One model call as a trace
One model call as a trace. In this video you'll turn a single call to a model into a span, and read it as data. When an agent gives a wrong answer, or takes a minute to answer at all, the screen shows you nothing. OpenTelemetry is an open source, vendor neutral standard for traces, metrics and logs. A trace is the story of one request, made of spans: timed operations with a name and attributes. The semantic conventions for generative AI, called GenAI, name those attributes. They are still in development, so every version is pinned. In this lesson you will install the OpenTelemetry packages at pinned versions, wrap one model call in a span named by the GenAI conventions, and read that span in a file with jq, and in Jaeger, a trace viewer. Before you start, you need Python 3, jq, Docker, and a model endpoint that speaks the OpenAI API, with its key. Three packages. The OpenAI client talks to any endpoint with that interface. The OpenTelemetry SDK, the software development kit, creates the spans. The exporter package sends them over OTLP, the OpenTelemetry protocol. This download is sped up in the video. The SDK is at version one point forty-five. The semantic conventions package ends in b, for beta, and its GenAI names are deprecated there, because they moved to a repository of their own. So I write the attribute names as plain strings. This file holds all the OpenTelemetry set-up. A tracer provider creates spans, and a processor hands each finished span to an exporter. When no OTLP endpoint is set, the exporter is the console exporter. I point it at a file, so each span is one line of JSON, a text format. The chat function wraps one model call in a span. The span name is the operation and the model. The attributes follow the conventions: operation, provider, model, then the response identifier, the finish reason and the token counts. My endpoint reports zero tokens, so the code estimates them from the text length and marks the span as estimated. The text of prompts is opt-in, because it can hold personal data. The program that makes the call is short. It sets up tracing, asks one question, and prints the answer. The answer is ordinary output. The span is what is new. This command makes the one line of JSON readable. The name says chat and the model. The attributes hold the operation, the model, why it stopped, and the token counts. A model call is now data you can query. The same span in a trace viewer. Jaeger is an open source tracing back end with a web page, and it runs in one container. Port sixteen six eight six is the web page. The OTLP receiver listens on four three one eight in the container, and this machine publishes it on four three one nine, because four three one eight is taken here. Both ports are open to this machine only. When the standard endpoint variable is set, the setup function sends the span over OTLP instead of writing it to the file. Jaeger's search page. I pick the service, release agent, the name the setup function gives the program, and press Find Traces. One trace, with one span: the model call. I open it. The span is one bar, as long as the call took. The icon shows that Jaeger recognizes the GenAI attributes. Its GenAI tab shows the operation, the model and the token counts: the same attributes that jq printed, now in a page you can search. In this lesson you learned that a span is one timed operation with a name, attributes and a status, that the GenAI conventions name the model, the token counts and the finish reason, and that one environment variable sends the span over OTLP instead of to a file.