AI observability has to connect system health with task quality

Introduction

The production dashboard is green.

The model endpoint is available. Latency has barely moved. Error rates are normal. Token consumption is within range. The external tools are responding.

Yet something has changed.

A quality scorer running against sampled customer-support interactions shows that answers about returns are becoming less grounded. The deterioration appears only in one task class, and mostly when a particular retrieval path is involved.

Nothing in the infrastructure dashboard would have raised an incident.

Nothing in the quality score alone explains where to look.

That is the missing connection in much of MLOps observability for task quality.

Production AI produces at least two kinds of evidence: whether the system is technically healthy and whether the behaviour it produces remains acceptable. Useful observability has to preserve enough common context that those two sets of signals can be examined together.

The goal is not another dashboard.

It is the ability to answer a harder production question:

What was happening inside the system when quality changed?

At a glance

• Conventional operational metrics such as latency, errors, availability and resource use remain necessary for AI systems. They do not establish whether an answer was grounded, an instruction was followed or a tool was used appropriately.
• Quality scores are equally incomplete when they cannot be connected to the model, retrieval path, tool activity, configuration or task class behind the interaction.
• Modern AI observability therefore needs a correlation layer joining runtime telemetry with application-quality evidence.
• A production trace or session can provide that shared reference, while dimensions such as task type, model, prompt or configuration identity and component path make aggregate quality trends diagnosable.
• The useful production view combines technical health with evidence that the AI behaviour remains acceptable.

System health and task quality answer different questions

Traditional observability tells teams whether requests completed, how long they took, which dependencies failed and whether the infrastructure is behaving normally.

AI systems need that information too.

They also introduce another set of production questions around groundedness, completeness, instruction adherence, tool use and application-specific behavioural rules.

Microsoft Foundry’s current observability model places operational and quality evidence side by side. Its production monitoring integrates with Azure Monitor Application Insights and exposes signals including token consumption, latency, error rates and quality scores, while tracing and evaluation add behavioural evidence around the application.

Neither signal family replaces the other.

An answer can arrive quickly and still be poor. A high-quality answer can come from a path whose latency or resource consumption has become unacceptable.

Production engineering needs to see both conditions.

Task quality needs a precise meaning

For this article, task quality is deliberately narrower than eventual business success.

It means whether a production interaction satisfies application-defined behavioural criteria: groundedness, relevance, completeness, instruction adherence, tool-use quality, deterministic business rules or similar measures.

That distinction matters.

A loan assistant can produce a high-quality answer while the customer later abandons the application.

A support interaction can satisfy its behavioural criteria while an unrelated fulfilment system fails hours later.

Those downstream outcomes still matter, but they belong to another measurement layer.

Observability first needs to connect what the AI system did with the quality of that behaviour.

Operational metrics alone can produce a false sense of health

Consider a retrieval-augmented support application.

Its p95 latency remains stable. Model errors stay close to zero. Retrieval is available. Token consumption has not moved enough to attract attention.

A new batch of support content enters the knowledge base and begins outranking more authoritative policy material for a narrow set of queries.

The system is functioning.

The answers are getting worse.

No infrastructure threshold has been crossed, so an ordinary service-health alert has no reason to fire.

The quality signal has to become part of what the production system watches.

Quality monitoring becomes useful when it lands on production telemetry

MLflow 3 provides a concrete implementation of this connection.

Its production monitoring can run scorers continuously against a configurable sample of live traces and attach the resulting assessments as feedback to the evaluated trace. The same scorers used during development can be reused in production, while monitoring views expose quality trends over time.

Databricks’ production-monitoring setup and managed scorer lifecycle are currently labelled Beta.

The architectural value is straightforward.

A poor groundedness result retains the identity of the production interaction it assessed.

Teams can then examine which model served it, which application configuration was active, what retrieval or tool activity occurred and how the interaction behaved operationally.

Quality becomes another observable property of real production traffic.

The trace is a correlation surface, not the final answer

Distributed tracing gives related work a common execution identity.

OpenTelemetry’s GenAI conventions can represent model operations, token usage and related GenAI attributes, while current instrumentation also covers agent and tool operations.

That does not mean every quality incident should begin with somebody reading a large trace tree.

The more useful first step is often aggregation.

If poor-quality interactions share a trace, session or other stable production identity, teams can group them by the conditions surrounding the behaviour and narrow the problem before reconstructing an individual run.

The trace is the correlation substrate.

The operating view comes from what can be aggregated across it.

Observability needs dimensions that explain variation

Suppose overall response quality falls by three percentage points.

That number becomes much more useful when production telemetry can show whether the decline is concentrated in one task class, one model deployment, one configuration, one retrieval strategy, one tool path, one region or another behaviourally meaningful cohort.

Perhaps invoice questions account for almost the entire decline.

Perhaps one model deployment performs differently from the rest.

Perhaps a new retrieval configuration corresponds with lower groundedness.

Perhaps latency and quality deteriorate together whenever one external service participates.

The right dimensions depend on the workload. They should be chosen because they help explain variation, not because they are easy to attach to telemetry.

Configuration identity belongs beside quality telemetry

AI applications can change behaviour without changing their model.

Prompt, retrieval, tool and orchestration configuration can matter alongside model identity.

Observability needs enough of that configuration identity to interpret production changes.

A quality drop beginning at 14:10 is more useful if telemetry also shows that a prompt alias moved at 14:03.

An unusual cost increase is easier to investigate when model identity is preserved.

A grounding problem becomes more tractable if interactions can be separated by the retrieval configuration active before and after a change.

Full configuration documents do not need to be copied into every trace.

Stable identifiers are usually enough to connect behavioural evidence with the production state that generated it.

Continuous evaluation closes part of the monitoring gap

Microsoft Foundry describes observability as a combination of evaluation, monitoring and tracing across the AI lifecycle.

Its production monitoring can combine operational measures such as latency, token consumption and errors with evaluation outcomes, while captured traces can be evaluated after deployment.

Foundry also supports evaluating interactions already captured in Application Insights, including traces emitted by agents outside Foundry when they follow supported OpenTelemetry conventions.

Evaluation therefore does not have to end at deployment.

Some of the behavioural criteria used during development can continue to measure sampled production traffic.

That creates continuity between what the organisation said mattered before release and what it watches after release, without turning production monitoring into another promotion gate.

Google is putting quality scores into the monitoring system itself

Google’s Agent Platform makes the same architectural move from another direction.

Its observability stack uses Cloud Trace, Logging and Monitoring to capture execution paths, events, errors, latency and token usage, while its Gen AI evaluation capabilities add behavioural quality assessment.

Google’s quality-alert workflow can export numeric online-evaluation scores into Cloud Monitoring.

Those scores can trigger incidents when quality falls below a defined threshold, with the alert linked back to the traces associated with the deterioration. Google’s documentation uses Task Success as one example of a quality metric that can be monitored over a time window.

Quality can therefore participate directly in production incident detection rather than living only inside an evaluation report.

A useful dashboard preserves both layers

A production AI dashboard does not need to force every measure into one composite health score.

Operational and quality signals can stay distinct while sharing enough context to investigate together.

Signal layer	Examples	What it can reveal
Runtime health	Latency, error rate, availability	Whether the service and dependencies are functioning
Consumption	Tokens, model calls, tool calls	How much work the system is performing
Component behaviour	Retrieval activity, tool latency, provider/model identity	Which parts of the application participated
Quality	Groundedness, relevance, completeness, adherence, safety	Whether the resulting behaviour remained acceptable
Context dimensions	Task class, configuration, region, channel	Where deterioration is concentrated
Feedback	User ratings, reviewer labels, deterministic outcomes	Additional evidence about real production behaviour

The value comes from being able to filter these layers against one another.

“Quality is falling” should become “quality is falling for this type of work under these production conditions.”

Aggregate averages can hide the incident

A global quality score can be as misleading as a global latency average.

Suppose 95% of traffic consists of simple informational requests and 5% consists of policy-sensitive cases.

If the simple requests remain excellent while policy-sensitive answers deteriorate sharply, the estate-wide average may barely move.

The same problem appears across languages, models, countries, application versions and tool paths.

Cohort-level monitoring exposes failures that disappear inside the aggregate.

The useful cohort depends on the application, and the observability design needs enough granularity to surface concentrated deterioration without producing unusable telemetry cardinality.

Quality sampling should follow cost, risk and traffic shape

Operational telemetry is relatively inexpensive to collect broadly.

Model-based quality evaluation can be much more expensive.

MLflow’s production-monitoring design supports configurable sampling so teams can trade coverage against evaluation cost.

Uniform random sampling is a reasonable starting point, but it can miss rare behaviours that matter.

A low-volume policy-sensitive workflow may disappear inside high-volume routine traffic. A newly launched task class may deserve temporary oversampling. Interactions that already trigger a deterministic anomaly may justify deeper evaluation.

The production policy can therefore combine broad deterministic checks with heavier coverage for high-consequence cohorts and lower sampling rates for expensive model-based judges.

Sampling determines what deterioration the organisation has a reasonable chance of seeing.

That makes it part of observability design rather than merely a cost-control parameter.

Quality alerts should narrow the cohort and route the incident

A latency alert and a groundedness alert describe different failures.

They may also need different responders.

If model requests begin timing out, the platform team may own the first investigation. If responses remain fast but policy adherence falls, the application or domain team may be better placed to act. When both move together around one dependency, ownership may cross several teams.

The alert should also help narrow the affected traffic.

A useful observability stack can show whether deterioration clusters around a particular task class, model, retrieval source, application version, region or tool path.

Once the cohort is narrow enough, detailed trace inspection becomes worthwhile.

Aggregate detection first, targeted reconstruction second.

That separation keeps production monitoring efficient while preserving a route into deeper debugging when the evidence warrants it.

Production evidence can be retained for later evaluation

Production traces often contain interactions that pre-release testing did not anticipate.

Preserving suitable production evidence can support later evaluation and improvement work.

The decision about how those cases enter a future release dataset or influence promotion belongs to the evaluation and release process rather than to production observability itself.

Here, the observability requirement is simply to preserve enough trustworthy evidence that later analysis remains possible.

Quality telemetry needs measurement identity too

Quality scores are not timeless facts.

Their interpretation can change when the scorer, judge model, rubric, sampling policy or monitored population changes.

A groundedness trend of 0.91 last month and 0.84 this month is difficult to interpret if the evaluator changed midway through the period.

Production-quality telemetry therefore needs enough measurement context to establish how each result was produced: evaluator identity and version, relevant rubric or configuration, sampling method and the population that was eligible for evaluation.

Otherwise, a monitoring change can look like an application regression.

Observability cost needs an explicit budget

Tracing, storing telemetry and continuously scoring production interactions all cost money.

Richer telemetry can also introduce privacy and retention obligations when prompts, responses, retrieval content or tool data are recorded; OpenTelemetry’s GenAI conventions explicitly warn that several of these fields can contain sensitive information.

The architectural decision here is how much evidence is worth collecting for its diagnostic value.

High-value metadata can often be retained broadly, while richer traces and expensive evaluation may be collected more selectively.

An observability system that captures everything indiscriminately can become economically difficult to operate without becoming proportionally better at diagnosis.

The production view should answer a small set of questions

A useful observability architecture should let teams establish whether the application is technically healthy, whether behavioural quality remains acceptable, where deterioration is concentrated and what production conditions distinguish poor interactions from good ones.

From there, it should identify which individual traces deserve deeper inspection.

That progression reduces the distance between detecting a quality problem and knowing where in the production estate to investigate it.

FAQs

What is AI observability?

AI observability combines conventional operational telemetry with signals specific to AI behaviour. Depending on the system, that can include traces, model and tool activity, token usage, configuration identity, evaluation scores, user feedback and other evidence needed to understand production quality.

Why are latency and error rates not enough for GenAI monitoring?

Latency and error rates show whether the service is functioning reliably. A GenAI application can remain fast and available while producing less grounded, less relevant or otherwise poorer outputs. Production monitoring therefore needs behavioural quality signals alongside infrastructure health.

How can AI task quality be monitored in production?

Teams can apply deterministic scorers, LLM judges, user feedback or application-specific evaluators to sampled production interactions. The resulting quality signal becomes much more useful when it is linked to the trace, session, configuration and task context that produced the interaction.

Should every production interaction be evaluated by an LLM judge?

Usually not. Continuous LLM evaluation can be costly. Critical or deterministic checks may justify broad coverage, while expensive quality judges can use sampling. The appropriate rate depends on risk, traffic volume, evaluator cost and the importance of the behaviour being monitored.

How is production observability different from pre-release evaluation?

Pre-release evaluation asks whether a candidate has enough evidence to be promoted. Production observability watches the behaviour of the deployed system under real traffic and helps localise deterioration. They can reuse evaluators and evidence, but they answer different operational questions.

A green system is useful only when the behaviour is green too

AI observability does not need to discard the disciplines that already keep software reliable.

Latency still matters.

Errors still matter.

Availability still matters.

Resource consumption still matters.

They simply describe only one part of production health.

An AI system also produces behaviour whose quality can deteriorate while every conventional service metric remains inside tolerance.

The observability architecture becomes useful when those two views are connected closely enough to investigate together.

The question is no longer only:

Is the system running normally?

It also has to be:

When task quality moves, can we see which production conditions moved with it?

That is the connection AI observability now has to preserve.