MLOps, LLMOps and AgentOps belong to different failure surfaces

Introduction

A customer-service agent stops resolving address-change requests.

The underlying model is available.

Latency is normal.

The retrieval service returns documents.

Tool calls are succeeding.

Nothing obvious is down.

Yet completion falls because the agent has started calling the customer-profile tool before checking whether the request requires identity verification. The tools work exactly as designed; the sequence is wrong.

A traditional model-health investigation would struggle to explain the failure.

So would a review limited to response quality.

The problem exists in the execution of the task.

This is why the distinction among MLOps, LLMOps and AgentOps failure surfaces matters more than the terminology itself.

Each discipline emerged around a different production problem. They overlap, but they do not look for failure in quite the same place. Treating them as interchangeable can leave an enterprise with excellent model monitoring, sophisticated LLM evaluation and an agent whose execution logic is still wrong.

At a glance

• MLOps is strongest when the primary production object is a trained model and its data, deployment and performance lifecycle.
• LLMOps extends operations into generative applications where prompts, context, retrieval and output quality can change behaviour even when the model artifact remains stable.
• AgentOps introduces another failure surface: multi-step execution through tools, state and orchestration, where the order or progression of otherwise healthy operations can be wrong.
• The useful distinction is not organisational ownership. It is where evidence of failure first becomes visible.

MLOps failures tend to appear around the model lifecycle

Traditional MLOps is built around a production object with a relatively clear centre: the machine-learning model.

The surrounding discipline manages how that model is developed, registered, approved, deployed and monitored.

AWS currently describes SageMaker MLOps in terms of experiments, workflows, lineage, model registration, deployment, monitoring and automation. Its Model Registry manages model versions, metadata, lineage, approval status and production deployment.

The associated failure surface is familiar.

Training and production data diverge. Input distributions shift. A new model performs worse than its predecessor. Serving infrastructure becomes unavailable or slow. A deployed artifact or configuration is wrong.

Production investigation often begins with the relationship among data, model, version and endpoint.

For existing customers, SageMaker Model Monitor also illustrates the kinds of signals associated with that surface: data quality, model quality, bias drift and feature-attribution drift. AWS now notes that Model Monitor is no longer open to new customers, so it is better treated here as an established example of the model-monitoring pattern than as a new platform recommendation.

MLOps remains valuable because those failures have not disappeared.

The difficulty starts when the model is no longer the component whose state best explains the application’s behaviour.

An LLM application can fail while the model remains healthy

Consider a knowledge assistant using a hosted foundation model.

The model endpoint has not changed.

Token latency is stable.

Error rates are normal.

Yet answers become less reliable after the team changes its system prompt and expands the retrieval corpus.

From the model-serving layer, production may look healthy.

From the application layer, something important has changed.

Generative applications introduce variables that sit outside the conventional trained-model lifecycle: prompts, retrieved context, conversation history, response constraints and application-level evaluation criteria.

That is much of the practical territory described as LLMOps.

MLflow 3’s current GenAI tooling is organised around this wider application object. Databricks describes tracking, evaluation and observability for GenAI applications and agents, including prompt and app versioning, tracing, human feedback and automated scorers.

The failure surface therefore moves.

A conventional predictive system may expose deterioration through model-quality or data-distribution metrics.

An LLM application can retain a healthy endpoint while the generated behaviour becomes less grounded, less complete or less suitable for the task.

The operative question becomes:

Is the application producing acceptable behaviour from this model, prompt and context?

Prompt changes create production changes of their own

Suppose a claims assistant is instructed to summarise the claim, identify missing evidence and recommend the next action.

A prompt revision intended to make responses shorter removes the instruction to identify missing evidence.

The model remains the same.

Retrieval remains the same.

The assistant becomes more concise.

It also starts producing incomplete recommendations.

Model drift does not explain the regression.

MLflow’s Prompt Registry reflects this independent lifecycle. Prompt versions are immutable, while aliases such as production and staging can point deployed applications at different versions. Databricks documents changing those aliases without requiring an application-code redeployment.

The useful operational evidence is prompt lineage plus application evaluation.

That is a different place to start looking than the model endpoint.

Retrieval can make a generation problem look like a model problem

A response can be wrong because the model reasoned badly.

It can also be wrong because the application supplied the wrong evidence.

A relevant document may disappear from an index. Ranking behaviour may shift. Permissions may suppress a source. Chunking may make the important passage difficult to retrieve. A corpus may be technically available but stale.

To the user, all of those failures look like a bad AI answer.

Operationally, the defect may sit upstream of generation.

MLflow’s current evaluation and monitoring stack works over traces and supports scorers for qualities such as correctness, groundedness, safety and completeness.

That makes application quality visible even when ordinary model-serving telemetry is uneventful.

MLOps is still needed underneath.

LLMOps adds evidence about the assembled generation path.

AgentOps begins with the execution of the task

Now extend the system again.

Instead of producing an answer, the application has to complete work.

A procurement agent must inspect a request, identify an approved supplier, check spending authority, obtain the required information and create the correct procurement record.

The individual components can remain available while execution goes wrong.

The agent may invoke the right operations in the wrong order.

It may carry the wrong state from one step to the next.

It may choose an inappropriate tool even though the tool itself is functioning.

It may enter a path that is valid technically but inconsistent with the business process.

These failures are about orchestration and progression through work.

Microsoft Foundry’s agent telemetry reflects that broader execution surface. Its OpenTelemetry-based conventions represent agent invocations, tool execution and memory operations as distinct parts of the run.

The model call is therefore only one part of the production evidence.

For AgentOps, the task path itself becomes operationally relevant.

The three surfaces can fail independently

The distinction becomes clearer when the same production system is viewed from three operating surfaces.

Operating surface	Characteristic production question	Example failure
MLOps	Is the model/data/deployment lifecycle behaving as expected?	Prediction quality deteriorates after input-data drift
LLMOps	Is the generative application producing acceptable behaviour from its prompt, context and model?	Answers become ungrounded after retrieval behaviour changes
AgentOps	Is multi-step execution progressing correctly through tools and state?	Healthy tools are invoked in the wrong sequence for the business process

The boundaries are not absolute.

A model regression can degrade an LLM application. Retrieval failure can disrupt an agent. Prompt instructions can influence tool choice.

The distinction is useful because the first useful evidence often appears in different places.

For one problem, model versions and data distributions may explain what changed.

For another, prompt lineage and retrieved evidence are more informative.

For an agentic failure, execution order, state progression or tool choice may carry the answer.

Different failure surfaces require different evidence

A model team may need model versions, feature distributions, prediction quality and deployment state.

A generative application adds evidence around prompts, retrieved context and evaluated outputs.

An agentic workload extends that record into orchestration, tool use and changing task state.

MLflow 3 increasingly uses traces as a foundation for GenAI evaluation and production monitoring, while Microsoft Foundry captures ordered agent activity and tool execution.

These are not arguments for three separate monitoring estates.

They show why model-serving metrics alone cannot explain every class of production failure once the application above the model becomes more complex.

Healthy lower layers can coexist with failure above them

The difficult production cases are often the ones where the infrastructure underneath the problem appears normal.

A model endpoint can be healthy while retrieval gives the application weak evidence.

A generative answer can look plausible while an agent chooses the wrong next operation.

Several tool calls may be perfectly valid while the orchestration path itself is inconsistent with the intended process.

The meaning of “green” therefore depends on the level being measured.

Model latency might remain stable while application quality deteriorates.

Application quality might remain acceptable while an agent begins following the wrong execution path.

These signals are not contradictory.

They describe different production surfaces.

This is one reason a single aggregate notion of “AI health” becomes less useful as systems acquire more behaviour around the model.

Ownership should follow the failure, not the acronym

The terminology can easily become an organisational debate.

Does the MLOps team own prompts?

Does an application team own LLMOps?

Is AgentOps a platform responsibility?

There is no universal answer.

A model-quality regression may belong with the model and data lifecycle owners.

A grounding problem may require the application, retrieval or evaluation owner.

An execution-order problem may involve orchestration logic, tool contracts and the business process owner.

Several teams may participate in the same incident.

The point of distinguishing failure surfaces is to shorten diagnosis and make ownership clearer once the failure is understood, not to create three new departmental boundaries.

Later operating disciplines do not replace earlier ones

AgentOps sits on top of infrastructure and models that still have to work.

LLMOps still depends on reliable model serving.

An agent may still rely on retrieval, prompts and model quality even though its distinctive failures appear in execution.

The production stack is cumulative.

As more behaviour moves above the model, more places become capable of changing the final outcome.

That is why enterprises can need MLOps, LLMOps and AgentOps concerns at the same time.

The labels overlap because the systems overlap.

Their diagnostic value comes from knowing which surface is currently failing.

The practical boundary is the question you cannot answer

One useful way to identify a missing operating discipline is to look at the production questions the current system cannot answer.

Can we determine whether the model or its input distribution changed?

Can we establish which prompt and retrieved context produced a poor answer?

Can we tell whether the agent followed the intended execution path through its tools and state?

If one of those questions repeatedly requires manual reconstruction across disconnected systems, the operating boundary probably stops too early.

That matters more than whether a particular platform feature is marketed as MLOps, LLMOps or AgentOps.

The labels are useful only when they point teams toward the evidence they were previously missing.

FAQs

What is the difference between MLOps and LLMOps?

MLOps primarily manages the lifecycle of machine-learning models, including training, versioning, deployment and production quality. LLMOps extends operations into generative applications where prompts, retrieval, context and response quality can alter behaviour independently of the model artifact.

What is the difference between LLMOps and AgentOps?

LLMOps focuses on the behaviour and operation of LLM-powered applications. AgentOps adds the execution surface created when systems use tools, maintain state and progress through multi-step work. The distinctive failure may therefore sit in orchestration rather than in the generated response itself.

Does AgentOps replace MLOps?

No. Production agents still depend on models, serving infrastructure, data and often retrieval. AgentOps adds operating concerns around execution and task progression while retaining dependencies on the MLOps and LLMOps layers underneath it.

Can a system be healthy in MLOps but unhealthy in LLMOps?

Yes. A model endpoint can have normal latency, availability and model-level metrics while an application produces weak answers because a prompt, retrieval source or context configuration changed.

What makes an AgentOps failure different?

An AgentOps failure can emerge from the sequence and state of execution rather than from the health of an individual model or tool. The agent may use available components incorrectly, follow an unsuitable path or carry the wrong state into the next step.

Different production objects create different failures

MLOps, LLMOps and AgentOps are often discussed as stages in the evolution of one discipline.

The deeper distinction is the production object each one is trying to keep under control.

MLOps centres on the model lifecycle.

LLMOps has to account for generated behaviour assembled from model, prompt and context.

AgentOps has to account for execution through a changing sequence of decisions, state and tools.

Those objects overlap, but their failures do not always appear in the same evidence.

A production team therefore needs to ask one question before deciding what to monitor or who should investigate:

Where can this system fail even when everything underneath it still looks healthy?

That answer defines the failure surface.