The expensive part of AI data infrastructure is increasingly context delivery

Introduction

The model team cuts inference spend.

Requests move to a cheaper model. Prompt caching improves. Output length comes down.

The application bill barely moves.

Behind each request, the system is still refreshing enterprise data, embedding changed documents, maintaining search indexes, storing retrieval structures, querying warehouses, enforcing access filters and keeping low-latency context services available.

Those costs are becoming easier to see in production platforms.

Snowflake, for example, breaks Cortex Search spending into warehouse compute for refresh, embedding compute, serving compute, storage and cloud-services compute. Serving charges can continue while a service is available even when no query is being served.

Microsoft now separates billing for agentic retrieval from the model tokens used for query planning and answer synthesis.

The visible model call is becoming one part of a larger AI data infrastructure context delivery cost.

For many enterprise systems, the expensive engineering problem is increasingly keeping the right context ready, current, searchable and accessible when the model needs it.

At a glance

• Context delivery includes more than adding retrieved text to a prompt. Enterprise applications may have to ingest, transform, embed, index, refresh, search, rank, permission-filter, cache and serve the underlying information.
• Some of those costs are transactional. Others continue while context infrastructure remains available.
• Agentic retrieval can increase context work because one task may generate several searches or subqueries before synthesis.
• Lower model prices therefore do not guarantee proportional reductions in total AI application cost. The data path around the model may become the more persistent part of the bill.

The model bill is no longer the whole AI data bill

Model pricing is unusually visible.

Input tokens, output tokens and cached tokens fit neatly into a pricing table. Teams can compare models, shorten prompts or route simpler work to cheaper inference.

Context delivery is harder to see because the spending is distributed across infrastructure.

A production assistant may need to:

• read source systems;
• identify changed records;
• transform and chunk documents;
• generate embeddings;
• materialise indexes;
• store vector and textual representations;
• run retrieval;
• apply semantic ranking;
• filter results according to permissions;
• query structured systems;
• assemble context;
• and keep that infrastructure responsive enough for an interactive application.

None of this is unusual architecture.

What has changed is how much of it can happen for one AI interaction.

Anthropic’s description of context engineering includes system instructions, tools, MCP resources, external data and message history as parts of the state an agent may need to curate repeatedly as a task progresses.

The context window sits at the end of a data-supply process.

Enterprise infrastructure pays for much of that process before the tokens reach the model.

Context delivery has a standing cost

Retrieval infrastructure is often described as though its cost begins when somebody searches it.

That is only one charging model.

Snowflake Cortex Search illustrates the wider economics. Its service can incur warehouse compute when source data is refreshed, embedding charges when text is inserted or updated, storage for materialised data and indexes, and serving compute while the search service is available. Snowflake’s documentation explicitly notes that serving cost can continue even when no queries are being served.

In May 2026, Snowflake added automatic suspend and resume for Cortex Search so inactive services could stop consuming serving compute. In July, it added resource budgets that can place monthly spending limits on individual Cortex Search services and automatically suspend them or revoke access when limits are exceeded.

Those features make sense because retrieval has become an infrastructure workload worth managing independently.

The same pattern appears elsewhere.

Azure AI Search offers dedicated capacity billed through hourly Search Units and a serverless model that charges for compute and indexed storage. Premium capabilities such as agentic retrieval can add separate usage charges.

An enterprise can therefore incur context-delivery cost before the final model invocation is considered.

Fresh context creates recurring data work

A static index is relatively easy to reason about financially.

Enterprise context rarely stays static.

Customer status changes. Orders move. Policies are amended. Products are added. Documents are replaced. Permissions change.

Every important change can create work somewhere in the context pipeline.

Snowflake notes that Cortex Search embeddings are recalculated for inserted or updated rows. Its warehouse cost also depends partly on the change rate of underlying data and the target lag configured for refresh. Lower target lag combined with higher data change rates can increase refresh-related compute.

Confluent exposes the same economic issue through a different architecture.

Its Real-Time Context Engine continuously materialises streaming business state for agents. Billing includes separate compute and storage meters. Compute is influenced by data scanned and streaming activity rather than simply counting user queries, while materialised context continues to incur storage charges.

The point here is economic rather than an argument that every agent needs real-time infrastructure.

Freshness has a cost curve.

The more frequently an organisation expects context to follow changing source state, the more often parts of the context-delivery machinery may have to run.

One task can trigger several context operations

The user may see one question.

The infrastructure may see considerably more work.

Consider an agent asked:

Which of our strategic customers have open support escalations, contracts renewing this quarter and declining product usage?

The task may require separate retrieval over account definitions, support records, contract data and product telemetry.

An agentic retrieval system can also decompose a complex question into several subqueries before combining the evidence.

Microsoft’s Azure AI Search documentation makes that cost split explicit. Agentic retrieval uses retrieval tokens during subquery execution and semantic ranking, while model-based query planning and answer synthesis create a separate model charge.

One business request can therefore produce a small fan-out of context work.

That fan-out matters because enterprise teams often measure AI consumption at the user-request or model-call level.

Neither necessarily reveals how many retrieval operations, index lookups, SQL queries or context transformations happened underneath.

Search size can matter even when query volume is modest

Context cost does not always scale most strongly with the number of prompts.

The amount of information kept ready can matter too.

Snowflake charges Cortex Search serving compute according to the volume of indexed data, including the vector embeddings stored with it. Its documentation gives worked examples showing that increasing index size or maintaining multiple vector indexes raises ongoing serving cost even without a corresponding increase in query count.

Azure’s serverless search pricing similarly separates compute use from indexed storage charged by GB per month.

This creates an architectural trade-off that is easy to miss during early pilots.

Adding another repository to an enterprise assistant can look almost free from the application’s point of view.

The additional context may require another ingestion path, more stored index data, more embeddings, more refresh activity and a larger search surface.

A hundred new users are not the only way an AI system becomes more expensive.

A much larger knowledge footprint can do it as well.

Caching changes context economics

Context can also be expensive because the same information is repeatedly processed.

A large system instruction, document collection or reference package may appear in many requests.

Google describes this directly in its Vertex AI context-caching guidance: repeatedly sending substantial contextual information increases cost and latency because the same tokens have to be processed again. Context caching lets applications reuse precomputed input tokens, with supported cached Gemini tokens priced below ordinary input processing.

Caching does not make context free.

Explicit cache storage itself has a price, and the application still has to decide what deserves to be cached, when it should expire and when changed enterprise information should replace it.

The economic question moves from repeated computation toward reuse.

Stable context can often be amortised.

Highly volatile context has fewer opportunities for that reuse.

The shape of enterprise data affects the context bill

Two assistants with identical models and identical request volumes can have very different infrastructure economics.

One might answer questions from a relatively stable policy library.

Another may operate across customer records, tickets, transactions, telemetry and rapidly changing operational data.

The second system has more context to maintain, more changes to propagate and potentially more sources to retrieve from during each task.

The difference does not originate in the model.

It comes from the shape of the information environment around it.

Several characteristics matter:

Context characteristic	Economic consequence
Large knowledge footprint	More index and storage capacity
High change rate	More refresh, transformation and embedding work
Low acceptable staleness	More frequent context updates
Many repositories	More ingestion, synchronisation and retrieval paths
Complex questions	More retrieval subqueries and ranking work
Repeated stable context	Greater opportunity for caching
Fine-grained access rules	More filtering and permission-aware retrieval work

Model price alone therefore gives an incomplete comparison between AI applications.

The same model can sit behind very different data-supply systems.

Cheap inference can expose expensive context architecture

Imagine that a new model cuts inference cost by half.

An application that spends most of its money on model tokens benefits quickly.

A system whose cost is dominated by large indexes, constant refresh, retrieval traffic and context-serving infrastructure may see a much smaller reduction in total spend.

After the obvious model-level optimisations have been made, the remaining bill can expose how much of the application’s economics sit elsewhere.

A cheaper model can leave the context estate doing almost exactly the same work.

This does not mean context delivery will dominate every AI workload. Long generations, heavy reasoning, large multimodal inference or high-volume model use can still make inference the largest line item.

The strategic change is that enterprise teams now have another substantial cost surface that can remain sticky even as model economics improve.

Context cost should be visible per workload

Data infrastructure costs are often allocated by platform.

AI applications are usually measured by product, team, user or model.

Context delivery sits awkwardly between those views.

A shared search service may support several agents.

One agent may trigger many retrievals per task.

Another may consume a large but slowly changing index.

A third may rely on small amounts of context that must be refreshed continuously.

Without workload-level attribution, all three can appear to have roughly the same AI data architecture.

They do not have the same economics.

Useful operational measures can include:

• context-serving spend by application;
• embedding and refresh cost by knowledge source;
• indexed data footprint;
• retrieval operations or subqueries per completed task;
• cache reuse;
• data-change volume driving context refresh;
• cost of context that is retrieved but never used in the final task.

Snowflake’s CORTEX_SEARCH_DAILY_USAGE_HISTORY view separates Cortex Search consumption into categories including SERVING, EMBED_TEXT_TOKENS and BATCH, making this kind of attribution more practical.

The useful question is which part of the context pipeline is consuming money without improving the application’s decisions.

More context can have negative economics

Teams often respond to weak AI answers by giving the system more information.

Add another repository.

Increase the retrieval count.

Include more history.

Keep more documents in the index.

Pass a larger context package to the model.

Some of those changes improve performance.

Others simply increase the amount of infrastructure work performed before the same answer is produced.

Anthropic’s context-engineering guidance argues for curating the most useful context rather than filling the context window indiscriminately. It highlights just-in-time retrieval, compaction and structured memory as ways to manage a finite context resource.

There is an infrastructure analogue.

An economically efficient context system delivers enough high-value information to improve the task without continuously paying to prepare, store and retrieve material that rarely affects the outcome.

Data architecture choices now appear in AI margins

Context-delivery architecture increasingly shows up in the business case.

Indexing a larger share of the enterprise estate can increase storage and embedding work. Near-live context can increase refresh activity. Separate domain indexes can create additional serving footprints. Multi-pass retrieval can multiply query work, while low-latency requirements may justify capacity that remains ready between requests.

Any of those choices may be justified by better accuracy, freshness or user experience.

They still have a cost.

When an enterprise asks what one AI-assisted case, investigation, recommendation or agent task costs, the answer increasingly needs to include the data infrastructure required to deliver its context.

FAQs

What is context delivery in enterprise AI?

Context delivery is the infrastructure work required to make relevant enterprise information available to an AI application when it needs it. It can include ingestion, transformation, embeddings, indexing, search, ranking, permission filtering, caching, structured queries, refresh and low-latency serving.

Why can context delivery be expensive?

Context infrastructure can create both persistent and usage-driven costs. Enterprises may pay for indexed storage, serving capacity, embeddings, source refreshes, retrieval operations, semantic ranking, queries and continuously materialised state before the final model invocation is considered.

Is context delivery more expensive than model inference?

Not in every workload. Inference can remain the dominant cost for reasoning-heavy, high-token or multimodal applications. The important shift is that enterprise context has become a separate and sometimes substantial cost surface that may not fall when model prices decline.

Does RAG reduce AI infrastructure cost?

RAG can avoid placing an entire knowledge base directly into every prompt, but it introduces retrieval infrastructure of its own. The economics depend on index size, refresh frequency, search volume, embedding work, retrieval complexity and how much retrieved context actually improves task outcomes.

How can enterprises reduce context-delivery cost?

The biggest opportunities usually come from reducing unnecessary indexed data, avoiding excessive refresh, improving retrieval precision, reusing stable context through caching, limiting redundant retrieval passes and attributing context infrastructure spend to the applications that actually consume it.

The cost is moving toward the information path

A model can become cheaper without making the surrounding application equally cheaper.

Enterprise information still has to be located, prepared, refreshed, governed and served.

As agents use more systems and tasks require richer evidence, that work happens more often and across more infrastructure.

The useful financial question is therefore becoming more specific:

How much are we spending to put the right enterprise context in front of the model for one useful task?

That number can tell a very different story from the token bill.