A customer score has value when it changes a decision. A score calculated in advance, stored for days, and never used still consumes compute and creates data that your teams must retain and govern. As customer experiences move closer to real time, the design question is no longer just which inference to run. It is where to run it, when to run it, and whether a device or a customer-owned service should do the work.
Three patterns help make that choice: batch inference owned by the customer, real-time inference requested when needed, and a hybrid model that uses on-device inference where supported and customer-owned inference elsewhere. Each changes the relationship between usage, compute, and stored output.
Shifting from scheduled scores to inference at the moment of need
Batch scoring is familiar. A model processes a population on a schedule, and the resulting scores are written to a store for later use. That pattern suits use cases where the score remains useful over time and downstream systems need a persistent value.
The trade-off is that batch cost is largely committed before anyone uses the result. Some scores become stale before activation. Others are never read at all. The organization has still run the job and stored its output. For high-volume use cases, the gap between scores produced and scores acted on deserves attention alongside model accuracy.
A real-time pattern changes the trigger. Rather than calculating every score in advance, the experience requests inference when a qualifying interaction needs it. Tealium Functions can support this approach by calling a customer-owned inference service on demand. The decision moves closer to the customer moment, and the system can avoid generating outputs for interactions that never call for them.
This does not make real-time inference free. It changes the cost profile. Compute is tied to requests rather than a scheduled population-wide run, and the implementation must account for request volume, service capacity, response time, and reliability. The output can also be limited to what the activation needs, reducing the volume of stored scores when persistent storage is not required.
A third pattern adds on-device inference where the use case and device support it. In a hybrid design, the device handles inference locally for supported interactions, while a customer-owned service handles the remaining interactions. The model and placement decision determine the split. The aim is not to move every inference to a device; it is to avoid sending work to a central service when local inference is appropriate.
How the architecture works
Batch: calculate first, use later
A scheduled pipeline selects a population, runs the model, and writes scores to a customer-owned store. Activation systems read those scores when they need them. This is a good fit when downstream consumers need a durable score or when the underlying data changes slowly enough for scheduled refreshes.
The operational questions are straightforward: how often does each score get used, how quickly does it lose value, and how long must it stay available? If those answers are unclear, the organization is paying to produce and retain outputs without knowing whether they affect a decision.
On demand: request inference when a use case needs it
In an on-demand flow, an interaction reaches the orchestration layer, which invokes a Tealium Function to request a result from the customer-owned inference service. The response can then inform the next action in that experience. The exact request and response contract depends on the service and use case, so teams should define it before implementation.
Hybrid: use local inference where it fits
A hybrid design routes supported interactions to on-device inference and sends other interactions to the customer-owned inference service. The response path then returns the result to the experience that needs it. The precise routing depends on device capability, model placement, privacy requirements, and the decision's latency needs.
Where the use case supports this split, the customer-owned service handles fewer requests. That can reduce central compute and the amount of output that needs to be stored. The total number of useful inferences can remain similar; what changes is where those inferences run and how many central calls are necessary. Validate this with measured workloads rather than assuming the hybrid pattern always costs less.
Example use cases
A time-sensitive experience
A site interaction signals that a customer is considering an offer. If the decision depends on a fresh model result, an on-demand request can return context for the next experience step. The team should define the latency budget and the fallback behavior before connecting the request to a live journey.
A durable planning score
A durable planning score teams over a defined period. Batch processing and persistent storage can be appropriate here. The key is to measure whether the stored score remains current and whether consumers actually use it.
A device-supported interaction
For an experience with a supported device and model, on-device inference can handle the decision locally. A hybrid design can reserve the customer-owned service for devices or use cases that do not support local inference. Teams should verify model compatibility, result consistency, and how each path is monitored.
Why Tealium's role matters
The orchestration layer determines when an interaction needs a model result and where the next action takes place. That makes it an important part of the inference design, not a replacement for the model or the customer-owned service.
- Connect the moment to the decision: Use customer interactions and defined journey logic to identify when a result is needed.
- Call services as needed: Tealium Functions can connect an eligible interaction to a customer-owned inference endpoint, subject to the agreed request, response, and operational design.
- Keep model ownership clear: The customer's model or inference service performs the inference. Tealium carries the relevant context into the flow and supports the action that follows.
- Choose what to retain: Design the response path around the downstream need. A persistent score is useful when consumers require it; a transient result does not need to become another long-lived data asset by default.
- Compare architectures with evidence: Track call volume, compute consumption, response time, unused outputs, and storage over representative workloads. The right pattern depends on those measurements and the use case.
The bigger takeaway
Inference cost is not only a question of model size or cloud price. It is also a question of how many results you generate, how many are used, where computation happens, and how much output you retain.
Batch processing remains a sound choice when scores need to persist. On-demand inference fits decisions that depend on a live interaction. A hybrid approach places some work on the device and keeps customer-owned inference available for the rest. Start with the decision, map the moment it needs a result, and choose the architecture that serves that moment without producing data nobody uses.
