Volver al blog

From a One-Shot Prompt to a Traceable AI System

Featured image: From a One-Shot Prompt to a Traceable AI System

Over the past few weeks, I have been focused on understanding what it means to move from a single-prompt integration to an AI system that can evolve, be observed, and be maintained.

LiteraryTrip started with a simple MVP: a Supabase Function received a request and ran a prompt to generate a literary route. That was enough to validate the idea, but not enough to understand the system when something went wrong.

Building the next version was not just about changing the prompt. It meant separating responsibilities, modelling states, introducing validation, and making every stage of generation visible.

This article summarises that journey: from the first one-shot flow to an architecture based on workers, LangChain, LangGraph, tracing, and authoritative publication.

The MVP: perfect for validating the idea

The first version had a deliberately simple flow:

request → Supabase Function → prompt → model → generated response

The function received the work's data, built the prompt, and asked the model for a structured literary route.

That design had a major advantage: it enabled fast learning. Before designing a complete platform, we could verify whether the product made sense.

The problem appeared later, when the question stopped being “can we generate a route?” and became:

  • What exactly happened during this generation?
  • Which model and prompt version were involved?
  • Did the model, validation, or evidence fail?
  • Was the response repaired or generated again?
  • Why was one route published while another was discarded?
  • Can we reconstruct the process without storing sensitive content?

A prompt can produce a response. On its own, it cannot reliably explain that response's complete lifecycle.

The first decision: separate generation from the system

The most important change was to stop treating generation as a single operation.

Generation became part of a pipeline with explicit responsibilities:

enqueue
  → canonical identity resolution
  → job claim
  → runtime selection
  → orchestration
  → projection
  → authoritative publication

The entry Edge Function no longer has to solve everything. It validates the request, controls access, and creates a job with the required context.

Then a worker claims that job and runs the appropriate pipeline according to the contract and capabilities persisted on the job itself.

This preserves compatibility with the legacy runtime without silently falling back from the new runtime.

The separation was organised around several boundaries:

Layer Responsibility
enqueue-generation Authentication, initial validation, quota, and job creation
process-generation-jobs Claiming, retries, terminal states, and worker execution
Literary orchestrator Running the generation process as data
Projection Converting the result into the publishable payload
Publication Persisting authoritatively and idempotently

One particularly important decision was that the orchestrator should not publish directly to the database.

The capable pipeline returns data. The worker remains responsible for persistence, publication, and closing the operational lifecycle.

LangGraph: turning generation into a process

LangGraph made it possible to express generation as a state graph instead of hiding all the logic inside a model call.

The main flow became:

prepare_input
      ↓
generate
      ↓
validate_proposal
   ┌──┴───────────────┐
valid               invalid
   ↓                    ↓
evidence        repair_proposal
   ↓                    ↓
classify_evidence ←─────┘
   ↓
assess_publishability

If validation exhausts the repair budget, the flow records quality_exhausted and ends as a non-publishable result.

What does generate actually do?

generate is the node that calls the model, but it does not directly contain the provider implementation.

It receives the current state, runs an injected generator, applies a deadline, and records the operation through the tracer.

Conceptually:

proposal = await trace("generate", request, () =>
  withDeadline(signal => generator(request, signal), 30_000)
)

This makes the node testable and keeps the orchestrator independent from any specific provider.

The model is not the only actor

The nodes do not necessarily represent different models or autonomous agents.

They represent stages with distinct responsibilities:

  • prepare_input prepares and traces the input context.
  • generate requests a proposal from the model.
  • validate_proposal checks structure, coordinates, counts, and quality.
  • repair_proposal requests a correction with specific instructions.
  • evidence enriches candidate places.
  • classify_evidence applies the classification policy.
  • assess_publishability decides whether the result can be published.

The distinction is fundamental: the model call is one part of the process, not the whole process.

Bounded repair, not infinite retries

When a proposal fails validation, the system does not retry it without limits.

The repair_proposal node receives the normalised error and builds specific instructions to correct the proposal.

It then returns to validate_proposal. The cycle can repeat only within the semantic budget defined by the job policy.

This separates two concepts that could previously be confused:

  • Operational retry: the worker runs a job again because of a transient failure.
  • Semantic repair: the model corrects a proposal that does not satisfy the contract.

They are not the same problem and should not use the same counter or policy.

Explicit state and reducers

The graph maintains explicit state with fields such as:

request
proposal
repairCount
validationError
places
publishability
terminalDecision

Each state update has a defined reducer. This means repair does not lose the previous proposal, the counter is not accidentally reset, and the validation error remains available to determine the next transition.

This detail may seem small, but it is one of the differences between chaining calls and modelling a state machine.

The traceability we gained

The refactor added traceability across several dimensions.

Execution identity

Generation can now be followed through a correlated identity:

correlation_id → job_id → trace_id → attempt

The context is persisted when the job is created instead of being reconstructed later from separate logs.

The worker uses the persisted trace_id and maintains the relationship between the input event, processing, and terminal result.

Observable stages

Tracing records operations such as:

  • canonical identity resolution;
  • cache lookup;
  • prompt loading;
  • provider call;
  • corroboration;
  • projection policy;
  • publication;
  • terminal generation closure.

The orchestrator also records the graph stages, including generation, validation, repair, evidence, and the publishability decision.

Explicit terminal outcomes

Generation no longer ends with a simple “success” or “error”.

The lifecycle distinguishes outcomes such as:

  • generated;
  • low_quality;
  • failed;
  • paused_resolution;
  • retry_scheduled.

This allows an operational tool to distinguish a low-quality result from a technical failure or a job that still needs to be retried.

Model runs and duration

Model attempts are recorded with operational information such as duration, result, error, trace, and attempt number.

Observability stops being merely a collection of messages and starts working as a queryable record of system behaviour.

Langfuse and the local ledger serve different purposes

The Langfuse integration provides an external, time-based view of the execution.

It makes it possible to follow spans related to a generation without automatically sending all content produced by the model.

Tracing is encapsulated behind DeveloperTracer. This means the orchestrator knows a tracing interface without being coupled to a specific implementation.

Alongside external tracing, the system maintains a local LangGraph ledger:

literary_graph_runs
literary_graph_steps

That ledger records the graph execution and its steps with sequence, node, state, duration, and sanitised snapshots.

The combination is useful because it addresses two different needs:

  • Langfuse helps observe execution operationally.
  • The local ledger preserves a durable, governed representation of the business process.

Traceability without turning it into a data leak

Observing more does not mean storing everything.

Ledger snapshots pass through a field allowlist. Counts, decisions, error codes, and policy versions are retained, but not the complete literary content.

The design excludes data such as the following from telemetry:

  • direct user identifiers;
  • complete prompts;
  • complete model responses;
  • HTTP error bodies;
  • internal retry instructions;
  • cache keys;
  • URLs and locations without an operational need.

Observability therefore becomes a data contract, not a console.log with more information.

Provenance and quality: generation is not verification

Modularity also made it possible to separate generation from data provenance.

The route uses canonical identity so that text entered by a user does not become the direct source of truth.

Evidence contracts include source, title, locator, retrieval date, version, rights, licence, and source level.

The system also distinguishes a verified factual claim from corroboration by another model.

AI corroboration may reduce the risk of hallucination, but it is not equivalent to factual verification.

This distinction is essential when the result feeds a product experience rather than just a demo.

Modularity through contracts and adapters

The orchestrator depends on interfaces, not concrete implementations.

The main contracts include:

  • RouteProposalGenerator for generating or repairing proposals;
  • LiteraryEvidenceSource for enriching places with evidence;
  • LiteraryOrchestrator for running orchestration;
  • runtime ports for separating the legacy flow from the LangGraph flow.

This makes it possible to replace a provider, evidence source, or tracer without rewriting the entire graph.

It also makes it possible to test behaviour with controlled dependencies without invoking a real model in every test.

Modularity does not mean having many files. It means every change has a clear place and a contract that protects the rest of the system.

Before and after

Before After
One function concentrated much of the flow Entry, worker, orchestrator, projection, and publication have separate responsibilities
The model returned a final response The graph governs generation, validation, repair, and decision-making
Failures were difficult to reconstruct Every job has correlation, trace, attempt, and a terminal result
Retries and corrections could be mixed together Operational retry and semantic repair have distinct policies
Telemetry could contain too much context Observable data is governed by allowlists
Persistence was close to generation The orchestrator returns data and publication is authoritative and idempotent

How this changed the way I think as an AI Engineer

This work left me with several important ideas.

1. The prompt is a component, not an architecture

A prompt can solve the first product problem. It does not solve identity, state, validation, retries, privacy, or publication on its own.

2. Quality must live outside the model

The model proposes. The system validates, repairs within limits, and decides whether the result meets the publication policy.

3. Observability is designed from the start

Adding logs at the end helps investigate symptoms. Designing correlated identity and terminal states makes it possible to reconstruct the complete story.

4. Modularity reduces the cost of learning

When the provider, evidence source, and tracer sit behind contracts, it is possible to experiment without turning every experiment into a complete migration.

5. AI needs engineering around it

LangChain and LangGraph do not replace system design. They help express execution, but quality also depends on contracts, policies, persistence, security, and tests.

What I can claim and what I still cannot

The refactor demonstrates improvements in structure, traceability, privacy, modularity, and lifecycle control.

It would not be correct to claim that it has reduced latency, cost, or the production error rate without comparable metrics.

It would also not be correct to present corroboration by another model as factual verification.

To me, this caution is part of the mindset shift: an architecture can be better designed without that authorising us to invent production results.

Closing

LiteraryTrip started as a direct call to a model because that was the right way to validate the idea.

Later, it needed to become a system because the important questions were no longer only about generation.

They were questions about control, traceability, quality, security, and evolution.

Moving from a one-shot prompt to an architecture with workers, LangGraph, LangChain, tracing, and authoritative publication was not about adding complexity for its own sake.

It was about making explicit the complexity the product already had.

For me, that is the central lesson of this transition towards AI Engineering.

The model call produces a proposal. The architecture determines whether that proposal can be understood, validated, and turned into a product.

#AIEngineering #LangGraph #LangChain #Supabase #Observability