Beyond the LLM Wrapper: Engineering Paglipat’s AI Concierge

The chat box is the least interesting part of Paglipat’s AI concierge.

Relocation questions combine rules that change, personal circumstances, owned travel guides and live prices. A thin wrapper would send all of that to a model and trust whatever came back. It might remember an old visa rule, invent a date for a hotel search or keep calling a tool after the useful work was done.

I built the opposite. The model can decide which bounded tool fits a question, but TypeScript decides whether the arguments are valid. Verified data supplies visa rules. The hotel and flight services supply prices. The Worker owns memory, privacy, budgets and retention.

This is the deeper AI layer behind Paglipat, an independent product I described in the wider Astro, Go and Flutter architecture. It is a production walkthrough, not an external-client case study or a claim that every assistant needs this exact stack.

Key Takeaways

• Tool calls are proposals. Every argument crosses a deterministic validation boundary before code executes it.
• The router can handle four formulaic categories, but it cannot author a factual reply. Ambiguity escalates.
• Memory is persisted for useful follow-up questions, then bounded by turn, token, replay and retention limits.
• Privacy and cost checks run before inference. The cheap rejection happens before the expensive dependency.
• Prompt caching is a tested billing invariant, not an optimisation I hope remains intact.
• The implementation has a ledger, kill switch, retention job and runbook. Its model-level evaluation is still manual, and I label that limitation plainly.

The model sits inside a deterministic system

The production path is a Cloudflare Worker in front of a hosted model API, D1, KV, a generated guide corpus, an owned hotel service and a third-party flight-price API. The current model configuration uses xAI’s OpenAI-compatible chat-completions API with grok-4.3. The provider is a dependency inside the design, not the design itself.

The Worker starts with controls that do not need a model: parse the request, check the daily budget, increment the IP bucket and verify a new session with Turnstile. A router then decides whether a formulaic request can end locally or whether the concierge loop should run. The loop may call tools, but it has a hard iteration cap.

That order is deliberate. An invalid body should not spend a network request. A bot that has exhausted an IP allowance should not reach Turnstile. A greeting should not reach the full prompt and tool set. A failing tool should become a bounded failure result, not tear down the response.

The static Astro site still owns the content and UI. The Go services still own travel data. D1 owns durable chat state and the authoritative reconciliation ledger for usage the Worker successfully records. KV provides expiring rate and spend keys. Keeping those responsibilities separate is the same reason I split the rest of Paglipat’s live architecture.

Tool calls are proposals, not permission

The concierge exposes five tools: check_visa, get_checklist, read_guide, search_hotels and flight_prices. Their descriptions state when to call them, not only what they return. That matters because a conservative model can answer from memory unless the trigger is explicit.

Here is the shortened shape of the hotel tool in production:

{
  name: 'search_hotels',
  description:
    'Search live hotel availability and prices. Use this when the user ' +
    'asks where to stay in a specific city and has given, or can give, ' +
    'dates. Do not guess dates.',
  input_schema: {
    type: 'object',
    properties: {
      city: { type: 'string' },
      checkin: { type: 'string', description: 'ISO yyyy-mm-dd.' },
      checkout: { type: 'string', description: 'ISO yyyy-mm-dd.' },
      adults: { type: 'integer', minimum: 1, maximum: 10 },
    },
    required: ['city', 'checkin', 'checkout', 'adults'],
    additionalProperties: false,
  },
}

The JSON schema helps the model construct a valid proposal. It is not an authorisation boundary. The handler calls a separate validator before it resolves a city or touches the hotel API. Invalid input returns a structured failure with an instruction to ask for the missing detail.

This gives the system two independent checks. The model sees a strict schema. The runtime still assumes the model may send malformed JSON, invent a field or supply a date that does not exist.

The boundary also narrows what comes back. The hotel handler maps each result to the few fields the answer needs: name, star rating, review score, nightly rate, breakfast and booking URL. It deliberately excludes Agoda’s free-text overview. Third-party prose is an unnecessary prompt-injection surface when structured fields can answer the question.

The router can decline work, but it cannot answer facts

The first version of the router could have been another small chatbot: classify the request and compose a short reply. That would have made the cheap tier capable of smuggling a factual claim past the concierge. “You do not need a visa for a short stay” contains no price or day count, but it can still be dangerously wrong.

I removed that capability structurally. The router returns strict JSON and can handle only greeting, thanks, off-topic or injection. The Worker, not the model, renders canned localised copy for those categories.

export function parseRouterOutput(raw: unknown): RouterVerdict {
  const escalate = (category: string): RouterVerdict =>
    ({ tier: 'escalate', category });

  if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) {
    return escalate('malformed');
  }

  const result = raw as Record<string, unknown>;
  if (result.tier === 'escalate') {
    return escalate(typeof result.category === 'string'
      ? result.category
      : 'unknown');
  }
  if (result.tier !== 'handle' ||
      typeof result.category !== 'string' ||
      !HANDLE_CATEGORIES.has(result.category)) {
    return escalate('malformed');
  }

  return { tier: 'handle', category: result.category as HandleCategory };
}

Malformed, unknown and ambiguous outputs escalate. The safe failure costs another model call; it does not invent a travel rule. Once a session has escalated, subsequent turns stay with the concierge so “what about my spouse?” retains the right context.

This is tiered routing by responsibility and token budget. Both current model constants happen to be grok-4.3; I am not claiming that two different model families are deployed. The useful boundary is what the router is permitted to do.

Five tools connect the conversation to verified and live systems

The clearest way to understand the tools is to follow two questions.

A DTV visa question goes through check_visa. The handler validates every field, loads a versioned rules file, verifies the consulate identifier, then runs deterministic eligibility and cost functions. The result includes verifiedAt, a source note and mustSayVerifiedAt: true. The model can explain the result, but it does not supply the financial threshold from memory.

A hotel question follows a different path. Dates are mandatory because the wrong dates can return perfectly plausible wrong prices.

The handler resolves an English city name to the catalogue’s city identifier, queries the Go hotel service, limits the result to three properties and wraps the changing price with an explicit note. If the dependency fails, it returns hotels_unreachable. The model can tell the visitor it could not check live rates; the Worker does not leak a raw upstream failure.

The remaining tools follow the same principle:

read_guide is worth calling out. A slug must pass a conservative regular expression and exist in the generated CHAT_CATALOG. That second gate stops a retired document from remaining fetchable because an old JSON file still happens to exist at the origin.

This is retrieval, but not vector RAG. The system selects from a deterministic catalogue and fetches an owned document by exact slug. For rules where the wrong answer matters, a typed deterministic calculator is a better fit than asking similarity search to find the right paragraph.

Memory is useful only when it is bounded

Without memory, a follow-up such as “what about my spouse?” has no referent. Unbounded replay creates the opposite problem: every turn becomes slower, more expensive and more exposed than the one before it.

Paglipat stores sessions and turns in D1. Before a concierge request, it loads the last six turns that have a non-empty model-authored answer, oldest first. Router-handled greetings are excluded because canned copy carries no useful context. A stream that emitted some text and then failed is currently stored and can be replayed; only an empty failed response is excluded. Filtering incomplete streamed answers is therefore a remaining reliability improvement, not a control the present implementation has.

The turn insert and the session counter update run in one D1 batch:

await db.batch([
  db.prepare(
    `INSERT INTO chat_turns
       (session_id, created_at, question, locale, tier,
        tools_used, guides_read, ph_distinct_id, answer)
     VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`
  ).bind(/* bounded turn fields */),
  db.prepare(
    `UPDATE chat_sessions
     SET turn_count = turn_count + 1,
         output_tokens = output_tokens + ?
     WHERE id = ?`
  ).bind(t.outputTokens, t.sessionId),
]);

That atomicity is a cost control. The same counters enforce the session limit. If the turn were committed but its counter update failed, the persisted state would silently undercount future usage.

This is session continuity, not a claim of long-term personalised memory. It is bounded by six replayed turns, thirty total turns, sixty thousand output tokens and a retention job.

Privacy controls belong before inference

Privacy is easiest to promise in copy and hardest to prove in a request path. The concierge makes several choices before a prompt leaves the Worker.

The request parser constrains message length, locale, currency, UUID shape and the current page. The page must be a same-origin path. It rejects both protocol-relative paths beginning // and backslashes, because the WHATWG URL parser can normalise a backslash into a cross-origin URL.

The IP address is hashed with SHA-256 and a salt before it is stored. Raw IPs do not enter D1. New sessions require a valid Turnstile challenge based on whether the session exists, not whether the caller supplied a UUID. That distinction closed a bypass where a caller could generate a fresh, well-formed ID and avoid the “new session” challenge.

There is also a deliberately narrow inbound filter for passport-machine-readable zones, passport-number patterns and grouped payment-card numbers. When it fires, three things happen:

1. the visitor receives a safe local response;
2. the model never receives the text;
3. the stored question becomes [redacted sensitive input].

This is not comprehensive PII detection, and I do not describe it that way. It is a cheap pre-inference barrier for the most obvious material that should never enter this travel assistant.

Transcripts are deleted after ninety days by a scheduled Worker handler. The cost ledger is kept separately because it stores model, token counts, cache-token counts and micro-USD, not question text or an IP hash. That separation makes long-run reconciliation possible without retaining the conversation indefinitely.

Cost control is a chain, not one token cap

One max_tokens value does not control the cost of a tool-using conversation. A public endpoint needs limits around the request, session, model loop and whole day.

The current configured bounds are deliberately visible in one pure constants module:

Those are limits, not evidence of savings or normal user behaviour.

The guard chain is ordered by cost: parse, budget KV read, IP bucket, then Turnstile. The daily budget is checked again inside the tool loop before every model step. When the provider returns usage successfully, the Worker prices the input, output and cache-read tokens and writes them to the D1 reconciliation ledger.

KV provides fast expiring counters, but its increment is a read followed by a write. Concurrent requests can lose an update. I accept that for a blast-radius limiter and say so in the code; it is not the accounting system. D1 is the reconciliation record.

The runbook adds the last layer: an emergency command pushes the current day’s spend key over the cap. Once that KV update is visible, subsequent requests stop before a model call. A code-level zero budget is one deploy away.

Prompt caching became a tested billing invariant

The largest input to the concierge is the stable part: system rules, the article catalogue, the verified visa digest and tool schemas. Dates, locale, currency, current page and conversation history are volatile.

If a volatile value enters the prefix, the API does not report a configuration error. Cache reads quietly fall to zero and every turn pays the full input rate. I therefore build the request with a stable first message and move request-specific context into the final user message:

const messages: XaiMessage[] = [
  {
    role: 'system',
    content: buildSystemText(args.catalog, args.visaDigest),
  },
  ...args.history.map((turn) => ({
    role: turn.role,
    content: historyContent(turn.content),
  })),
  {
    role: 'user',
    content:
      `${buildContextPreamble(args.ctx)}\n\n` +
      wrapUserMessage(args.message),
  },
];

Tool ordering is frozen for the same reason. Reordering equivalent schemas changes the prefix bytes and discards reuse. Tests assert that dates, locale, page and session values do not leak into the stable system text.

The deployment runbook also checks the result in production. Across the first six earlier production concierge turns recorded in that runbook, the ledger observed 53,702 cache-read tokens against 11,662 written input tokens. That is a small, dated operational sample used to verify the expected one-write-many-reads shape. It is not a representative cache-hit rate, benchmark or cost-saving claim.

The tool loop has to fail safely

The concierge streams text while also accumulating tool calls. When a call arrives, the Worker parses its JSON arguments, validates and executes the handler, then appends a role-specific tool result before the next model step.

for (let i = 0; i < TOOL_MAX_ITERATIONS; i++) {
  const still = checkBudget(await readSpendToday(env.CHAT_KV, dayKey));
  if (!still.ok) {
    await send({ type: 'error', error: 'daily_budget_exhausted' });
    break;
  }

  const step = await streamChatCompletion({
    apiKey,
    convId: sessionId,
    body: { ...params, messages },
    onText: async (text) => send({
      type: 'text',
      text: sanitizeAnswer(text),
    }),
  });

  await bill(MODEL_CONCIERGE, step.usage);
  if (step.tool_calls.length === 0) break;
  // Parse, validate, run and append each tool result.
}

Malformed tool JSON becomes an invalid-input outcome. An unknown tool becomes unknown_tool. A handler exception is caught and returned as tool_failed. The model receives a small hint to acknowledge that it could not retrieve the data rather than fill the gap from memory.

The browser and model also receive different views of a successful result. Deep links and result cards go directly to the browser as typed SSE frames. The model sees only the data it needs to explain the answer. A failed tool sends no empty card, because an empty card looks like a successful search with no results.

The loop stops after eight iterations even if the model still wants tools. Content filtering and length termination become explicit terminal events. If a visitor navigates away during a card write, that decoration cannot abort billing and turn persistence. In finally, the stream closes and the Worker attempts to record the turn exactly once.

Production work continues after deployment

The repository includes an operations runbook because “deployed” is not the final state of a public model endpoint.

It covers where the public Turnstile site key and private secret belong, how to migrate D1 remotely, why the static site must deploy before the Worker that fetches its guide corpus, how to confirm cache reads, how to trigger rate and budget controls, how to purge transcripts manually and how to query spend by day and model.

The automated suite exercises request parsing, budgets, rate limits, Turnstile responses, sensitive-input detection, prompt construction, routing, tool schemas, handlers, cost pricing, retention and the xAI streaming adapter. The production red-team table adds eight model-level cases: changing visa rules, refusal to collapse conditional answers, tool-grounded financial proof, unsupported-country boundaries, prompt injection, missing hotel dates, greetings and off-topic questions.

All eight were recorded as passing when the runbook was written. They are still manual, paid checks. They do not form an automated merge gate, and they do not prove a general accuracy percentage.

The observability boundary is similarly honest. The system has structured logs, tool names, guides read, token accounting and a cost ledger. It does not yet have one trace joining router decision, every tool latency, model latency, feedback and business outcome. The stored ph_distinct_id is currently null, so I cannot claim a measured link from a conversation to an affiliate conversion.

What I would improve next

The next step is not another tool. It is a tighter quality loop.

I would turn the existing manual cases into a versioned set of fifty to one hundred scenarios covering router decisions, should-call and should-not-call behaviour, argument validity, source facts, refusals, languages and multi-turn repairs. Deterministic checks can run on every change. Paid model comparisons can run on demand or nightly and report quality, latency and estimated cost by prompt and model version.

I would also add a privacy-safe trace identifier across router, model and tool steps, then join explicit feedback and consented conversion events without making raw conversation text the default telemetry.

Retrieval experiments belong after that. The current exact catalogue and typed calculators are a strength for high-risk rules. If broader discovery across the travel corpus becomes the problem, I can compare lexical, embedding and reranked retrieval against a labelled query-to-source set instead of adding a vector database by fashion.

The model is one dependency inside the product

Paglipat’s concierge is useful because the probabilistic part has a deterministic shell. Tools provide facts. Validators decide what may execute. Memory has limits. Privacy checks happen before inference. An operational kill switch can stop further model calls. The runbook tells me what to inspect when any of those assumptions changes.

That is what I mean by applied AI engineering: not model training, and not a chat box attached to a landing page. It is product engineering around a model that can be useful, expensive and wrong.

You can try the live product at paglipat.com, see the wider Paglipat case study, or compare it with the smaller, deliberately constrained davthecoder.com site assistant. My Applied AI and ML journal keeps the boundary between shipped product AI and the deeper ML work I am still learning explicit.

Frequently Asked Questions

Is Paglipat’s concierge a RAG system?

It retrieves owned documents from a generated catalogue by exact slug, but it does not use embeddings or a vector database. High-risk visa facts come from deterministic rules and calculators. I describe it as tool-based retrieval, not vector RAG.

Is it a multi-agent system?

No. It is one bounded request pipeline with a router and a concierge tool loop. Calling every model boundary an agent would make the architecture sound more autonomous than it is.

Does Paglipat train or fine-tune a model?

No. It uses a hosted model API. The engineering work is orchestration, tools, validation, grounding, privacy, cost control, persistence, testing and operations rather than model training or fine-tuning.