Introduction

Every post in this series has quietly touched a piece of the same problem. Building Agentic Workflows in Java said toolUse.input() is untrusted and must be validated before it reaches your code. Building Reliable LLM Applications in Java said the model will confidently invent facts, so ground it and get typed output instead of parsing prose. Neither post named the thing underneath both statements: anything that crosses from outside your code into the model, or from the model back into your code, is untrusted input — a request body from the network, not a trusted internal value. This post names that boundary directly and gathers the defenses in one place — prompt injection (direct and indirect), input validation, output validation, and PII redaction — with the SAFE pattern shown beside every unsafe one it replaces, since this is the security-forward capstone of the series.

Introduction

Every post in this series has quietly touched a piece of the same problem. Building Agentic Workflows in Python said a tool’s input is untrusted and must be validated before it reaches your code. Building Reliable LLM Applications in Python said the model will confidently invent facts, so ground it and get typed output instead of parsing prose. Neither post named the thing underneath both statements: anything that crosses from outside your code into the model, or from the model back into your code, is untrusted input — a request body from the network, not a trusted internal value. This post names that boundary directly and gathers the defenses in one place — prompt injection (direct and indirect), input validation, output validation, and PII redaction — with the SAFE pattern shown beside every unsafe one it replaces, since this is the security-forward capstone of the series.

Introduction

Building Reliable LLM Applications in Java put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Java put the same discipline in JUnit terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a JUnit test either passes or fails against a fixed expected value; an LLM’s output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assertEquals.

Introduction

Building Reliable LLM Applications in Python put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Python put the same discipline in pytest terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a pytest assertion either passes or fails against a fixed expected value; an LLM’s output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assert.

Introduction

Every LLM ecosystem now has at least one framework promising to make agents easier to build, and every framework post either oversells the abstraction or dismisses it outright. Neither is useful. The only honest way to evaluate a framework is to build the same thing twice — once on the vendor’s raw SDK, once on the framework — and compare what each version actually cost you and actually gave you back.

Introduction

Every LLM ecosystem now has at least one framework promising to make agents easier to build, and every framework post either oversells the abstraction or dismisses it outright. Neither is useful. The only honest way to evaluate a framework is to build the same thing twice — once on the vendor’s raw SDK, once on the framework — and compare what each version actually cost you and actually gave you back.

Introduction

We already covered picking the right model tier for the task and caching a large shared prefix in https://pg-blogs.netlify.app/posts/11-building-reliable-llm-apps-in-java/. Those two lines were the tip of a bigger discipline: LLM cost is not a fixed line item, it’s an engineering variable — one you can measure and shrink with the same rigor you’d apply to database query time or container memory.

This post goes deeper: how input/output pricing actually works, the exact cache_control shape and how to prove a cache hit rather than assume one, the Batches API for work that isn’t latency-sensitive, and model routing — using a cheap model to triage, escalating only the hard cases to a stronger one. The honest framing throughout: measure before you optimize. Every technique here has a cost of its own; applied to the wrong workload, “optimization” makes things slower or more expensive.

Introduction

https://pg-blogs.netlify.app/posts/10-building-reliable-llm-apps-in-python/ closed with a section on picking the right model per task and caching a shared prefix. That was the entry point into a bigger discipline: LLM spend is an engineering variable, not a fixed bill — one you can measure and reduce with the same rigor you’d apply to query latency or memory footprint.

This post goes deeper on four levers: how input/output pricing actually works and why the prefix is usually where the money goes, the exact cache_control shape and how to prove a cache hit instead of assuming one, the Batches API for work that isn’t latency-sensitive, and model routing — a cheap model triaging requests and escalating only the hard ones. The throughline is honest: measure before you optimize. Every lever here has its own cost; misapplied, it makes things slower or pricier, not cheaper.

Introduction

Retrieval-augmented generation (RAG) is the pattern behind almost every “chat with your documents” feature: instead of hoping a model already knows the answer, you find the passages most likely to contain it and hand them to the model as context. Done well, it’s the single highest-leverage technique for keeping an LLM application grounded in facts it didn’t memorize.

This post builds RAG from scratch — no vector database, no framework — so the mechanics are visible end to end: splitting documents into chunks, turning them into embeddings, searching them with nothing more than array math, reranking the results, and generating a grounded answer. It builds directly on the grounding discipline from Building Reliable LLM Applications in Java — “give the model the source material and instruct it to answer only from that material” — by showing where that source material actually comes from. We’ll close with the question every RAG design eventually has to answer honestly: when does retrieval beat simply pasting more into the context window?

Introduction

Retrieval-augmented generation (RAG) is the pattern behind almost every “chat with your documents” feature: instead of hoping a model already knows the answer, you find the passages most likely to contain it and hand them to the model as context. Done well, it’s the single highest-leverage technique for keeping an LLM application grounded in facts it didn’t memorize.

This post builds RAG from scratch — no vector database, no framework — so the mechanics are visible end to end: splitting documents into chunks, turning them into embeddings, searching them with nothing more than array math, reranking the results, and generating a grounded answer. It builds directly on the grounding discipline from Building Reliable LLM Applications in Python — “give the model the source material and instruct it to answer only from that material” — by showing where that source material actually comes from. We’ll close with the question every RAG design eventually has to answer honestly: when does retrieval beat simply pasting more into the context window?