Introduction

Building Reliable LLM Applications in Java put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Java put the same discipline in JUnit terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a JUnit test either passes or fails against a fixed expected value; an LLM’s output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assertEquals.

Introduction

Building Reliable LLM Applications in Python put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Python put the same discipline in pytest terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a pytest assertion either passes or fails against a fixed expected value; an LLM’s output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assert.

Introduction

Most Java codebases have plenty of tests and still get burned by production bugs. The problem is rarely quantity — it’s that the tests exercise the wrong things: they assert internal wiring instead of behavior, they mock away every collaborator until nothing real is left, or they cover the happy path ten times and the failure path zero times.

Good tests are fast, deterministic, and tell you something true about behavior. This post covers the practices that make JUnit 5 and Mockito tests actually earn their keep.

Introduction

Python’s testing tools are lightweight enough that it’s easy to write a lot of tests without writing good ones. A suite that mocks every collaborator, duplicates the same assertion ten times with different inputs pasted in by hand, or chases a coverage number will pass in CI and still miss real bugs.

pytest gives you fixtures, parametrize, and monkeypatch — the tools that make it just as easy to write the right tests as the wrong ones. This post covers how to use them well.