Skip to content
Aventheon

Tests first: the characterization-test approach to upgrading code nobody understands

October 12, 2026

You have inherited a module. It calculates something important, nobody on the team can explain why it does what it does, and it has no tests. Now you need to upgrade the framework underneath it.

The usual instinct is to read the code until you understand it, then write tests for the behavior you now believe in. That works for small code. For anything large, it is slow, and worse, your belief about what the code does is exactly the thing most likely to be wrong.

Characterization tests flip the order. You do not decide what the code should do. You record what it does, and then you hold that behavior fixed while you change everything around it.

What a characterization test is

The term comes from the legacy-code testing literature. A characterization test asserts the current behavior of a piece of code, whether or not that behavior is correct. If the code returns 17.3 for some input, the test says 17.3.

That sounds backwards, but the goal is different from a normal test. A normal test says "this is right." A characterization test says "this is what it does today, tell me if that changes." During an upgrade, change is what you are trying to detect.

You will sometimes capture a behavior that is plainly a bug. Leave it in the test, mark it with a comment, and decide separately whether to fix it. Mixing bug fixes into an upgrade makes it impossible to tell which change caused which difference.

The technique in four steps

  1. Pick seams. Find the narrowest functions or endpoints where you can feed input and observe output without touching the internals.
  2. Choose inputs. Real data is best. Production logs, a database snapshot (scrubbed of personal data), or captured requests all beat invented examples.
  3. Record outputs. Run the current code and save what it returns as the expected value.
  4. Replay after each change. Any difference is a signal to investigate, not necessarily a failure.

A small Python example

Suppose a pricing function nobody understands:

# legacy/pricing.py (we do not know why the rules are what they are)
def quote(order):
    ...

First, a script that records current behavior against a set of real-looking inputs:

# tools/record_golden.py
import json
from pathlib import Path
from legacy.pricing import quote

cases = json.loads(Path("fixtures/orders.json").read_text())
golden = {c["id"]: quote(c["order"]) for c in cases}
Path("fixtures/golden.json").write_text(json.dumps(golden, indent=2, sort_keys=True))

Run it once, on the code as it is today, and commit the output. Then the test:

# tests/test_pricing_characterization.py
import json
from pathlib import Path
import pytest
from legacy.pricing import quote

CASES = json.loads(Path("fixtures/orders.json").read_text())
GOLDEN = json.loads(Path("fixtures/golden.json").read_text())

@pytest.mark.parametrize("case", CASES, ids=lambda c: c["id"])
def test_quote_matches_recorded_behavior(case):
    assert quote(case["order"]) == GOLDEN[case["id"]]

That is the whole technique. It is deliberately dull. Now you can upgrade the Python version, the framework or the database driver, run the suite, and see exactly which cases moved.

If you prefer not to maintain the JSON by hand, snapshot libraries such as syrupy for pytest or Jest's built-in snapshot testing do the same job with less code. The idea is identical.

Making the recorded behavior trustworthy

A golden master is only as good as its inputs. A few habits matter.

Cover the branches, not just the volume. A thousand near-identical orders tell you less than thirty chosen to hit different rules. Run coverage while replaying your cases and look at what is never reached:

pytest --cov=legacy --cov-report=term-missing tests/test_pricing_characterization.py

Add cases until the important branches are exercised.

Freeze the sources of noise. Timestamps, random IDs, floating point ordering and dict iteration in older code all make outputs unstable. Inject a fixed clock, seed randomness, and sort anything unordered before comparing. If a test is flaky on the old code, you have learned something about the old code.

Isolate side effects. If the function writes to a database or calls an API, record what it tries to do rather than letting it do it. Capture the outbound call and assert on that.

Scrub personal data. Using production-derived fixtures is useful, but anything with personal information needs to be anonymized before it lands in a repository.

Where it fits in an upgrade

The sequence we would suggest looks like this:

  1. Characterization tests around the riskiest seams, passing on the current version
  2. Upgrade one version, run the suite, triage every difference
  3. For each difference, decide: expected consequence of the new version, or regression
  4. Update the golden file only for differences you have deliberately accepted, and note why in the commit
  5. Repeat for the next hop

Step 4 is the discipline. Re-recording the golden file to make the build green defeats the purpose. Each change to it should be a conscious decision with a reason attached.

Limits worth being honest about

Characterization tests prove that behavior is unchanged for the inputs you chose. They say nothing about inputs you did not. They also cement bugs and odd edge cases, which is fine for an upgrade but not a reason to keep them forever. As you come to understand the code, replace the recorded tests with ones that state intent.

They are also poor at UI behavior and at timing-dependent code. For those, end-to-end tests at a higher level (a handful of critical user journeys) cover what unit-level recording cannot.

Why this beats reading first

Reading code builds a mental model, and mental models of legacy code are unreliable. Recording behavior builds evidence. After a day of setting up a harness like the one above, you can change a dependency and learn within minutes whether anything observable moved. That is a better position than a week of reading and a hopeful merge.

If you have a module like this and are not sure where its seams are, bring it to a free 30-minute Triage. We can talk through where we would start recording. Or run the instant check to get a quick read on your stack first.