The problem
Testers usually get a story after it is refined and before it is built, and they have to turn it into test cases: what to check, with what data, and what the system should do. Done well, that covers the main path, every way the input can be wrong, the boundaries (zero, the limit, one past the limit) and the states the system can be in. Done in a hurry, it covers the main path and whatever the tester remembers from the last defect.
It breaks in two places. Cases drift away from the acceptance criteria: a criterion changes in refinement, the cases do not, and nobody notices until a defect is argued over. And cases are written from the story alone, without looking at the code, so the branches developers actually added, such as a retry, a fallback or a feature flag, are never tested on purpose.
Good looks like this: every acceptance criterion has at least one case, and you can see which; negative and boundary cases are written down rather than left to instinct; anything in the code that no criterion explains is raised as a question; and the cases arrive in a form your test tool or framework accepts.
How it works
Forge Test is built on demand in your engagement. It reads one story at a time with its acceptance criteria, ideally in the Given/When/Then form that Forge Spec writes. It can also read the code that implements the story, as a pull request diff or a set of files, and an OpenAPI contract when the story touches an API.
It is steered by your test standards: how you name cases, the priority levels you use, the techniques you expect (equivalence classes, boundary values, state transitions), and what test data you are allowed to use. These live in its rules, not in a prompt someone retypes each time.
It writes scenarios grouped by criterion, then cases under each scenario: preconditions, steps, test data and the expected result. Each case names the criterion it covers. Alongside the cases it lists gaps, meaning criteria it could not write a meaningful case for, usually because the expected result is not stated, and code paths with no criterion, such as an error branch the story never mentions.
Then it is checked. The output has to fit a typed schema, and a case that cites no criterion is rejected rather than shown. A coverage table lists each criterion and the cases against it, so an empty row is visible at a glance. That is a structural check, not a judgement on whether the cases are good.
A tester then reviews it, starting with the gaps. The cases come out as Gherkin feature files or as a CSV for your test management tool. Turning the cases into automated tests is a separate step, done by your developers or testers in your own framework.
Where it gets it wrong
Forge Test is good at coverage and traceability. It does not replace a tester’s judgement, and it fails in predictable ways.
- It cannot test what the criterion does not say. If “the user sees an error” never says which error, the agent cannot know the expected message. It lists the case as a gap with the question attached, rather than guessing.
- It writes too many cases. Boundary and negative cases multiply quickly, and some are not worth running. Priorities help, but the tester decides what goes into the regression pack.
- Code is not intent. Reading the code shows what the developer built, not what the business asked for. When the two disagree, the agent reports the mismatch. It does not decide which is right, and it never writes a case whose only purpose is to confirm the code.
- It does not know your environments. Data that exists in one environment and not another, or a third-party sandbox that behaves differently from production, is outside what it can see. The tester fills that in.
We would rather it leave a gap open than fill it with an expected result nobody agreed.
Why not just prompt a chatbot
A chatbot will give you a long list of test cases for any story. The list will look thorough, and you will not be able to tell which criterion each case covers, which criteria have no case at all, or which cases are padding. Forge Test is useful because of what sits around the model: your test standards in its rules, the requirement that every case traces to a criterion, the typed output that rejects one that does not, the coverage table, and the export to your tools. That is the part we build, and the part you keep.
What you get
- In your engagement: we build Forge Test for your stories and test tool, and from then on your testers review and prune its cases instead of writing them from a blank page.
- Under your brand: services firms can run it inside their own testing practice, with their own case templates and techniques in its rules.
- Built to own: we tune it to your test standards and test management tool, and hand over the source.