An AI agent can report that it finished its task. Another agent can report that its task is done too. Yet the software they produced together may still be incomplete, inconsistent, or incorrect.

A planner might write a requirements list, a developer might produce code, and a tester might run a test suite. Those activities matter, but their completion does not tell us whether the finished application does what the user asked. To answer that, we need to evaluate the final result against the original requirements.

A concrete example: a task-tracking application

Imagine a request for an application where users can create tasks, mark them complete, and see only their own tasks. A team of agents divides the work among planning, implementation, and testing.

The planner lists task creation and completion, but misses the requirement that users must see only their own tasks. The developer implements the listed features. The tester checks that tasks can be created and completed. Each agent can finish its assigned work, while the application still fails an important requirement.

That example is hypothetical, but the distinction is practical: task completion describes what the agents did. Final evaluation checks what the application actually delivers.

Completeness: is anything missing?

A solution is complete when it covers the requirements within the agreed scope. For the task-tracking application, that means checking task creation, completion, and access restrictions, not just checking that the application starts.

Start with the original request and turn each requirement into an acceptance criterion. An acceptance criterion is a condition the finished application must satisfy. For example:

• A signed-in user can create a task.
• The same user can mark that task complete.
• Another user cannot view that task.

Compare each criterion with the final application and its test results. A feature described in documentation or present as a button is not necessarily implemented. If a requirement is vague, record the assumption or resolve it before treating the check as conclusive.

Consistency: do the pieces agree?

A solution is consistent when its parts use compatible assumptions and do not contradict one another. An agent might implement an API that accepts a numeric task ID, while another builds a client that sends an incompatible value. Each part may look reasonable in isolation; the integrated application can still fail.

Check that interfaces, data formats, and access rules agree across the application. Integration tests help reveal disagreements that separate component tests can miss. Consistent code is not necessarily correct, however: all components can agree on the same wrong interpretation of a requirement.

Correctness: does it behave as required?

Correctness concerns the behavior of the final application for the requirements and cases being evaluated. A task may appear in the interface, but does marking it complete actually save the change? Does an unauthorized request return a denial without exposing the task?

Run functional tests for expected behavior and security tests for relevant risks. Include boundary and failure cases where the requirements call for them. A passing test shows that a particular check passed in a particular environment; it does not prove the application correct for every input or safe against every threat.

A prompt you can adapt

Use this prompt to guide an evaluation agent, or adapt it as a checklist for your own review. The agent needs access to the original requirements, the final files, and a suitable test environment to perform the checks.

Evaluate the final application against the original requirements.

1. List each requirement and its acceptance criterion.
2. Identify which requirements are implemented, missing,
   or cannot be verified from the available evidence.
3. Check whether components use compatible interfaces and data.
4. Run functional and relevant security tests when tools permit.
5. Report each test result, failure, and untested requirement.
6. State any assumptions and limits of the evaluation.

Do not mark a requirement as passed because an agent says
its task is complete. Do not claim a test ran if it did not.

Original requirements: [insert requirements]
Final application: [provide files and access instructions]

The prompt organizes the review; it does not guarantee that the evaluator will find every problem. Inspect the evidence, particularly for requirements involving security or other consequential behavior.

What to record in an evaluation

For each requirement, record the criterion, the check used, and the observed result. Distinguish a failed check from a requirement that could not be tested. For example, if the application cannot connect to its database, a reviewer may be unable to verify whether task completion persists. Calling that feature either a pass or a functional failure without further evidence would be misleading.

When comparing an AI agent team with a single agent, give both approaches the same requirements and evaluate their final outputs with the same checks. Also record differences in tools, time, and cost. Comparing agent activity alone does not establish which approach produced better software.

Benchmarks can help make parts of that evaluation repeatable. For example, SWE-bench evaluates proposed changes against real software issues and associated tests. Its results are useful within the benchmark’s scope; they do not replace testing an application against its own requirements.

Agent reports tell us how the work progressed. Evidence about completeness, consistency, and correctness tells us what the final solution actually delivers—and what remains unverified.