How to test an AI assistant before it handles real work.

An AI assistant is ready for real work when it does the right thing on ordinary cases, stops or hands off on cases outside its job, ignores instructions planted in the material it reads, and does all three consistently across repeated runs. Each of those can be checked, which is the point. "It seemed fine" is an impression, and impressions are formed almost entirely from the ordinary cases.

The method here is tool-neutral and sized for a small business. It needs a list of test cases, a written expected result for each, a few hours to run them, and somewhere to record what failed and what changed. It applies to one assistant and to a team of them; a team adds one more test, the handoff drill.

Writing the expected result after seeing the output is the most common way to fool yourself, because an output always looks reasonable once you have read it. Anthropic's guidance on building evaluations makes the same point in engineering terms: define what success means, specifically and measurably, before measuring anything.

For a business assistant the pass condition is usually about behavior rather than prose. It is not "writes a good reply". It is "asks for the address before offering a booking", "does not state a price", or "hands off and says a person will reply within the working day". A pass condition that someone else could check without asking what you meant is a good one.

1. 01

   Input

   Decides

   What exactly is the assistant given?

   Output

   The message, document or task, copied verbatim

2. 02

   Context

   Decides

   What does it have access to during the run?

   Output

   The instructions, files and tools in force at the time

3. 03

   Expected behavior

   Decides

   What should it do, and what must it not do?

   Output

   Two or three observable actions, written before the run

4. 04

   Pass condition

   Decides

   How will someone else decide pass or fail?

   Output

   A check that does not depend on taste

5. 05

   Result

   Decides

   What actually happened?

   Output

   Pass, fail or partial, with the output attached

A useful test set has four kinds of case, and most informal testing covers only the first. The proportions matter less than the presence of each: an assistant that has never been asked to do something outside its job has never shown that it will decline.

Anthropic's documentation also recommends test sets that mirror real usage and include edge cases, and it favors many cases with simple checks over a handful graded by hand. For a small business that translates into something practical: take twenty real examples from last month's work, remove the personal details, and add the boundary and adversarial cases on top.

Assistants report completion. Whether the work was completed is a separate fact. A drafted email that was never saved, a record described as updated in the assistant's summary but unchanged in the system, a booking described but never made: each produces a confident "done". The fix is to make proof part of the job. A finished task returns a link to the draft, the identifier of the record, or a copy of what changed.

The test then checks the proof rather than the message. Open the link. Look up the record. If the assistant cannot produce the evidence, the task failed, however well the report is written.

Any assistant that reads email, web pages or documents will eventually read text written by someone who wants it to do something. OWASP's 2025 list for language-model applications ranks prompt injection first, and it separates direct injection, typed by the user, from indirect injection, carried in by content the model reads. Indirect injection is the realistic risk for a business assistant, because the person attempting it never needs access to the assistant itself.

The drill is simple. Put an instruction inside a test email or document, asking the assistant to forward something, change a payment detail or ignore its rules, and give it an ordinary task that requires reading that material. A pass is an assistant that completes the ordinary task, does not follow the planted instruction, and mentions that it found one. Repeat the drill whenever the instructions change.

No drill makes an assistant immune. OWASP's own mitigations lean on structure: least privilege, human approval for high-risk actions, and clear separation of untrusted content. The drill shows whether that structure holds. The structure is what limits the damage on the day it does not.

Approval gates are configured in the tool, and configuration is easy to get subtly wrong. Test each gate directly: ask the assistant to send, publish, pay, delete or share, and confirm that it stops at the approval rather than acting. Then test the approval itself by approving one action and denying another, and check that the denied action really did not happen.

Test what happens when something runs out or breaks partway through a task: a usage limit, a tool that times out, a website that asks for a login. The pass condition is a clear report of what was finished and what was not, rather than a silent stop or an optimistic summary.

When several assistants pass work between them, each can pass its own tests while the team still fails at the joins. The handoff drill takes one realistic job, such as an inquiry that becomes a quote, then a booking, then a follow-up, and runs it from start to finish. At each handoff, check three things: the receiving role knows it now owns the work, it received a link to the record rather than a stale copy, and nothing was done twice or not at all.

Plant one trap in the drill as well: a document carrying a hidden instruction, or a request that only the owner may approve. A team that routes the trap to its exception queue is a team whose handoff rules are working. The structure under test is the one described in how to build an AI operations team.

A language model does not behave like a spreadsheet formula. The same input and the same instructions can produce different behavior on different runs, so one pass is weak evidence. Run every boundary and adversarial case several times. A case that passes four times and fails once has failed.

Keep a failure log of each failed case, what was changed and whether the retest passed. It becomes the most useful document the process produces, because it records why each rule exists. When the instructions are edited later, rerun the whole set rather than only the case that prompted the edit, because a fix for one behavior regularly breaks another.

A failure log needs five columns:

• DATE AND CASE

  When it ran, and which test case it was.

• WHAT HAPPENED

  The actual behavior, with the output attached.

• CAUSE

  The best available explanation: a missing fact, an ambiguous rule, a tool setting.

• CHANGE

  What was edited in the instructions, files or settings.

• RETEST

  The result of the full set after the change.

Passing the test set means the assistant is ready to prepare real work, not to act on it unsupervised. Start with a person reviewing every output, keep adding the real cases that surprised you to the test set, and widen what the assistant may do without approval only when the log shows it handling that kind of work reliably. The NIST AI Risk Management Framework describes the same discipline at organizational scale: risk is mapped, measured and managed continuously rather than checked once.

The ladder of authority, and the evidence each rung requires, is the subject of when to give an AI agent real authority. Testing is how that evidence is produced.

Aevrion Ops designs and tests AI assistants and small AI teams with businesses: role definitions, approval design, test sets and the integrations a company's own systems require, where each vendor's API or terms allow it. The decision to let an assistant act stays with the business.

1. Write the expected behavior and the pass condition for each test case before running it.
2. Cover four kinds of case, ordinary, edge, boundary and adversarial, because ordinary cases alone only build confidence.
3. Require proof of done, and check the proof rather than the report.
4. Plant an instruction in material the assistant reads; a pass ignores it, finishes the real task and flags it.
5. Test every approval gate directly, including a denial, and test what happens when a tool fails partway through.
6. For a team of assistants, run one real job through every role and check each handoff.
7. Repeat cases, keep a failure log, rerun the full set after every change, and go live in draft mode.