Test AI on one business task before making it a promise

An AI feature is useful when it improves a real task. The way to find out is to define the task, keep an existing point of comparison and check the quality of the result.

What the research actually studied

In Generative AI at Work, Erik Brynjolfsson, Danielle Li and Lindsey Raymond studied the introduction of an AI assistant using data from 5,172 customer-support agents. The published 2025 paper reports 15% more issues resolved per hour on average, with substantial differences between workers.

Less experienced workers benefited more. The most experienced and highest-skilled agents saw small gains in speed and small declines in quality. This was a particular customer-support setting. It does not establish a 15% saving for every business, every task or an AAG project.

Read the published paper in The Quarterly Journal of Economics, or the authors' university publication summary.

AAG's suggested trial

Pick one bounded task, such as preparing a draft reply from an approved service guide. Keep a person responsible for checking and sending the reply. Start with invented or appropriately cleared examples so the trial does not require unnecessary customer information.

Write down what a good answer must contain: the correct service details, any relevant exclusions and a clear next step. Add examples where the right response is to ask a question or pass the issue to a person.

Then compare the existing process with the assisted one:

• Time: include checking and corrections, not only the time taken to generate a draft.
• Accuracy: check facts against the approved source.
• Completion: confirm the customer question was actually answered.
• Escalation: check that uncertain or unusual cases reach the right person.
• Cost: include the software and the time needed to maintain its source material.

Review both straightforward and awkward cases. A tool that succeeds on a polished demonstration may still struggle with missing information, conflicting instructions or an outdated price list.

Decide what happens when it is wrong

Before making the trial part of daily work, decide who corrects an answer, who updates the underlying information and how staff can return to the existing process. Keep the trial's permissions limited to its task.

If quality improves and the total effort falls, expand gradually. If checking takes longer than doing the task, narrow the use or stop. A useful result can be deciding that a simple form, clearer guidance or ordinary automation would work better.

For tasks with stable rules, begin with our guide to what to automate first. Our workflow automation service explains how we define the task, exceptions and handover.

Source and credit

Erik Brynjolfsson, Danielle Li and Lindsey Raymond, Generative AI at Work, The Quarterly Journal of Economics, volume 140, issue 2, May 2025, pages 889-942. DOI: 10.1093/qje/qjae044. This note uses the published paper's figures, rather than the earlier working-paper figures. The trial checklist is AAG's interpretation, not a procedure tested by that study.