How to Adopt AI Agents: 4 Stages from Task Audit to Full Rollout

Adopt in 4 stages

Following these 4 stages reduces rework along the way. At every stage, write down what you decided on paper or in a file.

1. Task audit

First, list the department's work. For each job, fill in the following items.

Current time spent becomes the baseline for the later evaluation. Measure it at this stage, even roughly.

From the list, choose one first candidate. Good fits have fixed steps, results people can check, and mistakes that can be fixed in-house.

2. Small pilot

Once you have a candidate, pilot it in a narrow scope. Pilot one job. Keep the group of users small. Set the period in advance.

Anthropic advises those building AI agents to start with the simplest setup and add complexity only when needed. For highly autonomous agents, it also advises extensive testing in sandboxed environments with appropriate guardrails. Adopters can follow the same idea and start where the impact is small.

• Prepare test data. Before using production data, check behavior with fictional or past data
• Give the AI separate test accounts. Do not hand over production account permissions at the start
• During the pilot, a person approves every send and every publish
• Decide how to stop it. Write down who stops it and with what action

When you reach the point of entering real customer or employee data, check how personal data is handled. This is covered below under “What to decide in-house.”

3. Evaluate

Set the evaluation criteria before the pilot. If you set them afterward, you tend to pick only the numbers that look good.

• Accuracy: the share of outputs a person had to correct
• Time: how time per run changed from the time measured in the audit
• Review time: time spent checking and correcting
• Stops: how often it stopped due to errors or unexpected input
• User feedback: ease of use and where people got stuck

Even if work time drops, you gain nothing if checking and fixing take just as long. Look at work time and review time together.

Record results every time during the pilot. If you write them from memory afterward, the runs that went well stand out. A table with the date, items processed, items corrected, review time and reasons for stops makes later totals easier.

Keep questions for the provider alongside this table. You can then work out with the provider whether an issue can be fixed in settings or comes from how the AI agent is built.

Based on the results, choose one of three options: move to full rollout, change the scope or settings and pilot again, or not use it for this job. If you decide not to use it, record why. That helps when picking the next candidate.

4. Full rollout

When moving to full rollout, settle the points that could stay vague during the pilot.

• Manual: what the AI does and where people check
• Owners: the person who does daily checks, and a backup
• Exceptions: contacts and steps for when the AI stops or a wrong output goes out
• Review timing: when to revisit settings and scope
• Provider documents: where to keep usage notes, terms and setup guides

The “AI Guidelines for Business (Version 1.2)” from Japan's Ministry of Internal Affairs and Communications (MIC) and Ministry of Economy, Trade and Industry (METI) list points for businesses that use AI. These include following the usage notes set by the provider and staying within the intended scope, confirming the system works as specified, and properly storing and using the documents provided. Include steps to check these in your rollout manual.

Example: one job through the 4 stages

Take the job of entering contact form inquiries into a customer list and drafting replies, and follow it through the 4 stages.

1. Audit: a daily job with fixed steps. Today a staff member copies each inquiry by hand. Measure the work time per month
2. Small pilot: one staff member pilots it for a set period. First check behavior with past inquiries. Do not send replies; stop at drafts
3. Evaluate: record how many list entries a person corrected, time spent fixing drafts, and the number of stops. Compare with the time measured in the audit
4. Full rollout: name the person who checks and sends drafts, and a backup. Also set when to revisit how fields are split

Common pitfalls

• Scope too wide: hand over a whole department's work at once, and you cannot isolate what went wrong
• Criteria set afterward: with no baseline, you cannot decide whether to continue
• No one assigned to review: errors go out because no one looks at the AI's output
• Steps differ by person: the steps to give the AI are never settled, so the setup never comes together
• Looking only at the monthly fee: AI model fees and connected tool fees show up later

Each of these tends to happen when a stage is skipped. The more you are in a hurry, the more it pays not to skip the audit and the setting of criteria. You reach full rollout sooner that way.

What to decide in-house

Before moving through the stages, decide three things in-house: data handling, permissions, and owners.

Data handling

On June 2, 2023, Japan's Personal Information Protection Commission (PPC) issued an advisory on using generative AI services. It gives businesses two points of caution.

• When entering prompts that contain personal information, fully confirm that this stays within what is needed to achieve the purpose of use for that information
• Entering prompts that contain personal data without the person's consent, where the data is then handled for purposes other than producing the response, may violate the Act on the Protection of Personal Information (Japan). So fully confirm, among other things, that the provider does not use the data for machine learning

With an AI agent, data passes through several places: the agent itself, the provider of the AI model it uses, and connected tools. Ask each provider how data is handled at each point.

Permissions

Limit the permissions of the accounts you give the AI to what the job needs. Set actions that affect people outside—sending email, publishing posts, deleting data, payments—to require human approval.

Issue and manage API keys and passwords in-house. Make sure you can revoke them right away when the person in charge changes roles or you stop using the agent.

Owners

The AI Guidelines for Business say that businesses using AI should take responsibility for deciding whether to use the AI's output in their business. So name a business owner, and also decide who makes the call to stop it. The same guidelines also mention setting up, within reason, a contact point for inquiries from people involved.

Cost structure

The cost of an AI agent is the sum of several parts. Amounts vary widely by job and build, so we show only the structure here.

• Initial costs: setup and installation work, internal preparation
• The AI agent's monthly fee
• AI model fees: billed by the model provider when the agent runs on your own API key
• Fees for connected tools: plans for services you use, such as a CRM or social media management tool
• Staff time: checks, approvals and handling exceptions

When comparing options, compare this total, not just the monthly fee. You can estimate staff time from the review time measured during evaluation.

Adopting through Employee Store

On Employee Store, you can adopt AI agents with a one-time purchase or a monthly plan. Payments are processed by Stripe, and pricing is one-time, monthly, or a setup fee plus monthly. You can cancel a monthly plan from contract management in the deal room.

After purchase, the seller delivers what is needed for setup through the on-site delivery box. The buyer checks the delivery, confirms receipt, and can then write a review. You can use this same flow during a small pilot. Detailed steps are in the setup guide. The basics of AI agents are covered in What Is an AI Employee.