To measure AI ROI in projects, you need a baseline before the AI starts, one narrow task to test it on, and four numbers: time saved, quality kept, cost spent and how often people trust the output. Skip the baseline and you will end up guessing. This post gives you a method you can run in a month, with your own numbers instead of someone else's.
Key takeaways
- No baseline, no ROI. Measure the task before the AI touches it.
- Measure one task, not "productivity".
- Count review time as a cost. It is the one people forget.
- Approval rate tells you more about an agent than any demo.
- If the numbers are flat, that is a result too.
Why most AI ROI claims fall apart
Most AI ROI claims share three problems. They measure activity instead of outcomes, they ignore the cost of checking the AI's work, and they compare against a memory of how long things used to take.
You can avoid all three with a little discipline. You do not need a data team.
Step 1: Choose one task worth measuring
Pick a task that is:
- Repeated, at least weekly.
- Bounded, with a clear start and end.
- Owned by one person who can judge the result.
- Low risk, so a bad output does not cost a client.
Good candidates are a weekly status digest, sorting new requests, triaging bugs or drafting a PRD from a brief. A bad candidate is "help the team be faster".
Step 2: Record a baseline
For two to four weeks before the pilot, record how the task is done today. Keep it simple.
| What to record | How | Why |
|---|---|---|
| Time per run | A timer on the task | The core of the saving |
| Runs per week | Count them | Scales the saving |
| Rework | Did it need fixing after? | Quality before AI |
| Delay | Time from trigger to done | Speed of response |
In AlianHub, put a timer on the task each time someone does it. Logged time against estimates and timesheet data make the baseline less of a guess.
Tip: do not reconstruct the baseline from memory. People remember the bad weeks. Write it down as it happens.
Step 3: Run a narrow pilot
Turn on AI for that one task only. Keep everything else the same.
Start in "suggest only" mode. In AlianHub, that means an agent at L0 or L1, where every change goes to the AI Inbox for a person to approve. That gives you clean data on what the AI proposed and what people accepted.
Run it for the same length of time as the baseline.
Step 4: Measure four things
Time
Time saved is the baseline time minus the new time. The new time must include review.
Review time is not free. If you forget to count it, your ROI is fiction.
Quality
Did the output need fixing? Compare rework before and after. A faster digest that needs rewriting is not a saving.
Cost
Add up what the AI cost to run. In AlianHub, each agent has a monthly spend cap and a daily action limit, and the workspace can set a daily AI budget, so the cost side is visible rather than a surprise. AlianHub's AI controls also show cost per agent and per model.
Trust
Trust is the metric that predicts whether people keep using it. Two signals help:
- Approval rate: how many proposals were approved versus declined.
- Undo rate: how often people reversed something the AI did on its own.
AlianHub shows error rate, approval rate and cost per agent and per model, plus a quality view over 7, 30 or 90 days.
Step 5: Do the simple maths
You do not need a model. Use this:
- Time value = (baseline minutes per run minus new minutes per run, including review) x runs per period x an hourly cost you agree on internally.
- AI cost = spend for the period.
- Net = time value minus AI cost.
- Then check quality and trust. A positive net with a falling approval rate is a warning, not a win.
We are deliberately not giving you example numbers. Plug in yours.
Reading the results honestly
| What you see | What it probably means | What to do |
|---|---|---|
| Time down, approvals high | Good fit | Consider more autonomy on low-risk actions |
| Time down, approvals falling | The AI is drifting or the task changed | Review the skill and the brief |
| Time flat, approvals high | The task was not the bottleneck | Pick a different task |
| Time up | Review is costing more than it saves | Narrow the job or stop |
A pilot that tells you to stop has still paid for itself.
How to do this in AlianHub
- Baseline with timers. Start a timer each time someone does the task by hand for a few weeks.
- Assign the task to one agent. Pick it in the Agents tab at L1, so every change goes to the AI Inbox.
- Cap the cost. Set a daily action limit and a monthly spend cap before the first run.
- Log review time. Reviewers start a timer while they work through the AI Inbox.
- Read the health views. Check approval rate, error rate and cost per agent and per model over 7, 30 or 90 days.
- Decide. Promote to L2, keep, or pause it with Run now, Pause or Stop.
A business scenario
A product team tests the Reporter digest on its weekly leadership update. Timers record the old manual effort, then the reviewer's time once the agent drafts it.
After the pilot, the team compares time logged, spend against the cap and approval rate. The digest goes out on schedule, leadership sees stuck work sooner, and the lead has numbers from their own workspace to justify the next agent.
Common measurement mistakes
- Measuring several tasks at once, so nothing is clear.
- Changing the process and the AI at the same time.
- Counting only the AI's speed, not the human's review.
- Ignoring the cost of setup and tuning.
- Letting the person who chose the tool be the only judge.
When to expand
Expand when one task shows a clear net benefit and steady trust for a few weeks. Then:
- Move the agent from suggestions to low-risk actions.
- Pick the next task, and baseline it first.
- Keep the review date in the calendar.
For a list of what to check before giving an agent more freedom, see our AI agent checklist. For the common traps, see AI productivity myths.
FAQ
What is a good AI ROI for a project team?
There is no universal number, and we will not invent one. A good result is any task where time saved, minus review and AI cost, stays positive while quality and approval rates hold.
How long should an AI pilot run?
Long enough to see a pattern, usually a few weeks, and the same length as your baseline. Short pilots tend to measure novelty.
Should I include setup time?
Yes, at least once. Setting up skills, briefs and approvals takes time, and it belongs in the first period's cost.
Where do I see AI cost in AlianHub?
Each agent has a spend cap and action limit, and the AI controls page describes the health views for error rate, approval rate and cost per agent and per model.
Try the method on one task this month. You can run the pilot in the live demo first or self-host AlianHub for free, and share this with whoever signs off your AI budget.