Measuring the ROI of My AI Project Team
I’ve talked about building my AI Project Team.
I’ve introduced the agents.
I’ve talked about hiring my first AI agent and turning an idea into a workflow.
But now comes the question every project manager eventually has to ask:
Is it actually working?
It’s easy to get excited about AI.
It’s easy to build a cool workflow.
It’s easy to watch an agent summarize an inbox and think:
“That’s pretty neat.”
But “pretty neat” isn’t a business case.
If I’m going to build an army of AI agents, I need to know whether they’re actually making me more effective.
That’s where my 20% rule comes in.
What’s the 20% Rule?
My goal is simple:
Can AI help me reclaim at least 20% of the time I currently spend on repetitive, lower-value work?
I’m not saying AI should make me 20% faster at everything.
I’m not saying I should work 20% more.
And I’m definitely not saying:
“AI will eliminate 20% of my job.”
That’s not the objective.
The objective is to reclaim 20% of my working time that can be redirected toward higher-value activities.
Think about an eight-hour workday.
Twenty percent is roughly:
96 minutes.
That’s an hour and a half.
Every day.
That’s significant.
Where Does My Time Actually Go?
Before I can measure improvement, I need to understand where my time is going today.
As project managers, we often have a surprising amount of work that doesn’t feel like traditional project management.
For example:
- Reading email
- Searching for information
- Updating documents
- Creating status reports
- Writing follow-up emails
- Preparing meeting agendas
- Reviewing meeting notes
- Tracking action items
- Creating handover notes
- Updating project systems
- Finding previous decisions
- Chasing outstanding information
- Setting reminders
- Formatting information
- Copying information between systems
None of these activities are necessarily bad.
They’re part of the job.
But they aren’t all activities that require me.
And that’s where I see the opportunity.
Not All Time Is Equal
This is an important distinction.
Suppose I spend an hour preparing a status report.
If AI helps me reduce that to 20 minutes, I’ve saved 40 minutes.
But what do I do with those 40 minutes?
That’s where the real ROI comes from.
If I simply fill those 40 minutes with another administrative task…
I haven’t really improved my job.
But if I use that time to:
- Talk to a customer
- Help a team member
- Resolve a blocker
- Review a project risk
- Improve a process
- Think strategically
- Prepare for an important meeting
- Coach someone
- Build relationships
Now we’ve created value.
Time saved is only valuable when we use that time intentionally.
My AI Time Bank
I like to think about this as creating a Time Bank.
Every time an AI agent saves me time, I make a deposit.
For example:
Morning Email Agent
Save 20 minutes per day.
Status Report Agent
Save 45 minutes per week.
Handover Agent
Save 60 minutes per vacation.
Communication Agent
Save 15 minutes per day.
Information Agent
Save 30 minutes per week searching across systems.
Individually, these might not sound revolutionary.
But add them together.
10 minutes here.
20 minutes there.
30 minutes somewhere else.
Suddenly I’m talking about hours.
And hours become days.
The First Metric: Time Saved
The easiest metric to understand is:
How long did the task take before AI?
versus
How long does it take with AI?
For example:
| Task | Before AI | With AI | Time Saved |
|---|---|---|---|
| Morning email review | 45 min | 20 min | 25 min |
| Weekly status report | 90 min | 35 min | 55 min |
| Customer follow-up emails | 30 min | 15 min | 15 min |
| Handover notes | 120 min | 45 min | 75 min |
| Information searches | 30 min | 10 min | 20 min |
The exact numbers will vary.
That’s okay.
The point is to establish a baseline.
Measure before you automate.
Otherwise, how do you know whether you improved anything?
But Time Isn’t Enough
Here’s where it gets more interesting.
An AI workflow could save me an hour and still be a terrible workflow.
Why?
Because it could introduce errors.
Or miss important information.
Or create additional work.
Or require so much checking that the time savings disappear.
So I need more than one metric.
Metric #2: Accuracy
If my AI Action Agent identifies 20 action items, how many are actually valid?
Maybe:
18 out of 20.
That’s 90%.
That’s useful information.
But what about the two it missed?
Those could be more important than the 18 it found.
So I’m not only measuring:
“How much did AI find?”
I’m also asking:
“What did AI miss?”
Metric #3: False Positives
AI can sometimes see problems where there aren’t any.
An email says:
“We’ll need to revisit the timeline.”
AI might flag:
Potential schedule risk.
Maybe it is.
Maybe it isn’t.
That’s where false positives come in.
If my agent flags everything as a risk, eventually I stop paying attention.
That’s the AI equivalent of the project team that marks everything RED.
Eventually, nobody cares.
The objective is not maximum detection.
It’s useful detection.
Metric #4: Missed Items
This might be one of the most important measurements.
What did I miss without AI?
And what did the AI help me catch?
For example:
Customer follow-up identified: 8
Actions identified: 12
Potential issues identified: 3
Important item I would have missed: 1
That last number could be extremely valuable.
Because sometimes the ROI isn’t measured in minutes.
It’s measured in:
“That would have become a problem if we hadn’t caught it.”
Metric #5: Response Time
AI can also help improve how quickly I respond.
If an important customer request previously sat in my inbox for six hours before I noticed it, and now I identify it during my morning briefing…
That’s an improvement.
The question becomes:
How quickly can I identify and respond to things that matter?
Faster doesn’t always mean better.
But faster awareness can create better options.
Metric #6: Consistency
This is another benefit that’s easy to overlook.
People have good days.
People have busy days.
People go on vacation.
People get distracted.
Processes can become inconsistent.
AI workflows can help create repeatable processes.
For example:
Every morning:
Review → Identify → Prioritize → Brief
Every Friday:
Review → Summarize → Identify outstanding items → Prepare for next week
Before vacation:
Review → Capture → Organize → Handover
The workflow doesn’t wake up one morning and say:
“I don’t really feel like doing the status report today.”
Consistency matters.
Metric #7: Quality
Saving time doesn’t matter if quality drops.
If AI helps me create a report in 15 minutes instead of an hour, but I spend another 50 minutes fixing it…
I didn’t save time.
I created another problem.
That’s why the measurement needs to include:
Total human effort.
Not just:
AI generation time.
The question is:
How much human effort does it take to produce the final acceptable result?
That’s the number I care about.
The Human-in-the-Loop Tax
I’ve started thinking about another measurement:
The Human-in-the-Loop Tax.
Every AI workflow requires some level of human review.
That’s a good thing.
I want the human involved where judgment matters.
But I need to understand how much review is required.
For example:
AI creates first draft: 5 minutes
Human review: 10 minutes
Final editing: 5 minutes
Total:
20 minutes.
If the old process took 60 minutes, that’s a significant improvement.
But if the AI output takes 50 minutes to correct…
The workflow needs work.
The goal isn’t:
“AI generated something.”
The goal is:
“AI reduced the total effort required to produce something useful.”
The 20% Doesn’t Have to Come From One Place
This is important.
I’m not expecting one AI agent to save 20%.
That’s probably unrealistic.
Instead, I’m looking for lots of small wins.
Maybe:
Email: 5%
Reporting: 4%
Information retrieval: 3%
Communication: 3%
Meeting preparation: 2%
Handover: 2%
Action tracking: 1%
That’s:
20%.
It’s the combination that matters.
From Time Saved to Value Created
Eventually, I want to stop talking about time saved.
I want to talk about value created.
If I save two hours per week, what did I do with those two hours?
That’s the question.
Maybe I used them to:
Have better customer conversations.
Help a team member solve a problem.
Identify a project risk earlier.
Improve a broken process.
Prepare better for an important meeting.
Coach someone.
Think.
Yes.
Think.
Project managers don’t always get enough time to think.
We’re often too busy managing the work to step back and think about the work.
AI could help change that.
The ROI Equation
For me, the equation starts to look something like this:
AI ROI = Time Reclaimed + Quality Improvement + Risk Reduction + Better Outcomes
Not simply:
AI ROI = Faster Emails
That’s too narrow.
If an AI workflow helps me identify an issue earlier, that’s valuable.
If it helps me respond to a customer faster, that’s valuable.
If it prevents me from missing an action, that’s valuable.
If it gives me more time to coach my team, that’s valuable.
If it helps me manage more effectively without increasing my workload, that’s valuable.
The Dashboard for My AI Team
Eventually, I want my AI Project Team to have its own dashboard.
Something like:
AI Project Team Scorecard
⏱ Time Reclaimed
12.5 hours this month
📧 Emails Analyzed
1,247
📋 Actions Identified
83
🔄 Follow-Ups Identified
41
🚦 Potential Issues Flagged
14
📝 Reports Assisted
8
✉️ Communications Drafted
32
📚 Information Searches Assisted
57
⚠️ Items Requiring Correction
6
🎯 Estimated Time Reclaimed
18%
And there it is.
I’m approaching my 20% target.
That’s when this stops being an experiment.
It becomes a measurable productivity initiative.
The Danger of Chasing 20%
There’s one important caveat.
I don’t want to become obsessed with hitting 20%.
Because I could hit 20% by automating things that shouldn’t be automated.
That’s not success.
The objective isn’t:
“Automate as much as possible.”
The objective is:
“Automate the right things.”
Some activities should remain human.
Some decisions should remain human.
Some conversations absolutely should remain human.
The 20% should come from repetitive, predictable, information-heavy work where AI can provide assistance without taking away human accountability.
The Bigger Goal
The 20% rule isn’t really about 20%.
It’s about changing how I think about my work.
Instead of accepting repetitive tasks as:
“That’s just part of being a project manager.”
I’m asking:
“Does it have to be done this way?”
Could it be automated?
Could it be assisted?
Could it be simplified?
Could an AI agent do the first pass?
Could a workflow gather the information?
Could a system remind me?
Could AI prepare it so I only need to review it?
Those questions can lead to some pretty interesting possibilities.
My AI Team Has KPIs Too
Here’s the funny part.
I’ve spent years telling project teams that we need measurable objectives and KPIs.
Now I’m doing the same thing with my AI agents.
Every agent needs a job.
Every job needs an expected outcome.
Every workflow needs a measure of success.
And every workflow should be reviewed periodically.
In other words…
I’m managing my AI team like a project.
Which probably shouldn’t surprise anyone.
I am a project manager, after all.
The Ultimate KPI
At the end of the day, I don’t really care how many AI agents I have.
I don’t care how complicated my workflows are.
I don’t care how impressive the technology looks.
My ultimate KPI is much simpler:
Am I spending more time doing the work that matters?
If the answer is yes…
The AI team is doing its job.
If the answer is no…
Back to the drawing board.
20% Is Just the Beginning
Maybe I hit 20%.
Maybe I eventually hit 25%.
Maybe I discover that some workflows save far more time than I expected.
Maybe I discover others aren’t worth maintaining.
That’s part of the experiment.
Because this isn’t about building a perfect AI system.
It’s about continuously improving how I work.
Build.
Measure.
Learn.
Improve.
Then build the next one.
One agent.
One workflow.
One time-saver at a time.
And if I can reclaim even 20% of my time…
That’s not 20% less work.
It’s potentially 20% more opportunity to do the work only I can do.
And that’s the real ROI of my AI Project Team.
