"Mistral 7B outperforms Llama 2 13B on all benchmarks and matches Llama 1 34B on reasoning tasks, while being small enough to run on consumer hardware." — Mistral AI Blog, October 2023
One of the most common questions we get after delivering an AI system is: "How do we know if this is working?"
It's a question that should be asked before the project starts. But better late than never. And the answer is more structured than most people expect.
ROI for AI systems is real and measurable. But you need the right framework, because AI productivity gains are distributed, often indirect, and sometimes show up in places you weren't looking.
The Four Categories of AI ROI
When we work with clients on measuring AI value, we organise it into four categories:
1. Time Savings (Direct Labour Hours)
The most straightforward measurement: how many hours per week did this process take before, and how many does it take now?
Example: A professional services firm we worked with was spending ~40 hours/week across three employees manually processing and summarising client research reports. Their AI summarisation system reduced this to ~8 hours/week. That's 32 hours saved.Valuation: 32 hours × $85/hour average loaded cost = $2,720/week = ~$140,000/year.
System cost (infrastructure + ongoing maintenance): ~$24,000/year.
Year 1 ROI: roughly 5.8x. Year 2 and beyond (without rebuild costs): 7-8x.
How to measure: Time-tracking before and after. Be honest — the "after" time should include AI supervision time, not just assume the AI does it all.2. Quality and Error Reduction
Sometimes AI doesn't save time so much as it reduces mistakes. This is harder to put a number on but often more valuable.
Example: A client was manually entering data from supplier invoices into their accounting system. Error rate before AI: ~4% of entries had some kind of mistake, each requiring 20-30 minutes to find and fix. After implementing an AI extraction and validation layer: error rate dropped to <0.5%.Valuation: If they process 500 invoices/month and 4% have errors, that's 20 errors × 25 minutes = 8 hours of error correction/month. At 0.5%, it's 2.5 errors × 25 minutes = 1 hour. The saved correction time is 7 hours/month = ~$400/month in direct savings. But the bigger value is in downstream quality — cleaner books, fewer audit issues, less time at year-end.
How to measure: Track error rates before and after. Sample review of outputs. Audit trail.3. Throughput and Capacity
This is the category that's hardest to capture in a simple number but often represents the largest actual value: AI allows you to handle more volume without proportional headcount growth.
Example: A client's support team was handling ~200 tickets/day with 8 agents. After implementing an AI first-response and triage system that resolves ~35% of tickets automatically and pre-drafts responses for the other 65%, the same team handles 280 tickets/day with the same 8 agents. That's 40% more throughput — equivalent to adding 3 agents — without the hiring, training, and salary costs.Value of 3 additional agents at $60k loaded cost: $180k/year in avoided hiring. System cost: $30k/year. Net savings: $150k/year, not counting the improved response times and customer satisfaction.
How to measure: Tickets/volume handled per agent before and after. Customer satisfaction scores. Response times.4. Revenue Impact
Occasionally, AI creates direct revenue opportunities rather than cost savings. These are real but harder to attribute cleanly.
Example: A client's sales team was spending 40% of their time on administrative tasks — writing follow-up emails, updating CRM records, preparing meeting summaries. After AI automation of those tasks, selling time increased. In the 6 months after implementation, quota attainment improved from 78% to 89% across the team.The attribution isn't clean — other factors affect quota attainment. But the direction of causality is clear enough that it belongs in the ROI calculation with a confidence discount.
How to measure: Sales metrics before and after. Revenue per salesperson over time. Conversion rate changes. Apply a confidence discount (maybe count 50% of the improvement as AI-attributable, to be conservative).The Measurement Methodology
For every AI project, we recommend:
Baseline capture (before launch):- Identify the 2-3 primary metrics for each ROI category
- Measure current baseline values
- Document the measurement methodology so you can repeat it consistently
- Monthly in the first 3 months
- Quarterly after that
- Annual comprehensive review
Keep a running log of qualitative feedback and unexpected uses. Sometimes the highest-value use case wasn't the one you planned for.
What to Watch Out For
The "saved time gets absorbed" problem. If an employee saves 3 hours/week from AI automation, does the company actually capture that? Or does the time just fill with other low-value work? This is a management and process design problem, not an AI problem, but it affects the real ROI. Plan for how the saved time gets redeployed. The ramp-up period. In the first 1-3 months, the system is being refined, users are building habits, and the AI is often going through quality improvements. Don't measure ROI in month 1. Measure at month 3 when you have a stable baseline. Infrastructure costs are usually small. For most SMB AI systems, the ongoing infrastructure cost is $200-1,000/month. This is rarely the dominant cost factor. The dominant cost is either the build (one-time) or the people time around the system. Intangibles are real. Employee satisfaction, reduced cognitive load, faster response times, improved data quality — these matter even when they're hard to quantify. Don't dismiss them.The Bottom Line
The AI projects we've seen achieve the clearest ROI share a pattern: high-frequency processes (things that happen daily or weekly, not monthly), processes with measurable time or quality components, and processes where AI output quality is genuinely high enough to be trusted.
The projects that struggle to demonstrate ROI are usually low-frequency, one-off, or were built without clear success metrics from the start.
Measure it. The numbers are usually better than you expect.
Want to implement AI in your business? Book a 30-minute architecture call at blok-ai.com.