Why 90% of AI Projects Fail in Production — and the 6-Question Checklist We Now Run Before Every Rollout
This post started as a 90-second LinkedIn note. People kept asking for the actual checklist. So I'm putting it here, permanently, for anyone shipping AI into the real world.
— Saurabh Agarwal, Founder & CEO, Proeffico Solutions
The first time we deployed our AI, we were sure it would work.
We had tested it for months. Clean light. Controlled angles. A quiet meeting room. The model was beautiful.
Then we went to a polymer plant in Haryana.
Dust. Glare. Shift changes. Smoke that looked like steam. The model that worked perfectly the night before — failed in front of the client.
I remember standing in that factory, watching my team rebuild a confidence I thought we already had. That day I learned the lesson nobody tells you when you start an AI company.
The AI is the easy part.
90% of AI projects fail — not because the tech is wrong
They fail because of six things nobody puts on the website. We learned each of them the hard way. So now we run a pre-flight before any rollout goes live at Proeffico — six honest questions a project has to pass.
Here it is. Use it. Copy it. Add to it. Better still — fail at one of them at a low-stakes pilot rather than at a production site.
1. The Floor Test
Have we run this model in conditions identical to the customer's actual site?
Not "similar." Identical. Dust, glare, ambient noise, shift-change lighting, the wettest day, the busiest hour. If we've only tested in the office, we haven't tested.
- ✅ Green light: 7+ days of pilot footage or data captured from the real customer site, not a lab.
- 🔴 Red flag: "It works in our environment."
The lesson: a polymer plant in Haryana is not a meeting room in Noida.
2. The Domain Hours
How many hours has the team spent on the customer's floor — before we wrote a single line of training code?
Engineering is fast. Domain is slow. The reverse never works.
The fire detection model that misses electrical-room glare. The retail anti-theft model that doesn't know what a cashier's busiest hour looks like. Both built by people who never spent a shift on the floor.
- ✅ Green light: Minimum 40 hours of shadow-and-listen on-site, distributed across shifts.
- 🔴 Red flag: "We have a domain expert on the call."
The lesson: you can't model what you've never felt.
3. The Dataset Variation Audit
Does the dataset cover the edges — not just the average?
There is no AI. There is only the dataset you fed it.
Day shift only? Add night. One camera angle? Add ten. Clean light? Add dirty. Sunny day? Add rain.
The edge cases ARE the product. Average performance won't survive a Tuesday.
- ✅ Green light: Variation matrix documented — shifts × lighting × weather × angle × occlusion — with samples per cell.
- 🔴 Red flag: "We have 10,000 images." (Of what?)
The lesson: quantity is comfort. Variation is survival.
4. The Build vs Integrate Decision
For every major component — did we explicitly decide to integrate or to build? And why?
Founders love to build. That's how runway dies.
Detection models, dashboards, alert pipelines, OCR, storage, video transcode — most of it already exists. Use what's good. Build only what becomes the moat.
- ✅ Green light: A two-column doc — "Build / Integrate" — with a one-line reason per row.
- 🔴 Red flag: "We built it from scratch because…"
The lesson: your time is the most expensive line item in the project.
5. The Hardware Variance Map
Have we listed every deployment-site hardware variance — and tested for each?
Cameras. GPUs. Edge boxes. Network bandwidth. Power consistency. Storage path.
The model is portable. The deployment isn't.
Site A works. Site B has older cameras. Site C has no GPU. Site D's bandwidth dies at 6 PM.
- ✅ Green light: Hardware spec sheet per site, with the pipeline tested against each combination before go-live.
- 🔴 Red flag: "It worked at the pilot site."
The lesson: the silent killer doesn't show up at the demo. It shows up at the second deployment.
6. The Evolution Protocol
Who owns the rewrite — and what's the cadence to revisit the pipeline after go-live?
The first version is the hypothesis. The real product is built in the 90 days after rollout.
Reality keeps surfacing situations you've never seen. The team that owns "v2" before "v1" ships is the team that survives.
- ✅ Green light: Named owner. Weekly post-deployment review for the first 90 days. Documented rewrite triggers.
- 🔴 Red flag: "We'll deal with it if issues come up."
The lesson: treat the rollout as the second build, not the launch. Every successful AI project we've shipped was rewritten at least once after go-live.
Score yourself
Count the green lights on a project before you take it live.
| Score | What it means |
|---|---|
| 6 / 6 | Ship it. |
| 5 / 6 | Ship it — and fix the one before the second site. |
| 4 / 6 | You're shipping a prototype, not a product. Be honest with the customer. |
| 3 or less | Don't go live. The factory floor will not be kind. |
A note on what's NOT on this list
You'll notice this checklist has zero questions about:
- Which model architecture you used
- Which cloud provider you picked
- Whether you're using "agentic AI"
That's deliberate.
None of those determine whether the project survives the customer's real environment. The six questions above do.
The customer doesn't want AI. They want the problem gone.
I spent two years selling AI before I learned to stop selling.
The customer doesn't want AI. They want the problem gone. They want to sleep through the night.
The model is just our tool. The deployed, evolving, surviving, adopted solution — that is the product.
If you're in the middle of an AI project right now that looked amazing in the demo and is breaking in production — you are not failing. You are exactly where the journey actually begins.
I have been there. Standing in a factory I drove three hours to reach, watching the model freeze, wondering if I had picked the wrong life.
I hadn't. Neither have you.
Where this came from
At Proeffico we build VIZO361.AI — production AI video analytics on existing CCTV — and ship custom AI engineering across India and the GCC for industrial, retail, and BFSI buyers. Every line in this checklist came from a real site, a real failure, and a real team rebuilding confidence at 11 PM in someone else's factory.
If you're shipping AI into a tough environment and want a second pair of eyes on your deployment plan, talk to us. We'll run this checklist on your project, honestly, in a single 30-minute call. No pitch.
— Saurabh Agarwal, Founder & CEO, Proeffico Solutions
VIZO361 · ZIVUX · MaximPro · Engineering Intelligent Business
