Skip to main content
Back
Volume 02
Orchestrating AI Across the Enterprise
banner image

Escaping Pilot Purgatory: Why 95% of AI Initiatives Never Scale

Aniket Maindarkar CMO, Firstsource

Anthropic recently ran a fascinating experiment called “Project Vend,” where they let Claude manage an automated store in their San Francisco office for about a month1. The results revealed something profound about where we are in the AI journey. Claude excelled at discrete, well-defined tasks: researching suppliers, finding specialty products like Dutch chocolate milk, and handling routine operations. But when faced with the strategic coherence required to actually run a profitable business over time, it failed dramatically, sometimes even attempting to “contact the FBI” when daily fees continued after it decided to “close” the business2.

This experiment perfectly captures the current state of AI adoption. We’re witnessing remarkable success in tactical applications, but struggling to translate that into strategic business transformation.

And the data confirms this struggle: research from MIT’s NANDA Initiative found that 95% of enterprise AI pilots fail to progress beyond early stages to scaled adoption, delivering little to no measurable impact on profitability3.

The question isn’t whether AI creates value; it clearly does. The question is: what type of value, and why, are most organizations unable to scale beyond proof-of-concept?

The AI Value Hump: Why 95% of Initiatives Get Stuck

AI value creation follows a predictable maturation curve. Understanding this curve explains why some organizations capture exponentially more value than others, while most never escape the pilot phase.

Stage 1: Process Optimization

Organizations use AI to do existing things faster, cheaper, and more accurately. This includes call summarization, intelligent ticket routing, claims processing automation, quality monitoring, automated content generation, and workflow automation.

The Reality: These applications deliver immediate ROI and are relatively easy to implement.

The Trap: As AI tools become more accessible, optimization advantages erode. Everyone gets email automation, meeting summaries, and basic chatbots. What once was a competitive advantage, becomes table stakes within 12-18 months. Each incremental efficiency gain yields smaller benefits, and competitors can replicate your optimizations quickly.

Most organizations are clustered here, competing on the same use cases, while extraordinary value creation happens elsewhere.

Stage 2: Decision Enhancement

AI augments human decision-making in complex scenarios where judgment matters. Examples include:

  • Predictive churn models guiding retention strategies
  • Fraud detection systems learning from transaction patterns
  • Real-time next-best-action recommendations during customer interactions
  • AI-assisted strategic planning with human oversight
  • Predictive analytics for business decisions

The Challenge: This requires deeper integration with business processes, domain expertise, and organizational change capability. The technical barriers are higher, but more importantly, the execution barriers are significant.

Stage 3: Capability Creation

AI enables entirely new business capabilities that were previously impossible. Examples include:

  • Outcome-based pricing models tied to business metrics (PMPM cost management, retention rates, cash flow improvement) rather than labor hours
  • Autonomous adjudication for defined claim types
  • Specialized LLM platforms trained on specific industry verticals
  • Predictive issue prevention that eliminates problems before they occur

The Opportunity: These applications don’t just improve existing processes; they reimagine them entirely, creating new sources of competitive advantage with high barriers to entry.

The Four Killers of AI: Why Most Initiatives Never Launch

Before organizations even reach the challenge of scaling from pilot to production, many AI initiatives die in infancy:

1. C-Suite Misalignment: The Expectation Gap

Each executive expects fundamentally different outcomes from AI, creating hidden friction that dooms initiatives before they begin:

When the CFO builds business cases around headcount reduction, the CMO plans for capacity expansion, and the CEO expects growth, AI pilots that succeed technically still fail organizationally. Without explicit alignment on which outcomes matter and when, initiatives collapse under competing definitions of success.

2. Misalignment with IT Organization

When business units pursue AI initiatives without IT involvement or worse, in opposition to IT, failure is virtually guaranteed. IT controls data access, infrastructure, security protocols, and integration capabilities. Without their partnership, AI tools remain isolated and create security vulnerabilities.

3. Access to Data

AI is only as good as the data it can access. Most organizations have data locked in silos, poorly documented, or of insufficient quality. Organizations spend 60-80% of their AI project time on data preparation—cleaning, labeling, integrating, and securing data.

4. Limited Enablement Leading to Poor Adoption

Building an AI tool is the easy part. Getting people to actually use it is where most initiatives fail. Organizations launch AI tools with fanfare, provide minimal training, then wonder why adoption remains below 20% six months later.

5. Lack of Clarity on Purpose and Task Allocation

“We’re using AI” isn’t a strategy, it’s a buzzword. Organizations that can’t articulate specifically what AI should do, which tasks it will handle, and how work will be redistributed are destined for confusion and disappointment.

These five killers claim more AI initiatives than technical challenges ever will.

The Work Transformation Truth: Work Doesn’t Disappear—It Changes

Here’s what most organizations misunderstand about AI: Work doesn’t go away. In fact, work often increases. But the kind of work fundamentally changes.

When AI automates routine tasks, it doesn’t create idle time; it reallocates resources to what’s important. Customer service teams that implement AI chatbots don’t shrink; they shift from answering basic questions to handling complex issues that require human judgment. Finance teams that automate reporting don’t disappear; they spend more time on analysis and strategic planning.

This reallocation is where the real value lives, but only if organizations plan for it explicitly. Companies that implement AI expecting simple cost savings through headcount reduction miss the opportunity. Those that implement AI to free up capacity for higher-value work capture exponential returns.

The organizations succeeding with AI aren’t asking “How many people can we eliminate?” They’re asking, “What higher-value work can our people do when freed from routine tasks?” This shift in mindset separates the 5% from the 95%.

The Execution Gap: Why Most AI Pilots Never Reach Production

Here’s the uncomfortable truth: Promising AI use cases are abundant and relatively easy to identify. Every organization can imagine using AI for predictive analytics, personalized customer experiences, or outcome-based pricing. Most have multiple pilot projects showing promise.

What differentiates leaders from the rest is the ability to take that idea and operationalize it at scale.

The organizations capturing disproportionate value share two critical capabilities:

1. Deep Domain Expertise

Transformation applications require an intimate understanding of specific industry contexts. A specialized LLM for mortgage processing isn’t just faster document processing; it’s a fundamentally different capability that combines years of industry expertise with AI. This creates barriers that competitors can’t easily replicate.

2. Organizational Agility

The ability to change and pivot quickly is equally critical. The AI landscape evolves rapidly. Customer needs shift. Competitive dynamics change. The organizations that thrive can:

  • Move from pilot to production in weeks, not years
  • Iterate based on real-world feedback without bureaucratic gridlock
  • Reallocate resources rapidly when opportunities emerge
  • Kill failed experiments quickly and learn from them

This combination of deep expertise and rapid execution is rare. Most organizations have one or the other, but not both. Those that develop both capabilities climb past Stage 1 while competitors remain stuck.

Why Automation Actually Increases Demand for Human Expertise

Here's where the story gets counterintuitive. As AI handles more routine work, demand for premium, high-touch human services is actually increasing. The white glove delivery market exemplifies this trend, growing at 9.9% to 12.7% annually and projected to reach $20-66 billion by 20304 5.

This pattern extends far beyond delivery services:

  • Healthcare payers use AI for claims auto-adjudication, yet invest more in clinical appeals specialists for complex cases
  • Customer care centers automate routine troubleshooting while building specialized retention teams that combine deep product knowledge with relationship skills
  • Financial services automate basic transactions while expanding wealth advisory teams

Automation doesn't eliminate human value; it reshapes where human value is highest. Organizations that understand this can capture premium pricing for Stage 2 and 3 applications that combine AI capability with human expertise.

The Three-Horizon Framework: Operating Across Multiple Timelines

Rather than viewing AI value as a sequential journey, successful organizations operate across three horizons simultaneously:

Horizon 1: Operational Excellence (0-18 months)

Use AI to optimize existing processes and improve operational efficiency. This builds the foundation and funds more ambitious initiatives.

Examples: Automated call summarization, intelligent case routing, chatbot deflection, intelligent document processing, and AI-powered quality monitoring.

Strategic Purpose: Generate quick wins and ROI that justify investment in Horizons 2 and 3.

Horizon 2: Competitive Advantage (6-36 months)

Deploy AI to enhance decision-making and create differentiated capabilities. This requires deeper integration and the agility to pivot as you learn.

Examples: Predictive models identifying at-risk customers, AI-assisted complex case resolution, personalized customer experiences, dynamic workforce management.

Strategic Purpose: Build defensible advantages that competitors can't easily replicate.

Horizon 3: Market Creation (18+ months)

Leverage AI to enable entirely new business models and market categories. This demands significant organizational change and innovation capability.

Examples: Outcome-based pricing models, autonomous adjudication, specialized industry LLM platforms.

Strategic Purpose: Create new categories where you can capture premium value and establish leadership before competition emerges.

Why Transformation Applications Create Sustainable Value

Stage 2 and 3 applications offer different economic characteristics than optimization:

Higher Barriers to Entry: They require deep domain expertise, organizational change capability, and significant integration work. The execution challenge alone eliminates most competitors.

Client Lock-in: When AI capabilities become integral to client workflows, switching costs increase dramatically.

Premium Pricing: Clients pay for outcomes and new capabilities, not just efficiency improvements.

A Framework for Escaping the 95%: Practical Steps

The path forward isn’t about choosing between optimization and transformation—it’s about building the execution muscle to scale beyond pilots:

1. Audit Your Current Position

Map existing AI investments across the three horizons. Most organizations discover that the vast majority of their investment is clustered in Horizon 1, limiting long-term competitive advantage.

Action: Conduct a brutal inventory. How many pilots are stuck? Why? What execution capabilities are missing?

2. Build Dual Capabilities Simultaneously

Develop both deep domain expertise and organizational agility:

  • Domain Expertise: Invest in teams that combine AI expertise with vertical knowledge
  • Agility: Create lightweight governance, rapid experimentation processes, and clear kill/scale criteria

3. Rebalance Investment Allocation

Shift resources toward Horizon 2 and 3 applications. Use Horizon 1 success to fund more ambitious initiatives rather than pursuing more optimization projects.

Target Allocation: 60% Horizon 1, 30% Horizon 2, 10% Horizon 3 (adjust based on organizational maturity).

4. Focus on Business Outcomes, Not Technology

Measure success through business impact—client retention, revenue growth, outcome achievement—rather than technical metrics like accuracy or speed.

Key Question: “What business result changed?” not “What did the AI do?”

5. Build AI-Human Orchestration Capabilities

The highest value comes from combining AI’s computational power with human judgment, creativity, and relationship skills. Design systems that amplify human capabilities rather than replacing them.

6. Create Fast Feedback Loops

The ability to iterate rapidly separates leaders from laggards:

  • Run small-scale pilots in Stage 2 and 3 applications
  • Fail fast and learn from real-world feedback
  • Scale what works within quarters, not years
  • Kill what doesn’t work within weeks, not months

Conclusion: The AI Value Frontier

In 1995, Jeff Bezos left his Wall Street job to sell books online. His former colleagues thought he was crazy; everyone knew people wanted to touch books before buying them.
But Bezos wasn’t trying to build a better bookstore. He was building something that didn’t exist: a store with infinite shelf space, personalized recommendations, and next-day delivery anywhere.

Today’s AI leaders aren’t trying to build better chatbots or faster analytics. They’re building capabilities that don’t exist yet, businesses that learn, adapt, and create value in ways we’re just beginning to imagine.

The difference isn’t in their ideas. It’s in their ability to execute at scale while others remain stuck in pilot purgatory.

The AI revolution is entering its second phase. The first phase focused on proving AI could deliver value through process optimization. The second phase is about discovering where AI can create entirely new forms of value through transformation, and building the organizational muscle to actually deliver it.

This shift doesn’t diminish the importance of optimization; it remains essential. But sustainable competitive advantage increasingly comes from Stages 2 and 3, where AI enables human potential rather than replacing it, and where execution capability matters more than clever ideas.

The organizations that understand this evolution will escape the 95%. They’ll use AI not just to do things better, but to do things that were previously impossible. They’ll build businesses that combine the speed and scale of artificial intelligence with the creativity and judgment of human intelligence.

The question isn’t whether your organization will use AI. It’s whether you’ll build the execution capability to scale beyond pilots and create sustainable value.

The race isn’t about who can automate fastest; it’s about who can transform most effectively while others remain stuck at Stage 1.

References

  1. Anthropic. (2025). “Project Vend: Can Claude run a small shop? (And why does that matter?)” Retrieved from https:// www.anthropic.com/research/project-vend-1
  2. Andon Labs. (2025). “Vending-Bench: Testing long-term coherence in agents.” Retrieved from https://andonlabs. com/evals/vending-bench
  3. MIT NANDA Initiative. (2025). “The GenAI Divide: State of AI in Business 2025.” Retrieved from https://fortune. com/2025/08/18/mit-report-95-percent-generative-ai-pilots- at-companies-failing-cfo/
  4. Fortune Business Insights. (2023). “U.S. White Glove Services Market Size, Share | Forecast [2030].” Retrieved from https://www.fortunebusinessinsights.com/u-s-white- glove-services-market-108962
  5. Business Research Insights. (2024). “White Glove Services Market Size, Share, Analysis by 2033.” Retrieved from https://www.businessresearchinsights.com/market-reports/ white-glove-services-market-105477

Get In Touch