He shares a case study from a large enterprise AI team working on a knowledge and search system, where the core problem was guessing user context rather than designing a clear way for users to set it.
By reframing the problem around user needs, the team could ship quickly and avoid wasting time, money, and trust. The core message is that stalled AI projects usually suffer from clarity issues, not technical difficulty, and outside perspective can help teams see that sooner.
Let’s jump into the discussion:
Most AI projects stall because teams lose clarity on the user problem. Complexity creeps in and progress slows. This pattern repeats. Projects stall, confidence fades, and decisions get deferred.
Where does progress break down? When does AI complexity replace clarity? And when do teams stop learning from users?
How to know when the wheels are coming off your AI project:
You can’t demo a real system on real data in 2 weeks.
Not polished. Not production-ready. Just working.
If you can’t do that, something’s broken. And it’s probably not technical.
At CES last week, where I presented my interactive training on prompt engineering, Presidential Science Advisor Michael Kratsios put it bluntly: the U.S. is investing hundreds of billions into AI infrastructure to enable companies to create real ROI. But massive infrastructure investment isn’t enough. You have to apply it to real ROI. That’s the key to winning the AI game.
He’s right. But here’s what I’m seeing on the ground:
I ran a 1-day Snowball Sprint workshop in Japan recently. We went from idea → framing → real RAG registry with voice and substitution → working agentic Python application.
One day.
Participants were beyond excited. Then I asked: “Are you going to do this at your company?”
The room got quiet.
“The nail that sticks out gets hammered down.”
They’d need 6 months of internal approvals to run a 1-day POC.
The government is investing billions to enable ROI. But most organizations can’t give themselves permission to capture it.
The article above is about a team that spent 9 months and $2M+ without anything demoable. The problem wasn’t AI complexity—it was solving the wrong problem at the wrong layer.
The wheels come off AI projects when organizations can’t move fast enough to learn.
“They’d need 6 months of internal approvals to run a 1-day POC.”
This is true. Recently went through 14 months to align on engagement. So, there seems to be an appetite at the top, and a willingness with teams (however tentative)… it seems there’s an operational problem of understanding how to support an initiative. Would you agree @greg_nudelman
I almost fully agree with the article @greg_nudelman , HOWEVER, there’s a gray line that I think appears where there are times when complex technical pieces that just take time do exist.
An example is when we were implementing report filtering, the SQL was just so complex that it took a few weeks to get right.
Perhaps somewhat of a red flag, but something that was completely necessary to get a reasonable feature working. (The whole thing took over 6 months)
Do you think that there’s some leeway with the red flag rule, where sometimes things do just take a long time to implement?
Yes @Bryan — I think most orgs cannot move at the speed required by AI-first processes, which is why the entire balance of the industry is slowly pivoting toward fast iterative AI-first investments, as I’ve seen at CES. Companies that are moving fast against real ROI targets are building up incredible capabilities and will be winners in the AI-driven universe. I am glad to do my small part in providing hands-on team training through delivering real projects – the only way to learn, IMHO. SnowballSprint.com
@ben could you provide more info? My first book was on Search and SQL is my bread and butter. I’m not saying you need the whole thing working – just a thin slice, a small but complete use case – with a fraction of the data.
This is interesting, especially for people here who are trying to connect their work to business impact. How have you seen product teams align around this objective in practice?
With the implementation, there was a core file that dealt with all of the filtering logic. There wasn’t any good way to get partial filtering without making everything else unusable to a user.
It simply just took several months to get to a good spot working on top of legacy code and architecture.
@greg_nudelman love this thinking. I’m curious how you begin to identify ‘impossible problems’ within an AI context? I imagine general familiarity is helpful (though not for the engineers in your case study example), but are there other indicators for spotting these traps in advance?
Over 16 years in the industry I have developed the Snowball Sprint framework, which I wrote up in my latest book, UX for AI. Here’s the overview: https://www.snowballsprint.com/how-it-works One of the essential evaluation exercises is Digital Twin. Here’s a free Digital Twin chapter from my book, UX for AI (#1 New Release on Amazon): Digital Twin Book Chapter | UX for AI
Yes, that sounds like a nightmare. However, I do want to assert that for most problems there are ways to accelerate. For example, did you want to keep this file-based architecture? Why not move to a modern architecture like Elastic? Or SQL? If the tables are in decent shape you can usually get something working in a few weeks. Maybe slice off some operators and build a parallel filter harness? I’m sorry I don’t have particulars. The standard approach is to build a view to abstract the legacy implementation and use the API to spin up something clean. Again sounds like you’ve solved this already, but if you have a budget to do something more, I’d love to see if I can help: Calendly
