The problem you're living with

Tell me if you've heard this one.

Your team has a problem. You've got an app thats not working quite hitting the mark. Maybe its because you have a gap in your product offering that nobody has addressed. Maybe you are trying to reach a new customer that could open up an additional revenue stream but nobody's sure how to get in front of them.

Then somebody on the team comes up with an idea. Thanks to modern AI coding tools, that idea evolves into a demo, and the demo is actually really cool. It felt like magic to see the idea come to life, and look how fast you got to this point. Riding the wave, you showed it to a boss and that boss said the four words that set the whole initiative into motion: "this is really cool."

So next, the team doubled down on the idea. They gave it a cool name, were rewarded with a bigger budget, and a put together a roadmap. They got more people onto the project. They thought of more cool features. They added functionality. They started dreaming bigger...

Then time passed. And then more time passed. Over time, the excitement started to fade and doubt started to creep in. Questions started creeping in like:

Is this actually helping anyone?

Does this project still make sense?

The most uncomfortable part about these questions is that you cant really tell, because nobody decided up front how you would be able to tell. The only evidence we had up front was that somebody put together a great demo, and a leader liked it.

Why this happens

Let's be honest about why. The team skipped the product work. There were steps that should have been taken between the idea and the roadmap phases. Lets start with:

Understand the user and do the research to prove it.

Don't rely on assumptions or the loudest voice in the room. You need to run real research. Talk to real people, get real feedback, and find the earned secrets that only come from that diligent effort. Try running interviews, sending out surveys, observe, and listen. The more work you put into finding the truth, the more you will trust the answers that start to emerge.

Ask them questions about how they work today. What hurts? What have they already tried? These pain points are low hanging fruit that can shape your product into something they love.

Define success, and capture a quality baseline.

Now that you've put the work in to understand the customer, you've likely established some baseline metrics around the established pain points. AI can help accelerate this for you to help you group and categorize trends that may be hidden in your collected feedback. Once you have an established baseline, you can set a target for the incremental improvements you'd like to make. This will help set the vision for your product and influence what features you should build first.

Decide how you'll know whether it's working.

Now that you know what to build, and what you need to measure, you can set real improvement targets measured against those earned secrets we mentioned above. If you knew what the customer pain points were up front and have datapoints that you can measure against, you have a great opportunity to go back to those same customers with your proposed solution and take new measurements with your new features. The best products build the feedback loops right into their products so the product teams can iterate and refine their product on the fly.

How you can fix it

The three steps above lay a pretty solid foundation. There are some additional steps I'd recommend you take on top of that you can use evidence to drive decisions.

Decide the verdict in advance.

We touched on this above in point number 3 but lets take it a step further. Before you launch, agree with your team on what results would mean the project should continue, change direction, or stop. Writing this down ahead of time takes the emotion out of it. If you agree up front what the decision thresholds are, its much easier to follow later when the data starts to roll in.

Test with real behavior, not opinions.

As you are iterating, put the simplest version in front of your test users that you established your baselines with. Build only what is needed to demonstrate the added value and measure the impact it has on the user. This can be done via in person observations, in app usage analytics, surveys, or any other feedback loop concept you have.

Watch for things like, would they come back to use this feature again? Do they trust that it works? Do they feel their pain points being alleviated? Remember that what people do is far more telling than what they say in a survey. For example, asking someone "would you use this?" could get you a "yeah, I think so" whereas giving somebody that product to react to might result in another learned product secret.

Present the evidence to leadership, not just the sizzle.

When the time is right, bring the demo and the results to your boss. Tell a story with the journey that you've been on. Lead with the established pain points and baseline metrics. Help that leader feel the pain that the customer is feeling and back it up with the data you worked hard to collect up front.

From there you can introduce your product that addressed exactly those pain points and have a fast follow with real numbers collected during your testing that validate you're actually onto something.

This turns your previous feedback of "this is really cool" into "this is great work". It's also what gets projects funded with confidence. Leaders don't feel like they are taking risks when you share the story of the product journey you've been on and the data you now have to back it up.

A real life scenario

I was working on a significant project at one of the world's premier manufacturing companies. Leadership gave me and my team a clear objective: Turn a documented process that was printed on foldable pamphlets as a pocket reference and make it something digital that folks could read on their phones. The request was so straight forward. We could have built exactly that and everybody would have said "cool, thanks".

To get ourselves familiarized with the process, we started talking to folks that use the pocket reference on a daily basis. Those conversations led us down the path of an eventual global research even that garnered 700 responses from every region, every skill team, and had a balanced mix of experience levels.

What we learned from that research even changed the entire project. Turns out, people were pretty cool with the pocket reference and if we were to create a digital version, they'd take it but.. they didn't really want it and didn't feel like they were missing it.

What we did learn however was that one layer deeper, people were struggling to find the information they needed to do their daily jobs. The pocket reference highlighted procedural steps that teams were supposed to follow, but teams didn't know how to execute those individual procedures. Their solution was to ask someone else who had done the job before with no way to verify that the other person followed proper protocol to begin with.

Finding the right information at the right time emerged and the clear problem statement for us and we rallied around the question:

"How might we make the path to great work impossible to miss?"

From that point on, every experiment that we ran had a purpose. We iteratively implemented features that addressed documented pain points identified during our research event and every single feature aligned to the common theme of making the path to great work impossible to miss by giving users the right information at exactly the right time. If we knew where they were on their journey, and we knew what they were supposed to do at that step, then we could serve that information up to them in a timely manner. Leveraging AI we could also give them contextually aware answers. If we knew who they were, what program they were part of, and what role they were in, we could give them answers that related specifically to them.

The best part about this approach was that because we were tying every feature to a known problem identified during our research event, we had a baseline metric that we could measure against as we continued rolling out new features. We could tell our story with data and take the emotional attachment out of it. Eventually we took this project to pilot and when we met with teams that were completely reluctant to taking on another tool, our one minute demo quickly changed their tone to "I need this, when can I have it?".

What comes next?

At Small Machines AI, I help teams pressure-test an AI idea before they pour months into building the wrong thing. Our design sprints are structured to set you on a path to success just like this article laid out. We're currently taking on new engagements and would love to help you find the path to great work.

Reach out to us anytime on any of our social channels or send an email to letstalk@smallmachines.ai