Custom AI products are arriving at full speed. We've all been sold the promises of cost reduction and efficiency gains and honestly, what's not to love about that potential upside, right?

Maybe you work at a company thats been building your own custom ai workflows. Maybe you've opted instead to pay a vendor to build them for you. Some of those products are probably even working exactly as you'd hoped, which is great!

But I have a question for you, how is that return on your investment looking so far? Have you reached those cost reduction numbers you were hoping for? Are your token costs climbing as adoption grows?

If your entire product was built using a single model, there's a good chance that answer is yes. Lucky for you, there's an equally good chance that you can benefit from implementing a decision model into your product stack.

Wait, what is a decision model? Is that something new? Oh wait, yeah I remember hearing about something called JEV i think. I saw a few posts about that. Umm...

Keeping up with all of this can be overwhelming. Hang in there and take a deep breath. We're going to get through this together.

A New Category Arrives

Let's start with decision models. What are they and where did they come from? For most of us, JEV 1.13 was our introduction to decision models. It was launched by TypeSafe AI on September 15th, 2026 and the model's arrival kicked off a flurry of decision model launches to follow: Cloudflare Clef, AWS Strands, and OpenAI Decisions to name a few.

The underpinning technology is easy enough to understand and I'll keep the explanation simple. Traditional language models answer questions with paragraphs of text. JEV takes a different approach. It doesn't use paragraphs of text to answer decision questions, it just makes decisions and has a probability attached to it. For example: (Yes - 85% confidence). It also accomplishes this feat in a fraction of a second, for a fraction of the cost of a traditional LLM. Based on our testing, which we'll dive into shortly, $1 was good for about 50 thousand decisions.

So why does any of this matter? Well, think about all of the potential decision points that your current AI solution has. JEV has the potential to make routing decisions faster, and cheaper. These routing decisions also introduce an opportunity to split the work up and distribute tasks to smaller cost effective models. This is important because not every task should be handled by the biggest and best models.

Okay, this is sounding pretty great. Should I implement a decision model into my product stack? Do you have any numbers I can take a look at to help me make my decision?

Lucky for you, we've doe exactly that and yes, we do have numbers to share. But first, lets set the parameters for our experiment.

The Experiment

If there’s anything that I’ve learned along this AI journey, it’s that you can’t automatically take marketing material at face value. I’ve been burned by adding a new AI model to my app after reading featured release notes and taking them at face value as truth. The model didn’t work as advertised and I had long days of painful rework added to my backlog as a result. So before we take a formal stance on decision models, we decided to test the technology on our own proprietary app, Authored By. Will the promised efficiency gains hold up under real testing without compromising quality or performance? Here’s the problem statement we’re testing against:

Let me share a little bit more about our test subject. Authored By is a self authoring AI story building consumer app. The concept is simple enough to grasp. Users talk out loud to the app and the testimonials that they give are transformed into readable chapters. Over time, those chapters accumulate into stories that you can share with your friends and family.

With Authored By, the app has 3 key moments that we hypothesized could greatly benefit from a decision models:

  1. Is this shared moment something that should be added to a chapter?
  • (Referenced as “Enough to write” in chart below)
  1. Do we have enough moments recorded to generate a chapter?
  • (Referenced as “Ready to write” in chart below)
  1. Does the story still have a good flow with the newly generated content?
  • (Referenced as “Story flow” in chart below)

These three moments are all currently handled by Claude 5.5 Sonnet. Every decision generates paragraphs of text in order to do its evaluation. Something that could be hypothetically improved by JEV.

We added JEV to our codebase and ran simulations of each of the 3 scenarios above 30 times.

Here are the results:

As it turns out, JEV did a pretty great job at slashing our Claude token cost for 2 out of the 3 scenarios. For “Enough to write” we dropped our Claude token cost from $2.58 down to $1.08 per thousand jobs. JEV allowed us to stop the unnecessary content generation previously involved in the decision making process. This might feel like minimal savings but if you are working at an enterprise supporting an app that has hundreds of thousands of calls, those savings will start to feel like real numbers. By our calculation, $1 of JEV tokens buys you about 50k decisions. Each of these decisions saved about $0.0015 versus our existing Claude only setup. By that math, $1 of JEV tokens pays off by saving us roughly $75 dollars in Claude tokens by eliminating unnecessary content generation.

“Ready to write” saw nearly identical 59% efficiency gains.

“Story flow” on the other hand saw a much smaller 15% gain, mainly due to the way our logic is engineered. In order to process the decision of whether the story maintained a good flow, the new content needed to be generated and evaluated anyways.

It’s also worth pointing out that the accuracy of the decisions being made were not as high with JEV as they were with Claude. Claude passed every decision evaluation while JEV made the right evaluation 90% of the time.

So what does it all mean?

For us, the biggest win of the experiment wasn’t the amount of tokens saved by implementing a decision model. Our app doesn’t currently have the level of traffic that would warrant huge cost savings.

Implementing the decision model did however allow us to simplify our existing orchestration layer. The clearly defined decision points allowed us to clean up our logic and define clear handoffs to individualized agents that were designed to do specific jobs. No more handing every single task to Claude Sonnet 5.5. The individual agents can use the right model for the job giving us the range, flexibility, and efficiency our app needed. It also helps us to future proof our logic. If a more efficient model emerges that one of our agents could benefit from adopting, we can quickly swap out that agent’s logic and andjust our orchestration layer.

Reach out anytime

If this blog resonated with you or you and your organization are struggling with any of the content we covered today, free to reach out @ letstalk@smallmachines.ai. We have a tried and true design and delivery process that we live by and would love to build the path to great work, together.