Agentic chat is hot right now and for good reason. It's a technology thats revolutionizing the way we interact with our customers. In addition to those interactions, we're also unlocking unprecedented insights and getting next level with the metrics dashboards we are able to create.

In this post, we're going to get up to speed on agentic chat solutions and help you decide if this is a solution that your business should be investing in. I'll walk you through a comprehensive real world example that i hope inspires the way you think about this technology and simplifies the whole solution into something you can easily digest and speak to with confidence.

So let's dive in.

The first question you probably have is: What is agentic chat anyways?

It's a chatbot, right?

Yes, let's start there!

It's like a chatbot that can do stuff, right?

Yes! See, you know more than you give yourself credit for.

It's like, a lot of chatbots put together, isn't it?

Um, not exactly. But thats ok.

You're well on your way and in 5 minutes you'll understand this enough to explain it to your coworkers.

The best way to think about an agentic chat solution is to imagine it in layers. There are 4 important layers that I want you to understand: the user experience layer, the orchestration and decision layer, the agent layer, and the data layer.

The easiest way to understand this is to build out a simple but real world example you can relate to. So let's pretend we're a typical corporate enterprise that has a sales team that needs to stay on top of their invoices. We want to build a chat solution that can help employees ask questions and get answers as it pertains to those invoices.

The User Experience Layer

We'll start at the top with the UX layer. This is the only layer that the customer sees. It's the actual screen you interact with. You'll enter your messages, maybe upload an attachment, or maybe you'll leave a voice memo. You'll hit send and the system will do something and give you a response. You can customize this experience to suit your customer needs but regardless, this is our UX layer. Let's say our user asks the question:

"Whats going on with that purchase order?"

So what actually happens when I ask my agentic chat solution a question? I'm glad you asked! Interpreting that question is exactly what happens in the orchestration and decisioning layer. At this level, our app needs to interpret the request from the user and determine what to do with it. Depending on what the user wants to do, the orchestration and decisioning layer is going to hand off the job to be completed to a specialized agent. Our agentic chat solution can have any number of specialized agents. We'll get to that next.

In order to call these agents, We need to make sure we are collecting the right information from the user so that the agents can do their jobs correctly. Above, our user asked the question "Whats going on with that purchase order?". At this layer, our app will determine which agent has the ability to look into purchase orders. It has alot of agents at it's disposal and has custom logic written to determine when and why an agent should be called. It will also understand exactly what information it needs to collect from the user before it can confidently send the agent out to do it's job. With the question asked above, our solution will recognize that it needs more information. Let's send a message back to our user that helps us get that missing info:

"I'll look into that for you but first, I need you to share the purchase order number".

The user shares the additional detail and the orchestration and decisioning layer kicks off the right agent behind the scenes.

Agent Layer:

At this level, we may have anywhere from a few, to an army of agents. Every one of them will have a separate but clearly defined job to do.

But what are agents exactly?

The best way to think of agents are to think of them as specialized workers that are designed to do a specific repeatable task really well. Our agentic chat solution capabilities are going to stem from an army of specialized agents that might have many capabilities and those need to add capabilities to our chat solution, we can add new agents with the desired capabilities and introduce new logic into our orchestration and decisioning layer so it understands when to call this new agent.

From our example above, our purchase order question is going to rely on a specialized purchase order agent. This agent has the ability to interact with our data layer and pull all of the relevant information order the user provided details on. When provided with the right criteria, we can confidently expect this agent to execute it's task.

I want to take a second to touch on all of the tasks an agent can actually do. With our invoicing system, maybe we want to process payments, place new orders, cancel existing orders, run credit scores, apply for loans, or look up internal process documents. Agents can be created to do all of these tasks. The important thing to remember about agents is that they should have singular goals and purpose.

An agent that does one thing really well is better than an agent that does many things inconsistently.

The more logic you introduce to a single agent, the more opportunity you introduce for mistakes. Take a look at my previous post on clean code principles if you want to dive deeper into this subject.

Data Layer:

We're down to the final layer. Many of the hypothetical agents we discussed above will need to interact with various data systems. We may have a purchase order database, an inventory database, a product detail database, we might have sharepoint drives, we might even have external data sources that we are pulling in via third party APIs. Quite frankly it doesn't really matter.

What matters is that each agent understands how it will connect to the data that it needs and take action on that data. I'm going to keep this section short and sweet. Our agent finds the particular invoice record from our invoice table and sends it back up to the orchestration and decisioning layer to be handed off to the UX layer for the user to read. We've now completed a single pass through our agentic chat solution.

Getting Next Level with Analytics:

One of the best things about agentic chat solutions is opening the door to analytics dashboards and insights that we havent seen before. Think of it this way, traditional analytics dashboards have alot of statistics and trends derived from record counts. Alot of bar charts, pie charts, and line charts. These are great! But wouldn't it be nice to have insights into "why?" trends may be emerging? Owning your own agentic chat solution means you have the ability to dive into those conversations pair contextual awareness to the trends. Maybe you'll realize that new hires are asking the same types of questions and as a result we need to create better onboarding documentation. Maybe you'll realize that one particular onboarding document gets alot of the same types of questions and there is an overwhelming level of confusion as it relates to a particular process flow.

Agentic chat records allow us surface those insights. We can tell the content manager that paragraph 4 of the onboarding document is getting alot of questions from our new hires and we might need to revise our process flow diagram. We can see that alot of folks that have worked here for 6 months still dont really understand the inventory system and the training video we use isn't landing the way we thought.

You can even design your agentic chat solution in a way that allows you to gather the insights you are after. You can engineer a solution that processes inquries on the fly and populates your insights dashboard the moment they are surfaced.

At Small Machines AI, we specialze in creating agentic chat solutions and we'd be happy to show you some of the solutions we've built including: we would love to do the same for you. Upon request,

  • learning comprehension agents that greade and teach users against known learning objectives
  • agentic advisory boards that leverage agentic personas to provide professional feedback
  • a daily testimonial solution that transforms continued entries into structured and formalized stories
  • a job search tool that leverages conversation to understand exactly what a user is looking for in their next career, finds the open position, and drafts your outreach email
  • enterprise retrieval and action systems
  • human in the loop agentic chat support systems

Good luck to you as you venture out onto your agentic journey. We'd love to hear about the great things that you create and are happy to help you get there if you are looking to partner.