What we want from autonomous agents is to act on their own. To decide what is the best next course of action and do it. Pretty simple on a paper as we as humans do it all the time.
If you are working in a company there is variety of complexity of decisions, right? There is a decision where out of four teams a customer team should be rooted to be serviced the best and there is a decision about outsourcing your customer team offshore.
I want to talk about the latter ones.
I want to make argument and make you realise, how LITTLE information about decisions we make in business we actually record. And it’s not that LLM companies don’t have access to it. It’s that it doesn’t exist or it doesn’t reflect the reality you would expect agents to understand.
We want agents to make their decision based on some past training and on fly information they can retrieve. For an AI model to be trained on it, it has to exist right? It needs to be written down somewhere.
There is a few arguments that can be made.
1) LLMs already contain all the public internet learning, so it contains within it good representation and model to make business decisions.
2) Some of the LLMs are/will be trained on large amount companies of data and these companies (e.g. big 4 consulting) have the needed information.
What does a working ML model we want to make decisions look like?
Now, let’s talk about a decisioning machine learning model that works in our lives now. Insurance companies are using models to predict how much discount at renewal they should offer you for you to not leave. Let’s talk about how it works on a high level.
- Insurance companies have data about products you have, how long you’ve been with them, what segment you are, how often you change products etc.
- Then they run experiments, either manually created or more often based on the model recommendation.
- They record if you have renewed or not and then if you have cancelled it in free to cancel period.
- They continually tweak the model to find the minimum discount they have to give you not to lose you.
What it means they have some useful description of who you are and what you do. They don’t know that you wake up at 3am every morning worrying about getting fired from work. But they know your income, how many kids you have, how wealthy you are, where you live and how often you tend to switch providers.
Then they build a model around binary decision. It might effectively say “what discount do we need to offer for 85% chance this person stays with us?”. The outcome is renewal or not. True or false. One or zero.
I want you to think for a minute about what amount of data they have to predict a renewal outcome. They have the quite extensive data about you, they run experiments and have data from those and they know if you taken the action or not. The whole package. For a simple “Give them 10% discount and see what happens”.
Now we know how good models work in production. Let’s get back to more complicated decisions.
Let’s say you want to choose between bringing some CRM system in. If you were to write down all the important data points that goes into that decision what would they be? Would they be different if you would be considering Vibe-coding it yourself, buying SaaS or getting 3rd party to fully manage it?
Is there 40 data points that you could objectively weight them for autonomous agent to try to decide which is the best option?
How much does it cost? How much does a change cost? How many changes do we need to make in the future? Do we have the skill in-house? Do we want to have the skill in-house? Do we have a time to build the skill in-house? Is it a blocker for scaling now? What about security and data privacy? What about integration with other systems? What about customisation? Do you want it or now? How much does functionality does it provide now and how much do you buy into the company?
All of them are fairly sensible right? Could possibly be written down somewhere?
Now let’s go into the factors ones we don’t tend to write down.
No one will blame me for choosing the big vendor. I really want to use it, so I can put it on my CV. If we hire big 4 consulting and they fail, I can blame them without getting fired. I own shares in the company. I can’t stand our IT, so I want to outsource it, because then I can control it. If I will create a big project, I can get myself VP role. I already spend a lot of money on projects that weren’t successful, I need to go with the cheapest approach otherwise it will raise red flags. My manager said so, but I don’t understand why.
Not that we don’t write down the ones that are easy to write down, we have a huge amount of factors that drive a decision that we don’t anyone ever to know.
Are you going to tell that to an AI Agent? So it can log it and learn from it?
What we also don’t have is the expectation of what should happen and what shouldn’t happen after the decision is executed. I.e. How do we decide if the decision was good? And we don’t have it for very human reason. If it was good decision, no one cares and there is much more work to be done. If it was poor decision, the person who made it doesn’t want to bring it up and get it evaluated.
“No one was fired for hiring IBM.” is such a common saying capturing the challenge.
So we don’t document the factors, we don’t write down the expectations and we don’t document the outcome and success.
And if you wonder that you have seen a lot of documentation about vendor selections and projects and decisions.
What we tend to have is justifications of decisions. The decision was made and let’s document why it was the right decision. That’s majority of the business management slides and consulting slides, software architecture documentation etc. I’m not saying that it’s wrong, but it is a retrospective explanation after the decision is made and before execution happens.
If we want to move further to genuinely autonomous AI agents making more complex decisions, we have to get much better at transparency and documenting and even than I am making argument that the world of higher level decisions is so complex that it’s very hard to try to capture how it was made.
To address the the two arguments from earlier, which I myself smartly asked.
I will start with the second one that the data exists in private estates. What was trying to explain is that we don’t have it, we have justification of a decision. 90% of data project fails, is there forensic analysis for those? Perhaps one or two slides as “lessons learned” after half an hour meeting, but not enough for autonomous agents. The consulting companies have very different incentive and factors than companies paying them, even if they would have it, it would largely be the wrong ones for autonomous agents to consider.
And if we think about that it learned from the public internet. I would want to you think about where from? LinkedIn content has no connection to reality as it is PR, marketing and sales platform, so if the LLM would learn from there, how will it know how to make any business decisions except for what content is popular and gets traction on social media? Great training data set for that, because that’s what it is.
Somehow the LLMs are good at giving confidently answer regardless if it’s wrong or right, but then explaining why it was right. Bombs you with arguments and sometimes made up stats to convince you. Feels like it was trained on texts that justify decisions. Although, it is just my feeling and we will have to wait for some proper research.
Question for all companies as they expect autonomous agents to take over more work; what decisions do you actually want them to do and is it set up to make it?
Just above average 