Last week, Salesforce CEO and co-founder Marc Benioff made his opinion known: Copilot AIs are out and AI agents are in. Benioff is quoted as calling Copilot AIs “nasty” and likening them to “Clippy” the much-despised Microsoft Word assistant of the late 1990s.
I’m someone who personally experienced how useless Clippy was back in the day, I literally laughed out loud at the comment. But as someone who is immersed in Enterprise-Grade GenAI Contract Management, I had to wholeheartedly agree that Marc was making a really important point about how the industry needs to evolve to provide real value to businesses.
Benioff’s reasons for this negative opinion include hearing from Salesforce customers that the currently available copilot AI models, many of which were developed by OpenAI, aren’t delivering value. AI copilots are plagued by inaccurate data, hallucinations, and an all around failure to perform tasks that are usable and useful for business.
It’s worth noting at this point that AI is cycling through the same stages most new and disruptive technologies do. From a fad that promises to transform life as we know it, to an over-hyped and underperforming cliche, to finally landing in a nuanced place where we begin to understand what its capabilities and limits really are and what it’ll take to be successful with the technology. In other words, it’s gone from “AI can do everything!” to “AI is useless” to “Here’s what AI can really do and here’s the work you have to put in to get those results.”
While reality has largely fallen short of our initial hype from a couple of years ago of what large language models (LLMs) would be able to do, it’s still true that what we’ve got right now is something more powerful in terms of machine intelligence and our ability to communicate with that intelligence through natural language than ever before, and by many orders of magnitude.
But it’s important to acknowledge that the sci-fi idea that ‘out of the box’ LLMs and generative AI are something you can simply throw information into, ask a nuanced and important business question, and automatically get the right answer isn’t where we’re at today, and may not be for many years, if ever.
Luckily, Benioff went on to clearly address the way to solve this problem, not in some distant future, but today. He’s quoted as saying:
“Many customers are reporting the OpenAI models are not delivering high levels of accuracy and resolving even basic customer service issues for them,” Benioff said. “The lack of grounding, the lack of access to the metadata, the data itself, the sharing model, all of the components of a platform that are then needed to be able to achieve this kind of level of accuracy.”
What Benioff is specifically talking about is the importance of data grounding in the enterprise. A critical technique to create guardrails and boundaries for the AI to work within. When done correctly, generative AI can and will provide responses and perform analyses that are both accurate and useful. When done incorrectly, it’ll answer whatever questions you ask but those answers will be useless at best and nonsensical at worst.
What is data grounding?
To answer this question, it’s useful to first understand what is not data grounding. ChatGPT was trained on a vast percentage of the internet, originally up through mid-2021. Newer versions have been trained up through September 2023 and can even use Bing to search the web for more recent sources. While this sounds great in theory – imagine a brain that contains every bit of information ever published online – the reality is it’s a recipe for disaster if you’re looking for accurate outputs.
With no grounding in data that is known to be useful or specific to a business, and no boundaries on what it’s using for inputs, ChatGPT will often make up nonsensical answers to questions. And because it has the ability to provide incorrect answers in a format that seems very trustworthy, it can be difficult for a user to know the difference between an accurate answer and an AI hallucination. Imagine how little value a model like this provides to an enterprise organization that’s hoping to use generative AI to quickly perform complex analyses on its vast collection of data.
Data grounding, on the other hand, is the process of connecting AI to real-world data sources, thus limiting its inputs to clean, organized, accurate enterprise data with the right guardrails in place to ensure accuracy. When your AI is grounded in enterprise data, you’re not able to ask it anything and everything. You are, however, able to ask specific questions with answers buried in vast quantities of your organization’s data and expect accurate and valuable answers.
Clean, organized data is vital to data grounding
Using generative AI doesn’t relieve an organization from getting its house in order. On the contrary, to use AI effectively, an organization absolutely must start with clean and organized data. Otherwise, you’re trapped in the garbage-in-garbage-out loop where AI makes for a good product demo but has no hope of achieving ROI.
At Pramata, we’ve learned there is a very specific way of doing this data grounding that will lead to huge wins for our customers. Especially for those who want to use AI to access the data from their contracts to perform sophisticated business analyses.
We’ve created a Contract AI Knowledge Engine that’s powered by a company’s cleansed and organized contracts so that when they begin to use AI, it’s grounded in their own data and nothing else. Only with this foundation can an enterprise use LLMs and AI to get insights and make strategic business decisions.
How grounded data allows AI to turn contracts into insights
All of this may sound theoretical, so it’s helpful to offer some real world examples. At Pramata, we’ve developed a system for cleansing and organizing enterprise contracts into a format that’s easy for generative AI to access and work with, without going off the rails and creating its own “facts.”
So, what does that “data grounding” look like in contracts? Here’s are some questions to ask yourself:
- Do you have all of your contracts in one place, minus the garbage? If not, you run the risk of hallucinations due to duplicate document records, irrelevant files or missing contracts, amendments, and exhibits.
- Do you have the contracts organized by counterparty and grouped into hierarchies based on the order of precedence? If not, AI may make up those connections or wrongly infer what the current, prevailing terms are for any analysis.
- Do you have clauses captured, categorized and stored in a standard way? If not, you will quickly overflow context windows when asking questions.
- Are you able to isolate precise contracts, and sections of contracts when asking questions? If not, you are prone to slow, inaccurate responses.
Without this grounding of questions in the context of specific contract data, you run the risk of investing in a very expensive “Clippy” for your contracts. But, with this grounding in place, you have the potential to transform how you manage your agreements and unlock value well outside of the traditional “contracts = legal” equation:
- Automatically generate playbooks for contract negotiation based on what’s been agreed to in the past
- Identifying potential opportunities to consolidate vendors based on who has the best terms and pricing for a given set of goods and services.
- Find revenue leakage in contracts and automate price escalators, purchase commitments and more.
- Compare invoices with contracts to identify billing errors
- Identify white space and upsell/cross-sell opportunities by comparing customer contracts and analyzing which products customers don’t have that they should be purchasing from you
This list could go on and on, but the point is that when you work with a partner like Pramata that’s built solutions specifically to work with AI in this way, you truly can achieve the promise of generative AI in a way that the copilot model has failed to do. This is because of data grounding and the rules we write for AI to follow within the Pramata platform.
As Marc Benioff has moved away from the copilot model and stated that the future of AI is grounded in enterprise-grade data and metadata that will make it valuable to businesses and hallucination-free, we wholeheartedly agree!
At Pramata, we’re in the business of helping teams across every company from Legal to Sales, Procurement to Finance, get value from their contract data. We know that how you organize that data to get solutions with generative AI is critical.
Ironically, many of our biggest customers use Pramata to ensure that clean contract data flow into their CRMs.
So, Marc, if you’re listening, we should talk!
Because clean data is key to your strategy and your mission here, and a lot of the largest companies in the world, even those using Salesforce, haven’t yet cracked the code of getting clean contract data into their CRM. As we can all agree, it doesn’t matter how intelligent your AI is, if it’s not grounded in clean, accurate, organized data with clear rules and boundaries, it might as well be “Clippy 2024.”
