Read the Webinar Transcript Below:
Introduction and Speaker Backgrounds
James (Contract Management Solutions Expert, Pramata): Hello everyone. Welcome to today’s webinar with Pramata. We’re excited to host this webinar and share practical tips on how general counsels can think about implementation of contract AI.
Before we begin, quick introductions. My name is James. I’m a Contract Management Solutions Expert with Pramata. Before Pramata, I was an engineer by education, but I dove into academic technology transfer where I managed technology portfolios, negotiated agreements, and managed relationships with inventors and life science industry folks. From there, I went to work for a CLM provider, working closely with legal teams to address frustrations that lawyers and attorneys face in their daily contract work. Now I’m with Pramata, trying to bridge the gap between customer pain points and solving their real-world challenges using Pramata.
AC Agarwal (VP of Product, Pramata): Hi everyone. My name is AC Agarwal. I am the VP of Product at Pramata, based in the San Francisco Bay area where we’re getting an unusually cool summer this year. My background: I’ve been in legal tech for over 15 years with deep experience working directly with customers to implement solutions. Currently I’m managing the overall roadmap of Pramata’s solutions and products. Over the past decade I’ve been involved in using AI in some form, and especially the last couple years, we’ve been immersed in the deep end of using the latest generative AI technologies and embedding that in every piece of the platform component we have in a purpose-built way to help solve real-world use cases.
The AI Mandate for Legal Teams
James: There’s a common mandate we’re seeing across industries, especially for legal teams: they’re being told to, “Use AI”. That sounds broad, but it has significant implications. What does that really mean in practice?
Legal teams, especially during my years negotiating contracts, thought contract AI was this esoteric thing that was out of reach—something we couldn’t grasp, especially as practicing lawyers. By nature of being very meticulous, scrupulous, and detail-oriented, we often get afraid of how we would use AI within our daily practice, let alone how to use it within an organization to share information when we’re supposed to be the gatekeepers of that information in terms of contracts.
Given this mandate, what we’re going to cover is the best way to tackle that from a practical standpoint. If you just try to think about using contract AI without a strategic approach or framework, it seems very daunting. But with the advancement of different LLMs and technologies, and legal teams starting to embrace that they really need to start using contract AI.
I think folks today will take away some practical steps and, more importantly, be empowered to go back to their organization and strategize with business stakeholders to lay out a roadmap for what it would look like for their business to use AI.
Audience Poll Results
James: We conducted a short poll asking how often you currently use AI tools for things like contract review, legal research, or other legal tasks. The results were all over the place. A group of folks have never used any AI tools for contract review or anything else within their daily work, while others use it a few times a month, some weekly, and forward-thinking super users use it daily. This gives us an idea that we have folks on different paths along the AI journey from thinking about it all the way to using it on a daily basis.
Measuring AI ROI: Case Study from Axios
James: This survey from Axios shows how legal teams measure AI ROI. This is especially relevant for folks thinking about evaluating contract AI vendors because it’s a very large business decision. When legal is given a budget or general counsels are thinking about investing in a new tool, it’s not cheap. There’s an investment cost—both monetary and business process changes.
These metrics show ways you can measure potential ROI when executives ask how you’re going to measure it. We’re not trying to buy an AI tool just for the sake of buying a contract AI tool. These are things you can bring up to your executive, C-suite, CEO, or CTO to explain how legal teams who have invested and are in later stages of maturity in their AI journey are measuring AI ROI.
Today’s presentation focuses on two key metrics:
- Measuring increased contract throughput (60% of respondents) – handling larger volumes of contracts
- Measuring quality improvements over current processes (55% of respondents)
“We’re dealing in a world where legal teams are being pretty lean and it’s very costly to be onboarding lawyers on a constant basis. By using contract AI, many teams are seeing success in their ability to handle an increased load of contracts with the same resources.”
James: It’s not just about speed and volume. As attorneys, one of the most important things is that we make sure we hit the nail on the head the first time. When dealing with contract negotiations, you want to ensure that if you’re implementing contract AI, you want it to be accurate every time, reliable so that you don’t have to second-guess yourself. If you have to second-guess the data that AI is producing, you’d have to go back and reevaluate it, which creates more work—the opposite of what we want.
Implementation Framework: Priority-Based Approach
James: When determining how to implement contract AI, you have to break it down into bite-sized pieces, focus on things you want to prioritize, and work your way up. If you try to focus on implementing AI into every single process all at once, it gets too overwhelming.
We’ve seen success with our customers breaking things down into different priorities and buckets using this pyramid approach:
Priority 1 (Foundation): High Volume, Lower Risk Transactions
- NDAs – High volume but lower risk by nature
- Many customers find success starting with NDA workflows
Priority 2: Ad Hoc Legal Research Requests
- “Can you find me a contract?”
- “Can you find this MSA?”
- Questions about specific terms in agreements
- Audit-related requests
Priority 3 & 4: Strategic and Complex Work
- Contract drafting and negotiation (bread and butter of legal teams)
- Strategic projects that make long-term business impact
“You really want to be focusing and prioritizing your time on complex relationships and negotiations. As a GC, you don’t want to be stuck in the weeds of handling NDAs or ad hoc research requests coming from all over the organization.”
James: Each organization will have their own path and different priorities that may not map exactly 100% to this paradigm, but the key takeaway is figuring out what low-hanging fruit you could apply AI to. This formula seems to be working well, but in your organization, there’s probably a set of things you could apply AI to. Getting that quick win helps you gain confidence, especially since 25% of the audience hasn’t used contract AI yet. It’s important to get that first win because it helps gain confidence and buy-in.
Use Case 1: High Volume Transactions – NDAs
James: Let’s go over high volume transactions. In this case, NDAs—I know everyone loves NDAs. That’s where we all started in negotiating agreements, but it’s still a very pertinent use case because many legal teams struggle with sheer volume. It’s not necessarily that NDAs are the most difficult agreements in the world, but the issue is that it compounds. When dealing with high volumes of NDAs and other simple agreements, it bogs down your legal team and takes focus away from more complex negotiations. This is where AI comes in to make the whole process efficient, giving legal teams the ability to focus and prioritize more complex work.
The Complete NDA Process
James: NDAs might seem pretty simple in terms of language and negotiation, but if we look at the whole process, things get more complicated:
- Requesters submit NDAs to the legal team
- Legal team triages – contracts admin assigns NDA to appropriate legal rep
- Search for existing NDAs across multiple repositories (Google Drive, Slack, email, hard drives)
- Review the NDA – compare third-party NDA against playbook, look for conflicting terms
- Communication with stakeholders and other parties for redlining
“Even with today’s digitization of PDFs, searching for an NDA or agreement sometimes takes a lot of time. Back in my day working on contracts 10 years ago, I was dealing with paper copies—I’d have to go to the storage room and look for an NDA.”
James: While NDAs themselves might seem simple, if you look at the entire process, that’s where folks see hurdles and where volume creeps in. It’s not that legal takes 20 days to negotiate an NDA, but all this time compounds. There might be waiting for responses, internal stakeholders on vacation, and various business interruptions. Even with a very lean process for NDAs, we typically see total turnaround time averaging two to three days. Compound that across hundreds of thousands of NDAs your legal team processes—that’s a lot.
Ad-Hoc Approach vs. Integrated Contract AI Approach
James: There are two ways to think about using contract AI:
- Ad hoc approach – using out-of-the-box LLMs for specific tasks
- Holistic integrated approach – encapsulating an entire process
Ad-Hoc Example: Using ChatGPT for NDA Review
James: One approach, if you’ve never used contract AI tools before, is taking an ad hoc approach. You can take an out-of-the-box LLM like ChatGPT or Claude, feed it contracts, and ask it to compare a third-party paper versus your template.
In this example, we provided a third-party NDA for review, our template NDA, and asked ChatGPT to compare the two and tell us the differences.
The results show a clause-by-clause comparison. If you’ve never used this before, it’s quite fascinating how quick this is—we’re talking about a couple of seconds, maybe a minute if your internet’s slower. ChatGPT was able to return different comparisons, going through different clauses, showing what’s in the third-party paper versus our template, and showing key differences. That’s pretty good for an out-of-the-box LLM that shaves off time if you were to review it manually.
Limitations of the Ad-Hoc Approach
James: While ChatGPT is good at very specific tasks, there are limitations. We asked ChatGPT as a follow-up: “Do we have an existing NDA with this other party in the first place?” This harks back to that process diagram—first you’d want to search your repositories for an existing NDA with the customer. If you do, you don’t need to go through the review process and can return that fully executed NDA.
Unsurprisingly, ChatGPT cannot identify this—it says you need to check your contracts database or ask your legal/procurement team. The limitation is that while ChatGPT is good at specific tasks, it’s not connected to a contracts repository, your drive folder, Slack channels, or wherever you’re storing contracts.
“The point is that even if you sped up the review process, what happens if you find that you have an NDA with the other party? You just wasted your time doing that review, even though it was AI.”
James: Even if AI can speed up certain tasks and components, you have to address all the other steps and think about this more holistically. That’s why we encourage folks on the AI journey: while you can apply AI today in ad hoc approaches, we recommend taking a more holistic approach and implementing AI throughout your entire process. That’s where you’ll see benefits compounded and eventually get to self-service, giving your legal team time to focus on complex relationships and negotiations.
Problems with Ad-Hoc Approach at Scale
James: With the ad hoc approach, you can do things like NDA review and analysis, and AI will return pretty decent results—enough to realize there’s benefit in time savings and accuracy. But the problem is at a macro scale: Are you really going to manually feed ChatGPT hundreds of NDAs against your company template?
That starts to become a problem because at scale, we see issues. With larger, enterprise customers, you have hundreds of thousands of these high-volume transactions in any given month. When trying to do this at scale, the ad hoc approach breaks down.
Integrated Contract AI Approach Benefits
James: With an integrated approach using contract AI across the entire process, you start seeing benefits like:
- Sales teams can generate their own NDAs – Sales AEs in Salesforce generating hundreds of cases to legal teams asking for NDAs can now generate their own NDA using AI for drafting
- Dramatic reduction in legal workload – Instead of dealing with 100% of incoming requests, legal teams now only deal with 20% that involve further negotiation and redlining—the complicated ones that legal should invest time and energy in to
- No more random Slack messages – Self-service mechanism eliminates midnight requests like “Do we have an NDA?” especially important for global companies with time zone differences
- Focus on complex matters – Legal teams can focus on things that require negotiation and drive business value
Integrated Contract AI Approach Demo
AC: You’re probably wondering what this integrated approach looks like and if there’s a lot of work to get this in place. I’m going to address both of those things. Whether you use Pramata or a different provider, the main thing we want to highlight is seeing the art of the possible with contract AI and how you can get to that integrated process.
Streamlined Request Process
AC: First, you need a mechanism to have a steady flow of requests coming in. The question over the wall, Slack message, or urgent email is really disruptive. You need a steady way of getting requests.
When somebody makes a request for “I need an NDA for RGS Corporation,” immediately AI should check: do you already have an NDA in place? If there is, it tells you immediately and gives you a link directly to that contract. It seems simple or minor, but it’s one less request coming into legal for processing. It’s three days faster to return a response to the other party and maybe three days faster to close the deal. These are minor points, but they accumulate in terms of benefits.
Not only do we want this intake process in one place, we want to make sure it’s easy to access from where people live—whether that’s Outlook or inside a Salesforce opportunity.
Self-Service NDA Generation
AC: [Demonstrates self-service NDA generation in Pramata Contract AI Platform]
What we should be able to do is have the user input information into the system that gives them everything they need to influence the standard template without changing any clauses—only the parts of the standard NDA that need to change. They get full visibility into the request status and can download the auto-generated NDA in minutes. We have an NDA being developed where only things that need to be changed, like the party name and effective date, are modified.
“What could have taken three days to just draft something that’s on our standard template is now not only accelerated, but it’s actually self-serviceable by the rest of the business.”
Third-Party NDA Processing
AC: It’s not always as simple as “We already have an NDA” or “standard NDA.” Often it’s a scenario where it’s a third-party NDA. When somebody uploads a document into the request system, rather than waiting until it gets assigned to somebody in legal and reviewed over a couple days, there should be an automated way to perform a check that AI is doing: “Can this NDA be signed as is or not?”
That check shouldn’t be checking against a standard checklist—it should be checking against YOUR checklist. What are the things you care about in your organization? That should give a heads up to whoever’s making the request that there are a few things that pass your checklist and a few things that don’t. It should provide guidance to the other side—maybe a sales rep—saying if the other party can agree to these few things, then we can sign this NDA immediately, even though it’s not on our paper.
“That is deflecting things away from legal. It is empowering other business stakeholders and giving guidance on how to deal with other parties. Rather than addressing 100% of incoming NDA requests, maybe you’re only addressing 50% now or 30% or 20%”
Contract Review with AI Assistance
AC: In cases where the NDA doesn’t meet certain criteria, it goes to the general counsel or contract administrator view. They can open the document directly inside Microsoft Word and run a detailed check showing what met criteria and what didn’t.
They can use AI assistance embedded directly inside Microsoft Word to see that governing law didn’t meet criteria (State of Florida vs. preference for Delaware) and use AI capabilities to speed up the redlining process.
Transparency and Implementation Speed
AC: The vendor you’re thinking about should have transparency into the contract AI. It should explain why the AI is doing the analysis and giving red flags, providing more explanation than just displaying the red flag. That transparency of what the AI is doing and thinking is critical, even if you have embedded your organization’s policies.
“The way we design our products here at Pramata is that a key principle is people should be able to get going and running on all of this within days, not weeks, not months, but within days.”
AC: We’ve seen this play out with our customers who have adopted the technology. AI is at an unprecedented level. If you can use ChatGPT to get a quick summary within seconds, you should be able to do this integrated approach in a fairly easy way. It’s not the old way where you have to do a heavy IT project to get things off the ground.
Use Case 2: Ad Hoc Legal Research Requests
James: Next we’ll go through ad hoc legal requests and how you can implement contract AI to solve challenges with these requests. Legal teams are bogged down with requests and questions coming from the organization that take attention and focus away from complicated matters. Things like:
- “What are our data breach terms?”
- “Where’s an MSA for a customer?”
- “When does this customer renew?”
While folks might think of these questions as not important—just answer them as they come—what we’ve seen is that this is actually a huge time sink for legal teams that they probably don’t realize.
When thinking about ad hoc questions, it’s not just about using AI to increase contract throughput. You want to provide self-service mechanisms at a higher level where business units can answer questions about contracts themselves.
The “Cake” Analogy for Request Complexity
James: We think about ad hoc legal research requests in this cake analogy—different levels of complexity with each request:
- Base Layer (Simple): Searching for an MSA – pretty straightforward, find the contract and give it back
- Second Layer (Moderate): Data breach obligations – find the contract AND extract the data breach clause
- Top Layers (Complex): Multi-layered requests – “Give us this information AND provide a recommendation” which involves analysis
As you get higher in stacking that cake, requests become more complex and require more resources.
Ad Hoc Approach Example: Renewal Date Extraction
James: [Shows ChatGPT extraction of renewal dates from MSA]
We uploaded an MSA to ChatGPT and asked it to analyze and give us the extracted term date and renewal date. ChatGPT is pretty good—it identified the effective date and expiration date, and we could see the end date seemed accurate.
But what if I told you this wasn’t the whole story? What if this MSA was signed a decade ago, but since then, there have been multiple amendments? That’s where the issue becomes with ChatGPT—the answers are only going to be as accurate as all the context you provided. If we provided it with the MSA and every single related agreement that might have affected the term, ChatGPT would have given us a more accurate result.
Because it doesn’t have all that contextual information about the customer relationship, we see issues where if you have complex relationships with your customer base and many related agreements, are you really going to feed ChatGPT not only the MSA but all the future amendments? Who’s going to keep track of that? That becomes laborious and a problem at scale.
AI Hallucination Example
James: Even ChatGPT can be wrong. We provided it with amendments this time to see if it would give us the accurate renewal date. For context, we ran this two days ago on July 22nd. It says the renewal date is June 13th, but that’s already passed. If you’re a contracts manager looking at this result, you’re thinking, “Wait, did this expire?”
One thing you might realize is that sometimes AI hallucinates. If you don’t have appropriate guardrails in place and provide it with all the data, have all the prompting in place to get accurate results every time, you might see AI hallucinate. If you can’t trust the data, are you really going to trust giving access to all other business units to self-service?
“This is where it becomes a problem where ad hoc might work if you just wanted to do extractions for one agreement, but for these complex relationships that we deal with on a daily basis, it becomes impractical at scale.”
The Pramata Integrated Approach for Ad Hoc Requests
AC: The approach we’ve taken when designing our product to eliminate all these ad hoc requests is to produce the most pristine data around contracts and organize and prep them to be ready for contract AI so that whenever questions come in, it’s easy to answer.
It’s the same adage: garbage in, garbage out. I recently read that you can have the world’s best prompt, but better context or better input information will always beat the world’s best prompt in producing more reliable AI output.
Data Organization and Preparation
AC: You’ve got all these contracts scattered in maybe one, two, or three different drives. Sometimes they’re labeled correctly, sometimes they’ve got the wrong thing in the right folder.
What we emphasize is that you need to:
- Get rid of all duplicates
- Remove things that aren’t actually contracts
- Organize them into the right accounts
Contracts are complicated. They’re not just the one contract in the four corners of the document—there are different relationships, entities they’re related to. How do you organize all those into a clean repository so that at any given time, if somebody says “Help me find a contract,” you don’t have four different folders for the same entity?
You should be able to go into that folder and understand the full relationship of that contract history because it’s never usually a question about just one contract. Usually the question from finance, sales, or other departments is about the entire relationship history. For that, you need to know not just what happened in the MSA, but what did those amendments do? What did the transactional documents like order forms do? Understanding that whole relationship is critical.
“This is an essential part. If you want to offload all these ad hoc requests and get reliable results from contract AI, in our experience, you really have to have that solid context.”
AI-Powered Self-Service with Rich Context
AC: Once you have that solid context, you can start doing things like grabbing the right data from documents and tracking history. Renewal dates are never easy—they start off easy in the first document, but then relationships change, they get amended, something changed where they extended the term to two years. Keeping track of that history and ensuring you have a steady state of all newly signed contracts coming into this view is really important.
Once you do that, AI seems easy. Once you do that hard work, then you can start to do interesting things.
[Demonstrates data breach notification obligations query]
With rich context going into AI, it’s not just looking at this case and telling you what the answer is, but it’s analyzing: “Here’s what the original MSA said, but it was modified by the second amendment.” It has links to the actual thing, so you can verify the contract AI output.
“That’s the beauty of this rich context—it helps you feel the confidence to then open up the system, open up access to other people in the organization and let them self-serve in a secure manner.”
AC: From this place, you can do other things we saw: find a contract, answer data breach or any other term questions. Then you can look at this holistically across your entire set of contracts and do analytics. Where do I have upcoming renewals? Maybe with vendors, I don’t want them to auto-renew, and I can quickly run a report and export this to provide to my business partners.
The key thing is that if you want to eliminate all these interruptions that happen throughout the day and focus on strategic things, having that pristine repository of clean data constantly being updated is critical.
Advanced Contract AI Capabilities
James: In these couple of screenshots (see webinar), you can see what our customers have been able to do, focusing on priority three and four at a strategic level. They’ve been able to implement and customize our contract AI capabilities for their business needs. This includes building custom dashboards and reporting for the C-suite around tariffs and stack ranking vendors based on their own risk.
General Adoption & Implementation Tips
James: Here are are few overall general tips when you’re looking at AI implementation adoption:
- Choose the right vendor – a true partner that offers AI that integrates seamlessly into your existing workflows and business processes
- Start with small wins – from AI replacing repetitive, manual tasks to AI integrated into the entire process
- Think beyond legal – this really is a team effort, not just within legal teams, but also the AI vendors you may choose
Ready to learn more about how Pramata can help you get started with contract AI? Contact us today!
