Blog

The Summer 2023 legal-tech conference season covered the interplay between human labor, Symbolic AI, and Generative AI.

When ChatGPT arrived in November 2022, what followed was the breakneck speed of competitive LLM rollouts, as well as week-by-week announcements of the latest and greatest Generative AI development. As a result, even the most sophisticated of the legal tech intelligentsia from law firms, legal departments, and legal-tech companies are asking this question: What’s next? The two best-attended legal-tech events of the summer sought to answer that question. The American Association of Law Libraries (AALL) and ILTACON. Between them, common themes offered a panoramic view into how we’ll make our way through this LLM-induced noise.

Below are key takeaways, with this as the recurring theme: the industry’s shifting roles and responsibilities are shifting. That’s especially true for those tasks that can be performed alternately by Humans, Symbolic AI, and Generative AI.

A Tale of Two Conferences

Dickens today could well write "It was the best of times, it was the best of times." Has the legal technology sector ever had it this good? Excitement from even the most Luddite lawyers is jaw-dropping. Myriad law-firm innovation personnel and Knowledge Management professionals have expressed statements like “Lawyers who before couldn’t care less about tech — they’re now asking ‘When can I use AI?’ It’s unprecedented!” Of course it is: LLMs are unprecedented.

On one hand, we have extraordinary, near-miraculous tools and platforms that appear to promise to revolutionize how we conduct legal research, manage caseloads, and even interact with clients. On the other hand, there's a palpable sense of uncertainty, even dread, among legal professionals about the implications of these technologies for their careers and the profession as a whole.

Maybe it is the “worst of times” after all? Probably not: We’re in for some exciting, good times.

AALL and ILTACON sought to address attendees’ concerns head-on: not with answers, but with the right questions — coupled with some practical ways that law firms and legal departments can leverage technology — both using traditional Knowledge Management and with the help of our new LLM friends. Both conferences highlighted the immense promise and perils of legal tech, but what was most interesting was a common thread running through both: the role of humans as they interact with both Symbolic AI and Generative AI.

Some definitions:

• Humans are, of course, how nearly all tasks have historically been performed: skilled lawyers and their allied legal professionals, billing hours and cranking out tasks.
• Symbolic AI or "Good Old-Fashioned AI" (GOFAI), relies on explicitly programmed rules — handling tasks like document tagging, data extraction, and other items quickly and effectively. TurboTax is Symbolic AI. Docket Alarm is Symbolic AI. Much of vLex is Symbolic AI. Hand-written computer code, following patterns to provide human-like output constitutes Symbolic AI. If it follows a pattern, a coder can create code to interpret that pattern through Symbolic AI.
• Generative AI (e.g., GPT-4, Claude, Bard) is a subset of machine learning that can create new content based on its training data. Generative AI is useful for tasks like document summarization, automated responses, and even creating first drafts of some legal documents.

Below, we’ll discuss how these three methods are evolving, how they were presented and discussed in the AALL and ILTACON conferences, and what they might mean for the future of the legal profession.

AALL: Law Librarians as the Original Prompt Engineers

Focus on Human Skills

Few legal professionals know how LLMs will change the world more than librarians. After all, they were “prompt engineers” before prompt engineering was cool. Isn’t a Boolean search the coolest Old School prompt?

AALL’s focus on law librarians, legal information professionals, and others deeply involved in the curation and dissemination of legal knowledge has served this year’s audience very well. The event spotlighted the vital role humans play in contextualizing, filtering, and delivering legal information. While there was plenty of buzz about AI tools that can automate aspects of legal research, the consensus was clear: humans are irreplaceable for tasks that require a deep understanding of legal frameworks, ethical considerations, and nuanced interpretations.

Role of Symbolic AI

If I had a dollar for every time someone at AALL said the word “structure,” that would probably pay for my AALL 2024 admission. What’s old is new again. Or perhaps, structured data is evergreen. It’s always been important. And in the world of LLMs, structured data — perhaps as provided as Symbolic AI — is more important than ever.

Where in the past, tagging meant humans laboriously deciding “Is it an X or is it a Y?” But with Symbolic AI, legal research databases like Docket ALarm can use Symbolic AI — through countless lines of handwritten code — to classify and tag legal documents. For example, Docket Alarm’s code can determine that a docket entry reflects an Order Granting Summary Judgment — and place the appropriate tags. Then, Docket Alarm can say the likelihood of being granted Summary Judgment in this jurisdiction, before this judge, for this Nature of Suit. It’s not going to predict the future. But it will tell you past results, which tend to indicate where to place litigation bets.

Machines won’t replace human experts, and the fruits of Symbolic AI labor will considerably expedite the lawyers’ resultant insights.

Emergence of Generative AI

Beyond Symbolic AI, the belle of the ball was Generative AI. How could it not be? With jaw-dropping performance on the Bar Exam, everyone in the industry is reckoning with the question of “What do we do now?” At AALL, the librarians sought to provide some answers:

1. LLMs could be used to draft initial versions of legal documents.
2. They could be used to update templates based on recent legal developments.
3. They could free human professionals to focus on more complex tasks.

These are all, of course, the promises of Generative AI. We’re all wondering if those promises will come true.

ILTACON: Where Law Meets Tech

Human-Centric Approach

Of all the conferences, few bring together tech-savvy legal professionals and vendors showcasing the latest in legal technology like ILTACON. Beyond AALL, ILTACON went far deeper into the business of law: client interactions, case management, data analytics, pricing, business development. But like at AALL, the overarching message at ILTACON was the importance of a human-centric approach. Many discussions and panels focused on how technology could assist but not replace human skills. Whether it's client relations or augmenting a brief’s linguistic flourish: the human intellect remains undefeated. For now.

Symbolic AI vs. Generative AI

As with AALL, the ILTACON presenters and attendees emphasized the importance of tagging and structuring the law’s otherwise unstructured data. How to perform data extraction? Maybe Generative AI, — but Symbolic AI (“Good Old Fashioned AI”) might be able to do it better/faster/cheaper.

Many ILTACON sessions delved into ethics and governance around these technologies. The specter of Avianca sanctions loomed large. Many pounded the necessary drum: Deploy Generative AI only with Human oversight.

But while humans are overseeing Generative AI: Look what it can do!

• Produce first-draft legal documents.
• Perform sentiment analysis.
• Summarize whole documents.
• Generate client communications.

The biggest question: How much time will it save? That question was asked. But of course, it remains unanswered. While the technology remains nascent, it made the ILTACON crowds more energized than ever. We live in exciting times.

Intersections and Common Themes

The Balance of Power: Humans vs AI

Both AALL and ILTACON provided the prevailing notion that while Generative AI technologies could prove invaluable tools for automation and analysis, humans will continue to dominate in areas requiring empathy, creativity, and contextual understanding. Today, Generative AI cannot touch the most-intelligent of our legal professionals.

• Humans are best at: Client interaction, strategic planning, and court appearances, where emotional intelligence and nuance are paramount. And of course AI oversight: Nobody forget AI oversight!
• Symbolic AI are best at: Data sorting, document tagging, data analytics. All tasks that can be rules-based and don't require “understanding.”
• Generative AI is best at: Document summarization, drafting of legal documents, low-stakes document generation, and taking subsets of large data sets, processing them to produce new legal documents and insights.

The Symbiosis of Symbolic AI and Generative AI

A recurring theme in both conferences was that symbolic AI and generative AI can work in tandem to deliver more effective solutions:

• Symbolic AI can provide the most-effective Retrieval Augmented Generation (RAG) — using pre-defined tags and/or vector databases to find relevant materials from massive datasets. And then taking those reduced-document results to hand the baton to Generative AI.
• Generative AI can then take these filtered materials and generate summaries, draft initial versions of legal briefs or memos, and perform other more complex tasks that require a semblance of 'understanding.' Symbolic AI and Generative AI can work together!

Build vs. Buy

In both AALL and ILTACON, the "Build vs. Buy" question flowed both across stages and in the hallways. In particular, organizations mused whether they should build their “own LLMs,” or buy fine-tuned models. They also asked whether organizations should “build” systems to tag their own data — or instead “buy” ready-tagged data?

Of course, choosing to manually tag data in-house means you have full control over the data structure. But this approach can be resource-intensive, involving expensive human labor and time taken away from billable tasks.

On the other hand, buying pre-structured data — tagged by Symbolic AI — might be more consistent and cost-efficient. Legal tech providers like vLex can leverage their economies of scale, allowing legal firms to benefit from both the company’s legal-tech expertise and the cost savings. Firms who opt to buy well-structured data can focus on their core capabilities, rather than distracting their timekeepers from the task of manual data tagging.

Naturally, "Build vs. Buy" discussions at AALL and ILTACON usually end with “your mileage may vary,” but those miles often end in easing the burden on overtaxed timekeepers and their allied professionals.

The Human Oversight

Both events stressed the importance of human oversight. Whether setting the parameters for Symbolic AI or reviewing documents generated by Generative AI, the human touch remains indispensable for ethical and accurate outcomes. Humans do, in fact, remain undefeated. For this round, at least.

CONCLUSION: The Harmonious Triad of Humans, Symbolic AI, and Generative AI

The takeaway from both AALL and ILTACON is that the near-future legal-tech battle between humans and machines is not a zero-sum game. Rather, the message from both conferences appears to be a harmonious triad — where each excels in their proper zone:

• Humans provide insights, prompting skills, creativity, and contextual understanding — along with massive amounts of oversight.
• Symbolic AI is a cheaper and often more-accurate method of tagging and analytics.
• Generative AI for summarizing and creating from existing data sets —even better if well tagged — and with large amounts of human supervision.

Instead of pondering the role of technology in the legal field, remember: it's not an 'either-or' but a 'better together.” If Dickens were to have attended AALL and ILTACON, the venerable Victorian would probably be optimistic: “It was the best of times; it was the best of times.”