Josef unveils AI pre-processing engine for legal sector

Yesterday

Josef, the legal technology company, has announced a major upgrade to its AI-powered Q&A module, Josef Q. The update introduces a new document pre-processing engine, designed specifically to improve the accuracy and reliability of AI-generated legal and compliance responses.

Co-founder and COO of Josef, Sam Flynn, emphasised the importance of refining knowledge sources within AI workflows, stating, "It doesn't matter how sophisticated the agentic workflow that you're trying to build, if you don't get the knowledge sources right, it is going to be constrained in the value that it gives to you."

A New Approach to Legal AI

The new engine, built after two years of research with customers such as Bupa and L'Oréal, alongside academic collaborations with institutions like NYU and Cornell Tech, aims to address longstanding challenges in legal AI. Flynn described the problem with traditional retrieval-augmented generation (RAG) approaches, which often split legal documents arbitrarily, leading to missing or fragmented information.

"If you've used a Microsoft Copilot or a generic tool, you've uploaded your documents and suddenly it's splitting at strange places, or you've lost whole parts of the document—that is the augmented generation part going awry," Flynn explained.

To combat these issues, Josef has implemented three key innovations in its pre-processing engine:

Hierarchy-Based Chunking – The engine respects the inherent structure of legal documents, analysing sections, clauses, and paragraphs rather than arbitrarily dividing text. "So it is breaking down that document, but it's breaking it down with respect to the internal structure," Flynn said. "It's not gonna split clauses; it's gonna understand how sub-clauses relate to clauses."
Data Augmentation – This process introduces controlled variations, ensuring that AI models are exposed to different phrasings and contexts within legal language.
Contextual Enrichment – By adding metadata and semantic cues, the engine helps AI tools interpret legal text more accurately. "The tool doesn't just recognise this all as one big slab of text," Flynn noted. "It knows that there are headings, subheadings, clauses, and lists. It recognises those and, through a process of data enrichment, tells the tool what the structure of the document is and where all this relevant information is."

Hyper-Accurate Responses for Legal and Compliance Teams

One of the key benefits of the new pre-processing engine is its ability to improve the accuracy of AI-generated answers. During a demonstration, Flynn showed how a legal Q&A tool built with Josef could quickly and reliably answer a common in-house legal query: "Who can sign for a 5K invoice?"

"It comes back with a hyper-accurate answer to the question," Flynn said. "It does that by understanding where in the underlying document that answer should be drawn from."

The updated system also includes interactive answer sources, allowing users to audit AI-generated responses instantly. "You'll see here that this section is complete, it's not split in the middle after 50 tokens. It goes from the header all the way through to the next header. That seems super basic, but there are very few RAG tools that are doing that well in legal and compliance," Flynn noted.

The Future of RAG in Legal AI

While some in the AI community have speculated that RAG is being replaced by autonomous agent-driven models, Josef remains committed to evolving and optimising RAG-based approaches for legal applications. "RAG is certainly changing and evolving, but it is not dead," Flynn asserted. "Getting this part right is hyper-critical, and with Josef Q's new pre-processing engine, you can do that faster and more easily than ever."

Josef plans to build on its latest advancements with additional features in the near future, including automatic question categorisation and built-in tool strength assessments. "There's so much more to show you," Flynn said, adding that the company is eager to demonstrate its new capabilities to legal and compliance teams seeking to streamline their workflows.

With this latest update, Josef aims to enable legal and compliance professionals to create AI-powered tools that provide reliable and accurate support across their organisations—without requiring extensive technical expertise or IT involvement.

Share on: