Tens of thousands of contracts arrive every year as PDF and DOCX files from vendors, suppliers, subcontractors, and insurers. Each one needs categorization, field extraction, and routing into an ERP system. Oligamy built a custom LLM trained on roughly 2,000 labeled examples entirely within the EU. It reached 98.2% first-pass accuracy while staying fully GDPR compliant, and it cut processing time per contract from 15 to 25 minutes down to 2 to 3 minutes, removing about 95% of the manual categorization work.
Key takeaways
- A custom LLM fine-tuned on roughly 2,000 labeled domain examples reached 98.2% first-pass accuracy. Modeling with a larger context window points to 99.1% as achievable.
- GDPR compliance and high accuracy coexist when training stays inside the EU and the model is client specific, never shared.
- Document classification only pays off when it integrates tightly with the ERP: field extraction, a mapping preview, human verification, and a feedback loop that returns corrections into retraining.
- Auto-flagging the ambiguous 1.8% for human review prevents silent errors while keeping throughput high.
- In-production retraining on thousands of new examples improves edge cases without reprocessing the historical archive.
The contract classification problem at scale
Construction and real estate enterprises sign contracts with many parties: crews, equipment suppliers, subcontractors, workers, material vendors, insurers, and regulatory bodies. Those contracts span vendor agreements, purchase and sale contracts for materials and equipment, subcontractor labor agreements, equipment leases, permits, insurance certificates, and attachments such as mortgages or guarantees.
This client received several thousand to tens of thousands of contracts a year, all requiring human review and manual categorization. A specialist would read each contract, decide its category, extract the key fields (vendor name, dates, amounts, certificate references), and map the structured data by hand into the ERP system or a CRM.
Manual processing took 15 to 25 minutes per contract. For an organization handling tens of thousands a year, that meant thousands of specialist hours spent on categorization alone. The previous vendor's data entry portal was slow and unclear, which added friction to an already heavy workflow. Misclassifications propagated into the ERP and corrupted downstream analytics and compliance reporting.
Why generic AI falls short
Off the shelf document classification APIs and generic LLMs had two fatal gaps.
First, generic models lack domain knowledge of construction contracts. They understand legal language broadly, but not this client's categories, ERP field mappings, or business rules. A generic model might handle a standard equipment supply agreement, then fail on edge cases: a parts order that also includes on-site labor, or a vendor agreement that doubles as an equipment lease.
Second, the client's GDPR obligations ruled out sending contracts to shared API endpoints. European regulation requires explicit data residency and data sovereignty. Pushing contract content to a third party SaaS platform, where it might be pooled with training data from competitors, was not an option.
Oligamy's approach to AI contract automation was to fine-tune an OpenAI base model with training performed entirely inside the EU. The team labeled roughly 2,000 contract examples, used them to fine-tune the base model, and produced a dedicated, client specific LLM. Training never left EU boundaries. The model was never trained on competitor data and never shared with other clients. That combined the capability of a large model with the compliance the client needed. This is the core of how we build AI Solutions for regulated, document-heavy operations.
Building the pipeline: intake to verified ERP entry
The solution is not a single LLM. It is a six-step pipeline that moves a document from intake to a verified place in the ERP.
Intake. Documents arrive as PDF or DOCX. The system extracts raw text and metadata (filename, arrival date, format). OCR handles scanned amendments and low quality PDFs.
Classification. The fine-tuned LLM reads the contract and outputs four things: the predicted category (vendor agreement, equipment order, labor contract, permit, insurance, attachment), a confidence score, the reasoning behind the call, and an ambiguity flag for multi-category contracts.
Field extraction. A second pass pulls structured data: party names, start and end dates, amounts, certificate or license references, attachment links, signatory details, and the domain-specific fields the ERP expects.
ERP mapping. Extracted fields are pre-mapped to the correct ERP table and columns. The system shows a preview, for example "this contract will be entered into Vendor Agreements with vendor name, start date, and end date." This step matters because it lets a person verify placement before anything commits.
Human verification. An operator reviews the classification, the extracted fields, and the preview. If it is correct they approve and the data commits. If not, they fix it. Every correction is logged and fed back for retraining.
Automatic error detection. In production the system watches for patterns: recurring misclassifications, systematic extraction errors, format changes. Those patterns raise alerts and feed scheduled retraining runs.
About 98.2% of contracts clear this pipeline on the first pass. The remaining 1.8% are auto-flagged as ambiguous and sent straight to human review. The result is a hybrid workflow with no silent errors: every document is either classified, verified, and committed, or flagged for full review when the model is unsure.
Results: accuracy, speed, and labor impact
The deployed system classified 98.2% of contracts correctly on the first pass across all categories. Tests with a larger context window suggest the ceiling could reach 99.1%, but the current model already sits well above any manual baseline.
Time per contract dropped from 15 to 25 minutes to about 2 to 3 minutes. Operators no longer read and categorize from scratch. They review a pre-filled, pre-classified form and spot-check it.
Manual categorization work fell by roughly 95%. As an illustration, an organization processing 10,000 contracts a year at 20 minutes each spends around 3,300 specialist hours on intake. The same volume on this system needs 500 to 750 hours, which frees more than 2,500 hours a year for exception handling, compliance audits, and vendor relationships.
During the post-launch phase the model took in roughly 5,000 more contract examples. That continuous learning sharpened edge cases and new document patterns without reprocessing the entire archive.
GDPR compliance and data sovereignty
Many EU enterprises assume cloud plus AI equals risk. This project shows the opposite: GDPR compliance and 98% accuracy sit together.
All training ran inside the EU. No contract text, metadata, or extracted field left European boundaries during training or inference. The model ran as a dedicated client instance, not a shared resource, so this client's data never mixed with another organization's, and no competitor data ever improved its performance.
The build followed GDPR requirements end to end: data governance, access controls, audit logging, and retention policy. The OpenAI base model was used inside a compliance framework designed for enterprise deployments. A dedicated instance also shrinks the attack surface. A shared third party endpoint adds dependency and liability, while a controlled, client-owned model does not.
Implementation timeline and team
The project ran from September 2025 through January 2026, about 16 weeks from kickoff to full production rollout. The core team was three people: a Product Manager, an AI Team Leader, and an AI Senior Developer.
Phase 1, restricted pilot. The team hand-labeled a small base of contracts, set the category taxonomy, and trained the model in a restricted environment. The pilot passed and confirmed the targets were reachable.
Phase 2, technical improvements and scaling. The labeled set grew to roughly 2,000 contracts. The team added categories, improved field extraction, scaled the custom training, integrated the output with the ERP, and built the preview and verification interface.
Phase 3, enterprise rollout. The system went live across the organization. Operators were trained, and monitoring and error detection were configured.
Phase 4, in-production retraining. The model kept learning from real work. Thousands of new contracts were processed, corrected by operators, and used to retrain it, while the automatic error detection patterns were refined.
The timeline could have been shorter, but formalities and compliance work added weeks. Audit procedures, security reviews, and regulatory sign-offs do not accelerate. For teams that need this kind of capacity on demand, we run it through Dedicated Teams.
Parallel work: portal modernization
Alongside the LLM project, Oligamy rebuilt the client's legacy construction management portal, where specialists enter project data: apartment details, square meters, material breakdowns, and labor costs.
The old portal dragged. Page load sat at 8 seconds against an industry norm of 2 to 3 seconds, and the data entry flow was unclear, with too many clicks and context switches. The rebuild cut page load to 1.2 seconds and simplified the data entry flow by about 40%. The two efforts compounded: the LLM pipeline sped up contract intake, while the rebuilt portal made project data faster to enter and maintain. This kind of legacy modernization is part of our Digital Acceleration work.
Why this matters for enterprise operations
Classification and field extraction are the first steps of any document workflow into an ERP. When they are manual, everything downstream waits, and that is where the hours pile up.
The fine-tuned approach works because it combines three things: domain training on real labeled examples, tight integration with the systems that consume the output (ERP, CRM), and a verification workflow that keeps people in control. Operators see every classification decision and can correct it.
The same document classification AI pattern transfers to other high-volume workflows: procurement (purchase orders, invoices, delivery notes), HR (employment contracts, certifications), legal (case files, discovery), healthcare (records, claims), and financial services (loan applications, regulatory filings).
Key technical decisions
- Base model plus fine-tuning. Starting from an OpenAI base model and fine-tuning on client data was faster and cheaper than training from scratch.
- Training kept inside the EU. Committing to EU only processing added weeks but was non-negotiable for GDPR and client confidence.
- Dedicated client instance. A client specific model, not a shared multi-tenant pool, removed data leakage risk and allowed aggressive customization.
- Hybrid verification. The 98.2% threshold means 1.8% gets flagged for a human, which removes silent errors while keeping throughput high.
- Continuous retraining. Learning from corrected examples in production improved edge cases without disrupting the live system.
Scalability
The same pipeline extends to new document types without new infrastructure. Each added category or document class reuses the intake, extraction, verification, and retraining loop already in place, so the labor savings compound as volume grows or as more workflows move onto the model.
Looking ahead
A larger context window could push accuracy toward 99.1% and remove manual verification on most documents. In practice, the jump from 98.2% to 99% changes little day to day, because the bottleneck is already gone: classification is fast and operators verify in batches.
The next step is extending the system to more document types (invoices, purchase requisitions, compliance filings) and automating more of the routing logic downstream. Oligamy builds custom AI for high-volume document workflows. Contact Oligamy to talk through your own requirements.
FAQ
What accuracy can a fine-tuned LLM reach for document classification?
This project reached 98.2% first-pass accuracy on roughly 2,000 labeled examples, and modeling suggests 99.1% with a larger context window. Real results depend on training data quality, category complexity, and document clarity. Start with a pilot on a few hundred annotated examples to set a baseline before production.
Can an LLM stay GDPR compliant while staying accurate?
Yes. Training entirely inside the EU, using a dedicated client instance instead of a shared pool, and keeping data in region is fully compatible with 98%+ accuracy. Compliance and performance are not a trade-off here. They require disciplined data governance, which this project proved.
How long does it take to build and deploy a custom document classification LLM?
About 16 weeks on this project, from September 2025 to January 2026, with a team of three (Product Manager, AI Lead, Senior Developer). That included regulatory and compliance formalities that cannot be rushed. It can be faster with lower regulatory overhead or existing annotated data.
How does the LLM integrate with an existing ERP?
Classification output feeds field extraction, which structures the data for ERP mapping. A preview shows the operator exactly where the document will land before commitment. Corrections are logged and feed retraining, which creates a continuous improvement loop.
How many documents still need a human?
About 1.8% are auto-flagged as ambiguous and routed to a person. The other 98.2% run through automated classification and operator verification. The hybrid model removes silent errors while keeping throughput and operator efficiency high.
Explore AI Solutions for document workflows, or Digital Acceleration and Dedicated Teams for enterprise ERP integration projects.