Adopting local AI in a patent firm
Workable even at small scale. An overview of hardware, software and the limits of local language models in patent practice.
Article last updated on 28 June 2026
Key insight: In drafting and large parts of patent prosecution, we work with content the model cannot know. The presumed cloud advantage from broader training knowledge largely falls flat there. What matters is the quality of the context, not the size of the model. That makes local AI on a four-figure hardware investment sufficient for this class of work.
Local AI is within reach even for smaller firms. A well-equipped workstation with a gaming-class GPU or NVIDIA's compact DGX Spark is enough to run current open-weight language models such as Gemma 4 entirely on the firm's own equipment. The one-off hardware investment sits in the four-figure range. What it takes beyond that is a clear sense of where local models are strong and where they are not.
What "local" means here
Two meanings depending on the setup. In the simplest case, the practitioner's own machine — laptop or workstation — processes the texts itself; the invention disclosure does not even leave the hard drive. In a multi-user setup, a central workstation lives in the firm and is reached over the internal network; the data leaves neither the building nor lands with a cloud provider. Either way: there is no cloud contract that confidentiality hangs on. It hangs on the firm's own IT.
Where the cloud advantage shrinks
In drafting, in writing an office-action response or a position paper, we work with texts the model cannot know: an invention disclosure, a specific claim set, a specifically cited prior-art document. None of this content sits in the training data of ChatGPT or Claude — it is new.
The presumed cloud advantage of broader world knowledge and more versatile reasoning largely falls flat in this class of tasks. What actually counts is the quality of the context — for an invention disclosure, for instance, whether the invention is described clearly and in an enabling way, and whether the advantages, technical effects and distinguishing features over the prior art come through clearly — and context engineering, the discipline of giving the model exactly that context and not more in the prompt. That is the practitioner's job, not the model's. And it does not change when you move from cloud to local.
Two hardware paths
Either path comes in at a four-figure one-off investment.
Path A · Workstation with a gaming GPU
A machine with 32 GB of RAM and a dedicated graphics card in the RTX 3090 or 4090 class (24 GB VRAM). That runs Gemma 4 in its 26-billion-parameter variant comfortably at roughly 40 to 60 tokens per second, fast enough for interactive drafting. Apple-Silicon workstations (Mac Studio, Mac Pro, MacBook Pro with 32 GB of RAM or more) are an equivalent alternative.
Path B · NVIDIA DGX Spark
A compact workstation with 128 GB of unified memory, designed for local AI inference. The DGX Spark even runs models up to 200 billion parameters; on the larger variants at around 32 to 33 tokens per second, not production-scale for many parallel users, but plenty for confidential patent work.
Which path fits better depends on whether a second model or the next model generation is already on the roadmap. A workstation with an RTX 4090 carries today's relevant models without trouble; the DGX Spark adds headroom upwards.
Software stack
The hardware runs an inference framework that loads the model files, fits them to the available hardware, and exposes an API. Three frameworks are practical: Ollama (lightweight, ideal for automation), LM Studio (graphical interface, good entry point) and Msty (with an "Ask your data" RAG feature).
The actual language model is downloaded separately. Two recommendations:
- Gemma 4 (26B) — a well-balanced variant for local use. Apache 2.0 licence including a patent grant, so no licence-side risk for patent work.
- Qwen 3 in a fitting size — strong on structured text, a solid all-rounder.
For Office integration (connecting Word and Excel directly to a local model) there is a small Python bridge built on FastAPI, described in the knowledge article Office + AI Integration.
Anyone who wants their own document corpora searchable, for example a back catalogue of office actions or the firm's own files, sets up RAG with ChromaDB and Ollama; the build instructions are in the RAG Workshop Guide.
Step-by-step installation of every component is covered in the Local AI workshop module, with installation videos for Ollama and LM Studio and a curated model selection by available hardware.
What still has to come from the practitioner
Local models hallucinate just like cloud models do. Even in a closed environment without cloud risk, the duty of evidence is a non-negotiable part of every workflow: every model statement has to be backed by a verbatim quote and reference from the underlying source.
On top of that: local models react more strictly to prompts than cloud models. Vague instructions are ignored or misread. Working with Gemma 4 or Qwen 3 means writing branches as explicit IF-THEN rules, giving few-shot examples, and defining output templates verbatim. The patterns are collected in the knowledge article Prompt engineering for local LLMs.
The AI already running in your tools
A local model only shuts one of several doors through which client data leaves the building. Confidential content often reaches a cloud model without anyone having decided so on purpose. Embedded assistants are the most common route: Copilot in Microsoft 365, Gemini in Google Workspace, the Zoom AI Companion in meetings. They are often switched on by default or a single click away, and each of them can send document or meeting content to an external model on terms nobody reviewed. The local workstation does not help if the same application also runs through Word's embedded Copilot.
The second, quieter route is the memory feature. Chat tools increasingly remember earlier conversations and pull them in as context. What was entered for client A last week can surface while drafting for client B: a cross-matter bleed that the local-or-cloud decision tree does not catch, because it builds up inside a single tool over time.
In practice that means going through which of the existing tools bring AI features and on what terms, then deciding deliberately what stays, what gets switched off, and what is walled off for matter work (a separate profile or account). For confidential work, turn the memory feature off or use a fresh, temporary chat, the same hygiene as context engineering, where the point is likewise to control exactly what the model gets to see.
The first 30 minutes
Anyone who wants a first impression without committing to an investment can start with a regular office laptop and a small model such as Gemma 3n (E4B). It is enough to try out the patterns from the prompt-engineering article; output quality is limited, but the behaviour of local models becomes tangible. For full drafting quality, the step up to a workstation pays off.
Step-by-step installation is covered by the Local AI workshop module. Half an hour, and the entry point is in place.