AI strategy
Local AI for business: when it makes sense and what you need
Local AI for business means running a language model on your own computers or servers, without sending data to an outside provider. This guide covers when it pays off, the hardware and software involved, its limits and a ten-minute test to decide.
What local AI for business means, and why law firms and clinics ask for it
Local AI for business is a language model that runs on a computer or server you control, rather than on the servers of OpenAI, Google, Microsoft or Anthropic. When a solicitor asks a local model to summarise a case file, the text is processed on that machine and does not travel to any outside provider.
This is possible because several companies publish open-weight models that anyone can download and run on their own hardware. They include Mistral AI, a French company, Google with its Gemma family and Meta with Llama. Free tools such as Ollama or LM Studio make it easier to install one on an office computer.
The people asking most often are law firms, accountants and clinics. They handle information covered by professional confidentiality or health data, which Article 9 of the GDPR treats as a special category. For them, keeping documents inside the building matters more than having the most capable model available.
When running AI locally makes sense, and when it does not
It pays off when two conditions meet: data you would rather not entrust to a third party, even under a business contract with safeguards, and narrow tasks that a mid-sized model handles well. Summarising a report, sorting incoming email, pulling dates and parties out of a contract or searching an internal archive are good candidates.
It also helps if you work somewhere with a poor connection, if you want the model to stay the same from one month to the next, or if usage is very high and repetitive. The cost then looks more like an equipment purchase than a bill that grows with use.
It does not pay off if your team mostly needs the best reasoning available: drafting a complex pleading, working through a strategy in depth or handling very long documents. The large cloud models are still better at that. Nor does it make sense if nobody, inside the company or at a provider, will look after the machine, update the software and keep an eye on who has access.
What you need to run AI locally
We are talking about a small business with one powerful computer or a small server. Exact requirements depend on the model you pick.
A machine with enough memory
The graphics card's memory, or the shared memory on computers with Apple chips, decides what size of model you can load. Compressed versions of a model take less room at some cost in quality. For several people at once, you usually need a server with a graphics card.
Software to run the model
Ollama runs on macOS, Windows and Linux. Note that it also offers cloud models that run on Ollama's own servers, so check that the model you pick is a local one. LM Studio is another desktop option.
A model with a suitable licence
Each open-weight model has its own licence, and some set conditions on commercial use. Read it before using the model at work, and test several models on your own documents.
A link to your documents
A model on its own knows nothing about your files. For it to answer from your case files or protocols, it needs a search over those documents, and you need to decide which folders each person may query.
Someone to look after it
Software and model updates, backups, user accounts and a usage log. It is small but steady work, so assign it from day one.
The limits of local AI worth knowing
Less capable than the large models
The models that fit on office hardware are smaller. They summarise and extract well, but make more mistakes on long chains of reasoning, legal or medical nuance and very long documents.
Speed depends on the machine
As Ollama itself puts it, speed depends on the hardware. On an ordinary laptop, a long answer can take long enough that staff stop using it.
The GDPR still applies
Processing in-house cuts out transfers to third parties, but your company remains the data controller. You still need to control access, know which data is processed and why, and keep a record of it.
Mistakes do not go away
A local model gets things wrong just as a cloud model does, sometimes more often. Every summary or extracted figure that goes to a client or patient needs a professional's review.
A ten-minute test: do you need local AI?
Answer yes or no to each question. Three or more yeses and a local pilot is worth trying. One or none, and a business cloud plan is usually simpler.
Is your data confidential by law or health data?
Client files, medical records, clients' tax details. If so, the case for running AI locally is strong.
Are the tasks narrow and repetitive?
Summarising, sorting, extracting fields, searching an archive. A mid-sized model does this kind of work well.
Can you or a provider maintain a machine?
Someone with time to install, update, back up and manage accounts, in-house or outsourced.
Would several people use it every day?
With daily, repeated use, the hardware cost is shared out. For occasional use it rarely pays.
Can you accept somewhat lower quality?
If your tasks need the best reasoning available, the gap with the large models will show.
Example
A Leeds law firm tries a model in its own office
A hypothetical law firm in Leeds with eight solicitors wants to summarise the documents clients send in and find dates and deadlines in long contracts. The partners do not want those documents leaving the firm, and they answer yes to four of the ten-minute test questions.
With their IT provider, they install an open-weight model on a computer with a graphics card on the internal network. They connect it to a test folder of closed files and set one rule: the model drafts the summary and the list of deadlines, and the solicitor in charge always reviews it.
For six weeks they compare the model's summaries with what a solicitor would have written. For general work, such as emails with no client data or slides for a talk, the firm uses a business cloud plan with the provider's contractual safeguards.
To measure it, they log the time each summary takes before and after, review included, the errors they find and their type, and how often a summary has to be redone. Those figures tell them whether to extend the model to all open files or stay with the cloud for everything.
Local, cloud or both: deciding where your money goes
This is not only a technical choice. It starts with knowing which data each task involves, who does it and what happens if the output is wrong. With that map, some tasks go to a local model, others to a business cloud plan, and some need no AI at all.
Obinova is an AI agency based in Barcelona and Paris. In an AI audit we map your processes with your team, the data they touch and the tasks AI can change, and recommend where each option fits. If the process calls for it, we build the workflows in n8n, which can be installed on your own server and connected to a local model.
Frequently asked questions
What is local AI?
It is a language model that runs on a computer or server your company controls, rather than on a provider's servers such as OpenAI's or Google's. The text and documents you give it are processed on that machine. It relies on open-weight models, which you download, and on software such as Ollama or LM Studio that runs them.
Is a local LLM safer than ChatGPT?
It reduces one specific risk: data reaching a third party. Security also depends on who can access the machine, backups and updates. A poorly protected computer in the office can be less safe than a business cloud plan with sound contractual safeguards and careful configuration.
What hardware do I need for a local LLM?
It depends on the model's size. What matters most is the memory available to the model: the graphics card's memory or, on computers with Apple chips, the shared memory. A small model runs on a good office computer. For several users at once or larger models, you need a server with a graphics card.
How much does it cost to run AI locally?
The most common tools and many models can be downloaded free, so the cost lies in the hardware, setup, connecting your documents and maintenance. It depends on the number of users, the model size and the tasks. At Obinova we give you a quote after the first meeting, once we understand the case.
Is local AI automatically GDPR compliant?
No. Processing data on your own machines avoids sending it to a provider, but your company remains the data controller. You need to know which personal data is processed, for what purpose, who has access and how long outputs are kept. With health data, which the GDPR treats as a special category, the requirements are stricter.