What islocal AI?
The word local covers two different routes, and only one of them keeps the promise that no text leaves the building.
Get in touch
Let's talk about your project.
First we check whether the project fits your business model. Then you get a proposal with phases and effort.
Talk about local AI or call: +49 151 1576 5566Local AI means a company runs its AI application in a way that keeps control over data and system in house. There are two routes behind that phrase, though: either the language model itself runs on your hardware, or the application and documents sit with you while the model is called as a provider's service.
- Only when the model itself runs in house does no text leave the company network.
- Open models run on your own hardware, but they are usually smaller than the strongest cloud models.
- Germany's Data Protection Conference regards technically closed systems as preferable.
Which kind of local AI protects your data, and how far?
Most of what is sold as local AI for business follows one of two routes. On the first, you download an open language model and run it in your own data centre or with a host of your choice, often called on-premise AI. On the second, the application, file stores and knowledge base sit on your server, and only the text excerpts a task needs go to a provider's model. Both can fairly be called local, so ask any vendor one question first: where does the model run?
What you need to run a local model yourself
Hardware needs depend on the size of the model. On 5 August 2025 OpenAI released two open models, gpt-oss-120b and gpt-oss-20b, under the Apache 2.0 licence. According to OpenAI the smaller one runs with 16 GB of memory, while the larger one fits on a single data centre graphics card with 80 GB. If several people use it at once, you need headroom on top.
Then there is quality. Smaller models have less world knowledge, says OpenAI, and are more prone to making up facts. That rarely matters when you summarise or search your own documents, but it does for longer analyses. And someone has to look after hardware, updates and security, so costs shift rather than disappear.
Does local AI make data protection easier for you?
Yes, but it does not do the job for you. In its guidance of 6 May 2024, Germany's Data Protection Conference separates closed systems in a contained environment from open systems a provider runs, for instance as a cloud service, and prefers the closed ones. A legal basis, informing the people affected, their rights and, where a high risk is likely, a data protection impact assessment remain your job, wherever the model runs.
Frequently asked questions
Is a local AI as good as ChatGPT?
For many everyday tasks, yes. For demanding work we would test it on real examples from your business first.
Do I need a data processing agreement for a cloud model?
Often, yes. The German Data Protection Conference frequently treats an external provider running an AI application for you as a processor under Article 28 GDPR, and then a matching agreement belongs with it.
Where to go deeper
Local AI for Business: What “Local” Really Means
AI Assistant for Business: Types, Uses and Data Protection
Terms you should know in the same context
AI assistant · RAG (retrieval augmented generation) · Vector database · LLM (large language model) · AI hallucination · Back to the AI glossary A to Z
Local or cloud is less a matter of belief than a matter of tasks and data.