# Local LLM (self-hosted language model)

> A local LLM is a large language model that runs on hardware a business controls, such as its own computer or server, instead of on an AI vendor's cloud service, so prompts and documents stay within that environment.

Also called: self-hosted LLM, private AI, on-premises AI, offline AI model, Ollama.

Openly available models, such as the Llama, Mistral, and Gemma families, can be downloaded and run on a workstation or server with enough memory, ideally with a capable graphics card. Ollama is an open-source tool that downloads and runs these models and exposes them through a local API. Open WebUI is an open-source, self-hosted chat interface that connects to Ollama, giving staff a familiar chat window in the browser.

The main reason to run a model locally is control over data. Prompts, client files, and outputs are processed on your own equipment and are not sent to a third party, which can simplify confidentiality and data residency questions for legal, medical, and financial work. Costs also take a different shape: hardware and electricity instead of per-use fees.

Models small enough to run on modest hardware are generally less capable than the largest cloud models, and responses can be slow without a suitable graphics card. Running it yourself means you own the updates, access control, backups, and security of that server. Local does not automatically mean secure. Test with real tasks before buying hardware. We set up private AI using Ollama and Open WebUI on machines the client controls.

## Where we do this work

- [Private local AI](https://techpitsolutions.systems/services/private-local-ai)
- [AI tools and integration](https://techpitsolutions.systems/services/ai-tools)

## See also

- [Retrieval-augmented generation (RAG)](https://techpitsolutions.systems/glossary/rag): Retrieval-augmented generation (RAG) is a technique in which an AI system first searches a set of documents for passages relevant to a question, then gives those passages to a language model so the answer is based on that source material.
- [AI agent](https://techpitsolutions.systems/glossary/ai-agent): An AI agent is software that uses a large language model to work toward a goal by deciding which steps to take, calling tools such as a search, a CRM, or a calendar, and acting on the results with limited human input.
- [Data protection laws](https://techpitsolutions.systems/glossary/data-protection-laws): Data protection laws are the rules that govern how organizations collect, use, store, and share personal information. Examples include the EU and UK GDPR, California's CCPA, Canada's PIPEDA, Australia's Privacy Act, and New Zealand's Privacy Act.

---

Source: https://techpitsolutions.systems/glossary/local-llm

TechPit Solutions · Run by AI, backed by real people. A remote team working with businesses in the US, UK, Canada, Australia, New Zealand and Europe. Email info.techpit.co@gmail.com, WhatsApp +92 344 3360585. Book a 15-minute call: https://techpitsolutions.systems/contact