The 9 best local AI models for your own hardware

The interesting question is rarely “which open model is the best”. It is: which is the best one that still fits in my memory. That is where the matter is settled. A model that does not fit entirely into graphics memory falls back to the CPU and becomes orders of magnitude slower.

So this list is sorted by memory class rather than as a ranking. Six hardware tiers from 8 GB to a server rack, plus three special cases: best German, best code and the European question. Every figure comes from our model catalogue, which currently checks 48 open models against their official model cards. If you would rather start from your own hardware than from a model list, enter it in the hardware calculator and get the same selection filtered. Whether running models yourself is worth it at all is decided one step earlier, in the guide to local AI models.

The short version

Class	Recommendation	Memory	Licence	German
8 GB	Qwen3-VL 8B	7.5 GB	Apache 2.0	good
16 GB	gpt-oss 20B	16 GB	Apache 2.0	good
24 GB	Qwen3.6 27B	18 GB	Apache 2.0	strong
48 GB	Llama 3.3 70B	43 GB	Llama Community	good
96 GB	Mistral Medium 3.5 128B	75 GB	Modified MIT	strong
Server	Kimi K3	1,560 GB	Kimi K3 License	good
German	Gemma 4 31B	20 GB	Apache 2.0	strong
Code	Devstral Small 2 24B	16 GB	Apache 2.0	good
Europe	Teuken 7B, EuroLLM 9B	6 to 7.5 GB	Apache 2.0	strong

Every name links to the official model card. That is where the weights sit for download, the licence in full, and the figures our catalogue checks against. If you just want to run the model afterwards, pull it with a single line via Ollama or LM Studio.

How we picked

This belongs in the open, otherwise a list like this is worth nothing.

The memory footprint applies to quantisation level Q4_K_M, the default in Ollama and LM Studio, plus a context window of roughly 8K. The rule of thumb behind it is about 0.55 GB per billion parameters, rounded by hand. Two families fall outside that calculation because they are trained natively in MXFP4 rather than quantised after the fact: for gpt-oss the vendor’s stated minimum applies, for Kimi K3 the weight of the published files, because Moonshot never named one.

For mixture-of-experts models the total parameter count is what matters for memory, since every expert has to be loaded. Only the active ones matter for speed. That is exactly where the speed advantage of these models at equal size comes from.

The capability rating is editorial. The catalogue keeps a value from 0 to 100 per model, which it uses to sort which model a memory budget can carry at most. That is not a measured benchmark but a reasoned assessment. Where two models sit close together, this article is therefore decided not by the value alone but by German and licence. Where that happens, it is stated. As an independent cross-check, the model database additionally carries the ELO rating from LMArena, where anonymised models compete blind and real users decide. If you prefer measured figures to votes, Artificial Analysis covers the same open models with throughput, latency and price per million tokens.

The hard facts (parameters, native context window, licence) are checked against the vendor’s official model card, each with its own verification date. As of this list: 6 August 2026.

1. Up to 8 GB: Qwen3-VL 8B

7.5 GB memory, Apache 2.0, 256K context, German good

The entry class, meaning a common consumer graphics card or a laptop with 8 GB of unified memory. Qwen3-VL 8B delivers the most capability per gigabyte here: it understands images, holds 256,000 tokens of context and handles tool calling, meaning it can invoke external functions.

The caveat: in German it only rates as good, not strong. If you need better German at the same memory footprint and can do without image understanding, take Ministral 3 8B. That is the only real trade-off in this class.

2. Up to 16 GB: gpt-oss 20B

16 GB memory, Apache 2.0, 128K context, German good

A mixture-of-experts model with 21 billion parameters, of which only 3.6 billion compute per token. That makes it considerably faster than its size suggests, and it brings adjustable reasoning depth. Because it ships natively in MXFP4, OpenAI names 16 GB as the minimum in the model card rather than the 11.5 GB the rule of thumb would produce. The difference sounds small and is the one between a 12 GB card you may already own and a new one.

This is the closest call on the whole list. Mistral Small 3.1 24B needs the same memory, sits one point below in the rating, but is strong rather than good in German and comes from the EU. If you mostly process German documents, take Mistral Small in this class.

3. Up to 24 GB: Qwen3.6 27B

18 GB memory, Apache 2.0, 256K context, German strong

The clearest winner in the field. Qwen3.6 27B leaves behind everything in this class that demands more memory: Gemma 4 31B and Qwen3 32B need 20 to 21 GB and still rank below it. Add image and video understanding, 262,000 tokens of context, strong German, strong code and Apache 2.0.

Anyone setting up a workstation for local AI today lands here with very high probability. The 24 GB class is the point where local models stop being a compromise. What is still out of reach above it shows in the distance to the closed frontier: Claude Opus 5 can do things no open model of this size manages, it just does not run in your server room.

4. Up to 48 GB: Llama 3.3 70B

43 GB memory, Llama Community, 128K context, German good

On raw capability, Llama 3.3 70B wins this class, and by a clear margin. It is the first model on the list that reaches into the flagship class.

Only it is a win with two deductions. In German it is merely good, and it sits under the Llama Community License rather than a free licence. The smaller Qwen3.6 27B from the class below is strong in German and Apache 2.0. For German-language document work, the extra investment in 48 GB is therefore hard to justify: you buy capability and pay with language quality and licence freedom.

5. Up to 96 GB: Mistral Medium 3.5 128B

75 GB memory, Modified MIT, 256K context, German strong

This is where the conflict from tier 4 resolves. Mistral Medium 3.5 is the strongest model on the list that also brings strong German, it comes from Europe and holds 256,000 tokens of context.

Three alternatives are worth a look when memory is tight. Mistral Small 4 119B saves five gigabytes at 70 GB, stays strong in German and sits under Apache 2.0 rather than Modified MIT. NVIDIA’s Nemotron 3 Super rates even higher at the same 70 GB and offers a context window of one million tokens. gpt-oss 120B fits on a single H100 at 80 GB. The latter two are only good in German, though.

6. Server hardware: Kimi K3

1,560 GB memory, Kimi K3 License, 1M context, German good

The largest openly available model there is: 2.8 trillion parameters, 104 billion of them active, a context window of one million tokens. That is no longer a workstation, that is a server with multiple cards.

The real point of this class hangs on the memory figure. 1.56 terabytes is roughly twenty H100 cards, in practice three servers with eight each. The fact that only 104 billion parameters compute per request changes nothing: a mixture-of-experts model has to keep every expert loaded, the active parameter count buys speed and never a smaller machine. If you are genuinely provisioning server hardware and not chasing the maximum, DeepSeek V4-Pro takes you further, demanding 880 GB at 1.6 trillion parameters and sitting under MIT. What running K3 costs per month, and at what volume it pays off against an API, we worked through once.

7. Best German per graphics card: Gemma 4 31B

20 GB memory, Apache 2.0, 256K context, German strong

When German language quality is the only criterion and the hardware should stay at one card, Gemma 4 31B is the safe choice. Google also put the Gemma 4 generation under Apache 2.0 for the first time, whereas Gemma 3 still sat under the more restrictive Gemma Terms. In LMArena the 31B model ranks third among open models, which confirms the editorial rating from the outside.

In the capability rating it sits behind Qwen3.6 27B, which also offers strong German and needs less memory. Gemma stays in regardless, because among the models rated strong it delivers the most consistent German phrasing. If you have both cards free, run them against your own texts.

8. For code: Devstral Small 2 24B

16 GB memory, Apache 2.0, 256K context, German good

A model from Mistral specialised in programming, built for agentic work across multiple steps rather than individual code snippets. The memory footprint is the remarkable part: 16 GB, so the same class as an all-rounder.

With more hardware, Qwen3-Coder-Next (52 GB) and Kimi K2.7 Code (560 GB) offer stronger specialists. For the size that sits in a normal office, Devstral Small 2 is the best compromise between capability and memory. A specialist does not replace an all-rounder, it comes alongside one: the reason why one model is never enough.

9. For Europe: Teuken 7B and EuroLLM 9B, with a caveat

6 to 7.5 GB memory, Apache 2.0, 4K context, German strong

These two belong on the list because they are often sought and rarely described honestly. Teuken 7B comes from the Fraunhofer project OpenGPT-X, was trained on the Jülich supercomputer JUWELS and covers all 24 official EU languages. EuroLLM 9B comes out of the EU-funded UTTER project and covers 35 languages. Both sit under Apache 2.0, both are strong in German.

And both have a context window of 4,096 tokens. That is enough for a chat, but not for a quote, a contract or a set of minutes. With Teuken there is what the model card states on top of it: the training data ends in September 2023. The funding programme itself expired on 31 March 2025, after roughly 14 million euros from the German Federal Ministry for Economic Affairs. For document work, neither is suitable.

This is the uncomfortable part: sovereign European models exist, they are cleanly licensed and they speak good German. It is just that the use cases companies want local AI for practically all run past 4,096 tokens. If you want Europe in your stack, you get it through the Mistral family, not through the research models. And if you solve it via where it runs rather than where the weights come from, the five questions to ask a provider are in EU hosting and GDPR.

The licence question is smaller than its reputation

“Open weights means restricted use” is a stubborn belief. The catalogue says otherwise: 34 of the 49 models sit under Apache 2.0 or MIT, so under a genuinely free licence. The restrictions concentrate in a few families, above all Meta’s Llama Community License and the Gemma Terms of the third Gemma generation.

In practice that means the licence is rarely the bottleneck, it is a filter you set once. If only Apache 2.0 and MIT qualify, two thirds of the field still remain, the top included. Exactly how much, you can see in the catalogue when you set the licence filter to free.

If none of these nine fit

This list answers one particular question: the best model per hardware class. It replaces neither the comparison nor the decision in principle.

All 49 models, filterable by memory, licence, context window and German quality, are in the model database. If you think from the hardware rather than the model, take the calculator. Whether running your own pays off for you at all, or whether EU hosting is the calmer answer, is settled in the guide to local AI models. And what separates a corporate LLM from ChatGPT in the first place is here.

And then you need it in daily work

The question “local or cloud” is usually posed as though it were a switch you flip once. In practice it is an assignment: the draft contract goes to the local model, the research to the strongest one currently available.

That is exactly how RelationFlow is built. Every model on this list can be wired in as its own endpoint, via Ollama, OpenRouter or any OpenAI-compatible API, and then appears in the same model picker as GPT and Claude. It runs through your own account, counts against no token budget and is included in every plan, the free one too. How to set it up is in the documentation under Bring Your Own Model (BYOM).

What that changes day to day: your people stop switching tools depending on how sensitive a document is. They pick the model that fits the task in the same chat, and the company documents sit in the same place for both.

What this looks like with your own documents, the demo shows you in a few minutes.