Local LLM vs private LLM server: when your laptop is not enough

Running a model on your own laptop with Ollama takes one command, and for many people that is the whole answer. This guide shows where that stops: how much memory each model size needs, what happens when more than one person uses it, and which hardware covers each case.

The model sizes are the 4-bit files in the Ollama library on 13 October 2026. The memory column is only the model itself. Each conversation needs more on top, and that part decides when one machine is no longer enough.

How much memory a local LLM needs

Two things take memory: the model weights and the conversation.

The weights. At 4-bit quantization a model takes a little over half a byte per parameter. The Ollama library lists Qwen3-8B at 5.2 GB, Qwen3-14B at 9.3 GB, Qwen3-32B at 20 GB, Llama 3.3 70B at 43 GB and gpt-oss-120b at 65 GB. At 8-bit Llama 3.3 70B grows to 75 GB, and at full 16-bit precision to 141 GB.

The conversation. While the model reads and writes, it keeps a key-value cache for every token in the context. Its size follows from the model's configuration: 2 (key and value) × layers × KV heads × head size × bytes per value.

For Llama 3.3 70B that is 2 × 80 × 8 × 128 × 2 bytes, or 320 KB for every token at 16-bit. A conversation of 8,000 tokens needs about 2.5 GB, and the full 128,000-token context needs about 43 GB, as much as the 4-bit model itself. Qwen3-32B needs 256 KB per token, so 32,000 tokens take 8 GB.

So a 70B model that "fits" in 48 GB fits with a short context only. Long documents, large codebases and agents that resend the whole history need the memory for the cache as well.

When a laptop or one GPU is enough

A local model is a good fit when:

• One person uses it. By default Ollama answers one request per model at a time (OLLAMA_NUM_PARALLEL is 1) and uses a context of 4,096 tokens, according to its FAQ.
• The model is 32B or smaller. Qwen3-32B at 20 GB fits on a 32 GB card such as the GeForce RTX 5090 with room for context. Smaller models run on most recent laptops.
• The work is occasional. Drafting, summarising, trying prompts, coding help. If the laptop is closed at night, nothing breaks.
• The data is yours alone. A local model is the simplest way to keep a document on your own machine.

For learning, testing and personal use, start here. It costs nothing extra, and it shows whether an open model is good enough for your task before you spend money on anything bigger.

When you need a private LLM server

The limits show up in four places.

1. The model does not fit. Llama 3.3 70B at 43 GB does not fit on one 32 GB card. gpt-oss-120b needs 65 GB, and Qwen3-235B-A22B needs 142 GB. These models run on cards with 80 to 96 GB or on several cards together.

2. More than one person uses it. Every parallel conversation needs its own cache. Ollama's FAQ states that memory grows with OLLAMA_NUM_PARALLEL × context length. Twenty colleagues on Llama 3.3 70B with 8,000 tokens each would need up to 52 GB of cache on top of the 43 GB model, about 95 GB in the worst case. Serving engines such as vLLM use PagedAttention to give memory only to tokens that exist and continuous batching to serve many requests at once, so the real figure is often lower. It still does not fit on a laptop.

3. It has to run all the time. A chatbot on your website, a support assistant or an AI agent that checks mail every few minutes cannot depend on a laptop that sleeps, travels and updates itself.

4. The data belongs to the company. Customer records, contracts and source code may not leave company control. A private server with one owner, a contract and a known location answers that question for auditors more easily than a set of personal laptops does.

Local LLM, private server or API: which fits whom

Our server has eight RTX PRO 6000 Blackwell Server Edition cards with 96 GB each. One card holds gpt-oss-120b with room for the cache. Two cards hold Qwen3-235B-A22B at 4-bit. Eight cards can run one copy of a mid-size model per card, or one large model across all of them. Details are on the RTX PRO 6000 server page, and data handling is covered on the private LLM hosting page.

Moving from local to a server

The step is smaller than it looks:

1. Test the model locally first. If a 32B model already does the job on your laptop, you may not need a 70B model at all.
2. Pick a serving engine. Ollama is simple and handles a few users. vLLM and SGLang are built to serve many requests in parallel.
3. Keep the same API. Ollama and vLLM both offer an OpenAI-compatible endpoint, so applications built against your laptop usually need only a new address.
4. Set the context you need. Larger context means more cache per user. Decide the limit from your real documents, not from the model's maximum.

Questions

What is the difference between a local LLM and a private LLM? A local LLM runs on the device in front of you. A private LLM runs on hardware that you or your company control, which can be a server in a data centre. Both keep the model and the data out of a shared public service.

Can I run a 70B model on a laptop? Only on a laptop with very large unified memory, and then for one person and a modest context. Llama 3.3 70B needs 43 GB for the 4-bit model alone, plus about 2.5 GB for every 8,000 tokens of conversation.

Does quantizing the cache help? Yes. Ollama can store the cache at 8-bit or 4-bit when Flash Attention is enabled, which cuts its memory roughly in half or to a quarter. Test the answers afterwards, since lower precision can change them.

How do I estimate memory for a model not listed here? Take the 4-bit file size from the model page, then add the cache: 2 × layers × KV heads × head size × 2 bytes, multiplied by the tokens per conversation and the number of conversations at once. The layer and head counts are in the model's config.json.

Sources

• Ollama library, file sizes and context windows: llama3.3, qwen3, gpt-oss, checked on 13 October 2026.
• Ollama: FAQ, on OLLAMA_NUM_PARALLEL (default 1), the default context of 4,096 tokens, memory per parallel request and K/V cache quantization, checked on 13 October 2026.
• Model configurations on Hugging Face: Llama 3.3 70B Instruct (an ungated copy of Meta's file: 80 layers, 8 KV heads, head size 128) and Qwen3-32B (64 layers, 8 KV heads, head size 128), checked on 13 October 2026. The per-token cache sizes are our calculation from these values at 16-bit.
• vLLM: documentation, on PagedAttention and continuous batching, and its OpenAI-compatible server. Ollama: OpenAI compatibility.
• NVIDIA: GeForce RTX 5090 (32 GB GDDR7) and RTX PRO 6000 Blackwell Server Edition (96 GB).
• NEOXIS: pricing, from $1.70 per GPU-hour.