Guide
Ollama out of memory: diagnose the load failure before changing models
Ollama OOM or a model that loads but crawls? Read the `ollama ps` processor/context split, see why context length changes memory, and choose the smallest fix.
Published 2026-10-19 · Updated 2026-10-19 · Every sourced claim below carries its own link and as-of date.
Quick answer
An explicit out-of-memory or allocation error points to memory pressure from weights, KV cache, runtime overhead, or other memory users. Other load failures can come from unsupported model/backend/driver combinations. If a model loads slowly, inspect PROCESSOR and CONTEXT: CPU placement can be intentional or a split, and is only one possible bottleneck.
An explicit out-of-memory or allocation error points to memory pressure, but a generic load/startup failure can also come from an unsupported model, backend, driver, or GPU configuration. A model that loads but feels slow can reflect CPU placement, GPU contention, a long context, or another system bottleneck. Treat the error text and `ollama ps` as clues, not a complete hardware diagnosis.
1. What kind of failure do you have?
| What you see | Likely cause | First check |
|---|---|---|
| An explicit out-of-memory / allocation error | Weights + KV cache + runtime overhead may exceed the memory available to the process; other apps can use part of the pool | Reduce context first; if the same explicit error remains, check the quant and other memory users |
| A model fails to load without an explicit memory error | Could be a model, runtime/backend, driver, or GPU-compatibility issue rather than capacity | Check the exact error and the runtime's supported hardware/model path before changing quant |
| Model loads, but generation or prompt processing crawls | CPU placement, GPU/CPU splitting, memory contention, long context, or another bottleneck | `ollama ps` → PROCESSOR and CONTEXT; compare with the placement you intended |
| It worked yesterday but fails now | A larger context setting, another loaded model, or a driver/app update | Check CONTEXT in `ollama ps`; compare the settings |
| The request is rejected before any memory math | The runtime/quant/backend combination is gated | Our checker shows the gate reason; this is not an OOM |
2. Read the `ollama ps` columns correctly
Ollama's official CLI reference documents the columns. SIZE is resident size, not the download size or total system RAM. PROCESSOR is the placement signal. Newer versions also print the allocated CONTEXT.
| Column | What it means | Why it matters here |
|---|---|---|
| NAME / ID | The loaded tag and digest | Confirms which artifact is actually running |
| SIZE | Resident model memory | A loaded model can be larger than its file because of KV cache and overhead |
| PROCESSOR | GPU/CPU placement | A mixed split can indicate offload; 100% CPU may be an intentional CPU-only run. Placement alone does not diagnose speed. |
| CONTEXT | Allocated token window | Larger context increases KV-cache memory |
| UNTIL | Idle unload time | Explains why a model disappears after a pause |
Run ollama ps in a terminal, copy everything, and paste it here. The pasted text stays in your browser; analytics receives counts only. We map only exact tags we have verified; an unknown tag is shown as unknown, not guessed.
3. The four levers, in order
- Lower the context. KV-cache memory grows with the token window. If you set 64K for a task that only needs 8K, you are paying memory for tokens you are not using.
- Use a smaller quant. Q5/Q6/Q8 files need more memory than Q4. This is a real quality tradeoff; we do not claim a quant is lossless.
- Use a smaller model. A 14B at Q4 is not the same memory class as a 32B at Q4. Check the exact quant file size on the model page.
- Add memory or remove competition. Close other GPU-heavy apps, avoid counting shared VRAM you do not control, and remember that multi-GPU layer-split pools memory but does not multiply speed.
4. Ollama's own context defaults
Ollama chooses a default context from the available VRAM. Those defaults are not a promise that the model plus your task will fit; they are a starting point. The FAQ still describes a flat 4096 default, while the dedicated context page lists these VRAM-based values — check `ollama ps` for what actually ran.
| VRAM | Ollama's documented default context |
|---|---|
| < 24 GiB | 4K |
| 24–48 GiB | 32K |
| ≥ 48 GiB | 256K |
Ollama's docs also advise using the model's maximum context for best performance and avoiding CPU offload. That is a performance recommendation, not a fit guarantee: on a memory-limited machine, a smaller window may be the only clean fit. Check the allocated CONTEXT and PROCESSOR in `ollama ps`, then choose the tradeoff explicitly.
Set context for a single server session
OLLAMA_CONTEXT_LENGTH=32768 ollama serveUse this command only when starting the server from this shell. If Ollama's desktop app or an OS service already owns the server, configure that running process instead; a new shell variable will not change it. The Ollama FAQ documents platform-specific environment setup. In the desktop app, use the context-length slider in settings, then confirm with `ollama ps`.
5. KV-cache quantization is a separate lever
Ollama can quantize the K/V cache independently of the model weights. Its FAQ documents a global OLLAMA_KV_CACHE_TYPE setting: f16 is the default, q8_0 uses roughly half the KV-cache memory, and q4_0 roughly a quarter, with a quality tradeoff that may be more noticeable at high context. This is not the same thing as choosing a Q4/Q5/Q8 model file.
Ollama documents this path as active when Flash Attention is enabled. It uses Flash Attention automatically where the backend and device support it; OLLAMA_FLASH_ATTENTION=1 forces it on. On an unsupported backend, the K/V setting is not a memory guarantee. Ollama also notes that the quality impact depends on the model and task; high-GQA models may be more sensitive.
| Ollama K/V cache type | Relative KV memory | Documented tradeoff |
|---|---|---|
| f16 (default) | Baseline | High precision; highest memory |
| q8_0 | About 1/2 of f16 | Very small precision loss; usually no noticeable quality impact |
| q4_0 | About 1/4 of f16 | Small-to-medium precision loss; may be more noticeable at higher context |
Set the K/V cache type for an Ollama server session (requires Flash Attention)
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve6. Check the exact setup before you buy or upgrade
Our checker computes the weights, KV cache, activations, and runtime overhead for your exact hardware, model, quant, and context. It shows a fit grade and a speed tier — not a fabricated tokens-per-second number.
- Open the interactive checker with your hardware and model: /can-i-run.
- If you have not chosen a machine, use /stack to build a full setup within a budget.
- For Macs and mini-PCs, remember the pool is the GPU-addressable slice of unified memory, not the advertised total.
- For multi-GPU, the pool adds but the speed does not multiply; the checker labels layer-split setups.
7. Prevention checklist
- Record the quant label and context you actually ran, not just the model name.
- Leave OS/desktop headroom; the model is not the only GPU consumer.
- Re-check after an Ollama update or a model tag change; a registry default can move.
- If a model loads but crawls, fix placement/context before buying more hardware.
- If it fails with an explicit out-of-memory/allocation error, reduce context first — it is the cheapest reversible change. For other load errors, check runtime and hardware compatibility before changing quant.
Common questions
- Does `ollama ps` show total system RAM?
- No. SIZE is the resident model memory Ollama reports, not total RAM. PROCESSOR shows the GPU/CPU placement, and CONTEXT shows the allocated token window in newer versions.
- Will increasing context always make the model fit better?
- No. Context increases KV-cache memory. A larger window can turn a clean fit into a tight fit or an out-of-memory failure. Choose the smallest window that holds your task.
- Can TokenAssemble tell me if my exact setup will OOM?
- It computes a fit verdict from sourced hardware and model data with stated assumptions. It cannot see other apps using your GPU, driver behavior, or a runtime's internal allocation details, so treat the verdict as a preflight check, not a guarantee.
- Does Ollama's KV-cache quantization change the fit verdict?
- Our engine assumes an fp16 KV cache, so the displayed memory requirement does not include a q8_0/q4_0 cache saving. Ollama documents OLLAMA_KV_CACHE_TYPE as a global setting that requires Flash Attention. Changing it can reduce actual KV memory, but we do not present that as a guaranteed fit or a lossless quality change.
Sources checked
- Ollama context-length documentation · as of 2026-10-19
- Ollama CLI reference (`ollama ps`) · as of 2026-10-19
- Ollama FAQ — K/V cache and server environment variables · as of 2026-10-19
- Ollama download page · as of 2026-10-19
- TokenAssemble methodology · as of 2026-10-19
Sources are checked on the stated dates. A vendor page can change after that date; if it contradicts this guide, the vendor page wins. We do not present vendor performance claims as our own measurements.
Related checks and guides
- Check your exact setup
Fit, grade, and context at your real memory budget.
- Ollama runtime guide
Install, context defaults, MLX preview, and caveats.
- Qwen3.8 + Splash on Apple silicon
Official requirements and the engine's data boundary.