Fitting a 35B model into 16 GB, then checking it gets things right

The setup

A desktop PC running an AI agent entirely offline.

Web search, GitHub, Cloudflare and browser control come from Goose's extensions and MCP servers, not from the model. The model's job is to call those tools reliably and get the details right. It also needs vision for screen-based computer control.

Starting point: three stacked failures

Goose answered every message with null result from llama cpp. Each problem was hidden behind the one before it.

Read the logs before changing settings. Each error message named its own cause.

Why mixture-of-experts models fit this card

An MoE model stores all its weights but only computes with a few of them per token.

Per gigabyte, an MoE model is somewhat less capable than a dense one. But on a card the dense model doesn't fit, MoE wins clearly. The dense model was dropped.

Speed tuning

Most of the gain came from where the model's pieces sit in memory, not from exotic settings.

The big one: keep every layer on the GPU

Raising the context to 128K looked free on a 10K-token prompt. A 79K-token prompt showed the real cost: Qwen3.6's generation fell from 29 to 5.5 tok/s. The real launch arguments explained why. LM Studio's automatic placement put 22 whole layers on the GPU and 18 on the CPU, attention included. So the CPU was doing attention over the entire chat.

The fix is the standard one for MoE models: every layer on the GPU, with only some layers' experts kept in system RAM (llama.cpp's --n-cpu-moe, LM Studio's "MoE experts on CPU"). Attention stays on the GPU, and only the cheap, rarely used expert weights wait on RAM.

Every step of every sweep

Each sweep step changed one setting, reloaded the model and timed a ~10K-token prompt and a ~79K-token prompt. The highlighted row is the setting the picker chose: fastest long-chat generation with VRAM and pinned RAM both at or under 14 GB.

Other settings that mattered

• mmap off: 61–99% faster prompt reading on every model. With mmap on, RAM-resident weights are pages of the model file. With it off they're ordinary memory, which the GPU appears to read much faster. The speedup is measured; the explanation is likely but not proven.
• Physical batch (ubatch) 2048: about 2× faster prompt reading than the default 512, for 0.4 GB more VRAM.
• Q8 K/V cache: saves 1.2 GB of VRAM at roughly 3% speed, which buys headroom for the desktop.
• CPU threads, flash attention: no measurable difference.

Two memory limits, not one

VRAM is the obvious limit. The second is GTT: system RAM the GPU driver pins for the GPU, capped at about 15.6 GB here. Experts kept in RAM are pinned in GTT, so moving more of them off the GPU trades one limit for the other. With 80% of experts in RAM, GTT filled, the load failed and the kernel logged 2,832 Not enough memory errors in one burst.

Vulkan beat ROCm on every model

LM Studio's ROCm runtime was slower and used about 5 GB more VRAM than estimated, so far less of each model fit:

Every ROCm run also logged thousands of init_user_pages: Failed to get user pages: -1 errors. ROCm was asking for write access to the read-only memory-mapped model file. Turning mmap off removed the errors but didn't change the speed. The server log also showed missing hipBLASLt kernels for this GPU (gfx1201) in LM Studio's ROCm bundle. That's LM Studio's to fix.

GLM-4.7-Flash: a runtime limit

GLM's attention cache was far larger than LM Studio estimated. Its long-prompt reading stuck near 110 tok/s at every placement and batch size, 16× slower than Gemma. That pattern points to the runtime, not the settings: LM Studio's Vulkan build likely lacks a fast path for GLM's attention type. That's inferred from the measurements, not confirmed.

Quality: does it get the job right?

Speed says nothing about accuracy. Each tuned model ran real Goose tasks headless, scored automatically and then checked by hand.

Six easy tasks barely separated the models (Qwen3.6 12/12, Gemma 11/12). So a harder set followed, with traps, conflicting sources and hidden checks. Each model ran each task twice, with at most 40 tool steps and 25 minutes per run.

What the failures were

• H6 measures step efficiency as much as accuracy. Only Qwen3.6 finished within 40 steps (28 and 36). Gemma reused stale page references. Flint saved every page snapshot to a file and read it back, two steps per look; it had flagged 3 of the 4 right orders when it ran out. In daily use Goose would simply ask to continue.
• Gemma's H5 failure was a real bug: an inverted condition (nextIsYearly = !isCurrentlyMonthly) meant the toggle could never switch. It rewrote the file 12 times instead of debugging, and timed out.
• GLM's H3 error was consistent: both runs totalled 30 full $300 payments ($9,000) instead of $8,815.01, missing the smaller final payment.

Temperature matters

Qwen3.6 runs at its publisher's precise-coding temperature of 0.6, while Gemma ran at Google's general default of 1.0. Rerun at 0.6, Temperature fixed Gemma's coding reliability but not its step efficiency.

The scorers needed checking too

Before any model ran, every scorer was tested against a correct answer and a deliberately wrong one. That caught two wrong expected values in the hidden tests, which would have failed a perfect model. After the runs, three correct answers turned out to be marked down: two briefs that noted a handover in the owner column, and a loan answer that said "cheapest" instead of "cheaper". They were re-scored.

Reading every brief and loan explanation by hand also found slips the scorers passed. Qwen3.6 had two: a brief said someone was "out through mid-November" when the email gave no return date, and one loan breakdown didn't add up despite correct totals. Gemma and Flint had none.

Read the actual answer before trusting either a pass or a fail.

Final setup

Qwen3.6 35B A3B on Vulkan: mmap off, 128K context, every layer on the GPU, 60% of experts in system RAM.

The saved LM Studio settings file for Qwen3.6 (~/.lmstudio/.internal/user-concrete-model-default-config/qwen/qwen3.6-35b-a3b.json), load section:

{ "key": "llm.load.contextLength",                  "value": 131072 },
{ "key": "llm.load.numParallelSessions",            "value": 1 },
{ "key": "llm.load.llama.tryMmap",                  "value": false },
{ "key": "llm.load.llama.keepModelInMemory",        "value": false },
{ "key": "llm.load.llama.acceleration.offloadRatio","value": 1 },
{ "key": "llm.load.numCpuExpertLayersRatio",        "value": 0.6 },
{ "key": "llm.load.llama.kCacheQuantizationType",   "value": { "checked": true, "value": "q8_0" } },
{ "key": "llm.load.llama.vCacheQuantizationType",   "value": { "checked": true, "value": "q8_0" } },
{ "key": "llm.load.llama.physicalBatchSize",        "value": 2048 }

The other models

• Recursive Flint: as accurate as Qwen3.6 on everything except step efficiency, and the most careful with details, but about 2.7× slower per task. A good second opinion for code when time doesn't matter.
• Gemma 4 26B A4B QAT: the fastest, and clean on details. Set its temperature to 0.6 for coding and agent work.
• GLM-4.7-Flash: not recommended on this machine. It's slow at every setting and got a loan total wrong twice.

Lessons

Things that cost time, so you don't have to repeat them.

1. Read the server's real launch arguments. /proc/<pid>/cmdline showed settings the UI implied were applied but weren't.
2. Benchmark long prompts. 128K context looked free at 10K tokens and cost 80% of generation speed at 79K.
3. Never run two model servers on a GPU that drives your screen. A leftover ROCm server kept 13.4 GB and the next load crashed GNOME.
4. Watch GTT as well as VRAM. Moving experts to RAM trades one limit for the other.
5. Block suspend during long GPU jobs. "Performance" power mode doesn't stop the idle timer. Use systemd-inhibit --what=sleep:idle.
6. Journal every step. A START with no END marks exactly where a crash happened.
7. Use the publisher's sampling settings. Gemma at temperature 1.0 made a logic bug it didn't make at 0.6.
8. Test the tests. Scorers had wrong expected values and false negatives; both were caught only by checking them.
9. Small things: lms chat prints stats to stderr; a 300 KB prompt is over Linux's 128 KB argument limit, so pipe it on stdin; Playwright MCP blocks file://, so serve test pages over local HTTP.

How it was run

A small Python harness, standard library only, running unattended as systemd user services.

• Speed sweeps write each setting into LM Studio's per-model settings file, reload, verify the launch arguments, then time two prompts (two measured runs after a warm-up). A random string at the start of each prompt stops llama.cpp reusing its cache.
• Crash safety: kill leftover servers and require VRAM under 1.5 GB before each step. Watchdogs kill the server above 15.0 GB VRAM or 14.8 GB GTT. The run stops if the kernel logs GPU memory errors, and the original settings file is restored if the run is killed. The watchdogs stopped 15 steps before they could take the desktop down: 7 in the ROCm tests and 8 in the tuning sweeps.
• Settings picker: fastest long-chat generation with ≥ 2 GB VRAM headroom. It applies the publisher's sampling preset and confirms a flag-free load reproduces the tested arguments.
• Evaluation: goose run per task and model, with session exports kept for hand review. Scorers check code with hidden tests, numbers against computed truth, and pages in a real browser.

Everything on this page is generated from the run journals: data/results.json. The harness lives in a private repository.

Log

Results so far, newest first.

• Oct 25, 2026
  Hard tasks complete for all four tuned models; Gemma retested at temperature 0.6. Qwen3.6 confirmed as the daily model.
• Oct 24, 2026
  128K tuning sweeps for Qwen3.6, Gemma 4, Flint and GLM-4.7-Flash. Easy-task evaluation. Qwen3.8 dropped.
• Oct 24, 2026
  ROCm vs Vulkan comparison with a crash-safe harness; mmap-off discovery.
• Oct 24, 2026
  Fixed Goose: moved to LM Studio, new session, larger context.

Next up: rerun H6 with an 80-step limit, and more runs per task so small differences become meaningful.

Sources

• LM Studio model catalog: sizes, capabilities and MoE parameter counts
• Unsloth run guides: Qwen3.6, Gemma 4, Flint, GLM-4.7-Flash: sampling, memory needs, known issues
• Recursive Flint model card, LM Studio GLM-4.7 page
• All benchmark and evaluation figures were measured on the hardware above, Oct 24–25, 2026.