Agens Volundr 32B v1: long agent sessions on hardware you own
Agens Volundr 32B v1 is out under Apache-2.0. It is an open model of about 32 billion parameters in which only 18 of 72 layers keep a KV cache, built so long agent sessions can stay on machines you own.
Key points
- 54 KDA layers with a fixed-size state, 17 BCSA layers and 1 dense layer: only 18 of 72 keep a KV cache
- 262K-token context window; retrieval validated to 107K tokens
- Repetition loops in our SWE-bench agent sessions: 36 of 50 in the Preview, 0 of 50 in v1
- BF16, INT4, GGUF and a 16 GB build for Agens Engine, on Hugging Face and ModelScope
The problem we wanted to solve
Agent sessions get long: tool output, files and logs pile up quickly. In a conventional model every attention layer keeps a KV cache that grows with every token, so at long context the cache, not the weights, decides what fits on your hardware. We wanted long agent sessions to stay on machines a company can own.
What's inside
54 of the 72 layers use KDA, a linear attention that folds the conversation into a fixed-size state. 17 layers use BCSA, our compressed sparse attention: an exact window over the latest 4,096 tokens, with older context pooled 4:1 into blocks and a learned indexer that picks the 512 most relevant blocks. One dense layer keeps a full view. Engram adds a hashed n-gram memory: a 2-gram and a 3-gram lookup table at each of layers 2 and 22, four tables in all, and mHC passes information between layers along four residual streams instead of one.
What v1 fixes
The Preview looped in 36 of 50 SWE-bench agent sessions; v1 loops in none of them (BF16 and INT4). Thinking is now bounded: a median of 120 tokens per step, the longest 12K, none at or over 16K. Follow-up turns are 2.9× faster with session continuation (Blockway sglang build, INT4, one 48 GB GPU, greedy replay of 20 agent transcripts).
On our SWE-bench Verified mini set v1 solves 27 of 50 (Preview: 22) with the median task time down from 124 to 82 minutes, and τ²-bench goes from 89 to 92 of 120. LiveCodeBench v6 rises from 63.4 to 67.9 and AIME 2025 from 76.7 to 80.0. Not every score went up: the Preview is higher on HumanEval, MATH-500, GPQA Diamond and IFEval, and the model card has the full comparison.
Builds and how to run them
BF16 runs on two 48 GB GPUs or one 141 GB GPU. INT4 (24.6 GB) runs on one 48 GB GPU with our sglang build, and the model PR is open upstream in SGLang. The 16 GB build (2.82 bits per weight) loads in Agens Engine, our open-source inference engine, within a 16 GiB GPU budget. GGUF builds (BF16, Q8_0, Q4_K_M, text only) need the volundr branch of our llama.cpp fork. Weights are on Hugging Face and on ModelScope (BlockwayLink).
Where it comes from
Volundr is derived from Qwen3.8-27B (Apache-2.0): we transplanted a new 72-layer architecture (KDA, BCSA, Engram, mHC) onto its weights, then ran continued pre-training and agent post-training. Every language-stack tensor was retrained. The full lineage is on the model card.
Known limitations
The 16 GB build can loop or answer wrongly with thinking on and a long document (32K+ tokens) in the prompt; use thinking off or the INT4 build for long documents. Long-document question answering with thinking on can also degenerate on BF16 and INT4, mostly at medium reasoning effort, and we are working on it. Retrieval is validated to 107K tokens; the full 262K window has not been re-validated on this checkpoint.
We are a small team and there is a lot still to improve. If you try it, tell us where it breaks.