Blockway

OPEN MODEL · v1 · OUT NOW

Agens Volundr

Long context, without the memory bill. Only 18 of its 72 layers keep a KV cache. An open model of about 32 billion parameters, built for tool use, coding and long agent sessions, on machines a company can actually own.

  • ~32B · dense
  • 72 layers
  • 18 keep a KV cache
  • 262K context window
  • Apache-2.0
  • BF16 · INT4 · GGUF · 16 GB build
sparse attention · KV cache (17)dense attention · KV cache (1)linear attention · fixed-size state (54)

262K-token context window; retrieval validated to 107K tokens (needle-in-a-haystack 100/100). The full 262K window has not been re-validated on this checkpoint.

NEW IN v1

What v1 fixes

Repetition loops

36 of 50 SWE-bench agent sessions in the Preview, 0 of 50 in v1 (BF16 / INT4)

Runaway thinking

median 120 tokens of thinking per step, longest 12K, no step at or over 16K

Follow-up turns

2.9× faster with session continuation (Blockway sglang build, INT4, one 48 GB GPU, greedy replay of 20 agent transcripts)

SWE-bench Verified mini

22 → 27 of 50; median task time 124 → 82 minutes

τ²-bench

89 → 92 of 120

LiveCodeBench v6 · AIME 2025

63.4 → 67.9 · 76.7 → 80.0

Not every score went up: the Preview is higher on HumanEval, MATH-500, GPQA Diamond and IFEval. The full comparison is on the model card.

THE PROBLEM

Long context is a memory problem before it's a compute problem

In a conventional model, every attention layer keeps a KV cache: a copy of what it has seen, which grows with every token. Stack seventy-two of them and, at long context, the cache, not the weights, decides what fits on your machines.

Volundr changes the shape of that curve. 54 layers hold a fixed-size state. Only 18 keep a cache.

context length →cache memory →every layer cachesVolundr: 18 of 72 cache
Schematic, not to scale.

FIVE IDEAS, ONE MODEL

What's inside, and why it matters

KDA · 54 LAYERS

A memory that doesn't grow

Kimi Delta Attention is linear attention: instead of keeping every past token, each layer folds the conversation into a recurrent state of fixed size.

Why it matters: Three quarters of the model carry the whole session forward in the same amount of memory on turn 5 as on turn 500. That's what makes long agent sessions practical on hardware you can buy.

  • fixed-size state
  • no KV cache

TOKENS IN

STATEfixed size

BCSA · 17 LAYERS

Sharp up close, wide at a distance

Blockway Compressed-Sparse Attention reads the latest 4,096 tokens exactly. Everything older is pooled 4:1 into blocks, and a learned indexer picks the 512 most relevant blocks to read.

Why it matters: Every token reads a fixed budget: the recent past in full detail, plus the parts of the distant past that matter. The model can look back across the whole window without paying to look at all of it.

  • 4,096-token exact window
  • far field pooled 4:1
  • top 512 blocks

FAR FIELD · POOLED 4:1 INTO BLOCKS

indexer picks the top 512 blocks

EXACT WINDOW

last 4,096 tokens, every one

DENSE · 1 LAYER

One full view

A single layer of classic full attention closes the stack: here, every token can see every other token.

Why it matters: Fifty-four layers stay lean and seventeen stay selective, and one keeps the complete picture.

  • full attention
  • layer 72

ENGRAM · LAYERS 2 AND 22

A memory of common patterns

Engram is a hashed n-gram memory: a 2-gram and a 3-gram lookup table of 1M rows × 512 at each of layers 2 and 22, four tables in all.

Why it matters: Looking a pattern up is cheaper than recomputing it, and each token reads only a few rows. That's why Agens Engine and the GGUF build can keep the tables in system RAM and free GPU memory.

  • hashed 2-gram and 3-gram memory
  • layers 2 and 22
  • can live in system RAM

GPU MEMORY

weights and caches

SYSTEM RAM

Engram tables, in Agens Engine and the GGUF build

mHC · 4 STREAMS

Four lanes instead of one

A conventional transformer passes information between layers along one residual stream. Volundr uses four in parallel.

Why it matters: More room to carry information through 72 layers of depth, so the model doesn't have to squeeze everything into a single path.

  • four residual streams

DENSE, NOT MIXTURE-OF-EXPERTS

What you see is what you run

About 32 billion parameters, and every layer runs on every token. No hidden expert pool to keep resident, so the footprint is exactly what it says on the label, and capacity planning for your own machines stays simple.

Where it comes from

Derived from Qwen3.8-27B (Apache-2.0): a new 72-layer architecture (KDA, BCSA, Engram, mHC) transplanted onto its weights, then continued pre-training and agent post-training. Every language-stack tensor was retrained.

AVAILABLE NOW

What ships

BF16

full precision

Runs on

two 48 GB GPUs, or one 141 GB GPU

Blockway/Agens-Volundr-32B-v1

INT4

24.6 GB

Runs on

one 48 GB GPU, Blockway sglang build

Blockway/Agens-Volundr-32B-v1-INT4

16 GB build

2.82 bits per weight

Runs on

a 16 GiB GPU budget, with Agens Engine

Blockway/Agens-Volundr-32B-v1-Engine-16GB

GGUF

BF16 · Q8_0 · Q4_K_M, text only

Runs on

Blockway's llama.cpp fork (volundr branch)

Blockway/Agens-Volundr-32B-v1-GGUF

Apache-2.0 open weights

on Hugging Face and ModelScope (BlockwayLink), for commercial use

Agens Engine

our open-source (MIT) inference engine, built for Volundr: github.com/BlockWayz/agens-engine

sglang

Blockway's build (github.com/BlockWayz/agens-sglang); the model PR is open upstream (sgl-project/sglang#45093)

Speculative decoding

a DFlash2 drafter for one or two concurrent streams, and a 0.22 GB MTP draft head for Agens Engine

16 GB BUILD + AGENS ENGINE

The whole context window, in a 16 GiB budget

The 16 GB build loads in Agens Engine, which keeps compute on the GPU and moves lookup tables and long-context KV to system RAM.

98–150

tok/s, greedy, short context, with drafter

67–93

tok/s, greedy, after a 219K-token document

239K

tokens in one conversation, peak 15.5 GiB

Measured on one 48 GB GPU (compute capability 8.9) held to a 16 GiB budget by the engine's hard cap, single conversation. Not measured on a physical 16 GB card.

More about Agens Engine →

EVALUATION

Benchmarks

Every model run by Blockway on one harness, with the same settings. v1 is ahead of Qwen3.8-27B on competitive coding and competition maths; Qwen3.8-27B still leads on agentic coding, multi-turn tool use, instruction following and GPQA.

BenchmarkAgens Volundr 32B v1Volundr PreviewQwen3.8-27B
LiveCodeBench v62025-02 to 2025-05, best of 2 runs67.9▲ +8.463.459.5
HumanEvalgreedy completion78.7▲ +1.381.777.4
SWE-bench Verified mini50 tasks, Codeway harness54.044.058.0
BFCL v4 irrelevancenon-live, official harness82.1▲ +0.480.881.7
τ²-benchairline, retail, telecom76.774.279.2
BFCL v4 parallelnon-live, official harness94.092.094.0
AIME 2025avg@4, best of 2 runs80.0▲ +6.776.773.3
MATH-50096.8▲ +0.298.296.6
IFEvalprompt-strict87.287.688.4
IFBenchprompt-strict64.366.369.3
GPQA Diamondbest of 4 runs, 6K-token thinking budget80.384.385.4
MMLU-Pro1,400-question subset80.079.980.4
CMMLU2,010-question subset85.986.586.4

▲ v1 ahead of Qwen3.8-27B, in points. Best score in each row in bold.

Thinking on, temperature 0.6 and the same output-token limit for every model unless noted; answers cut off by the limit count as wrong. GPQA Diamond: 6K-token thinking budget, then the answer is forced, best of 4 runs for every model. τ²-bench: agent at temperature 0, Qwen3.8-27B as the user simulator for every model; the 13 retail tasks that need an LLM judge are scored as failed for every model. BFCL v4: non-live categories, official harness at temperature 0.001. SWE-bench Verified mini: a 50-task subset run through Codeway, Blockway's coding-agent harness, with each model's default sampling and identical limits. Qwen3.8-27B was run by us on the same harness; these are not its publisher's figures. Run by run (best shown): GPQA Diamond v1 79.3 / 78.8 / 80.3 / 78.8, Preview 79.3 / 79.8 / 83.3 / 84.3, Qwen3.8-27B 83.8 / 84.8 / 82.3 / 85.4; AIME 2025 v1 78.3 / 80.0, Preview 76.7 / 72.5, Qwen3.8-27B 70.0 / 73.3; LiveCodeBench v1 66.4 / 67.9, Preview 63.4 / 63.4, Qwen3.8-27B 59.5 / 58.8.

Smaller builds

The INT4 and 16 GB builds against BF16 v1. The 16 GB build trades some accuracy for size, mostly on hard reasoning and code.

BenchmarkBF16INT416 GBSetting
AIME 202580.0—80.0avg@4, best of 2 runs (bf16 mean 79.2); 16 GB: best of 4 (mean 76.7)
MATH-50096.8—96.4
GPQA Diamond80.381.373.7best of 4 runs, 6K-token thinking budget then a forced answer; INT4 mean 78.9
HumanEval78.773.873.2greedy completion
IFEval87.2—86.5prompt-strict
BFCL v4 parallel94.0—94.0non-live, official harness
BFCL v4 irrelevance82.1—81.716 GB: provisional, 8 of 240 hit a 32K client cap

16 GB build vs BF16: mean KL 0.117, top-1 agreement 90.4%. Full tables, settings and known limitations are on the model cards.

KNOWN LIMITATIONS

Good to know

  • 262K-token context window; retrieval validated to 107K tokens (needle-in-a-haystack 100/100). The full 262K window has not been re-validated on this checkpoint.
  • 16 GB build: with thinking on and a long document (32K+ tokens) in the prompt, it can loop or answer wrongly. Use thinking off, or the INT4 build, for long documents. A revised 16 GB build is planned.
  • BF16 and INT4: long-document question answering with thinking on can also degenerate, mostly at medium reasoning effort. Thinking off always answered in our tests. We're working on it.
  • The DFlash2 drafter is for one or two concurrent streams; many-user serving uses the configurations without it.
  • GGUF needs the volundr branch of Blockway's llama.cpp fork; stock llama.cpp doesn't load it until support is merged upstream.

Models we train. Hardware you own.

Agens Volundr 32B v1 is out. Download the weights, run them with Agens Engine, and tell us where it breaks.