If you run long-context agents and keep bumping into KV-cache VRAM, the useful claim here is concrete: at 256K context the project reports BF16 attention KV at 72 MiB fixed, versus 8,192 MiB for the Qwen3.5-4B baseline — about 99.1% smaller. Total persistent history per sequence in their batch runtime is listed as 97.76 MiB, scaling as O(1) in length. Those are company / report figures, not independent audits.
Throughput claims on one Hopper 80GB GPU, for a 32K-in / 32K-out best-batch workload: 12,124 tok/s output versus 1,889 for Qwen3.5-4B — about 6.42×. Same setup notes 512 concurrent sequences and peak allocated memory around 61.26 GiB. Again: attributed numbers from the EdgeRock materials.
What RMA actually changes
Architecturally the conversion keeps the backbone’s GDN layers, then replaces eight full-attention layers with a parallel of sliding-window attention and RMA. Local window size is 1024; the RMA bank holds 1024 records per KV group. The pitch is that distant facts live in the anchor bank instead of an ever-growing full KV tensor.
Their distant-memory eval is the clearest “does the bank do anything?” number in the package: RMA on 94.1% versus off 0.4%. That gap is why the constant-memory story is interesting beyond a pure throughput chart.
Illustration: O(n) full-attention KV wall versus a fixed RMA bank — company-reported 72 MiB BF16 attention KV at 256K.
Quality after conversion
On AIME 2024–2026 mean pass@1 the report lists 82.9% for EdgeRock-4B, against the strongest evaluated bounded-state baseline they cite — RWKV-7 G1j 13.3B at 18.8%. Versus its own backbone, EdgeRock retains 96.3% of Qwen3.5-4B’s AIME mean. The model is text + image; visual mean is listed at 84.9% (95.3% of the backbone).
Tradeoffs exist. Materials note drops on some suites (including LiveCodeBench / GPQA-style evals) relative to the unmodified Qwen3.5-4B. Treat the conversion as a deliberate memory/throughput trade, not a free upgrade on every benchmark.
Yes for the artifacts: Hugging Face (rockmatrix-ai/EdgeRock) and GitHub are globally reachable. Runtime is Apache-2.0; weights follow the model license on the card. Optimized path wants CUDA; a portable PyTorch CPU/CUDA path is also described. Formats listed: BF16, NF4/INT8/FP8, MLX, GGUF, ONNX, LiteRT previews. Important: GGUF needs their custom RMA backend — it is not a drop-in for stock llama.cpp. MLX / Apple Metal is called out as not tested on the card. Rock Matrix is a Chinese listed company (岩山科技); the open release is still usable worldwide if you accept those runtime caveats.
Where to look
- Weights: huggingface.co/rockmatrix-ai/EdgeRock
- Code: github.com/rockmatrix-ai/EdgeRock
- Report: arXiv 2610.10118 / rockmatrix.ai docs
Honest caveats
- Vendor / report metrics — KV sizes, tok/s, AIME means, and distant-memory scores come from company and paper materials; not third-party audited here.
- Custom runtimes — GGUF without their RMA backend will not match the pitch; do not assume stock llama.cpp or untested MLX Metal.
- Quality tradeoffs — strong AIME retention vs backbone, but some coding/knowledge suites drop; read the card before swapping production agents.
- CUDA-first optimized path — portable PyTorch exists, but the Hopper batch numbers assume the optimized stack.
For builders who already ship long-context agents on 4B-class GPUs, EdgeRock is a rare open conversion that attacks the KV wall directly — with the usual Chinese-lab packaging: HF + GitHub first, and you bring (or build) the right backend.