Research & Outputs

Papers from the lab

Working papers and technical audits from Tawakkul Labs, with the open source products that implement them.

4 published4 papers340 lines of documentationLast updated 19 Sept 2026

Papers

The Asymmetric Turn: Inside DeepSeek V4.1 Flash and Why It Wins

Tawakkul Labs · 19 Sept 2026

DeepSeek V4.1 Flash replaces the uniform decoder with a 40 layer Causal Encoder Decoder: 20 encoder layers read the prompt with 8B active parameters and 20 decoder layers write with 16B. Alongside a compressed sparse attention scheme and a much smaller KV cache, it changes what long context agent work costs. This paper examines the architecture, the reported benchmarks, the API economics, and the honest caveats, and asks what a vendor reported win on agent tasks means for teams building on African hardware budgets.

ai ml architecture deepseek moe inference 
ReadPDF

MoE at the Edge: Offloading and Small Mixture of Experts Models for Constrained Hardware

Tawakkul Labs · 17 Sept 2026

A technical survey of two parallel lines of work: expert offloading, which moves mixture of experts weights between memory tiers so large models run on constrained devices, and small mixture of experts models, which compress the architecture itself to sub billion scale. It covers the routing consistency caveat, predictive offloading, adaptive precision, expert merging, the on device MoE scaling law, and the honest open question of whether MoE actually helps on consumer hardware, all examined for what it means on African hardware budgets.

ai ml moe edge offloading slm survey 
ReadPDF