BaseCompute releases Qwen3.8-27B quants with lower KL divergence than same-size Unsloth builds

The three Apache-2.0 builds target local inference and report lower KL divergence than same-size Unsloth quantizations, pending independent replication.

2 min read

Computer hardware components and cables inside a case

Computer hardware components and cables inside a case

BaseCompute has released BaseQuant V1, three quantized builds of Qwen3.8-27B aimed at local inference through llama.cpp and the company’s BaseRT runtime. The Small, Medium and Large variants occupy 10.93GB, 16.41GB and 20.82GB respectively, giving users a direct quality-versus-memory choice for consumer workstations and compact AI systems.

The headline result is about distribution fidelity, not task accuracy. Across six public prompt sets, BaseCompute says its method produced 15% lower KL divergence than same-size Unsloth builds at the 3- and 4-bit tiers; the large tier improved by 3%. On a held-out set, the reported gap ranged from 17% to 34%. Lower KL divergence means the quantized model’s token probabilities stayed closer to the source model. It does not by itself prove better answers, reasoning or application success.

The comparison was run by the maintainer, and no independent reproduction accompanied the launch. Results can also depend on calibration data, backend kernels, prompt distribution and the exact competing quantization settings. Users should therefore benchmark the builds on their own workloads before treating the reported ranking as general.

The model card and weights are Apache-2.0 licensed. The supporting BaseRT repository, maintained by BaseCompute, supplies a local runtime, CLI, OpenAI-compatible API and bindings for the project’s .base format. It had 237 total GitHub stars, 19 forks and 52 commits when checked October 22, 2026. That is a point-in-time popularity count, not evidence of measured recent growth; no earlier star snapshot was available.

Sources: BaseCompute announcement · Model card · BaseRT repository

Qwen3.8-27BLocal AIllama.cppQuantizationBaseCompute