Mentor Palokaj: AI Model Compression & Quantization | TPN, Bittensor Subnet 65 | Ep. 106

Mentor Palokaj, one of the technical leads at TPN (True Performance Network, Bittensor Subnet 65), explains how far you can shrink a large language model before it stops being intelligent. He walks through quantization from 16-bit weights down to ternary, why each halving costs more than the last, and why the outlier values that get rounded away are where much of the intelligence lives. The conversation also covers why nobody yet agrees on how to measure an LLM's intelligence, how TPN plans to score miners on KL divergence as well as benchmarks, and the team's route from a TAO bridge to a VPN subnet to model compression. *In this episode, you'll learn* - Why TPN walked away from a profitable VPN subnet in May 2026 and rebuilt around model compression - How quantization actually works, and why 16 bits to 8 costs a few percent while 4 to 2 costs thirty or forty - Quantization-aware distillation: using the original model as the teacher for its own compressed student - Why public benchmarks get overfitted, and how Fluid Bench samples a new test on every run - KL divergence as the planned structural check on how far a compressed model's outputs drift from its parent - Ternary weights, Hadamard rotations, and the outlier problem those rotations are meant to solve - What the TPN workbench does when you hand it a Hugging Face model and a hardware ceiling - Where compression actually matters: robots and battery-powered devices with no internet connection *Chapters* 0:00 - Highlights 1:01 - From a TAO Bridge to a VPN Subnet 4:28 - The 440 Upgrade and a Six-Hour Rethink 6:42 - Profitable, but Not Parabolic 9:42 - What TPN Actually Does to a Model 10:03 - Quantization, Explained With a Ruler 11:36 - Quantization-Aware Distillation 12:25 - Nobody Knows How to Measure an LLM 13:15 - Anyone Who Can Cheat Will Cheat 13:49 - Fluid Bench, a Benchmark That Moves 15:13 - The TPN Workbench and Optimize 16:57 - An LLM as a Plinko Machine 20:02 - Toward Models That Are Experts 21:12 - Dee

Watch the video on Ventura Labs (YouTube) ↗

Taosis indexes announcements, articles, posts and releases about Bittensor subnets from their own channels and keeps a record of each. The text above is the source’s; the figures are read from the chain by Taosis.