Parallax: Building the Decentralized "Linux of AI" With No Off Switch | Jon Durbin at Exploit

About SN64 Chutes

Jon Durbin closes Day 1 of Exploit with a case for open, resilient and ubiquitous artificial intelligence. Starting with his personal motivation for building Chutes and Parallax, Jon examines the physical and economic limits of centralized AI infrastructure: data-center expansion, power constraints, consolidation of the open-source stack, privacy, dependence on open-weight models and the difficulty of decentralized training. He then introduces Parallax, an approach to decentralized model training designed around systems with no single off switch. The architecture combines sparse mixture-of-experts models, ternary weights, low-bandwidth distributed training and commodity hardware with the goal of making inference extremely fast, cheap and widely available. The thesis: reduce the cost of intelligence toward the cost of the energy required to produce it, make models reproducible and auditable, and build what Jon describes as the “Linux of AI.” CHAPTERS 00:00 — Why open source matters 03:10 — Llama, Self-Instruct and open-weight AI 07:40 — The incentive: permissionless access to intelligence 08:47 — The physical limits of AI infrastructure 11:14 — Concentration of the AI stack 14:03 — Encrypted inference and privacy 16:11 — Dependence on open-weight models 17:00 — Alignment through visibility and open source 19:20 — Why decentralized training is difficult 21:16 — Training with no off switch 22:30 — Introducing Parallax 22:53 — Large Language Monkeys and repeated sampling 24:13 — Ubiquitous, cheap inference 24:43 — Sparse ternary models 31:03 — 60 tokens/sec on a phone 31:59 — A decentralized training architecture 36:37 — Results from the Epsilon training run 39:56 — Two days vs. one month 40:11 — Commoditizing model training 40:30 — $11.27 per billion training tokens 41:52 — Almost 18,000 tokens/sec on a 5090 44:25 — Training an 80B model over Starlink 45:50 — Eliminating trust 46:59 — Building the Linux of AI 47:31 — Q&A

Watch the video on Opentensor YouTube ↗

Taosis indexes announcements, articles, posts and releases about Bittensor subnets from their own channels and keeps a record of each. The text above is the source’s; the figures are read from the chain by Taosis.