Pretraining a model used to mean a datacenter and a nine-figure budget.
About SN64 Chutes
Chutes@chutes_aiPretraining a model used to mean a datacenter and a nine-figure budget. Jon did it on rented RTX 5090s over the open internet for $11.27 per billion tokens. The whole 8B run came to about $6,500. Retail hourly GPU prices, 574B tokens, 52 hours. The syncers were four CPU VMs with no GPU in them, and there was no object store anywhere in the system. If a model costs about what a used car costs, a lot more people get to own one. That's the point of Parallax.
SN64 Chutes at a glance
SN64, Chutes (α token ش), is one of the independent markets on the Bittensor network, each with its own α token and its own price. Its α currently trades at 0.067908 τ (≈ $18.88), against 200,240 τ of pool liquidity. On-chain, the best-delegated validator on it yields about 33.76% APR (simple) and 40.15% APY (compounded), emissions add 0.008963 τ to the pool each block, epochs run every 360 blocks. It was registered at block #4,531,295.
More from SN64 Chutes
Taosis indexes announcements, articles, posts and releases about Bittensor subnets from their own channels and keeps a record of each. The text above is the source’s; the figures are read from the chain by Taosis.