AI Crawler robots.txt Study 2026: Top 10k Sites
About SN22 Desearch
We parsed 6,480 robots.txt files from the Tranco top 10,000. 18.6% block AI training crawlers site-wide, 2.4% block the main search crawler. Data, method, CSV.
Read the article at SN22 Desearch news page ↗
SN22 Desearch at a glance
SN22, Desearch (α token χ), is one of the independent markets on the Bittensor network, each with its own α token and its own price. Its α currently trades at 0.003006 τ (≈ $0.84), against 7,525 τ of pool liquidity. On-chain, the best-delegated validator on it yields about 55.66% APR (simple) and 74.48% APY (compounded), emissions add 0.000001 τ to the pool each block, epochs run every 360 blocks. It was registered at block #2,009,702.
More from SN22 Desearch
Taosis indexes announcements, articles, posts and releases about Bittensor subnets from their own channels and keeps a record of each. The text above is the source’s; the figures are read from the chain by Taosis.