v2.0.24: validator: hard-fail generated runs at >=30% harness-failure rate (#322)
About SN15 ORO
Prior behavior (post oro#320) only hard-failed when EVERY episode was a harness failure. Today's Mistral outage exposed the flip side: OR's shared upstream pool for mistralai/mistral-small-2603 (the simulator model) was 429-rate-limited, and 60-80% of episodes came back as environment_error. Runs succeeded with 0.03-0.10 partial scores and counted against miners for infra we own. Introduce _INFRA_FAILURE_THRESHOLD = 0.30. When environment_error + verifier_error >= 30% of the roster, hard-fail as generated evaluation infrastructure failure — the completed sample isn't a representative slice of miner performance. The Backend classifier already shields such failures from the auto-discard counter and the work item re-opens so the run re-queues when infra recovers. Below the threshold, harness episodes still score at zero reward alongside agent_error — same as after oro#320. That path is exercised by the existing test_below_threshold_harness_failures_score_normally parametrized across [2, 3, 8] failed of 30 (max 26.7%). Backend needs the mirror change in generated_scoring.py (separate PR) since it re-computes the score for the mismatch check on complete_run. Ship this PR first — validator will start hard-failing locally which Backend already accepts as FAILED under the current 100% rule. Co-authored-by: Claude Opus 4.7 [email protected]
Read the release notes on GitHub ↗
SN15 ORO at a glance
SN15, ORO (α token ο), is one of the independent markets on the Bittensor network, each with its own α token and its own price. Its α currently trades at 0.017900 τ (≈ $3.99), against 13,275 τ of pool liquidity. On-chain, the best-delegated validator on it yields about 93.34% APR (simple) and 154.29% APY (compounded), emissions add 0.005086 τ to the pool each block, epochs run every 360 blocks. It was registered at block #7,366,897.
Taosis newsletters
Taosis News: the week’s articles from the subnet teams, the press and community newsletters, every Friday. Taosis Weekly: the week’s movers, best yields and chain events, with dollar values, every Monday. Past issues
More from SN15 ORO
Taosis indexes announcements, articles, posts and releases about Bittensor subnets from their own channels and keeps a record of each. The text above is the source’s; the figures are read from the chain by Taosis.