News·SN15 ORO GitHub·Sep 18, 2026, 17:44 UTC
v2.0.24: validator: hard-fail generated runs at >=30% harness-failure rate (#322)
About SN15 ORO
Prior behavior (post oro#320) only hard-failed when EVERY episode was a harness failure. Today's Mistral outage exposed the flip side: OR's shared upstream pool for mistralai/mistral-small-2603 (the simulator model) was 429-rate-limited, and 60-80% of episodes came back as environment_error. Runs succeeded with 0.03-0.10 partial scores and counted against miners for infra we own. Introduce _INFRA_FAILURE_THRESHOLD = 0.30. When environment_error + verifier_error >= 30% of the roster, hard-fail as generated evaluation infrastructure failure — the completed sample isn't a representative slice of miner performance. The Backend classifier already shields such failures from the auto-discard counter and the work item re-opens so the run re-queues when infra recovers. Below the threshold, harness episodes still score at zero reward alongside agent_error — same as after oro#320. That path is exercised by the existing test_below_threshold_harness_failures_score_normally parametrized across [2, 3, 8] failed of 30 (max 26.7%). Backend needs the mirror change in generated_scoring.py (separate PR) since it re-computes the score for the mismatch check on complete_run. Ship this PR first — validator will start hard-failing locally which Backend already accepts as FAILED under the current 100% rule. Co-authored-by: Claude Opus 4.7 [email protected]
Read the github release at SN15 ORO GitHub ↗
SN15 ORO at a glance
SN15, ORO (α token ο), is one of the independent markets on the Bittensor network, each with its own α token and its own price. Its α currently trades at 0.027481 τ (≈ $6.91), against 14,072 τ of pool liquidity. On-chain, the best-delegated validator on it yields about 54.6% APR, emissions add 0.008811 τ to the pool each block, epochs run every 360 blocks. It was registered at block #7,366,897.
More from SN15 ORO
Taosis indexes announcements, articles and releases about Bittensor subnets from their own channels and keeps a record of each. The text above is the source’s; the figures are read from the chain by Taosis.