This is part of our series on a year of Bittensor experience, leading up to our anniversary at the 13th of July. This will be an exciting day, as we will not only change the font of this website, but also announce the relaunch of our subnet on a totally different subject than LLM training. After discussing miner archetypes and incentive landscapes, validator opacity, logging and data, and the singular validator, we now take a look at known validator concepts, so that we can then discuss a new concept for validation. Please let us know what you think in our Discord channel!
We haven’t looked at all 128 subnets in detail, so this might be not entirely new, but we think we have a new concept to reward miners, that solves a lot of issues we’ve described in this series. Before we go into details let’s do a recap on well-known scoring mechanisms.
For LLM training there is the concept where miners publish a model, where the model is scored against other models, and the best model gets the highest reward. Miners all compete by trying to achieve the same goal, and their relative score (in LLM terms: their loss value on a particular dataset) is the thing they optimize for. The game is ongoing; there is no end to it other than some change of competition parameters. With decaying advantage factors there is a time dimension added to the quality on which submissions are scored.
Prediction subnets fall into the same broad category: multiple miners trying to solve the same puzzle and relative quality of work that can be assessed at some point in time.
Other subnets have a short term goal and time-bounded challenges, where every epoch some work needs to be delivered, and the miner scoring the highest score on that work, wins the epoch. If the globally optimal solution can be found within the epoch, this evolves into a contest where quality of miners is equal, and speed is rewarded – the first one to deliver, wins. Otherwise the challenge is to get the best score within the time limit of the epoch.
Yet other subnets, that are more aimed at selling a marketable service, or digital commodity, have validators that send challenges to miners, who solve them, and send the results back. The challenges might be real end-user requests (e.g. image/video tagging or enhancement), or synthetic requests that are mixed in, just to assess miner quality. The miner is rewarded for doing work, if its quality meets some threshold, often with a (small or large) quality factor on top of it, so that miners are incentivized to deliver the highest quality they can – this will earn them more and cost them less in terms of deregistrations.
The next article will elaborate on our new concept.
0 Comments