This is part of our series on a year of Bittensor experience, leading up to our anniversary at the 13th of July. This will be an exciting day, as we will not only change the font of this website, but also announce the relaunch of our subnet on a totally different subject than LLM training. After discussing miner archetypes and incentive landscapes, validator opacity, logging and data, we are now ready to describe a part that is central to our new subnet design: the singular validator. Please let us know what you think in our Discord channel!
The design of Bittensor was originally to have multiple validators that would check miner quality, and sell the digital commodities they get from miners. Although this was a good concept from a theoretical standpoint – because validators would keep each other in check and stay honest, while receiving something of value from miners, to sell – in practice it led to many problems.
First of all, it is hard work to run a validator. It is code, written by someone else, often not so stable, and running it often requires significant devops skill. Operators who run validator code as provided by subnet owners (times 64, times 128 or even more) are not to be envied. Updates may come at any time, and should be rolled out fast, or the validator risks falling out of sync with miners and other validators, which comes at the cost of losing validator reward.
“Weight copiers” decided to copy the work of other validators (the “weights” they set on chain) so that they don’t have to do any real work. Often, sadly, they can even predict the average miner weights better than the actual validators, giving them a higher “vtrust” and hence a higher validator reward, despite them doing no real work. Although much has been tried to fix this, weight copiers are still successful in many subnets.
Another problem is where validators disagree on which miner is best, for example because they score the work of miners based on a random selection of samples, adding some noise to the score. Their disagreement from the average miner score makes that they get a low vtrust value, and corresponding low reward – this is why weight copiers can earn more than real validators. Subnet builders have to factor this aspect in when writing validator code.
As of the implementation of “child hotkeys” (CHK) in the Bittensor blockchain, a validator may decide to have another validator do the work for them. This was the first step in the direction we see many subnets heading, where only one validator does the validation, and all others either copy weights or CHKs the real validator. The real validator is then often run by the subnet owner.
We see this as a good development, at least for the near term. In practice, there is no significant trust issue, as long as the subnet owner publishes its validator code, or is at least transparent on how the internal mechanics work. The operational advantage of running one validator is obvious – the downside of course being that it introduces a single point of failure for the subnet. Another benefit is that no scoring mismatch can arise between different validators, and the problem of weight copying is eliminated.
Most importantly, having only one validator allows to easily keep long running miner state, subsequently allowing for more elaborate weight setting, which is instrumental in solving issues such as those described in the optimal incentive landscape.
The next articles will discuss how having a singular validator allows for new ways of scoring and rewarding miners.
0 Comments