This is part of our series on a year of Bittensor experience, leading up to our anniversary at the 13th of July. This will be an exciting day, as we will not only change the font of this website, but also announce the relaunch of our subnet on a totally different subject than LLM training. After discussing miner archetypes, incentive landscapes and validator technicalities, we now discuss a new concept for validation, that we think is crucial for our new subnet idea. Please let us know what you think in our Discord channel!
The concept is the culmination of ideas we gathered after building and tweaking a subnet, mining various kinds of subnets, having felt the burden of running a validator on our own subnet, running 24/7 mining infra on various other subnets, debugging validator codebases, taking part in many discussions about Bittensor and analyzing/debugging/fixing core issues in btcli/bittensor/subtensor. We think we found a way to address many technical and operational issues. Of course we may have missed an aspect or nuance here and there – if you think so, please join the ongoing discussion in our Discord channel!
Our new concept consists of a pool of tasks, administered by the validator, where miners are free to choose which task they try to solve. The first one to solve a task, gets the associated reward. Tasks that are not solved, increase in value over time – they have no deadline. Simple tasks are probably solved by miners that are good at automating processes. Their reward is in their volume – building a bot that earns a lot of small amounts, on relatively simple challenges, will be worthwhile. Complex tasks, that are hard to automate and require research, will accrue an ever higher reward. Instead of building a bot, a miner may decide to crack a hard problem and hit the jackpot every few days or weeks. Idle mining does not deliver, because un-earned reward will be burned.
In essence, this rewards speed and not score, so the concept works best for binary concepts (task is solved or not solved) or for concepts with a clear quality threshold (good enough is good enough), although you can imagine including a quality score, where the reward for the task is increased by some factor – where still the fastest submission that crosses the minimum threshold gets the reward. It could also work with synthetic tasks, that can be added to the work pool at a constant pace.
Note that compared to existing concepts that are found in a majority of subnets, this concept is somewhat of a crossover between “bandwidth” subnets – where initiative is with the validator – and “setCommitment” subnets – where initiative is with the miner. In the new concept, the miner takes the initiative to select a task out of a pool of tasks that is managed by the validator. Note that the miner does not “claim” the task, he simply submits a resolution to the problem, and if he is first to solve it, he gets the associated reward, and the task is marked as solved. This is a small conceptual change, but its impact may be significant. Miners will not be forced to be on standby 24/7 under penalty of losing their reward, or worse, their registration. This should take some devops stress away from them, or equivalently, give miners that enjoy mining, but dislike hosting high-availability infra, a fair chance.
If some miner would find a research paper that describes a new method of solving a problem, giving them the edge to solve 10 outstanding tasks that have accrued a high reward, all at once – good for them. It will be a nice reward for being able to connect the dots, faster than other miners. Our clients will be very happy to see their problems solved, as soon as the knowledge to solve them became public. I include this example, because in the SN29 Novelty Search we had an intriguing discussion about whether winning an SN9/SN29 LLM training challenge with a good model, taken from the internet, would be fair play. I thought, and still think, it is. We reward good work – we don’t care how you do it. It is only our job to make sure the actual work is done.
We think that in concrete implementations of this, miners can easily be incentivized to collect winnings on one UID, instead of registering multiple UIDs. One simple way is to raise their score to the power of some f>1, because (2s)f>2sf . Another way is to impose a rate limit, that is gradually increased proportional to the success of the miner, in such a way that it is always beneficial to have one UID with more than twice the rate limit of two UIDs.
Conceptually, having a singular validator is key to this approach. A setup with multiple validators however is possible, and may be desirable, so that they can sell the digital commodity produced to their own clients. The validators would have to jointly decide on miner scores and weights, because individually they may have wildly differing views on miners – which is a logical conclusion from them serving different clients, with different kinds of tasks, where different miners may decide to specialize in. The validators will together act as one singular validator in the sense that they come to an agreement about weights to set on chain. (We ignore the inter-validator trust issues for now.)
In recent discussions on Discord, opinions were voiced that running multiple validators is essential to the proper working of Bittensor. This may be the case, and we see no fundamental objection to this, as was clear from the previous paragraph. However, we think that for a first iteration of the to-be launched subnet, it is better to run a singular validator. As soon as the subnet is up and running, and validators have signed their own deals with customers, a pivot can be made to split the singular validator. By having the singular validator already conceptually split up in parts that produce and consume auditable structured data, this should be doable without a redesign or rewrite.
Tomorrow we will draw the contours of what we think are suitable subjects for Bittensor subnets.
0 Comments