The (over)analyzing miner

This is part of our series on a year of Bittensor experience, leading up to our anniversary at the 13th of July. We discuss 9 miner archetypes before digging deeper into incentive landscapes. After describing honest, rogue, calculative, blockchain-savvy, overfitting, and devops miners, we turn our attention to the (over)analyzing miner. If you’re still reading along, the contours of proper subnet design should be almost clear now. Please let us know what you think in our Discord channel!

The (over)analyzing miner dives deeper than anyone else into the mines of the subnet to see if there is an edge that nobody else has found yet. AI provides deeper mines than any other technology. We view this miner as a (successful) honest miner, so in that sense it differs from many of the miner types discussed before. In practice we have seen overanalyzing miners being chastised for alledged foul play – to our strong disagreement. This miner plays in a different league than the overfitting miner, who wilfully produces garbage to win.

In one subnet, a brand new LLM needed to be trained with its size between a specified minimum and maximum number of weights. Smaller LLMs, heavily trained on the same dataset, already existed. Instead of starting training from scratch, one miner decided to:

• take the best available smaller LLM,
• add almost-zero weights to increase it to the right size,
• train it just a little, and
• submit it as a new model.

This of course was far from trivial and required careful analysis on how models can grow without losing their embedded knowledge. However, it was far more effective than spending a sports car worth of GPU hours on a training run (yes…), or than spending weeks of R&D on optimized pipelined parallel training code (yes…), to eventually arrive at a model that could also be produced with a few lines of Python code in under a minute.

The real issue here was that the challenge contained a minimum size for the model, which is a strange restriction, once you understand how these models work. Making models smaller is hard, making them larger is trivial.

The general problem here is that the subnet owner must not be outsmarted by the miners, while the reason for running a subnet for others to mine, is to get smarter people on board (this is the big paradox of Bittensor). We think that a good subnet owner must also be a good miner – on his or her own subnet – to prevent such issues from creeping in.

Tomorrow we discuss a special breed of miner: the clueless miner.