This is part of our series on a year of Bittensor experience, leading up to our anniversary at the 13th of July. We discuss 9 miner archetypes before digging deeper into incentive landscapes. After discussing the honest, the rogue and the calculative miner, we’re looking at the blockchain-savvy miner today. Read more…
This is part of our series on a year of Bittensor experience, leading up to our anniversary at the 13th of July. We discuss 9 miner archetypes before digging deeper into incentive landscapes. After discussing the honest and the rogue miner, the calculative miner is considered. Please let us know Read more…
This is part of our series on a year of Bittensor experience, leading up to our anniversary at the 13th of July. We discuss 9 miner archetypes before digging deeper into incentive landscapes. The previous article discussed the honest miner. Please let us know what you think in our Discord Read more…
With our first anniversary fast approaching on the 13th of July, it’s time to look back on our wild ride in Bittensor, in our role as miners, subnet owners, observers and contributors. We will be sharing our insights in a few articles, the first of which you are reading now, Read more…
As explained in our previous article, growing a model in most dimensions in quite simple, but increasing the hidden size comes with a few problems. This article dives deep and shows how it can be done. Why, again? We can simply grow a model’s MLP (intermediate size) or number of Read more…
In this post, we revisit model comparison metrics. Should we compare sample-by-sample, or group several samples together? Or does it make more sense to “pack” samples (i.e. join multiple samples with an EOS token in between)? Does length matter? And what’s up with these “pages” in the dataset? TL;DR A Read more…
In this blog, and in our Discord channel, we discuss training in detail. A topic that is often overlooked, is how to grow a model. Especially in incentivized, collaborative and distributed training, this is a key ingredient. This post explores the concept of model growth with concrete Python code examples Read more…
This post discusses one of the essential ingredients for training a model: how to see whether your model is actually improving, in other words, how do you benchmark the result of your training? Please read our prior blog posts (e.g. On incentive (part 1)) to understand the concept of sample Read more…
In the previous post on incentive we described how loss values are used to score models against each other, and how the relative standing between models changes over time due to decaying advantage. From that conceptual idea a few steps are needed to get to a ranking of models and Read more…
One of the first objectives for subnet 29 was to see whether the sample packing during training makes a difference for the achievable loss. TL;DR Sample packing makes significant difference. Per-token loss of a trained model is generally higher when training on packed samples, and evaluating on single samples. Or, Read more…