Subnet creation changelog:
TODO: immediate post-launch steps
- Spread the word about SN29 renaming to actors not using on-chain identity data
- (Suggest to those actors to use on-chain identity data…)
- Implement competitions to allow training for specific goals
- Clean up mining codebase and publish on GitHub
- Analyze possibilities and implement on-chain announcement of validator startup with version information
- Finalize testing with arbitrary tokenizer (immediately saves up to 800m of 6.9b parameters on reference model)
- Launch first additional competition (weight fraction 0.1) with arbitrary tokenizer
- Collect results and feedback on subnet and competition performance
2024 Q3
- Consult with community and create a list of competitions, scheduled well in advance
- Research model merging tactics to unlock distributed training potential of subnet
- Draft shortlist of finetuning targets for niche models
- Launch first additional competition for niche model
- Publish
model_surgeon.py, a commandline tool to modify models
2024 Q4
- Strive to have 5 niche models in training
- Provide boilerplate code for web applications, host apps showcasing top models
- Research external benchmarks to assess subnet efficacy
2025 and beyond
- Pretrain-as-a-Service: commercial opportunities
- Finetune-as-a-Service: commercial opportunities