LLMTrainer
A comparison of pretraining framework for LLM
Install / Use
/learn @ifromeast/LLMTrainerREADME
LLMTrainer
对比 LLM 的多种预训练方案的性能，方案包括：
- Torch DDP
- FSDP
- DeepSpeed
- FlashAttention V2
- ColossalAI
- Sophia Optimizer
- Lightning Fabric
详细介绍见 https://zhuanlan.zhihu.com/p/648094197?
