yulan-team/YuLan-Mini
Text Generation
•
Updated
•
670
•
27
YuLan-Mini
A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.
Note A highly capable 2.4B lightweight LLM using only 1T pre-training data.
-
yulan-team/YuLan-Mini-Datasets
Updated • 302 • 8 -
YuLan-Mini: An Open Data-efficient Language Model
Paper • 2412.17743 • Published • 60
yulan-team/YuLan-Mini-Before-Annealing
Updated • 36 • 6Note The model & optimizer states of the last curriculum phase before learning rate annealing.
yulan-team/YuLan-Mini-Phase20
Updated • 8 • 2Note The model & optimizer states of the 20th curriculum phase.