Last week we released a collection of new open source work from Meta FAIR — including Meta Lingua, a lightweight and self-contained codebase design to train language models at scale. Lingua is designed for research and uses easy-to-modify PyTorch components in order to try new architectures, losses, data and more. By sharing the code, we hope Lingua will enable researchers to focus on the important work they’re doing while using the platform to support efficiency and reproducibility.