Summary [2210.02414] GLM-130B: An Open Bilingual Pre-trained Model arxiv.org
686 words - html page - View html page
One Line
arXivLabs provides a framework to develop and share arXiv features, and GLM-130B is an open-source bilingual pre-trained language model with 130 billion parameters supported by the Simons Foundation.
Key Points
- arXivLabs is a framework for developing and sharing new arXiv features
- GLM-130B is an open-source bilingual pre-trained language model with 130 billion parameters
- GLM-130B offers significant outperformance over GPT-3 175B on a wide range of popular English and Chinese benchmarks
- GLM-130B can be effectively inferred on 4 RTX 2080 Ti (11G) GPUs or 8 RTX 3090 GPUs
- Support for the GLM-130B project was provided by the Simons Foundation and member institutions
- arXiv provides status notifications, mailings, and contact information
Summary
174 word summary
Get status notifications via arXiv Operational Status, Web Accessibility Assistance, and Privacy Policy. To subscribe to arXiv mailings, click here. To contact arXiv, click here.
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on their website. It is committed to values of openness, community, excellence, and user data privacy. Projects include CORE Recommender, Connected Papers, Spaces, Replicate, ScienceCast, Papers with Code, Smart Citations, Litmaps, and Bibliographic Explorer.
The article provides a DOI link to access code, training logs, related toolkit, and lessons learned for using 100B-scale models. The GLM-130B model weights are publicly available, and it can be effectively inferred on 4 RTX 2080 Ti (11G) GPUs or 8 RTX 3090 GPUs. It offers significant outperformance over GPT-3 175B on a wide range of popular English and Chinese benchmarks. We introduce GLM-130B, an open-source bilingual (English and Chinese) pre-trained language model with 130 billion parameters, an attempt to surpass GPT-3. Support for this project was provided by the Simons Foundation and member institutions. We acknowledge their support.