When Open AI’s GPT-3 model made its debut in May of 2020, its performance was widely considered to be the literal state of the art.
Capable of generating text indiscernible from human-crafted prose, GPT-3 set a new standard in deep learning.
But oh, what a difference a year makes. Researchers from the Beijing Academy of Artificial Intelligence announced on Tuesday the release of their own generative deep learning model, Wu Dao, a mammoth AI seemingly capable of doing everything GPT-3 can do, and more.
First off, Wu Dao is flat-out enormous. It’s been trained on 1.75 trillion parameters which is a full ten times larger than the 175 billion GPT-3 was trained on and 150 billion parameters larger than Google’s Switch Transformers.
In order to train a model on this many parameters and do so quickly — Wu Dao 2.0 arrived just three months after version 1.0’s release in March — the BAAI researchers first developed an open-source learning system akin to Google’s Mixture of Experts, dubbed FastMoE.
This system, which is operable on PyTorch, enabled the model to be trained both on clusters of supercomputers and conventional GPUs.
This gave FastMoE more flexibility than Google’s system since FastMoE doesn’t require proprietary hardware like Google’s TPUs and can therefore run-on off-the-shelf hardware — supercomputing clusters notwithstanding.
Wu Dao also showed off its ability to power virtual idols and predict the 3D structures of proteins like AlphaFold.
- The Automation-Human Balance Takes Shape in Security
- 3 Tactics to Accelerate a Digital Transformation
- Putting Production on Repeat with Machine Tool Automation
- AI in manufacturing: Optimizing costs and enabling the workforce
- RPA: Why you need to care about this totally unsexy technology
- Buildings IOT Implements Smart Building Management System for Thor Equities’ 800 Fulton Market Development in Chicago
- Artificial Intelligence (AI) in Energy
- Automation Tools Are Key to a Successful Hybrid Workplace
- IoT technology can ease food supply chain pressures
- When Big Data Goes Wrong: 3 Common Issues and Possible Solutions