When Open AI’s GPT-3 model made its debut in May of 2020, its performance was widely considered to be the literal state of the art.
Capable of generating text indiscernible from human-crafted prose, GPT-3 set a new standard in deep learning.
But oh, what a difference a year makes. Researchers from the Beijing Academy of Artificial Intelligence announced on Tuesday the release of their own generative deep learning model, Wu Dao, a mammoth AI seemingly capable of doing everything GPT-3 can do, and more.
First off, Wu Dao is flat-out enormous. It’s been trained on 1.75 trillion parameters which is a full ten times larger than the 175 billion GPT-3 was trained on and 150 billion parameters larger than Google’s Switch Transformers.
In order to train a model on this many parameters and do so quickly — Wu Dao 2.0 arrived just three months after version 1.0’s release in March — the BAAI researchers first developed an open-source learning system akin to Google’s Mixture of Experts, dubbed FastMoE.
This system, which is operable on PyTorch, enabled the model to be trained both on clusters of supercomputers and conventional GPUs.
This gave FastMoE more flexibility than Google’s system since FastMoE doesn’t require proprietary hardware like Google’s TPUs and can therefore run-on off-the-shelf hardware — supercomputing clusters notwithstanding.
Wu Dao also showed off its ability to power virtual idols and predict the 3D structures of proteins like AlphaFold.
- Why digital transformation is an ongoing process for enterprises
- Discover the power of your sexuality with AI
- IBM’s Project CodeNet will test how far you can push AI to write software
- The race to understand the thrilling, dangerous world of language AI
- The Struggle To Stop AI From Cheating On Tests
- Chatbots: AI tech can be useful in real-world business applications right now
- Google AI Researchers Are Dreaming Up a New Species of Search Engine
- Pitfalls to avoid when using AI for the first time
- Using Low-Code and No-Code in IoT App Development
- Gartner Says 70% of Organizations Will Shift Their Focus From Big to Small and Wide Data By 2025