GPT-3
2020 text-generating language model / From Wikipedia, the free encyclopedia
Dear Wikiwand AI, let's keep it short by simply answering these key questions:
Can you list the top facts and stats about GPT-3?
Summarize this article for a 10 year old
Generative Pre-trained Transformer 3 (GPT-3) is a large language model released by OpenAI in 2020. Like its predecessor, GPT-2, it is a decoder-only[2] transformer model of deep neural network, which supersedes recurrence and convolution-based architectures with a technique known as "attention".[3] This attention mechanism allows the model to selectively focus on segments of input text it predicts to be most relevant.[4] It uses a 2048-tokens-long context[jargon], float16 (16-bit) precision, and a hitherto-unprecedented 175 billion parameters, requiring 350GB of storage space as each parameter takes 2 bytes of space, and has demonstrated strong "zero-shot" and "few-shot" learning abilities on many tasks.[2]
Original author(s) | OpenAI[1] |
---|---|
Initial release | June 11, 2020 (beta) |
Repository | |
Predecessor | GPT-2 |
Successor | GPT-3.5 GPT-4 |
Type | |
Website | openai |
On September 22, 2020, Microsoft announced that it had licensed GPT-3 exclusively. Others can still receive output from its public API, but only Microsoft has access to the underlying model.[5]