GPT-3 is the most advanced language model ever created, a product of an effort by OpenAI to create a publicly available system that can be used to advance research and applications in natural language. The model itself published less than three months ago, is an autoregressive language model with 175 billion parameters and was trained with a dataset that includes almost a trillion words.
Impressive as that may be, it is difficult to get some intuition of what such a complex model, trained on billions of human-generated texts, can actually do. Can it be used effectively in translation tasks or in answering questions?
To get some idea of what a sufficiently high-level statistical model of human language can do, I challenge you to have a look at this conversation with GPT-3, published by Kirk Ouimet a few days ago. It relates a dialogue between him and GPT-3 on the topic of COVID-19. The most impressive thing about this conversation with an AI is not that it gets many of the responses right (others not so much). What impressed me is that the model was trained with a dataset created before the existence of COVID-19, which provided GPT-3 no specific knowledge about this pandemic. Whatever answers GPT-3 gives to the questions related to COVID-19 are obtained with knowledge that was already available before the pandemic began.
This certainly raises some questions on whether advanced AI systems should be more widely used to define and implement policies important to the human race.