Edge 256: The Architecture and Methods Powering ChatGPT
An overview of the AI techniques behind OpenAI's new supermodel
ChatGPT has been one of the most popular artificial intelligence(AI) agents ever created. The model has taken the data science community and the internet by storm pushing the boundaries of creativity across all industries. Despite the immense popularity of ChatGPT, there have been very little discussion about the AI techniques behind its magic. Many of the techniques behind ChatGPT are going to be the foundation of the upcoming GPT-4 which promises to be one of the most impressive models in AI history.
The main ideas behind ChatGPT were pioneered by another OpenAI’s , InstructGPT which was released earlier this year. InstructGPT fine tunes GPT to follow instructions which opens the door to a wider set of human interactions . ChatGPT takes some of the ideas pioneered by InstructGPT to a whole new level with a very novel architecture and training process.