GPT-4: a quantum leap into the future of artificial intelligence
On March 14, 2023, OpenAI ushered in the next era and introduced the latest version, GPT-4, to the world. With this upgrade, the AI company fundamentally revolutionized its predecessor GPT-3 and added new features.
The fundamental difference from GPT-3 is the added ability to recognize images. You can insert images and generate information from them. Another milestone is the integration of GPT-4 into the new version of Microsoft's Bing search engine. GPT-4 is 82% more accurate than GPT-3 at answering questions and makes up answers less often. So GPT-4 is more than just an upgrade to GPT-3 – it's a quantum leap into new spheres and offers a deep look into future developments.
What's the difference between GPT-3 and GPT-4?
To understand the scale of GPT-4's advances, it's important to look at each innovation:
- Thanks to its larger context, GPT-4 can process about 50 pages of English text. That means longer documents & code can be summarized and processed. Dialogs are also improved thanks to a more complex memory. As a result, the model forgets less and can work more extensively with prompts.
- With improved reasoning, the latest version outperforms its predecessor once again. GPT-4 proves that it can handle tasks such as academic exams and tests with above-average success. Accordingly, it develops broader and more complex skills in areas such as algebra/math and language comprehension.
- With the new image input feature, users can insert images. GPT-4 then produces interpretations, conclusions and various other insights. For example, it can generate websites and extract information from documents.
- GPT-4 also shines with changed AI behavior. This makes it possible to customize the model by phrasing instructions accordingly. In other words, it can generate dialogs based on previous conversations with the chatbot.
- Another advantage of GPT-4 over GPT-3 is broader data access. As a result, GPT-4 can work with far more diverse and larger datasets and be trained on them.
GPT-4 – a true AI revolution
The datasets mentioned and the resulting possibilities are another milestone in GPT-4's revolutionary program:
- The biggest difference between GPT-3 and GPT-4 is said to be the number of parameters used to train them. GPT-3 was trained with 175 billion parameters, making it the largest language model ever built at the time. For GPT-4, figures of up to 100 trillion parameters have circulated, although OpenAI has never confirmed an official number.
- Text data calculation is another important building block of ChatGPT. Here too, GPT-4 has grown considerably in terms of data volume. The amount of text input in ChatGPT is calculated in so-called "tokens". As mentioned, GPT-4 can process more text data than GPT-3. The maximum for GPT-4 is 32,000 tokens, compared with about 8,000 tokens for its predecessor.
💡 The GPT model family processes text using tokens, which are common sequences of characters found in text. These models understand the statistical relationships between these tokens and excel at producing the next token in a sequence of tokens. The tokens are calculated in OpenAI's own "tokenizer".
💡 The tokenizer shows how prompts are split up by ChatGPT or GPT-3, but also by other AI tools. One to two short English sentences correspond to about 30 tokens; two German sentences with the same content can sometimes take up twice as many tokens. OpenAI itself states a limit of 4,097 tokens for GPT-3 and thus also ChatGPT. Text beyond this limit is ignored by the AI. The number of tokens can therefore also be used to calculate the costs for AI users.
GPT-4 and natural language processing
With self-learning speech recognition systems, GPT-4's highly complex models can be built in a structured way. Methods and techniques for machine processing of natural language play a central role here:
Natural language processing (NLP) tries to capture natural language and process it on computers using rules and algorithms. It combines various methods and findings from linguistics with modern computer science and artificial intelligence. The goal is to enable the broadest possible spoken communication between humans and computers.
As a result, both applications and machines should be controllable by voice. Since computers can't draw on experience to understand language better the way humans do, they have to use AI and machine learning algorithms and methods. NLP has to capture language in the form of sound or character strings and extract the meaning, and there are various methods for this. Subfields of NLP used here include:
- Speech recognition
- Segmenting captured language into individual words and sentences
- Recognizing the base forms of words and capturing grammatical information
- Recognizing the function of individual words in a sentence
- Extracting the meaning of sentences and phrases
- Recognizing sentence context and relationships between sentences
GPT-4 and large language models
Large language models (LLMs) are part of natural language processing (NLP) research, which deals with developing models that can understand and generate natural language. They use neural network technology in particular to enable understanding and generating language. An LLM is therefore a specific model within NLP that can be applied to a wide range of tasks. NLP is the broader field dealing with processing and analyzing natural language, and LLMs are part of it.
The LLM is an integral part of the development of GPT-4. The data large language models (LLMs) are trained on usually comes from publicly available sources such as the internet. Some of the most commonly used data sources for training LLMs are:
- Wikipedia: A large share of the data used to train LLMs comes from Wikipedia articles. These articles are translated into many different languages and cover a wide range of topics.
- Book and article databases: LLMs are often trained on data from books and scientific articles available in online databases.
- Social media: Tweets, posts and other social media content are often used to train LLMs, as they contain a large amount of natural language.
- Other publicly available sources such as news articles, forum posts and online comments
What's new with GPT-4o and GPT-o1
GPT-4o
The new GPT-4 model now comes with an added "o". In this case it stands for omni, because the model can process images, text and audio. It's said to be twice as fast and 50% cheaper than its predecessor GPT-4 Turbo. The model also comes with a desktop app for Mac users, a forward-looking response to the various browsers integrating their own AI models. So far, the GPT-4o model is free for all users (with a cap on the number of requests).
The new model can think along in context and help, but the biggest leaps are in audio processing. It can respond to audio input in just 232 ms, almost as fast as in a natural conversation between two people. The AI can also tell calm breaths from excited ones and express emotions in synthetic speech. Videos and photos can be analyzed live to recognize emotions.
Earlier models often struggled with humor, which the new model improves on, and GPT-4o makes up answers much less often than its predecessors.
GPT-o1
The biggest difference between GPT-o1 and other models is that it works on the chain-of-thought principle. The problem is broken into smaller sections that are worked through one after another. If a section doesn't lead to a solution, other options are played through in the background. The model is meant to question everything first, which puts the quality of the answers ahead of speed. That's why the GPT-o1 model is especially suited to complex tasks in math, coding and scientific work.
Another development is GPT-o1-mini, which is cheaper and faster to use and particularly good at logical step-by-step thinking. The GPT-o1 model, however, has broader background knowledge – if that's relevant to the question, you're still better off with the o1 model.
Conclusion: rapid developments at ChatGPT
Some companies and organizations also use their own data to train LLMs, especially when they have specific applications the models are meant to support. This data can come from customer feedback, emails or chat logs, for example.
In conclusion, with GPT-4 OpenAI has developed an outstandingly complex and advanced AI model and, within a short time after releasing GPT-3, added impressive features and enormous data structures. The speed of technical progress shows again and again in the improved ChatGPT models.
We're incredibly excited about further developments in the market and are always working on building these innovations into our platform, melibo.
