OpenAI is facing difficulties with Orion, the next-generation model that powers its AI. The company is having issues in certain areas with the performance gains achieved with the successor to GPT-4. This is quoted in a report from The Information as an OpenAI official claims that Orion has made “much less” quality gains than the transition from GPT-3 to GPT-4.
We’ve also heard some OpenAI researchers say that Orion “is not reliably better than its predecessor [GPT-4] at handling certain tasks.” What tasks are these? Apparently, encoding is a weakness, and Orion probably can’t beat GPT-4 in this area, but Orion’s linguistic skills have also been noted.
So it seems to work (relatively) well for common queries, and tasks like summarizing and rewriting text. But for those hoping to use AI as a programming aid, these rumours don’t seem all that promising.
Underlying Problem for OpenAI and Orion
OpenAI seems to have hit a wall when it comes to the data available to train its AI. As the report makes clear, there is a “dwindling supply of high-quality text and other data” that can be used in pre-training to improve the skill of LLMs (large-scale language models) at solving more complex problems, like correcting programming errors.
While these LLMs have already grabbed many of the low hanging fruit, it is now becoming significantly harder to find high-quality training data, slowing progress in some ways.
Moreover, this training is becoming increasingly computationally intensive, making Orion (and subsequently other AI models) significantly more expensive to develop (and operate). Of course, users of AI will eventually foot this bill in some way, and there is even talk of making the development of more advanced models effectively “economically impossible”. With growing concerns about climate change, not to mention the environmental impact of large data centers consuming more electricity from the power grid. While this report should be treated with caution, there are worrying signs here that herald a serious reality check for future AI development.
This information also suggests that a different approach could be taken to continually improve AI models after initial training. In fact, it looks like this may be necessary. Orion is expected to launch in early 2025 (not sooner as some rumors suggest). It’s possible that OpenAI will completely change their AI naming scheme with this next-generation model, so it likely won’t be called ChatGPT-5.
