Would you pay $200 a month for unlimited ChatGPT? What if it’s able to “reason”? OpenAI thinks you just might.
As part of its “12 days of Shipmas,” where the company is announcing new features for 12 days straight, OpenAI is finally bringing its first reasoning model out of preview, as well as adding unlimited access to it and all OpenAI models to a $200 monthly subscription plan.
Called OpenAI o1, the reasoning model has been available in preview since September, with paying ChatGPT members able to send 30 messages a week to o1-preview and 50 messages a week to the more lightweight o1-mini. Now that it’s in full release, as CEO Sam Altman explained during a livestream today, Plus and Team members will still be limited in how much they can use it (Enterprise and Edu members will also have to wait a week to access it), but it’ll supposedly be much more powerful when they do.
One of the biggest issues surrounding AI is hallucination, or when it simply gets something wrong. Because an AI chatbot can only rely on its training, it can’t normally tell what’s real and not, and will present falsehoods with the same confidence as facts.
Reasoning AI is an attempt to fix that. With a reasoning model, an AI will break a prompt down into multiple parts, addressing each one at a time and doing its best to check its prior conclusions for accuracy before moving on, all while showing you its thought process. It might also take more time to respond than your typical model, to help prevent errors.
This is called “chain of thought,” and while testing o1-preview and 01-mini, Lifehacker editor Jake Peterson had luck with both simple prompts (is a hot dog a sandwich?) and more complex ones (generate a 6x6 nonogram puzzle that looks like the letter Q when solved). The early version of the bot took over a minute to generate responses when necessary, and provided him with a drop-down menu allowing him to scroll through its “thought process.”
This ensured both he and the bot could easily debug and understand where mistakes came from, and with the final o1 model, OpenAI is promising that it has reduced “major errors on difficult real-world questions by 34%” and that the model is generally now “about 50% faster.”
In particular, OpenAI released charts promising the new model is over 50% more reliable than the non-reasoning GPT-4o model in coding and over 40% more reliable in competition math. These are all internal numbers, and OpenAI wasn’t exactly clear about how it’s testing or measuring these models, but those are pretty big boasts.
It’ll likely take some time for experts to do their own, independent testing, so it’s possible you’ll see a little cold water thrown on these claims soon. A recent study from Apple, for instance, found that o1’s “reasoning” abilities are still more akin to “sophisticated pattern matching.”
That’s where the catch comes in. OpenAI actually says it has a better version of o1 ready, but it comes with a hefty price tag. Announced alongside OpenAI o1 was ChatGPT Pro, a new membership plan that gives unlimited access to all OpenAI models, as well as unlocks o1’s “pro mode.”
“In evaluations from external expert testers, o1 pro mode produces more reliably accurate and comprehensive responses, especially in areas like data science, programming, and case law analysis,” OpenAI wrote in a blog post.
Essentially, Pro Mode allows the model to use more compute and take more time, resulting in a little over 10% more reliability depending on the task. Is that little bit of extra performance worth it? Well, it might be if you’re a medical researcher or other power user, which is probably why OpenAI is awarding 10 grants to “leading institutions in the U.S.,” which will give them free access to ChatGPT Pro.
Everyone else will have to decide how far they want to stretch their wallet, although OpenAI isn’t strictly targeting enterprise customers here, with the announcement livestream saying that o1 pro mode is also targeting “power users” who are “already pushing the models to the limits of their capabilities on tasks like math, programming, and writing.”
While OpenAI o1 will probably be a bit cost prohibitive for most people for now, even if they’re not looking at its pro mode (ChatGPT Plus is still $20 a month), the company did say that it’s looking to improve the model’s usability for “everyday use cases” beyond “really hard math and programming problems.” As part of today’s release, the model is now supposed to answer simple questions “really quickly,” while taking longer for harder questions, as opposed to dawdling on all queries.
With that, OpenAI is paving the way for o1 to potentially replace its non-reasoning models down the line. That could be a big boon for free users, although it’s not likely to happen anytime soon.
In the meantime, sources have told The Verge to expect Sora, OpenAI’s text-to-video model, to be released during the “12 days of Shipmas” event.
