According to a recent Bloomberg article, major AI companies like OpenAI, Google, and Anthropic are struggling to build their next-generation models, indicating a slowdown in the pace of innovation.
ChatGPT's Incremental Improvements
The article suggests that the model known as "Orion" at OpenAI did not meet the company's desired performance, according to two people familiar with the matter. This is not entirely surprising, as each iteration of a technology like ChatGPT is unlikely to see the same level of dramatic improvement as the initial versions. Just like new iPhone releases, the incremental changes become less noticeable over time.
The article also mentions that an upcoming iteration of Google's "Gemini" software is not living up to internal expectations, and Anthropic has seen the timeline slip for the release of its long-awaited "Cloud 3.5 Opus" model.
The author suggests that the companies may be facing various bottlenecks, such as compute power and available data. However, they also note that there are potential post-training unlocks, such as Chain of Thought and test-time compute, that could provide performance and scaling improvements.
Continued Innovation and Investment
Despite these apparent slowdowns, the author believes these are not significant stumbles for the companies, and they will continue to innovate. The massive amount of investment in the AI industry suggests that these challenges are temporary, and breakthroughs are likely to come in the near future.
Greg Brockman's Return to OpenAI
In another story, the article discusses the return of Greg Brockman to OpenAI. Brockman, the former CTO of the company, had been on an extended leave and vacation. Now, he is back, working with Sam Altman to find the right technical role for him within the organization.
The article also covers the story behind the acquisition of the ChatGPT.com domain. Apparently, Dharmesh Shah, the founder of HubSpot, had purchased the domain for $10 million and then sold it to OpenAI, likely earning a significant profit in the process.
Llama 3.2 Vision Capabilities
The article then discusses the latest update to the Llama model, which now includes support for Meta's Llama 3.2 Vision models. This allows the model to perform tasks such as reading prescriptions, analyzing receipts, and generating graphs.
Palantir and Anthropic Partnership
The article also covers the partnership between Palantir and Anthropic, where Palantir will be the first industry partner to bring Anthropic's Cloud models to classified environments for U.S. government intelligence and defense operations. This raises concerns about the potential implications of such powerful AI models being used in sensitive government applications.
The article then highlights the release of Quant 2.5 Coder, which is described as potentially the best open-source coding model available. The models are available in various sizes, from 32 billion to 500 million parameters, and are integrated with the Llama platform.
Microsoft's Tiny Troop Project
The article also discusses Microsoft's new open-source project called Tiny Troop, which allows the simulation of people with specific personalities, interests, and goals, similar to a "tiny world" simulation.
TSMC's Chip Manufacturing Restrictions
The article then covers the news that TSMC has been forbidden from manufacturing 2-nanometer chips outside of Taiwan, which could hinder the company's progress in the U.S. market and potentially lead to a fallout in U.S.-Taiwan relations.
The article also mentions the release of AlphaFold 3, the latest version of DeepMind's protein folding prediction model, which is now open-source and available for anyone to experiment with.
Diffusion-Based Video Game Engines
The article then discusses the emergence of a new trend in video game engines, where diffusion-based models are used to emulate environments like Minecraft in real-time, without the need for traditional game engines. The article highlights the open-source Lucid V1 project as an example of this technology.
Waymo's Expansion in Los Angeles
The article also covers the expansion of Waymo's robot taxi service in Los Angeles, which is now available to anyone within an 80-square-mile area of the county, no longer limited to an invitation-only program.
The article delves into the latest developments surrounding Mir Moraadi, the former CTO of OpenAI, and the team he is assembling for his new venture. This includes the recent departure of Manana Chen, a research program manager at OpenAI, to join Moraadi's new company, along with other former OpenAI employees.
Impressive Robot Demonstration from China
The article concludes with a discussion of a new demo video showcasing the impressive agility and capabilities of a robot developed by a Chinese company called Deep Robotics. The robot's ability to climb, move on two or four legs, and navigate various terrains is described as "absolutely insane to watch."
The article also covers the release of Nause Research's Forge Reasoning API, which allows the application of test-time compute techniques, similar to those used in OpenAI's Orion and Anthropic's models, to any uploaded model. This could potentially improve the performance of open-source models.
Meta AI Rayband Glasses Updates
Finally, the article discusses the latest updates to Meta's AI Rayband glasses, including new features such as voice-controlled video sharing, hands-free actions, and integration with Meta's AI chatbot.
Part 1/7:
The Slowing Pace of AI Innovation
According to a recent Bloomberg article, major AI companies like OpenAI, Google, and Anthropic are struggling to build their next-generation models, indicating a slowdown in the pace of innovation.
ChatGPT's Incremental Improvements
The article suggests that the model known as "Orion" at OpenAI did not meet the company's desired performance, according to two people familiar with the matter. This is not entirely surprising, as each iteration of a technology like ChatGPT is unlikely to see the same level of dramatic improvement as the initial versions. Just like new iPhone releases, the incremental changes become less noticeable over time.
The article also mentions that an upcoming iteration of Google's "Gemini" software is not living up to internal expectations, and Anthropic has seen the timeline slip for the release of its long-awaited "Cloud 3.5 Opus" model.
Potential Bottlenecks and Algorithmic Unlocks
[...]
Part 2/7:
The author suggests that the companies may be facing various bottlenecks, such as compute power and available data. However, they also note that there are potential post-training unlocks, such as Chain of Thought and test-time compute, that could provide performance and scaling improvements.
Continued Innovation and Investment
Despite these apparent slowdowns, the author believes these are not significant stumbles for the companies, and they will continue to innovate. The massive amount of investment in the AI industry suggests that these challenges are temporary, and breakthroughs are likely to come in the near future.
Greg Brockman's Return to OpenAI
In another story, the article discusses the return of Greg Brockman to OpenAI. Brockman, the former CTO of the company, had been on an extended leave and vacation. Now, he is back, working with Sam Altman to find the right technical role for him within the organization.
ChatGPT.com Domain Acquisition
[...]
Part 3/7:
The article also covers the story behind the acquisition of the ChatGPT.com domain. Apparently, Dharmesh Shah, the founder of HubSpot, had purchased the domain for $10 million and then sold it to OpenAI, likely earning a significant profit in the process.
Llama 3.2 Vision Capabilities
The article then discusses the latest update to the Llama model, which now includes support for Meta's Llama 3.2 Vision models. This allows the model to perform tasks such as reading prescriptions, analyzing receipts, and generating graphs.
Palantir and Anthropic Partnership
The article also covers the partnership between Palantir and Anthropic, where Palantir will be the first industry partner to bring Anthropic's Cloud models to classified environments for U.S. government intelligence and defense operations. This raises concerns about the potential implications of such powerful AI models being used in sensitive government applications.
Quant 2.5 Coder Release
[...]
Part 4/7:
The article then highlights the release of Quant 2.5 Coder, which is described as potentially the best open-source coding model available. The models are available in various sizes, from 32 billion to 500 million parameters, and are integrated with the Llama platform.
Microsoft's Tiny Troop Project
The article also discusses Microsoft's new open-source project called Tiny Troop, which allows the simulation of people with specific personalities, interests, and goals, similar to a "tiny world" simulation.
TSMC's Chip Manufacturing Restrictions
The article then covers the news that TSMC has been forbidden from manufacturing 2-nanometer chips outside of Taiwan, which could hinder the company's progress in the U.S. market and potentially lead to a fallout in U.S.-Taiwan relations.
AlphaFold 3 Release
[...]
Part 5/7:
The article also mentions the release of AlphaFold 3, the latest version of DeepMind's protein folding prediction model, which is now open-source and available for anyone to experiment with.
Diffusion-Based Video Game Engines
The article then discusses the emergence of a new trend in video game engines, where diffusion-based models are used to emulate environments like Minecraft in real-time, without the need for traditional game engines. The article highlights the open-source Lucid V1 project as an example of this technology.
Waymo's Expansion in Los Angeles
The article also covers the expansion of Waymo's robot taxi service in Los Angeles, which is now available to anyone within an 80-square-mile area of the county, no longer limited to an invitation-only program.
Mir Moraadi's New Venture
[...]
Part 6/7:
The article delves into the latest developments surrounding Mir Moraadi, the former CTO of OpenAI, and the team he is assembling for his new venture. This includes the recent departure of Manana Chen, a research program manager at OpenAI, to join Moraadi's new company, along with other former OpenAI employees.
Impressive Robot Demonstration from China
The article concludes with a discussion of a new demo video showcasing the impressive agility and capabilities of a robot developed by a Chinese company called Deep Robotics. The robot's ability to climb, move on two or four legs, and navigate various terrains is described as "absolutely insane to watch."
Nause Research's Forge Reasoning API
[...]
Part 7/7:
The article also covers the release of Nause Research's Forge Reasoning API, which allows the application of test-time compute techniques, similar to those used in OpenAI's Orion and Anthropic's models, to any uploaded model. This could potentially improve the performance of open-source models.
Meta AI Rayband Glasses Updates
Finally, the article discusses the latest updates to Meta's AI Rayband glasses, including new features such as voice-controlled video sharing, hands-free actions, and integration with Meta's AI chatbot.