OpenAI's DevDay brings Realtime API and other treats for AI app developers
It's been a tumultuous week for OpenAI, full of executive departures and major fundraising developments, but the startup is back at it, trying to convince
You are viewing a single comment's thread from:
At its 2024 DevDay event, OpenAi, a pioneering artificial intelligence (AI) startup, unveiled a slew of new tools and features designed to enhance its AI models and entice developers to build applications on its platform. Despite recent executive departures, including the chief technology officer and chief research officer, OpenAI's chief product officer, Kevin Weil, reassured attendees that the company's progress would not be hindered.
One of the most significant announcements was the Realtime API, which enables developers to create apps with low-latency, AI-generated voice responses. This feature bears some resemblance to ChatGPT's Advanced Voice Mode, but with some key differences. The Realtime API offers six distinct voices, which are not compatible with third-party voices to prevent copyright issues. This feature is expected to revolutionize the way developers build voice-enabled applications, allowing them to create more engaging and interactive experiences for users.
OpenAI also introduced vision fine-tuning in its API, enabling developers to utilize images in addition to text to fine-tune their applications of GPT-4o. This feature is expected to significantly improve the performance of GPT-4o for tasks involving visual understanding, such as image classification, object detection, and more.
The company also announced prompt caching, which allows developers to cache frequently used context between API calls, reducing costs and improving latency. According to OpenAI, developers can save up to 50% using this feature, while Anthropic promises a 90% discount. This feature is expected to be particularly beneficial for developers who rely heavily on OpenAI's API for their applications.
Furthermore, OpenAI introduced model distillation, which enables developers to use larger AI models, such as o1-preview and GPT-4o, to fine-tune smaller models, such as GPT-4o mini. This feature is expected to provide cost savings and improve the performance of smaller AI models, making it an attractive option for developers working with limited resources.
Notably, OpenAI's DevDay event did not include any announcements regarding the GPT Store, which was introduced last year. Additionally, the company did not release any new AI models during the event, including OpenAI o1 and the video generation model, Sora.
Overall, OpenAI's new features and tools aim to convince developers to build AI apps on its platform, which is operating in an increasingly competitive space. By providing developers with more powerful and flexible tools, OpenAI is positioning itself as a leader in the AI development space, and its new features are expected to have a significant impact on the industry.