San Francisco, CA—At the OpenAI Dev Day 2024, OpenAI introduced four new features, expanding its AI offerings with real-time capabilities, vision fine-tuning, and more. The event, held on October 1, featured workshops, live demos, and hands-on sessions for developers.
Here is Sam Altman’s take on OpenAI Dev Day
OpenAI Dev Day opening keynote from the video:
1. Launch of GPT-4 Turbo: Introduced with extended context length and reduced costs.
2. New Features: Added JSON Mode and reproducible outputs for better user control.
3. Partnerships: Collaborations with platforms like Canva and Zapier to enhance user experiences.
4. Focus on Innovation: Emphasis on AI safety, privacy, and empowering developers.
5. Live Demonstrations: Showcased capabilities of voice and vision features in practical applications.
New AI Features Announcements:
1- Real-Time API
The biggest announcement that they’ve made is this whole concept of a real-time API. This will give us the ability to build voice apps that just go directly from our device to their API, processing the voice and responding in voice as well. This innovation is designed for use cases like virtual assistants and live translation, allowing simultaneous multimodal outputs and function calling to trigger real-world actions. The API, priced at approximately $0.30 per minute, adds value for developers who want to build responsive, voice-based apps.
/ Olivier Godement, OpenAI’s Head of Product, emphasized that this technology is designed to make AI more accessible, catering to users who prefer speaking over typing
2- Vision Fine-Tuning framework
A new Vision Fine-Tuning framework also takes the spotlight, empowering developers to customize models with control over hyperparameters like epochs and learning rates. Integration with Weights and Biases offers detailed tracking tools to monitor fine-tuning jobs and performance. The vision fine-tuning API allows you to fine-tune GPT-4 with texting text and images in and out. You’ll probably end up hosting the images yourself somewhere on the web. Pricing for this is $25 per million tokens for training and $15 per million output tokens.
3- Model Distillation AP
OpenAI’s Llama 3 models, including the 1B and 3B versions, use model distillation techniques to provide lightweight, high-performance alternatives to their larger counterparts. This approach preserves core functionalities while significantly enhancing speed, making these models suitable for real-time applications on mobile devices and edge computing. By reducing computational overhead, model distillation ensures faster deployment and helps businesses lower cloud and operational costs without sacrificing accuracy or versatility.
4- Prompt Caching
The introduction of Prompt Caching aims to reduce latency by reusing frequently processed inputs and optimizing response times for applications that rely heavily on predictable queries. By optimizing server loads and response times, Prompt Caching ensures seamless user experiences during high-traffic periods, enabling businesses to handle increased demand efficiently without compromising performance.
O1 Model Unveiled with Code Generation Capabilities and Focus on Safety
The O1 model was also showcased during the event, demonstrating its ability to generate and architect code. In one example, a developer successfully created an iPhone app by describing the idea, highlighting the model’s potential beyond text-based interactions.
OpenAI emphasized safety throughout the event, hinting at future tools like a “safety API” to give users more control over AI usage. “We’re committed to running continuous safety evaluations on fine-tuned models,” the company stated.
The announcements signal a major leap forward in the AI landscape, with follow-up events scheduled for London on October 30 and Singapore on November 21, 2024. Developers can now explore the newly released tools and refer to OpenAI’s updated documentation for deeper insights.
Manahil Samuel holds a Bachelor’s in Computer Science and has worked on artificial intelligence and computer vision She skillfully combines her technical expertise with digital marketing strategies, utilizing AI-driven insights for precise and impactful content. Her work embodies a distinctive fusion of technology and storytelling, exemplifying her keen grasp of contemporary AI market standards.
