OpenAI — the company behind ChatGPT — held its second annual DevDay in the fall of 2024. It unveiled several massive updates, tools, and features to delight attendees and livestream viewers. What were these changes? Will they benefit developers? Here’s a breakdown of the event and what it means for the future of ChatGPT. OpenAI’s 2024 DevDay DevDay — a portmanteau of developer and day — is a global conference hosted by OpenAI. While 2024’s updates were unveiled in San Francisco on Oct. 1, the company plans to visit London on Oct. 30 and Singapore on Nov. 21. This is in contrast to last year when the only event was in San Francisco. OpenAI’s reasoning for expanding its scope is twofold. For one, it wants to be closer to its global audience. The second reason is that developers requested more time to learn from each other after 2023’s DevDay. So, while there will technically be three events, every major update has already been revealed. The most recent conference was the company’s second. It held its first almost one year prior, in November 2023, where it announced a JSON mode, custom models, and a copyright protection tool, among other things. Since 2023’s DevDay was packed with massive updates, many developers expected 2024 to be just as big, if not bigger. While many expected OpenAI to unveil the next iteration of ChatGPT, which started training in 2024, it announced it would prioritize application programming interface (API) advancements and developer tools. This DevDay focused on ChatGPT-4o and GPT-4o mini, which were released in 2024 as successors to GPT-4. The “o” in GPT-4o stands for omni, emphasizing the model’s multimodality. It outperforms GPT-4 in many ways. It supports more languages,has a 128,000-token context window, and a knowledge cutoff date of late 2023. As artificial intelligence goes, it is among the most powerful, fast models today — and it has just received multiple massive updates. 5 Major Updates Revealed During DevDay Five major updates were unveiled during DevDay 2024. While they became available to developers on Oct. 1, some are only for paying customers. Each has unique implications for developers. 1. Model Distillation Model distillation involves leveraging a larger model to teach a smaller one. OpenAI announced multiple features for this cost-effective training method. The first is stored completions, which lets developers automatically keep input-output pairs on the platform permanently. This enables them to seamlessly generate datasets for distillation. The second model distillation tool is a beta of an Evals product. It lets developers conduct distillation end-to-end on the platform to measure their model’s performance. Instead of manually creating scripts and tinkering with a mess of disparate logging tools, they can easily create and run custom evaluations. 2. Vision Fine-Tuning Vision fine-tuning is a customization feature that makes it possible to fine-tune for GPT-4o and 4o-mini with images — not just text. This way, models will have a stronger understanding of visual input. As a result, they perform better in object recognition, detection, image search, and classification tasks. 3. Prompt Caching OpenAI’s prompt caching provides automatic discounts on input the model has recently seen. Instead of paying full price, developers get 50% off — and faster processing speeds — starting at about 1,024 cached tokens. This feature is available on GPT-4o, GPT4-o mini, o1-preview, and o1 mini. Since 1,000 tokens represent approximately 750 words, companies will save on commonly reused prompts. Notably, specifics on how long the cached storage is saved vary depending on whether the user is active during peak or off-peak hours. However, they don’t have to go out of their way to manually apply the discount. 4. Higher Rate Limit A rate limit controls how often users can access the server to make requests within a certain time frame. OpenAI announced it is doubling its rate limit for GPT-4o, going from 5,000 requests per minute to 10,000. This change may not seem as grand as the others, but it will be just as significant in the long run. 5. Realtime API In the past, developers needed three different platforms to create a speech-to-speech model — one for transcription, one for inference, and one for text-to-speech. With Realtime API, they can now integrate fast, natural speech-to-speech conversations into their applications. They have six preset voices to choose from. Realtime API was previously exclusive to ChatGPT as Advanced Voice Mode. Now, professionals can build models that have lifelike speech-to-speech discussions. The delays are shorter and the voices are clearer. Users can even interrupt the AI midsentence to steer the conversation in a different direction. Applications of an Updated ChatGPT Model distillation is one of the most significant features unveiled during the 2024 DevDay since it allows inexperienced professionals to build robust models. It has applications in health care, finance, and retail. It could even be used in a nontechnical setting like education. Since upwards of 60% of educators already use AI in the classroom, integration would be relatively seamless. Notably, model distillation has proven to be an effective tool for improving open models on various tasks. However, it doesn’t reach the teacher model’s level of performance. This slight drop in accuracy and speed may be unacceptable in high-risk, fast-paced industries. However, users can spend extra time fine-tuning their model if needed. Vision fine-tuning is just as significant as model distillation. Developers could create highly accurate image detection and classification models. Trained models could evaluate X-rays to identify cancerous cells, analyze surveillance video to identify intruders, or improve quality control in high-risk settings like aerospace manufacturing. Prompt caching’s real-world impact isn’t as large, but it is still important. It may even be the feature developers are most excited about. After all, it’s something they’ve been waiting years for. Automatically getting half off commonly used prompts could save individuals and companies a lot of money. Notably, prompt caching isn’t exclusive to GPT-4o and GPT-4o mini — Anthropic’s Claude and Google’s Gemini already have it. For reference, Gemini offers a 75% discount on input tokens. However, the key difference is that it isn’t automatic — users must use CachedContent.create to create a cache and explicitly specify usage when defining the model. Realtime API is last but not least. This feature could revolutionize speech-to-speech conversations, enabling seamless communication between humans and AI. Businesses could leverage it for customer service, scheduling, translation, or summarization. They could use it as an assistant, companion, or support tool. What These Updates Mean for ChatGPT’s Future The significance of focusing on API advancements and developer tools for this year’s DevDay cannot be ignored. Instead of rolling out a flashy new model at the big annual conference to make headlines and generate hype, OpenAI is ensuring its foundation is strong. These updates will encourage external innovation and generate insightful feedback from users. These updates could kick-start a growth period for OpenAI, causing speech-to-speech and image-focused models to trend. At the very least, other AI engineers and businesses will look toward this company as inspiration. Similar updates from competitor models will likely follow in the near future. As for the implications of Realtime API, prompt caching, vision fine-tuning, and model distillation, the one-day conferences in London and Singapore will likely be microcosms of what’s to come. Attendees and livestream viewers should expect to see developer-created content, insights, and questions featured at the upcoming events. OpenAI — the company behind ChatGPT — held its second annual DevDay in the fall of 2024. It unveiled several massive updates, tools, and features to delight attendees and livestream viewers. What were these changes? Will they benefit developers? Here’s a breakdown of the event and what it means for the future of ChatGPT. OpenAI’s 2024 DevDay DevDay — a portmanteau of developer and day — is a global conference hosted by OpenAI. While 2024’s updates were unveiled in San Francisco on Oct. 1, the company plans to visit London on Oct. 30 and Singapore on Nov. 21. This is in contrast to last year when the only event was in San Francisco. plans to visit London on Oct. 30 plans to visit London on Oct. 30 OpenAI’s reasoning for expanding its scope is twofold. For one, it wants to be closer to its global audience. The second reason is that developers requested more time to learn from each other after 2023’s DevDay. So, while there will technically be three events, every major update has already been revealed. wants to be closer wants to be closer The most recent conference was the company’s second. It held its first almost one year prior, in November 2023, where it announced a JSON mode, custom models, and a copyright protection tool, among other things. Since 2023’s DevDay was packed with massive updates, many developers expected 2024 to be just as big, if not bigger. While many expected OpenAI to unveil the next iteration of ChatGPT, which started training in 2024, it announced it would prioritize application programming interface (API) advancements and developer tools. This DevDay focused on ChatGPT-4o and GPT-4o mini, which were released in 2024 as successors to GPT-4. The “o” in GPT-4o stands for omni, emphasizing the model’s multimodality. It outperforms GPT-4 in many ways. It supports more languages, has a 128,000-token context window, and a knowledge cutoff date of late 2023. As artificial intelligence goes, it is among the most powerful, fast models today — and it has just received multiple massive updates. has a 128,000-token context window , has a 128,000-token context window 5 Major Updates Revealed During DevDay Five major updates were unveiled during DevDay 2024. While they became available to developers on Oct. 1, some are only for paying customers. Each has unique implications for developers. 1. Model Distillation Model distillation involves leveraging a larger model to teach a smaller one. OpenAI announced multiple features for this cost-effective training method. The first is stored completions, which lets developers automatically keep input-output pairs on the platform permanently. This enables them to seamlessly generate datasets for distillation. The second model distillation tool is a beta of an Evals product. It lets developers conduct distillation end-to-end on the platform to measure their model’s performance. Instead of manually creating scripts and tinkering with a mess of disparate logging tools, they can easily create and run custom evaluations. 2. Vision Fine-Tuning Vision fine-tuning is a customization feature that makes it possible to fine-tune for GPT-4o and 4o-mini with images — not just text. This way, models will have a stronger understanding of visual input. As a result, they perform better in object recognition, detection, image search, and classification tasks. 3. Prompt Caching OpenAI’s prompt caching provides automatic discounts on input the model has recently seen. Instead of paying full price, developers get 50% off — and faster processing speeds — starting at about 1,024 cached tokens. This feature is available on GPT-4o, GPT4-o mini, o1-preview, and o1 mini. starting at about 1,024 starting at about 1,024 Since 1,000 tokens represent approximately 750 words , companies will save on commonly reused prompts. Notably, specifics on how long the cached storage is saved vary depending on whether the user is active during peak or off-peak hours. However, they don’t have to go out of their way to manually apply the discount. represent approximately 750 words represent approximately 750 words 4. Higher Rate Limit A rate limit controls how often users can access the server to make requests within a certain time frame. OpenAI announced it is doubling its rate limit for GPT-4o, going from 5,000 requests per minute to 10,000. This change may not seem as grand as the others, but it will be just as significant in the long run. it is doubling its rate limit it is doubling its rate limit 5. Realtime API In the past, developers needed three different platforms to create a speech-to-speech model — one for transcription, one for inference, and one for text-to-speech. With Realtime API, they can now integrate fast, natural speech-to-speech conversations into their applications. They have six preset voices to choose from. have six preset voices have six preset voices Realtime API was previously exclusive to ChatGPT as Advanced Voice Mode. Now, professionals can build models that have lifelike speech-to-speech discussions. The delays are shorter and the voices are clearer. Users can even interrupt the AI midsentence to steer the conversation in a different direction. Applications of an Updated ChatGPT Model distillation is one of the most significant features unveiled during the 2024 DevDay since it allows inexperienced professionals to build robust models. It has applications in health care, finance, and retail. It could even be used in a nontechnical setting like education. Since upwards of 60% of educators already use AI in the classroom, integration would be relatively seamless. 60% of educators already use AI 60% of educators already use AI Notably, model distillation has proven to be an effective tool for improving open models on various tasks. However, it doesn’t reach the teacher model’s level of performance. This slight drop in accuracy and speed may be unacceptable in high-risk, fast-paced industries. However, users can spend extra time fine-tuning their model if needed. proven to be an effective tool proven to be an effective tool Vision fine-tuning is just as significant as model distillation. Developers could create highly accurate image detection and classification models. Trained models could evaluate X-rays to identify cancerous cells, analyze surveillance video to identify intruders, or improve quality control in high-risk settings like aerospace manufacturing. Prompt caching’s real-world impact isn’t as large, but it is still important. It may even be the feature developers are most excited about. After all, it’s something they’ve been waiting years for. Automatically getting half off commonly used prompts could save individuals and companies a lot of money. Notably, prompt caching isn’t exclusive to GPT-4o and GPT-4o mini — Anthropic’s Claude and Google’s Gemini already have it. For reference, Gemini offers a 75% discount on input tokens. However, the key difference is that it isn’t automatic — users must use CachedContent.create to create a cache and explicitly specify usage when defining the model. offers a 75% discount offers a 75% discount Realtime API is last but not least. This feature could revolutionize speech-to-speech conversations, enabling seamless communication between humans and AI. Businesses could leverage it for customer service, scheduling, translation, or summarization. They could use it as an assistant, companion, or support tool. What These Updates Mean for ChatGPT’s Future The significance of focusing on API advancements and developer tools for this year’s DevDay cannot be ignored. Instead of rolling out a flashy new model at the big annual conference to make headlines and generate hype, OpenAI is ensuring its foundation is strong. These updates will encourage external innovation and generate insightful feedback from users. These updates could kick-start a growth period for OpenAI, causing speech-to-speech and image-focused models to trend. At the very least, other AI engineers and businesses will look toward this company as inspiration. Similar updates from competitor models will likely follow in the near future. As for the implications of Realtime API, prompt caching, vision fine-tuning, and model distillation, the one-day conferences in London and Singapore will likely be microcosms of what’s to come. Attendees and livestream viewers should expect to see developer-created content, insights, and questions featured at the upcoming events.

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

5 Key Updates Unveiled at OpenAI's DevDay 2024

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

6 Key Learnings From the 2025 AI Governance Survey

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

6 Key Learnings From the 2025 AI Governance Survey

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps