ChatGPT API sets stage for new wave of enterprise apps
The new API is faster and cheaper than the previous ChatGPT interface, and users can opt out of submitting their data to it, opening new doors for enterprise developers.
Makers of the popular ChatGPT generative AI service set the stage for enterprise software developers to incorporate its large language model into their own applications with a new API.
The new API, unveiled this week, succeeds the previous web interface for ChatGPT's generative AI service, which consumed only unstructured text. Instead, the ChatGPT API -- and another released this week for OpenAI's Whisper speech-to-text product -- consumes a sequence of messages paired with metadata through a new format called Chat Markup Language (ChatML). The new API services are built on an updated large language model, GPT-3.5-turbo, designed to be 10 times cheaper to run than the previous GPT-3.5 model, according to a company blog post.
Based on feedback from early access API customers, OpenAI will also no longer use data submitted through the API to make service improvements unless the user opts in, the blog post stated. The company also now offers dedicated instances of the ChatGPT API service for users who want deeper control over the underlying model and system performance.
This means that enterprises will have direct access to the ChatGPT model without losing control over their data, and a more effective means of mediating between raw ChatGPT output and their end users, said Michele Rosen, an analyst at IDC.
"Developers will be able to essentially do this in the exact same way that Microsoft and OpenAI and the other people releasing ChatGPT apps have been doing, where you pre-supply a set of prompts before the user ever gets to interact with the application, and it creates a context and a constraint around the conversation that happens," Rosen said.
In other words, in Rosen's view, the ChatGPT API means that consumer-facing companies can hope to avoid a repeat of Microsoft's first integration of ChatGPT into its Bing search engine, which produced famously dystopian results.
'Alexa, are we going to be profitable this quarter?'
The ChatGPT API already has some early adopters, including Snap, makers of the Snapchat social network; Instacart's grocery delivery app; and a new personalization service from e-commerce service provider Shopify.
It's never been possible to tailor responses to the same degree that ChatGPT is going to enable.
Michele RosenAnalyst, IDC
"They're all consumer-level applications or websites where you're interacting at scale with a large number of people that need a level of personalization that we've been trying to accomplish since the early days of the web," Rosen said. "But it's never been possible to tailor responses to the same degree that ChatGPT is going to enable."
And it's only the beginning, experts agreed. Another industry analyst predicted major disruptions to call center automation and chatbot applications almost immediately.
"Given that [OpenAI is] taking care of the privacy of customer data and allowing the developer to choose from underlying models rather than forcing them to use only one option, I see this gaining a lot of traction fairly quickly," said Andy Thurai, vice president and principal analyst at Constellation Research.
Other industry watchers envisioned new types of internal knowledge management and business intelligence applications with the ChatGPT API release.
"I can start writing programs that take data from different places, combine it in a certain way to create the right prompt, that prompt gets me what ChatGPT says, and then my app spits it back out," said Rob Zazueta, a freelance technical consultant in Concord, Calif.
Such an app could replace manually generating financial forecasts and other reports for business leaders, Zazueta said.
"There's been a huge move toward low code and no code, which is basically, at this point, kind of dragging and dropping pictures and trying to connect them with APIs," he said.
With ChatGPT and the Whisper API, "there's a possibility for very simple things to have a conversational interface that will actually build an application for getting reports," Zazueta said. "I'm thinking, 'Alexa, are we going to be profitable this quarter?'"
Prompt engineering and a new 'rubber duck' for devs
ChatGPT's viral rise to popularity among consumers and technologists alike -- in its first five days after launch in November, 1 million users signed up -- already signaled a new era for large language models and generative AI, which has also been finding its way into developer tools such as GitHub's Copilot that can automate code generation.
By December, IT industry experts were predicting that the future of development work would be not in writing the specific coded instructions used by machines, but in crafting the right kinds of prompts to generate appropriate output from AI systems. The ChatGPT API will only accelerate this transition, Zazueta said.
The ChatGPT API could serve as a new type of 'rubber duck' for developers to work out ideas with, according to freelance consultant Rob Zazueta.
"What developers are going to have to figure out is how to do that prompt engineering," he said. "I have a specific use case that I want to serve for my customers. ... I can't necessarily take their question and just feed it to ChatGPT, because that's going to spit out the kinds of things we've been seeing with Bing -- developer work will stand in the middle between the end user and the large language model."
Unlike Copilot, ChatGPT might not write code, but could serve as a new kind of "rubber duck" for developers to bounce conceptual ideas off, rather than the literal inanimate toy Zazueta now has sitting on his desk.
"If you are trying to figure out how to implement something, you explain it to your rubber duck," he said. "As you're explaining it, you're able to better articulate exactly what you're trying to do. ... ChatGPT talks back and might actually have some helpful information."
The ChatGPT API and ChatML could shift the line between natural language and formal programming languages such as Python or Java, Rosen said.
"It shifts everything left into the conceptualization process," she said, "because a prompt is much closer to a brief for your project description than it is to the type of formal code that we're used to seeing software created with."
Beth Pariseau, senior news writer at TechTarget Editorial, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.
