Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
Last week felt like a lifetime in AI. Sure, the week before was fast-paced and so was the week before that, but this one — really, a lifetime. And somehow I just couldn’t let it go. While the rest of the world went about its business, I noodled about the implications of OpenAI’s latest ChatGPT chess move.
You’ve likely heard by now that OpenAI unveiled plugins that link its conversational AI ChatGPT to the real world. Developers can connect their apps and data to ChatGPT, letting it summon information (stock ticks, headlines), regurgitate docs, or act on your behalf (book travel, place takeout orders).
The news immediately got Twitter buzzing, with many seeing the initial list of available ChatGPT plugins, including Expedia, Instacart, Zapier and OpenTable, as a signal of OpenAI‘s ambitions to further its dominance by turning ChatGPT into a developer platform. Others marveled at the simple yet powerful fact that the plugins make it possible for ChatGPT to browse the internet for real-time information.
So how was my weekend? Well, I considered some of the other big ways ChatGPT plugins will shift the AI landscape. Here are five possibilities:
1. Help minimize ChatGPT hallucinations
According to OpenAI, plugins offer the potential to tackle various challenges associated with large language models, including “hallucinations,” keeping up with recent events, and accessing (with permission) proprietary information. By integrating explicit access to external data — such as up-to-date information online, code-based calculations, or custom plugin-retrieved information — language models can strengthen their responses with evidence-based references.
But one software developer and AI researcher, Chomba Bupe, pointed out on Twitter that while the plugin API itself may be factual, ChatGPT can “still hallucinate when formulating a prompt from the user & it can further introduce unwanted details when reinterpreting results.”
2. Share data to gather data
I had an interesting talk today with Tim Hwang, founder and CEO of FiscalNote, whose plugin is one of the initial batch that OpenAI announced on Thursday. FiscalNote aggregates legislation, regulations and government filings from thousands of federal, state and local agencies; uses AI models to structure it and normalize it; and delivers personalized data feeds to company clients.
FiscalNote partnered with OpenAI on its plugin partly as a way to combat misinformation with solid data about government filings. “I do feel like the data that we’re contributing is very important to making sure that the technology is able to deliver factual outcomes,” said Hwang.
But, he added, the plugin also adds value to FiscalNote, because the company receives data from OpenAI on ChatGPT queries, which it can then add to its own AI models. “We serve up a portion of our data to OpenAI. We don’t open up the entire firewall,” he said. “We can see what people are querying at any given time, which enables us to go back and refine our own data collection efforts and dig deeper into areas that people want.”
3. Put the future of websites in peril
What does it mean if ChatGPT can crawl websites for their content for free? Will it kill websites and website traffic, for example, if fewer people browse websites since they can get a direct answer right from ChatGPT?
As Riad Benguella, an Automattic engineer, blogged a couple of weeks ago: “Why would I open a browser if I can just talk to a bot and get an instant customized reply?”
4. Create security issues for sensitive data
A few days before the ChatGPT plugins were announced, OpenAI confirmed a glitch that made it possible for some users to see other customers’ credit card details.
But users of a Y Combinator forum pointed out the potential risks of ChatGPT plugins when it comes to security: “It is possible to use these unreleased plugins by setting up match-and-replace rules through an HTTP proxy. There are only client-side checks to validate that you have permission to use the plugins and they can be bypassed,” said one user.
5. Hello, vector databases
OpenAI released three of its own ChatGPT plugins, including an open-source knowledge-retrieval plugin “to be self-hosted by any developer with information with which they’d like to augment ChatGPT.” It allows users to “obtain the most relevant document snippets from their data sources, such as files, notes, emails or public documentation, by asking questions or expressing needs in natural language.”
For plugins retrieving information, developers need to access a vector database that indexes and searches documents and acts as a “long-term memory” for the application. The ChatGPT plugin allows developers to choose one of several vector database options, including Pinecone. “Everyone is rushing to build [a ChatGPT plugin] now,” said Pinecone VP of marketing Greg Kogan. “Companies who build plugins will need this vector database component, this long-term memory.”
