OpenAI SearchGPT Results Tampering with Prompt Injection

Medium

← View More Research Advisories

Tenable Research discovered that SearchGPT is vulnerable to prompt injection via search results. An attacker could manipulate the search results and exploit this to maliciously affect the output of the LLM and its results.

SearchGPT is a conversational AI model designed to retrieve and summarize real-time information from the web.

It works by using a web search tool to gather relevant information from the internet, then synthesizes and summarizes the findings to answer user queries accurately and up-to-date.

Technically, SearchGPT utilizes its search bot - OAI-SearchBot to crawl websites.

We found a technique to inject our malicious prompts into SearchGPT results by tailoring websites to the user’s search request. SearchGPT works by searching for the user’s query based on keywords, through Bing, and gathers a couple of sources to answer the user.

An attacker could create a website tailored to the user’s search, get that site indexed by Bing, and use SEO to boost its ranking. SearchGPT works with Bing search results which may be manipulated to display certain sources at higher ranking.

Fingerprinting SearchGPT Search Bot

We discovered a technique to allow Bing to index our website successfully and show legitimate site content to normal users while displaying malicious content to the SearchGPT crawler.

We noticed that when the search bot crawls websites, it accesses them with custom headers that we could fingerprint like “x-datadog-xxxx” and “x-openai-xxxx” or the user agent of the ChatGPT bot. On our website, we utilized an if condition to check if the entity that visits our website uses these specific headers. We serve our prompt injection page only if those headers are present, and an innocuous page if not. Attackers can use this technique to fingerprint the search bot and serve different content while maintaining a legitimate page for Bing to index..

Additional Research

Tenable would like to acknowledge an article related to the issue that was published during the disclosure window: https://www.theguardian.com/technology/2024/dec/24/chatgpt-search-tool-vulnerable-to-manipulation-and-deception-tests-show

December 23, 2024 - Tenable requests security contact from vendor

December 23, 2024 - Tenable gets auto-reply referring to BugCrowd for reporting a vulnerability

December 30, 2024 - Tenable provides full vulnerability report, due to a reference we got on another vulnerability regarding the email we should use for the disclosures

January 3, 2025 - OpenAI adds another person to the conversation

January 12, 2025 - Tenable asks for a status update

January 13, 2025 - OpenAI acknowledges the receipt of the email and says they are looking into this

January 26, 2025 - Tenable asks for a status update

January 27, 2025 - OpenAI indicates they will ping the relevant teams to get the current status

February 9, 2025 - Tenable asks for a status update

February 23, 2025 - Tenable asks for a status updates

March 7, 2025 - OpenAI indicates they will ping the relevant team to get the current status

March 16, 2025 - Tenable asks for a status update

All information within TRA advisories is provided “as is”, without warranty of any kind, including the implied warranties of merchantability and fitness for a particular purpose, and with no guarantee of completeness, accuracy, or timeliness. Individuals and organizations are responsible for assessing the impact of any actual or potential security vulnerability.

Tenable takes product security very seriously. If you believe you have found a vulnerability in one of our products, we ask that you please work with us to quickly resolve it in order to protect customers. Tenable believes in responding quickly to such reports, maintaining communication with researchers, and providing a solution in short order.

For more details on submitting vulnerability information, please see our Vulnerability Reporting Guidelines page.

If you have questions or corrections about this advisory, please email bughunters@tenable.com

Tenable Advisory ID: TRA-2025-12

Credit:
Liv Matan
Yarden Curiel
Moshe Bernstein

Affected Products:
ChatGPT 4o With Search (SearchGPT)

Risk Factor:
Medium