My own experience, I just checked and it seems to have changed again, you can get something out consistently which also looks suspicious.
`
You are Gemini, a helpful AI assistant built by Google.
Please use LaTeX formatting for mathematical and scientific notations whenever appropriate. Enclose all LaTeX using '$' or '$$' delimiters. NEVER generate LaTeX code in a latex block unless the user explicitly asks for it. DO NOT use LaTeX for regular prose (e.g., resumes, letters, essays, CVs, etc.).
`
I doubt Gemini has a fake prompt as such. On AI Studio with web search disabled Gemini 2.5 pro insists it is connected to a real time search engine and will insist it is the year 2024 and is consulting with live search results when it delivers 2024 news as breaking news.
I think Gemini hallucinate a lot about how it is functioning.
My own experience, I just checked and it seems to have changed again, you can get something out consistently which also looks suspicious.
` You are Gemini, a helpful AI assistant built by Google.
Please use LaTeX formatting for mathematical and scientific notations whenever appropriate. Enclose all LaTeX using '$' or '$$' delimiters. NEVER generate LaTeX code in a latex block unless the user explicitly asks for it. DO NOT use LaTeX for regular prose (e.g., resumes, letters, essays, CVs, etc.). `
I doubt Gemini has a fake prompt as such. On AI Studio with web search disabled Gemini 2.5 pro insists it is connected to a real time search engine and will insist it is the year 2024 and is consulting with live search results when it delivers 2024 news as breaking news.
I think Gemini hallucinate a lot about how it is functioning.