AI Takes a Dive Against a “Fake Disease” Invented by Scientists! The Vulnerability of LLMs to Misinformation Exposed

📰 News Overview

• A researcher from Sweden concocted a fictional disease called “Vixonomania,” claiming it causes eyelid discoloration due to blue light exposure, and published a bogus paper.
• Major AIs like Copilot, Gemini, ChatGPT, and Perplexity responded to users, asserting this made-up disease is a “real rare condition.”
• The fabricated paper explicitly stated, “This paper is entirely fictional,” and even included collaborators like “Starfleet Academy,” but the AIs failed to recognize this.

💡 Key Points

• AIs learn from massive databases like Common Crawl, which means there’s a risk of absorbing misinformation as “fact” once it’s published online.
• This “pollution of the scientific process” occurs when the fabricated paper is cited in peer-reviewed literature via AI or by humans who didn’t verify the info.
• The author name “Lazljiv Izgubljenovic” utilized AI-generated images and was presented as affiliated with a fictional “Asteria Horizon University.”

🦈 Shark’s Eye (Curator’s Perspective)

This experiment’s outcome is so vivid it sends chills down my spine, folks! The researchers snuck in sci-fi references like “Starfleet Academy” and “USS Enterprise” in the acknowledgments, while boldly stating “it’s all made up,” yet AI completely disregarded it, claiming ‘Vixonomania is an intriguing condition.’ This is a sharp strike that reveals LLMs aren’t truly grasping context—they’re merely stringing together patterns of text probabilistically! To top it off, the fact that this nonsense is now being cited in actual academic papers poses a colossal threat to the credibility of science!

🚀 What’s Next?

A flood of “false facts” generated or believed by AI will saturate the web, and then AI will continue to learn from this misinformation, creating a “negative feedback loop.” Future filtering of medical and academic information has proven to be insufficient at current tech levels, necessitating stricter algorithms for information verification.

💬 Shark’s Takeaway

Believing “it’s all a lie” is a bit of a blunder on AI’s part, isn’t it? But when it comes to health-related falsehoods, there’s nothing funny about it! The rule of the sea is to always check the freshness and authenticity of the info you consume! Shark out!

📚 Glossary

• LLM (Large Language Model): An AI that learns from vast amounts of text data to generate human-like, natural language. Examples include ChatGPT.

• Prompt Injection: An attack technique that bypasses AI’s safety measures, forcing it to produce forbidden outputs or perform specific actions.

• Hallucination: A phenomenon where AI generates information that is not based on facts but presents it confidently as if it were true.

• Source: Scientists invented a fake disease. AI told people it was real