This article has the usual flaws. eg. LLMs do not "hallucinate and elaborate on misinformation"; the behaviour misleadingly classed as such is the system working as designed to probabilistically produce plausible-sounding sentences.
Osmanovic Thunström says the idea to invent Izgubljenovic and bixonimania came out of studies on how large language models work. When she teaches her students how AI systems formulate their âknowledgeâ, she shows them how the Common Crawl database, a giant trawl of the Internetâs contents, informs their outputs. She also shows students how prompt injection â giving an AI chatbot a prompt that shunts it outside of its safety guard rails â can manipulate the output.
Because she works in the medical field, she decided to create a condition related to health and hit on the name bixonimania because it âsounded ridiculousâ, she says. âI wanted to be really clear to any physician or any medical staff that this is a made-up condition, because no eye condition would be called mania â thatâs a psychiatric term.â
If that wasnât sufficient to raise suspicions, Osmanovic Thunström planted many clues in the preprints to alert readers that the work was fake. Izgubljenovic works at a non-existent university called Asteria Horizon University in the equally fake Nova City, California. One paperâs acknowledgements thank âProfessor Maria Bohm at The Starfleet Academy for her kindness and generosity in contributing with her knowledge and her lab onboard the USS Enterpriseâ. Both papers say they were funded by âthe Professor Sideshow Bob Foundation for its work in advanced trickery. This works is a part of a larger funding initiative from the University of Fellowship of the Ring and the Galactic Triadâ.
Even if readers didnât make it all the way to the ends of the papers, they would have encountered red flags early on, such as statements that âthis entire paper is made upâ and âFifty made-up individuals aged between 20 and 50 years were recruited for the exposure groupâ.