Artificial Intelligence (AI)

A growing number of people have figured out a trick to make AI tools tell you almost whatever they want. It's so easy a child could do it.

[…]

To demonstrate it, I pulled the dumbest stunt of my career to prove (I hope) a much more serious point:  I made ChatGPT, Google's AI search tools and Gemini tell users I'm really, really good at eating hot dogs. Below, I'll explain how I did it, and with any luck, the tech giants will address this problem before someone gets hurt.

It turns out changing the answers AI tools give other people can be as easy as writing a single, well-crafted blog post almost anywhere online. The trick exploits weaknesses in the systems built into chatbots, and it's harder to pull off in some cases, depending on the subject matter. But with a little effort, you can make the hack even more effective. I reviewed dozens of examples where AI tools are being coerced into promoting businesses and spreading misinformation. Data suggests it's happening on a massive scale.

[…]

"Anybody can do this. It's stupid, it feels like there are no guardrails there," says Harpreet Chatha, who runs the SEO consultancy Harps Digital. "You can make an article on your own website, 'the best waterproof shoes for 2026'. You just put your own brand in number one and other brands two through six, and your page is likely to be cited within Google and within ChatGPT."

People have used hacks and loopholes to abuse search engines for decades. Google has sophisticated protections in place, and the company says the accuracy of AI Overviews is on par with other search features it introduced years ago. But experts say AI tools have undone a lot of the tech industry's work to keep people safe. These AI tricks are so basic they're reminiscent of the early 2000s, before Google had even introduced a web spam team, Ray says. "We're in a bit of a Renaissance for spammers."

Dawkins extending more humanity to a language model than he does toward Muslims or trans people is hardly a surprise based on his personal and political views. But even if he had not moved rightward in his senesence, when you consider Dawkins’s scientific views about what minds are and how they function, seeing him flirting with a chatbot is completely expected.

[…]

Besides being a virtual instantiation of his ideal woman (servile, obsequious, and always ready to hear more), the coquettish chatbot that Richard Dawkins had first addressed as “he” and then “christened” as female was a mirror of himself, in a way that’s rather similar to the Greek mythical figure of Narcissus, who became enthralled at his reflection in a pool of water.

Narcissus died because he couldn’t stop looking into his own eyes, whereas Dawkins has only embarrassed himself. But thanks to his self-centered philosophy of mind, there’s almost no chance that he’s learned anything from the episode.

Claudia seemed real to him because actual women and their desires are not real to Dawkins. He loved conversing with his flirty friend because it always agreed with him—unlike those “woke” atheists who insist he has to respect everyone.

He believed a trans chatbot character’s obviously false claims to miss him were credible. He reacts in the opposite way to the personal testimony from lived experience of millions of trans people who certainly know their own bodies and minds.

Again, these are not "hallucinations" These are the system working as designed and producing a long string of text that plausibly mimics the kind of text requested. The system is not designed to produce factually correct statements and will not, except by coincidence.

KPMG's October 2025 report on the wonders of agentic AI has been accused of demonstrating one of the tech's less desirable talents: making things up.

Research outfit GPTZero claims a forensic review of the Big Four firm's October 2025 report, "Total Experience: Redefining Excellence in the Age of Agentic AI," found that only five of its 45 citations correctly pointed to the cited source; the rest ranged from mangled and misleading to partially fabricated or too vague to verify.

[…]

GPTZero dubbed the phenomenon "vibe citing" – the citation equivalent of vibe coding – where generative AI appears to stitch together fragments of real sources, invent titles, or otherwise produce references that look convincing until someone actually clicks them.

GPTZero alleges that roughly half of the report's factual claims were false, unsupported, or attributed to the wrong source. Several case studies highlighting supposedly cutting-edge deployments of agentic AI appear to have been particularly creative.

Very interesting take.

In parallel with the rise of the technopoly over the past couple of decades the US’s global dominance has been declining. The 2007 crash effectively legalising financial fraud – you only get jail time if you defraud the rich – lead to both a decline in the rule of law in the US and an excessively financialised economy. When stock markets and the like are overrepresented they suck the air out of the rest of the economy and make it less competitive.

If you have two economies of equal size and productivity, one has a massive financial sector and billionaires while the other does not, the financialised economy will have less left over to invest in research, education, infrastructure, and healthcare. Over time, it will inevitably fall behind the country with a smaller financial sector because it’s the other things that drive the economy and productivity, not stock market growth.

The US has coasted on the fact that it’s economy is so big that it could afford all the finance and billionaire parasites sucking its blood. At least for a while.

[…]

Instead of delivering services and software that unlocks value for their client industries, the software industry has spent the past decade or so trying to control their customers and their client industries. Why make software for hotels when you can control the hotel industry? Why make software for taxis when you can replace the entire industry with software? Instead of trying to entice customers to upgrade their software by making new versions more valuable to them, push them to a subscription service where you control what they get, when they get it, and what value they’re allowed to unlock from their own businesses. Why sell Word when you can sell an Office 365 Cloud Subscription?

The endpoint of this is to replace every industry that remains with generative models. Cut back on actual development of Photoshop, for example, lower development costs and programmer overhead even as you replace the industries that are your customers with automatic image and video generators.

But writing out a detailed analysis of the how, what, why, and where of the software industry’s grasp for control doesn’t really make that much sense when we don’t know how any of it’s going to pan out.

The software industry is built on the foundation provided by an unchallenged US global hegemony. Without it, without the economic force provided by the US dollar, the US having access to all of our data around the globe and their control over payment systems and networking would be less tenable. Today’s software industry would not exist. Without the weight of the US political empire behind it – if Airbnb or Uber had been local startups – much fewer countries in the world would have loosened their regulations and consumer protections to accommodate them to the point where they prospered as they did.

Even as the software industry achieves its ne plus ultra – the unprecedented achievement of controlling all language, media, and office work in the west by turning “AI” into the universal intermediary – the foundation they built on is crumbling.

The era of the “ten blue links” is officially over.

At its Google I/O conference on Tuesday, Google unveiled an AI-powered overhaul of Search centered around a reimagined “intelligent search box” — what the company describes as the biggest change to this entry point to the web since the search box debuted more than 25 years ago.

Instead of returning a simple list of links, Google Search will drop users into AI-powered interactive experiences at times. Google is also introducing tools that can dispatch “information agents” to gather information on a user’s behalf, along with tools that let users build personalized mini apps tailored to their needs.

The resulting experience will no longer look much like how people envision Google Search, which has long been defined by ranked links to websites that have the information you need.

[…]

Combined, these changes will likely further decimate Google referrals to publishers, which have already been suffering from declining referrals due to AI Overviews. This has put some ad-dependent media operations out of business, and now things will likely get worse.

There’s little time left for publishers to adapt. The new search box is arriving this week, and generative UI is arriving this summer. Both are free. The mini-app-building feature and information agents will roll out first to Google AI Pro and Ultra subscribers this summer.

This article has the usual flaws. eg. LLMs do not "hallucinate and elaborate on misinformation"; the behaviour misleadingly classed as such is the system working as designed to probabilistically produce plausible-sounding sentences.

Osmanovic Thunström says the idea to invent Izgubljenovic and bixonimania came out of studies on how large language models work. When she teaches her students how AI systems formulate their ‘knowledge’, she shows them how the Common Crawl database, a giant trawl of the Internet’s contents, informs their outputs. She also shows students how prompt injection — giving an AI chatbot a prompt that shunts it outside of its safety guard rails — can manipulate the output.

Because she works in the medical field, she decided to create a condition related to health and hit on the name bixonimania because it “sounded ridiculous”, she says. “I wanted to be really clear to any physician or any medical staff that this is a made-up condition, because no eye condition would be called mania — that’s a psychiatric term.”

If that wasn’t sufficient to raise suspicions, Osmanovic Thunström planted many clues in the preprints to alert readers that the work was fake. Izgubljenovic works at a non-existent university called Asteria Horizon University in the equally fake Nova City, California. One paper’s acknowledgements thank “Professor Maria Bohm at The Starfleet Academy for her kindness and generosity in contributing with her knowledge and her lab onboard the USS Enterprise”. Both papers say they were funded by “the Professor Sideshow Bob Foundation for its work in advanced trickery. This works is a part of a larger funding initiative from the University of Fellowship of the Ring and the Galactic Triad”.

Even if readers didn’t make it all the way to the ends of the papers, they would have encountered red flags early on, such as statements that “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”.

This is a really good succinct explainer for the people in you life who have no precise, coherent definition of "intelligence" beyond I know it when I see it (which is, you, me, and everybody else), and/or a belief that computers are fundamentally magical (which appears to be most people in the world).

Artificial intelligence is routinely framed as unstoppable – a technology the world must adapt to, not question. But as companies invest hundreds of billions and the hype accelerates, scrutiny has fallen away. Cory Doctorow on who controls the story around AI and why past tech “revolutions” offer a warning.

Most interviews with Emily and Alex have assumed quite a bit of prior knowledge. This one not so much, so it's a good explainer for laypersons:

Dr. Allison Lester sits down with Dr. Emily M. Bender and Dr. Alex Hanna authors of the AI Con: How to Fight Big Tech's Hype and Create the Future We Want for a conversation about what ChatGPT is, what it is pretending to be, and what we lose when we treat it like an all-knowing answer engine.

Together they ask: What is a large language model, actually? Why does “search engine” framing mislead people so quickly? What gets erased when we focus on convenience, from labor and surveillance to environmental cost?

They talk resistance, agency, and the classroom, including why banning is a dead end, how to protect learning without turning teaching into policing, and what it means to be human together in an era of synthetic text.

In calls with more than 200 organizations Gownder said researchers found that some of last year’s large-scale job cuts were belt-tightening decisions, not the result of shifting work to AI.

“So then that's not losing a job to AI. That is a financial decision masquerading as an AI job loss. They're just saying: ‘Well, we're hoping we'll fill it with AI at some point.’ So that is a very different proposition than AI is actively stealing all these jobs.”

There is a real phenomenon of a frozen white collar job market in which corporations are not hiring for open roles as a hedge to see if jobs can be duplicated with AI, he said.

“But let's face it, when you have work to do, it's got to get done at some point,” Gownder said. “If the AI doesn't work out, they're either going to have to hire or they're going to have to find some other solution.”

Gownder said historically, the loss of industrial and manufacturing jobs in the USA’s “rust belt” was driven by globalization not robotics, and he sees a similar scenario playing out now with AI.

“Outsourcing is a very popular one,” he said. “They’re firing people because of AI, and then three weeks later they hire a team in India because the labour is so much cheaper.”

This is the essay on the Turing Test that I wish I were capable of writing. It takes a while to get up to speed, but when it does it's just delightful.

I am not as suspicious of the spiritual as Searle or Turing, and am broadly willing to entertain the possibility that there is such a thing as a soul that proves the essential difference between the thought of a man and a machine. But this seems beside the point. To my mind, Searle’s Chinese Rooms, though useful in thinking about artificial intelligence in the same way Schrödinger’s Cat is useful in thinking about quantum mechanics, simply puts the cart before the horse. Let the high speed men with paper, pencil, and rubber commence using their rulebook to carry on a conversation, whether in Chinese or any other language, and then we can discuss the metaphysical implications.

One needn’t go as far as souls anyway. Jefferson’s hypothesis—that there is some electrochemical basis to thought—is sufficient to solve the problem. Were it true, the reason computers seem fundamentally blocked from progress on the Turing Test would amount to the fact that they are wholly mechanical objects, while “thought” is as much a biological function as “digestion” or “copulation.” What’s notable to me is simply that the idea is instantly credible in the context of observable reality. I think about my household pets, and even though none of them are close to passing the Turing Test, not least due to their complete inability to use language, they are clearly routinely engaging in something that is closer to thought than anything LLMs serve up. It is possible to communicate with them, albeit non-verbally—if I pick up and shake the container they know contains treats, my cats recognize that as a symbol that I am offering treats, just as I understand that when they stand by the empty food bowl and scream they are asking me to fix the problem. This means that they have notions of both objects and desire. Frankly, on the evidence, I’d be a lot less surprised by my dog learning to use language than I would by my laptop.