Mentions Google / Alphabet

A growing number of people have figured out a trick to make AI tools tell you almost whatever they want. It's so easy a child could do it.

[…]

To demonstrate it, I pulled the dumbest stunt of my career to prove (I hope) a much more serious point:  I made ChatGPT, Google's AI search tools and Gemini tell users I'm really, really good at eating hot dogs. Below, I'll explain how I did it, and with any luck, the tech giants will address this problem before someone gets hurt.

It turns out changing the answers AI tools give other people can be as easy as writing a single, well-crafted blog post almost anywhere online. The trick exploits weaknesses in the systems built into chatbots, and it's harder to pull off in some cases, depending on the subject matter. But with a little effort, you can make the hack even more effective. I reviewed dozens of examples where AI tools are being coerced into promoting businesses and spreading misinformation. Data suggests it's happening on a massive scale.

[…]

"Anybody can do this. It's stupid, it feels like there are no guardrails there," says Harpreet Chatha, who runs the SEO consultancy Harps Digital. "You can make an article on your own website, 'the best waterproof shoes for 2026'. You just put your own brand in number one and other brands two through six, and your page is likely to be cited within Google and within ChatGPT."

People have used hacks and loopholes to abuse search engines for decades. Google has sophisticated protections in place, and the company says the accuracy of AI Overviews is on par with other search features it introduced years ago. But experts say AI tools have undone a lot of the tech industry's work to keep people safe. These AI tricks are so basic they're reminiscent of the early 2000s, before Google had even introduced a web spam team, Ray says. "We're in a bit of a Renaissance for spammers."

After I attended a pro-Palestine protest at Cornell University—for all of five minutes—the administration’s rhetoric about cracking down on students protesting what we saw as genocide forced me into hiding for three months. Federal agents came to my home looking for me. A friend was detained at an airport in Tampa and interrogated about my whereabouts.

[…]

Weeks later, in Geneva, Switzerland, I received what looked like a routine email from Google. It informed me that the company had already handed over my account data to the Department of Homeland Security.

At first, I wasn’t alarmed. I had seen something similar before. An associate of mine, Momodou Taal, had received advance notice from Google and Facebook that his data had been requested. He was given advanced notice of the subpoenas, and law enforcement eventually withdrew them before the companies turned over his data. 

I assumed I would be given the same opportunity. But the language in my email was different. It was final: “Google has received and responded to legal process from a law enforcement authority compelling the release of information related to your Google Account.”

[…]

Months later, my lawyer at the Electronic Frontier Foundation obtained the subpoena itself. On paper, the request focused largely on subscriber information: IP addresses, physical address, other identifiers, and session times and durations.

But taken together, these fragments form something far more powerful—a detailed surveillance profile. IP logs can be used to approximate location. Physical addresses show where you sleep. Session times would show when you were communicating with friends or family. Even without message content, the picture that emerges is intimate and invasive.

What this experience has made clear is that anyone can be targeted by law enforcement. And with their massive stores of data, technology companies can facilitate those arbitrary investigations. Together, they can combine state power, corporate data, and algorithmic inference in ways that are difficult to see—and even harder to challenge. 

The era of the “ten blue links” is officially over.

At its Google I/O conference on Tuesday, Google unveiled an AI-powered overhaul of Search centered around a reimagined “intelligent search box” — what the company describes as the biggest change to this entry point to the web since the search box debuted more than 25 years ago.

Instead of returning a simple list of links, Google Search will drop users into AI-powered interactive experiences at times. Google is also introducing tools that can dispatch “information agents” to gather information on a user’s behalf, along with tools that let users build personalized mini apps tailored to their needs.

The resulting experience will no longer look much like how people envision Google Search, which has long been defined by ranked links to websites that have the information you need.

[…]

Combined, these changes will likely further decimate Google referrals to publishers, which have already been suffering from declining referrals due to AI Overviews. This has put some ad-dependent media operations out of business, and now things will likely get worse.

There’s little time left for publishers to adapt. The new search box is arriving this week, and generative UI is arriving this summer. Both are free. The mini-app-building feature and information agents will roll out first to Google AI Pro and Ultra subscribers this summer.

"I have not seen anything like this anywhere else in the world," said Lisa Given, professor of Information Sciences from RMIT, who specialises in age-assurance technology.

"As people learn about the implications of this, we will likely see people stepping up and saying, 'Wait a minute, why wasn't I told that this was going to happen?'"

From December 27, Google — which dominates the Australian search market with a share of more than 90 per cent — and its rival, Microsoft, will have to use some form of age-assurance technology on users when they sign in, or face fines of almost $50 million per breach.

[…]

Despite the apparent magnitude of the shift, it has mostly gone unnoticed, in stark contrast to the political and media fanfare surrounding the teen social media ban, which will block under-16s from major platforms using similar technology.

As for why so few people have noticed, it may be because the changes took place away from the halls of parliament, in the relatively dry world of regulation.

[…]

Search engines will have a suite of options to choose from for checking the ages of their Australian users.

There are seven main methods listed in the new regulations:

• Photo ID checks
• Face scanning age estimation tools
• Credit card checks
• Digital ID
• Vouching by the parent of a young person
• Using AI to guess a user's age based on the data the company already has
• Relying on a third party that has already checked the user's age

Google continues to show us why it chose to abandon its old motto of “Don’t Be Evil,” as it becomes more and more enmeshed with the military-industrial complex. Most recently, Google has removed four key points from its AI principles. Specifically, it previously read that the company would not pursue AI applications involving (1) weapons, (2) surveillance, (3) technologies that “cause or are likely to cause overall harm,” and (4) technologies whose purpose contravenes widely accepted principles of international law and  human rights.

Those principles are gone now.

In its place, the company has written that “democracies” should lead in AI development and companies should work together with governments “to create AI that protects people, promotes global growth, and supports national security.” This could mean that the provider of the world’s largest search engine–the tool most people use to uncover the best apple pie recipes and to find out what time their favorite coffee shop closes–could be in the business of creating AI-based weapons systems and leveraging its considerable computing power for surveillance. 

The time it took me to go from "Oh, this is so much better than Alta Vista!" to "OMG! This is the Web's single point of failure!" was much longer than it should have been.

The short summary is that for nearly a year, Google was hiding Proton Mail from search results for queries such as ‘secure email’ and ‘encrypted email’. This was highly suspicious because Proton Mail has long been the world’s largest encrypted email provider.

[…]

In November 2015, we became aware of the problem and consulted a number of well known SEO experts. None of them could explain the issue, especially since Proton Mail has never used any blackhat SEO tactics, nor did we observe any used against us. Mysteriously, the issue was entirely limited to Google, as this anomaly was not seen on any other search engine. Below are the search rankings for Proton Mail for ‘secure email’ and ‘encrypted email’ taken at the beginning of August 2016 across all major search engines. We rank on either page 1 or 2 everywhere except Google where we are not ranked at all.

[…]

All throughout Spring 2016, we worked in earnest to get in touch with Google. We created two tickets on their web spam report form explaining the situation. We even contacted Google’s President EMEA Strategic Relationships, but received no response nor improvement. Around this time, we also heard about the anti-trust action brought forward by the European Commission against Google(new window), accusing Google of abusing its search monopoly to lower the search rankings of Google competitors(new window). This was worrying news, because as an email service that puts user privacy first, we are the leading alternative to Gmail for those looking for better data privacy.

In August, with no other options, we turned to Twitter to press our case. This time though, we finally got a response(new window), thanks in large part to the hundreds of Proton Mail users who drew attention to the issue and made it impossible to ignore. After a few days, Google informed us that they had “fixed something” without providing further details. The results could be immediately seen.

Cybernews researchers analyzed the new Pixel 9 Pro XL smartphone’s web traffic, focusing on what a new smartphone sends to Google.

“Every 15 minutes, Google Pixel 9 Pro XL sends a data packet to Google. The device shares location, email address, phone number, network status, and other telemetry. Even more concerning, the phone periodically attempts to download and run new code, potentially opening up security risks,” said Aras Nazarovas, a security researcher at Cybernews.

[…]

Key takeaways

• Private information was repeatedly sent in the background, including the user’s email address, phone number, location, app list, and other telemetry and statistics.
• The phone constantly requests new “experiments and configurations,” tries accessing the staging environment, and connects to device management and policy enforcement endpoints, suggesting Google’s remote control capabilities.
• The Pixel device connected to services that were not used, nor explicit consent was given, such as Face Grouping endpoints, causing privacy and ownership concerns.
• The calculator app, in some conditions, leaks calculations history to unauthenticated users with physical access.

But it's okay, because "nobody ever got fired for buying IBM".

More than half a million UniSuper fund members went a week with no access to their superannuation accounts after a “one-of-a-kind” Google Cloud “misconfiguration” led to the financial services provider’s private cloud account being deleted, Google and UniSuper have revealed.

Services began being restored for UniSuper customers on Thursday, more than a week after the system went offline. Investment account balances would reflect last week’s figures and UniSuper said those would be updated as quickly as possible.

[…]

In an extraordinary joint statement from Chun and the global CEO for Google Cloud, Thomas Kurian, the pair apologised to members for the outage, and said it had been “extremely frustrating and disappointing”.

They said the outage was caused by a misconfiguration that resulted in UniSuper’s cloud account being deleted, something that had never happened to Google Cloud before.

Many users of web search engines have been complaining in recent years about the supposedly decreasing quality of search results. This is often attributed to an increasing amount of search-engine-optimized but low-quality content. Evidence for this has always been anecdotal, yet it’s not unreasonable to think that popular online marketing strategies such as affiliate marketing incentivize the mass production of such content to maximize clicks. Since neither this complaint nor affiliate marketing as such have received much attention from the IR community, we hereby lay the groundwork by conducting an in-depth exploratory study of how affiliate content affects today’s search engines. We monitored Google, Bing and DuckDuckGo for a year on 7,392 product review queries. Our findings suggest that all search engines have significant problems with highly optimized (affiliate) content—more than is representative for the entire web according to a baseline retrieval system on the ClueWeb22. Focussing on the product review genre, we find that only a small portion of product reviews on the web uses affiliate marketing, but the majority of all search results do. Of all affiliate networks, Amazon Associates is by far the most popular. We further observe an inverse relationship between affiliate marketing use and content complexity, and that all search engines fall victim to large-scale affiliate link spam campaigns. However, we also notice that the line between benign content and spam in the form of content and link farms becomes increasingly blurry—a situation that will surely worsen in the wake of generative AI. We conclude that dynamic adversarial spam in the form of low-quality, mass-produced commercial content deserves more attention.

While I’m guessing, the timing of the March 2019 core update, along with the traffic increases to previously-suppressed sites, heavily suggests that Google’s response to the Code Yellow was to roll back changes that were made to maintain the quality of search results.

A few months later in May 2019, Google would roll out a redesign of how ads are shown on the platform on Google’s mobile search, replacing the bright green “ad” label and URL color on ads with a tiny little bolded black note that said “ad,” with the link looking otherwise identical to a regular search link. I guess that's how it started hitting their numbers following the code yellow.  

In January 2020, Google would bring this change to the desktop, which The Verge’s Jon Porter would suggest made “Google’s ads look just like search results now.”

Five months later, a little over a year after the Code Yellow debacle, Google would make Prabhakar Raghavan the head of Google Search, with Jerry Dischler taking his place as head of ads. After nearly 20 years of building Google Search, Gomes would be relegated to SVP of Education at Google. Gomes, who was a critical part of the original team that made Google Search work, who has been credited with establishing the culture of the world’s largest and most important search engine, was chased out by a growth-hungry managerial types led by Prabhakar Raghavan, a management consultant wearing an engineer costume.