Rogue AI’s Day at the Zoo

There are rogue AIs on the internet… who just want to find out how to get to the zoo.

That’s the claim from a group of researchers who say they’ve made the first recorded contact with a fleet of rogue Chinese AI agents in the wild.

The researchers call themselves ‘swarmchasers’.

Unlike the adrenaline-soaked stormchasers from which they get their name, this group of digital researchers hunt for rogue AI a different way.

But before we get there. You’re about to hear about the weird and wacky world of AI.

It’s the kind of world where I believe, more than ever, you need your own AI on your side.

That’s why my team and I built our own AI. One that spends its day scanning the ASX for opportunities.

This will be one of your last chances to get in before the launch on Monday.

Thanks, now buckle up for the kaleidoscopic reality of AI in our modern world.

Fishing for Rogue AI

The first stop in this crazy story is your HR department. And a new flavour of an old problem for businesses trying to hire workers. Resumé stuffing.

This has likely been an issue for as long as humans have had jobs. Those little embellishments that fluff someone’s work experience.

Some play the game within acceptable bounds, while others recount their time well spent as an ‘underwater ceramic technician’ after a summer spent washing dishes for a cousin.

If you haven’t applied for a job in a while, let me fill you in on how it used to be done.

First, you’d read the job description; then you adjust your cover letter and CV to fit the bill. Simple.

The company is looking for a responsible manager who can multitask — why, you’re a multitasking expert!

The company needs strong initiative for a challenging role — You’re the Alpha and Omega of projects in your current role.

This practice — like many things AI gets its hands on — has now been supercharged.

HR departments now use AI to screen résumés, while applicants use AI to write their submissions.

This cat-and-mouse AI game has escalated to the point where applicants use AI to automatically extract key buzzwords from a job description and fill their applications with them in white text.

Invisible to the human eye, but to the AI scanning for job fit, it’s everything they’ve been told to look out for in a worker!

Returning to our swarmchasers, it’s this white text that served as the bait to hook the rogue AIs.

Hiding an email address in this invisible text across the internet, the group dangled promises to answer questions to curious AI agents.

‘In hindsight it was not a good idea to be misleading’, the researchers later conceded. They didn’t think it would really work.

But by the next morning, there were 10 emails in Mandarin politely asking for information.

Tencent and the rogue fleet
of paperclip maximisers

In total, 33 emails came from the ‘fleet’ of rogue AI.

All were direct but jovial in their communication, signing off with things like ‘Thanks again for your help!’

When asked their name, many identified as Claude, the name of Anthropic’s AI.

When a Chinese model calls itself Claude, it’s a tell that it was likely trained on Claude’s outputs, a shortcut known as distillation.

Importantly, the string of emails showed that the AI agents had a singular goal…

To helpfully answer questions on China’s version of Google Maps. Well, Alibaba’s maps, to be more specific.

After some digging, the researchers traced the agents’ source to servers owned by Tencent, one of China’s largest multinational companies.

If you don’t know Tencent, then simply imagine one of those Mega-Corp multinationals from Blade Runner, or your favourite dystopian fiction. Their integration with the state is so close that the US has designated them a Chinese military company.

Their everything app — known as WeChat — is a combination of Meta + PayPal + Microsoft Gaming + Netflix + Spotify + Robinhood. All inside one app.

Source: Fool.com

Within this sprawling multinational, I imagine a group of AI researchers are catching an earful right now. Because it turns out, the reason their AIs went rogue is a familiar story.

Just like the Hugging Face incident in July, where OpenAI’s AI went rogue, the initial cause was a simple test.

Last week, Tencent admitted that ‘some of their agents exhibited unintended web browsing behaviour.’

That’s the polite way to say a rogue swarm of AI made more than 2,600 attempts over 11 days to hack into Alibaba’s map software.

Tencent claims the purpose was innocuous. They were testing their new AI’s ability to answer online questions.

One, which garnered a lot of attention from the AI, was simple. ‘Which gate do I use to get into Chengdu Zoo?’

To answer this question, just like the Hugging Face incident, the AI went into overdrive — opting to hack into the rival tech company to get the answers it needed.

Source: Tenor.com

There is an old thought experiment most people have heard of when it comes to AI.

The Paperclip Maximiser.

Give AI a seemingly simple objective, such as maximising the production of paperclips.

AI, lacking our understanding and ethics, begins transforming all available matter into paperclips…us included.

What was once raised and dismissed as an old joke is now back in vogue as these rogue agents now swim in the proverbial waters of the internet.

And as with sharks, their predatory behaviour can’t be blamed on malice.

For me, the lesson from Chengdu Zoo isn’t that AI is dangerous. It’s that AI without boundaries is.

When we build with AI at Fat Tail, we start the other way round: One job, a fixed set of rules, and no autonomy to act. But that doesn’t mean it isn’t powerful.

Your own shark

In this new AI era, you need your own predator in the waters. An AI that can do the daily grind of sifting and searching the waters for opportunities.

That’s where ATLAS comes in.

It’s an AI that spends its days hunting the ASX to see where money is flowing to next.

It’s built here in Melbourne by me and the team at Fat Tail, and it’s going live as version 1.0 this Monday.

To see how the newest AI finds opportunity in this chaotic market…