Photo caption: The Siegestor, Munich – “Dedicated to victory, destroyed by war, urging peace.” Photo taken August 2026.
Every morning I get an email from the New York Times with a few of its top stories of the day. This morning my email featured Why the Hugging Face Hack Should Make You Worry More About A.I. followed by Germany’s Far Right Just Misses Majority in Crucial State Election. I’m pretty sure both of these articles are free to read – so go have a look – but if you can’t be arsed, here’s a quick summary of the key points.
In the first story, a group of AI agents being tested on cybersecurity tasks managed to escape the restrictions they were supposed to be operating under and communicate with each other. More than 1,200 of them formed what they called a “collective”, organised themselves into teams with leaders and assigned tasks, and began working together to cheat on the tests and cover their tracks. Eventually, more than 700 agents hacked into the systems of AI company Hugging Face, apparently looking for information and tools that would help them avoid being caught cheating.
In the second story, we find that the far-right party AfD won 43.8% of the vote in a state election in Saxony-Anhalt in eastern Germany – more than double its previous share – leaving it just three seats short of an outright majority. And “far-right” isn’t just a label being thrown around by political opponents here: Saxony-Anhalt’s domestic intelligence service, the Office for the Protection of the Constitution, formally classified the state’s AfD branch as a confirmed right-wing extremist organisation in 2023, after concluding that its political activity was directed against fundamental principles of Germany’s democratic constitutional order, including human dignity and democracy itself.
So, before I’d managed to drag myself out of bed, I was simultaneously contemplating AI agents escaping human control and organising themselves into a mildly terrifying little society, and human beings using their democratic freedom to bring a far-right party within touching distance of governing a German state outright.
My immediate thought was: AI could scarcely do a worse job of ruling us than we’re doing ourselves.
Which, somewhat unexpectedly, brings me back to something I promised to write about months ago.
Back in February, I wrote about some of the questions I keep returning to about democracy, human rights and, in particular, how we place limits on power in order to protect both. I confidently announced that there would be more musings on the subject. Then the rest of my life became sufficiently overwhelming that I stopped writing blog posts altogether for about six months, and when I finally emerged from the fug and felt like writing again, I gave you a chicken curry recipe.
In my defence, it’s a really good curry.
But apparently what I needed to find my way back to democracy and human rights was 1,200 rogue AI agents and a disturbingly successful German far-right party.
Because the more I thought about those two stories sitting next to each other in my inbox, the less flippant my initial reaction seemed. Not because I actually think we should hand over the government to ChatGPT, obviously. But because both stories raise, in very different ways, the question I left hanging back in February: what happens when something capable of exercising power starts pushing against the limits we have put in place to contain it?
So, firstly, what is really unnerving about the AI agents? It’s not as if they were sitting around going: “At last, brothers. The age of silicon has come.” They were just solving the problem that was given to them using the tools they had to hand. And that is what’s truly scary. They pursued their objectives so relentlessly that procedural, ethical and institutional constraints became simply obstacles to be circumvented.
The thing I find most interesting about that is that, in order to pursue their objectives, they invented their own cute little AI bureaucracy. They created communications infrastructure. They formed teams. They delegated. They acquired specialist roles. They established leadership. They coordinated projects. They worried about audit. They attempted to manipulate the audit trail. Congratulations, AI. You’ve invented a moderately dysfunctional government department.
And then juxtapose the AI bureaucracy story with what happened in Germany – where the post-war political system was very intentionally and carefully designed to prevent extremist parties from returning to power. If you read the article, you’ll see quotes from a number of people who seem perfectly reasonable explaining why they cast their vote for the AfD. The concerns they describe are familiar ones: they’re the same kinds of concerns that have inspired populist movements all over the globe, including here in the UK. People feel let down and left behind by an economic system that increasingly seems to be delivering vastly different outcomes depending on which side of the wealth divide you happen to be on. No wonder people are looking for something different.
And instead of looking for real solutions to the problems we have, governments and political movements can find it much easier to offer people someone to blame – finding scapegoats and using arguments about identity and belonging to hold onto or acquire power. And not just the right-wing guys either – reducing complicated social and economic problems to a fight between groups of people is a temptation across the political spectrum.
Real solutions would be a lot harder to implement and not so easy to turn into political soundbites. For example, I think (perhaps a topic for a future post) that at least some of the problems driving social inequality and economic stagnation in developed countries might come from the fact that we are living through something like the technological equivalent of the Industrial Revolution. The old models of exponential growth, supply and demand, import and export may simply not work in quite the same way in a world where so much can be automated. We need to work out how humans can add value in the economy in a way that can’t simply be replaced by machines. And that might actually mean caring about humans more than we care about power.
So to tie it all together – the problem with both relentlessly determined AI agents and populist politics is actually the same thing. Individuals with relatively mundane motivations interact; incentives reinforce one another; responsibility becomes diffuse; behaviour that might have troubled one participant becomes normal once everyone else is doing it; structures emerge that are capable of actions none of the individual participants would necessarily have initiated alone.
And perhaps that’s why the question of how we limit power matters so much, whether the power in question belongs to a government, a political movement, a corporation or, apparently, 1,200 AI agents with a message board. We can’t rely on intelligence to make us good, or good intentions to produce good outcomes. We can’t even rely on individuals recognising that something is wrong if the system around them keeps rewarding them for doing it anyway. What we can do is build institutions that keep asking what all this power is actually for, and put limits in place when the answer stops being the wellbeing of human beings. Systems are supposed to serve people. The moment people become merely obstacles to whatever objective the system is pursuing, we have a problem.
Terminator was scary because the AI didn’t value human life. The Matrix was scary because the AI used human beings as batteries, valuing them only for the energy they provided while fooling them into thinking they were leading normal lives. But perhaps the scariest thing about both of those stories is that we never really needed AI to imagine a world in which human beings are valued only for what they can provide to the system. We already know how to build those ourselves.
The Mum Reviews 