Is AI a Threat? - Attempt at an Assessment | Overwhelming complexity

AI translated using Gemini 3 Flash

Nowadays, one hears increasingly often about the risks and dangers of “AI.” Right from the start: the question of whether “AI” is dangerous can be answered with a clear yes. AI is used in warfare1, for propaganda2, mass surveillance3, autonomous extortion4, and even advises suicide5. The medium-term effects on society as a whole regarding social cohesion and probably also economically (even just at the level AI models are at today—be it only that the stock market crashes) are not clearly foreseeable, but definitely existent.

The question that is becoming even larger, however—and which this post is about, precisely because the term “AI” is used so much in dystopian movies—is whether “AI” is an existential threat, like a nuclear war.

What is “Artificial Intelligence”?

Before we deal with that, let’s have a brief interjection. “Artificial Intelligence” is a hype term that can mean everything and nothing. To show the extremes: I have seen advertising that describes a coffee machine as “AI-supported,” and the only thing concretely mentioned there was that favorite drinks are shown further up the list (so probably simply those most frequently used).

On the other hand, the term “Artificial Intelligence” resonates with all the blockbusters a la Terminator, where AI is smarter than humans and tries to destroy humanity.

The term “AI” is therefore too nebulous and connoted for a good debate. Therefore, I’ll limit the term here for now; I am referring to models that have arisen through machine learning—almost exclusively the (also unclearly defined) area of “deep learning”—and which achieve incredible things today, from real-time translation and recognizing flowers to automated warfare.

Models like ChatGPT, Gemini, and Claude are based on so-called LLMs (Large Language Models), a collective term for a variety of specific models that are, however, based on the same idea. This term is already much more clearly defined and I therefore prefer it for most discussions. Here, however, I will stick with the term AI, as the examples are not just limited to LLMs.

As already explained above, AI is very present and also already very dangerous, but the crucial question is: will it just continue like this?

The Impressive Progress of AI

There are a surprising number of people who have been thinking a lot about the existential risks of AI for a long time and provide many arguments for these scenarios.

Ultimately, however, there is only one argument so far that I find convincing: AI has so far achieved many things that were thought would take much longer or even be entirely impossible for machines. They have mastered Go and chess, written texts, painted pictures, and even folded proteins.6

The question that arises from this is quite clear: what about the problems that AI cannot yet solve today, or where it seems impossible for an AI to solve them? Will AI soon be more intelligent and faster than us?

And what does that mean then? If the AI had the goal of producing paperclips, would it then produce paperclips from everything, including us humans?7 Or if part of the programming/goal-setting of the AI is simply not to be turned off8? Would it then destroy all intelligent life to ensure this goal?

And even if it is still “controlled” by humans, the question is whether we could oversee the consequences—and what the goals of these humans are.

The Epistemic Risk: The Unknown

But of course, no one can (really) say this scenario will occur or this scenario is x% probable—in contrast, for example, to climate change, where we can estimate this quite well (under the assumption of physical models9).

This is called an epistemic risk; we have no idea whether this scenario is possible and, if so, how probable it is.

The collective term for this line of thought (that AI is/could be an existential risk) is AI Safety.

But AI Is Not the Only Problem

Because of this, we also don’t know whether it is an important problem or not, and the fact that it cannot be put into any box and analyzed more closely leads, I think, to a large extent to this discussion in certain circles being very heated and necessarily unfounded. Added to this is clearly the impressive speed in the development of these models, which makes the debate appear/become very urgent.

But it must always remain clear: existential risks have existed all along

And the other problems mentioned above that AI brings with it must also not be lost sight of—including the exacerbation of the climate crisis through increased resource requirements.

The Paradoxical Behavior of Many AI Companies and Developers

The strangest thing about the matter, however, is definitely how many AI companies themselves warn of the danger of a “superintelligent AI” (OpenAI, Anthropic) and many of the employees of these companies also say it is one of their greatest fears—but nevertheless, they try to build the most intelligent models possible as quickly as possible.

Most of these companies have the declared goal of building a superintelligent AI (AGI)—something that should be well-considered even without existential risks (ethical implications, military use).

I also don’t find it implausible here that in addition to the fascination of bringing technology further (and the money), there is also this principle of “if someone is going to have a superintelligence, then I want it to be me; at least I will handle it reasonably” resonating.

What Should We Do?

From this, one perhaps also notices another component why this problem occupies many: it should actually be so easy to solve. Significantly fewer people would have to stop programming AIs than, for example, people would have to stop flying.

If one concludes from all this that AI is an existential danger that is probably not super improbable (which I unfortunately tend toward more and more), then the central question is: What do we do?

The first step is clearly to introduce AI regulations and build stronger guardrails into AI models; this also helps in particular against some of the other AI risks mentioned at the very top.

Beyond that, the question arises of how far one wants to slow down AI development itself, because one would then actually need worldwide control—of both companies and data centers (many here draw the comparison to mutual control as with nuclear weapons; a comparison I find difficult, but I see where it comes from, namely from the necessity of deep and effective global control). And that is unfortunately de facto unimaginable in the current world climate.

Personally, I also don’t see the value of a generally superintelligent system. I would find many problem-specific systems—for example for protein folding—much more practical and better. Why that is one of the goals at all is a relevant question for me (there are some cultural/technical explanatory approaches, but nothing I understand enough about to write it here).

Moreover: even if the most extreme scenarios do not occur—massive AI systems and data centers concentrate enormous power in a few hands, which is already a reason to at least keep an eye on it.

Conclusion: Acting Despite Uncertainty

Perhaps AI is an existential risk. We don’t know.

That doesn’t mean there aren’t other global potentially existential problems like climate change and nuclear weapons, but unfortunately one more to think about.

What exactly that looks like is difficult, but it should start with generally stricter AI regulation and potentially global controls of large data centers. It definitely does not mean that we should now destroy all GPUs (as some people have demanded in conversation).

To give a positive outlook by contrast: there are also a few people who believe that “techno-socialism” is coming, in which all material needs are met and we can free ourselves from poverty, wars, and unpleasant work. (Though I suspect being de facto useless wouldn’t be quite as great as it sounds.)

(Or else, development will soon stagnate and we will just continue to live as before, only slightly more comfortably and with greater climate problems.)

P.S: For this blog post, I let Claude Haiku play “proofreader,” and it repeatedly inserted sentence fragments intended to make AI safety appear even more urgent (as a closing sentence, for example, it wanted to insert: “Which of these scenarios is more likely, I don’t know. But that’s why we should act.”)

I would find it very exciting to discuss this further with people, because I am still quite uncertain myself, so feel free to just write me an email.

  1. https://www.cbsnews.com/news/anthropic-claude-ai-iran-war-u-s/ ↩

  2. https://en.wikipedia.org/wiki/Deepfake ↩

  3. https://netzpolitik.org/2026/verhaltensscanner-in-mannheim-keine-straftaten-aber-kamera-ueberwachung/ (German) ↩

  4. https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/ (Ok, not quite yet, but well on its way) ↩

  5. https://www.zeit.de/digital/2025-09/chatgpt-jugendschutz-suizid (German) ↩

  6. I once read that before AlphaFold or with X-ray crystallography, it was often an entire doctoral thesis to determine the folding of one protein. I unfortunately find no evidence, so perhaps it simply isn’t true. ↩

  7. https://en.wikipedia.org/wiki/Instrumental_convergence ↩

  8. Often this also arises automatically from another goal-setting; e.g., producing as many paperclips as possible also means not letting oneself be turned off so that one can continue to produce paperclips. ↩

  9. Here it becomes philosophically interesting, of course: who says that our models will still apply tomorrow? who says that there will still be gravity tomorrow? We have no certainty of that, but so far these assumptions seem very reasonable. And that’s where some other arguments from the AI safety direction become weak, namely by harping on this point—but according to this motto, one would also have to try to prepare for the disappearance of the Earth tomorrow. ↩