We’ve Been Here Before. Sort Of.
Image credits: EWY Media | Adobe Stock
I spend a lot of time with physicists. A lot. Enough that Christopher Nolan’s Oppenheimer resonated with me personally as well as scientifically. That film did an outstanding job of capturing the intoxicating pleasure of being around people who are very, very good at thinking about hard things; the sizzling energy of a community suddenly realizing that what was theoretical yesterday might be buildable tomorrow; the deeply held conviction that understanding nature is, in itself, a worthy pursuit.
It also delivered a heartbreaking portrayal of the moment when understanding nature stopped being the whole story.
What the Atomic Age Teaches Us About AI Safety
I’ve been thinking about that moment quite a bit over the last week, as headlines have been filled with prominent researchers leaving frontier AI companies because they believe the technology that they are helping to build could pose an existential risk to humanity. These leaders aren’t pointing to the “ordinary” risks of a powerful technology, like job displacement, misinformation, bias, surveillance, and the concentration of wealth and power. No, they’ve been talking about human extinction (and, I’m forced to conclude, the extinction of many other species in the same apocalyptic scenario).
It doesn’t matter whether I agree with their estimates of the probability of this threat, or its timing. This moment is extraordinary, and it’s worth pausing to reflect on what history may teach us about it.
The Future Has a History
Imagine working on something all day, surrounded by some of the smartest people in the world. Now, imagine doing that while believing that there’s a nontrivial chance that the thing you are collectively creating could end the human “experiment.” When you lie down in your bed each night, you decide to get up and go to work tomorrow.
That daily decision keeps bringing me back to the physicists, and not just Oppenheimer and the rest of the Manhattan Project crew. I also think about Heisenberg.
Werner Heisenberg was one of the architects of quantum mechanics and one of the greatest physicists of the twentieth century. During World War II, he became a central figure in Germany’s nuclear research program. What exactly happened there has never been satisfactorily resolved.
After the war, Heisenberg suggested that German physicists had, in some sense, avoided building Hitler an atomic bomb. Historians have spent decades arguing over whether this was true. Perhaps Heisenberg deliberately dragged his feet. He may have wanted to build a bomb, but misunderstood the requirements. Maybe Germany simply lacked the industrial infrastructure and political commitment necessary for a Manhattan Project. Like most people confronting impossible circumstances, his motivations may have been contradictory even to himself.
I think that ambiguity is more useful to us than the question of whether Heisenberg was secretly a hero. Here’s the real question:
💡 What does a scientist owe humanity when the thing they are helping to make may cross a civilizational threshold?
This question is critical because walking away doesn’t make the knowledge disappear. Once nuclear fission was discovered in 1938, no one could pretend that it didn’t exist. Heisenberg couldn’t. Neither could Oppenheimer, Szilard, Bohr, Fermi, or anyone else.
Discovery is a One-Way Door
Image credits: Adria Vidal | Adobe Stock
Today’s AI researchers face a version of the same problem. Once you believe that sufficiently capable AI is possible, quitting your job doesn’t make it impossible. It just means that you are no longer one of the people building it.
Which creates a terrible (and infinite) ethical loop. Here’s how I’d code it:
Around and around we go in the infinite loop.
What happens when competition enters the story? Let’s go back to our atomic analogy.
One of the most powerful motivations for the Manhattan Project was the fear that Nazi Germany might build an atomic weapon first. Once you accept that possibility, actions that would otherwise seem morally unthinkable become not merely defensible but necessary.
Today, we hear versions of this argument about AI:
What if another company gets there first? (where is there?)
What if we slow down and our competitors don’t?
What if China gets there first?
What if the people who care about safety step aside and leave development to the people who don’t?
It seems to me that, like in the atomic era, these questions lure us to the same awful place. That’s the place where everyone can sincerely believe that slowing down would make the world safer while simultaneously believing that they cannot afford to be the one who slows down.
No Moment of Decision?
World War II B-29 bomber Enola Gay on display at Smithsonian Air & Space Museum, Chantilly VA | Image credits: Kelly Verdeck/Wirestock Creators | Adobe Stock
Here’s what I think may be fundamentally different between the atomic age and the AI age: With the atomic bomb, there was a reasonably clear decoupling between building the weapon and using it. The scientists at Los Alamos could build a bomb. Someone else would decide whether to drop it. I don’t think this was moral evasion or cowardice; there was basically one person who could decide to use or not use the bomb, and none of them were that person.
I’m not sure I see a distinction like this with AI. There may be no equivalent of the moment that the Enola Gay took off for Hiroshima. Instead, there seem to be thousands of decisions around AI:
Give the model access to tools.
Let it write code.
Allow it to operate autonomously for longer.
Connect it to financial systems.
Let agents communicate with other agents.
Give it access to laboratories.
Deploy the more capable model because the competitor already has.
Automate one more layer.
Each decision can be individually rational. You’ve probably asked yourself whether to give your favorite LLM access to your email, or your calendar. Although each decision can look incremental, the cumulative transition may be enormous. You’re only one person. What happens when companies, industries, and governments enter this decision cascade?
💡 Also, atomic bombs can’t build more bombs. But recursive AI can build more powerful AI. That’s another important difference between these two moments.
Some days, I think that we are a species that is reasonably good at noticing cliffs (and yet, here we are …). I’m confident that we humans are much worse at noticing slopes.
AI may be a slope. Unlike atomic weapons, AI is a general-purpose technology. The same underlying capabilities that worry AI safety researchers also design medicines, accelerate scientific discovery, improve education, help us understand climate systems, engineer new materials, democratize expertise, and increase human flourishing in ways that are difficult to imagine from where we sit today.
What would happen if we hopped in my time machine, headed back to the Manhattan Project, and told those scientists, “Hey, the thing you’re building might destroy civilization. But it could also cure all human disease along the way. Go or no go?”
I wouldn’t wish this moral proposition on my worst enemy. Which brings me back to Heisenberg and, perhaps more importantly, to Niels Bohr.
Who Gets to Say No?
The two men met in occupied Copenhagen in 1941, in a conversation that has become almost mythological because neither man ever gave a completely satisfactory account of what was said. One interpretation is that Heisenberg was probing whether physicists themselves might somehow agree not to build atomic weapons.
Whether or not that’s true, it’s a fascinating idea, isn’t it? A tiny group of humans understands something that almost nobody else understands yet. They realize that this knowledge could transform civilization. Could they simply agree not to cross the line? Would that even work?
💡 I don’t think the real issue here is whether those physicists were good people. The problem was the system around them: war, nations, fear, competition, secrecy. Power.
Eventually Bohr became deeply concerned that atomic weapons could not be managed simply through the consciences of individual scientists or the secrecy of individual governments. The technology had created a problem that required institutions capable of operating at the same scale as the technology itself. You can read his famous 1950 letter about this here.
When an AI researcher resigns today because they believe the technology they are building could threaten humanity, it’s easy to focus on the individual. Are they right? Are they exaggerating? Do I believe their probability estimate for extinction, or the timeline they assign to it?
I think that’s the wrong place to look. Because whether the probability is 20% or 2% or 0.2%, something historically crucial is happening when people at the technological frontier believe that the consequences of their work may exceed the capacity of institutions to govern it.
Consider the institutions. Corporations build things to generate profit. Markets reveal what people will pay for. Nations pursue strategic advantages. None of these institutions is designed to answer the question of whether humanity should cross a particular technological threshold. (I’m leaving aside religious institutions here because this post is already too long … but I’ve paid a lot of attention to Pope Leo’s AI-focused encyclical this year.)
So who gets to decide? Are we willing to leave that to emergent insight due to competition? I don’t think I am.
When Conscience Isn’t Enough
Image credits: Google DeepMind | Unsplash
Let’s circle back to our Heisenberg analogy one last time.
AI is not the atomic bomb. The historical circumstances are profoundly different. The technologies are profoundly different. The uncertainties are profoundly different.
The human predicament, the moral predicament, might be the place of learning. In both cases, a relatively small community discovers something extraordinary. Discovery becomes engineering, which becomes competition, which generates urgency. That’s how a scientific question gets transformed into a civilizational one.
💡 I am a scientist. Curiosity is one of the deepest organizing principles of my life. I have spent my career believing that understanding more about the (natural) world is overwhelmingly good—and I still believe that.
I also know that knowledge and wisdom are not the same thing. Which is why the physicists of the 1940s have been on my mind. By the time everyone understands that a scientific discovery has become a political, ethical, and civilizational problem, the scientists may already have built it.
“Stay or resign?” can no longer solve the problem. Why have we, yet again, constructed a system in which so much may depend on that choice in the first place?
What do we do when our ability to discover something outruns our ability to decide, together, whether it should exist?
I believe that we can build institutions as ambitiously as we build technologies. We can widen the circle of people who get to decide. We can create new norms before the old ones fail us. We can refuse the fairy tale that technological trajectories are inevitable because the genie is already out of the bottle.
The future is not something our technologies do to us. It is something we build with them.
Can the conversation move as fast as the science this time? Let’s find out. And let’s not forget to take Bohr’s words to heart:
“Any widening of the borders of our knowledge imposes an increased responsibility on individuals and nations through the possibilities it gives for shaping the conditions of human life. The forceful admonition in this respect which we have received in our time cannot be left unheeded and should hardly fail in resulting in common understanding of the seriousness of the challenge with which our whole civilization is faced. It is just on this background that quite unique opportunities exist to-day for furthering co-operation between nations on the progress of human culture in all its aspects.”
About Tiffany
Dr. Tiffany Vora speaks, writes, and advises on how to harness technology to build the best possible future(s). She is an expert in biotech, health, & innovation.
For a full list of topics and collaboration opportunities, visit Tiffany’s Work Together webpage.
Get bio-inspiration and future-focused insights straight to your inbox by subscribing to her newsletter, Be Voracious. And be sure to follow Tiffany on LinkedIn, Instagram, Youtube, and X for conversations on building a better future.
For the Curious and the Voracious
Subscribe to Be Voracious, Tiffany’s newsletter, for thoughtful conversations on science, innovation, and leadership, delivered straight to your inbox.
Buy Tiffany a Cup of Coffee | Image credits: Irene Kredenets via Unsplash.
Donate = Impact
If this article sparked curiosity, inspired reflection, or made you smile, consider buying Tiffany a cup of coffee!
Your support will:
Spread your positive impact around the world
Empower Tiffany to protect time for impact-focused projects
Support her travel for pro bono events with students & nonprofits
Purchase carbon offsets for her travel
Create a legacy of sustainability with like-minded changemakers!
Join Tiffany on her mission by contributing through her Buy Me a Coffee page.
Our Editorial Process
We believe AI is most valuable when it strengthens human thinking rather than replaces it. Throughout our editorial workflow, we use AI to transcribe conversations, organize information, brainstorm, challenge assumptions, improve clarity, automate repetitive tasks, and pressure-test our reasoning before publishing.
Editorial responsibility remains ours. AI helps us think, write, and work more effectively, but judgment, accountability, and our relationship with readers remain human.