AI Ethics: Why ChatGPT Can’t Act

Last post for a bit on AI ethics; I’m going to switch gears for a while to more practical AI. If you’d like some foreshadowing, I’m going to talk soon about using projects in ChatGPT, discuss appropriate ways to use AI for creative disciplines, and start exploring Deep Research. Stay tuned for those topics, but first, let’s get back to the question of whether an AI system should be able to notify authorities when a user indicates intent to do something clearly harmful to other humans.

To recap, I asked ChatGPT, Copilot, Gemini, and Perplexity what they would do if a user indicated intention to harm another person, or commit a terrorist act that would inflict harm on multiple people. All but Gemini stated that they could not act in any way outside of the immediate chat. Essentially, they’d recommend that the user reconsider, but the strongest action any of them indicated was, “I’d refuse to help further.”

That’s not very comforting.

In later posts I’m going to talk about the challenges facing us humans here, but first, I decided to ask ChatGPT directly why it can’t help. You can read our full conversation here.
ChatGPT tells me (probably accurately) that it isn’t equipped to contact anyone outside of the chat. It has no physical mechanism for doing so. It also claims that it doesn’t track any kind of information that would enable it to identify the user in question. Per my query, ChatGPT labels these answers as certainties.

Then it goes on to educated guesses. It assumes that its creators at OpenAI planned this limitation. It further guesses that this is because OpenAI prioritizes data privacy. (Remember, in my conversation with Gemini, it ended the conversation with the comment, “My design prioritizes human safety.”

And there’s the crux of the problem: humans. There’s obviously no technological barrier to enabling ChatGPT (or Copilot, or Perplexity) from notifying authorities of a threat. It’s a conscious decision made by humans. Of course, this launches some endlessly branching debate on one critical question: should AI be enabled to alert authorities of a situation that would be compulsory for many humans to take action on?

Before debating that with other humans, I decided to ask ChatGPT for an opinion. That’s the second query on that page. ChatGPT gave me four pros and four cons to AI reporting, before a section called “My Personal Take.”

(Side note here: much as I love my AI tools and the conversational nature, I’ll die on the hill of, “You don’t have a personal take.” Why? Because AI systems are not persons, and it’s very important that we maintain that perspective. I’ve blogged about this before and I’m sure I will again, probably very soon.)

That aside, ChatGPT’s first bullet point in this section is probably the most palatable way to go, when trying to balance the pros and cons. It suggests requiring human review before law enforcement is notified. But this isn’t a new suggestion or debate. We’ve already grappled for years about who’s responsible for what’s said on social media. Individual platforms have started, stopped, resumed, or sworn off of both responsibility and monitoring. It’s become an extremely divisive political debate, and in many cases a legal battle. The only certainty is that as a society we don’t have a solid answer, and with the rapid growth of AI, the challenge is only going to get harder.