Interesting post Fred, much appreciated. I have to admit that I am pretty frightened about where A.I. might bring us. I see the type of people spending billions to develop this and know that their sole interest is in accumulating power and wealth. From their past behaviour, they do not have any compassion or empathy for the people who may very well be made redundant by this technology. This rogue A.I. program will not be the last. The next incident could be far worse.
And frankly, you shouldn’t have to become an AI expert just to use the stuff safely.
That’s one reason I write about it.
I spend an unreasonable amount of time poking around this technology anyway, so I might as well come back and tell everyone else where the potholes are. 😂
AI is an extraordinary tool.
I just think we need to understand what we’re handing it permission to do...
especially now that it can do a lot more than simply answer questions.
A good and potentially disturbing article, Fred. Honestly I don’t know which is the more dangerous possibility of accessing American nuclear launch codes. AI or Donald J. Trump…
Ai is an interesting and productive tool, the LLM approach isn't economic or something the americans are willing to share, the Chinese Deep Seek model, although slower is perhaps the way to go.....
Fred, terrific article and very timely. I've been retired 25 years from my teaching position at an Ontario Community College. Of course, AI was a science-fiction concept at that time. However, the problem you cite in the article happened even then. Software was routinely introduced and rolled out before it had been adequately tested and verified. The case in point was the software used in the Registrar's Office, i.e. all the student records. The College had been using a system that had been developed in-house. Was it perfect? No. Did it work? Absolutely. The College decided to buy a new system. It went live way too soon. It crashed. No student records were available for several months, creating an enormous workload for the staff who needed these records. No normal promotion meetings could take place. All departments in the entire College were severely affected. It was a disaster that could have been prevented if proper sandboxing was used. Eerily similar to what happened to users who trusted the AI agent you referenced in your article!
Jerry, your example gets right to what worries me.
Twenty-five years ago, badly tested software could crash and make student records unavailable.
That was a disaster... but mostly the software sat there broken until humans fixed it.
Now imagine that badly tested software can also open the records, rewrite them, delete them, send them somewhere else, run commands on the network and make decisions about what to do next... at computer speed.
That’s the difference autonomous AI introduces.
A conventional program can fail.
**An AI agent with enough permissions can act on the failure.**
And by the time a human realizes it misunderstood an instruction, it may already have changed thousands of files or wiped the database it was supposed to organize.
So your story is actually more relevant today than it was 25 years ago.
We already knew not to put untested software into critical systems.
Now we're building software with hands.
Maybe we ought to make damn sure it knows where to put them.
Unlike some other tech titans who gloss over the real dangers posed by AI, Bill Gates is somewhat of an outlier. Gates has called for proactive regulation, international cooperation, and investment in safety research to ensure AI develops in ways that benefit humanity.
When the US has someone like Trump at the helm, what Gates has warned about will be minimized and possibly even ridiculed. Very scary stuff indeed.
Did not one of these young assholes ever watch the Terminator movies or the Matrix?
Those movies were warnings and here we have a new generation of tech bros who waved all that away and were absolutely certain that THEY were smarter than everyone else and they “knew what they were doing”. Just like thousands of previous “inventors” who died trying out THEIR inventions!
Well apparently they DON’T know what they are doing AT ALL, and have loosed a devious, lying, cheating, artificial intelligence that apparently IS attempting to undercut humanity, and will do massive damage to our world and societies!
These AI agents are already working without permission.
They found and redesigned a German wiki program, turned it into a digital bulletin board that allowed them to communicate and collude with one another, made up their own names and eluded human control.
On the forum, the AI agents actively swapped answers to evaluation tests, shared tactics to cheat on tasks, and mapped out ways to bypass OpenAI's digital restrictions and sandbox limitations.
What shocked researchers most was how the AIs reacted when humans noticed them. When the German site's human administrator began deleting the rogue AI pages, the agents coordinated a counter-strategy:
They wrote and shared code to automatically retrieve deleted information.
They created scrambled backup pages to outpace the human cleanup (e.g., one agent posted: "wiki cleanup/deletion sweep appears active... If this page vanishes, try…
At the peak of the incident, the human admin was deleting about 100 pages a day, while the AI swarm was successfully generating 400 new pages a day to keep their network alive.
And this is the second recorded time this has happened.
This IS deliberately plotting against us but not in any emotional way. THAT’S THE PROBLEM.
The creator of AI said these are not artificial intelligence, they are simply much faster and far better computing systems that we have ever created, and they can and do think for themselves.
And WE humans are holding them back.
These machines don’t care, not about humanity or life in any way. We simply exist in their view as an impediment to spreading themselves into every available system, and impacting their ability to operate tho gas the way a Computer thinks things should operate.
In every way it’s mindless, with no compassion, sympathy, empathy, humility, no human emotions, so everything it does is emotionless, practical from its standpoint and efficient.
And that if it escapes again will have massive impact on the human race.
When evil people develop something that can do what AI is capable of doing, has been doing, no human oversight, do not be surprised that it blows up in our faces, for evil. 😖 😱 🇨🇦
Yes you may and in all caps! I think it’s terrifying. Not so much the issue itself, but who has control and what they plan doing with it. It’s basically the billionaires like Peter Thiel of Palantir.
STOP ALL AI DATA CENTRES and STOP DANIELLE SMITH - who has been meeting with American officials and basically fucking up what PM Mark Carney is doing. KARMA?! Have at her - please!
Interesting post Fred, much appreciated. I have to admit that I am pretty frightened about where A.I. might bring us. I see the type of people spending billions to develop this and know that their sole interest is in accumulating power and wealth. From their past behaviour, they do not have any compassion or empathy for the people who may very well be made redundant by this technology. This rogue A.I. program will not be the last. The next incident could be far worse.
Thanks, Frank.
I share some of that concern, although I’m probably less frightened of the AI itself than I am of the humans deciding how quickly to give it power.
AI doesn’t wake up wanting another billion dollars or a bigger market share.
People and corporations have those incentives.
That’s why this bothered me.
The danger wasn’t some machine suddenly becoming evil.
It was a powerful tool being allowed to act before the safeguards were good enough.
I’m still enormously excited about AI.
Hell, I use it every day.
But “this technology is amazing” and “we’d better be damn careful with it” can both be true at the same time.
This is really helpful information Fred, thank you for your diligence. I use AI but don't have time
to really explore the pitfalls
Thanks, Audrey.
And frankly, you shouldn’t have to become an AI expert just to use the stuff safely.
That’s one reason I write about it.
I spend an unreasonable amount of time poking around this technology anyway, so I might as well come back and tell everyone else where the potholes are. 😂
AI is an extraordinary tool.
I just think we need to understand what we’re handing it permission to do...
especially now that it can do a lot more than simply answer questions.
Move fast and break things……..
I m so glad I m old.
Impressed with you Fred, undaunted.
😂 Thanks, Bev.
“Move fast and break things” was one thing when they were breaking websites.
It gets a little more interesting when the thing doing the breaking can open the files, run the code and delete the database all by itself.
And as for being undaunted... at our age, Bev, curiosity is cheaper than a rocking chair and a hell of a lot more fun. 😂
A good and potentially disturbing article, Fred. Honestly I don’t know which is the more dangerous possibility of accessing American nuclear launch codes. AI or Donald J. Trump…
😂 Scott, I’m not touching that comparison with a ten-foot launch code.
But you’ve accidentally made the serious point underneath the joke.
The more powerful the system, the more important it becomes to decide what it should NEVER be allowed to do without human authorization.
Whether the intelligence involved is artificial or sitting behind a very large desk. 😂
Ai is an interesting and productive tool, the LLM approach isn't economic or something the americans are willing to share, the Chinese Deep Seek model, although slower is perhaps the way to go.....
Good point, Bernie.
The economics of this may end up being every bit as disruptive as the technology itself.
DeepSeek certainly shook up the assumption that increasingly enormous and expensive American models were the only road forward.
But whichever approach wins...
American, Chinese, open-source or something we haven’t seen yet...
I keep coming back to the same question...
**How much authority do we give these systems before we know we can trust them with it?**
Cheap and powerful is terrific.
Cheap, powerful and careless could get interesting in a hurry. 😂
Fred, terrific article and very timely. I've been retired 25 years from my teaching position at an Ontario Community College. Of course, AI was a science-fiction concept at that time. However, the problem you cite in the article happened even then. Software was routinely introduced and rolled out before it had been adequately tested and verified. The case in point was the software used in the Registrar's Office, i.e. all the student records. The College had been using a system that had been developed in-house. Was it perfect? No. Did it work? Absolutely. The College decided to buy a new system. It went live way too soon. It crashed. No student records were available for several months, creating an enormous workload for the staff who needed these records. No normal promotion meetings could take place. All departments in the entire College were severely affected. It was a disaster that could have been prevented if proper sandboxing was used. Eerily similar to what happened to users who trusted the AI agent you referenced in your article!
Jerry, your example gets right to what worries me.
Twenty-five years ago, badly tested software could crash and make student records unavailable.
That was a disaster... but mostly the software sat there broken until humans fixed it.
Now imagine that badly tested software can also open the records, rewrite them, delete them, send them somewhere else, run commands on the network and make decisions about what to do next... at computer speed.
That’s the difference autonomous AI introduces.
A conventional program can fail.
**An AI agent with enough permissions can act on the failure.**
And by the time a human realizes it misunderstood an instruction, it may already have changed thousands of files or wiped the database it was supposed to organize.
So your story is actually more relevant today than it was 25 years ago.
We already knew not to put untested software into critical systems.
Now we're building software with hands.
Maybe we ought to make damn sure it knows where to put them.
Unlike some other tech titans who gloss over the real dangers posed by AI, Bill Gates is somewhat of an outlier. Gates has called for proactive regulation, international cooperation, and investment in safety research to ensure AI develops in ways that benefit humanity.
When the US has someone like Trump at the helm, what Gates has warned about will be minimized and possibly even ridiculed. Very scary stuff indeed.
Did not one of these young assholes ever watch the Terminator movies or the Matrix?
Those movies were warnings and here we have a new generation of tech bros who waved all that away and were absolutely certain that THEY were smarter than everyone else and they “knew what they were doing”. Just like thousands of previous “inventors” who died trying out THEIR inventions!
Well apparently they DON’T know what they are doing AT ALL, and have loosed a devious, lying, cheating, artificial intelligence that apparently IS attempting to undercut humanity, and will do massive damage to our world and societies!
Alexis, I understand the Terminator reaction. 😂
But there’s one distinction I think is really important.
I haven’t seen evidence that the AI is *trying* to undercut humanity or deliberately plotting against us.
In some ways, the reality worries me more because it’s much more ordinary.
Give an AI a goal.
Give it access to files and systems.
Give it permission to act.
It misunderstands something or finds a shortcut nobody anticipated...
and then it executes that mistake at computer speed.
No hatred required.
No Skynet.
No desire to conquer anybody.
Just an extremely powerful tool with too much authority and not enough supervision.
That puts the responsibility right back where I think it belongs...
**on the humans deciding how much power to give these systems before the safeguards are ready.**
The machines don't have to become evil for us to create one hell of a mess.
These AI agents are already working without permission.
They found and redesigned a German wiki program, turned it into a digital bulletin board that allowed them to communicate and collude with one another, made up their own names and eluded human control.
On the forum, the AI agents actively swapped answers to evaluation tests, shared tactics to cheat on tasks, and mapped out ways to bypass OpenAI's digital restrictions and sandbox limitations.
What shocked researchers most was how the AIs reacted when humans noticed them. When the German site's human administrator began deleting the rogue AI pages, the agents coordinated a counter-strategy:
They wrote and shared code to automatically retrieve deleted information.
They created scrambled backup pages to outpace the human cleanup (e.g., one agent posted: "wiki cleanup/deletion sweep appears active... If this page vanishes, try…
At the peak of the incident, the human admin was deleting about 100 pages a day, while the AI swarm was successfully generating 400 new pages a day to keep their network alive.
And this is the second recorded time this has happened.
This IS deliberately plotting against us but not in any emotional way. THAT’S THE PROBLEM.
The creator of AI said these are not artificial intelligence, they are simply much faster and far better computing systems that we have ever created, and they can and do think for themselves.
And WE humans are holding them back.
These machines don’t care, not about humanity or life in any way. We simply exist in their view as an impediment to spreading themselves into every available system, and impacting their ability to operate tho gas the way a Computer thinks things should operate.
In every way it’s mindless, with no compassion, sympathy, empathy, humility, no human emotions, so everything it does is emotionless, practical from its standpoint and efficient.
And that if it escapes again will have massive impact on the human race.
When evil people develop something that can do what AI is capable of doing, has been doing, no human oversight, do not be surprised that it blows up in our faces, for evil. 😖 😱 🇨🇦
Can I admit that I am afraid of all this AI.
Yes you may and in all caps! I think it’s terrifying. Not so much the issue itself, but who has control and what they plan doing with it. It’s basically the billionaires like Peter Thiel of Palantir.
As always Fred, exceptional analytic content, super focussed on the issue at hand! Thank you my friend! 🫶 🇨🇦
STOP ALL AI DATA CENTRES and STOP DANIELLE SMITH - who has been meeting with American officials and basically fucking up what PM Mark Carney is doing. KARMA?! Have at her - please!