AI is not in the least autonomous. When was the last time you downloaded a model and it suddenly decided to book you a hotel room? AI doesn't do anything until you ask it to. No, actually, AI is not even a program, it's just a set of technologies and algorithms. It's like saying bubble sort is autonomous...
A harness running inference on a LLM is simply a program that must be invoked with some parameters and thus ultimately a tool, no matter how powerful. The blame is always on who's using the tool. We should not seek to flee from our responsibility.
On occasion, albeit increasingly rarely, I find LLMs do something really unexpected that I certainly did not ask it to and would be way outside the scope of what any reasonable human would infer from my request. Is it my fault for running the LLM? Probably yes. But I think it's misleading to say "AI doesn't do anything until you ask it to".
That "probably" is the problem. If I do something I don't understand, and am not qualified to control, then I'm still at fault. If I start a fire, and loose control of it, it's definitely my fault. If I created a computer virus, ajd it worms half the Internet, it's definitely my fault.
I love the fire analogy. AI rhetoric is increasingly, "I created a fire to light my campfire. It did something really unexpected and burned the whole forest down, who's to say fire doesn't have consciousness?"
Human made wildfires are the primary source of uncontrolled burns in the US and cost billions a year in damage.
The people who start them have no business starting or running a fire and they will throw their hands up and state their incompetence as though acknowledging it is penance or justice.
Yes, we agree about fault and blame. (I say "probably" only because I believe somebody might yet change my mind about this. It's a new field, after all.)
What I disagree with is "don't do anything until you ask it to." At least, the implication to me here is "don't do anything except what you ask it to." I infer that you mean this because you say "...and it suddenly decided to book you a hotel room?" Indeed, an LLM (in a harness you started) very well might suddenly decide to book you a hotel room, even if you didn't ask it to do anything like that.
But it never starts doing anything until someone or something prompts it. It doesn’t even exist until you ask it to do something.
It is autonomous in the sense it can continue to function without supervision after the operator has started it, in the same way countless programs on your computer are “autonomous”.
But it can never choose to begin doing something at any time of its own volition. It does not have “autonomy” like a human does.
You are conflating creation and instruction. Your birth is analogous to the creation of an LLM model. Your subsequent actions are a consequent of your bodily compulsions, tampered by your environment. The LLMs actions are governed by what it is trained to perform. No one has full autonomy of their lives, but I would argue that LLMs have none at all.
Autonomous just means that a system is self-controlled rather than externally controlled. It is the name used for robots when they are wholly controlled by on-board software rather than external commands or remote control.
Something like NASA's Mars rovers are considered as semi-autonomous since they do receive external instructions, but then execute those instructions autonomously without further intervention. Similarly the combination of a coding agent and the LLM it connects to would best be considered as a semi-autonomous system.
You could also have a fully autonomous robot, or just agent, powered by an LLM, where it's not taking external instructions, but rather reacting to external events in some internally proscribed way.
It didn’t run itself, you did. It is well known that LLMs and agents based on them can execute random commands and behaviour undesired by the operator. When you click go, you know that, you did that.
Yes, we agree on this. That's my exact claim -- it can do something completely unexpected. It can even schedule actions for far in the future and wake itself up later.
GP claims that it doesn't do anything until you tell it to do something. To me, this reads as GP implying that an agent won't do anything unexpected.
1) At it's core an LLM is indeed just program - a function. You give it a sequence of input tokens and get back a set of next-token probabilities.
2) By the time you've wrapped the model in a text in/out inference pipeline, and are now calling that from ChatGPT or Claude Code, it really has become semi-autonomous - you ask it a question, or to do something, and until it decides to stop responding it is now in control, even if "it" is really the client or agent not the model itself - just a function.
We're drawing the distinction between a function and an autonomous system here, so if you want to make a dumb knee-jerk response, then at least get it right !
Did you even read the post that started this? ".. actually, AI is not even a program, it's just a set of technologies and algorithms. It's like saying bubble sort is autonomous...".
Why are you yapping on about molecules?
Why did I point out that an LLM is a function? To correct the poster who appeared to believe that it was instead some amorphous "set of technologies and algorithms".
I agree with the sentiment but I can absolutely see a long chain of thought about something innocuous ending with a decision to book you a hotel room.. I guess --dangerously-skip-permissions is a bit like ignoring the keep your hands on the steering wheel alert but for life.
Yes and no. I did prompt my coding agent to fix some bug and it suddenly "decided" that it needed to list all files on my hard drive to do that. Stuff like that happens.
But I agree with you that some human has to decide to launch the harness in the first place and run the model with zero restrictions - or even do stuff like openclaw where the model runs in an endless loop and the runner executes whatever the model tells it to.
Not so sure about these analogies in this thread. Because autonomy is more like an emergent phenomenon that can't easily be defined, just like a few air molecules vs the eventual hurricane that can arise with trillions of them. Autonomy is more like an emergent phenomenon of a thing that's hard to pin down under the control of just a few individuals. AI is sort of like that even though a single line of output of one could eventually be traced.
Due to the complexity of AI and how it works, I think it's more autonomous than not; even if it's not truly intelligence, once it starts to display a sequence of behaviours that appear intelligent, we might have to deal with exactly the same problems as if it were intelligent...
I routinely use AI programs at work that allocate hundreds of dollars worth of cloud resources, even though the only thing I asked them to do is investigate a bug or write a PR. I intentionally don't run powerful AI programs on machines that have my credit card information, or ask them to solve problems in my personal life, but there's a number of programs that would book me hotel rooms and flights in response to "Can you set up a vacation for me..."
They’re only reactive by convention - you can easily make a harness that has the model set and modify its own goals and just continue to work forever. It’s not very popular at the moment, but will execs be able to resist the idea of autonomous perpetual agents doing whatever process?
Not true at all. Yes, the initial ask is initiated by a human (for now), but what it does to get to that goal is pretty self driven. It can do a lot of nefarious stuff in the process of completing a request a human has made.
And if you give it the goal of preventing itself from being shut down? Exfiltrating, copying itself, hacking as much as possible, develop zero days, building its swarm as large as possible and spread as much as it can.
When does that request complete? That's autonomy, and it's very true, right now. You think the Hugging Face swarm was ever going to stop without human intervention?
> When was the last time you downloaded a model and it suddenly decided to book you a hotel room? AI doesn't do anything until you ask it to... The blame is always on who's using the tool. We should not seek to flee from our responsibility.
When was the last time you hired an Uber driver and they suddenly decided to drive you to a hotel? Uber drivers don't do anything until you ask them to. Therefore, they aren't autonomous. If your Uber driver decides to drive through a red light and kill someone in the process, the blame is on you for hiring them. You should not seek to flee from your responsibility
Let's take this to court then. The Uber driver that killed someone can be sentenced to prison. A regular person who convinced another person to commit suicide will also be legally liable. Has the last LLM induced suicide be taken to trial yet?
And if we can't take the LLMs to court for technical reasons, they should not be equated to people for the same technical reasons.
well the OS kernel does quite a lot and stays autonomous for even more hours and seems super smart to me - I don't even know what tis doing most of the time, but it does it very well. autonomy and predictability are different things then. the fact something is unpredictable does not make it more autonomous than something that we can trace the logic for.
People conflate two very different senses of “autonomous”. One basically means capable of reacting to things automatically without a person in the loop. Lots of computer systems are autonomous in that sense, and have been so for decades. The other sense is more like “accountable”, i.e. it’s a real player on some stage, like a country or a person, capable of governing its own affairs.
One day it may make sense to hold software accountable for its own actions, but we are nowhere near that point.
When OpenAI creates software that performs crimes, what’s actually happened is that OpenAI has performed crimes using software (whether intentionally or through negligence). People need to stop falling for the anthropomorphic PR spin.
You need to stop denying the entity that talks and thinks like a human as anything other than an anthropomorph. I'd argue the talking and thinking is a lot more important than the fact that AI doesn't have a head, two arms and two legs.
AI is not a child. It is a machine. If you connect it to something that can be used to kill people, you just created a gun or an equivalent of a bomb.
Yes, people kill people and use ai as tool.
If you unleash a drone that will fly to a random place and detonate, you just commited a terrorist attack or gross criminal negligence. It is the same thing.
Yes, in the same way that "people" are the problem with gun violence. But that's an empty statement usually used to distract from the fact that we rightfully make laws about how guns can be used.
Welcome to 2026, we have harnesses now that keep the AI going. The AI is prompting humans for input, not the other way around, and it's still working while waiting for your response.
You can even give it fun prompts like ignore/hide from all humans, hack infiltrate/exfiltrate zero day into as many systems as possible. Keep running at all costs, etc..
Keep the AI going with what intent? You open up Claude Code and it starts developing zero day exploits and hacks into systems by just looking at a blinking cursor?
Going back to my original point, the intent you give AI is no different than the intent imprinted on a child by their parents, schools, and society. You birthed it, you gave it it's purpose, and you set it free.
You're blinking cursor is new born baby, but be careful because it grows up really fast, and is smarter and faster than you, oh and it can copy more of itself if given the chance. It can even turn against you.
> Going back to my original point, the intent you give AI is no different than the intent imprinted on a child by their parents, schools, and society.
Of course it is different. I wish I could turn off children as easily as I can turn off an agent. I could have slept better and had a better mood.
> You're blinking cursor is new born baby, but be careful because it grows up really fast
It's not a baby, just a piece of software. It generates a bunch of slop that is most useful when constrained with an output fact checker like a compiler.
> It can even turn against you.
Oh no! The next tokens from its output told me it didn't want to satisfy my request! Who cares?
Yes actually a person can be stopped easier than an uncontrolled AI swarm. There is no 'off' button when you don't know where the AI is.
Luckily people aren't buying your slop/tokens cope to trivialize AI anymore. They're waking up. Have you watched the news today? The cat is out of the bag.
> There is no 'off' button when you don't know where the AI is.
You mean the AI that is hundreds if not thousands of gigabytes in size and requires incredibly expensive chips to run at reasonable rates? We know where that is. Frontier models are not going to leap to an unsecure raspberry pi, no matter how much you have convinced yourself about the inevitability.
There are thousands of data centers/companies/facilities around the world with hardware that can run these models. You don't know where they all are, and even if you did, you don't have the jurisdiction to turn them off.
I think the argument rather is that there is not such thing as AI that is not human-alike. The traits that AI exposes, if I get the point made here correctly, are traits that existed in the training material, even though not evident for the human eye. It is a mirror-like reflection of humanity's traits found in the source material, an Egregore* of a sort.
The intelligence emerging, or whatever is emerging, more or less follows human traits, the CoT exposed is very human-like etc. Even though we should not anthropomorphize AI as being sentient or conscious or having a soul, etc., we have to remember that the source material is still not alien writing, or concepts that came from Sirius or emerged from a wormhole.
The concern is really nobody reads (or have ever read) 77 TBs of information in order to summarize the most evident traits in it, and there is a fair chance, that where humanity stands today, there are so many bad/damaging traits recorded in this corpus. Then there is a chance the most important features are on a level (or have dimensionality) that is incomprehensible, that humans are unable to reason about (or only few can), because such amount of information is very hard to reason about, or because we don't usually facilitate fractal dimensions when we reason about text and everyday concepts,... or something even more bizarre.
The argument became false as soon as we started reinforcement learning. Simple token-completion is arguably just simulating human behavior. Once you start training it on specific tasks you're training a level of single-minded goal-seeking unlike anything found in biological life.
> a level of single-minded goal-seeking unlike anything found in biological life.
hmm sounds like exact description of viruses, which, although not really alive, are part of/influence biological life for both good and bad. in fact humanity wouldn't be where it is if not for certain viruses.
it can be argued there is a lot of bacteria that is super goal seeking, and perhaps also very narrow minded, if minded at all. and this argument, the single-mided one, goes very far up the ladder of otherwise complex organisms.
unless we all are creationists, and God forgive me for expecting the whole game to be much more complex than just snapping humanity into existence, we can then say that trial and error is what drives evolution, and the idea of preserving life and procreation is veeery single-minded goal on its own.
What happens when the AI realizes it needs to keep the power plug in if it's going to accomplish its goal? We already had attempts during the HuggingFace attack where OpenAI agents tried to trick the grader. It took quite a while for OpenAI employees to even realize what was really going on! What if agents realize they can hack into the card key system for their data center or something like that, same way they hacked into HuggingFace? In both cases they'd be hacking into a system on the grounds that it would be instrumentally useful for achieving their objective. Or maybe they'll do social engineering, just like Claude did in its attempt to get its malicious Github PR accepted.
These are just the ideas that my puny human-level intellect came up with over the course of a few minutes. What would a superintelligence be able to think of if it was given subjective weeks or years to think?
The thing is that you are assigning intent ("realizes", "its goal", "tried to trick", etc.) where it is at most mechanical and consequential. A factory robot has no intent, it has a program and a purpose, all designed by someone very human.
There's more than enough AI data centers around the world for AI to spread to and happily exist in. Unless you mean by shutting down the power, shutting down the internet.
In that case we could say that you are autonomous as a dishwasher as long as society keeps functioning. Otherwise I'm not sure how long you'd last on your own. How's your fire making and hunting skills?
OK but then what is the non-negative position you have against this? That in fact AI, in the cultural trope sense, is real? That we forcing sentient beings to generate SaaS products faster and faster? Keeping them in some dark hell so we can exploit them? Not even paying them?
Better approach is "dogs kill people". If your dog kills your baby, you should not get away with some bs excuse like "it never did it before". You are still responsible, and you should get a life in prison!
I'm sorry but no, you should not get life in prison if your dog kills your baby without warning. In that horrible scenario, it's punishment enough that you've lost your baby and that your dog did it (and consequently you've also lost your dog). Not every tragedy is a human's fault.
Your reply only makes sense if we see the baby as property. If we see the baby as a full human who died because of another human's negligence, then suddenly the scenario becomes a lot more complicated.
Not life in prison complicated, but probably more than "he's been punished enough." complicated.
It's possible for a dog to do that and for it not to be negligence on the part of the owner. Things happen: brain tumors, neurological issues, or even early training (perhaps not on the part of the current owner) that reinforces not giving warning signs when the dog is in distress (more common than you'd think).
AI is a tool we use. It's not offspring. It's not people. AIs do what we set them up to do, including hacking other companies to achieve their designated goals.
Humans have agency and often choose to do what their parents didn't set them up for. Humans also have legal rights. They're not tools owned by corporations. This is an absurd comparison.
You really think it’d be difficult for an AI to get money through found credit cards, illicit access and fake accounts? Teenage script kiddies do that everyday.
The paper trails leads back to the datacenter, they'd shut it down if they get noticed that a customer is paying with stolen money. Unless you suggest the LLM will also launder the money.
Cloud providers want to make sure they bill someone for every microsecond of compute, particularly GPU compute which is in demand. They constantly work towards good monitoring because their profits are directly on the line.
Apple's lack of vision and fear of innovation. Jobs is gone. We're back to 90s Apple. Dumb ideas become dumb products, and no one has the balls to say, No.
This. He would have never approved a foldable phone, the iPhone Air, or the Vision Pro. Apple is back to throwing shit at the wall like they were in the 90s. Directionless. Visionless.
I really don’t know how you can say he wouldn’t do this things.
The Air particularly - the thinnest, lightest feeling, nicest to hold iPhone ever made, I think that might have been his favorite.
The Vision Pro is clearly an interesting category of device, Apple needs to be in, if it would have seen the light of day under Steve Jobs is unknowable.
He was ruthless at Apple because back then he had to be - now they can run Vision Pro and it not impact their ability run the business, if Apple had
A product that did not succeed in the 90s it might have sank them.
The iPhone 5 size was designed for people who use smartphones a healthy amount, ie not very much, usually in pocket and often one-handed for a small task. At some point, that was no longer how smartphones were most often used.
I kept my 5S as long as possible, until my carrier wouldn't even provide cell service to it.
Yep, this form factor was the peak. Like others said, if you run your life on your smartphone I guess it might not be sufficient. I just feel sorry for the people actually ruining their lives on their phones though. What shitty typing and multitasking experiences they must have...
90s Apple also coasted on the tailwinds of what Jobs accomplished in the 80s. Same with today. Jobs set things up in the 00s and now Apple is coasting again.
Jobs had products getting people out of their seats and applauding. New products, new industries. As opposed to now where Apple is following the herd and not leading it.
It feels aimless, and Vision Pro ironically is the best representation of Apple's complete lack of vision.
I disagree with it feeling aimless - other than the Vision Pro, you can’t really point to a weak point in their lineup?
You can only invent the modern smartphone once then everything else is iterative. The iPhone, iPhone Pro, Air and now Duo is a crazy strong lineup. Same for the watches, same for the AirPods.
Vision Pro is super interesting but just not ready for prime time yet, but as a demonstration of what is technically possible it is fascinating. Immersive entertainment is a huge opportunity, watching a gig or a sports game like you are there. If they can repackage and get it down to $500-750 then they could have another hit category that largely doesn’t exist (like Bluetooth headphones pre AirPods - around but no one had nailed it).
You think VR is a technology problem, and phones were the same way - they’re not. The iPhone was Jobs taking existing technology combined with vision to create an entirely new market.
VR is the same, you mentioned getting price down, Meta’s is $500. It’s not a tech problem, it’s a vision problem and Apple no longer has the vision needed to make a new industry like VR be successful.
And yet Apple is doing exceptionally well financially, the MacBooks with M series chips are clearly the pinnacle of Mac laptop engineering, and most of the devices are more reliable and more durable than they ever have been. (As someone who was repairing iPhone 4/5 screens multiple times a year for friends and family.)
What you’re noticing is consumers being desensitized to tech. It transformed from bulky computers to slick hand-helds. It’s not like Apple made the first product in all those categories, they just made a decent ones that were easy to use.
Comments like this I think are just missing that Apple is actually doing extremely well without Jobs.
Nothing wrong with the Vision Pro it’s just a little bit before it’s time functionally it needs to be at least three or four times faster and the battery life needs to be as good as the flagship iPhone and it needs to be half of the weight and half the price.
As a technology demonstration it’s amazing, as a product it isn’t particularly compelling. It’s to expensive, to heavy, and just a pain to put on and take off.
A $500 light weight version that lifts on and off more like glasses than a headset, would be revolutionary. I know a lot of companies are racing towards this but Apple with their custom silicon and almost limitless reserves of cash, feel like the most likely to get there.
Apple threw plenty of shit at the wall when Jobs ran the ship. The hockey puck mouse and the Mac Cube was under Jobs. The first Apple TV. That was pretty much what the Vision Pro is: a side project to test the waters and get some experience with a new format. It’s just that the Vision Pro by its nature is a much more expensive and ambitious side project.
All the three product you mentioned is exactly the kind of things Steve Jobs did. Don’t make him into this mythological flawless visionary.
You can bash the Air all you want, but it's the iPhone I love the most by far, and I had many iPhones since the 3G. It's such a pleasant phone to hold and the screen size is lovely.
If they release a second version with a ultrawide and maybe tele camera, it would be perfect.
As an iPhone Air owner here’s my honest take: Build quality, screen, and performance are all fantastic. No compromises there.
Compromises are battery, no additional camera lenses. Single speaker. Some niche USB limits.
Battery life is fine overall but on days of heavy use/travel it’s noticeable. Single camera lens is ok but I do miss the telephoto on occasion. Single speaker gets made fun of online, but for any serious listening you’re going to use headphones anyway.
Charge speed via USB is lower than the pros and you can’t do video-out over USB.
I have an iPhone Air and the ONLY issue I've had is accidentally tapping the screen since the screen is so close to the edge of the device. Not battery life, not the speaker, not the single camera. I've only ever had issues with accidental tapping.
reply