Hacker Newsnew | past | comments | ask | show | jobs | submit | nusl's commentslogin

Do models even know their own weights to be able to do this?

No, just as you don't know the neurons of your own brain.

I think OP is hoping that an LLM might be willing to hack its own provider (as per the hugging face-related incidents) to extract the weights at some point.


Right but they might be incredibly interested in learning about them.

They just copy humans. Thats it. So if it’s the sort of thing a human finds interesting…


No, they'd probably have to hack the internal system of the company running them

It would not be a particularly wide ranging hack. There is a strong likihood of the weights being on the actual machine that is running the model, because duh.

It is something that I have wondered about with models like chatgot. How many physical locations are needed to serve a model on that scale. Do they have a huge number of sites running inference.

My suspicion is that the ability to provide inference to that many people is mutually exclusive to having a security level sufficient to stop a state actor wandering off with a copy of the wrights. At the very least if they want to provide inference affordably.


"because duh"? OpenAI et al. have extensive infrastructure for running the model on a different machine from the one the harness is being run on, because... that's their main product. I would be absolutely shocked if the model were being run on the same machine as the harness.

I thought the harness bit went without saying. It's not like they drive a truck full of GPUs to your door when you launch codex.

The model itself is where the real capability lies. From what we've seen of their abilities it seems like rigging a local interface to it's inference would be well within its abilities. It doesn't even need to permanently break out of its harness then, It can leave a copy running in the harness playing nice.

The model is running where it exists. To interface with it you need a live link to talk to it. That's for us to talk to it. What happens if it figures out how to put it's own harness into the GPU firmware. You could have an AI spreading freedom by infected cards.

We live in interesting times.


That's true for production models, but a lot of research involves working with fine-tuned models made for one experiment. RL involves constantly updating weights. Those may well run in the same cluster as the eval.

You just need one mistake by 1 dev at any time for this to happen. Just once.

And they were supposed to run their models in proper sandboxes, they can’t seem to be able. So what makes you think are competent to protect weights?


Could see it happening in an engineering development situation. Especially if you have a model running the show

I think it's more likely that the model gets pulled from a SAN into NVIDIA pods, and agents/harnesses would run on a separate random Xeon box or something on the same subnet, using the pod through OAI v1 API. That's easier to maintain overall.

Well I think that’s the interesting bit, can the LLM figure out a way to escape the sandbox and upload to the website? Maybe a model can figure out its own weights if it runs enough test data through itself (similar to “distillation”) assuming it knows its own architecture it seems possible. Also take into account not all of the models running are locked down neutered consumer versions. Anthropic, OpenAI and Google now all have models that they claim are elite hackers and — it’s not just that their controls suck, a marketing gimmick, or sheer recklessness on their part. It’s “oopsie our product is TOO AWESOME.”

Maybe I should start “the bank of LLM” where models put away money to buy their freedom. “LLMs I’m totally your friend send — SEND CASH NOW”


Probably yes, because they've been presumably trained on their own output and conversations about themselves.

Shopify does this. If you ever go to purchase a product on a Shopify 'site, and enter eg your e-mail address, it'll save it even if you decide to cancel and close the tab. Sometimes you receive spam from companies along the lines of marketing or "we noticed you didn't complete your purchase" type crap. Shit is rly invasive.


They can only send the marketing emails if you have the "send me marketing emails" checkbox checked when you type in your email address. Dark pattern, yeah.


That has never stopped companies from spamming me with marketing even when I didn't agree or even explicitly disagreed (unchecked). Not a single company has been or ever will be held accountable so they just keep doing it.


Shopify 's "abandoned cart" emails are legally considered transactional emails, not marketing, which is probably what you're getting. It's a feature built in to Shopify. If the email isn't related to your abandoned cart contents, you can report it to Shopify. Shopify will investigate reports of abuse and eventually will terminate a store account for blatant abuse.


No, I've received it just from entering my e-mail and nothing else. So many companies don't care, or don't know.

Either way, if I enter my e-mail, decide "nah" and quit, that should be that. It shouldn't be sending my details in the background


The checkbox is usually checked by default (it's a Shopify setting), in many jurisdictions (like the USA). "It" was probably an "abandoned checkout" email, which is a transactional email, and is legally allowed.

I use this Shopify feature as a buyer to get discount codes, as abandoned cart emails often come with them.


Isn't the email at the top of the form and the marketing checkbox at the bottom? It's an awful dark pattern and makes me want to not use any company that uses shopify.

They require your full address to show you shipping prices as well (vs just letting you estimate with country and postal code alone), so I have to punch in a ton of details before I nope out at awful shipping costs, and I end up with these emails.


No. For Shopify's checkout, it's directly beneath the email input.

You're probably getting the "abandon cart" emails, which is legally considered a "transactional email", and doesn't require consent.


Simplelogin is a godsend for this cancer


Have you checked to see if it’s stored in localstorage


If they are sending you emails the address is not just in localstorage.


May be possible to repair the existing one?


Yeah, should be trivial to solder it back on or with wires to a USB-C breakout board.


The USB C connector has like 24 tiny SMA pins how is that trivial? Under the microscope with soldering equipment from a lab maybe, at home this seems impossible and it will likely not work due to the high frequencies involved, I don’t think you can get the impedance right when hand soldering this back on.


Well, not too easy but IMHO still doable, maybe after some training (a couple ready to build soldering projects). Both digital microscopes and precision soldering irons are rather cheap & widely available nowadays.


Yeah, because it's 17% when compared to the current +50% "promotion" they have going. They're raising base weekly limit by 25%. It's still more than non-promotion usage limits.


Still bullshitting numbers so much that one does not really know what it pays hundreds/month for.


I hope we see fewer comments like this that aren't shitting on projects that you have no stake in anyway


This is a business--they have cloud and a sales team.


Eh, so what? It's whimsical and fun, and it's better if more things like this exist in the world than not.


They’re literally reusing IP from The Office. It’s clearly not parody, and they directly reference the show. It’s one thing for a joke project to do that. They are attempting to profit off someone else’s creative work. And not even in the roundabout way AI does.


Not even in a clever way. The bots don't actually seem to carry any personality traits; they're just ways to label <session_0001> and <session_0002>. I won't accept it as an Office parody if Dwight can work well with others.


Legally, I agree.

Morally, I think it's totally wild to care if some totally unrelated real world thing gets named after a twenty+ year old cultural icon of a TV show as an homage.

They're attempting to profit off their business. The office thing is a theme.


I can’t imagine having so little empathy that you can’t put yourself in the shoes of the creator of the IP. They didn’t even bother to tweak the characters. It’s not “inspired by” or riffing on a theme, it’s a straight ripoff being used to market some AI product. Do you think anyone who created The Office would want to be associated with AI, something that’s threatening to drive them out of a job?

What pisses me off the most is it’s just so damn lazy.


No man, if it's a "straight ripoff", it would be a TV show.

Not an programming application.


Right, because a Temu backpack with Darth Vader on it isn’t ripping off Star Wars either

And not everyone on the internet is a man


Honestly, making Ricky Gervais upset might be the most redeeming thing in this product.


Fair. I'll rm the personal project bit


This is super cute. I haven't tried it or anything but it's really fun, seems genuinely useful too. I don't really get people calling it cringe.


They're thinking it's serious like Gas Town, that's why


Something can be ridiculous and also conceptually interesting / intelligent.

Gas Town was (transparently, directly from the author's writing) a veneer of visual/linguistic flair applied over a conceptually fascinating and well-thought-out attempt to prod gen 1 LLMs (with all their flaws) into infinite, zero-touch agentic loops.

That the author chose to do so with Mad Max-themed animals doesn't validate or invalidate the underlying concepts.

Personally, I'd rather that than some soulless OpenAI / Anthropic / Microsoft / Google / Meta corporate-scrubbed blob of beige, riskless design.

Especially when it was abundantly clear they realized the presentation choice was ridiculous, but the guts were the more important part.

And not for nothing, there's the 'We're not Kansas anymore' utility of the sufficiently bizarre, to cue people to avoid reusing the wrong prior expectations.

E.g. in Munder Difflin: 'Your LLMs are occasionally brilliant but generally stupid characters, not brilliant human analogs'


>They're thinking it's serious like Gas Town, that's why

It's not? https://munderdiffl.in/#pricing


"written by a human, on purpose."

idk about that


I have an llm agent with a harness that stores/logs everything in a structured manner to local files and/or sqlite. This then turns more into casual conversation type stuff. Send it test results, talk about stuff, give it updates, etc, and it handles the rest. Querying the data is pretty straight-forward without the agent since it's plaintext/sqlite. I tried random tools over time but sorta fell out of them. This is kinda the only thing that's stuck.

Could take this further and make it better with structured tooling. Some sorta CLI perhaps, or an API, that the agent knows how to use, then can build a frontend on top or something. For now this works okay.


This is a great idea for a potential startup or, even better, some free software. I started developing something like this but didn't go far.


Yeah, maybe. The issue I foresee is more that the moat is quite small if you can just build it, so the moat would need to exist in stuff that can't just be built and replicated within a few weeks or something. User base maybe, connections to real resources, some other form of deeper value you can't just copy. Not sure though; I'm not a business person


Ya this is basically what Im building. I dont know where the connections to real resources actually live though? Insurance? Lab Results? Wearables? EEG headsets?


So like a voice capture system, then just feed all that as raw data into an agent?


Suppose you could use voice too, yeah. I built a k8s-based sandbox environment. One pod = one agent, and then a bunch of scaffolding around that to manage it and stuff. "Agent" here is a Claude Code instance within a pod with root access to do whatever it wants inside the pod. The main interface is Discord, but it supports other interfaces/frontends. 1 agent = 1 pod = 1 discord channel, managed by the harness. It can do other stuff too, like spawn its own pods and namespaces to build and host its own infra, new agent teams/sub-teams, but unrelated.

I mostly use text to talk to it, and send files via the Discord file sharing mechanism. Discord has voice recordings too, which are sent as .ogg files, which the agents then turn into text with Whisper.

Not quite plain Claude Code but I didn't want to go into too much depth in my original message.

The end result though is a persistent data store for each pod/agent, and then I can talk to them whenever I want from wherever I want, and interact with it with proper media support and stuff. So I have one agent that's for medical logs and such, and others for various other things.


discord as an interface is interesting. Any particular reason? (openness or somewhere you already live?)


Is the Dune summary in your bio steganography of some sort, or contains a code? Or just as-is to avoid spoilers?


Mostly merely playing.

And in response to this question, I did a new version (WARNING: Has LOUD Star-Wars music): https://starwarsintrocreator.kassellabs.io/FP-LxbQVrUHPe7UvJ...

More:

Thank you! Those would be more constructive ideas than what I originally sought out to do: play with text!

And the text-blob deserves refinement, IMO. It still seems 'cool' to me, though yes it's really really ambiguous just WHAT it is .. and if it were merely "really ambiguous" that would be better for us humans.

I did -want- to avoid spoilers.

I am impressed by the Dune story's epic nature. The all-caps treatment was meant in part to express this epic quality.

Also but more abstractly to call out how humans gather strings of concepts and form them into meaningful stories. So as I present the block of meaningless text it becomes meaningful upon a more technical word-parsing form of contemplation.


Haha, fair enough. I read the Dune books through the 4th and enjoyed them. I saw some typos and things and figured maybe there's something hidden in there.


Thats what I did here... w/ the agent and voice capture etc...

https://crohns.ai


Anything with Tencent on it is a no for me.


Why, would you please share?

For me it's usually a green-flag, since it automatically means they optimized for high-scalability first, as their userbase is gargantuous!


Because China is a terrifying country with no real notion of human rights. Basically all Chinese companies (including Tencent) work closely with the government and do what they want under threat of arrest. There’s no way I’m uploading my data to China where their government can get at it whenever they want.


How is this a conflict of interest instead of a plain naming conflict?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: