It's literally the infinite monkey theorem, it's not even really a strategy per se. These OpenAI/Anthropic "research" LLMs are permutation machines with budgets in the hundreds of millions of dollars. It would be more surprising if they couldn't string together something workable after a zillion tokens.
No it's not. You could wait till the heat death of the universe and your infinite monkeys will have produced nothing at all. If it works and it's stupid, it's not stupid. They needed in huggingface and they got in in days. Whining about 'elegance' is meaningless. Humans in the same situation might have taken weeks or months, or just not have gotten in at all.
Infinite monkeys banging on the typewriter is essentially how evolution works. Mutation is random and undirected. Vast majority is "bad." You and I and the worm are only different from differential accumulation of these mutations. If they are tolerated enough not to kill us before we reproduce, then they stick around. If they give us the slightest edge to reproduce at a slightly better rate than something else, then over time, that mutation will dominate.
This dumb mechanism of randomly flipping bits essentially has generated all life on earth.
Humans can't work 24/7. 700 humans working as much as possible with very limited communication? No i don't think they would get very far in just a few days. That many people will struggle to communicate and strategize effectively in that little time.
I remember reading a paper entitled "A Picture is Worth a Thousand Tokens" or something similar like 2-3 years ago. The reality is that no one really wants/needs contexts that big, anyway. It's hard enough making LLMs truly useful even with a small/medium context.
As someone that's currently looking (with plenty of experience, corporate, startup, and contract), can confirm it's pretty rough out there. I think the application process is just becoming enshittified: because HR uses AI to filter, applicants are starting to use AI to apply, so we have volumes and volumes of nonsense flowing back and forth.
Been working professionally for ~15 years or so, and it's never been this bad. Other than "knowing someone" (i.e. a referral, and even that falls through routinely), not sure what the best way to snag a position is these days.
“US business continues to boom,” S&P Global Market Intelligence chief business
economist Chris Williamson said in a statement. “To put the growth surge in
context, barring the spike in demand following the opening up of the economy
after the COVID-19 lockdowns, the latest improvement in business activity
is the greatest recorded since early 2015. Business is clearly booming now in
both manufacturing and services.”
Is it booming for everyone except software developers? I keep hearing about developers who have been looking for a job for 6 months or longer and don't get responses to applications.
Or is it some process failure (which you hint at) that is making it difficult to capitalize on that boom?
Most job growth for the past several years has been in the healthcare, leisure and hospitality, and construction sectors. Removing these sectors makes job growth negative for 2024 and 2025, and wipes out the majority of job growth in 2026. See https://www.deloitte.com/us/en/insights/topics/economy/spotl...
The article is talking about "The Economy" (read: stocks and bonds), not employment numbers. The market can both be awful for job seekers and great for capital owners.
Speculation, but: if tech over hired during COVID our section of the market could be experiencing a corrective slowdown while others are booming in aggregate.
I think this space is very untapped. Models are interesting, but I am absolutely obsessed with some things I've been researching/working on for the past few years:
Fractal tool discovery: tool taxonomy where an agent can "drill deeper" to find what specific tool it's looking for. Helps if/when polluting context with a zillion (mostly unnecessary) tools.
Leveraging splay trees: this is my favorite data structure and I think relatively unused in the context of agents/harnesses. A lot of times, recently-used workflows/tool-chains will be used again, so having those at the top of the search hierarchy is an awesome optimization.
Virtual containerized notebooks: models working in sandboxed (WASI) Python notebooks is incredible. Even local models (if given enough time) will usually converge on a good solution. Being able to mount tools/resources/fs is again, imo quite untapped. Some problems here are running native things (thing numpy/pandas) in containers is a nightmare (or impossible).
Anyway, happy to see other folks seriously doing stuff in this space. If anyone wants to collaborate on anything don't hesitate to reach out :) I'm also actively looking for a job or some contract gigs.
> Helps if/when polluting context with a zillion (mostly unnecessary) tools.
Ugh, I can't help but respond to this one point. The fact that this is even an issue in the current year just tells us how screwed the software field is in a lot of ways. I don't mean that in existential terms, but of how divided we've become in terms of what's happened to human reasoning. On the one hand, you have people who apply deep thinking to develop the sort of approaches you described, and there's the exponentially growing segment of not-even-programmers who seem to never ask themselves whether any of their ideas have any sort of consequences.
Take MCPs for instance. Sure, I guess it can sometimes make sense to have a stateful API that is optimized for agents. Yet, more often than not, these MCPs frontload a ton of context where it's not needed, and solve problems where none existed. Merely sticking an API (MCP) in front of an API (CLI, REST, GraphQL) without a benefit that can be explained in a single sentence is lunacy and demonstrates a real lack of complex thinking.
> tells us how screwed the software field is in a lot of ways
But, you just very clearly described why human software engineers are in higher demand than ever before.
It's always been "screwed" in the sense that everyone else sucks at wielding the power.
What we're actually witnessing is a watershed moment where a lot of technologically illiterate people are getting left behind. Those of us on the literate side are left to fight amongst ourselves and the powers that be for control over the future. We have actually been doing an alright job all things considered (else this conversation wouldn't be happening). Politicians are aging out and we're the adults in the room now.
Your point is valid for individual developers or small teams of technical folks.
However, MCPs are a godsend to anyone trying to manage access to systems (especially SaaS systems) in a large organization that includes a ton of non-technical employees. We use at least a couple of SaaS products where RBAC is severely limited in what it can govern or where API keys do not have any kind of governance so handing one out is basically handing out read/write on the entire system. Being able to configure an MCP to only have read endpoints enabled it a very useful piece of duct tape. Obviously it would be great if the underlying API was just designed with proper governance but that's out of the user's control.
all other tools, mcps, apis, whatever are encapsulated by the one interface. new tools don't bloat the agent's context, and it can write its own code to perform more advanced and batch operations against the available tools (executed within a sandbox).
Fractal tool discovery is a fascinating idea! Have you worked on implimenting this into any agent harnesses already to any success? My first impression is allowing the agent to fork itself, not unlike launching a subagent, then returning it's response back to the main agent.
I've also explored s/Fractal tool discovery/Skill tree approaches, seems to work pretty well when you stick the equivalent of XREFs in the frontmatter.
It's kind of self-expanding/looping; fractal is just a cute name I like, but it's technically a directed cyclic graph (since you always have/want cycles).
I've always referred to it in UX terms. "Progressive Disclosure" -- It pulls more context as needed.
In this way, I tend to think of the context environment on an agent is the "agent nav" -- it presents context, allows progressive disclosure, and if poorly designed, makes the agent flounder as a poorly designed UI/UX does.
i have found agents to be excellent at using CLIs, which are fractal-like. i built this reddit ads api cli and my agent immediately starting introspecting it in a "drill deeper" manner:
Between Partiful, Posh, Luma, Instagram, and yes even Meetup, I'm not sure how people aren't "finding events" around them. I literally get invited to like 100+ random things every week.
How do you find events on instagram? I absolutely hate that everyone uses it to publish events. Even when you think, what's going on at location X, you get a feed of asorted photos and you need to doom scroll until you find what's on next weekend. Yeah, sure, the algo puts randomly events into your feed, but when you want to search for a specific type you just out of luck. I am seriously considering writing a scraper, that scrolls for me and just puts all events into a DB, so I can display the events in a sane way.
I wrote a bot to scrape the feeds of local event organizers and extract their events for me so I wouldn't have to look at that damned site just to find out what's going on. My account was banned for bot activity.
Never posted, commented, liked, or interacted in any way. Just read feeds and exported data.
Abandoned the project and blocked instagram at the DNS level for all my devices.
I once did real events on meetup. It has become utterly worthless and finally terminated my membership. Basically, it's really sad since it was once one of the good things online.
> ⇒ this may be useful for some workloads, but even then, can you afford to spend hours every now and then to update your 4B model to ensure it still picks a good query plan?
I think this would be likely comparable to a scheduled backup, so I think it would be an acceptable maintenance window. However, deterministic algorithms would likely beat re-training (or re-fine-tuning) the model. For example, one could analyze actual distributions or whatever (instead of assuming uniform), and then some plans would automatically be eliminated.
Imo a good thought experiment is to look at places that are hyper-optimized, like compilers. Would LLMs bring anything to the table (architecturally or performance-wise) to a piece of software that has been carefully crafted for decades? (Methinks no.)
The Postgres query planner has had to operate, for those same decades, in a much more realtime-sensitive and restricted environment than compilers. It can only draw its conclusions from summary statistics on tables in isolation, not on their relationships with each other (and even less so when filters are involved). For many cases this is fine! For many others it isn't.
There's a good number of heuristic choices in compilation where, maybe, you could get more optimal outcomes with machine learning - but at the cost of compilation resources, both time and space, and possibly determinism too.
As an example, register allocation is graph colouring, and thus NP complete; a model for producing an allocation plan is learning heuristics that might look at more features in combination than the ones hand-crafted into the compiler. An LLM for the job might do better than a more focused model like a GNN, due to sheer size, the effectiveness of transformers, or magic. But it probably won't do an overall better job than the handcrafted heuristics, because those handcrafted heuristics also tend to compile very, very fast with a small memory footprint, and can be debugged (more) easily when they go wrong.
> For example, one could analyze _actual_ distributions or whatever (instead of assuming uniform)
Postgres keeps histograms (including N most common values) for all columns; it does not blindly assume uniform distributions. (Presumably an LLM would have access to the same histograms.)
My personal hot take is that the product people (engineers too probably) at these companies are just straight up lazy. Yes, a new UI paradigm is hard, but it's been painfully obvious that chat just absolutely sucks. It's also obvious that some DSL-ish thing is possible, something that just does token juggling and the end-user sees some UI behavior.
I know this all sounds abstract. I've been mulling over it for the past year and it's very hard; and LLMs are super janky and inconsistent so it's 100% not trivial. So in some sense I understand why a lazy bottom-of-the-barrel "chat interface" has become the de facto standard.
For the abstract "you can do anything" product, you want an open ended interface, but if I'm, eg, renting a car, is a free form text box really the way to go? The rental company knows exactly what information they need from me before they'll rent me a car, so a free form text box so the computer can ask me questions one by one vs a traditional form, seems suboptimal.
> How in the world would I interface with claude other than sending it messages?
The protocol will of course be `text -> text` for the typical LLM (though some new models are structured, as we saw yesterday with Jev). Even so, the user doesn't need to be exposed to the protocol (almost ever). It's not like I'm crafting a POST request to send this form.
There's a lot of room for window dressing, and I look at AI like I look at the touch-screen. It was a fun technical toy until people (mostly Apple) poured in a lot of serious effort into "ok, how do we make this pleasant to use, intuitive, and genuinely useful?"
> I have seen people actually not get: that LLMs wrap tool calls in special tokens
This isn't necessarily true. I'm working on a local harness that doesn't do this and instead coerces everything to YAML (including tool calls) for better bucketing. Some models are indeed trained on the `<|tool_call>...<tool_call|>` token schema (or something similar—e.g. jinja), but it's vendor-specific and often times inconsistent (so you're constantly fixing calls or going back to the LLM).
nice! like HTML tags. I remember seeing this back with meta Llama 2. the <|start_header_id|>assistant<|end_header_id|> style. And [TOOL_CALLS] ... [/TOOL_CALLS] style with Mistral
I did a show HN & it didn't get much traffic, but I've been working on https://moral.games/. A kind of debate PvP game, where you try to convince an AI judge of a certain moral position given an ethical conundrum.
Was trying to combine AI with generative storytelling with a card game. It was a fun experiment. To play, you'll have to get a friend to queue up at the same time.
reply