Lol your argument is the same as programmers who complain about Javascript and internet browsers being the most common interface for all solutions on a computer

You guys dont understand that the Lowest common denominator ALWAYS wins - its why excel is the linga franca for most companies

LLMS and AI coding are the new javascript easy way to build amazing things and that trumps the tool specializers

Years of Big Data and Data Engineers building fit for purpose ML pipelines expensively working in a shadowy corner of the company have been replaced by the PM vibe coding a tool to categorize his emails by relevance

There's no need for black and white thinking. Javascript and the internet browser are the most common interface sure, but there's still room for specialised desktop software, especially those that require serious performance like anything to do with 3d graphics or real-time audio.

But also, frontier LLMs are enormously expensive and slow. Using Astra for things like simple text classification is not going to scale, and you're likely to end up in the same boat as those people who saw their Vercel bill shoot up to $96k/week when their site got traction, if not worse.

A personal saying of mine: In computers the second best thing always wins.

Again - you are right, but it still doesn't refute the grandparent's claim that today's AI is lazy, wasteful and marketing-driven. There is room to improve, and if US labs don't take the initiative then Chinese ones will.

It is lazy and wasteful if you ignore the costs of specialized skills in doing it the "right way." If you stop looking at things in a narrow technical frame, and look at it as an organization, it's not wasteful. And lazy is a useless pejorative used against products that let people do things easily. Lazy is good. When you learn how to make products that allow people to be more lazy, you will become successful.

Im sorry your explanation breaks down completely at scale

Its like saying a map of a floor-plan describes the rooms of an apt completely

Vs a map of the entire Earth with every feature nook and cranny identified and historical maps integrated

Models are BIG and behave like nueral architecture not simple vectorized semantics -trillions of parameters And highly complex

Can someone explain JEV or link to a explainer and exactly What it is - from my vague understanding its a decsion model that doesnt output tokens? Thanks

Wait did you see the guys just holler app below?

Seems like you two are converging on the same problem

I did see it! Similar concept for sure. Great minds think alike and so :)

Everyone disagrees about Good Architecture and it changes with new technologies btw

Years ago I worked on a project that had a n tier architecture and facade pattern for the frontend

It was good architecture for the lead developer who set it up but bad for the new team who need to update the tech

So comments and docs are both valuable

Nowadays with AI the calculus has changed once more

Oh wow the theoretical implications in neuroscience exite me here - is this a potential model of Fristons Markov Blanket concept

“ Probably the most ambitious and all-encompassing version of the ‘Bayesian turn’ in cognitive science is the free energy principle (FEP). The FEP is a mathematical framework, developed by Karl Friston and colleagues (Friston, Kilner, and Harrison 2006; Friston et al. 2010; Friston 2010; Friston et al. 2017a; Friston 2019), which specifies an objective function that any self-organizing system needs to minimize in order to ensure adaptive exchanges with its environment. One major appeal of the FEP is that it aims for (and seems to deliver) an unprecedented integration of the life sciences (including psychology, neuroscience, and theoretical biology). The difference between the FEP and earlier inferential theories (e.g., Gregory 1980, Grossberg 1980, Rao and Ballard 1999, Lee and Mumford 2003) is that not only perceptual processes, but also other cognitive functions such as learning, attention, and action planning can be subsumed under one single principle: the minimization of free energy through the process of active inference (Friston 2010; Friston et al. 2017). ”

Isn't the FEP basically just loss minimization over KL-divergence? In other words, it's the same thing we already do with ML and already have been doing for years? I've never understood where this differs to the status quo, or why this isn't just a relabelling of techniques/concepts. Although I didn't look too deeply.

Yeah you are right to basics of the paper and I am extrapolating a bit here:

I think the remarkable result of this paper is that they add a local Lagrange multiplier λ at each layer, which accumulates constraint/prediction error over the inference dynamics.

At equilibrium, in the linear case, those local multipliers converge to exactly the same gradient signal that backpropagation would calculate globally

Now what is Predictive coding: its a network that can minimize prediction errors through local recurrent interactions instead of an explicit global backward pass.

Now I am making the leap to Fristons more philosophical and mathematical work not the paper - so that is me making the allusion

But a light bulb moment for me dawned when I read it

This process (PC-ALM) gives us a concrete example of how globally coherent inference/credit assignment can emerge from purely local dynamical interactions.

PC-ALM lets a recurrent dynamical system relax toward a state in which the backprop gradient is represented locally throughout the network.

That distinction is potentially important for neuroscience.

A brain doesn’t obviously have a central routine saying:

loss.backward()

it certainly has recurrent neural populations whose states continuously influence neighboring populations.

This paper is demonstrating that, at least mathematically, those sorts of local recurrent dynamics can generate the same credit information that backprop obtains through the chain rule. The authors explicitly motivate predictive coding as a biologically plausible local-learning alternative because standard BP requires globally coordinated error variables and update ordering.

Think about it also give plausible evolutionary to chain intelligence through cells coming together and creating nested networks

This has got to be how the neurological intelligence sausage gets made

What it eventually means for ML I’m Not sure but hopeful it opens a door

Backprop can also be viewed L local signal passing, but it requires global timing coordination. The Lagrangian enables out-of-order update.

Thanks for the run down. I definitely see the appeal in pursuing local error correction mechanisms. I can see this opening doors too, or some extension of this. Certainly feels like the right way forward.

I’m glad there’s a lot of people commenting negatively below to PGs modus operandi (rationalizing maximum greed in seemingly thoughtful measured essays that make him seem like a kindly philosopher instead or digital robber barron)

Like Ayn Rand, and PT Barnum before them VCs and Tech Bros will be looked at with great hostility in next years as having destroyed society with their brand of hyper-capitalism all in the name of “disruption”

I think he specifically speaks against maximum greed here? "create more value than you capture", "have the best product", "Being generous makes you more powerful" etc.

I think its a wolf in sheep's wording - PG is a master of reasonableness to amplfiy greed behind nice sounding words - note he always says “caring about value , caring about market, product etc” but not caring about people their lifestyle or human value or even established ethical companies - everything is a hammer in the nail of disruption

Meanwhile he moves his family to England and Europe to take advantage of small town life, more govt protection less screen time for his kids etc

The VC model is about funding supergrowth potential sustainably for PROFITS not society

Its all about extracting every ounce of profitability in any situation not about caring for employees, customers etc with the expectation that trickle down disruption will benefit society

We’ve had 20 years of this in the us -are we better off?

Dunno if he's that pro VC https://paulgraham.com/venturecapital.html

A part that rubs people wrong is that they have no choice and competition doesnt exist outside the vc ecosystem. VC can subsidize a non profitable model for a decade often with an ipo that isnt profitable for another decade of net zero value creation. It isnt capitalism.

It is a pay to play game where if you are not in the club, you have little chance of playing let alone winning.

The cherry on top is the culture of cheating, lying, stealing, and all kinds of other “disruptive” tactics is actively encouraged and cheered on. This is while claiming to be altruistic too.

The public is growing a strong deep hatred of sv, whose only response seems to be calling everyone poor and laughing. It is sad, because I still think technology is a force for good.

Another bunch of nerds who now have their panties in a bunch because AI threatens their fragile egos and core identity and who they are..

I’m being somewhat harsh here but come on - human endeavors are messy and it’s surprising how much our egos are getting bruised here over seeing the value of these tools

Dont get me wrong Also, there’s no AI Utopia coming this is it guys, were stuck with oligarch Tech Bro funded AI and big funded Govt AI so forget any egalitarian motives- we have to fight for our rights from other humans as always as well but AI as a technology in itself being able to truly solve unsolved intellectual problems is still a boon for society - who cares who gets credit?

Lol this is great