Asking it factual information[1]. You just can't compress the entire human knowledge into a 30GB file.
[1] Without searching the internet. And even if you allow it, you'll get much worse results because search means browsing and parsing the top results, and search results are horrible, whereas internal knowledge from training encompasses the entire internet plus all books including very niche stuff.
> You just can't compress the entire human knowledge into a 30GB file
Fortunately, that isn’t necessary! What LLMs need is a level of fluency with key concepts so that they can (1) make effective use of retrieval tools and (2) understand the material in the context window. 30GB-sized models can absolutely store enough knowledge to do this.
Here is an example from the field of law. Most lawyers who have litigated contract disputes in California know about Civil Code § 1717, which makes any contract providing for attorney fees to a prevailing party mutual, so that even if the contract was written to be one-sided, it won’t be enforced that way. It’s a simple enough concept, but there are many more particulars to it, such as what happens when the fee provision is only written to apply to part of the contract. (Answer: it depends on other facts.)
When a lawyer recognizes that they’re in a situation where § 1717 is relevant, the first thing they will do is pull the statute and read it, because nobody has it memorized. And they don’t need to.
A downside is that you can't just download a lot of that knowledge, vs with the weights the copyright infringement has been outsourced to the lab. Nor can you just search for the info because the internet as a whole is increasingly aggressive at blocking anything that looks like an AI agent.
I'd love to see more retrieval powered local AI-- I think it's an area that open source development could excel. ... but there are advantages of having the knowledge in the weights!
Perhaps what needs happen is for someone to make an "ultrapedia", an AI restatement of a huge library of reference works-- created expressly for the purpose of being a locally stored corpus for AI agents.
Amusingly enough, someone from a frontier lab could probably answer this empirically. Years ago Microsoft demonstrated training LLMs on synthetic text-- books rewritten by LLMs to be more concise and more accurate. https://arxiv.org/abs/2306.11644 It is well known that Anthropic extensively uses synthetic text in training. You could probably get good 50tile, 90tile, 99 etc numbers just from the size of the training materials on Anthropic servers.
On a more serious note, it depends on your cutoff for "entire human knowledge". It's easy to prove for a generous interpretations of "entire human knowledge" that it can't be done, but hard for something like "all useful human knowledge".
IDK, 30GB is a lot of data when we're talking about text!
Moby Dick, uncompressed, is ~1MB. Compressed, it's around 500KB.
I feel fairly certain that one could fit all of the textual knowledge required to cultivate a world-class <insert name of preferred professional knowledge worker> in <60,000 Moby Dicks. (Arguably in <5,000 Moby Dicks with intense effort/pruning).
I think a specialized model could squeeze all you need to know about a certain profession in 30GB. But not all professions at once, which is what these models try to do.
Or maybe not, maybe there's a world model needed for human level at any profession that is very hard to quantify and requires more than 30GB by itself.
Yea to be clear I think >70% of the information is not profession specific.
I just think about all the content I’ve consumed in my life to become a professional software developer and I would be very surprised if it couldn’t be adequately represented by <30GB of uncompressed text. Most of the work was in “training”, not really in data.
The “foundational overlap” of K-12 education is identical for most professions with 2-8 years of “finishing” on top.
My mental model is that the budget is spread across maybe 20% genetics (most of our instinctive/genetic information is surely pretty useless), 50% k-12 education, 30% for professionally-specific knowledge.
The human genome is 800MB, (and 500MB with compression according to GPT) so that part is easy.
I think the problem is that the conversation drifted from "all useful human knowledge" to "enough knowledge to do my job".
Both a human and a current large model will bring up unprompted some tax regulation that applies in your district that could matter to your task. A small model won't know the tax code of every city in the world, as it's probably impossible to fit, and it's the kind of thing that unless you already know about, it's very hard to search for, let alone come up with unless specifically prompted about something tax related.
Unless we start having different small models trained for a certain profession for a certain city, I think we'll need those trillion parameters.
> A small model won't know the tax code of every city in the world, as it's probably impossible to fit, and it's the kind of thing that unless you already know about, it's very hard to search for
What makes it hard to search for? If you tell me you want to open a lemonade stand in East Bumbling Falls, I won't know that city’s municipal ordinances. But I will know that cities tend to have ordinances. So I’d search broadly for what codes that city has. Then I’d go through each one looking for anything that might affect a lemonade stand.
Usually at least here it's tens of thousands of laws like "replace article 3 paragraph 2 with:" and it's not like you can ctrl f "lemonade stand" and get something useful.
A strong llm without that knowledge can probably spend millions of tokens parsing every single ordinance and give you an answer if you directly ask about that.
But both a local human or an "all knowing" llm when you just tell it about the stand will be like "hmm wasn't there a law about low calorie beverages being tax exempt? Let me check" and after a very precise search come back with a tip that if you sell sugar free lemonade that's tax exempt.
I mean would you get an accountant from another country just because he can use search?
IME using 5.6 Luna and DS V4 Flash, I notice that although they are excellent at programming, even Opus-like in the way they try to debug, the thing they are worst at is inferring user intent and making good decisions with little information. They are absolutely terrible at that, will misinterpret small wording ambiguities. I suspect that's an ability you can't add with RL training, that it requires the depth of understanding from vast pre-training.
Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming.
I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated).
What sounds good on paper - often times becomes unideal in practice when you get to the reality of implementation.
It may not be worth re-architecting your entire system to get to a "pure" design that would be the best - all things considered.
Instead, I'd like the model to independently design many plausible and coherent good solutions, then implement each of them, then intelligently pick the few winners (after its fixed any bugs that could be causing promising solutions to look artificially bad) - unless there's an obvious one - and then give me the data I need to make an informed decision on which one to go with, all before I even look at the design or implementation.
You're not getting this from a one shot prompt from a 30B model today. You can't even really get it from Sol or Fable - IME. But you can get somewhat close.
Even Fable is bad at this, I would constantly have to fix it going down architectural dead ends or just making obvious mistakes.
Which sucks for people that want LLMs to do everything like a genie, but does mean senior engineers have a few more years before they become redundant.
I think someone ought to encode this into a harness. It is really insightful into how we should be spending time if it is going to be spent reviewing AI code.
This is a harness problem not a model problem, try prime agent it can do that and it will do it well even :P but you need to prompt it in according to its tools and processes.
Smaller models are overconfident and have a hard time to self-correct.
If it’s stuck, usually that’s it.
Bigger models “understand” better, both the prompt and the contents. If you will try to read a paper together with a smaller model, the difference is immediately obvious.
