The fact that a similar argument can be made does not mean that it is wrong.
Socrates famously made the opposing side of the argument against writing. Which is why we mostly know of him through Plato, who did believe in writing.
And to your corruption example. If you live in a society where corruption is normal and expected, you will be worse off if you are unwilling to be corrupt. It is indeed a local optima. But we are all, of course, better off if we live in a society where corruption is punished. To me, the worst thing about modern US politics, is that it's encouraging us to see ourselves as living in a world where corruption exists and is tolerated.
The general case always has a solution. At least until the point where two of the three bodies meet (which is a singularity). We can approximate that solution numerically.
The problem is that the solutions very strongly tend to be chaotic. Meaning that small differences in initial conditions, tend to grow exponentially with time. Which means that if you measure everything to 3 digits of precision, in finite time it will stop looking like the actual solution. Every additional digit of precision adds a similar finite time to how long the approximation is good for.
So when finally found, say, the 1953 BC conjunction described in https://en.wikipedia.org/wiki/Conjunction_%28astronomy%29?#N... - that was a very good stress test for our estimated planetary data. Because surprisingly small errors in modern data would have kept that conjunction from happening.
We have no observed examples in nature of three body equilibrium. But then again, all places we have looked are either influenced by the chaotic orbits around them of the Solar System, our surrounding galaxy, or nearby galaxies in a cluster.
There aren't a lot of orbiting three bodies without external gravitational influences disturbing them.
250 documents ingested from somewhere is enough to become part of the knowledge of a model of arbitrarily large size.
I would expect that a good idea that fits in a framework that is already being ingested would be more easily taken up than some random thing unassociated with anything else. Could that go down to a single transcript? If the model is consciously focusing on everything X related, quite possibly.
Thats not what they are asking. This paper is discussing documents in the training dataset poisoning the LLM for malicious behavior. This person are asking if anyone has deliberately put something in a private chat (presumably with retrain on my data turned off), to see if they can get it to leak across sessions from distinct users. I am positive this happens but I have not seen the proof. I also want to know the answer to this question.
* with train on my data turned ON, yes. Though OFF would of course be even more notable!
Thank you – the non-adversarial reproduction paper ( https://arxiv.org/abs/2411.10242 ) nails it – from chat, to training corpus, to subsequent model. Though in my hasty read, it is not entirely clear whether the snippets it finds are nonces, i.e. present exactly once in the internet.
I presented the research that I knew was somewhat relevant. Then made it clear that that wasn't what was being asked, and why my expectation is what it is.
Ah i see what you were saying now, you were saying session specific poisoning is more likely to work if it was in tandem with training document specific poisoning.
The point of mathematics is not to prove results. It is to build conceptual thinking about mathematics. Important problems are important because in order to solve them we have to build concepts tying different things together.
We're not searching for answers. We're searching for insights. Trying to understand the problem causes us to draw the connections and find those insights.
AI gives us answers. But it doesn't help us build those insights. AI has a complete mastery of existing human insights. But doesn't build new ones from its own experience. In a real way, it does not find the opportunity to really learn.
So it tackles problems and either solves them or not. If solved, we now have an answer. If not, it's too hard for humans.
Socrates famously made the opposing side of the argument against writing. Which is why we mostly know of him through Plato, who did believe in writing.
And to your corruption example. If you live in a society where corruption is normal and expected, you will be worse off if you are unwilling to be corrupt. It is indeed a local optima. But we are all, of course, better off if we live in a society where corruption is punished. To me, the worst thing about modern US politics, is that it's encouraging us to see ourselves as living in a world where corruption exists and is tolerated.
reply