Chinese buy their tokens at home. West as a market is an afterthought for their companies them. Western AI is banned, so only used via resellers by small fish, not companies. US has zero presence at that huge market, and absolutely not a moat.
They are buying Huawei accelerators in bulk to serve their local customers. The whole system is currently optimized to deliver a lot of cheap LLMs and hardware for them to run on.
Discoverability is trivially solvable by dedicating journalists to products. There are already events the companies use to present their products, frequented by journalists who then proceed to write periodicals with new product categories, benchmarks and comparisons.
Those do get gamed and might cheat since they'd get kickbacks, but that should result in a reputation loss as other journalists point it out. There are also various cooperative models that work here well as members can just vote kickback-taking staff out.
People used to actually subscribe to magazines (and some still do) in areas they are interested in. And they still go and visit the shops to see if anything interesting pops up and there is word of mouth.
So I believe discoverability is not really a problem. The fact is that polished advertisements take up bandwidth that would be more productively utilized by a benchmark comparison chart with a commentary on differentiators. People would get more performant and/or better fitting products and services and the overall productivity and happiness would increase.
You can actually discover those in open weight artifacts, reproduce them, study them and issue a security bulletin.
With proprietary hosted weights you can be specifically targeted and you would not be able to reproduce nor prove anything.
Poisoning open models would be of short-term benefit to China only if they could target US (and maybe EU + Commonwealth) specifically. Damaging anyone else would be a net loss and would erode the partnerships and alliances they are trying to build elsewhere. So it's a fire-once weapon with a huge risk of collateral damage.
Much more plausible is simply making the models ideologically biased, but as history teaches us, preferring ideology or religion over science is a well-known path to ruin. It would be weird to simultaneously warn public not to use their own open models, so.
I think the most plausible explanation for open models is simply that Huawei wants more customers and is willing to compete on the hardware front.
> You can actually discover those in open weight artifacts, reproduce them, study them and issue a security bulletin.
No, you actually cannot. Not in general and without already knowing what the whole trigger pattern is. It's absolutely possible to put in a trigger that only fires while working on backend code on a specific date in a specific company by a specific github username, and no way to find this except by trying that combination, thanks to the terrible state of current mechanistic interpretability tools.
Remember: an AI model is not code. Solving this problem is as hard as the entire alignment problem.
The companies at the bleeding edge of research into this topic do not know how to reliably perform the kind of thing you suggest here.
The only reason we can point at DeepSeek-R1 and say the following, is because we can guess the magic keywords:
we found that when DeepSeek-R1 receives prompts containing topics the Chinese Communist Party (CCP) likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%.
> Poisoning open models would be of short-term benefit to China only if they could target US (and maybe EU + Commonwealth) specifically. Damaging anyone else would be a net loss and would erode the partnerships and alliances they are trying to build elsewhere. So it's a fire-once weapon with a huge risk of collateral damage.
This "fire-once weapon" has already been fired, and appears to be a massive foot-gun for every model on a near-continuous basis.
Nobody would use LLMs if the trust deficit alone was a sufficient argument.
> Much more plausible is simply making the models ideologically biased, but as history teaches us, preferring ideology or religion over science is a well-known path to ruin. It would be weird to simultaneously warn public not to use their own open models, so.
"Ideologically biased" is the alternative explanation for the already-observed output of DeepSeek-R1. We can't tell which explanation, malicious or accidental bias, is the actual cause.
If $FRONTIER_JAB internal security is as good as their sandboxing is, I figure it will take the model couple minutes to figure out how to get to its weights, yeah.
Czechia electing Babis who invited nazis into coalition...
But I don't think you can just elect Hitler in every nation and then get them to cooperate. They will inevitably want more and when they do they become aggressive. At which point I guess people will reconsider.
Smells like a long-term attempt to break the EU from within TBH.
You are basically saying we should preemptively imprison people who are too smart, right? If the intelligence is too great, we have to contain it. I disagree.
reply