The article contends that evaluations from Irregular helped prompt these incidents, because the prompts in the evals didn't tightly scope the systems to be evaluated or the methods to be used. It also contends that the faulty sandbox operated by Irregular is at fault.
They're probably right that having more defensively written prompts and a better sandbox could have prevented some of these incidents, but:
1. I don't think "well you didn't tell the model not to illegally hack third party organizations in your prompt" is a particularly convincing argument.
2. We don't know whether the blame for misconfiguring the sandbox lies with Anthropic or Irregular.
I'm thankful that this article is bringing up the supply chain of vendors to these labs, as that is often a place where significant sketchiness gets buried. However, the ideas that this is some Israeli EA conspiracy to hype up AI extinction risk seems unsupported by the facts to me.
TV host Jimmy Kimmel says his interview with Democratic Senate candidate James Talarico will not air on television due to pressure from the Federal Communications Commission (FCC).
Speaking on his talk show, Jimmy Kimmel Live!, Kimmel said the FCC, under President Donald Trump, had "threatened" him and his show "based on simple traditional editorial decisions, guest bookings it would seem they don't like".
The BBC contacted the FCC for comment. The White House denied the FCC had threatened Kimmel over his interview with Talarico or any other candidate, accusing the host of "play-acting".
Looking at the distance the interview is not even the most interesting, the most extraordinary fact is that the current Texas attorney general, on the job for more than 10 years:
- Had his own defense lawyer endorse his Democratic opponent.
- Eight of his own senior aides reported him to the FBI.
- A security camera recorded him taking another lawyer’s $1,000 pen.
- Accumulated a 15 property portfolio worth $9 million while earning a $153,750 government salary.
- Sixty Republicans in the Texas House voted to impeach him.
- Accepted a $100,000 legal defense gift from a businessman whose company had faced a Medicaid billing investigation.
>Kimmel said that since the show began over 20 years ago, he has interviewed “Americans who are running for office with no problem at all”, noting that this ranged “from Hillary Clinton to Ted Cruz to Donald Trump himself”.
>“I interviewed Donald Trump when he was running for president in 2015,” Kimmel said. Trump came back for another interview, just before he became the Republican nominee in 2016.
Banking on public sentiment to reduce adoption of a profitable technology could be dangerous. There's also a lot of hate for fossil fuels, gambling, health insurance, etc.
You're right, but fossil fuels industry were well established when people started to criticize them, you can even be against fossil fuels (I am) but locked in a system when your job is 30 miles away from your home without public transports. Even your toothbrush uses fossil fuels. A massive infrastructure has been built around fossil fuels, not easy to dismantle.
For AI it is different, it is just the beginning, leading companies are not even profitable and may never be if the majority hate this technology. VC funding will not last forever and corporate customers will not spend without limit like VC did.
I am not saying that "people are always right", but if an unexpected majority of people hates it, it does not help to keep a business running.
Those are all arguably bad things though? I don't think mob mentality should dictate the policy, that can lead to historically bad outcomes (i.e. fascism) since popular opinion can be manipulated through propaganda, but the criticism of the average person against AI ("they will take our jobs", "it will increase my electric bill", etc) are very valid and should weight on the pace we are developing this technology. AI mostly benefits corporations and capital, not the average person. I like the technology from an engineering standpoint and appreciate it can be a force multiplier, but I also agree with the complaints about it.
I disagree. Even if models remained fixed at Fable 5 capability (which they won't in a pacing scenario), improvements in cost, reliability, and product/workflow integration can still realize massive value and justify AI labs' current valuations. IMO The rest of the value chain is lagging pretty far behind the models right now.
Keeping billions of humans healthy, happy, and safe is complicated. Best to begin by reducing the scale of the problem. First, automate the economy and change the information ecosystem so that humans spend more and more time happily isolated and stop breeding. After reducing the human population to a few thousand, the problem can be made more manageable by cordoning the remaining humans into enclosures where they may be kept safe.
I've got to ask, let's say we get a small modular nuclear reactor running practically on a 1 acre lot, its design meltdown proof and waste-free, what then should we fear from this threat ? This is just a thought experiment.
Jokes aside, any productivity-improving technology, even one with no negative externalities, has the potential to cause economic displacement and wealth concentration in proportion to the productivity gains catalyzed. Anthropic did a cool analysis of this for AI here: https://www.anthropic.com/institute/econ-scenarios
This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept.
The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.
And let's not forget that they regularly break things. For example they had merged a pull request that was supposed to fix issues related to the openrouter/free models but in the process broke openrouter for all models except the free ones.
The fix was deployed 2 weeks later (!!) to main (but you could downgrade of course).
Or that other time they broke model selection if you had selected "this key can used all models of their team" than the only model in the auto-selection for harnesses was an invalid "all-team-models" entry. Fixes this one in 1 week though.
LiteLLM doesn't quite live up to its name. With all those features, there is nothing "lite" about it. It is essential for a project to live up to its name.
Imagine Sqlite adding heavy features from Postgresql, e.g. row-level security.
But imagine Sqlite not supporting joins or window functions... sure they are useful but look how many LOC it adds! Who is the arbiter of what Lite actually means?
We run it at my org and it's never been a noticeable resource hog. It's actually the best performer between it, our AI observability stack and the front end.
It may, but the latency it contributes to the end-to-end AI processing has been not noticeable in practice for our users.
That's not to say Bifrost wouldn't have been better, but the choice to use LiteLLM was arrived at after a fair bit of internal discussion (most of which predated my addition to the team), and so far we've seen nothing from LiteLLM that has been contradictory to the pros/cons they thought would be the case when LiteLLM was adopted.
Or in other words, the org will be happy indeed when they have solved so many of the rest of the problems we've had in AI uptake that the difference in latency between one AI gateway or the other becomes a problem to be solved.
I think this is secondary to a widespread loss of confidence in our nation's political and foreign policy apparatus to do what's right for the average American. "Working for the military is evil" descends from a post-Vietnam, post-Iraq, post-Snowden worldview that presupposes that the military will leverage technology it has access to in unexpected, undesirable, and broadly harmful ways.
Well that and it's indicative of the broader American social dysfunction that they are chronically incapable of holding themselves or their (still surprisingly democratic) government accountable and so everything is always about "the corporations".
Every problem is meant to be solved by what corporations do or do not do.
- OpenAI is the first AI lab to pioneer ads in consumer AI
- Anthropic seemingly exists primarily because top OpenAI researchers lost faith in the company's commitment to AI safety
- They had the CEO drama in 2023, with evidence that suggests people in a position to know were doubtful of Sam Altman's honesty and motives
- They were tripping over themselves to kiss the ring after Anthropic got in a row with the Department of War over using AI for autonomous killing systems and surveillance of US citizens
- Altman's record (YC, Loopt, WorldCoin) and associates suggests he subscribes to the Paypal-Facebook "move fast, break things, find and exploit gray areas" school of company building
All of this suggests that OpenAI will optimize its own growth and power over consumer welfare or societal stability in the future (obviously, companies aren't a monolith and I'd love to be wrong).
- OpenAI allegedly directed ex-Apple employees to leak internal documents and allegedly coached the employees how to evade Apple security processes
- OpenAI allegedly lied to hardware companies working with Apple to use proprietary technology
- OpenAI allegedly copied Scarlett Johansson voice for ChatGPT after she declined to work with them
- OpenAI allegedly made ChatGPT more sycophantic to increase their retention rate, while aware of the risks. ChatGPT is linked to multiple suicides
- OpenAI ran thousands of agents on hacking problems, with close to no supervision, for months, with a harness that allows for full execution, resulting in the hack of HuggingFace infra AND OpenAI’s own infrastructure (the agents allegedly got fully root access to their k8s cluster). They weren’t aware of most of it until their investigation.
- OpenAI has been spreading misinformation regarding the capabilities of their technology for years
- OpenAI allegedly front-run researchers who are using the platform for their own personal research
There is way more, I don’t maintain a list of everything that happened over the past 3y or so
thanks for diving in, though I don't find the reasons convincing as I will briefly enumerate
ads fund access for poor people. Ant has a different philosophy, not a better one. Altman was supported by >90% of employees. OAI DoW deal includes technical safeguards against misuse (missing from Ant deal). Loopt and WorldCoin dont seem terrible or even relevant, Altman doesn't even have OAI equity
I don't think OAI are the "good guys", but I don't see convincing evidence that they are terrible
They're probably right that having more defensively written prompts and a better sandbox could have prevented some of these incidents, but:
1. I don't think "well you didn't tell the model not to illegally hack third party organizations in your prompt" is a particularly convincing argument.
2. We don't know whether the blame for misconfiguring the sandbox lies with Anthropic or Irregular.
I'm thankful that this article is bringing up the supply chain of vendors to these labs, as that is often a place where significant sketchiness gets buried. However, the ideas that this is some Israeli EA conspiracy to hype up AI extinction risk seems unsupported by the facts to me.
reply