Why are there results for opus5.5 on arc-agi 1 and 2 but not 3?

https://arcprize.org/results/anthropic-claude-opus-5-5

wait what debacle?

Probably referencing how when Astra came out it was only 1 point ahead of 5.6 sol.

( why is the x-axis on the first chart in descending order ? )

There are no facts only interpretations. - Frederick Nietzsche

But honestly, it is because numbers are like people; torture them enough and they'll tell you anything.

since it was sso for their codex/chatgpt account, presumably they used an existing chatgpt github connector available to such accounts.

Somehow the auth'd token for the forum was available within that VM, so having RCE they could've replayed it? idk

  > but the commit was not documented as a security fix and received no CVE.

There must be an entire class of open source commits that unknowingly fixed security bugs without being tagged as security fixes that one could look for missed backports. Scary.

"quick" tunnels you can make by installing cloudflare software and running a command, without a cloudflare account, using a fixed "trycloudflare.com" domain and not your own domain.

existing (slow) tunnels you can create once you've set up your own domain for cloudflare to manage its DNS, installing cloudflare software, logging in to your account, and running a similar command.

edit: my bad, quick ones aren't new at all

This has been the behaviour when you launch but don’t connect a specific tunnel for at least a couple of years

Quick tunnels have existed since at least 2022 (when I was using them)

The things in the 'what should be self-driving' section are the exact things i give to juniors so they can build their mental model of the codebase.

Now, I agree it'd be fantastic to have that automated, but how do we then share that context with the humans that will drive the non-self-driving actions?

So there's a strategy shift here. They launched financial services specific tools and now law?

Is the play here a set of specialized harnesses using their best general model?

  > the Tanstack compromise is very likely to have been the leak vector
  ....appears to have been backdoored to extract an API key with authorization to read the private codebase.
  ...
  >  immediately rotated all required tokens & credentials to prevent further incidents.

Rotating the API key doesn't quite put them in a position to "prevent further incidents" does it? The next PyPI/npm supply chain issue will just get the new key?

I suppose whatever they use that key for should be reviewed and re-scoped if possible?

Does github let you restrict where you can originate requests using a given API key? or are we just not there yet?

In my experience, Luna is not able to find non-trivial impacts from the changes at hand.

I've seen this a few times on relatively simple changes on complex codebases.