Came here to say this. They have oh-my-pi in the benchmark but not pi, but those are very different animals. pi is lightweight out of the box so has very little start-time overhead. (And will not spin up agents like crazy.) pi might do worse if those things are actually important for solving the problem, but it certainly has a shot at being most efficient.
hi Albert from the Strands team. We can look into a Pi run. Our researcher is working on a deep dive for our benchmarks, so we got resources to test against other harnesses
You could in theory require consistency of the spec with the product, but this is only valuable as a “source” if you ensure some kind of reproducibility. E.g., 9/10 times this spec + opus produces code that is equivalent, measured appropriately.
For me the first point, openness to 3rd party, is the decider. I don’t want to build tooling around a completely closed model. I liked being able to use pi, and now I exclusively use my own harness which I modify the way I want. Not possible with Anthropic subscription.
I often have the urge to design my own harness too (once I have more time). But even with the current mainstream harnesses out there, there's just to many hurdles if you wanted to mainly stick with anthropic models and need the subsidized pricing (from a sub).
Acting with urgency is a bit at odds with discovering flaws in your plan. If you're sprinting you're less likely to notice smells and things that are inelegant, more likely to paper them over. That said, there is a time for urgency. Just not every single task.
I know this is intended as a joke but there is something to it. The public market has a way of elevating profit over everything else, even if you start with the best intentions. Remember “don’t be evil”?
If this is true, though, then given the way their chat operates, this might be more dangerous than it seems.
One of the things I like about ChatGPT is its memory, the way it kind of seamlessly, but not excessively, ties back to earlier discussions. It's huge for usability (for me).
But this also means that you should expect that if "improve the model for everyone" becomes unclicked (leaving aside for a moment the fact that that is ridiculous) then they have a reasonable argument that your decision implies to all conversations. Because recall is part of their thing. So it's not just your chats going forward that are at risk. As soon as you see that unclicked, it's reasonable to expect that your history is irretrievably theirs now. You don't even have to think they are especially nefarious for this to be true.
Argument for only wins because of the weight of the lawyers they can bring to bear. Clearly ephemeral recording for the purposes of processing violates the spirit of the statute at least. And there is this, from the MGL:
The term ''interception'' means to secretly hear, secretly record,
or aid another to secretly hear or secretly record the contents of
any wire or oral communication through the use of any intercepting
device by any person other than a person given prior authority by
all parties to such communication.
Oh wait, they are not helping another person to "hear", only to see the words spoken or a summary thereof. Oops, MGL has you there too:
The term ''contents'', when used with respect to any wire or
oral communication, means any information concerning the identity
of the parties to such communication or the existence, contents,
substance, purport, or meaning of that communication.
So if you wear a watch in a conversation without consent, and then after the fact the watch tells you who was in the conversation or what it was about, that seems like a facial violation. Maybe it falls under a common carrier exemption, but then that puts legal liability on the wearer not Apple.
This may become a case of we don't enforce the laws against large companies.
Even beyond this, if you create circumstances where mass commission of consequential felonies is an inevitable result, I suspect that is also illegal. 100% this just gets ignored by the Mass attorney general though.
I have a hard time believing you are asking this question in good faith.
However I will respond with what I might expect to see play out:
If Sony makes a Walkman and you record with that Walkman it stays on that Walkman and you have a recording. After Sony sells you the Walkman Sony no longer interacts with you to record something, all of your recording is based on your deciding to record, and Sony has absolutely nothing to do with any recording you make.
If you use Siri recap it moves between devices and Apple analyses it to give you a recap of what was said, obviously it is something that would need to be argued in court but depending on the pertinent state laws it seems reasonable to assume that it would be argued that Apple had the recording in its possession at one point, and that they did something on an individual basis with your recording. Sony does nothing individually with the recordings, Apple does, Apple Analyzes the recording and gives you a recap.
Why would the company providing analysis of illegal recordings be liable if that's the product they're selling? Well, perhaps, as in the case of the MA statute quoted, there is something in the law saying if you do analysis of an illegal recording you are liable. That would be a pretty big "Why" in my experience.
All of this is of course open to legal interpretation because in the U.S nobody knows for sure if it is legal or not until the rulings establish the precedent. However you can often guess pretty accurately.
On edit: I had forgotten the exact content of the MA statute, going back and looking at it, it probably isn't a liability for Apple, but it isn't 100% sure it isn't (I'd say a pretty good chance though) plus as this is on a state basis, there could easily be another state that words things in such a way that it might more likely be a problem for Apple.
since they didn't specify a specific state I suppose it probably varies, but yeah, there is obviously one case in which it looks like a pretty clear cut violation.
reply