That's incredible. Sure, not practical for most applications, but if you really want a local top tier model, you can run it on anything as long as you are patient.
As someone with a healthy amount of RAM, but just a 16GB GPU, I am wondering what kind of work I could queue up for overnight runs. I thought the best models were fully out of reach, but the 128GB CPU only test had a 1.8 tokens/second. While not speedy, you could probably do something with that given extensive coffee breaks. This speed simulator[0] demos what it looks like.
So while SSD streaming is interesting I'm not sure it's exactly the same thing as the per-layer embedding that is being utilized in tandem with streaming here. To utilize per-layer embedding, it would have had to be trained that way, which GLM 5.2 was not.
more interesting would be using some kind of FPGA to logic glue each RAM socket interface bitplane to a hard drive (so a collection of hard drives with ridiculous collective bandwidth). Perhaps a single RAM socket contains actual RAM and the linux kernel would have to be modified to only use the real RAM memory region for OS and inference software, with the inference software rewritten to stream LLM weights deterministically from the hijacked RAM slot physical memory regions. Obviously the FPGA can't truly achieve the CAS latencies over the HDD (unless the HDD firmware was rewritten so it can predict the next deterministic token sufficiently in advance to cache the result and stream it just in time to FPGA then "RAM" socket...) but even if the HDD firmware can't be reprogrammed for some reason, the FPGA knows what memory address will be deterministically fetched next, so it can make the requests to the parallel array of HDD's ahead of time.
My guess is because the ESP32's flash is only ~1/4 the bandwidth of the internal SRAM. If you do this on a more powerful system not only is the gap much wider but you also have much more compute you need to keep fed with bandwidth to be efficient.
It's also mapped into the address space so there's very little extra latency in grabbing the embedding as opposed to something like nvme that will have to setup a command list, submit it to the drive's microcontroller, wait for the op to be processed, etc.
OwnerRez | Software Tester/QA | Full Time | Remote
OwnerRez is a vacation rental software platform for hosts that automates the entire process from properties, bookings, and the overall guest experience across channels like Airbnb, Vrbo, and Booking.com, as well as direct booking websites.
We're looking for a Software Tester with 5+ years of experience working with software engineers to test new features, look for regressions, document bugs, and identify inconsistencies. Experience with automated testing is a plus, but we mainly do manual testing.
OwnerRez | Remote (UTC -10 to UTC +4) | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for senior full stack engineers to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database. We're also looking for ops skills and desire to join our on-call rotation.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB. AWS sysops/devops experience preferred.
GenAI/LLM API knowledge and agentic coding experience (Cursor, Claude Code, Copilot etc.) a huge plus!
OwnerRez | Remote (GMT -8 to GMT +3) | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for senior full stack engineers to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database. We’re also looking for ops skills and desire to join our on-call rotation.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB. AWS sysops/devops experience preferred.
OwnerRez | Remote (GMT -8 to GMT +3) | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for senior full stack engineers to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database. We’re also looking for ops skills and desire to join our on-call rotation.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB.
OwnerRez | Remote (GMT -8 to GMT +3) | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for a senior full stack engineer to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database. We’re also looking for ops skills and desire to join our on-call rotation.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB.
OwnerRez | Remote | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for a senior full stack engineer to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database. We’re also looking for ops skills and desire to join our on-call rotation.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB.
OwnerRez | Remote | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for a senior full stack engineer to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database. We’re also looking for ops skills and desire to join our on-call rotation.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB.
OwnerRez | Remote | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for a senior full stack engineer to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database. We’re also looking for ops skills and desire to join our on-call rotation.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB.
OwnerRez | Remote | Senior Full Stack Engineer | Full Time
OwnerRez is a vacation rental software platform for property managers and owners that integrates with channels like Vrbo, Airbnb, TripAdvisor, and booking.com as well as direct websites to manage vacation rental properties, bookings, and guests -- making the entire process automated.
We're looking for a senior full stack engineer to join our team to design and develop features end to end -- from web UI to backend business logic, services, and the database.
Our stack: .NET (MVC, jQuery on the frontend, WebAPI on the backend), MySQL, Redis, DynamoDB.
