

2·
2 years agoThis is basically the same thing as what the big platforms do. You’re just offloading the decisions of what to see to a neural network and hope it’s deciding correctly. I’m not sure what a solution would be but I’m not sure I would put my eggs in the llm/ai basket. Not without a lot more details from the models on why they made a decision.
It’s likely wrong reporting, the m5 ultra gets 614 GB/s today. The m3 pro gets over 800. The rumored m5 pro is 1.2TB/s. Likely this will be 2.4 TB/s, but half the reason for high bandwidth in nvidia chips is somewhat offset in the unified scheme. Mlx needs a lot less copy data around when the gpu can just read it directly.