• 0 Posts
  • 144 Comments
Joined 3 years ago
cake
Cake day: July 18th, 2023

help-circle









  • My response was only in regard to the speed in which something like IoT devices can operate.

    You’re right that goals/directions of LLMs can be persisted anywhere, but a swarm or fleet of AI running on IoT devices will be severly hampered by compute (based on current models). Which was the main point I was making, in that there is only so much you can do to distribute their processing. Meaning you can parallelize the number of models, but can’t speed them up beyond a certain limit. This is based on the fact that some form of a layer and it’s weights must exist within the memory of a single device.

    The downside to this limitation is that it is temporary, eventually IoT devices will probably be powerful enough, or the worms could just attack phones/laptops and easily get the compute they need. I think the real risk of IoT devices is low security mixed with low “supervision” (no one would easily notice their IoT device is doing extra processing).



  • Glad to hear I’m not the only person who has had some difficulties. And yeah, you are right that it should be fine for hosting services, which is why I mentioned my specific use cases.

    However, I think calling out lack of support is always a good thing as you never know what support you may need down the road. I definitely have run into some issues due to their supported OS I’m running and its surprisingly out of date for a board that only released a couple years ago.




  • OrangePi support is terrible. Maybe it’s better with the 5, but the 4 I’ve struggled with getting a library to work with the GPIO pins, documentation on the NPU is limited, and I can’t get it to recognize a standard serial camera.

    It’s possible I’m just trying to do weird things, but it’s a much worse experience than I’ve ever had with a raspberry pi and I regret not spending more to just get the pi. I’ve lost probably 10+ hours just trying to deal with some things I’d consider basic, and had to create weird workarounds that hurt performance in other areas.