Serving 70B-scale LLMs efficiently on low-resource edge devices [pdf]

📰 Hacker News · simonpure

Serving 70B-scale LLMs efficiently on low-resource edge devices [pdf]. 58 comments, 248 points on Hacker News.

Published 3 Oct 2024