From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap

arXiv:2410.20791v3 Announce Type: replace-cross Abstract: The rapid expansion of foundation models (FMs), such as large language models (LLMs), has given rise to FMware, software systems that integrate FM(s) as core components. While building demonstration-level FMware is relatively straightforward, transitioning to production-ready systems presents numerous challenges, including reliability, high implementation costs, scalability, and compliance with privacy regulations. Our paper conducts a se