Overview
The f-inference project introduces a paradigm shift in Foundation Model (FM) deployment by transitioning from centralized, cloud-bound architectures to a unified Resource-driven Computing Continuum (RCC). Spanning Cloud, Edge, and IoT tiers, the RCC decentralizes intelligence by placing models closer to data sources and end users. Central to this approach is an active Edge tier capable of orchestrating lightweight, task-specific FM experts. These models collaborate horizontally across peer nodes and vertically across IoT devices, reducing cloud reliance and boosting operational resilience. By integrating compressed model variants down to the IoT level, the RCC enables localized micro-inference that minimizes latency, energy consumption, and bandwidth costs. Advanced orchestration techniques—including Mixture of Experts (MoE), dynamic query routing, early-exit strategies, and localized caching—optimize real-time resource allocation while preserving data sovereignty within local operational boundaries. Ultimately, the RCC democratizes access to state-of-the-art AI, offering a scalable, sustainable, and robust framework for distributed, privacy-preserving digital transformation.
