Contributions by Fredrik Ståhl and Kristoffer Wallin
In this blog post, we explore Funnel’s process toward adopting Rust for some of our high-performance, scalable services. This transition has significantly enhanced system reliability and is increasingly shaping the choice of technology within Funnel’s backend infrastructure.
The shift to Rust began during the pandemic when Funnel’s development primarily relied on Python. In search of ways to improve type safety and system efficiency, Rust was introduced. With Funnel processing around 18-20 TB of data daily, optimizing performance and reducing runtime issues became a priority. Rust’s static typing, compile-time checks, and ownership model provided a robust way to catch issues early, especially in concurrent environments.
Rust was initially tested on a few isolated services to assess its potential. The early outcomes were promising, especially for handling high-traffic workloads. The added control over system resources and performance gains led to broader adoption across more critical services. Rust’s compiler checks encouraged a more thoughtful and precise approach to coding, leading to improved productivity over time.
Efficiently processing large volumes of data is critical for Funnel, and Rust’s low-level capabilities have proven to be a strong fit for this purpose. The absence of a garbage collector allows for greater control over memory management, ensuring more predictable performance under heavy loads. This has been essential for maintaining the stability of Funnel’s data processing services.
Transitioning from Python to Rust presented some initial challenges. Adapting to concepts like Rust’s memory ownership model and the borrow checker required some adjustments. However, these constraints enforced a structure that helped eliminate common concurrency bugs, which became increasingly important as Funnel scaled its backend systems.
There were also challenges related to the maturity of Rust’s ecosystem. Although the language is robust, the library support is not yet as comprehensive as that of established languages like Python or Java. Despite this, Rust’s package manager, Cargo, has been reliable in managing dependencies, and the supportive community has played a significant role in overcoming some of these limitations.
Since adopting Rust, Funnel’s development strategy has evolved with clearer guidelines on when to use different programming languages. Rust is not mandated across the board but is encouraged for projects where performance and type safety are crucial. This approach aligns with Funnel’s commitment to continuous improvement and efficient service delivery.
For those interested in exploring Rust, starting with small, isolated services is a practical approach. In a microservices architecture, adding a Rust-based service allows for a manageable way to test the language’s capabilities without requiring a full-scale system overhaul. While Rust may not be the right fit for every project, it offers clear advantages for performance-critical services where type safety and control over system resources are essential.
Discover more on the Funnel Dev Blog!