TL;DR: Roofline and Arm built end-to-end data-tiling for Arm’s SVE in MLIR and IREE, making AI code generation adapt to the vector width found at runtime and delivering an average 30x speedup over the same stack without data-tiling. Because that infrastructure was built independently of any specific ISA, porting it to RISC-V’s RVV took weeks rather than months, with support for hundreds of AI models from day one and better performance than ExecuTorch on 16 of 22 models on the SpaceMiT X60.
Scalable vector ISAs such as Arm’s SVE and RISC-V’s RVV are replacing fixed-width extensions like Arm NEON. They let silicon vendors implement hardware with vector widths that suit their product, while a single binary still runs across all of them, including hardware generations still on the roadmap. This newly won flexibility now requires software support in the compiler: Vector-length-agnostic code generation for AI models needs to adapt tile shapes and memory layouts effectively to the available vector width instead of assuming fixed-width vector targets. To get a scalable software solution that brings the latest AI models to these different chips in no time, the compiler needs to be built with a reusable and adaptive infrastructure.
This webinar shows how Roofline’s infrastructure-based approach using MLIR and IREE allowed us to migrate the stack from one vector extension to another in mere weeks, while supporting hundreds of AI models from day one. Together with Arm, we built end-to-end data-tiling for Arm’s SVE in MLIR and IREE, where tile sizes are derived from the runtime vector length rather than baked in at compile time. The hard part was not generating SVE instructions, but making the compiler stack itself scalable. Data-tiling turned out to matter more than the vectors themselves, with an average speedup of 30x over non-data-tiled SVE code and gains over both IREE’s and ExecuTorch’s backends for the fixed-width vector extension NEON. We could then reuse most of our infrastructure to port our support to RISC-V’s scalable vector extension RVV. Because the infrastructure and our optimizations are independent of the specific ISA, enabling a second scalable target required only weeks rather than the months we needed to build the infrastructure around Arm SVE. Roofline’s RVV support outperforms ExecuTorch on 16 of 22 models on the RISC-V hardware SpaceMiT X60.
You will leave this webinar knowing how the MLIR and IREE infrastructure lets Roofline build a reusable software stack that can be applied across different implementations of scalable hardware, across different ISAs and also largely across different hardware families.