Custom embedded Linux distribution based on Yocto for an AI accelerator evaluation kit

This thesis covers the work done while developing a custom embedded Linux dis- tribution in partnership with Dutch company Amarula Solutions, tailored for an Artificial Intelligence (AI) accelerator manufacturer’s needs. The targets for this Linux-based Operating System (OS) were the evaluation kit for a new AI accelera- tor card to be introduced on the market, which is based on a commercially available board, as well as an entirely custom board developed by the manufacturer. The objective was to build a Linux-based OS that would allow deployment of AI ap- plications on the edge, making efficient use of the Rockchip RK3588 System on Chip (SoC) and adding support for the new AI accelerator. At the start of the work, AI, Neural Processing Units (NPUs) and the rise of AI on embedded systems was explored, with its hardware demands and typical appli- cations, as well as open-source software and its state of the art. The implementation was done through the Yocto Project, using a modern workflow entirely based upon open-source tools. The project incorporates Git for version control, Gerrit for col- laborative code reviewing and Gitea as a remote repository host, integrated with Jenkins for the Continuous Integration (CI) process and Sphinx for documenta- tion. The initial target was the Linux kernel version 5.10, which was later upgraded to version 6.1. The distribution was configured to allow real-time video processing through GStreamer, a key application for AI edge computing the performance of which was thoroughly benchmarked and tested, finding the results to be in line with expectations. Support for Docker was added and a vendor-provided container was integrated in the distribution. The project finally consisted in a single Yocto capable of building the embedded Linux distribution for the two targets, resulting in the accelerator card being fully functional on both, providing competitive performance and polished user experience. The results show the feasibility of creating a high quality final product which is employed in real-world market scenarios, all while using state of the art versions of open-source tools and software.