ML Inference For Embedded Applications Reference Design

The design integrates deep learning inference into embedded applications using a System-on-Chip, enhancing performance across various industrial and technological fields.

Machine learning inference for embedded applications is becoming increasingly important as it allows devices to process data locally, enabling real-time decision-making without constant cloud connectivity. This capability is vital in environments where quick responses are crucial, such as autonomous vehicles or medical monitoring systems. Performing inference on-device reduces bandwidth needs, enhances data privacy, cuts costs, and ensures functionality in remote or disconnected environments. Additionally, this technology supports the development of tailored, adaptive applications, paving the way for smarter, more autonomous devices across various sectors. The reference design from Texas Instruments (TI) illustrates the use of TI Deep Learning (TIDL)/Machine Learning on a Sitara AM57x System-on-Chip (SoC) to implement deep learning inference in an embedded application. 

The design demonstrates running deep learning inference using C66x DSP cores (present in all AM57x SoCs) and Embedded Vision Engine (EVE) subsystems, which function as dedicated deep learning accelerators on the AM5749 SoC. It is suited for any application aiming to incorporate deep learning or machine learning inference into an embedded setting. Those looking to quickly start with a deep learning network or assess their network’s performance on an AM57x device will find a comprehensive guide on using TIDL in the Processor SDK.

– Advertisement –

TIDL facilitates the execution of real-time deep learning inference at low power using both the C66x cores and the EVE subsystems. It comprises a suite of open-source Linux software packages and tools designed to offload deep learning inference compute workloads from Arm cores to the EVE subsystems and C66x cores. This document describes how to develop and deploy CNNs for image classification, object detection, and pixel-level semantic segmentation on the AM5749 SoC.

The design’s applications span a diverse range of fields, including automated sorting equipment, optical inspection systems, vision computers, and code readers. It extends further to industrial and logistics robots, currency counters, ATMs, and patient monitoring systems. Additionally, it is utilised in building automation, industrial transport, and sectors such as space, avionics, and defence, showcasing its broad adaptability and utility.

– Advertisement –

The embedded system features include deep learning inference capabilities on the AM57x SoC, with a scalable performance using the TI deep learning library (TIDL) on the AM57x. This can utilise C66x cores alone, EVE subsystems alone, or a combination of both. The system is optimised for reference CNN models that support object classification, detection, and pixel-level semantic segmentation.

A comprehensive walk-through of the TIDL development flow is provided, covering training, import, and deployment of models. Additionally, benchmarks of various popular deep-learning networks on the AM5749 are included to demonstrate system capabilities.

TI has tested this reference design on the AM5749 IDK EVM and incorporates the TIDL library on the C66x core and EVE subsystem. It also includes reference CNN models and a Getting Started guide to assist new users in deploying and utilising the technology effectively. To read more about this reference design, click here.