Newsroom > Blog
December 20, 2022
Efinix recently released the Efinix TinyML Platform delivering, for the first time, an intuitive and fast time to market way to create, optimize and deploy artificial intelligence (AI) models on efficient FPGA devices at the far edge of the network. Leveraging the configurability of the Sapphire RISC-V core and the inherent efficiency of the Efinix fabric, the TinyML platform unlocks the potential of highly accelerated AI running on cost-effective, low-power platforms at the far edge. Designers now have a framework with which to innovate and to deploy AI rapidly and intuitively.
Today’s modern world is generating data at an ever-increasing pace. So much data in fact that we cannot possibly analyze and interpret it all and the vast majority goes unused, untapped, and discarded after its short shelf life expires. Increasingly artificial intelligence (AI) is being used to comb through these seas of data looking for insights that can be flagged as important and worthy of more attention. Historically, these AI algorithms have been run in data centers where almost limitless computing power can be applied to the problem. Unfortunately, in many cases, the cost and latency involved in a round trip to a data center make this approach unviable.
What is needed is the ability to perform AI at the far edge of the network where the data is created and where it still has local context. If this were possible, more data could be cost-effectively analyzed with almost zero latency. Processing at the far edge is typically provided by microcontrollers that have low operating power, a small physical size but also correspondingly low compute capability. In most cases, notoriously compute intensive AI algorithms cannot run on edge microcontrollers with sufficient accuracy and latency to deliver the required quality of service.
Recently, the open-source community has spent a great deal of effort in optimizing AI algorithms, quantizing their parameters, and adapting them to run in lower precision mathematical functions to make them more microcontroller friendly. TensorFlow Lite, a microcontroller centric version of the popular TensorFlow AI development environment, can take standard AI algorithms and quantize them to run more efficiently using microcontroller resources. The results however regularly fall short of the desired performance.
With this dynamic in mind, Efinix set itself a challenge of delivering a development environment that would take the standard TensorFlow Lite output and would intuitively and rapidly accelerate the output. Using the highly efficient fabrics of its Trion and Titanium FPGA products it would deliver fast time to market with an AI solution delivering the required performance. The result was the Efinix TinyML Platform.
The Efinix TinyML Platform takes the TensorFlow Lite for Microcontrollers (TFLite Micro) C++ Library and ports it to run on the Efinix Sapphire RISC-V SoC. Once an AI model is quantized with TensorFlow Lite, it is analyzed by the Efinix model profiler to discover which of the TFLite Micro C++ Library primitives it contains. A key differentiator of the Sapphire RISC-V is its ability to implement custom instructions within the FPGA fabric. By creating custom instructions that are optimized for the application, the RISC-V processor can be accelerated by many orders of magnitude, running the application at near hardware speed. The Efinix model profiler has access to a large library of custom instructions that implement the TFLite Micro C++ primitives and these are made available to the designer through the graphical user interface of the profiler.
From within the profiler, the designer is free to select accelerators from the library and include them in the hardware design files of the FPGA project. The accelerators can be configured to trade off size vs performance to tailor the resulting design to the required parameters. Hardware design files of the selected accelerators are automatically generated to include in the Efinix Edge Vision SoC reference design.
With custom instructions instantiated in the FPGA design, all that remains is to call the hardware custom instructions instead of the equivalent operations in the TFLite Micro library. The result is a massively accelerated AI algorithm that has large portions of the model running in custom hardware under software control. Running on the Sapphire RISC-V SoC, the model has low operating power while the hardware acceleration of the FPGA fabric delivers the required hardware performance.
The Efinix TinyML platform has been contributed to the open-source community and is available on the Efinix GitHub . It is delivered with the model profiler and configurable accelerators as well as example accelerators to provide templates for users to develop their own custom instruction accelerators for both AI and application performance enhancements. By leveraging the example designs and documentation, designers can become familiar with the techniques required to replace TFLite Micro C++ Library primitives with custom instructions either automatically created by the profiler or custom designed for the requirements for the selected application. In this way, hardware acceleration is made available to the AI core in a turnkey automated environment and the designer is free to use the same techniques to massively accelerate application-level code running on the RISC-V core.
Full documentation is available for download along with Jupyter notebooks to guide the user through the entire creation, training, quantization, and deployment phases of their end AI system. The platform is based around the popular Efinix Edge Vision SoC Framework to speed time to market with edge applications and to give designers a template for application accelerators within the standardized application “sockets” made available within that platform. In this way, designers can rapidly deploy edge systems that contain pre and post processing of sensor data and highly accelerated AI analysis. Application code running on the multi-core capable Sapphire RISC-V processor ensures that the results of the AI analysis can be rapidly characterized, and appropriate action can be taken at a system level.
The Efinix TinyML platform leverages the configurability of the Sapphire RISC-V core, the efficiency of the Efinix Quantum® fabric and the expanding libraries of reference designs and examples to deliver a turnkey method of creating and deploying AI solutions at the edge in a cost-effective and low power framework. Visit the Efinix TinyML GitHub and follow along with the tutorials to get the next level of information about how to work with this platform and deploy AI at the far edge.