Process Geometries, the Edge, and AI:
the Role of FPGAs

Some of the most advanced embedded applications rely heavily on a link back to a data center or the cloud in order to execute their highest compute workloads. There is one high-compute workload that is common to some of the most advanced embedded application areas, such as robotics and sensor fusion: AI inference and training. Edge compute nodes are a critical enabler of highly responsive, highly accurate systems that implement AI as part of their primary functionality. Devices embedded in the field and edge servers both need a powerful AI-capable chipset that can enable these applications and many others.

Where the Edge and AI Converge

As edge computing can support many different applications, it is not only used to support AI on embedded devices and IoT platforms. Some applications that implement AI as part of their primary functionality need to periodically access additional compute resources that are not available on the device. In the past, this would be done accessed through a cloud platform, with an application on the device acting as the intermediary. Edge computing changes this dynamic by providing application-specific compute resources closer to the end device.

Network Topology in Edge Computing With AI

The role of edge servers and end users in a network changes when AI is involved. Edge computing supports AI functionality in an end device in a few important ways:

• Data from end devices can be aggregated and used for training in an edge server, then the model on the end device can be updated.
• The edge server can be used entirely for training and inference, with results delivered to the end device with a low-latency connection.
• In applications with multiple AI models and functions, only the higher-compute inference and training workloads can run on an edge server, while simpler AI models can run on the end device.

With edge computing being implemented with the goal of low-latency service provision, edge compute systems need a chipset that can support service provision, application execution, and AI inference/training simultaneously. The chipset also needs to be reconfigurable to support AI training and inference.

How FPGAs at the Edge Enable Embedded AI

FPGAs are an ideal option for implementing AI functionality both on end devices and in edge servers. They can occupy the typical role of an AI accelerator on an expansion card without upgrading existing systems. The alternative for newer systems is to implement everything in an FPGA as the primary chipset on end devices and in an edge server. As reconfigurable components, the AI models they instantiate in silicon can be periodically updated so that the system is always optimized.

The system architectures of edge server nodes and user devices that can implement AI applications with an FPGA are shown in the following table. These architectures can support a variety of embedded applications that deliver services to end users. For comparison, the traditional chipset that might be used with an edge compute node is highlighted in the first row.

	End-User Device	Edge Server
Traditional chipset	MCU/MPU with ASIC accelerator	MPU with GPU accelerator
FPGA-based chipset	Small FPGA, or MCU with small FPGA accelerator	MPU with large FPGA accelerator, or entire system on large FPGA
Peripherals	Supporting interfaces instantiated on-device	Memories, storage, etc.
Other components	Sensors, power, networking components	Networking components (uplink and downlink)
Application	Small embedded application for data capture Some pre-processing and low-compute inference can be performed on the device	Main server application for large workloads Edge server application controls data flow to the end device and the cloud

Implementation with an FPGA on an edge node can help reduce the server’s size and power consumption through elimination of the GPU. We can also see that the architecture sets up a dynamic where the end user device and the edge server node can share computing resources and workloads.

The end-user device architecture is essentially a data collection and pre-processing device, although it can support low-compute inference tasks with an optimized AI model instantiated in the FPGA. With acceleration and quantization in modern AI/ML packages like TensorFlow Lite, they have become competitive compute platforms for AI inference again. However, they still cannot provide the same level of dedicated compute or power efficiency found in a small FPGA.

Next, at the server level, a larger FPGA can handle much higher compute AI workloads that take too long or too much power on the end-user device. FPGAs used in an edge server can be implemented as the primary processor for smaller server nodes that need mid-range compute. For the highest compute workloads, implementation of an FPGA as an accelerator alongside a traditional chipset is the best path forward for providing AI-enabled services.

Summary

As we have seen from the above points, FPGAs have a role to play as the primary chipset in an edge node, or as part of the chipset implemented in an end-user device. FPGAs offer highly specific compute resources that are also reconfigurable, something which perfectly accommodates AI inference and training. By splitting AI inference and training tasks across an end-user device and an edge computing node, it’s possible to continuously optimize AI models and compute within a broader application.

Developers that want to accelerate production of AI-capable edge computing nodes and end-user devices should use the Efinix FPGA SoCs based on RISC-V cores. The RSIC-V SoC core IP from Efinix can be tailored to some of the most challenging embedded applications, including vision, AI, sensor fusion, and much more.