Lattice Blog


[Blog] Revolutionizing Edge AI: The Role of FPGAs in Smart Camera Optimization

[Blog] Revolutionizing Edge AI: The Role of FPGAs in Smart Camera Optimization
Posted 06/18/2024 by Mark Hoopes, Director of Automotive & Industrial Segment Marketing

Posted in

Smart cameras are gaining widespread adoption in our technology-driven world. These standalone vision systems are equipped with sensors, computing power, and AI-driven decisioning capabilities, and do more than just capture images – they extract information and execute actions. Consider this: In 2022, the smart camera market was valued at $3.48 billion, but projections indicate it will soar to $8 billion by 2031.

With an unprecedented deployment of AI-powered smart cameras, the demand for solutions that offer low power consumption, minimal latency, enhanced security, and adaptability is on the rise. These features ensure that smart cameras remain efficient, sustainable, and versatile across various applications, all while optimizing system performance.

Field programmable gate arrays (FPGAs) are at the heart of this technological evolution. At the 2024 Embedded Vision Summit, Lattice Semiconductor and Etron Technology America gave a presentation on “Optimizing Endpoint Smart Cameras with Low Power FPGA and DRAM offerings.” This blog breaks down the presentation and explores how developers can utilize FPGAs to meet Edge AI demands and optimize smart cameras. Read on to learn about key components of FPGAs that render them ideal for smart cameras.

The Power of FPGAs at the Edge

FPGAs are powerful and flexible solutions that enable developers to develop scalable Edge AI applications while reducing latency, increasing power efficiency and bandwidth, and improving data privacy. There are several FPGA capabilities that stand out for smart cameras in particular: flexibility, ultra-low power, scalable performance, programmability, and security.

Smart cameras are changing rapidly amid AI’s continual advancements. Developers must be able to make changes to a device when new improvements, algorithms, safety features, updates, or requirements arise. FPGAs serve as an adaptable hardware solution that can help developers keep up with this evolving environment. With inherent flexibility and programmability, they are an ideal choice for Edge computing because they can be easily modified – even after being deployed – if functionality changes are needed.

Low power consumption is also vital for compact smart cameras because it enables superior reliability and quality, ensures continuous monitoring, and expands the scope of applications. FPGAs are capable of parallel processing, which allows AI applications to deliver top-tier performance-per-watt thanks to their low clock frequency operation.

Lastly, FPGAs have enough embedded memory and logic / DSP resources to implement many functions completely on chip, while also being flexible to support efficient external memory for larger processing needs. This flexibility is crucial for Edge computing, where adapting to a wide variety of applications is essential. They also offer secure device configurations to help drive cyber resilience across Edge AI environments, which is critical for reliable and safe operations.

Common Smart Camera Architectures

There are three primary Edge AI smart camera architectures. The spectrum ranges from complex system on a chip (SoC) applications that have large amounts of external memory to smaller units that possess shared memory. Each architectural option has its own unique set of pros and cons. Their use cases are dependent on the performance requirements of the application at hand.

Media SoCs are typically the largest, most complex architecture and require eight watts or more. They meet all developer needs, however they can often be overkill when looking to optimize the solution for low power and cost. The next common choice is a combination of small MCUs and general-purpose FPGAs – which often require about 5-10 watts. The smallest, highly optimized architectural option for low power Edge solutions utilizes a low power FPGA with integrated Soft RISC-V® CPU and external RPC DRAM, with total power near one watt.

[Blog] Revolutionizing Edge AI: The Role of FPGAs in Smart Camera Optimization Image 1

For architectures that utilize FPGAs, developers can also tap into external memory to achieve larger, more complex processing tasks.

Avoiding Overprovisioning Memory

Avoiding overprovisioning compute horsepower and memory is critical as developers implement innovative solutions into their smart camera structures. When it comes to smart camera memory, the biggest consideration to keep in mind is that more is not always better. There is no extra credit for unused excess capability.

Extra bits and excess memory bandwidth can have a significant effect on performance, power, and costs. Most applications will benefit from having less memory while operating at lower power levels. Developers should strive to meet the minimum amount of computing power that ensures the application can function effectively. This is a more practical approach that will boost efficiency and reduce operational costs.

Developers should also aim to identify proactive power savings opportunities that do not compromise component availability and system reliability. For example, using series bus termination with CHIP SCALE packages can result in significant power savings compared to parallel termination. Series termination offers faster bus settling times with power dissipation of 840 mW, while parallel termination dissipates 3.6 W, resulting in 2.76 W in power savings, while still preserving signal integrity and reliability.

Paving the Way for a Connected Future

In an era defined by technology, the widespread adoption of smart cameras is reshaping industries and revolutionizing how we perceive and interact with our surroundings. As the smart camera market surges, demands for low power, minimal latency, enhanced security, and adaptability are at an all-time high. FPGAs are an ideal solution, offering adaptability and parallel processing capabilities suitable for rapidly evolving use cases. Their ability to optimize and tune Edge AI models and Image Signal Processing also helps to reduce complexity while still meeting application needs and reducing power and cost.

When developers harness the power of FPGAs to optimize smart camera performance, they can pave the way for a more connected and intelligent future. To learn more about FPGAs and their role in Edge AI and smart cameras, reach out to speak with the Lattice team today.