[Blog] Innovative FPGA Technology Enabling Low Power, Modular, Small Footprint USB Solutions

Developing USB technology presents unique challenges, primarily due to the need for robust connectivity, speed, and power management within constrained device form factors. Compatibility across a wide range of devices, varying data transfer speeds, and demands for low latency and high-power efficiency place additional pressure on engineers to innovate within stringent technical limits. Engineers must integrate USB functionality into increasingly smaller modules, balancing functionality with design limitations.

This blog summarizes some of the typical solutions used in industry for high performance USB 3 devices and introduces a new architecture that saves power and area while expanding flexibility and ease of use.

Lattice recently announced a new FPGA family with a native USB 3.2 Gen 1 called the Lattice CrossLinkU™-NX. This blog describes the device in detail besides the product datasheet. Some salient features of the CrossLinkU-NX device are:

• Offers up to 5 Gbps data transfer speeds with integrated USB 2.0 and 3.2 support
• Significantly reduces board size and power consumption due to higher integration.
• Supports a variety of interfaces (MIPI, LVDS, CMOS) and advanced security features (AES256 and ECC256).
• Provides low power standby and versatile DSP resources for optimized AI performance.
• Ideally suited for industrial, automotive, and consumer applications.

tinyVision.ai has worked closely with Lattice Semiconductor over the past few years to develop a complete USB solution that integrates advanced FPGA technology with modular, high-performance USB functionality. This collaboration has enabled tinyVision to leverage Lattice’s CrossLinkU-NX FPGA platform, pushing the boundaries of efficiency, scalability, and flexibility in USB-enabled designs. Customers benefit from a complete solution including hardware, RTL, and firmware that they can modify per their requirement and not just an application note.

By combining tinyVision's expertise in firmware and modular hardware design with Lattice's cutting-edge FPGA capabilities, this collaboration has delivered an innovative solution that redefines USB connectivity for modern embedded systems and multimedia applications.

Industrial Cameras Today Require Efficient Bridging

Low or high-speed USB solutions are ubiquitous today in many devices with limited bandwidth requirements and found in many low-end microcontrollers. However, to move large amounts of data for, say an industrial camera, software defined radio or such applications, USB SuperSpeed is required.

FPGAs are typically used as a signal gathering and processing frontend in a large number of applications such as industrial cameras, handheld ultrasound, logic analyzers and software defined radios. To communicate with USB 3, they are typically tied to a USB bridge chip.

Figure 1. A typical USB3 peripheral utilizing an FPGA and a USB3 bridge chip.

The interface between the FPGA and the bridge chip is typically a parallel FIFO interface. Bridge chips are simple data pumps and incapable of touching the data flow itself. They are typically able to add headers/footers to conform to a protocol requirement. The task of signal processing, as required in an ISP (Image Signal Processor) for various camera related functions such as AEC, AGC, AWB, Gamma, lens correction and so on, are left to the FPGA to deal with.

CrossLinkU-NX FPGAs: A Game-Changer in USB Device Design

The CrossLinkU-NX device plays a significant role as it takes the 2 chips usually paired together: the FPGA and the USB 3 interface, and merges them into a single part.

Figure 2. Improved system architecture using the Lattice CrossLinkU-NX device

The advantage is a smaller footprint, lower power, lower BoM (Bill of Materials) and manufacturing and a single toolchain to master and manage than other competing USB 3 bridge solutions on the market. With a line rate of 5 Gbps and measured throughput of 3.4 Gbps at the UVC layer, there is plenty of bandwidth available for high resolution, high frame rate and multiple camera systems.

Architecturally, the data FIFO can be pulled into the FPGA. This flexible architecture enables potentially new applications which were simply not possible earlier (e.g., now able to allocate USB endpoints to various peripherals under the designer’s control as opposed to a limited number as dictated by the chip vendor.) This enables support for multiple camera streams quite easily, for example.

Board design is also far simpler due to a significantly lower parts count (hence, fewer supply chain issues) as well as direct sensor interface without the need to route the large number of FIFO wires across the board, and no inter-chip control signals are needed.

Figure 3. Conceptual block diagram of a USB device enabled by the CrossLinkU-NX FPGA

tinyVision.ai's Innovative Approach

tinyVision’s solution consists of the following components:

1. SoM (System on Module) for fast prototyping with a path to production.
2. Compact RTL built around the USB hard IP to maximize USB bandwidth and efficiency.
3. Zephyr RTOS drivers for this IP that abstract the complex USB protocol.
4. Toolflows and complete reference designs that make good jumping off points for developers.
5. Customer Support by providing design services from driver development through complete product development.

Figure 4. tinyVision.ai tinyCLUNX33 SoM MIPI to UVC converter

The modular hardware and firmware architecture supports interchangeable components, allowing teams to add USB functionality to their products with minimal work.

The SoM (System on Module) hardware supports USB 3.2 Gen1, ensuring high-speed data and low power, fitting even constrained form factors. A variety of adapter boards are available using the Syzygy standard to enable fast prototyping.

Building on the Zephyr RTOS, tinyVision have developed a USB-as-an-API approach, implementing a zero-configuration USB API with automatic and compliant USB descriptor generation. This makes USB integration seamless and enables developers to handle USB calls as they would handle other API requests. This significant simplification, combined with tinyVision’s commitment to open-source development, includes upstreaming its USB 3 driver and active involvement in developing Zephyr’s UVC (USB Video Class) driver, essential for video streaming.

The SoM is supported by a rich IP suite including the USB IP and SRAM controller that have been tested on the platform to simplify integration.

tinyVisionalso provides a suite of image and audio processing intellectual property (IP) utilizing StreamLogic technology. This IP suite supports low-latency stream-based processing ideal for real-time applications, such as video streaming and audio playback, enabling advanced multimedia capabilities within USB devices.

Figure 5. Sample of a complete Image processing pipeline using the Streamlogic no-code Graphical design tool. Audio and Neural Network IP is also available.

7 Benefits of the New USB Device Architecture

1. Low power consumption compared to a typical FPGA + dedicated USB3 FIFO chip
2. USB 3 bandwidth + High-performance signal processing: The USB solution is able to efficiently utilize the full bandwidth of the USB3 physical layer to provide sustained data rates of 3.4Gbps at the UVC layer. The FPGA also has significant on-chip compute resources that enable not only ISP but also advanced features like object detection/tracking.
3. Compact form factor: A tiny 3.1 mm x 7.4 mm CSP package is available enabling very small devices.
4. Programmable flexibility: The data flow architecture in a conventional FPGA + Bridge chip is quite constrained. By pulling the USB core to be co-located with the FPGA fabric, we enable far more interesting datapaths.
5. Strong Firmware and IP Support: tinyVision’s Zephyr port provides users with an industrial grade and modern RTOS with a huge library of open-source drivers as well as a strong and helpful community. StreamLogic technology as well as the MIPI to UVC and SRAM controller IP enable fast development.
6. Enhanced security features: Bitfile encryption as well as other industry leading security features secure the customer design.
7. Faster time-to-market: A System-On-Module approach enables very fast time to market since the most complex parts of the design have been already verified along with a functional RTL and Zephyr reference design.

Customers Leveraging Low Power, Modular, Small Footprint Solutions

Constructive Realities, is in the business of digitizing and organizing spatial data. They were in the process of developing a custom 3D vision system that included a co-located Time of Flight (ToF) and RGB camera and spatial tracking and reconstruction running on a small ARM SBC. The solution needed to be compact, battery-powered and low cost.

ToF processing is complex and requires significant memory bandwidth and compute on the host, forcing the application processor to handle all aspects of the vision pipeline. The default solution for this would be to use an FPGA to manage the RGB and ToF sensor as well as its processing and present the data as a stream to the end user over USB using a FIFO to USB 3 chip such as the Cypress or FTDI chipsets.

By eliminating the FTDI/Cypress bridge chip, our customer has been able to fit the design into a smaller form factor, save power and significantly reduce BoM complexity by eliminating the bridge chip supporting circuits (bridge chip, memory, power and other passives).

The flexibility offered by the tinyVision UVC solution stack enabled faster development by exposing the raw MIPI data from both sensors directly to the host where the original algorithms were developed. Subsequently, the computationally expensive ToF algorithms were ported by our customer to the SoM, enabling them to use an off the shelf part to reduce to the compute requirements on the connected system.

Zephyr formed a key part of the solution due to its modularity, small footprint and strong community support. This SoM will form part of their final product with minimal investment of developing an interposer card that carries the SoM, power, IMU and the 2 image sensors.

Enabling Reliable, Efficient, and Secure USB Connectivity

The collaboration between tinyVision and Lattice Semiconductor has produced a transformative USB device solution centered around the CrossLinkU-NX FPGA. This innovative approach combines USB 3.2 support, advanced FPGA capabilities, and comprehensive firmware support through Zephyr RTOS, delivering a compact, power-efficient, and highly flexible platform.

The solution significantly simplifies USB device development while reducing costs and time-to-market, making it ideal for applications ranging from industrial cameras to medical devices and beyond. These applications benefit from the solution's flexibility, scalability, and high-performance capabilities, enabling reliable, efficient, and secure USB connectivity in mission-critical systems.

Whether for real-time data processing, multimedia, or embedded systems, tinyVision’s technology supports the complex requirements of these diverse sectors.

These transformative USB device solutions from tinyVision will be showcased at the Lattice Developers Conference 2024, on Dec. 10-11. Register now and secure your spot at the Lattice Developers Conference, where you’ll gain unique opportunity to learn from seasoned FPGA experts and experience more than 75+ innovative technology demos from partners across multiple industries.

For more information about tinyVision’s USB solutions and services, email us at: sales@tinyvision.ai and visit us at www.tinyvision.ai. To learn more about Lattice CrossLinkU-NX FPGAs and how Lattice can help you accelerate your USB-based design development, reach out to speak with the team at Lattice.

Let us help you accelerate your USB device development with our proven expertise and innovative technology stack.