Smart Consumer Robots and Toys

Powerful, Low Power FPGAs Enable Next Generation Robots and Toys

Next generation of consumer robots and toys are embedding a range of sensors to drive rich user experiences and human-like interfacing. Designers are taking advantage of flexibility in sensors choices, powerful low power edge processors, and ability to aggregate data from multiple sensors to drive these devices.

Lattice’s smart solutions running on optimized low power FPGAs provide:

  • Low power on-device AI for object detection, identification, counting, and human-machine interfacing
  • ISP and other processing to collect quality image, sound, and data from sensors
  • Flexible and low latency sensor data aggregation, bridging, and buffering from a wide variety of sensors
  • Accurate, low power, and predictable motor control to drive devices’ movement

Jump to

Block Diagram

Robots and Toys

Example Use Cases

Low Power AI Processing

  • Low Power on-device NN based processing
  • Reduced data traffic to the cloud, improving security and privacy and reducing bandwidth usage
  • Object detection, classification, counting and HMI including gestures and voice based commands

Motor Control

  • Low latency programmable motor control
  • Ability to interface with a wide variety of motors
  • Predictable performance to meet safety requirements

Audio Bridging

  • Connect up to 8 microphones to a processor
  • Audio data buffering to offload the processor
  • Support I2S, PDM microphone interfacing
  • Up to 1 Mb of on device RAM for buffering

Image Sensor Bridging

  • Connect a wide variety of image sensors to processors
  • MIPI PHYs supports up to 2.5 Gbps/lane, up to four lanes
  • Flexible host interfacing including CSI, SPI, PCIe
  • Flexible processing for video data muxing and stitching

Sensor Fusion and I/O Expansion

  • Interface to wide variety of sensors to create rich user experience
  • Flexible preprocessing including arbitration, time stamping, and filtering
  • Create programmable sensor fusion algorithms

Low Latency Sensor Bridging

  • Take advantage of parallel FPGA architecture to simultaneously collect data from multiple sensors
  • Interface to wide variety of sensors to create rich user experience
  • Flexible preprocessing including arbitration, time stamping, and filtering

Reference Designs

Key Phrase Detection

Reference Design

Key Phrase Detection

Continuous searches for a key phrase utterance via a digital MEMS microphone. Can be re-configured to work with any trained word or phrase.
Key Phrase Detection
Human Face Identification Reference Design

Reference Design

Human Face Identification Reference Design

Uses a Convolutional Neural Network in the ECP5 FPGA to detect a human face, and match to known registered faces. Can be adapted to work with any other object.
Human Face Identification Reference Design
Human Presence Detection

Reference Design

Human Presence Detection

Uses Lattice sensAI IP to continuously search for the presence of a human and reports results. Can be adapted to detect any other object.
Human Presence Detection
Object Counting AI

Reference Design

Object Counting AI

An example object counting application based on the Lattice sensAI stack. Includes SPI, DDR IP blocks, ISP engine, 8 CNN engines and a counting / overlay engine
Object Counting AI
MIPI DSI/CSI-2 to OpenLDI LVDS Interface Bridge for CertusPro-NX

Reference Design

MIPI DSI/CSI-2 to OpenLDI LVDS Interface Bridge for CertusPro-NX

Modular MIPI/D-PHY Reference Design - MIPI DSI/CSI-2 to OpenLDI LVDS Interface Bridge reference design takes DSI or CSI-2 MIPI data and converts them to OpenLDI format on LVDS.
MIPI DSI/CSI-2 to OpenLDI LVDS Interface Bridge for CertusPro-NX

Demos

Key Phrase Detection

Demo

Key Phrase Detection

Uses artificial intelligence (AI) to detect a specific key-phrase using a tiny, low-power iCE40 UltraPlus FPGA
Key Phrase Detection
CSI-2 PCIe Bridge Demonstration

Demo

CSI-2 PCIe Bridge Demonstration

This design demonstrates the functionality of transferring MIPI CSI-2 camera video data to computer via PCIe with a Direct Memory Access (DMA) engine.
CSI-2 PCIe Bridge Demonstration
Human Counting AI Demo

Demo

Human Counting AI Demo

Human upper-body detection and counting demonstration utilizes Lattice’s ECP5 FPGA and a Convolutional Neural Network (CNN) acceleration engine
Human Counting AI Demo
Human Face Detection AI Demo

Demo

Human Face Detection AI Demo

Uses Lattice sensAI IP to detect human faces on a tiny, low-power iCE40 UltraPlus FPGA implementing AI at the edge. Adaptable to detect other objects.
Human Face Detection AI Demo

IP Cores

CNN Plus Accelerator IP Core

IP Core

CNN Plus Accelerator IP Core

CNN Plus IP is a flexible accelerator IP that simplifies implementation of Ultra-Low power AI by leveraging capabilities of Lattice FPGAs.
CNN Plus Accelerator IP Core
Convolutional Neural Network (CNN) Accelerator IP

IP Core

Convolutional Neural Network (CNN) Accelerator IP

Implement AI solutions with CNNs from common or custom networks. Configure up to 16-bit widths. Works with Lattice Neural Network Compiler software tool.
Convolutional Neural Network (CNN) Accelerator IP
Convolutional Neural Network (CNN) Compact Accelerator

IP Core

Convolutional Neural Network (CNN) Compact Accelerator

Implement AI solutions with CNNs or BNNs that have power consumption in the mW range. Works with Lattice Neural Network Compiler software tool.
Convolutional Neural Network (CNN) Compact Accelerator

Development Kits & Boards

CertusPro-NX Versa Board

Board

CertusPro-NX Versa Board

CertusPro-NX Versa Board supports a wide range industry standards such as MIPI, SFP+, 10 GbE, LPDDR4 and PCIe (Gen3) for rapid prototyping and testing.
CertusPro-NX Versa Board
Embedded Vision Development Kit

Board

Embedded Vision Development Kit

Three-board Video Interface Platform (VIP) modular kit. Includes 2 MIPI/D-PHY camera input, FPGA processing, HDMI output. More Input/Output boards available.
Embedded Vision Development Kit
CrossLink-NX Evaluation Board

Board

CrossLink-NX Evaluation Board

For general evaluation and development with CrossLink-NX, includes many flexible interfaces such as FMC and PMOD with generous device IO access
CrossLink-NX Evaluation Board
HM01B0 UPduino Shield

Board

HM01B0 UPduino Shield

A complete development kit for implementing Artificial Intelligence (AI) using vision and sound as sensory inputs to a low-cost, low-power iCE40 UltraPlus FPGA.
HM01B0 UPduino Shield

Support

Technical Support

Need Help? We're Here to Assist You

Quality & Reliability

Reference Material to Help Answer Your Questions