Lattice sensAI Stack

Accelerate Integration of Flexible, Low Power Inferencing at the Edge

The full-featured Lattice sensAI stack includes everything you need to evaluate, develop and deploy FPGA-based Machine Learning / Artificial Intelligence solutions - modular hardware platforms, example demonstrations, reference designs, neural network IP cores, software tools for development, and custom design services.

The Lattice’s FPGA-based machine learning solutions are highly flexible, easy to implement, low power (from under 1 mW-1 W), small (package sizes starting at 5.5 mm2) and priced for high volume production.

Lattice can help you accelerate deployment of always-on, on-device AI into a wide range of Edge applications including mobile, smart home, smart city, smart factory, and smart car products.

sensAI Stack

Jump To

Videos

 Lattice sensAI: Making PCs smarter and more aware Expand Image

Lattice sensAI: Making PCs smarter and more aware

Lattice sensAI smart vision sensing technology tracks the user's attention while using their device making PCs smarter and more aware of their surroundings than ever before, by combining the built-in camera with machine learning models running on Lattice’s industry-leading low power FPGAs.
Lattice sensAI: Protecting your PC with onlooker detection  Expand Image

Lattice sensAI: Protecting your PC with onlooker detection

Lattice sensAI vision sensing technology is designed to help protect PC users from visual hackers by warning the user with an on-screen alert, or automatically turning on privacy screen settings, when enabled.
Lattice sensAI: Better collaboration with face framing technology Expand Image

Lattice sensAI: Better collaboration with face framing technology

Lattice sensAI vision sensing technology helps to keep PC users in frame, even while shifting positions, and understands when new team members enter the frame and automatically refocuses to accommodate them.
Lattice FAD sensAIExpand Image

Lattice sensAI: accelerating low power AI at the edge

The full-featured Lattice sensAI stack includes everything developers need to evaluate, develop and deploy FPGA-based Machine Learning / Artificial Intelligence solutions - modular hardware platforms, example demonstrations, reference designs, neural network IP cores, software tools for development, and custom design services.

Design Methodology Overview

sensAI Block Diagram

  • Training done using standard Machine Learning frameworks such as Tensorflow
  • Example NN models customized for Lattice devices provided
  • Reference Designs for popular use cases provided for download or through sensAI Studio
  • Verilog based hardware implementation in the FPGA using Lattice Radiant or Diamond

Software Based Methodology

Software Based Methodology

  • Training done using TensorFlow and converted to TF Lite
  • Lattice Propel is used to build SoC in the FPGA including optimized ML acceleration
  • C++ software based implementation in the FPGA for ease of use
  • A Reference Designs provided for download

System Architecture Examples

The following diagrams are examples of how FPGAs are commonly used to power AI/ML solutions. Lattice FPGAs give you a powerful and scalable AI/ML solution that’s simple, low-power and small-footprint. Scroll down to see hardware demonstrations you can download and evaluate today.

Preprocessor: CrossLink-NX AI Accelerator

  • Optimized FPGA Architecture built on low power process (FDSOI)
  • Flexible interface to connect to high quality image sensors through MIPI CSI-2
  • Ability to pass video data through CSI-2 or PCIe with AI acceleration meta data

Stand-alone: ECP5/iCE40 UltraPlus FPGA AI enabled System

  • Always-on, integrated solutions on ECP5 or iCE40 UltraPlus FPGA
  • Low latency and secure implementation
  • FPGA resources can be used for system control

Preprocessor: iCE40 UltraPlus Event Trigger

  • iCE40 UltraPlus FPGA for always-on detection of key-phrases or objects
  • Wakes-up a high performance ASIC/ASSP for further analytics only when required
  • Reduces overall system power consumption

Preprocessor: ECP5 FPGA AI processor

  • Scalable performance/power with ECP5 based neural network acceleration
  • ECP5 based I/O flexibility to seamlessly interface to on-board legacy devices including sensors
  • Low-end MCU for flexible system control

Postprocessor: ECP/iCE40 UltraPlus AI Accelerator

  • Add AI acceleration without significant modification of the system
  • Scalable performance/power with ECP or iCE40 UltraPlus
  • ASIC/ASSP can handle data preprocessing and pass on inference data only

Reference Designs

Object Classification Reference Design

Reference Design

Object Classification Reference Design

A reference design for implementing object classification based on Mobilenet NN model running on Lattice CertusPro-NX low power FPGA
Object Classification Reference Design
Hand Gesture Detection

Reference Design

Hand Gesture Detection

Implements a low power AI based system to detect hand gestures using an IR image sensor
Hand Gesture Detection
Key Phrase Detection

Reference Design

Key Phrase Detection

Continuous searches for a key phrase utterance via a digital MEMS microphone. Can be re-configured to work with any trained word or phrase.
Key Phrase Detection
Human Face Identification Reference Design

Reference Design

Human Face Identification Reference Design

Uses a Convolutional Neural Network in the ECP5 FPGA to detect a human face, and match to known registered faces. Can be adapted to work with any other object.
Human Face Identification Reference Design
Human Presence Detection

Reference Design

Human Presence Detection

Uses Lattice sensAI IP to continuously search for the presence of a human and reports results. Can be adapted to detect any other object.
Human Presence Detection

IP Cores

CNN Plus Accelerator IP

IP Core

CNN Plus Accelerator IP

Implement Ultra-Low Power AI solutions with CNNs. Configure up to 16-bit widths. Works with Lattice Neural Network Compiler software tool.
CNN Plus Accelerator IP
Convolutional Neural Network (CNN) Accelerator IP

IP Core

Convolutional Neural Network (CNN) Accelerator IP

Implement AI solutions with CNNs from common or custom networks. Configure up to 16-bit widths. Works with Lattice Neural Network Compiler software tool.
Convolutional Neural Network (CNN) Accelerator IP
Convolutional Neural Network (CNN) Compact Accelerator

IP Core

Convolutional Neural Network (CNN) Compact Accelerator

Implement AI solutions with CNNs or BNNs that have power consumption in the mW range. Works with Lattice Neural Network Compiler software tool.
Convolutional Neural Network (CNN) Compact Accelerator
CNN Co-Processor Accelerator IP

IP Core

CNN Co-Processor Accelerator IP

A CNN co-processor accelerator engine for use with low power Lattice FPGAs. The engine can be used with a RISC-V processor to create an SOC and implement TF Lite-based acceleration applications that leverage the parallel compute and distributed resource capabilities of Lattice FPGAs.
CNN Co-Processor Accelerator IP

Development Kits & Boards

Embedded Vision Development Kit

Board

Embedded Vision Development Kit

Three-board Video Interface Platform (VIP) modular kit. Includes 2 MIPI/D-PHY camera input, FPGA processing, HDMI output. More Input/Output boards available.
Embedded Vision Development Kit
CrossLink-NX Voice and Vision Machine Learning Board

Board

CrossLink-NX Voice and Vision Machine Learning Board

Designed for low-power machine learning applications with Lattice sensAI and CrossLink-NX. Includes image sensors, microphones, HyperRAM, and expansion ports.
CrossLink-NX Voice and Vision Machine Learning Board
CertusPro-NX Voice and Vision Machine Learning Board

Board

CertusPro-NX Voice and Vision Machine Learning Board

Design AI use cases for the Edge quickly! This board along with the Lattice sensAI solution stack provide the tools for developing vision and audio-based AI applications.
CertusPro-NX Voice and Vision Machine Learning Board
HM01B0 UPduino Shield

Board

HM01B0 UPduino Shield

A complete development kit for implementing Artificial Intelligence (AI) using vision and sound as sensory inputs to a low-cost, low-power iCE40 UltraPlus FPGA.
HM01B0 UPduino Shield

Demos

Object Classification Demonstration

Demo

Object Classification Demonstration

Sample demonstration for object detection, classification, and tracking multiple objects running on a low power general purpose FPGA using CNN Model
Object Classification Demonstration
User Tracking and Onlooker Detection Demostration

Demo

User Tracking and Onlooker Detection Demostration

Sample demonstration for detection and tracking of multiple human faces running on a low power general purpose FPGA using CNN Model
User Tracking and Onlooker Detection Demostration
Key Phrase Detection

Demo

Key Phrase Detection

Uses artificial intelligence (AI) to detect a specific key-phrase using a tiny, low-power iCE40 UltraPlus FPGA
Key Phrase Detection
Hand Gesture Detection

Demo

Hand Gesture Detection

Uses artificial intelligence (AI) to implement hand gesture detection algorithm using a tiny, low-power iCE40 UltraPlus FPGA
Hand Gesture Detection
Human Counting AI Demo

Demo

Human Counting AI Demo

Human upper-body detection and counting demonstration utilizes Lattice’s ECP5 FPGA and a Convolutional Neural Network (CNN) acceleration engine
Human Counting AI Demo

Software Tools

The Lattice sensAI solution stack includes the Neural Network Compiler for easy integration of networks developed in TensorFlow,Caffe or Keras into Lattice FPGAs - no prior RTL experience required.

The output of the Neural Network Compiler can then be integrated into the standard Lattice FPGA development tool flows - Lattice Diamond for ECP5, or Radiant for iCE40 UltraPlus, and finally deployed to the FPGA in your system.

  • Neural Network Compiler – Rapidly analyze, simulate, and compile various networks for implementation onto Lattice CNN/CNN Compact Accelerator IP cores.
  • Lattice Radiant Software – FPGA design tool for design creation, importing IP, implementation, bitstream generation, downloading the bitstream onto an FPGA. Supports iCE40 UltraPlus device.
  • Lattice Diamond Software – FPGA design and exploration tool targeted for ECP5 family of FPGAs.
  • Lattice Propel Software – Soft SoC design for design, compilation and deployment of hardware designs into Lattice’s FPGAs.
  • Lattice sensAI Studio – End to end Model Zoo, data labeling, model training, validation and deployment tool.

Lattice User Presence Awareness Kit

A demo kit with Windows application showcasing the use of computer vision to enable extended battery life in PCs.

Custom Design Services

The Lattice sensAI stack includes an ecosystem of select, worldwide design service partners that can deliver custom solutions for a range of end-applications, including mobile, smart home, smart city, smart factory, and smart cars. Click here for more information on Lattice sensAI certified partners.

Documentation

Information Resources
Downloads
TITLE NUMBER VERSION DATE FORMAT SIZE
Lattice sensAI Stack Product Brochure
I0265 7.0 5/20/2020 PDF 1.1 MB
TITLE NUMBER VERSION DATE FORMAT SIZE
Accelerating Implementation of Low Power Artificial Intelligence at the Edge
WP-0014 1.1 11/28/2018 PDF 1.2 MB
Lattice SensAI 4.1: Tools and IP Transform Low-Power FPGAs into Intelligent AI/ML Edge Computing Engines
1.0 11/10/2021 PDF 766.4 KB
Rising Edge AI Requirements Demand Higher Performance Solution
WP-0019 1.0 8/17/2019 PDF 1.1 MB
Harnessing the Power of AI: An Easy Start with Lattice’s sensAI
WP-0017 1.0 1/31/2019 PDF 2.3 MB
TITLE NUMBER VERSION DATE FORMAT SIZE
Lattice Propel 2.0 Patch for Lattice sensAI 4.1
1.0 11/10/2021 EXE 48.3 MB

*By clicking on the "Notify Me of Changes" button, you agree to receive notifications on changes to the document(s) you selected.

Awards

The Electronics Industry Awards 2019

Internet of Things Product of the Year

EDN Hot 100 Product Award

Tools & Development

AI Breakthrough Award

Best AI-Based Solution for Engineering

Assodel Award 2019

Best Industrial and Automotive Related Hardware Product

China Electronic Market 2018 Editor's Choice Awards

Most Competitive FPGA Product

Like most websites, we use cookies and similar technologies to enhance your user experience. We also allow third parties to place cookies on our website. By continuing to use this website you consent to the use of cookies as described in our Cookie Policy.