Lattice sensAI Stack

Accelerate Integration of Flexible, Low Power Inferencing at the Edge

Related Products

Related Applications

The full-featured Lattice sensAI stack includes everything you need to evaluate, develop and deploy FPGA-based Machine Learning / Artificial Intelligence solutions - modular hardware platforms, example demonstrations, reference designs, neural network IP cores, software tools for development, and custom design services.

The Lattice’s FPGA-based machine learning solutions are highly flexible, easy to implement, low power (from under 1 mW - 5 W), small (package sizes starting at 5.5 mm2) and priced for high volume production.

Lattice can help you accelerate deployment of always-on, on-device AI into a wide range of Edge applications including mobile, smart home, smart city, smart factory, and smart car products.

sensAI 6.0 Stack

Jump To

Videos

Lattice Avant™ Launch Event – Dec. 5, 2022
Expand Video

Lattice Avant™ Launch Event – Dec. 5, 2022

On Dec. 5, 2022, Lattice unveiled Lattice Avant, a new FPGA platform purpose-built to bring the company’s power efficient architecture, small size, and performance leadership to mid-range FPGAs, offering best-in-class power efficiency, advanced connectivity, and optimized compute. Watch the livecast replay to experience the next level of FPGA innovation.
Lattice sensAI: Making PCs smarter and more aware
Expand Video

Lattice sensAI: Making PCs smarter and more aware

Lattice sensAI smart vision sensing technology tracks the user's attention while using their device making PCs smarter and more aware of their surroundings than ever before, by combining the built-in camera with machine learning models running on Lattice’s industry-leading low power FPGAs.
Lattice sensAI: Protecting your PC with onlooker detection
Expand Video

Lattice sensAI: Protecting your PC with onlooker detection

Lattice sensAI vision sensing technology is designed to help protect PC users from visual hackers by warning the user with an on-screen alert, or automatically turning on privacy screen settings, when enabled.
Lattice sensAI: Better collaboration with face framing technology
Expand Video

Lattice sensAI: Better collaboration with face framing technology

Lattice sensAI vision sensing technology helps to keep PC users in frame, even while shifting positions, and understands when new team members enter the frame and automatically refocuses to accommodate them.
Lattice FAD sensAI
Expand Video

Lattice sensAI: accelerating low power AI at the edge

The full-featured Lattice sensAI stack includes everything developers need to evaluate, develop and deploy FPGA-based Machine Learning / Artificial Intelligence solutions - modular hardware platforms, example demonstrations, reference designs, neural network IP cores, software tools for development, and custom design services.

Design Methodology Overview

sensAI Block Diagram

  • Training done using standard Machine Learning frameworks such as Tensorflow
  • Example NN models customized for Lattice devices provided
  • Reference Designs for popular use cases provided for download or through sensAI Studio
  • Verilog based hardware implementation in the FPGA using Lattice Radiant or Diamond

Software Based Methodology

Software Based Methodology

  • Training done using TensorFlow and converted to TF Lite
  • Lattice Propel is used to build SoC in the FPGA including optimized ML acceleration
  • C++ software based implementation in the FPGA for ease of use
  • A Reference Designs provided for download

System Architecture Examples

The following diagrams are examples of how FPGAs are commonly used to power AI/ML solutions. Lattice FPGAs give you a powerful and scalable AI/ML solution that’s simple, low-power and small-footprint. Scroll down to see hardware demonstrations you can download and evaluate today.

Preprocessor: CrossLink-NX AI Accelerator Block DiagramExpand Image

Preprocessor: CrossLink-NX AI Accelerator

  • Optimized FPGA Architecture built on low power process (FDSOI)
  • Flexible interface to connect to high quality image sensors through MIPI CSI-2
  • Ability to pass video data through CSI-2 or PCIe with AI acceleration meta data
Stand-alone: ECP5/iCE40 UltraPlus FPGA AI enabled SystemExpand Image

Stand-alone: ECP5/iCE40 UltraPlus FPGA AI enabled System

  • Always-on, integrated solutions on ECP5 or iCE40 UltraPlus FPGA
  • Low latency and secure implementation
  • FPGA resources can be used for system control
Preprocessor: iCE40 UltraPlus Event TriggerExpand Image

Preprocessor: iCE40 UltraPlus Event Trigger

  • iCE40 UltraPlus FPGA for always-on detection of key-phrases or objects
  • Wakes-up a high performance ASIC/ASSP for further analytics only when required
  • Reduces overall system power consumption
Preprocessor: ECP5 FPGA AI processorExpand Image

Preprocessor: ECP5 FPGA AI processor

  • Scalable performance/power with ECP5 based neural network acceleration
  • ECP5 based I/O flexibility to seamlessly interface to on-board legacy devices including sensors
  • Low-end MCU for flexible system control
Postprocessor: ECP/iCE40 UltraPlus AI Accelerator Expand Image

Postprocessor: ECP/iCE40 UltraPlus AI Accelerator

  • Add AI acceleration without significant modification of the system
  • Scalable performance/power with ECP or iCE40 UltraPlus
  • ASIC/ASSP can handle data preprocessing and pass on inference data only

Reference Designs

Object Classification Reference Design

Reference Design

Object Classification Reference Design

The Object Classification reference design shows examples on implementing machine-learning based object classification to edge devices applications.
CertusPro-NX, Avant-E, CrossLink-NX 
Machine Learning/AI, Lattice sensAI, Object Detection, Object Identification, Space 
Object Classification Reference Design
Hand Gesture Detection

Reference Design

Hand Gesture Detection

Implements a low power AI based system to detect hand gestures using an IR image sensor
iCE40 UltraPlus 
Machine Learning/AI, Gesture Detection, Lattice sensAI 
Hand Gesture Detection
Barcode Detection Reference Design

Reference Design

Barcode Detection Reference Design

Exhibits the barcode detection using CertusPro-NX Voice & Vision Machine Learning Board & its camera barcode detection potential based on Yolov5 NN models.
CertusPro-NX 
Machine Learning/AI, Lattice sensAI, Barcode Detection 
Barcode Detection Reference Design
Scene Segmentation Reference Design

Reference Design

Scene Segmentation Reference Design

Efficient and low power approach for implementing scene segmentation using Lattice CrossLink-NX FPGA
CrossLink-NX 
Lattice sensAI, Machine Learning/AI, SPI, Object Counting, Object Segmentation, Image Processing, Camera, Defense 
Scene Segmentation Reference Design
Key Phrase Detection

Reference Design

Key Phrase Detection

Continuous searches for a key phrase utterance via a digital MEMS microphone. Can be re-configured to work with any trained word or phrase.
iCE40 UltraPlus 
Audio AI, Machine Learning/AI, Lattice sensAI, Home Control Appliances & Security, IoT & Wearables, Notebooks & PCs, Printers, Smart Toys & Robots, Tablets 
Key Phrase Detection
Show More

IP Cores

Advanced CNN Accelerator IP

IP Core

Advanced CNN Accelerator IP

Calculates full layers of Neural Network including convolution layer, pooling layer, batch normalization layer, and fully connected layer.
CertusPro-NX, Avant-E 
Machine Learning/AI, Lattice sensAI, Object Detection, Object Identification, Home Control Appliances & Security, IoT & Wearables, CNN Bundle 
Advanced CNN Accelerator IP
CNN Plus Accelerator IP Core

IP Core

CNN Plus Accelerator IP Core

CNN Plus IP is a flexible accelerator IP that simplifies implementation of Ultra-Low power AI by leveraging capabilities of Lattice FPGAs.
CrossLink-NX, Certus-NX, CertusPro-NX 
Machine Learning/AI, Lattice sensAI, AR / VR & Drones, Home Control Appliances & Security, IoT & Wearables, Notebooks & PCs, Printers, Smart Toys & Robots, Tablets, UAVs, Automotive, CNN Bundle 
CNN Plus Accelerator IP Core
Convolutional Neural Network (CNN) Accelerator IP

IP Core

Convolutional Neural Network (CNN) Accelerator IP

Implement AI solutions with CNNs from common or custom networks. Configure up to 16-bit widths. Works with Lattice Neural Network Compiler software tool.
ECP5 / ECP5-5G 
Machine Learning/AI, Lattice sensAI, AR / VR & Drones, Home Control Appliances & Security, IoT & Wearables, Notebooks & PCs, Printers, Smart Toys & Robots, Tablets, Defense 
Convolutional Neural Network (CNN) Accelerator IP
Convolutional Neural Network (CNN) Compact Accelerator

IP Core

Convolutional Neural Network (CNN) Compact Accelerator

Implement AI solutions with CNNs or BNNs that have power consumption in the mW range. Works with Lattice Neural Network Compiler software tool.
iCE40 UltraPlus 
Machine Learning/AI, Lattice sensAI, AR / VR & Drones, Home Control Appliances & Security, IoT & Wearables, Notebooks & PCs, Printers, Smart Toys & Robots, Tablets, UAVs 
Convolutional Neural Network (CNN) Compact Accelerator
CNN Co-Processor Accelerator IP

IP Core

CNN Co-Processor Accelerator IP

A CNN co-processor accelerator engine for use with low power Lattice FPGAs. The engine can be used with a RISC-V processor to create an SOC and implement TF Lite-based acceleration applications that leverage the parallel compute and distributed resource capabilities of Lattice FPGAs.
CrossLink-NX 
Lattice sensAI, Machine Learning/AI, AR / VR & Drones, Home Control Appliances & Security, IoT & Wearables, Notebooks & PCs, Printers, Smart Toys & Robots, Tablets, UAVs, CNN Bundle 
CNN Co-Processor Accelerator IP

Development Kits & Boards

Embedded Vision Development Kit

Board

Embedded Vision Development Kit

Three-board Video Interface Platform (VIP) modular kit. Includes 2 MIPI/D-PHY camera input, FPGA processing, HDMI output. More Input/Output boards available.
CrossLink, ECP5 / ECP5-5G 
3D Video, Camera, DDR SDRAM, HDMI, Image Processing, Machine Learning/AI, Lattice sensAI, Signal Aggregation, Video Tx/Rx, VIP, Lattice mVision, IoT & Wearables, Smart Toys & Robots, Home Control Appliances & Security, Notebooks & PCs, Tablets, Printers, AR / VR & Drones, UAVs 
Embedded Vision Development Kit
CrossLink-NX Voice and Vision Machine Learning Board

Board

CrossLink-NX Voice and Vision Machine Learning Board

Designed for low-power machine learning applications with Lattice sensAI and CrossLink-NX. Includes image sensors, microphones, HyperRAM, and expansion ports.
CrossLink-NX 
CSI-2, CSI Camera, Camera, Microphone, PMOD, HyperRAM, Machine Learning/AI, Lattice sensAI, Automotive 
CrossLink-NX Voice and Vision Machine Learning Board
CrossLink-NX-33 Voice and Vision Machine Learning Board

Board

CrossLink-NX-33 Voice and Vision Machine Learning Board

CrossLink-NX-33 Voice and Vision Machine Learning Board is designed using Crosslink-NX 33K, ideal for machine learning applications.
CrossLink-NX 
CSI-2, SPI, CSI Camera, Camera, Microphone, PMOD, Machine Learning/AI, Lattice sensAI, Automotive 
CrossLink-NX-33 Voice and Vision Machine Learning Board
CertusPro-NX Voice and Vision Machine Learning Board

Board

CertusPro-NX Voice and Vision Machine Learning Board

Design AI use cases for the Edge quickly! This board along with the Lattice sensAI solution stack provide the tools for developing vision and audio-based AI applications.
CertusPro-NX 
Machine Learning/AI, DSP, PMOD, Camera, Lattice sensAI, Space, Automotive 
CertusPro-NX Voice and Vision Machine Learning Board
CrossLink-NX VIP Sensor Input Board

Board

CrossLink-NX VIP Sensor Input Board

Intended for use with the Lattice Embedded Vision Development Kit. Includes 4 camera inputs and HyperRam for embedded video applications with CrossLink-NX
CrossLink-NX 
VIP, Camera, CSI Camera, CSI-2, HyperRAM, Image Processing, Machine Learning/AI, Lattice sensAI, Parallel Video, PMOD, SPI Flash, Video Aggregation, Video Bridge, Video Tx/Rx, Lattice mVision, AR / VR & Drones, Home Control Appliances & Security, IoT & Wearables, Notebooks & PCs, Printers, Smart Toys & Robots, Tablets, Defense, Automotive 
CrossLink-NX VIP Sensor Input Board
Show More

Demos

Object Classification Demonstration

Demo

Object Classification Demonstration

This object classification demo provides a sample application for detecting, classifying, and tracking multiple objects running on CertusPro-NX FPGA.
CertusPro-NX, Avant-E 
Machine Learning/AI, Lattice sensAI, Object Detection, Object Identification, Space 
Object Classification Demonstration
User Tracking and Onlooker Detection Demonstration

Demo

User Tracking and Onlooker Detection Demonstration

Sample demonstration for detection and tracking of multiple human faces running on a low power general purpose FPGA using CNN Model
CrossLink-NX 
Machine Learning/AI, Lattice sensAI, Image Processing, Camera, FaceI dentification 
User Tracking and Onlooker Detection Demonstration
Barcode Detection Demonstration

Demo

Barcode Detection Demonstration

This demo implements an algorithm for barcode detection using the CertusPro-NX Voice and Vision Machine Learning Board.
CertusPro-NX 
Machine Learning/AI, Lattice sensAI, Barcode Detection 
Barcode Detection Demonstration
Key Phrase Detection

Demo

Key Phrase Detection

Uses artificial intelligence (AI) to detect a specific key-phrase using a tiny, low-power iCE40 UltraPlus FPGA
iCE40 UltraPlus 
Audio AI, Machine Learning/AI, Microphone, Lattice sensAI, Home Control Appliances & Security, IoT & Wearables, Notebooks & PCs, Printers, Smart Toys & Robots, Tablets 
Key Phrase Detection
User Background Blurring Demonstration

Demo

User Background Blurring Demonstration

Efficient and low power approach for implementing user background blurring using Lattice CrossLink-NX FPGA
CrossLink-NX 
Lattice sensAI, Machine Learning/AI, SPI, Object Counting, Object Segmentation, Image Processing, Camera 
User Background Blurring Demonstration
Show More

Software Tools

The Lattice sensAI solution stack includes the Neural Network Compiler for easy integration of networks developed in TensorFlow,Caffe or Keras into Lattice FPGAs - no prior RTL experience required.

The output of the Neural Network Compiler can then be integrated into the standard Lattice FPGA development tool flows - Lattice Diamond for ECP5, or Radiant for iCE40 UltraPlus, and finally deployed to the FPGA in your system.

  • Lattice sensAI Studio – End to end Model Zoo, data labeling, model training, validation and deployment tool.
  • Neural Network Compiler – Rapidly analyze, simulate, and compile various networks for implementation onto Lattice CNN/CNN Compact Accelerator IP cores.
  • Lattice Radiant Software – FPGA design tool for design creation, importing IP, implementation, bitstream generation, downloading the bitstream onto an FPGA. Supports iCE40 UltraPlus device.
  • Lattice Diamond Software – FPGA design and exploration tool targeted for ECP5 family of FPGAs.
  • Lattice Propel Software – Soft SoC design for design, compilation and deployment of hardware designs into Lattice’s FPGAs.

Lattice User Presence Awareness Kit

A demo kit with Windows application showcasing the use of computer vision to enable extended battery life in PCs.

Custom Design Services

The Lattice sensAI stack includes an ecosystem of select, worldwide design service partners that can deliver custom solutions for a range of end-applications, including mobile, smart home, smart city, smart factory, and smart cars. Click here for more information on Lattice sensAI certified partners.

Documentation

Information Resources
TITLE NUMBER VERSION DATE FORMAT SIZE
Select All
Lattice sensAI Stack Product Brochure
I0265 7.0 5/20/2020 PDF 1.1 MB
TITLE NUMBER VERSION DATE FORMAT SIZE
Select All
Bringing Laptops Into the 2020s
WP0024 2.0 7/1/2020 PDF 723.4 KB
Accelerating Implementation of Low Power Artificial Intelligence at the Edge
WP-0014 1.1 11/28/2018 PDF 1.2 MB
Lattice SensAI 4.1: Tools and IP Transform Low-Power FPGAs into Intelligent AI/ML Edge Computing Engines
1.0 11/10/2021 PDF 766.4 KB
Rising Edge AI Requirements Demand Higher Performance Solution
WP-0019 1.0 8/17/2019 PDF 1.1 MB
Harnessing the Power of AI: An Easy Start with Lattice’s sensAI
WP-0017 1.0 1/31/2019 PDF 2.3 MB

*By clicking on the "Notify Me of Changes" button, you agree to receive notifications on changes to the document(s) you selected.

Awards

The Electronics Industry Awards 2019 Expand Image

The Electronics Industry Awards 2019

Internet of Things Product of the Year
EDN Hot 2018Expand Image

EDN Hot 100 Product Award

Tools & Development

AI Breakthrough Award Expand Image

AI Breakthrough Award

Best AI-Based Solution for Engineering
Assodel Award 2019

Assodel Award 2019

Best Industrial and Automotive Related Hardware Product
China Electronic Market 2018 Editor's Choice AwardsExpand Image

China Electronic Market 2018 Editor's Choice Awards

Most Competitive FPGA Product

ABOUT LATTICE

SALES

SUPPORT

FOLLOW