Overview

This project intends to deal with integrating and building a object detection framework and its dectection based on camera and peripheral micro-controller (Arduino BLE 33 mBed) connected to a NVIDIA Jetson Nano acting as a hub. For the purpose of demonstration we would be attaching a low power camera attached to the micro-controller to act as an inference engine built over Machine Learning Framework to interact with Jetson Nano and form a feedback loop to alarm a system of detection of person in frame of camera. The aim of this project is to train a machine learning model and deploy over low-power, low-memory micro-controllers and form feedback system of the Embedded Network Hub (Jetson). This would include bringing up completely independent hardware which includes the controlling unit, sensors, and software bring up, which includes Operating systems, communication protocols, coding styles, and languages.

Using low-level micro-controllers is cheap and provides more opportunity (future goal) to scale the product under development and testing providing both financial and engineering benefits.

NVIDIA Jetson acts as a server for the peripheral boards(clients). The term voting technique indicates that at any given time the reading that is received from the majority of the sensors will be considered for further actions. Two Camera feeds are available to work with one resides on the NVIDIA Jetson and other connected to the Peripheral Board.

Major Goals:

• Yocto to compile meta-tegra.
• Running Tegra-demo-distro with Arduino Setup.
• Setup Camera board and Communication.
• Build deep-stream or opencv (hardware accelerated) running on Jetson Nano.
• Run AI Image recognition project working with on image on Host/Hub (Jetson) and/or Arduino BLE 33 with Camera.

System works as a hard real time system, which captures camera feed at set time interval irrespective of how much time the sensors take to send data over to Nano. The overaching goal of this project would be create a Hub and Client mechanism such that machine learning can be deployed on micro-controllers with and without GPU’s.

Block Diagrams

Hardware Block Diagrams

Software Block Diagrams

Functional Block Diagrams

See Software Block Diagram

Target Build System

We intend to use Yocto as the bring up tool to generate Linux image on NVIDIA Jetson Nano.

Hardware Platform

Intended hardware platforms for this project:

• Jetson Nano
• Arduino Nano 33 BLE
• OV7675 Camera

Open Source Projects

• Tensorflow
• Tensorflow Lite
• Tensorflow Micro
• Meta Tegra for Yocto
• Arduino CLI
• Arduino IDE
• TinyML

Previously Used Content

• Yocto
• Signal handlers
• Clocks
• Init Scripts and Rootfs Overlays

New Content

• Yocto Image for Jetson Nano _Markdown
• Creating Modules and scripts for Arduino BLE 33
• Interface for Jetson and Arduino _Markdown
• Deplying Tiny Machine Learning (TinyML)
• Case study for TinyML _Markdown

Shared Material

• Flashing Image to SD Card

Source Code Organization

• Project Webpage: Dynamically Generated Documentation
• Project Overview: Project Synopsis
• Project Schedule
• Caleb’s Repository: One-click-build script Focused
• Arpit’s Repository: Image Development Focused
• Zach’s Repository: Driver & Module Code Focused

Group’s Overview

Team Project Members

• Caleb: Yocto Image Setup and build for Jetson Tegra, Dockerfile 1-click-image
• Zach: Serial communication Between Subsystems
• Arpit: Machine Learning Framework and TinyML Setup and yocto compatibility.
  • Dive Deep into Tiny Machine Learning deployment
  • Integrate Camera and peripheral to detect Person in frame.
  • Code Porting and Tensorflow dependencies.

Schedule Page

• Project Schedule

Student Video Demos

• Arpit Savarkar’s Video
• Caleb Provost’s Video
• Zach Turner’s Video