Project Overview

Team Logo

Title: Video Pipeline for Machine Computer Vision

Client: JR Spidell

Advisor: Dr. Zambreno

Description: This is a computer vision video pipeline deployed on an FPGA.

Highlighted Technologies: OpenCV, PyTorch, Vitis-AI, and the Ultra96v2 FPGA

ROI Gif

Problem Statement

People with mobility and cognitive impairments, such as Cerebral Palsy, face significant challenges in maintaining independence and safety. Traditional wheelchairs often lack the advanced technologies needed to support these users, leaving gaps in autonomy, communication, and safety. Healthcare professionals and caregivers also struggle with the absence of real-time alerts for medical emergencies like seizures, increasing the risk of delayed responses. These challenges not only affect the quality of life for wheelchair-bound individuals but also limit opportunities for proactive care.

Our client wants to address these issues by developing assistive wheelchair technologies with features such as advanced mobility assistance and real-time seizure detection. This system aims to increase wheelchair user autonomy, improve safety, and reduce caregiver stress. Our team is collaborating with the client to develop a subsystem that detects, locates, and presents information on the user’s eyes in real time that will be used in future iterations of the client’s vision.

System Flow Diagram

Figure: Overall System Flow


Demo Gif

Figure: Video Processing Pipeline Demo

Team Members

Mason Inman

Mason Inman

Semantic Segmentation Quantization

Software Engineering Major continuing for M.S. in Artificial Intelligence.

James Minardi

James Minardi

Hardware Integration

Computer Engineering Major focussing on embedded computer graphics and machine learning.

Eli Ripperda

Eli Ripperda

Embedded Systems

Computer Engineering Major with industry experience in embedded systems and data engineering.

Lindsey Wessel

Lindsey Wessel

Machine Learning - Eye Locating

Software Engineering Major. Going into Computer Graphics.




Presentations

First Semester Final Presentation
IRP Presentation



Project Poster

Project Poster



Design Documents

4910 Design Document
4920 Design Document



4910 Weekly Reports

Report 1
Report 2
Report 3
Report 4
Report 5
Report 6
Report 7
Report 8
Report 9
Report 10



4920 Bi-Weekly Reports

Report 1
Report 2
Report 3
Report 4
Report 5



Lightning Talks

LT 1 - Product Research
LT 2 - Problems & Users
LT 3 - User Needs & Requirements
LT 4 - Project Planning
LT 5 - Detailed Design
LT 6 - Design Check-In
LT 7 - Prototyping
LT 8 - Ethics