Project Overview

Title: Video Pipeline for Machine Computer Vision
Client: JR Spidell
Advisor: Dr. Zambreno
Description: This is a computer vision video pipeline deployed on an FPGA.
Highlighted Technologies: OpenCV, PyTorch, Vitis-AI, and the Ultra96v2 FPGA

Problem Statement
People with mobility and cognitive impairments, such as Cerebral Palsy, face significant challenges in maintaining independence and safety. Traditional wheelchairs often lack the advanced technologies needed to support these users, leaving gaps in autonomy, communication, and safety. Healthcare professionals and caregivers also struggle with the absence of real-time alerts for medical emergencies like seizures, increasing the risk of delayed responses. These challenges not only affect the quality of life for wheelchair-bound individuals but also limit opportunities for proactive care.
Our client wants to address these issues by developing assistive wheelchair technologies with features such as advanced mobility assistance and real-time seizure detection. This system aims to increase wheelchair user autonomy, improve safety, and reduce caregiver stress. Our team is collaborating with the client to develop a subsystem that detects, locates, and presents information on the user’s eyes in real time that will be used in future iterations of the client’s vision.

Figure: Overall System Flow

Figure: Video Processing Pipeline Demo
Team Members
Mason Inman
Semantic Segmentation QuantizationSoftware Engineering Major continuing for M.S. in Artificial Intelligence.

James Minardi
Hardware IntegrationComputer Engineering Major focussing on embedded computer graphics and machine learning.
Eli Ripperda
Embedded SystemsComputer Engineering Major with industry experience in embedded systems and data engineering.

Lindsey Wessel
Machine Learning - Eye LocatingSoftware Engineering Major. Going into Computer Graphics.
Presentations
First Semester Final PresentationIRP Presentation
Project Poster
Project PosterDesign Documents
4910 Design Document4920 Design Document
4910 Weekly Reports
Report 1Report 2
Report 3
Report 4
Report 5
Report 6
Report 7
Report 8
Report 9
Report 10
4920 Bi-Weekly Reports
Report 1Report 2
Report 3
Report 4
Report 5
Lightning Talks
LT 1 - Product ResearchLT 2 - Problems & Users
LT 3 - User Needs & Requirements
LT 4 - Project Planning
LT 5 - Detailed Design
LT 6 - Design Check-In
LT 7 - Prototyping
LT 8 - Ethics