Webcam-pulse-detector

A python application that detects and highlights the heart-rate of an individual (using only their own webcam) in real-time.

View the Project on GitHub thearn/webcam-pulse-detector

Alt text

webcam-pulse-detector

UPDATE: Stand-alone (no dependancy) precompiled application now available!

The application can be run by simply executing the binary contained in the zip file for your platform. This code can also be run from source by following the instructions below.

A python code that detects the heart-rate of an individual using a common webcam or network IP camera. Tested on OSX 10.8 (Mountain Lion), Ubuntu 13.04 (Ringtail), and Windows 7 & 8.

Inspired by reviewing recent work on Eulerian Video Magnification, with motivation to implement something visually comparable (though not necessarily identical in formulation) to their pulse detection examples using Python and OpenCV (see https://github.com/brycedrennan/eulerian-magnification for a more general take on the offline post-processing methodology). This goal is comparable to those of a few previous efforts in this area (such as https://github.com/mossblaser/HeartMonitor).

This code was developed at NASA Glenn Research Center in support of OpenMDAO, under the Aeronautical Sciences Project in NASA's Fundamental Aeronautics Program, as well as the Crew State Monitoring Element of the Vehicle Systems Safety Technologies Project, in NASA’s Aviation Safety Program.

How it works:

This application uses OpenCV to find the location of the user's face, then isolate the forehead region. Data is collected from this location over time to estimate the user's heart rate. This is done by measuring average optical intensity in the forehead location, in the subimage's green channel alone (a better color mixing ratio may exist, but the blue channel tends to be very noisy). Physiological data can be estimated this way via a combination of photoplethysmology and the optical absorption characteristics of (oxy-) haemoglobin (see http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-16-26-21434).

With good lighting and minimal noise due to motion, a stable heartbeat should be isolated in about 15 seconds. Other physiological waveforms (such as Mayer waves) should also be visible in the raw data stream.

Once the user's heart rate has been estimated, real-time phase variation associated with this frequency is also computed. This allows for the heartbeat to be exaggerated in the post-process frame rendering, causing the highlighted forehead location to pulse in sync with the user's own heartbeat.

Support for detection on multiple simultaneous individuals in a single camera's image stream is definitely possible, but at the moment only the information from one face is extracted for analysis.

The overall dataflow/execution order for the real-time signal processing looks like:

Alt text

This signal processing design is implemented in the OpenMDAO assembly object defined in lib/processors.py.

The definition of each component block used can be found in the source files lib/imageProcess.py, lib/signalProcess.py, and lib/sliceops.py. The @bin and @bout blocks in the above graph denote assembly-level input and output.

Running from source:

Basic requirements:

OpenCV is a powerful open-source computer vision library, with a convenient numpy-compatible interface in the cv2 bindings.

If you want to run from source and modify UI or data output behavior, and make only minor changes to the signal processing, you can checkout and run the no_openmdao branch with no further dependancies. This branch implements a 'flattened' version of the master branch's OpenMDAO assembly, but as a plain python object.

Both the no_openmdao branch and the precompiled binary applications contain support for real time serial port and UDP output of the estimated heart rate.

However, if you would like to make significant or exploratory changes to the signal processing code (eg. multichannel support, PCA/ICA data factorizations, better filters, etc.), you should run the master branch with OpenMDAO support and build on what is already there (see instructions below).

Requirements for master branch (OpenMDAO support):

OpenMDAO is an open-source engineering framework that serves as a convenient environment to containerize the required real-time analysis, and allow for that analysis to be easily tweaked to specification and compared with alternative designs. Upon installation, OpenMDAO is bootstrapped into its own Python virtualenv, which must be activated before use (see the Quickstart section below). OpenMDAO requires python 2.6+, numpy, scipy, and matplotlib (see http://openmdao.org/docs/getting-started/requirements.html)

Running Windows, completely new to Python, but still would like to hack on the master branch source code? Full instructions for getting started with all requirements needed to run this code are available here

Quickstart:

On branch no_openmdao:

On branch master:

. OpenMDAO/bin/activate

Or on Windows:

OpenMDAO\Scripts\activate

python get_pulse.py

Usage notes:

TODO: