Camera-Based Non-Contact Continuous PPG Monitoring & Authentication System

Monday. June 10, 2019 - 4 mins

---

Intro

This technique aims at accurate and continious vital sign monitoring (and authentication) from mobile devices in a non-contact manner. More specifically, it monitors the heart rate and blood pressure by extracting the PPG (photoplethysmography) signal from the recorded videos of an individual’s skin such as face, forehead, palm, etc.

Related works were introduced and published in several papers (listed below as reference).

Summary

• Carried out signal amplification and PPG extraction based on facial videos.
• Calculated heart rate and blood pressure estimation based on extracted PPG signal.
• Implemented PPG-based user authentication with an average accuracy of 86.67%.

Motivaiton

It is crucial to continuously monitor vital signs to determine an individual׳s physiological and pathological conditions in daily life or clinical settings. Among them, heart rate (HR) and blood pressure (BP) play particularly important roles, as they are directly related to an individual’s mood, behavior, and physical activities. Abnormal changes in the physiological functions of the cardiovascular system could lead to life-threatening complications and would be reflected on the readings of HR and BP measurements.

The vital signs are usually monitored by medical professionals periodically. However, without continuous monitoring of those vital signs, some minor symptoms could go undetected and eventually result in clinical deterioration. The observable changes in vital signs are often present in a patient for 8 to 24 hours before a sudden life-threatening event, such as sudden cardiac arrest (SCA) and heart attack. Continuous monitoring of vital signs can allow experts to take precautionary actions to prevent such life-threatening events. However, most of the existing vital signs monitors adopted in the clinical settings are neither affordable nor portable, which has been prohibitive for their use in individual-centered pervasive healthcare (e.g., in home healthcare). Therefore, it is important to develop a low-cost monitoring system that can remotely and continuously measure vital signs with ease of use and convenience.

Rationale

• PPG: A technology being used to reflects cardiac activities by measuring blood volume changes in the superficial atrial structure.
• The subtle changes of blood volume caused by cardiovascular activities, can be captured by the camera as changes in color.

Evaluation Settings

• Vedio Length & Test Period: 10s/vedio, tested over 3 month
• Minimum Resolution & Frame Rate: 640 × 480 pixels @ 15/30 fps
• Illuminance: different light conditions (150-1200 lumen/m^2) and different backgrounds in the real life scenarios

Methods

• Face Detection & Tracking:
  1. Viola-Jones algorithm (detection)
  2. Kanade-Lucas-Tomasi (KLT) (tracking)
• Image Processing and Signal Amplification:
  1. Changes in color channels are low in magnitude
  2. Image Pyramid: filters out the redundant information, preserves important features of an image
  3. Laplacian pyramid: amplifies the signal of interest (frequency band of 0.5-4 Hz) and reconstructs the original image to make the color changes more significant
• PPG Extraction:
  1. Color Average and Pre-processing: combine pixel values of each color segment to a spatially averaged output over ROI, perform detrend and normalization
  2. ICA: to reduce the statistical dependencies between each color segment
  3. Moving window integration: to discard the unwanted ripple remained in the source signal and to obtain better intrinsic features of the waveform
• HR Estimation:
  1. Estimate the power spectral density (PSD) of the preprocessed signal using FFT
  2. The pulse frequencycan be estimated as the frequency associated with the highest power
• BP Estimation: (systolic BP)
  1. Pulse transit time (PTT) is the time taken by a pulse wave to travel from the left ventricle to a distant monitoring location
  2. Estimate PTT based on the peak to peak time (latency) between the PPG signals extracted from different selected ROIs on face
  3. Carry out calibration based on the correlation between the PTT and the reference BP data
  4. The absolute blood pressure (ABP) estimation is refined based on their correlation coefficients K1 and K2, as the following equation: BP= K1/PTT+K2
• Authentication:
  1. Feature extraction: discrete wavelet transform (DWT)
  2. Classifier: stacked autoencoder and softmax

Performance

• HR Estimation: 95% confidence (±1.96 SD) (1-5 BPM error)
• BP Estimation: (systolic BP) < ±10mmHg error
• Authentication: 66.67% - 100% accuracy (86.67% on average)

Potential Scenarios

• Authentication and vital sign (heart rate, blood pressure) monitoring and physical condition estimation for smart (truck) cockpit
• Authentication for home entertainment system
• Better and securer authentication for ATM

Reference

[1] Omkar R. Patil, Wei Wang, Yang Gao, Wenyao Xu, Zhanpeng Jin. “A low-cost, camera-based continuous PPG monitoring system using Laplacian pyramid.” Smart Health 9 (2018): 2-11.

[2] Omkar Patil, Wei Wang, Yang Gao and Zhanpeng Jin. “A Non-Contact PPG Biometric System Based on Deep Neural Network.” The 9th IEEE International Conference on Biometrics: Theory, Applications, and Systems (BTAS). IEEE, 2018.

[3] Omkar Patil, Wei Wang, Yang Gao and Zhanpeng Jin. “A Camera-Based Pulse Transit Time Approach for Non-Intrusive Blood Pressure Monitoring.” The 7th IEEE International Conference on Healthcare Informatics (ICHI’19).

Tweet Share