Pose Detection with OpenCV in OpenGL By Fina Beauchamp

Project Description

I decided to continue working with OpenCV after creating an Android camera filter app for my Senior project. This project analyzes different body parts from images and alters a model in OpenGL based off the location of each part.

Hands and Robot Scan

The result of the project with an image input, with both the hands, and head detected. The robot's arms are shifted slightly up because of the orientation of the hands in the image.

The Steps Used

  1. Specify the Haar-feature files (both hand and body detection)
  2. Start webcam or image capture
  3. For each frame, attempt detection. List of rectangles returned.
  4. Use largest “head” rectangle as the bounding box of the head in the frame.
  5. Use largest 2 “hand” rectangles as the bounding boxes of the left and right hands.
  6. Draw rectangles in the frame.
  7. Alter rotation of the robot arms and position based on head and hand position.


Image with hands and robot

This shows the robot moving to adjust to the detection of ther hands. The input is the image on the left and the robot output is on the right. The left hand detection is not acurate, however this image does show the robot accurately adjusting forthe hand's detection location.

This video shows the hand and face detection in real time. The hand-detection was not as great as face because it was not from OpenCV, but rather trained by someone else. Also, the webcam footage is mirrored from what is actually occuring.

This video shows the head detection with a bunny in OpenGL in real time. The webcam footage is mirrored from what is actually occuring.

Resources Used

Cascade Classifier Tutorial in OpenCV and C++. The tutorial helped me start off my project and familarize myself with OpenCV in C++.

Install OpenCV on Mac . This tutorial walked step by step on how to get OpenCV into a project using CMake on a Mac computer.