header photo

Level 3c - How To Improve Face Detection

QUESTION: "Face detection result is not up to my expectations/needs!  How do I improve the detection results?!"

Simple Solution:
  Is your problem something like: not all faces are being detected ? or the bigger, smaller, farther, a bit tilted/rotated faces are not detected?

You can solve this problem easily. The Solution includes to:
1- "Tune" the face detector to your needs by simply changing the parameter values in the call to DetectHaarCascade().Explained below.
2-Try various Haar Cascade XML files available for your object(face), you might find the best one that suits your requirements.  Try These XMLs for Face detection.

3-  Ensure GOOD image quality! For that, if possible,make sure you have web camera that gives GOOD quality images. the better the quality, the better the detection. OR you could apply image processing for improving image quality
4- Make sure the faces in the images are FRONTAL with as  little rotation or tilt as possible because
NOTE: if the face you're trying to detect is not even fitting into the image completely, then that's a NO-GO!  OR if the face is not a frontal face enough, i.e it is too much rotated or tilted, then you might want to align the face first(if possible): correct its rotation/tilt.


1. Improve Face Detection by Tuning The Detector's Parameters:
Mr. Robin Hewitt’s article: Seeing with OpenCV, proved to be my resource for learning how face detection works in OpenCV and thus in EmguCV. He has explained it SUPERBLY! You should read his complete article, it gives you an extensive know how of face detection in C++ using OpenCV.

However, we are here to learn face detection in C# using EmguCV and the concepts that apply to OpenCV also hold true for EmguCV , so I have arranged all the necessary things from his article, that you need to know, right here in this tutorial. With due respect, i will be quoting text from his article (in Maroon color). This way you can understand the concepts and use of of Haar Cascade classifier's parameters, from the ‘EmguCV and C#’ perspective.
“There are several parameters you can adjust to tune the face detector for your application”.

1.1- The Haar Cascade

I had mentioned in Level 3b - Implementing Face Detection that the first parameter in the call to DetectHaarCascade() is the XML file from which, trained data is loaded for the Haar classifier. There are several frontal face detector cascades in OpenCV (and EmguCV). The best choice for you will depend on your set up. It's easy to switch between them - just change the file name. Why not try each?

It's also possible to create your own, custom XML file using the HaarTraining application, in OpenCV's apps directory. Using that application is beyond the scope of this article.

DO NOT try training an XML if one already exists and is available for use over the internet. it'll waste your time and energy.

I use haarcascade_frontalface_alt_tree.xml because it had the best results, as far as I have tried, for frontal face detection: detects maximum possible faces in one go with least no. of false detections!


1.2- Scale Increase Rate

The second parameter in the call to DetectHaarCascade() specifies how quickly OpenCV should increase the scale for face detections with each pass it makes over an image. Setting this higher makes the detector run faster (by running fewer passes), but if it's too high, you may jump too quickly between scales and miss faces. The default in OpenCV is 1.1, in other words, scale increases by a factor of 1.1 (10%) each pass.

This parameter may have a value of 1.1 , 1.2 , 1.3 or 1.4.

I have set it to 1.4, i.e the highest: Meaning it will run the least number of passes, thus will be most fast BUT might miss out some faces! The lower the value, the more "thoroughly" haar detector will check the image for the "object"/"face", but naturally will take more time. decide which option you want more.

1.3- Minimum Neighbors Threshold

The third parameter in the call to DetectHaarCascade() is the ‘The minimum-neighbors threshold’ which sets the cutoff level for discarding or keeping rectangle groups as “face” or not, based on how many raw detections are in the group. This parameter’s value ranges from 0 to 4.

 “One of the things that happens "behind the scenes" when you call the face detector is that each positive face region actually generates many hits from the Haar detector. Figure 3 shows OpenCV's internal rectangle list for the example image, lena.jpg. The face region itself generates the largest cluster of rectangles. These largely overlap. In addition, there's one small detection to the (viewer's) left, and two larger detections slightly above and left of the main face cluster.  

Usually, isolated detections are false detections, so it makes sense to discard these. It also makes sense to somehow merge the multiple detections for each face region into a single detection. OpenCV does both these before returning its list of detected faces. The merge step first groups rectangles that contain a large amount of overlap, then finds the average rectangle for the group. It then replaces all rectangles in the group with the average rectangle.  

Between isolated rectangles and large groupings, are smaller groupings that may be faces, or may be false detections. The minimum-neighbors threshold sets the cutoff level for discarding or keeping rectangle groups based on how many raw detections are in the group. The C++ default for this parameter is three, which means to merge groups of three or more and discard groups with fewer rectangles. If you find that your face detector is missing a lot of faces, you might try lowering this threshold to two or one.  

If you set it to 0, OpenCV will return the complete list of raw detections from the Haar classifier. While you're tuning your face detector, it's helpful to do this just to see what's going on inside OpenCV. Viewing the raw detections will improve your intuition about the effects of changing other parameters, which will help you tune them.

Figure 3. OpenCV's internal detection rectangles. To see these, use min_neighbors = 0

I have used minimum neighbors = 4
i.e I want only an object to be marked as a face if it has the highest probability and vote of being the "object"(which is “face” in our case). put in another way, if i set minimum neighbor to a value n, then detector will mark an object as "face" in my image IF there is a group of n  rectangles (hits) identifying it as a "face" (see image above).
If you find that your face detector is missing a lot of faces, you might try lowering this threshold to two or one. 

Higher value = a stricter detection criterion!
lower value = a Linient detection criterion.

1.4- Canny Pruning Flag

The fourth parameter to DetectHaarCascade() is a flag variable. There are currently only two options: 0 or DO_CANNY_PRUNING.

If the Canny Pruning option is selected, the detector skips image regions that are unlikely to contain a face, reducing computational overhead and possibly eliminating some false detections. The regions to skip are identified by running an edge detector (the Canny edge detector) over the image before running the face detector.

Again, the choice of whether or not to set this flag is a tradeoff choice between speed and detecting more faces. Setting this flag speeds processing, but may cause you to miss some faces. In general, you can do well with it set, but if you're having difficulty detecting faces, clearing this flag may allow you to detect more reliably. Setting the minimum-neighbors threshold to 0 so you can view the raw detections will help you better gauge the effect of using Canny pruning.

1.5- Minimum Detection Scale

The fifth parameter in the call to DetectHaarCascade() is the size of the smallest face to search for. you can select the default for this by setting the scale to ,say, 25x25, as in our code.

“But what is the default? You can find out by opening the XML file you'll be using. Look for the <size> tag. In the default frontal face detector, it's
  <size> 24 24 </size>.
So, for this cascade, the default minimum scale is 24x24.

To set the minimum scale higher than the default value, set this parameter to the size you want.” In our code, 25 x 25 works ok, modify the value and see what happens.

“Depending on the image resolution you're using, this default size may be a very small portion of your overall image. A face image this small may not be meaningful or useful, and detecting it takes up CPU cycles you could use for other purposes. For these reasons, and also to minimize the number of face detections your own code needs to process, it's best to set the minimum detection scale only as small as you truly need.  

A good rule of thumb is to use some fraction of your input image's width or height as the minimum scale - for example, 1/4 of the image width. If you specify a minimum scale other than the default, be sure its aspect ratio (the ratio of width to height) is the same as the default's. i.e, aspect ratio should be 1:1.”

  PLEASE CHECK  THE SOLUTION HERE. - IF you still don't find your answer, then Please post your question as a comment in the following format:
"QUESTION: why is that....etc etc"


Tutorial by: Mahvish

Go Back

actually this tutorial amazing and great
I have learned too much from it
and it helped me too much
please keep it up .
we want more for eye detection please
you showed the steps in amazing way.

Very good tutorial. I am going to use the knowledge that i have gained here to do my project ;)

You done fantastic work here.
my final project related to this one(mouse controlling via face).thank you very much...

Miss Mahvish;
Thanks for your tutorials! I am Chinese.There are not books and tutorials about EMGUCV.My English is bad,But I want to say "Thanks" to you!

Excellent!! thank you!

Great tutorial, thanks!

Thanks for the tutorial,
its really helpful,

but I have a question,
Can we display all HAAR feature stage when it applied to a picture ?
I mean displaying the result all feature one by one,

I need it for my final project,

Thank you

hi Mahvish,
Your tutorial is awesome and useful.
my question is that i want to feed the form an image instead of a video stream

Thank you very much :)) very helpful !!