Motion Triggered Face Distortion

Abstract

  • We wanted to develop a video facial distortion program for entertainment purpose.
  • Our idea is inspired by smartphone application DouYin.
  • Our program would detect and track a face in image sequences. A selection of distortion can be applied onto the tracked facial area(s).
  • We have achieved our goals; our program is capable of tracking a facial area (largest facial area if multiple are present initially) throughout image sequence and apply one of four available distortions onto the facial area.
  • We have four distortion methods available to choose: sphere[1], gridline[2], wave[3], and rainbow[4]


    • Introduction

    Our goal is to design and implement a video facial distortion program for entertainment purpose. The program would take in a recorded and/or live video[5] feed as input. From the input video, the desired output will be our program would detect and track any face(s) throughout the image sequence. If the video presents multiple faces, the program would choose one face and apply certain facial distortions. If the initial chosen face is covered or disappears from the video frame, the program would re-apply facial distortion to the second target. Our idea is inspired by two smartphone applications, DouYin and Musical.ly; these two applications provide similar features as our proposed program. However, based on our preliminary test on these apps, we believe we can build an open source program to provide a similar or even better user experience through better facial detection and the inclusion of feature tracking algorithms.



    Teaser figure

    data structure


    Approach

  • Select a detected facial area as region of Interest (ROI).
  • Selected ROI is tracked from frame to frame.
  • If selected ROI is lost, a new ROI is selected.
    • data structure


    Qualitative Results

    Due to the nature of our project, it is hard to define and assess our results qualitatively. There is no real quality metrics for facial distortions other than personal preferences. The quality of facial detection is mainly based upon the imported library module (OpenCV) and our self-implemented image mask. A comparison of facial detection quality between different library modules would have no real relevance to our project and therefore is not included in our report.



    Conclusion and Future Work

    Our goal is to design and implement a video facial distortion program for entertainment purpose. From our demo, we showed that our implementation can successfully identifies human faces and distorts them according to the filters selected. Therefore, we have achieved our expectation. However, our design is far from perfect. There are many areas where we can improve; the first one is to have a better facial detection algorithm. In our current setup, the human face is tracked and shown inside a bounding box. To better detect actual facial region, we also implemented a mask that will better represent the curvature of a human facial region. However, this technique is not efficient in all scenarios; people don't have the same facial shape. Our facial detection also suffers when user are not facing the camera directly; if a person is turning away from the camera, our current facial detection sometimes cannot track consistently. User experiences can also need improvement. We can add additional “fun” features such as replacing user’s face with another face. Last, our current setup has no sound features, which will also be added in future works.

    Distortion Sample

    distortion_sample
    Distortion method: Sphere


    Reference

  • [1]. https://github.com/Tinker-S/SomeImageFilterWithPython
  • [2]. http://www.degeneratestate.org/posts/2016/Oct/23/image-processing-with-numpy/
  • [3]. http://blog.csdn.net/matrix_space/article/details/78345540
  • [4]. http://blog.csdn.net/matrix_space/article/details/72303014
  • [5]. https://pythonprogramming.net/haar-cascade-face-eye-detection-python-opencv-tutorial/