Formerly favorite* intern, Braedan Kennedy, has whipped up one of our favorite Slack apps yet. You Got Mailbot is a Slack app which posts a photo from the inside of our mailbox when we receive snail mail. As a bonus, You Got Mailbot uses AI to detect and post a snapshot of the first face it finds when the mailbox is opened.
You Got Mailbot posts a photo to Slack with the first face it detects when the mailbox is opened
You Got Mailbot is written in Python and runs on a Raspberry Pi Zero computer which, along with it’s tiny camera, is about as big as a pack of gum. The Raspberry Pi Zero connects to the network over wifi and starts recording video as soon as there is a spike in brightness when the mailbox is opened from the outside. The Pi Zero isn’t very powerful so it generally takes 3 – 6 minutes for it to process the video and run the face detection algorithms. Facial recognition is performed by the Raspberry Pi using Haar-cascade Detection in OpenCV. If a good face match is not found then the middle frame of the video clip is posted instead.
In addition to writing the code, Braedan designed and 3D printed a case and mount for a Raspberry Pi and camera which is mounted inside the mailbox facing the door. We ran a USB cable down into the mailbox to power the Pi.
Version 2 – The Artificial Intelligence Update
The Raspberry Pi Zero computer has many defining features, such as built in wifi/bluetooth capability and its minuscule form factor, which make it an ideal platform for projects such as You Got Mailbot. Unfortunately the computer is lacking in processing power and memory, both of which are necessary for performing Facial Recognition in real time. Because of this, sacrifices were made in terms of accuracy which resulted in You Got Mailbot not detecting faces very often.
(Face is clearly in the image, but the Haar-cascade missed it)
Facial Recognition using Artificial Intelligence (AI) and Machine Learning algorithms now become feasible options
The solution was to abandon the concept of real-time detection, because the processor cannot keep up, and implement a post-processing method. The camera now records video first and detects faces later. Post-processing allows for better detection accuracy in trade of performance losses since the facial detection does not have to happen in real time. Slower, and more computationally intensive methods of Facial Recognition using Artificial Intelligence (AI) and Machine Learning algorithms now become feasible options. You Got Mailbot v2 performs facial recognition through the use of a Neural Network that has been trained to detect faces with millions of sample images. It can now detect faces with certainty as high as 99.9999%!
(old method in blue, Neural Network in red)
Another benefit to using a Neural Network, besides incredible detection accuracy, is that it also returns the accuracy data (as shown in the figure above). The data can be used to sort the quality of the detected faces and upload the best image instead of uploading the first as done previously.
As with any project, there have been some amusing fails. Big boxes tend to block the camera or are too large to fit past the camera and need to be cut out 🙂
You can find the updated v2 source code to create your own You Got Mailbot in the Outside Open GitHub repo.
* Only
