The Little Computer that Could: YOLO Object Recognition and Raspberry Pi Part 2 of 3

 

You Only Look Once

One cure for motion detector false alarms in Part 1 is object recognition; the alarm is only raised if the intruder is recognized. YOLO is fast, trendy (deep neural net!) and will run on a Raspberry Pi. arunponnusamy comes straight to the point.

I already had numpy and openCV from Part 1, so it is just

# wget https://pjreddie.com/media/files/yolov3.weights

Now this is a massive file and I needed a copper Ethernet connection to the Internet to download it. Next you download the zip file from arunponnusamy for the files yolo_opencv.py, yolov3.cfg, yolov3.txt and dog.jpg. Do not use wget on github. And simply run it:

# python3 yolo.py --image dog.jpg --config yolov3.cfg --weights yolov3.weights --classes yolov3.txt

If you get 'OutOfMemoryError' you will probably need to reboot your Pi.

The output should be:

I substituted a frame from my IP Camera, and sure enough it recognized a person correctly:

# python3 yolo.py --image alarm.png --config yolov3.cfg --weights yolov3.weights --classes yolov3.txt

Output of yolov3.weights model

For convenience, see my github repository for copies of arunponnusamy's code. It took my Raspberry Pi 3 some 50 seconds to process the image, which is a little long but would probably keep up with the motion detector triggers of a few frames per hour. But arunponnusamy's reference is Joseph Redmon, who also mentioned a tiny version of the pre-trained neural net model. You will also need to download his yolov3-tiny.cfg, which I got by cloning from his github.

git clone https://github.com/pjreddie/darknet

Joseph's executable is a C program, but the input files are the same so I simply used his tiny model files with arunponnusamy's python script:

$ python3 yolo.py --image alarm.png --config yolov3-tiny.cfg --weights yolov3-tiny.weights --classes yolov3.txt

Note I also reused arunponnusamy's yolov3.txt which is the dictionary of object names. Joseph's dictionary seems to be baked into his executable file. The command finishes in 5 seconds, which is 10 times better. The output bounding box is different, but it did recognize the object as a person.

Output of yolov3-tiny.weights model

The idea is to use YOLO to filter out the false alarms from the motion detection program from Part 1, In Part 3, we will look at tiny-YOLO, Tensorflow, and SSD_Lite. The Raspberry Pi is truly the little computer that could.

Happy Trails.

The Little Computer that Could: motion detection with OpenCV and Raspberry Pi Part 1 of 3

One of the silver linings with this pandemic lockdown is you get round to doing one or two thing you always meant to do. For me it is vision systems. I happened to be testing an lcd panel using my Raspberry Pi 3. Like that little engine, I think I can ...

As it neared the top of the grade, which had so discouraged the larger engines, it went more slowly. However, it still kept saying, "I—think—I—can, I—think—I—can." - The Little Engine That Could

OpenCV seems as good a starting point as any. My Raspberry Pi 3 did not have the most recent image, so your mileage may vary.

# cat /proc/version
Linux version 4.19.66-v7+ (dom@buildbot) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1253 SMP Thu Aug 15 11:49:46 BST 2019

# cat /etc/debian_version
9.13

Using jeremymorgan's instructions, 

# apt-get update
# apt-get upgrade

After a really long wait, it finished. If like me your downloads get interrupted you can resume using:

# dpkg --configure -a

I also had to tweak jeremymorgan's instructions a little:

# wget https://bootstrap.pypa.io/pip/3.5/get-pip.py

# python3 get-pip.py

But the next command failed:

# pip install opencv-contrib-python

-su: /usr/bin/pip: No such file or directory

Wait, I just installed pip!

# whereis pip

pip: /etc/pip.conf /usr/local/bin/pip /usr/local/bin/pip2.7 /usr/local/bin/pip3.5 /usr/share/man/man1/pip.1.gz

Ah, my python install usually seeks its commands at /usr/bin, so

# ln -s /usr/local/bin/pip /usr/bin/pip

Now it completes:

# pip install opencv-contrib-python

Requirement already satisfied: numpy>=1.12.1 in /usr/lib/python3/dist-packages (from opencv-contrib-python) (1.12.1)

Installing collected packages: opencv-contrib-python

Successfully installed opencv-contrib-python-4.1.1.26

I'll be needing someting to play video, so

# apt-get install mplayer

OpenCV seems to speak native C++, but python seems like a good idea to me, so:

# pip3 install opencv-python

The Raspberry Pi has a built-in camera interface and all the OpenCV worked examples seems to use it, but I did not have a camera handy. What I do have is my old Trendnet  TV-IP422WN IP Camera. It works with an ancient, rickety and totally unsafe version of Microsoft Explorer (ActiveX - eeek!) but with a little bit of luck, and help from Aswinth Raj got it to work in plain http:

http://192.168.10.30/cgi/mjpg/mjpg.cgi

2021-08-04 update: The Trendnet TV-IP422WN also supports rtsp:

rtsp://192.168.10.30/mpeg4

The program to test your non-Raspberry Pi video camera is here.

Now Aswinth Raj managed to stream video from his DVR with OpenCV using rtsp. It is not http, but worth a try. His code uses cvui, so in it goes:

# pip3 install cvui

But when you first run motion_capture.py, there are errors. You need:

# apt-get install libatlas-base-dev

Then the python module numpy throws an exception. For some mysterious reason this made it work:

# apt-get remove python-numpy

# apt-get remove python3-numpy

# pip uninstall numpy

# pip3 uninstall numpy

# pip3 install numpy

# apt-get autoremove

This caused a crash and CPU reboot, after which

# dpkg --configure -a

# apt-get autoremove

# pip3 install numpy

And now, work it does. Here is my http version.

Next is motion detection, by automaticaddison. He has working python code for two methods: by absolute difference and by background subtraction. Both work without fuss; I only modified it for my Trendnet IP camera instead of the default Raspberry Pi camera.

Note the dog is also detected but the program chose the biggest contour

Now by any reasonable measure either program would solve the problem of motion detection using Raspberry Pi and OpenCV. True, the frame rate was not brilliant at 1 fps but it would detect most intrusions unless the intruder flat-out sprinted across the camera field of view.

My problem was I had mounted my IP camera to look out on the front lawn which pointed the camera northeast. When run 24/7 the morning sun and the evening shadow of the house, coupled with wind blowing through the vegetation caused almost continous triggering. It did not matter if absolute difference or background subtraction was used. When motion_capture.py was modified to save the triggering images, it used up a few gigabytes every day.

One way to reduce false alarms is to mask out problem regions. I used the cruder image cropping, which does work but over 24 hours essentially my entire background changes a few times a day. 

Adjusting the image threshold, including adaptive thresholding made little difference. Next I took a Adrian Rosebrock's weighted average (cv2.accumulateWeighted) of a few consecutive frames, and while this helped the false alarms were still annoyingly high.

The next thing to try was to only raise an alarm on say 5 consecutive triggers. This caused some intrusions to be missed, particularly if the intruder moved quickly. But there were still too many false alarms. A similarly crude method was to put upper and lower limits on the size of the detected bounding rectangle (cv2.boundingRect)

I ended up using all the methods in one after the other, and maybe got a few false alarms per hour. It sort of works, but was a little error-prone (ie misses intrusions). The program is background_subtraction.py. One possible improvement is Ivan Kudriavtsev's take on motion detection. The installation of Numba looks a bit daunting, so I left it for another time.

In Part 2, we will investigate if object recognition will help.

There you have it: the little Raspberry Pi that could do motion detection using OpenCV. Happy Trails.