The Little Computer that Could: Motion Detection with Outdoor Camera, Tensorflow and Raspberry Pi Part 3 of 3

 

"I have no spur to prick the sides of my intent, but only vaulting ambition, which o'erleaps itself and falls on the other." - Macbeth

Motion detection with an outdoor camera can be problematic, with wind, shadow and direct sunlight causing multiple false alarms. Suppressing these false alarms can result in genuine alarms being missed. One way is to filter all the alarms through an object recognition program.

The motion detection program of Part 1 will write all alarm image frames to the ./alarms/ directory. A modified object recognition program of Part 2 will inspect each alarm frame and if an object is recognized, will write it to another directory ./alarm/. The alarm filter program is called tiny-yolo_alarmfilter.py. This seemed to work well to start with, but with any new project, time will tell. For starters it seemed to think my dog is a cow, probably because she was sniffing at the grass.

tiny YOLO mislabelling dog as cow

Now my dog does have a temper, but calling her a cow is a little harsh. Even so, both dogs and cows qualify (in my opinion) as valid alarm triggers so it is not a show-stopper for now. It is particularly good at excluding changes in lighting and shadows. Proof positive that the little Raspberry Pi could, and did. 

Recognizing passing vehicles 

Ah, vaulting ambition ... if the Pi can recognize an object, maybe it can also track it. Most alarms are quite passive, except for the loud siren, which tends to annoy the neighbors and should only be used as a last resort. A pan/tilt camera like the Trendnet TV-IP422WN that visibly tracks the object is a lot more menacing, and should scare off the more timid intruders like birds and squirrels. But that is another blog post.

A tensorflow model looks promising, as it can be potentially speeded up with the use of custom hardware like the Coral USB acelerator for about the price of a Raspberry Pi 4.

Coral USB Accelerator

Installing tensorflow proved to be a bit hit and miss, but Katsuya Hyodo's github readme worked for me. This time I started with a squeaky clean version of Raspbian, 2021-05-07-raspios-buster-armhf.img.

Remember to uninstall the dud versions:

# pip3 uninstall tensorflow
# apt-get install -y libhdf5-dev libc-ares-dev libeigen3-dev gccgfortran libgfortran5 libatlas3-base libatlas-base-dev libopenblas-dev libopenblas-base libblas-dev liblapack-dev cython3 openmpi-bin libopenmpi-dev libatlas-base-dev python3-dev
# pip3 install pip --upgrade
# pip3 install keras_applications==1.0.8 --no-deps
# pip3 install keras_preprocessing==1.1.0 --no-deps
# pip3 install h5py==2.9.0
# pip3 install pybind11
# pip3 install -U --user six wheel mock
# wget "https://raw.githubusercontent.com/PINTO0309/Tensorflow-bin/master/tensorflow-1.15.0-cp37-cp37m-linux_armv7l_download.sh"
# sh ./tensorflow-1.15.0-cp37-cp37m-linux_armv7l_download.sh
# pip3 install tensorflow-1.15.0-cp37-cp37m-linux_armv7l.whl

A quick test:

# python3

Python 3.7.3 (default, Jan 22 2021, 20:04:44)

[GCC 8.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import tensorflow

>>> tensorflow.__version__

'1.15.0'

>>>

For object detection I used Edje Electronics. 

Packages tensorflow, libatlas-base-dev, libhdf5-dev, libhdf5-serial-dev I had already installed previously

# apt-get install libjasper-dev

# apt-get install libqtgui4

# apt-get install libqt4-test

I used version 4.4.0.46 because 4.1.0.25 could not be found

# pip3 install opencv-contrib-python==4.4.0.46

# apt-get install protobuf-compiler

# pip install --user Cython

# pip install --user contextlib2

# pip install --user pillow

# pip install --user lxml

# pip install --user matplotlib

Got the tensorflow models

# git clone https://github.com/tensorflow/models.git

Then SSD_Lite:

# wget http://download.tensorflow.org/models/object_detection/ssdlite_mobilenet_v2_c

oco_2018_05_09.tar.gz

# tar -xzvf ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz

The original tensorflow program, ./models/research/object_detection/TensorFlow.py got its images from the default Raspberry Pi camera, so I made a simpler version to take one frame (./341front.jpg in 640x480) at a time, uTensorFlow.py.

Note SSD_Lite misclassified a dog as sheep

The processing time was over 40s on my Raspberry Pi 3. My Coral USB accelerator will take more than a month to arrive, and it needs Tensorflow Lite, for which Edje Electronics has a very promising Tensorflow Lite repository, so why not. Notice this time the commands are as a sudoer user and not root, which I am told is the proper way to do things:

$ sudo pip3 install virtualenv

$ python3 -m venv tflite1-env

$ source tflite1-env/bin/activate

Then comes a whopper of a download:

$ git clone https://github.com/EdjeElectronics/TensorFlow-

Lite-Object-Detection-on-Android-and-Raspberry-Pi.git

$ bash get_pi_requirements.sh

Notice with Tensorflow Lite there is no Tensorflow module:

$ python3

Python 3.7.3 (default, Jan 22 2021, 20:04:44)

[GCC 8.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import tensorflow

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

ModuleNotFoundError: No module named 'tensorflow'

>>>

$ wget https://storage.googleapis.com/download.tensorflow.org/models/tflite/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip

$ unzip coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip

Archive:  coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip

  inflating: detect.tflite

  inflating: labelmap.txt

The original program is TFLite_detection_webcam.py but I need a version that works on individual image files, and not a video file, as in tiny-yolo_alarmfilter.py. The progam is tflite_alarmfilter.py, which took no time at all to write in python. You run it thus:

$ python3 tflite_alarmfilter.py --resolution='640x480' --modeldir=.

Processing alarm file 341front_20210804_094635_original.png

Processing alarm file 341front_20210804_094649_original.png

Processing alarm file 341front_20210804_094648_original.png

The framerate is now an astounding 2fps even without the Coral USB Accelerator. The detection seems slightly better with a more accurate bounding box without multiple boxes nested in the same object.

Tensorflow Lite SSDLite MobileNet v2: note accurate bounding box

tiny-YOLO: multiple bounding boxes over same object

I had originally planned to outsource the alarm files filtering to an x86 CPU, but the results with Tensorflow Lite made everything possible on the same Raspberry Pi 3, truly the little CPU that could. 

Happy Trails.