Optimus under the Hood: OpenCV with CUDA for Nvidia GT 640M GPU and Slackware 14.2

Optimus Prime stepping forth from laptop - AI-generated image from getimg.ai

 Never thought much about my laptop GPUs. Even less about Nvidia GPUs as I gave up on proprietary software 20 years ago. I was quite happy with the open source noveau driver, until Nvidia's cuDNN allowed OpenCV imaging programs to use Deep Neural Nets - AI.

Installing CUDA

Slowly, for it was a little cumbersome to hold your nose at the same time, I loaded the CUDA Linux toolkit into by GeForce GT710 desktop. The process was as unpleasant as ever - 10-year old proprietary software starts to look like abandonware, but the results were amazing. The GPU heated up like crazy and my desktop blew up, but OpenCV flew.

Acer Aspire M3-581TG

Suddenly there were low-cost possibilities for AI-enabled imaging systems - surveillance video, even augmented reality. And some of my old laptops (defenestrated, of course) had Nvidia GPUs. I started with an old Acer Aspire M3-581TG - it has an Nvidia GeForce 640M, or so the sticker on the keyboard says. 

lspci came up with a surprise - the GPU was an Intel GPU:

root@aspireM3:/$lspci

00:00.0 Host bridge: Intel Corporation 3rd Gen Core processor DRAM Controller (rev 09)

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)

00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)

00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)

00:16.0 Communication controller: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 (rev 04)

00:1a.0 USB controller: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 (rev 04)

00:1b.0 Audio device: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller (rev 04)

00:1c.0 PCI bridge: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 (rev c4)

00:1c.1 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 (rev c4)

00:1c.3 PCI bridge: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 4 (rev c4)

00:1d.0 USB controller: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 (rev 04)

00:1f.0 ISA bridge: Intel Corporation HM77 Express Chipset LPC Controller (rev 04)

00:1f.2 SATA controller: Intel Corporation 7 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)

00:1f.3 SMBus: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller (rev 04)

01:00.0 VGA compatible controller: NVIDIA Corporation GK107M [GeForce GT 640M] (rev a1)

07:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader (rev 01)

0d:00.0 Network controller: Qualcomm Atheros AR9462 Wireless Network Adapter (rev 01)

0e:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetLink BCM57780 Gigabit Ethernet PCIe (rev 01)

Now if I had read all the lines instead of stopping after the first 3, I would have noticed it also had an Nvidia GPU! the GK107M or GeForce GT 640M. It took quite a few weeks to recover from the shock - two GPUs in a laptop? The GPUs were switched in and out depending on whether graphics performance or power consumption was being prioritised. Nvidia called this its Optimus system.

GPU Switching

Now the GT 640M is quite an old GPU, and the best way would be to install CUDA/cuDNN/OpenCV on a matching Ubuntu distribution. But my M3-581TG had been defenestrated 10 years ago. It ran Slackware 14.2-current and was too much work on it to install new. 

Nvidia GPU, CUDA Toolkit, cuDNN and OpenCV are notoriously finicky and you need to get the versions just right. Not to mention your gcc, libraries and various Linux bits. CUDA and cuDNN are proprietary blobs so it is a matter of installing the various versions until one works. The first thing to do is to go past the Nvidia marketing guff and find out the GT 640M's GPU architecture. Its real name is the GK107 and the architecture is Kepler.

The you need to find the the GK107's Compute Capability, which from Nvidia is 3.0. From the cuDNN Support Matrix, the chances of it working with cuDNN 7.6.4, CUDA 10.1.243 and Linux driver at least r418.39 seems promising.

 First the driver. I started with slackbuild version, r460.67. Normally, you do a slackbuild with the Nvidia blob, but I had good results with Nvidia installer with the GT 710 so I downloaded it from Nvidia and ran it directly: 

#sh NVIDIA-Linux-x86_64-460.67.run

Now if you selected the dkms option the installer will fail and you will need to slackbuild dkms first.

$sh ./dkms.SlackBuild

$upgradepkg --install-new /tmp/dkms-2.8.4-x86_64-1_SBo.tgz

After which it needs to be run as a service, so

$vi /etc/rc.d/rc.modules.local

# Enable DKMS module rebuilding

if [ -x /usr/lib/dkms/dkms_autoinstaller ]; then

  echo "Running DKMS autoinstaller"

  /usr/lib/dkms/dkms_autoinstaller start

fi

dkms may result in build errors so in the end I deselected it. After the installer finished the original nouveau driver was blacklisted and the Nvidia driver loaded but my X windows would not start. It turned out I first need to lspci for the GPU bus number:

# lspci

01:00.0 VGA compatible controller: NVIDIA Corporation GK107M [GeForce GT 640M] (rev a1)

And enter it into a new xorg.conf:

# cat /etc/X11/xorg.conf

Section "Module"

    Load "modesetting"

EndSection

Section "Device"

    Identifier     "Device0"

    Driver "nvidia"

    BusID "PCI:1:0:0"

    Option "AllowEmptyInitialConfiguration"

EndSection

With X up, check the loaded driver:

$nvidia-smi

Sat Jun  8 21:53:52 2024

+-----------------------------------------------------------------------------+

| NVIDIA-SMI 460.67       Driver Version: 460.67       CUDA Version: 11.2     |

|-------------------------------+----------------------+----------------------+

| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |

| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |

|                               |                      |               MIG M. |

|===============================+======================+======================|

|   0  GeForce GT 640M     Off  | 00000000:01:00.0 N/A |                  N/A |

| N/A   62C    P8    N/A /  N/A |    149MiB /   981MiB |     N/A      Default |

|                               |                      |                  N/A |

+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+

| Processes:                                                                  |

|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |

|        ID   ID                                                   Usage      |

|=============================================================================|

|  No running processes found                                                 |

+-----------------------------------------------------------------------------+

Next is CUDA. The cuDNN compatibility matrix says 10.1.243 but I had good luck with CUDA 10.2.89 and it was very close to 10.1.243 so

$sh ./cuda_10.2.89_440.33.01_linux.run

Note I took care not to install the included GPU driver as I already had a working 460.67.

After that you will need to include the CUDA path ion your bash profile:

$cat ~/.bash_profile

PATH=$HOME/utils:/usr/local/cuda-10.2/bin:$PATH

export PS1="\u@\h:\w\$"

To test, there is a neat little test program, and:

$nvcc -o check_cuda check_cuda.c -lcuda

$./check_cuda

Found 1 device(s).

Device: 0

  Name: GeForce GT 640M

  Compute Capability: 3.0

  Multiprocessors: 2

  Concurrent threads: 4096

  GPU clock: 708.5 MHz

  Memory clock: 900 MHz

  Total Memory: 981 MiB

  Free Memory: 723 MiB

Next is cuDNN and from slackbuild is version to use is 8.0 but that did not work out with OpenCV so I dialed it down a notch to cuDNN 7.6.5. This time I went with slackbuild with a few mods to get it to work:

$cp cudnn.SlackBuild cudnn.SlackBuild-v8.0_11.0

$cat cudnn.SlackBuild

PRGNAM=cudnn

VERSION=${VERSION:-v7.6_10.2}

BUILD=${BUILD:-1}

TAG=${TAG:-_SBo}

CUDNN_VERSION=${VERSION%_*}

CUDA_VERSION=${VERSION#*_}

$ln -s cudnn-10.2-linux-x64-v7.6.5.32.tgz cudnn-10.2-linux-x64-v7.6.tgz

$./cudnn.SlackBuild

cuda/include/cudnn.h

cuda/NVIDIA_SLA_cuDNN_Support.txt

cuda/lib64/libcudnn.so

cuda/lib64/libcudnn.so.7

cuda/lib64/libcudnn.so.7.6.5

cuda/lib64/libcudnn_static.a

Slackware package /tmp/cudnn-v7.6_10.2-x86_64-1_SBo.tgz created.

$upgradepkg --install-new /tmp/cudnn-v7.6_10.2-x86_64-1_SBo.tgz

We have suffered losses, but we will install OpenCV ...

Next  is the biggie, OpenCV. This usually means lots of iterations. Amos Stailey-Young's page is a good place to start. What worked for me is OpenCV 4.3.0 and opencv_contrib 4.3.0. Untar them into their respective subdirectories.

The cmake is:

heong@aspireM3:~/cuda/opencv/build$cmake -D CUDA_NVCC_FLAGS="-D_FORCE_INLINES -gencode=arch=
compute_35,code=sm_35" -D CMAKE_BUILD_TYPE=RELEASE -D OPENCV_GENERATE_PKGCONFIG=ON -DBUILD_SHARED_LIBS=OFF -D CMAKE_INSTALL_PREFIX=/usr/local -D INSTALL_C_EXAMPLES=OFF -D BUILD_TESTS=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_EXAMPLES=OFF -D WITH_OPENEXR=OFF -D WITH_CUDA=ON -D WITH_CUBLAS=ON -D WITH_CUDNN=ON -D CUDA_ARCH_BIN=3.0 -D OPENCV_DNN_CUDA=ON -D OPENCV_EXTRA_MODULES_PATH=~/cuda/opencv/opencv_contrib-4.3.0/modules -D LDFLAGS="-pthread -lpthread" -D CUDNN_VERSION="7.6" ~/cuda/opencv/opencv-4.3.0/

Note the use of the Compute Capability number. cuDNN version number has to be explicitly specified as the cmake persistently fails to extract the cuDNN version number from its include files.

Then it is 

$make -j 4

and then

$su -c "make install"

 And seemed to have resulted in 2 files:

root@aspireM3:/$ls -lh /usr/local/lib/python3.6/site-packages/cv2/python-3.6

total 255M

-rwxr-xr-x 1 root root 255M Jun 16 22:58 cv2.cpython-36m-x86_64-linux-gnu.so

root@aspireM3:/$ls -lh /usr/local/lib/python2.7/site-packages/cv2/python-2.7

total 255M

-rwxr-xr-x 1 root root 255M Jun 16 22:57 cv2.so

And I simply did

$ln -s /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.cpython-36m-x86_64-linux-gnu.so /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.so

$export PYTHONPATH="/usr/local/lib/python3.6/site-packages/cv2/python-3.6/"

A very quick test is

$python3

Python 3.6.8 (default, Jan 13 2019, 13:36:07) 

[GCC 8.2.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import cv2

>>> 

Amos Stailey-Young's sample code did not work for me, but sr6033's code is very similar and worked well.

$python3 detect_faces_video.py  --prototxt prototxt.txt --model res10_300x300_ssd_iter_140000.caffemodel

[INFO] loading model...

[INFO] starting video stream...

[ WARN:0] global /home/heong/cuda/opencv/opencv-4.3.0/modules/videoio/src/cap_gstreamer.cpp

(935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, dura

tion=-1

For python2:

$export PYTHONPATH="/usr/local/lib/python2.7/site-packages/cv2/python-2.7/"

heong@aspireM3:~/cuda/opencv/build$python

Python 2.7.15 (default, Jun 17 2018, 22:57:51) 

[GCC 7.3.0] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import cv2

>>> 

"No sacrifice, no victory ..."

And there you have it: OpenCV 4.3.0 with CUDA 10.2.89 and cuDNN 7.6.5 running on the Nvidia GT 640M of an ancient Aspire M3-581TG laptop. Maybe my next laptop will have an Nvidia GPU with 8GB RAM ... what was it that Optimus Prime said? "Hang on to your dreams, Chip. The future is built on dreams."

The Littlest Computer that Could: OpenCV using Raspberry Pi 1 Model B Rev 2

 

First there was the 1B: Raspberry Pi One Model B.

Now out of production, the Raspberry Pi One Model B was released in 2012, earlier than the Model A. I had a few lying unused in my parts box, some damaged by defective power supplies, but mostly superseded by better versions like the Pi 2, 3 and 4s. The Pi 1 Model B was the slowest, had only 512MB DRAM and used the sdcard as mass storage. The USB functionality was questionable: the LAN chip was internally routed through the USB bus which crippled its throughput. And since the Pi was always touchy about its 5V input power adding basic functionality like keyboard, hdmi USB was always a hit and miss affair.

Left: sdcard, right: micro-sdcard

The sdcard is getting very hard to find. You find micro-sdcard with an adapter but these tend to develop contact problems and corrupt the onboard filesystem. To make matters worse, a Pi Model B which cannot boot will have no indication: there is just that red power LED on and nothing else, and it looks pretty much like a dead Pi. Some 60% of my discarded Pi 1s simply had microsd adapter contact problems and could not boot.

On the plus side, the Pi 1 drew the least power amongst the Pi series which meant most old Android phone chargers could power it. It also had audio and video jacks, which were very handy with retro electronics. 

I managed put one to use monitoring my solar panel, but most of the little jobs are better served by the ESP8266 or the Microchip PIC. If only the Raspberry Pi Model B could run OpenCV; with a bit of nifty image processing, it might find a use, perhaps to check if my front gate has been left open, or the air conditioner left running, or if the smoke alarm is beeping. 

Most of the time, the Pi 3 is the minimum recommended model, but there is no mention Raspberry Pi One cannot be used. No harm trying; and since the install process can be left alone, it is easily done on the side. 

Raspberry Pi 1 Model B Rev 2 with infamous microsd adapter

$dd if=2021-10-30-raspios-bullseye-armhf-lite.img of=/dev/sdc

After the standard install of Raspbian, I use raspi-config to turn on the ssh server and set a fixed ethernetIP. It can then be used as a headless (ie no monitor or keyboard) system via ssh from a host laptop or desktop. After which there is the usual obligatory

# apt-get update --allow-releaseinfo-change
# apt-get upgrade

And the Pi model:

root@pi:~# cat /sys/firmware/devicetree/base/model
Raspberry Pi Model B Rev 2

Jeremy Morgan's OpenCV install worked for my Pi 3 before, but now instead it stops:

# pip install opencv-contrib-python
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting opencv-contrib-python
Downloading opencv-contrib-python-4.5.4.60.tar.gz (150.7 MB)
|��������������������������������| 150.4 MB 93 kB/s eta 0:00:04Killed

From 'dmesg -T' it looks like I ran out of memory:

[Tue Nov 23 18:03:30 2021] [ 5714]     0  5714    87141    21594     104       0

    21597             0 pip

[Tue Nov 23 18:03:30 2021] Out of memory: Kill process 5714 (pip) score 349 or sacrifice child

[Tue Nov 23 18:03:30 2021] Killed process 5714 (pip) total-vm:348564kB, anon-rss

:86376kB, file-rss:0kB, shmem-rss:0kB

[Tue Nov 23 18:03:30 2021] oom_reaper: reaped process 5714 (pip), now anon-rss:0

kB, file-rss:0kB, shmem-rss:0kB

My free memory is:

# free -m

              total        used        free      shared  buff/cache   available

Mem:            369          18         285           0          65         304

Swap:         15358          21       15337

And I can get a litte more by editing the boot partition's config.txt:

# vi /boot/config.txt

Add:

gpu_mem=16

And comment out

#start_x=1

After a reboot I get more memory:

# free -m

              total        used        free      shared  buff/cache   available

Mem:            477          31         338           6         106         390

Swap:            99           0          99

 

But this is still not enough. Now I could increase the swap file in my micro sdcard, but the thrashing might wear it out as the number of write operations is limited. Instead I used one of the many ancient thumbdrives, discarded just because of their low capacities. I ended up using a compactflash card for its speed:

# dd if=/dev/zero of=/dev/sda bs=1M count=1024

# mkswap /dev/sda

# swapon /dev/sda

# pip install --upgrade pip setuptools wheel

# python -m pip install --upgrade pip

# pip3 install opencv-contrib-python

pip seems to have gone walkabout so,

#  ln -s /usr/local/bin/pip /usr/bin/pip

Failure:

# pip install opencv-contrib-python

    File "setup.py", line 381, in _classify_installed_files_override

      with open(os.path.join(cmake_install_dir, "python", "cv2", "__init__.py"),

 'r') as opencv_init:

  FileNotFoundError: [Errno 2] No such file or directory: '_skbuild/linux-armv6l

-3.7/cmake-install/python/cv2/__init__.py'

  ----------------------------------------

  ERROR: Failed building wheel for opencv-contrib-python

Failed to build opencv-contrib-python

ERROR: Could not build wheels for opencv-contrib-python, which is required to in

stall pyproject.toml-based projects

Vishwesh Shrimali's instructions seem promising, and his minimum requirement is for a Pi 2. There are more separate bash commands which increases the chances for a successful debug. Since a fail is near certain, I chose to key in the commands manually instead of running Shrimali's script.

root@pi:/root/opencv# apt-get -y purge wolfram-engine

root@pi:/root/opencv# apt-get -y purge libreoffice*

root@pi:/root/opencv# apt-get -y clean

root@pi:/root/opencv# apt-get -y autoremove

root@pi:/root/opencv# apt -y update

root@pi:/root/opencv# apt -y upgrade

root@pi:/root/opencv# apt-get -y remove x264 libx264-dev

root@pi:/root/opencv# apt-get -y install build-essential checkinstall

 cmake pkg-config yasm

root@pi:/root/opencv# apt-get -y install git gfortran

root@pi:/root/opencv# apt-get -y install libjpeg8-dev libjasper-dev libpng12-dev

root@pi:/root/opencv# apt-get -y install libtiff5-dev

root@pi:/root/opencv# apt-get -y install libtiff-dev

root@pi:/root/opencv# apt-get -y install libxine2-dev libv4l-dev

root@pi:/root/opencv# cd /usr/include/linux

root@pi:/usr/include/linux# ln -s -f ../libv4l1-videodev.h videodev.h

root@pi:/usr/include/linux# cd $cwd

root@pi:/root/opencv#

root@pi:/root/opencv# apt-get -y install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev

root@pi:/root/opencv# apt-get -y install libgtk2.0-dev libtbb-dev qt5-default                         

root@pi:/root/opencv# apt-get -y install libatlas-base-dev

root@pi:/root/opencv# apt-get -y install libmp3lame-dev libtheora-dev

root@pi:/root/opencv# apt-get -y install libvorbis-dev libxvidcore-dev libx264-dev

root@pi:/root/opencv# apt-get -y install libopencore-amrnb-dev libopencore-amrwb-dev 

root@pi:/root/opencv# apt-get -y install libavresample-dev

root@pi:/root/opencv# apt-get -y install x264 v4l-utils

            The following are optional:

root@pi:/root/opencv# apt-get -y install libprotobuf-dev protobuf-compiler

root@pi:/root/opencv# apt-get -y install libgoogle-glog-dev libgflags-dev 

root@pi:/root/opencv# apt-get -y install libgphoto2-dev libeigen3-dev libhdf5-dev doxygen

            Required python libraries

root@pi:/root/opencv# apt-get -y install python3-dev python3-pip

            Virtual environment:

fred@pi:~/opencv $ python3 -m venv OpenCV-4.0-py3

fred@pi:~/opencv $ echo "# Virtual Environment Wrapper" >> ~/.bashrc

fred@pi:~/opencv $ echo "alias workoncv-4.0=\"source /root/opencv/OpenCV-4.0-py3/bin/activate\"" >> ~/.bashrc                                       

fred@pi:~/opencv $ source /root/opencv/OpenCV-4.0-py3/bin/activate(OpenCV-4.0-py3) fred@pi:~/opencv $

 Increase the swap file from 100 to 1024:

(OpenCV-4.0-py3) fred@pi:~/opencv $ sudo sed -i 's/CONF_SWAPSIZE=100/CONF_SWAPSIZE=1024/g' /etc/dphys-swapfile

(OpenCV-4.0-py3) fred@pi:~/opencv $ sudo /etc/init.d/dphys-swapfile stop

[ ok ] Stopping dphys-swapfile (via systemctl): dphys-swapfile.service.      

(OpenCV-4.0-py3) fred@pi:~/opencv $ sudo /etc/init.d/dphys-swapfile start

[ ok ] Starting dphys-swapfile (via systemctl): dphys-swapfile.service.         

(OpenCV-4.0-py3) fred@pi:~/opencv $ pip install numpy dlib

(OpenCV-4.0-py3) fred@pi:~/opencv $ deactivate

This is an over 400MB porker of a file:

fred@pi:~/opencv $ git clone https://github.com/opencv/opencv.git

In retrospect I should have checked out 4.0.1 as having cv2.drawKeypoints() would have been handy.

fred@pi:~/opencv/opencv $ git checkout 4.0.0

fred@pi:~/opencv $ git clone https://github.com/opencv/opencv_contrib.git

fred@pi:~/opencv $ cd opencv_contrib

fred@pi:~/opencv/opencv_contrib $ git checkout 4.0.0

Then comes the config:

fred@pi:~/opencv/opencv/build $ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE

_INSTALL_PREFIX=/home/heong/opencv/installation/OpenCV-4.0 -D INSTALL_C_EXAMPLES

=ON -D INSTALL_PYTHON_EXAMPLES=ON -D WITH_TBB=ON -D WITH_V4L=ON -D OPENCV_PYTHON3_INSTALL_PATH=/home/heong/opencv/OpenCV-4.0-py3/lib/python3.5/site-packages -D WITH_QT=ON -D WITH_OPENGL=ON  -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules -D BUILD_EXAMPLES=ON ..

Now for the make:

fred@pi:~/opencv/opencv/build $ CMAKE_INSTALL_PREFIX=/usr/local

fred@pi:~/opencv/opencv/build $ export CMAKE_INSTALL_PREFIX

fred@pi:~/opencv/opencv/build $ make

Produces the error

In file included from /home/fred/opencv/opencv_contrib/modules/cvv/src/qtutil/f

ilter/sobelfilterwidget.cpp:3:

/home/heong/opencv/opencv/modules/imgproc/include/opencv2/imgproc.hpp:208:5: not

e:   �FILTER_SCHARR�

     FILTER_SCHARR = -1

     ^~~~~~~~~~~~~

make[2]: *** [modules/cvv/CMakeFiles/opencv_cvv.dir/build.make:453: modules/cvv/

CMakeFiles/opencv_cvv.dir/src/qtutil/filter/sobelfilterwidget.cpp.o] Error 1

make[1]: *** [CMakeFiles/Makefile2:12676: modules/cvv/CMakeFiles/opencv_cvv.dir/

all] Error 2

make: *** [Makefile:163: all] Error 2

From Nobuo Tsukamoto, in the file /home/fred/opencv/opencv_contrib/modules/cvv/src/qtutil/f

ilter/sobelfilterwidget.cpp added 'using namespace cv;' at line 13 thus:

#include "../../util/util.hpp"

#include "../filterfunctionwidget.hpp"

#include "../filterselectorwidget.hpp"

using namespace cv; // cmheong 2021-11-29

namespace cvv

{

namespace qtutil

{

SobelFilterWidget::SobelFilterWidget(QWidget *parent)

After which

fred@pi:~/opencv/opencv/build $ make

Produces the error

Scanning dependencies of target example_cpp_detect_mser

[ 89%] Building CXX object samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o

/home/fred/opencv/opencv/samples/cpp/detect_mser.cpp:28:10: fatal error: GL/glu.h: No such file or directory

 #include <GL/glu.h>

          ^~~~~~~~~~

compilation terminated.

make[2]: *** [samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/build.make:63:

samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o] Error 1

make[1]: *** [CMakeFiles/Makefile2:29519: samples/cpp/CMakeFiles/example_cpp_det

ect_mser.dir/all] Error 2

make: *** [Makefile:163: all] Error 2

From RajkiranVeldur, just do

root@pi:~#  apt-get install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev

After which

fred@pi:~/opencv/opencv/build $ make

Produces the link error

[ 89%] Linking CXX executable ../../bin/example_cpp_detect_mser

/usr/bin/ld: CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o: in functi

on `draw(void*)':

detect_mser.cpp:(.text.startup.main+0x1c70): undefined reference to `gluPerspect

ive'

collect2: error: ld returned 1 exit status

make[2]: *** [samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/build.make:134:

 bin/example_cpp_detect_mser] Error 1

make[1]: *** [CMakeFiles/Makefile2:29519: samples/cpp/CMakeFiles/example_cpp_det

ect_mser.dir/all] Error 2

make: *** [Makefile:163: all] Error 2

regpa mentioned that I need openGLU.so, but a brute-force search could not come up with one:

fred@pi:~/opencv/opencv/build $ sudo ls -lR / | grep -e openGLU 

fred@pi:~/opencv/opencv/build $

There is however, a file called libGLU.so mentioned by myinternetofthings:

fred@pi:~/opencv/opencv/build $ sudo ls -lR / 2>/dev/null | grep -e GLU.so

lrwxrwxrwx  1 root root       15 Sep 20  2015 libGLU.so -> libGLU.so.1.3.1

lrwxrwxrwx  1 root root       15 Sep 20  2015 libGLU.so.1 -> libGLU.so.1.3.1

-rw-r--r--  1 root root   358228 Sep 20  2015 libGLU.so.1.3.1

Added it to 2 separate files link.txt:

fred@pi:~/opencv/opencv/build $ cat samples/opengl/CMakeFiles/example_open

gl_opengl.dir/link.txt

/usr/bin/c++     -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-

dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wm

issing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -W

uninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comme

nt -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthr

ead -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee

 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG    -Wl,-

-gc-sections   CMakeFiles/example_opengl_opengl.dir/opengl.cpp.o  -o ../../bin/e

xample_opengl_opengl  -Wl,-rpath,/home/heong/opencv/opencv/build/lib -ldl -lm -l

pthread -lrt /usr/lib/arm-linux-gnueabihf/libGL.so ../../lib/libopencv_highgui.s

o.4.0.0 ../../lib/libopencv_videoio.so.4.0.0 ../../lib/libopencv_imgcodecs.so.4.

0.0 ../../lib/libopencv_imgproc.so.4.0.0 ../../lib/libopencv_core.so.4.0.0 /usr/

lib/arm-linux-gnueabihf/libGLU.so

fred@pi:~/opencv/opencv/build $ cat samples/cpp/CMakeFiles/example_cpp_det

ect_mser.dir/link.txt

/usr/bin/c++     -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-

dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wm

issing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -W

uninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comme

nt -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthr

ead -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee

 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG    -Wl,-

-gc-sections   CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o  -o ../.

./bin/example_cpp_detect_mser  -Wl,-rpath,/home/heong/opencv/opencv/build/lib -l

dl -lm -lpthread -lrt /usr/lib/arm-linux-gnueabihf/libGL.so ../../lib/libopencv_

gapi.so.4.0.0 ../../lib/libopencv_stitching.so.4.0.0 ../../lib/libopencv_aruco.s

o.4.0.0 ../../lib/libopencv_bgsegm.so.4.0.0 ../../lib/libopencv_bioinspired.so.4

.0.0 ../../lib/libopencv_ccalib.so.4.0.0 ../../lib/libopencv_cvv.so.4.0.0 ../../

lib/libopencv_dnn_objdetect.so.4.0.0 ../../lib/libopencv_dpm.so.4.0.0 ../../lib/

libopencv_face.so.4.0.0 ../../lib/libopencv_freetype.so.4.0.0 ../../lib/libopenc

v_fuzzy.so.4.0.0 ../../lib/libopencv_hdf.so.4.0.0 ../../lib/libopencv_hfs.so.4.0

.0 ../../lib/libopencv_img_hash.so.4.0.0 ../../lib/libopencv_line_descriptor.so.

4.0.0 ../../lib/libopencv_reg.so.4.0.0 ../../lib/libopencv_rgbd.so.4.0.0 ../../l

ib/libopencv_saliency.so.4.0.0 ../../lib/libopencv_sfm.so.4.0.0 ../../lib/libope

ncv_stereo.so.4.0.0 ../../lib/libopencv_structured_light.so.4.0.0 ../../lib/libo

pencv_superres.so.4.0.0 ../../lib/libopencv_surface_matching.so.4.0.0 ../../lib/

libopencv_tracking.so.4.0.0 ../../lib/libopencv_videostab.so.4.0.0 ../../lib/lib

opencv_xfeatures2d.so.4.0.0 ../../lib/libopencv_xobjdetect.so.4.0.0 ../../lib/li

bopencv_xphoto.so.4.0.0 ../../lib/libopencv_shape.so.4.0.0 ../../lib/libopencv_p

hase_unwrapping.so.4.0.0 ../../lib/libopencv_optflow.so.4.0.0 ../../lib/libopenc

v_ximgproc.so.4.0.0 ../../lib/libopencv_datasets.so.4.0.0 ../../lib/libopencv_pl

ot.so.4.0.0 ../../lib/libopencv_text.so.4.0.0 ../../lib/libopencv_ml.so.4.0.0 ..

/../lib/libopencv_dnn.so.4.0.0 ../../lib/libopencv_video.so.4.0.0 ../../lib/libo

pencv_photo.so.4.0.0 ../../lib/libopencv_objdetect.so.4.0.0 ../../lib/libopencv_

calib3d.so.4.0.0 ../../lib/libopencv_features2d.so.4.0.0 ../../lib/libopencv_fla

nn.so.4.0.0 ../../lib/libopencv_highgui.so.4.0.0 ../../lib/libopencv_videoio.so.

4.0.0 ../../lib/libopencv_imgcodecs.so.4.0.0 ../../lib/libopencv_imgproc.so.4.0.

0 ../../lib/libopencv_core.so.4.0.0 /usr/lib/arm-linux-gnueabihf/libGLU.so

After which

fred@pi:~/opencv/opencv/build $ make

Completes successfully. But there is still the installation. Set the environment variable CMAKE_INSTALL_PREFIX so that:

fred@pi:~/opencv/opencv/build $ echo $CMAKE_INSTALL_PREFIX

/usr/local

fred@pi:~/opencv/opencv/build $ sudo make install

And a quick test run:

fred@pi:~/opencv/opencv/build $ source /home/fred/opencv/OpenCV-4.0-py3/bin/activate

(OpenCV-4.0-py3) fred@pi:~/opencv/opencv/build $ python

Python 3.7.3 (default, Jan 22 2021, 20:04:44)

[GCC 8.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import cv2

>>> print(cv2.__version__)

4.0.0

>>> quit()

And there you have it, Raspberry Pi 1 Model B Rev 2, the littlest computer that could OpenCV.

The Little Computer that Could: motion detection with OpenCV and Raspberry Pi Part 1 of 3

One of the silver linings with this pandemic lockdown is you get round to doing one or two thing you always meant to do. For me it is vision systems. I happened to be testing an lcd panel using my Raspberry Pi 3. Like that little engine, I think I can ...

As it neared the top of the grade, which had so discouraged the larger engines, it went more slowly. However, it still kept saying, "I—think—I—can, I—think—I—can." - The Little Engine That Could

OpenCV seems as good a starting point as any. My Raspberry Pi 3 did not have the most recent image, so your mileage may vary.

# cat /proc/version
Linux version 4.19.66-v7+ (dom@buildbot) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1253 SMP Thu Aug 15 11:49:46 BST 2019

# cat /etc/debian_version
9.13

Using jeremymorgan's instructions, 

# apt-get update
# apt-get upgrade

After a really long wait, it finished. If like me your downloads get interrupted you can resume using:

# dpkg --configure -a

I also had to tweak jeremymorgan's instructions a little:

# wget https://bootstrap.pypa.io/pip/3.5/get-pip.py

# python3 get-pip.py

But the next command failed:

# pip install opencv-contrib-python

-su: /usr/bin/pip: No such file or directory

Wait, I just installed pip!

# whereis pip

pip: /etc/pip.conf /usr/local/bin/pip /usr/local/bin/pip2.7 /usr/local/bin/pip3.5 /usr/share/man/man1/pip.1.gz

Ah, my python install usually seeks its commands at /usr/bin, so

# ln -s /usr/local/bin/pip /usr/bin/pip

Now it completes:

# pip install opencv-contrib-python

Requirement already satisfied: numpy>=1.12.1 in /usr/lib/python3/dist-packages (from opencv-contrib-python) (1.12.1)

Installing collected packages: opencv-contrib-python

Successfully installed opencv-contrib-python-4.1.1.26

I'll be needing someting to play video, so

# apt-get install mplayer

OpenCV seems to speak native C++, but python seems like a good idea to me, so:

# pip3 install opencv-python

The Raspberry Pi has a built-in camera interface and all the OpenCV worked examples seems to use it, but I did not have a camera handy. What I do have is my old Trendnet  TV-IP422WN IP Camera. It works with an ancient, rickety and totally unsafe version of Microsoft Explorer (ActiveX - eeek!) but with a little bit of luck, and help from Aswinth Raj got it to work in plain http:

http://192.168.10.30/cgi/mjpg/mjpg.cgi

2021-08-04 update: The Trendnet TV-IP422WN also supports rtsp:

rtsp://192.168.10.30/mpeg4

The program to test your non-Raspberry Pi video camera is here.

Now Aswinth Raj managed to stream video from his DVR with OpenCV using rtsp. It is not http, but worth a try. His code uses cvui, so in it goes:

# pip3 install cvui

But when you first run motion_capture.py, there are errors. You need:

# apt-get install libatlas-base-dev

Then the python module numpy throws an exception. For some mysterious reason this made it work:

# apt-get remove python-numpy

# apt-get remove python3-numpy

# pip uninstall numpy

# pip3 uninstall numpy

# pip3 install numpy

# apt-get autoremove

This caused a crash and CPU reboot, after which

# dpkg --configure -a

# apt-get autoremove

# pip3 install numpy

And now, work it does. Here is my http version.

Next is motion detection, by automaticaddison. He has working python code for two methods: by absolute difference and by background subtraction. Both work without fuss; I only modified it for my Trendnet IP camera instead of the default Raspberry Pi camera.

Note the dog is also detected but the program chose the biggest contour

Now by any reasonable measure either program would solve the problem of motion detection using Raspberry Pi and OpenCV. True, the frame rate was not brilliant at 1 fps but it would detect most intrusions unless the intruder flat-out sprinted across the camera field of view.

My problem was I had mounted my IP camera to look out on the front lawn which pointed the camera northeast. When run 24/7 the morning sun and the evening shadow of the house, coupled with wind blowing through the vegetation caused almost continous triggering. It did not matter if absolute difference or background subtraction was used. When motion_capture.py was modified to save the triggering images, it used up a few gigabytes every day.

One way to reduce false alarms is to mask out problem regions. I used the cruder image cropping, which does work but over 24 hours essentially my entire background changes a few times a day. 

Adjusting the image threshold, including adaptive thresholding made little difference. Next I took a Adrian Rosebrock's weighted average (cv2.accumulateWeighted) of a few consecutive frames, and while this helped the false alarms were still annoyingly high.

The next thing to try was to only raise an alarm on say 5 consecutive triggers. This caused some intrusions to be missed, particularly if the intruder moved quickly. But there were still too many false alarms. A similarly crude method was to put upper and lower limits on the size of the detected bounding rectangle (cv2.boundingRect)

I ended up using all the methods in one after the other, and maybe got a few false alarms per hour. It sort of works, but was a little error-prone (ie misses intrusions). The program is background_subtraction.py. One possible improvement is Ivan Kudriavtsev's take on motion detection. The installation of Numba looks a bit daunting, so I left it for another time.

In Part 2, we will investigate if object recognition will help.

There you have it: the little Raspberry Pi that could do motion detection using OpenCV. Happy Trails.