Did I leave the Air Conditioner On? Indicator LED detection using Raspberry Pi and OpenCV

 

Air Conditioner Indoor Unit with Yellow Indicator LED

Air conditioners are essential in hot and humid Malaysia, especially if you want to work from home. Most of us have occasionally wondered if we have left it on after we left the house: the resulting electricity bill can be a nasty surprise. Most times you cannot do much about it, save for going back home to check.

But then, I managed to hack its infrared remote using an ESP8266, which made it an Internet of Things (IoT) device, which lets me turn it on and off from my smartphone. Now I have a need to know if I left the aircond turned on at home.  

When the indoor unit comes on, there is a beep and an orange LED lights up. The standard way is to mount an optocoupler diode in series with the orange LED, wire the optocoupler output to an ESP8266 and the resulting IoT will reliably report the aircond on/off status every time.

But variety, they say, is the spice of life, and I happened to have an obsolete Raspberry Pi Model B with OpenCV installed. And lots of ancient 640x480 webcams. Granted the lighting conditions would change through the day, but surely it can recognize that round orange light with some consistency?

USB webcam looking at the indicator LED from 3 feet away

It would be a bonus if software can be added later to detect that beep. That would have other applications like detection of smoke alarms' beeps, thunder, doorbell chimes and other interesting sounds. But that is another blog post.

OpenCV Raspberry Pi Model B with LAN connection (ie 'headless' mode)

 Isaac Vidas looks like a good starting point, first using an HSV transform to isolate the color of interest, then using cv2.HoughCircles() to precisely locate the LED itself. 

Image after HSV transform

I did need some additional help in setting the color thresholds required in his code to:

# Get lower orange hue

lower_orange_hue = create_hue_mask(hsv_image, [0, 0, 255], [0, 255, 255]) 

# Get higher orange hue 

higher_orange_hue = create_hue_mask(hsv_image, [0, 0, 255], [38, 255, 255])

There is a handy python script by nathancy here, and together with a HSV color wheel, my threshold values could be determined using several images of the aircond LED under different lighting conditions.

HSV Color wheel

  

Image after color filtering

Firstof, you will be needing a programs to view the USB webcam video and still frames. I use mplayer and feh:

# apt-get install mplayer

# apt-get install feh

You set the Pi via raspi-config not to run the X Server (ie the GUI desktop), but it helps to have the X libraries installed. From your laptop/desktop you just ssh in:

fred@pi:~ $ ssh -t -Y 12.34.56.78

And from there, mplayer should display the video on your desktop. This lets you position the camera properly.

fred@pi:~ $ mplayer tv://

To get 10 still frames after 10s (some cameras auto-adjust brightness):

fred@pi:~ $ mplayer -vo jpeg -frames 10 -ss 10 -brightness 25 tv://

You can use 'mplayer -loop 0' to display the still images, but they flash on and off rather annoyingly. I much prefer something like feh:

fred@pi:~ $ feh .images/image_on.png

And best of all, the openCV code will execute as if you were using the Pi's console (ie HDMI).

Having selected your camera position, you should probably make a set of images under different lighting conditions. I used a fragment of Isaac Vidas's code to do this, in particular to see the effect of lighting on the separate operations like blurring and HSV transformation. This is named webcamTest.py and is available on my github repository. You typically do:

 

fred@pi:~/checkLed $ source ~/opencv/OpenCV-4.0-py3/bin/activate

(OpenCV-4.0-py3) fred@pi:~/checkLed $

(OpenCV-4.0-py3) fred@pi:~/checkLed $ python ./webcamTest.py image_on.png 

Next, use the nathancy code, which I named hsvThresholder.py. 

(OpenCV-4.0-py3) fred@pi:~/checkLed $ python hsvThresholder.py

hsvThresholder.py: adjust the sliders at the bottom. Runs very slowly on a Pi B, so be patient and watch the console output in the window below

You want to adjust the various sliders in order to mask out all other regions of different color to your LED. A Raspberry Pi 1 Model B will be extremely slow here so patience is required. One way is to watch the bash console messages as they are much quicker to update than the picture. Copy the final settings from the console, which will be something like:

(hMin = 0 , sMin = 0, vMin = 85), (hMax = 28 , sMax = 255, vMax = 255)

My version of Isaac Vidas's code is named checkAC_led.py. and pretty much works as advertised, except it required a much larger (something like 6x diameter) image of the LED. I would have needed to mount my camera much closer, just 17cm from the LED. The other problem is the camera needs to be square over the LED as cv2.HoughCircles() do not detect ellipses very well. And line (ie hollow) circles worked better than a solid one.

Image with test circle added: this is the minimum size circle cv2.HoughCircles() will detect

Mounting my camera closer and square-on the LED is the correct solution. This also minimizes false alarms and improves reliability of detection. This probably means some sort of mounting bracket on the wall, and might get in the way when the air conditioner is being serviced. A software solution would be great, and the future beep detector would help filter out those false alarms ...

This led me to cv2.SimpleBlobDetection() code, which does much better with smaller and deformed circles. Take care to set minArea as large as possible: I actually counted the number of LED pixels in my HSV transform.

The gotcha here is that the HSV image has to be inverted for blob detection to work:

    h, s, image_gray = cv2.split(full_image)

    image_gray_neg = cv2.bitwise_not(image_gray) 

    detector = cv2.SimpleBlobDetector_create(params)

After conversion to grayscale and inversion

After successful blob detection

The final version, checkACvideo_led.py reads from the webcam instead of still image files, filters out false alarms based on the blob x and y coordinates and prints the air conditioner status. In its IoT form the print statement just needs to be modified to publish to an MQTT server like mosquitto.

So did I leave the air conditioner on? Hey Mycroft, is my air conditioner on or off?

Happy Trails

The Littlest Computer that Could: OpenCV using Raspberry Pi 1 Model B Rev 2

 

First there was the 1B: Raspberry Pi One Model B.

Now out of production, the Raspberry Pi One Model B was released in 2012, earlier than the Model A. I had a few lying unused in my parts box, some damaged by defective power supplies, but mostly superseded by better versions like the Pi 2, 3 and 4s. The Pi 1 Model B was the slowest, had only 512MB DRAM and used the sdcard as mass storage. The USB functionality was questionable: the LAN chip was internally routed through the USB bus which crippled its throughput. And since the Pi was always touchy about its 5V input power adding basic functionality like keyboard, hdmi USB was always a hit and miss affair.

Left: sdcard, right: micro-sdcard

The sdcard is getting very hard to find. You find micro-sdcard with an adapter but these tend to develop contact problems and corrupt the onboard filesystem. To make matters worse, a Pi Model B which cannot boot will have no indication: there is just that red power LED on and nothing else, and it looks pretty much like a dead Pi. Some 60% of my discarded Pi 1s simply had microsd adapter contact problems and could not boot.

On the plus side, the Pi 1 drew the least power amongst the Pi series which meant most old Android phone chargers could power it. It also had audio and video jacks, which were very handy with retro electronics. 

I managed put one to use monitoring my solar panel, but most of the little jobs are better served by the ESP8266 or the Microchip PIC. If only the Raspberry Pi Model B could run OpenCV; with a bit of nifty image processing, it might find a use, perhaps to check if my front gate has been left open, or the air conditioner left running, or if the smoke alarm is beeping. 

Most of the time, the Pi 3 is the minimum recommended model, but there is no mention Raspberry Pi One cannot be used. No harm trying; and since the install process can be left alone, it is easily done on the side. 

Raspberry Pi 1 Model B Rev 2 with infamous microsd adapter

$dd if=2021-10-30-raspios-bullseye-armhf-lite.img of=/dev/sdc

After the standard install of Raspbian, I use raspi-config to turn on the ssh server and set a fixed ethernetIP. It can then be used as a headless (ie no monitor or keyboard) system via ssh from a host laptop or desktop. After which there is the usual obligatory

# apt-get update --allow-releaseinfo-change
# apt-get upgrade

And the Pi model:

root@pi:~# cat /sys/firmware/devicetree/base/model
Raspberry Pi Model B Rev 2

Jeremy Morgan's OpenCV install worked for my Pi 3 before, but now instead it stops:

# pip install opencv-contrib-python
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting opencv-contrib-python
Downloading opencv-contrib-python-4.5.4.60.tar.gz (150.7 MB)
|��������������������������������| 150.4 MB 93 kB/s eta 0:00:04Killed

From 'dmesg -T' it looks like I ran out of memory:

[Tue Nov 23 18:03:30 2021] [ 5714]     0  5714    87141    21594     104       0

    21597             0 pip

[Tue Nov 23 18:03:30 2021] Out of memory: Kill process 5714 (pip) score 349 or sacrifice child

[Tue Nov 23 18:03:30 2021] Killed process 5714 (pip) total-vm:348564kB, anon-rss

:86376kB, file-rss:0kB, shmem-rss:0kB

[Tue Nov 23 18:03:30 2021] oom_reaper: reaped process 5714 (pip), now anon-rss:0

kB, file-rss:0kB, shmem-rss:0kB

My free memory is:

# free -m

              total        used        free      shared  buff/cache   available

Mem:            369          18         285           0          65         304

Swap:         15358          21       15337

And I can get a litte more by editing the boot partition's config.txt:

# vi /boot/config.txt

Add:

gpu_mem=16

And comment out

#start_x=1

After a reboot I get more memory:

# free -m

              total        used        free      shared  buff/cache   available

Mem:            477          31         338           6         106         390

Swap:            99           0          99

 

But this is still not enough. Now I could increase the swap file in my micro sdcard, but the thrashing might wear it out as the number of write operations is limited. Instead I used one of the many ancient thumbdrives, discarded just because of their low capacities. I ended up using a compactflash card for its speed:

# dd if=/dev/zero of=/dev/sda bs=1M count=1024

# mkswap /dev/sda

# swapon /dev/sda

# pip install --upgrade pip setuptools wheel

# python -m pip install --upgrade pip

# pip3 install opencv-contrib-python

pip seems to have gone walkabout so,

#  ln -s /usr/local/bin/pip /usr/bin/pip

Failure:

# pip install opencv-contrib-python

    File "setup.py", line 381, in _classify_installed_files_override

      with open(os.path.join(cmake_install_dir, "python", "cv2", "__init__.py"),

 'r') as opencv_init:

  FileNotFoundError: [Errno 2] No such file or directory: '_skbuild/linux-armv6l

-3.7/cmake-install/python/cv2/__init__.py'

  ----------------------------------------

  ERROR: Failed building wheel for opencv-contrib-python

Failed to build opencv-contrib-python

ERROR: Could not build wheels for opencv-contrib-python, which is required to in

stall pyproject.toml-based projects

Vishwesh Shrimali's instructions seem promising, and his minimum requirement is for a Pi 2. There are more separate bash commands which increases the chances for a successful debug. Since a fail is near certain, I chose to key in the commands manually instead of running Shrimali's script.

root@pi:/root/opencv# apt-get -y purge wolfram-engine

root@pi:/root/opencv# apt-get -y purge libreoffice*

root@pi:/root/opencv# apt-get -y clean

root@pi:/root/opencv# apt-get -y autoremove

root@pi:/root/opencv# apt -y update

root@pi:/root/opencv# apt -y upgrade

root@pi:/root/opencv# apt-get -y remove x264 libx264-dev

root@pi:/root/opencv# apt-get -y install build-essential checkinstall

 cmake pkg-config yasm

root@pi:/root/opencv# apt-get -y install git gfortran

root@pi:/root/opencv# apt-get -y install libjpeg8-dev libjasper-dev libpng12-dev

root@pi:/root/opencv# apt-get -y install libtiff5-dev

root@pi:/root/opencv# apt-get -y install libtiff-dev

root@pi:/root/opencv# apt-get -y install libxine2-dev libv4l-dev

root@pi:/root/opencv# cd /usr/include/linux

root@pi:/usr/include/linux# ln -s -f ../libv4l1-videodev.h videodev.h

root@pi:/usr/include/linux# cd $cwd

root@pi:/root/opencv#

root@pi:/root/opencv# apt-get -y install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev

root@pi:/root/opencv# apt-get -y install libgtk2.0-dev libtbb-dev qt5-default                         

root@pi:/root/opencv# apt-get -y install libatlas-base-dev

root@pi:/root/opencv# apt-get -y install libmp3lame-dev libtheora-dev

root@pi:/root/opencv# apt-get -y install libvorbis-dev libxvidcore-dev libx264-dev

root@pi:/root/opencv# apt-get -y install libopencore-amrnb-dev libopencore-amrwb-dev 

root@pi:/root/opencv# apt-get -y install libavresample-dev

root@pi:/root/opencv# apt-get -y install x264 v4l-utils

            The following are optional:

root@pi:/root/opencv# apt-get -y install libprotobuf-dev protobuf-compiler

root@pi:/root/opencv# apt-get -y install libgoogle-glog-dev libgflags-dev 

root@pi:/root/opencv# apt-get -y install libgphoto2-dev libeigen3-dev libhdf5-dev doxygen

            Required python libraries

root@pi:/root/opencv# apt-get -y install python3-dev python3-pip

            Virtual environment:

fred@pi:~/opencv $ python3 -m venv OpenCV-4.0-py3

fred@pi:~/opencv $ echo "# Virtual Environment Wrapper" >> ~/.bashrc

fred@pi:~/opencv $ echo "alias workoncv-4.0=\"source /root/opencv/OpenCV-4.0-py3/bin/activate\"" >> ~/.bashrc                                       

fred@pi:~/opencv $ source /root/opencv/OpenCV-4.0-py3/bin/activate(OpenCV-4.0-py3) fred@pi:~/opencv $

 Increase the swap file from 100 to 1024:

(OpenCV-4.0-py3) fred@pi:~/opencv $ sudo sed -i 's/CONF_SWAPSIZE=100/CONF_SWAPSIZE=1024/g' /etc/dphys-swapfile

(OpenCV-4.0-py3) fred@pi:~/opencv $ sudo /etc/init.d/dphys-swapfile stop

[ ok ] Stopping dphys-swapfile (via systemctl): dphys-swapfile.service.      

(OpenCV-4.0-py3) fred@pi:~/opencv $ sudo /etc/init.d/dphys-swapfile start

[ ok ] Starting dphys-swapfile (via systemctl): dphys-swapfile.service.         

(OpenCV-4.0-py3) fred@pi:~/opencv $ pip install numpy dlib

(OpenCV-4.0-py3) fred@pi:~/opencv $ deactivate

This is an over 400MB porker of a file:

fred@pi:~/opencv $ git clone https://github.com/opencv/opencv.git

In retrospect I should have checked out 4.0.1 as having cv2.drawKeypoints() would have been handy.

fred@pi:~/opencv/opencv $ git checkout 4.0.0

fred@pi:~/opencv $ git clone https://github.com/opencv/opencv_contrib.git

fred@pi:~/opencv $ cd opencv_contrib

fred@pi:~/opencv/opencv_contrib $ git checkout 4.0.0

Then comes the config:

fred@pi:~/opencv/opencv/build $ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE

_INSTALL_PREFIX=/home/heong/opencv/installation/OpenCV-4.0 -D INSTALL_C_EXAMPLES

=ON -D INSTALL_PYTHON_EXAMPLES=ON -D WITH_TBB=ON -D WITH_V4L=ON -D OPENCV_PYTHON3_INSTALL_PATH=/home/heong/opencv/OpenCV-4.0-py3/lib/python3.5/site-packages -D WITH_QT=ON -D WITH_OPENGL=ON  -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules -D BUILD_EXAMPLES=ON ..

Now for the make:

fred@pi:~/opencv/opencv/build $ CMAKE_INSTALL_PREFIX=/usr/local

fred@pi:~/opencv/opencv/build $ export CMAKE_INSTALL_PREFIX

fred@pi:~/opencv/opencv/build $ make

Produces the error

In file included from /home/fred/opencv/opencv_contrib/modules/cvv/src/qtutil/f

ilter/sobelfilterwidget.cpp:3:

/home/heong/opencv/opencv/modules/imgproc/include/opencv2/imgproc.hpp:208:5: not

e:   �FILTER_SCHARR�

     FILTER_SCHARR = -1

     ^~~~~~~~~~~~~

make[2]: *** [modules/cvv/CMakeFiles/opencv_cvv.dir/build.make:453: modules/cvv/

CMakeFiles/opencv_cvv.dir/src/qtutil/filter/sobelfilterwidget.cpp.o] Error 1

make[1]: *** [CMakeFiles/Makefile2:12676: modules/cvv/CMakeFiles/opencv_cvv.dir/

all] Error 2

make: *** [Makefile:163: all] Error 2

From Nobuo Tsukamoto, in the file /home/fred/opencv/opencv_contrib/modules/cvv/src/qtutil/f

ilter/sobelfilterwidget.cpp added 'using namespace cv;' at line 13 thus:

#include "../../util/util.hpp"

#include "../filterfunctionwidget.hpp"

#include "../filterselectorwidget.hpp"

using namespace cv; // cmheong 2021-11-29

namespace cvv

{

namespace qtutil

{

SobelFilterWidget::SobelFilterWidget(QWidget *parent)

After which

fred@pi:~/opencv/opencv/build $ make

Produces the error

Scanning dependencies of target example_cpp_detect_mser

[ 89%] Building CXX object samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o

/home/fred/opencv/opencv/samples/cpp/detect_mser.cpp:28:10: fatal error: GL/glu.h: No such file or directory

 #include <GL/glu.h>

          ^~~~~~~~~~

compilation terminated.

make[2]: *** [samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/build.make:63:

samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o] Error 1

make[1]: *** [CMakeFiles/Makefile2:29519: samples/cpp/CMakeFiles/example_cpp_det

ect_mser.dir/all] Error 2

make: *** [Makefile:163: all] Error 2

From RajkiranVeldur, just do

root@pi:~#  apt-get install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev

After which

fred@pi:~/opencv/opencv/build $ make

Produces the link error

[ 89%] Linking CXX executable ../../bin/example_cpp_detect_mser

/usr/bin/ld: CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o: in functi

on `draw(void*)':

detect_mser.cpp:(.text.startup.main+0x1c70): undefined reference to `gluPerspect

ive'

collect2: error: ld returned 1 exit status

make[2]: *** [samples/cpp/CMakeFiles/example_cpp_detect_mser.dir/build.make:134:

 bin/example_cpp_detect_mser] Error 1

make[1]: *** [CMakeFiles/Makefile2:29519: samples/cpp/CMakeFiles/example_cpp_det

ect_mser.dir/all] Error 2

make: *** [Makefile:163: all] Error 2

regpa mentioned that I need openGLU.so, but a brute-force search could not come up with one:

fred@pi:~/opencv/opencv/build $ sudo ls -lR / | grep -e openGLU 

fred@pi:~/opencv/opencv/build $

There is however, a file called libGLU.so mentioned by myinternetofthings:

fred@pi:~/opencv/opencv/build $ sudo ls -lR / 2>/dev/null | grep -e GLU.so

lrwxrwxrwx  1 root root       15 Sep 20  2015 libGLU.so -> libGLU.so.1.3.1

lrwxrwxrwx  1 root root       15 Sep 20  2015 libGLU.so.1 -> libGLU.so.1.3.1

-rw-r--r--  1 root root   358228 Sep 20  2015 libGLU.so.1.3.1

Added it to 2 separate files link.txt:

fred@pi:~/opencv/opencv/build $ cat samples/opengl/CMakeFiles/example_open

gl_opengl.dir/link.txt

/usr/bin/c++     -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-

dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wm

issing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -W

uninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comme

nt -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthr

ead -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee

 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG    -Wl,-

-gc-sections   CMakeFiles/example_opengl_opengl.dir/opengl.cpp.o  -o ../../bin/e

xample_opengl_opengl  -Wl,-rpath,/home/heong/opencv/opencv/build/lib -ldl -lm -l

pthread -lrt /usr/lib/arm-linux-gnueabihf/libGL.so ../../lib/libopencv_highgui.s

o.4.0.0 ../../lib/libopencv_videoio.so.4.0.0 ../../lib/libopencv_imgcodecs.so.4.

0.0 ../../lib/libopencv_imgproc.so.4.0.0 ../../lib/libopencv_core.so.4.0.0 /usr/

lib/arm-linux-gnueabihf/libGLU.so

fred@pi:~/opencv/opencv/build $ cat samples/cpp/CMakeFiles/example_cpp_det

ect_mser.dir/link.txt

/usr/bin/c++     -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-

dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wm

issing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -W

uninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comme

nt -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthr

ead -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee

 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG    -Wl,-

-gc-sections   CMakeFiles/example_cpp_detect_mser.dir/detect_mser.cpp.o  -o ../.

./bin/example_cpp_detect_mser  -Wl,-rpath,/home/heong/opencv/opencv/build/lib -l

dl -lm -lpthread -lrt /usr/lib/arm-linux-gnueabihf/libGL.so ../../lib/libopencv_

gapi.so.4.0.0 ../../lib/libopencv_stitching.so.4.0.0 ../../lib/libopencv_aruco.s

o.4.0.0 ../../lib/libopencv_bgsegm.so.4.0.0 ../../lib/libopencv_bioinspired.so.4

.0.0 ../../lib/libopencv_ccalib.so.4.0.0 ../../lib/libopencv_cvv.so.4.0.0 ../../

lib/libopencv_dnn_objdetect.so.4.0.0 ../../lib/libopencv_dpm.so.4.0.0 ../../lib/

libopencv_face.so.4.0.0 ../../lib/libopencv_freetype.so.4.0.0 ../../lib/libopenc

v_fuzzy.so.4.0.0 ../../lib/libopencv_hdf.so.4.0.0 ../../lib/libopencv_hfs.so.4.0

.0 ../../lib/libopencv_img_hash.so.4.0.0 ../../lib/libopencv_line_descriptor.so.

4.0.0 ../../lib/libopencv_reg.so.4.0.0 ../../lib/libopencv_rgbd.so.4.0.0 ../../l

ib/libopencv_saliency.so.4.0.0 ../../lib/libopencv_sfm.so.4.0.0 ../../lib/libope

ncv_stereo.so.4.0.0 ../../lib/libopencv_structured_light.so.4.0.0 ../../lib/libo

pencv_superres.so.4.0.0 ../../lib/libopencv_surface_matching.so.4.0.0 ../../lib/

libopencv_tracking.so.4.0.0 ../../lib/libopencv_videostab.so.4.0.0 ../../lib/lib

opencv_xfeatures2d.so.4.0.0 ../../lib/libopencv_xobjdetect.so.4.0.0 ../../lib/li

bopencv_xphoto.so.4.0.0 ../../lib/libopencv_shape.so.4.0.0 ../../lib/libopencv_p

hase_unwrapping.so.4.0.0 ../../lib/libopencv_optflow.so.4.0.0 ../../lib/libopenc

v_ximgproc.so.4.0.0 ../../lib/libopencv_datasets.so.4.0.0 ../../lib/libopencv_pl

ot.so.4.0.0 ../../lib/libopencv_text.so.4.0.0 ../../lib/libopencv_ml.so.4.0.0 ..

/../lib/libopencv_dnn.so.4.0.0 ../../lib/libopencv_video.so.4.0.0 ../../lib/libo

pencv_photo.so.4.0.0 ../../lib/libopencv_objdetect.so.4.0.0 ../../lib/libopencv_

calib3d.so.4.0.0 ../../lib/libopencv_features2d.so.4.0.0 ../../lib/libopencv_fla

nn.so.4.0.0 ../../lib/libopencv_highgui.so.4.0.0 ../../lib/libopencv_videoio.so.

4.0.0 ../../lib/libopencv_imgcodecs.so.4.0.0 ../../lib/libopencv_imgproc.so.4.0.

0 ../../lib/libopencv_core.so.4.0.0 /usr/lib/arm-linux-gnueabihf/libGLU.so

After which

fred@pi:~/opencv/opencv/build $ make

Completes successfully. But there is still the installation. Set the environment variable CMAKE_INSTALL_PREFIX so that:

fred@pi:~/opencv/opencv/build $ echo $CMAKE_INSTALL_PREFIX

/usr/local

fred@pi:~/opencv/opencv/build $ sudo make install

And a quick test run:

fred@pi:~/opencv/opencv/build $ source /home/fred/opencv/OpenCV-4.0-py3/bin/activate

(OpenCV-4.0-py3) fred@pi:~/opencv/opencv/build $ python

Python 3.7.3 (default, Jan 22 2021, 20:04:44)

[GCC 8.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import cv2

>>> print(cv2.__version__)

4.0.0

>>> quit()

And there you have it, Raspberry Pi 1 Model B Rev 2, the littlest computer that could OpenCV.

BB View 7" LCD on Debian Buster 10.3: Abandonware revisited

 

BB View LCD by element14. Note: the FPC cable when correctly inserted may not show printing topside

The BB View LCD came out about mid-2013, and was one of the earliest LCD panels for the beaglebone. It came with a "cape", ie a beaglebone interface board. I bought mine in December 2014 and could not get it to work. No doubt it worked for many other people: it came with kernel patches and source code for Angstrom, the TI SDK as well as Debian. It is still listed for sale in element14 and Mouser, but recently has become "unavailable".

Currently, there is a worldwide semiconductor shortage, and component leadtimes are now a few months. The temptation to press the BB View 7" LCD back into service became irresistible. I think my problem was I used the kernel patches with a later, 2014 version of Debian while the patches were mid-2013. The beaglebone black could recognize the cape and read its eeprom, but the LCD stayed blank, and the backlights kept blinking in perfect unison with the beaglebone CPU LED.

While no walk in the park, I thought it should possible to get it working. After all I have source code and it should only be a matter of time. And given the pandemic, I have four weeks of lockdown time, to be precise. And hopefully I should learn a little about kernel display drivers too.

First you need a beefy 5V power supply. I used a 5V@3A one. While the average current draw of the BB View LCD is not that high at 240mA, there are regular nasty spikes. I got the occasional CPU crash along with sdcard errors.

Beaglebone Black Serial Debug Header: Pin 1 is on the right

You also need to connect to the serial debug port. Any USB serial TTL adapter will do. The pinout is:

Beaglebone    Pin Description      USB Serial TTL Adapter
1                      Gnd                         Gnd
4                      TX                           RX
5                      RX                           TX

The Beaglebone Black manual specified an FTDI USB adapter, but I used a cheap CH340, which worked well enough. The baud rate is 115200, 8-bits and 1-stop with no hardware flow control.

FTDI Serial Debug cable. Click on picture for datasheet

The terminal program I use is minicom, set to 115200 baud, 8-bits, 1-stop and without hardware flow control. If you have the proper FTDI cable, you should use hardware flow control.

This may be a no-brainer to some, but an LCD's size is measured at its diagonal. The BB View LCD comes in 2 software-incompatible sizes, 4.3" and 7". For the unwary, a 7" LCD measures close to 4.3" on one side.

Also, the FPC cable is inserted with the metalled side facing down on both ends. And may not look like the picture on top. Each matching connector has a little lever which you pull up. Do not force the FPC cable.

The Debian image I used was the latest as of this writing, which is bone-debian-10.3-iot-armhf-2020-04-06-4gb.img.xz. 

To transfer the image to sdcard, I did:

# xzcat bone-debian-10.3-iot-armhf-2020-04-06-4gb.img.xz | dd of=/dev/sdb

Do take care to ensure /dev/sdb is the sdcard else you might end up wiping the drive on your laptop/desktop.

The changes to the sdcard /boot/uEnv.txt boot config file are:

dtb_overlay=/lib/firmware/BB-VIEW-LCD7-01-00A0.dtbo

disable_uboot_overlay_video=1

disable_uboot_overlay_audio=1

You can download the uEnv.txt file here. Notice that it specifies a binary device tree overlay file, BB-VIEW-LCD7-01-00A0.dtbo. The latest and greatest bone-debian-10.3-iot-armhf-2020-04-06-4gb.img.xz happened to have them pre-compiled. I have not seen it in earlier debian images, so this is indeed a stroke of luck. I have uploaded a copy here for your convenience. 

By the way it also has BB-BONE-LCD7-01-00A0.dtbo which also works except for an annoyingly dim backlight setting and the inability to adjust it.

These dtbo files are usually something that you get by recompiling the BB View Debian source files. However, the manufacturer supplied BB View Debian source file BB-VIEW-LCD7-01-00A0.dts did not work for me when compiled to dtbo files. On close examination it is missing pin definitions for 8 LCD panel data pins, lcd_data16 - lcd_data23.

Typically the beaglebone black boots by default from the eMMC, so you need to hold down the 'Boot' button on the beagleboard before you power on. The BB View cape gets in the way, but it helpfully provides a second 'Boot' button on the cape. 

Given the power demands of the BB View, I found it more reliable not to boot from the power switch. Rather there is a 'Power' button on the beaglebone black which you can use. 

The LCD displayed the login screen for tty1, and you can log in by plugging a USB keyboard into into the USB Host (ie USB Type B) connector.

Working BB View 7" LCD with Debian Buster 10.3

To display picture files from the command prompt I used fbi with a little help from Drew Fustini:

root@beaglebone:~# apt-get update

root@beaglebone:~# apt-get install fbi

# wget https://kernel.org/theme/images/logos/tux.png

# fbi -d /dev/fb0 -T 1 -a tux.png

For video I used mplayer:

# apt-get install mplayer

This worked for me as there is no audio when using the BB View LCD:

# export SDL_NOMOUSE=1

# mplayer -nolirc -vo sdl:driver=fbcon  -ao null  test.mpg

My test mpg file happened to have a resolution of 480x360. The BB View 7" LCD is 800x600 but mplayer could not quite get the scaling correct. Whan scaled manually this works:

# mplayer -nolirc -vo sdl:driver=fbcon  -ao null -x 800 -y 533 test.mpg

mplayer 

Debian 10.3 did not seem to recognise the touchscreen, but a little peek at the decompiled dtbo file mentioned a tscadc kernel modue:

        fragment@5 {

                target = <&tscadc>;

                __overlay__ {

                        status = "okay";

                        tsc {

                                ti,wires = <4>;

                                ti,x-plate-resistance = <200>;

                                ti,coordinate-readouts = <5>;

                                ti,wire-config = <0x00 0x11 0x22 0x33>;

                                ti,charge-delay = <0x400>;

                        };

                        adc {

                                ti,adc-channels = <4 5 6 7>;

                                ti,chan-step-opendelay = <0x098 0x3ffff 0x098 0x0>;

                                ti,chan-step-sampledelay = <0xff 0x0 0xf 0x0>;

                                ti,chan-step-avg = <16 2 4 8>;

                        };

                };

        };

A search of the dmesg log produced:

[Thu Jun 24 10:21:40 2021] input: ti-tsc as /devices/platform/ocp/44e0d000.tscadc/TI-am335x-tsc.0.auto/input/input0

Which led me to

# ls -l  /sys/devices/platform/ocp/44e0d000.tscadc/TI-am335x-tsc.0.auto/input/input0/event0

A quick check of the shows that tapping the LCD does indeed produce a response:

# hexdump -C -v /dev/input/event0

...

00000780  81 9f d5 60 e6 e6 06 00  03 00 01 00 2c 08 00 00  |...`........,...|

00000790  81 9f d5 60 e6 e6 06 00  03 00 18 00 92 03 00 00  |...`............|

000007a0  81 9f d5 60 e6 e6 06 00  00 00 00 00 00 00 00 00  |...`............|

000007b0  81 9f d5 60 1f 03 09 00  01 00 4a 01 00 00 00 00  |...`......J.....|

000007c0  81 9f d5 60 1f 03 09 00  03 00 18 00 00 00 00 00  |...`............|

000007d0  81 9f d5 60 1f 03 09 00  00 00 00 00 00 00 00 00  |...`............|

Another bit of electronics rescued from the scrap heap cannot be that bad.

Happy Trails.

PS

Why not use the element14 files and go with the BB View manual? Believe me, I tried. For 14 days. I started off with the oldest Debian images I could find, co-incidentally the same Debian 7.5 of 2014-05-14 that I used back then. And got the same result. And since there were 3 different patches, one each for Angstrom and TI SDK, I tried them as well but none worked. The Angstrom version was older 2013-06-20 and worked slightly better: the LCD backlight did not blink in concert with he CPU light and was quite controllable on its own. But in all cases the LCD stayed resolutely blank. 

There is a wealth of information on the subject, not to mention source code. But there is no schematic, so reading it would be harder. Two weeks later, I could rebuild the latest TI SDK, or even mainline Linux kernel using Linaro, but neither version worked with the BB View LCD. The problem as mentioned earlier was the device tree source code, which I suspect was incomplete for the 7" LCD. The 4.3" version probably worked, but I did not test this. 

It was starting to become clear why there was such heavy development over so many years: there is precious little standardization in ARM systems on how to interface to external boards. Each ARM board basically is unique, and back in 2013 this was hardcoded into the kernel source.

At some point each variation was defined in a 'device tree' that was compiled separately from kernel. In practice this simply means a separate sub-directory with its own Makefile all under the same master ARM build. 

Eventually, with a little encouragement from Linus Torvalds, the device tree code was shifted to the ARM bootloader, 'Das U-Boot'. This is where the Beaglebone Debug Serial Port comes in handy. Many of the BB View LCD problems are reported only as system console messages, ie to the Debug Serial Port, and may not even show up in the kernel log. 

Thus, unlike the manufacturer, Debian admirably maintained support for the BB View. As usual, the documentation became hopelessly out of date, as befits abandonware. 

After 2 weeks, Lady Luck took pity on me. I always have a late version of Beaglebone Debian, in this case Debian Buster 10.3 of 2020-04-06. The micro-sdcards are too tiny to label, and I swapped it into the BB View LCD system by mistake. It complained it missed the file BB-VIEW-LCD7-01-00A0.dtbo which just happened to be in the eMMC partition (as /dev/mmcblk1p1). One small change in the boot config file and it worked! Took all of 5 minutes.

Which leaves me 2 more weeks of lockdown time to fill. Maybe it's time to watch 'Das Boot' again ...

Das Boot: Wolfgang Petersen's 1981 classic

USB over IP: How to remote-access your USB devices over the network

 

First appearing in 2005 in Takahiro Hirofuchi's kickass paper, there are much better guides on USB/IP, like Linux Magazine's, Ridgerun's, etc. From 2009 USB/IP was accepted into mainline Linux kernel and up-to-date documentation on it tends to get lost in the Linux haystack. The wrinkle here is USB/IP running on Slackware 14.2-current as of July 2019. Unlike Debian/Ubuntu you cannot usually seamlessly install USB/IP (or most other things) on Slackware. A little DIY is in order.

I know that is cold comfort compared to Debian's 'apt install'. Indeed, Slackware too has its (a little rickety) SlackBuild scripts. But in USB/IP case there is no SlackBuild as it was already included in the kernel. Well, after a fashion.

But take heart: in 30 years of Slackware I seldom fail to install the things I want. This is a chance to poke a little into the innards of Linux. You have the source code in all its glory. And it is built beautifully - I say this as one who has seen Microsoft Windows NT source code [shudder]. 

And the experience will prove useful in the rare instances Debian's installs fail, like the time I installed the Java development kit and could no longer log into my desktop. This means when things fail the same methods work: paste the error into google and look for the patches and workarounds. But I digress; back to USB/IP.  

I should test my Acer aspireM3's USB webcam before the install breaks things:

# mplayer tv:// -tv driver=v4l2:width=640:height=480 -vo xv -tv device=/dev/video1

Let's verify that my Slackware kernel actuall has USB/IP:

root@aspireM3:~# ls -lR /usr/src/linux/ | grep -i usbip
-rw-r--r-- 1 root root  1172 Jan 26  2019 sysfs-platform-usbip-vudc
-rw-r--r-- 1 root root 23365 Jan 26  2019 usbip_protocol.txt

drwxr-xr-x 2 root root  4096 Jan 26  2019 usbip/
/usr/src/linux/drivers/usb/usbip:
-rw-r--r-- 1 root root 18694 Jan 26  2019 usbip_common.c
-rw-r--r-- 1 root root 10117 Jan 26  2019 usbip_common.h
-rw-r--r-- 1 root root  3969 Jan 26  2019 usbip_event.c
drwxr-xr-x  3 root root   4096 Jan 27  2019 usbip/

Looks good. A look at the kernel's config file shows that the binary has not been left out at compile time:

root@aspireM3:~# grep -i usbip /usr/src/linux/.config

CONFIG_USBIP_CORE=m

CONFIG_USBIP_VHCI_HCD=m

CONFIG_USBIP_VHCI_HC_PORTS=8

CONFIG_USBIP_VHCI_NR_HCS=1

CONFIG_USBIP_HOST=m

# CONFIG_USBIP_DEBUG is not set

And finally, locate the kernel modules (ie device drivers) themselves:

# ls -l /lib/modules/4.19.18/kernel/drivers/usb/usbip

total 128

-rw-r--r-- 1 root root 22576 Jan 27  2019 usbip-core.ko

-rw-r--r-- 1 root root 42480 Jan 27  2019 usbip-host.ko

-rw-r--r-- 1 root root 58016 Jan 27  2019 vhci-hcd.ko

And it runs OK:

# modprobe -v usbip-core

insmod /lib/modules/4.19.18/kernel/drivers/usb/usbip/usbip-core.ko

Let's launch the USB/IP server daemon:

# usbipd -D

-su: usbipd: command not found

Oops. The userspace binaries are not installed. Let's see if my Slackware installation has the userspace source code:

# ls -l /usr/src/linux/tools/usb/usbip/src

total 96

-rw-r--r-- 1 root root   440 Jan 26  2019 Makefile.am

-rw-r--r-- 1 root root  4406 Jan 26  2019 usbip.c

-rw-r--r-- 1 root root  1292 Jan 26  2019 usbip.h

-rw-r--r-- 1 root root  5404 Jan 26  2019 usbip_attach.c

-rw-r--r-- 1 root root  5169 Jan 26  2019 usbip_bind.c

-rw-r--r-- 1 root root  2922 Jan 26  2019 usbip_detach.c

-rw-r--r-- 1 root root  9713 Jan 26  2019 usbip_list.c

-rw-r--r-- 1 root root  6179 Jan 26  2019 usbip_network.c

-rw-r--r-- 1 root root  5295 Jan 26  2019 usbip_network.h

-rw-r--r-- 1 root root  1536 Jan 26  2019 usbip_port.c

-rw-r--r-- 1 root root  3494 Jan 26  2019 usbip_unbind.c

-rw-r--r-- 1 root root 15083 Jan 26  2019 usbipd.c

-rw-r--r-- 1 root root  1630 Jan 26  2019 utils.c

-rw-r--r-- 1 root root   863 Jan 26  2019 utils.h

Yes, I do. I just need to compile it but it fails:

/usr/src/linux/tools/usb/usbip# ./autogen.sh

/usr/src/linux/tools/usb/usbip# ./configure

/usr/src/linux/tools/usb/usbip# make

make  all-recursive

make[1]: Entering directory '/usr/src/linux-4.19.18/tools/usb/usbip'

Making all in libsrc

make[2]: Entering directory '/usr/src/linux-4.19.18/tools/usb/usbip/libsrc'

  CC       libusbip_la-usbip_device_driver.lo

usbip_device_driver.c: In function �..read_usb_vudc_device�..:

usbip_device_driver.c:106:2: error: �..strncpy�.. specified bound 256 equals destination size [Werror=stringop-truncation]

  strncpy(dev->path, path, SYSFS_PATH_MAX);

  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

usbip_device_driver.c:125:2: error: �..strncpy�.. specified bound 32 equals dest

ination size [-Werror=stringop-truncation]

  strncpy(dev->busid, name, SYSFS_BUS_ID_SIZE);

  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

cc1: all warnings being treated as errors

make[2]: *** [Makefile:471: libusbip_la-usbip_device_driver.lo] Error 1

make[2]: Leaving directory '/usr/src/linux-4.19.18/tools/usb/usbip/libsrc'

make[1]: *** [Makefile:497: all-recursive] Error 1

make[1]: Leaving directory '/usr/src/linux-4.19.18/tools/usb/usbip'

make: *** [Makefile:365: all] Error 2

It does not look serious. What used to be a compiler warning has now been classified as an error so the compiler obligingly stops. A quick google comes up with a patch:

https://patchwork.kernel.org/project/linux-usb/patch/20180721021232.GR14131@deca

dent.org.uk/

> +++ b/tools/usb/usbip/libsrc/usbip_common.c

> @@ -226,8 +226,8 @@ int read_usb_device(struct udev_device *

>       path = udev_device_get_syspath(sdev);

>       name = udev_device_get_sysname(sdev);

>

> -     strncpy(udev->path,  path,  SYSFS_PATH_MAX);

> -     strncpy(udev->busid, name, SYSFS_BUS_ID_SIZE);

> +     snprintf(udev->path, SYSFS_PATH_MAX, "%s", path);

> +     snprintf(udev->busid, SYSFS_BUS_ID_SIZE, "%s", name);

So I just edit usbip_device_driver.c:

# vi libsrc/usbip_device_driver.c

 And at line 106

        strncpy(dev->path, path, SYSFS_PATH_MAX);

 Is changed to

        // strncpy(dev->path, path, SYSFS_PATH_MAX); cmheong 2021-03-27

        snprintf(dev->path, SYSFS_PATH_MAX, "%s", path);

At line 125:

strncpy(dev->busid, name, SYSFS_BUS_ID_SIZE);

           Became

snprintf(dev->busid, SYSFS_BUS_ID_SIZE, "%s", name);

Anow for the file usbip_common.c line 230:

        // strncpy(udev->path,  path,  SYSFS_PATH_MAX); // cmheong 2021-03-27

        // strncpy(udev->busid, name, SYSFS_BUS_ID_SIZE);

Becomes

        snprintf(udev->path, SYSFS_PATH_MAX, "%s", path);

        snprintf(udev->busid, SYSFS_BUS_ID_SIZE, "%s", name);

Now some Slackware versions, depending on the date of your install, you might get an additional error:

  CC       usbip_network.o

usbip_network.c: In function âusbip_net_pack_usb_deviceâ:

usbip_network.c:91:32: error: taking address of packed member of âstruct usbip_

usb_deviceâ may result in an unaligned pointer value [-Werror=address-of-packed

-member]

   91 |  usbip_net_pack_uint32_t(pack, &udev->busnum);

      |                                ^~~~~~~~~~~~~

We just need to tell the compiler not to freak out and treat the warning as an error. This can be done by changing line 12174 of the 'configure' file:

/usr/src/linux/tools/usb/usbip$vi configure

EXTRA_CFLAGS="-Wall -Wno-error=address-of-packed-member -Werror -Wextra -std=gnu

99"

After which the compile completes successfully:

# make

make  all-recursive

make[1]: Entering directory '/usr/src/linux-4.19.18/tools/usb/usbip'

Making all in libsrc

make[2]: Entering directory '/usr/src/linux-4.19.18/tools/usb/usbip/libsrc'

  CC       libusbip_la-usbip_common.lo

  CC       libusbip_la-usbip_host_common.lo

  CC       libusbip_la-vhci_driver.lo

  CC       libusbip_la-sysfs_utils.lo

  CCLD     libusbip.la

make[2]: Leaving directory '/usr/src/linux-4.19.18/tools/usb/usbip/libsrc'

Making all in src

make[2]: Entering directory '/usr/src/linux-4.19.18/tools/usb/usbip/src'

  CC       usbip.o

  CC       utils.o

  CC       usbip_network.o

  CC       usbip_attach.o

  CC       usbip_detach.o

  CC       usbip_list.o

  CC       usbip_bind.o

  CC       usbip_unbind.o

  CC       usbip_port.o

  CCLD     usbip

  CC       usbipd.o

  CCLD     usbipd

make[2]: Leaving directory '/usr/src/linux-4.19.18/tools/usb/usbip/src'

make[2]: Entering directory '/usr/src/linux-4.19.18/tools/usb/usbip'

make[2]: Leaving directory '/usr/src/linux-4.19.18/tools/usb/usbip'

make[1]: Leaving directory '/usr/src/linux-4.19.18/tools/usb/usbip'

Now the server daemon runs:

# /usr/src/linux/tools/usb/usbip/src/usbipd -D

A quick check:

/usr/src/linux/tools/usb/usbip# ps -ef | grep -e usbip

root     23597     2  0 14:57 ?        00:00:00 [usbip_event]

root     23608     1  0 19:53 ?        00:00:00 /usr/src/linux/tools/usb/usbip/src/.libs/lt-usbipd -D

root     23835  1533  0 19:54 pts/0    00:00:00 grep -e usbip

Next we list the USB devices available to the server:

/usr/src/linux/tools/usb/usbip# /usr/src/linux/tools/usb/usbip/src/usbip list --local

 - busid 1-3.1 (0835:8501)

   Action Star Enterprise Co., Ltd : unknown product (0835:8501)

 - busid 1-3.2 (046d:c077)

   Logitech, Inc. : M105 Optical Mouse (046d:c077)

 - busid 1-3.3 (13ba:0017)

   PCPlay : PS/2 Keyboard+Mouse Adapter (13ba:0017)

 - busid 1-3.4 (10c4:ea60)

   Cygnal Integrated Products, Inc. : CP2102/CP2109 UART Bridge Controller [CP21

0x family] (10c4:ea60)

 - busid 1-3.5.1 (0835:8502)

   Action Star Enterprise Co., Ltd : unknown product (0835:8502)

 - busid 1-3.5.2 (046d:0829)

   Logitech, Inc. : unknown product (046d:0829)

 - busid 1-3.5.4 (0c45:62f1)

   Microdia : unknown product (0c45:62f1)

 - busid 2-1.3 (04f2:b300)

   Chicony Electronics Co., Ltd : unknown product (04f2:b300)

 - busid 2-1.4 (04ca:3006)

   Lite-On Technology Corp. : unknown product (04ca:3006)

Using the displayed busis, we use 'lsusb -v' to zero in on the webcam:

Bus 002 Device 003: ID 04f2:b300 Chicony Electronics Co., Ltd

  iManufacturer           1 Chicony Electronics Co., Ltd.

  iProduct                2 HD WebCam

  iSerial                 3 SN0001

 - busid 2-1.3 (04f2:b300)

Now I need to launch another kernel module:

# modprobe -v usbip-host

insmod /lib/modules/4.19.18/kernel/drivers/usb/usbip/usbip-host.ko

We now have the USB/IP server grab the webcam:

/usr/src/linux/tools/usb/usbip# /usr/src/linux/tools/usb/usbip/src/usbip bind --busid=2-1.3

usbip: info: bind device on busid 2-1.3: complete

Now we move over to the other (ie client) computer, and repeat the process of installing USB/IP. We then launch the USB/IP kernel client modules:

/usr/src/linux/tools/usb/usbip$modprobe -v vhci-hcd

insmod /lib/modules/4.19.62/kernel/drivers/usb/usbip/usbip-core.ko

insmod /lib/modules/4.19.62/kernel/drivers/usb/usbip/vhci-hcd.ko

Let's say the server's IP address is 12.34.56.78. We can see what USB device on the server is available over IP:

/usr/src/linux/tools/usb/usbip$src/usbip list --remote=12.34.56.78

Exportable USB devices

======================

 - 12.34.56.78

      2-1.3: Chicony Electronics Co., Ltd : unknown product (04f2:b300)

           : /sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.3

           : Miscellaneous Device / ? / Interface Association (ef/02/01). 

 

Ah it is a webcam ;7) We attach to it:

/usr/src/linux/tools/usb/usbip# /usr/src/linux/tools/usb/usbip/src/usbip attach -debug --remote=12.34.56.78 --busid=2-1.3

And we play the remote (ie server) webcam by using mplayer:

/usr/src/linux/tools/usb/usbip$ mplayer -cache 128 -tv device=/dev/video2:driver=v4l2:width=640:height=480:outfmt=i420 -vo xv tv://

Note that here I access /dev/video2, since my client laptop also has a webcam at /dev/video1, and the USB/IP attach made a new one at /dev/video2.  Also the server webcam resolution is 1080 x 720 but that seemed to mave maxed out my old 100Mbps fiber link. Dropping the resolution down to 640 x 480 worked out well.

You can see your attached USB device using:

/usr/src/linux/tools/usb/usbip$src/usbip port --remote=12.34.56.78

Imported USB devices

====================

Port 00: <Port in Use> at High Speed(480Mbps)

       Chicony Electronics Co., Ltd : unknown product (04f2:b300)

       3-1 -> usbip://192.168.1.4:3240/3-1.3

           -> remote bus/dev 003/003

Notice the port number 00. We will be needing it. Lastly once you are through with the webcam you release it so another computer can use it:

/usr/src/linux/tools/usb/usbip$src/usbip detach --port=00

usbip: info: Port 0 is now detached!

There you have it. USB/IP comes standard with Linux, and lets you access USB devices remotely. I had great fun spending an afternoon doing this to the boogie blues. Here's a video of Daisy tapping her feet to it:

Click on picture for the video

Happy Trails.

RESTful IoT with privacy & security: How to set up a Debian HTTPS Server

 

"It is quiet here and restful, and the air is delicious. There are gardens everywhere and police spies lie in the bushes ... " - Maxim Gorky

It is very tempting to use the ubiquitous HTTP web protocol for IoT. It is easy to test, and lets you operate your IoT from smartphones, tablets, desktops and even a computer program. Such a setup is called RESTful. It simply means your IoT device speaks the language of the web browser and web server. 

So we rush ahead with our RESTful API, and the IoT device is soon working and indispensable. Pretty soon we realize we need security and privacy: it won't do to have a hacker open the voice-controlled garage door ...

Our first line of defense is our WiFi password. It is reasonable to assume those living in the house should have access to WiFi and IoT. But what if we had guests or lodgers? Changing WiFi passwords can be a real bear, especially if you have 20-odd IoT devices. In fact it makes sense to localize the changes to a dedicated IoT server. RESTful, naturally.  

We will be needing some form of authentication: account names and passwords should do for now. Next we will need a reasonable amount of privacy, i.e., encryption so that someone else should not be able to lift the IoT password off the WiFi. That means HTTPS, or HTTP with SSL.

We start by implementing HTTPS server on a Linux system. The ESP8266 is known to be a little wobbly running HTTPS. ESP32 is better, but we can do without the complication for now. The traditional way is to use Apache. There are other, easier ways (like nginx, nodejs and even python) but Apache lets you run multiple servers right off the bat. This means you can keep your bad old HTTP server, add another HTTPS server on top of that and lets you support both your HTTP and HTTPS IoT devices.

From a bog-standard Debian (mine is a Beaglebone on eMMC), do the usual:

# apt-get update

# apt-get upgrade

Next, get Apache:

# apt install apache2

And while you are at it, you might as well make sure you have ssh. I got my DNS from duckdns.

Apache should come up complete with the stock webpage at http://localhost. Put your webserver files at /var/www/html/

To access the webserver from outside your WiFi access point, you will need a DNS server, but once you get it organized, a bog standard browser will display a warning before it will display your home page:

That means you need SSL, which usually costs money. You can opt for a self-signed certificate but this will produce a warning with most browsers. You then elect to disregard the warning and proceed, but this is a real problem if you are trying to sell the IoT device.

One way out is to get a 90-day certificate free from sslforfree. You just have to register, input your domain name and prove that you have access to the webserver, usually by uploading an sslforfree file to it. After it checks out the certificates can be downloaded. sslforfree links to a youtube video describing the process.

Do check out the video. I will simply list the differences relevant to a Debian installation. In Debian it is a simple:

# a2enmod ssl

Considering dependency setenvif for ssl:

Module setenvif already enabled

Considering dependency mime for ssl:

Module mime already enabled

Considering dependency socache_shmcb for ssl:

Enabling module socache_shmcb.

Enabling module ssl.

See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create s

elf-signed certificates.

To activate the new configuration, you need to run:

  systemctl restart apache2

I now need a configuration file for my HTTPS (or SSL) webserver. There is a template in Debian:

# cp /etc/apache2/sites-available/default-ssl.conf /etc/apache2/sites-available/secure.cmheong.duckdns.org.conf

secure.cmheong.duckdns.org being the domain name of my new HTTPS server. The parameters for the new server are put in:

# cat /etc/apache2/sites-available/secure.cmheong.duckdns.org.conf | head -n 16

<IfModule mod_ssl.c>

        <VirtualHost _default_:443>

                ServerName secure.cmheong.duckdns.org

                ServerAlias www.secure.cmheong.duckdns.org

                ServerAdmin webmaster@secure.cmheong.duckdns.org

                DocumentRoot /var/www/html

                # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,

                # error, crit, alert, emerg.

                # It is also possible to configure the loglevel for particular

                # modules, e.g.

                #LogLevel info ssl:warn

                ErrorLog ${APACHE_LOG_DIR}/secure.cmheong.error.log

                CustomLog ${APACHE_LOG_DIR}/secure.cmheong.access.log combined

You then check your configuration and do not proceed further until this passes:

# apachectl configtest

Syntax OK

Make sure your webserver is now accessible from the Internet. Usually this means setting up Port Forwarding in your gateway to forward all port 443 traffic to your server IP address.

# systemctl restart apache2

You will then need to prepare for the sslforfree test of webserver and domain name ownership:

# mkdir /var/www/html/.well-known

# mkdir /var/www/html/.well-known/pki-validation

Register with sslforfree, download the challenge file they provided and put it in the new directory. This is where the Debian ssh installation comes in handy. 

If the sslforfree challenge succeeds, then the certificates and private key will be generated as a zip file.

# unzip secure.cmheong.duckdns.org.zip

Archive:  secure.cmheong.duckdns.org.zip

 extracting: certificate.crt

 extracting: ca_bundle.crt

 extracting: private.key

You then move them to their final secure directories:

# cp -v ./sslforfree/*.crt  /etc/ssl/certs

'./sslforfree/ca_bundle.crt' -> 'certs/ca_bundle.crt'

'./sslforfree/certificate.crt' -> 'certs/certificate.crt'

# cp  ./sslforfree/private.key  /etc/ssl/private/private.key

Remember to delete the ./sslforfree directory. If you want to put the certificates in a different place you will need to update the site config file accordingly:

# cat /etc/apache2/sites-available/secure.cmheong.duckdns.org.conf | grep -i SSLCerti

                #   SSLCertificateFile directive is needed.

                #SSLCertificateFile     /etc/ssl/certs/ssl-cert-snakeoil.pem

                #SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

                SSLCertificateFile      /etc/ssl/certs/certificate.crt

                SSLCertificateKeyFile /etc/ssl/private/private.key

                SSLCertificateChainFile /etc/ssl/certs/ca_bundle.crt

As usual test the Apache configuration:

# apachectl configtest

And then restart Apache:

# systemctl restart apache2

The just aim your Chrome browser at https://www.yoursecureserver.com. If it worked you get something like this:

 

Now the traffic to and from the IoT RESTful server is encrypted. Note the sslforfree certificates expire in 90 days, but you are free to generate a new set. They will even email you a reminder. 

There you have it: a secure Internet-facing RESTful IoT server.

Happy Trails.

If at first you don't succeed, try try again: Flashing an old Beaglebone Black eMMC

 

2019 Hugo Award for Best Novelette: Zen Cho

Back in 2016 I bought a few Beaglebone Blacks, but did not get round to using them. I guess they were superseded by the later-model Raspberry Pi's. 

Beaglebone Black

The Pi has always been less reliable than the Beagleboard or Beaglebone. Broadcom USB subsystem was especially flaky and since the Pi USB bus handled both disk IO as well as Ethernet, you are often SOL as Broadcom is not really reknown for fixing things. Little things like industrial temperature rating of 85 degrees Celsius. And having to run Linux from sdcards, which are certain to wear out. Even worse the full-size sdcard sockets fail after a few years. Luckily the later Pi models use microsd sockets, but I am still stuck with a bunch of Pi Model Bs and their flaky sdcards.

The Pi had irresistible things going for it. It was way cheaper, had more addon modules (ie 'hats'), and best of all, it had Debian. There was no longer the month-long struggle to get Angstrom Linux to run properly. Yet those niggling problems ...

Getting my Beaglebone Black running was unexpectedly painless. I plugged it into my laptop USB port and it was running. It came up as /dev/ttyACM0 a serial port. No problem, all I needed was minicom, and the default settings of 115200 baud, 8 bits, 1 stop, no parity.

Default account is 'debian' and password 'tempwd'. There is no root password. I did not have to struggle with Angstrom. In fact I forgot to put in the sdcard. That meant it loaded from on-board mass storage. And it was Debian!

root@beaglebone:~# cat /etc/dogtag

BeagleBoard.org Debian Image 2015-03-01

Turned out the onboard memory was eMMC, still flash memory, but in IC form without those dreaded sockets. Memory is 512MB and there is no built-in WiFi; that would come only in the Beaglebone Black Wireless.

From bottom left: USB Master socket, microsd  and the elusive User-boot button

First order of business with Debian is to get it up to date. I connected it via copper LAN to my ADSL modem where it found the Internet on its own. But 'apt-get update' had errors, even though it technically did not fail:

root@beaglebone:~# apt-get update

W: Failed to fetch http://ftp.us.debian.org/debian/dists/wheezy/contrib/binary-a

rmhf/Packages  404  Not Found [IP: 64.50.233.100 80]

W: Failed to fetch http://ftp.us.debian.org/debian/dists/wheezy/non-free/binary-

armhf/Packages  404  Not Found [IP: 64.50.233.100 80]

W: Failed to fetch http://ftp.us.debian.org/debian/dists/wheezy-updates/main/bin

ary-armhf/Packages  404  Not Found [IP: 64.50.233.100 80]

'apt-get upgrade' finished OK:

root@beaglebone:/home/debian# apt-get upgrade

But the version was wheezy, and really old. Not to worry, I reached for the latest images, and downloaded AM3358 Debian 9.12 2020-04-06 4GB SD ImgTec. It only needed a tiny (4GB!) microsd card, and:

$xzcat bone-debian-10.3-iot-armhf-2020-04-06-4gb.img.xz | sudo dd of=/dev/sdc

7372800+0 records in

7372800+0 records out

3774873600 bytes (3.8 GB, 3.5 GiB) copied, 3001 s, 1.3 MB/s

But it did not boot from the microsd, and instead after an hour or so booted from eMMC. Time to Read the Manual. The manual is no fluffy faux-friendly 'Getting Started' guide; it reads like a datasheet with schematics in glorious abundance.

There is mention of a 'Boot button', where if held down and the beaglebone is power-cycled it will force a boot from sdcard. But it did not work - and eventually always reverted to the old eMMC wheezy Debian.

After wasting a couple of days ruling out a hardware malfunction, the problem had to be the Debian image. One hint was that Debian Image 2015-03-01 would not mount  Debian 2020-04-06. That would point to an ext3 filesystem incompatibility. The existing eMMC Debian code would be needed to mount and boot the new Debian before flashing can commence.

And yet all this  has been solved before; one compromise is to use an fossil filesystem (like FAT16) just for booting, and indeed Debian 015-03-01 had such a partition but not Debian 2020-04-06. Usually SoCs have separate bootrom to prevent bricking incidents like this (like its close cousin the Beaglebone White), but this is easy enough to test.

beaglebard.org maintains a complete archive of old images, and hoping for an intermediate Debian that will be compatible with both, I picked Debian 2016-12-09. Then it s a simple matter of:

$xzcat bone-debian-8.6-iot-armhf-2016-12-09-4gb.img.xz > /dev/sdc

And it booted from microsd, just like that.

root@beaglebone:~# cat /proc/version

Linux version 4.4.36-ti-r72 (root@a2-imx6q-wandboard-2gb) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 SMP Wed Dec 7 22:29:53 UTC 2016

You need to prepare the new image for flashing. Just find the file /boot/uEnv.txt and uncomment the last line:

cmdline=init=/opt/scripts/tools/eMMC/init-eMMC-flasher-v3.sh

To flash the eMMC, I added the 5V power cable, and powered off. Then with the microsd still in, held down 'User boot'  button and powered back on. This worked right off the bat; the blinkenlights did an impression of Pong, and when finished, all lit up, then shut down.

On powering up, and removing the microsd, I get:

root@beaglebone:~# cat /etc/dogtag

BeagleBoard.org Debian Image 2016-12-09

Our imugi is not yet a dragon, but this is clearly The Way. The trusty sdcard is then repurposed with:

$xzcat bone-debian-10.3-iot-armhf-2020-04-06-4gb.img.xz | sudo dd of=/dev/sdc

And now it booted off the microsd. Now all I need to do is to repeat the eMMC flashing process with the latest Debian Image, 2020-04-06. As a precaution I first upgraded without incident:

root@beaglebone:~# apt-get update

root@beaglebone:~# apt-get upgrade

Changed the default passwords did the usual sysadmin stuff. The eMMC flash went without incident, and unlike Zen Cho's Byam, my Beaglebone Black did not turn back but transformed its eMMC to Debian 2020-04-06.

Happy Trails.