Friday 27 April 2018

A clock that never needs to be set- ESP8266 WiFi-synchronized LED Clock with MAX7219

"Time, it needs time ..." Scorpions, Still Loving You
I like LED clocks. Waking up at night, you just need to glance at it for the time. No fumbling with the light switch, or peering at the radioactive hands of a mechanical clock. That way I have a much better chance of going back to sleep.

LED Clock


A smartphone works just the same except I have to keep moving it and it may not be there when I need the time. An LCD clock is much less legible. Or if the backlight is turned up, it is too bright as a bedside clock.

The trouble with an LED clock is they are not expensive enough to be made very accurate. Typically a quartz clock will be off by some 4 minutes a year. It gets worse if they lose mains power. They are battery-backed, and they keep the time, but with much less accuracy. Over a few power outtages, you can get an outrageously-wrong clock.

Ideally, my LED clock should synchronize itself over the Internet, much like my smartphone. Now if I can keep the large LED segment displays and replace the  electronics I would get my ideal clock.

The ESP8266 with a MAX7219 are perfect for this. The idea is to get the system working, then remove the MAX7219 7-segment displays and wire it to the much larger LEDs in the clock. There should be no trouble fitting an ESP-12E and MAX7219 into the clock.  

I would need a battery-backed 5V for the ESP8266. A lithium power bank circuit would do quite nicely. There is no need to set time as the unit will automatically correct itself once WiFi is available.

Earlier, I had a MAX7219 working with an ESP-12E NodeMCU ESP8266. The Arduino IDE has sample code for the WiFi time, ESP8266 NTPClient. I just need to paste my MAX7219 SPI code into it. No wiring needed.

ESP-12E ESP8266 Nodemcu with MAX7219


And it was so. In just one hour I have a self-correcting WiFi LED clock. The battery-backed power circuit still need to be designed and tested, the software may well need to be changed, and the ESP8266 does not boot up unless the MAX7219 is first disconnected, and last but not least it whole thing needs to be stuffed into the LED clock. 

It was over so quickly, it was only later I found there is already an existing ESP8266 project that does exactly that, courtesy of sebouh.

An Internet of Things for Industry sometimes require accurate reliable time, especially when it is part of a distributed control system. This would be a cheap way of adding NTP to an embedded control system.

There you have it, ESP8266 WiFi-synchronized LED Clock with MAX7219. Happy Trails.

/*
 * TimeNTP_ESP8266WiFi.ino
 * Example showing time sync to NTP time source
 *
 * This sketch uses the ESP8266WiFi library
 */

#include <TimeLib.h>
#include <ESP8266WiFi.h>
#include <WiFiUdp.h>
#include <SPI.h>

class ESP_MAX7219
{
private:
    uint8_t _ss_pin;
    void SS_High()
    {
        digitalWrite(_ss_pin, HIGH);
    }
    void SS_Low()
    {
        digitalWrite(_ss_pin, LOW);
    }

public:
    ESP_MAX7219(uint8_t pin):_ss_pin(pin) {}
    void begin()
    {
        pinMode(_ss_pin, OUTPUT);
        SS_Low();
        delay(1000);
        SS_High(); // Default to high
    }

    void writeData(const uint8_t MSB, const uint8_t LSB)
    {
        SS_Low();
        SPI.transfer(MSB);
        SPI.transfer(LSB);
        SS_High();
    }
};

ESP_MAX7219 esp(SS);


void send(const uint8_t msb, const uint8_t lsb)
{
  esp.writeData(msb, lsb);
  delay(10);
}
void init_MAX7219(void)
{
    send(0x0F, 0x00);
    delay(100);
    Serial.println("Test mode off");
    send(0x09, 0xFF);
    Serial.println("BCD Mode all 8 digits");
    delay(100);
    send(0x0A, 0x03);
    Serial.println("Quarter brightness");
    delay(100);
    send(0x0B, 0x07);
    Serial.println("8 Digit Scan");
    delay(100);
    send(0x0C, 0x01); // Normal Operation
    Serial.println("Shutdown off");
    delay(100);
}

uint8_t put_bcd(const uint8_t bcd)
{
  if (bcd>9)
    return 0x00;
  send(0x08, bcd);
  delay(10);
  return 0x01;
}

static const uint8_t bcd100_table[] = {
  0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09,
  0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19,
  0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29,
  0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39,
  0x40, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49,
  0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59,
  0x60, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69,
  0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79,
  0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89,
  0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99,
};

uint8_t put_bcd100(const uint8_t bcd)
{
  uint8_t c;

  if (bcd>99)
    return 0x00;
  c= bcd100_table[bcd];
  send(0x01, (c & 0x0F)); // Display LSN
  delay(10);
  send(0x02, (c>>4) & 0x0F); // Display MSN
  delay(10);
  return 0x01;
}
uint8_t put_bcd10000(const uint8_t bcd)
{
  uint8_t c;

  if (bcd>99)
    return 0x00;
  c= bcd100_table[bcd];
  send(0x03, (c & 0x0F)); // Display LSN
  delay(10);
  send(0x04, (c>>4) & 0x0F); // Display MSN
  delay(10);
  return 0x01;
}

uint8_t put_bcd1000000(const uint8_t bcd)
{
  uint8_t c;

  if (bcd>99)
    return 0x00;
  c= bcd100_table[bcd];
  send(0x05, (c & 0x0F)); // Display LSN
  delay(10);
  send(0x06, (c>>4) & 0x0F); // Display MSN
  delay(10);
  return 0x01;
}

void blank_display(void)
{
  send(0x01, 0x0F);
  delay(10);
  send(0x02, 0x0F);
  delay(10);
  send(0x03, 0x0F);
  delay(10);
  send(0x04, 0x0F);
  delay(10);
  send(0x05, 0x0F);
  delay(10);
  send(0x06, 0x0F);
  delay(10);
  send(0x07, 0x0F);
  delay(10);
  send(0x08, 0x0F);
  delay(10);
}
const char ssid[] = "****************";  //  your network SSID (name)
const char pass[] = "*****************";       // your network password
// NTP Servers:
static const char ntpServerName[] = "pool.ntp.org";
//static const char ntpServerName[] = "time.nist.gov";
//static const char ntpServerName[] = "time-a.timefreq.bldrdoc.gov";
//static const char ntpServerName[] = "time-b.timefreq.bldrdoc.gov";
//static const char ntpServerName[] = "time-c.timefreq.bldrdoc.gov";

const int timeZone = +8;     // Central European Time
//const int timeZone = -5;  // Eastern Standard Time (USA)
//const int timeZone = -4;  // Eastern Daylight Time (USA)
//const int timeZone = -8;  // Pacific Standard Time (USA)
//const int timeZone = -7;  // Pacific Daylight Time (USA)


WiFiUDP Udp;
unsigned int localPort = 2390;  // local port to listen for UDP packets

time_t getNtpTime();
void digitalClockDisplay();
void printDigits(int digits);
void sendNTPpacket(IPAddress &address);

void setup()
{
  Serial.begin(115200);
  Serial.println();
  Serial.println();
  Serial.println("TimeNTP Example");

delay(3000);
  Serial.println("Initializing SPI");
  SPI.begin(); // Delay sending output to MAX7219 to prevent it leaching power
  esp.begin();
  Serial.println("Initializing MAX7219 - 1st attempt");
  init_MAX7219();
  blank_display();
  put_bcd100(1);
  delay(3000);
  Serial.println("Initializing MAX7219- 2nd attempt");
  init_MAX7219();
  blank_display();
  put_bcd100(2);
  delay(3000);
  Serial.println("Initializing MAX7219- 3rd attempt");
  init_MAX7219();
  blank_display();
  put_bcd100(3);
  delay(3000);
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, pass);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("IP number assigned by DHCP is ");
  Serial.println(WiFi.localIP());
  Serial.println("Starting UDP");
  Udp.begin(localPort);
  Serial.print("Local port: ");
  Serial.println(Udp.localPort());
  Serial.println("waiting for sync");
  setSyncProvider(getNtpTime);
  setSyncInterval(300);

}

time_t prevDisplay = 0; // when the digital clock was displayed

uint8_t hours = 0;
uint8_t mins = 0;
uint8_t loop_ctr = 0;
void loop()
{
  if (timeStatus() != timeNotSet) {
    if (now() != prevDisplay) { //update the display only if time has changed
      prevDisplay = now();
  hours = hour();
  mins = minute();
  loop_ctr = second();
  put_bcd100(loop_ctr);
  put_bcd10000(mins);
  put_bcd1000000(hours);
    }
  }
}

void digitalClockDisplay()
{
  // digital clock display of the time
  Serial.print(hour());
  printDigits(minute());
  printDigits(second());
  Serial.print(" ");
  Serial.print(day());
  Serial.print(".");
  Serial.print(month());
  Serial.print(".");
  Serial.print(year());
  Serial.println();
}
{
  // utility for digital clock display: prints preceding colon and leading 0
  Serial.print(":");
  if (digits < 10)
    Serial.print('0');
  Serial.print(digits);
}

/*-------- NTP code ----------*/

const int NTP_PACKET_SIZE = 48; // NTP time is in the first 48 bytes of message
byte packetBuffer[NTP_PACKET_SIZE]; //buffer to hold incoming & outgoing packets

time_t getNtpTime()
{
  IPAddress ntpServerIP; // NTP server's ip address

  while (Udp.parsePacket() > 0) ; // discard any previously received packets
  Serial.println("Transmit NTP Request");
  // get a random server from the pool
  WiFi.hostByName(ntpServerName, ntpServerIP);
  Serial.print(ntpServerName);
  Serial.print(": ");
  Serial.println(ntpServerIP);
  sendNTPpacket(ntpServerIP);
  uint32_t beginWait = millis();
  while (millis() - beginWait < 1500) {
    int size = Udp.parsePacket();
    if (size >= NTP_PACKET_SIZE) {
      Serial.println("Receive NTP Response");
      Udp.read(packetBuffer, NTP_PACKET_SIZE);  // read packet into the buffer
      unsigned long secsSince1900;
      // convert four bytes starting at location 40 to a long integer
      secsSince1900 =  (unsigned long)packetBuffer[40] << 24;
      secsSince1900 |= (unsigned long)packetBuffer[41] << 16;
      secsSince1900 |= (unsigned long)packetBuffer[42] << 8;
      secsSince1900 |= (unsigned long)packetBuffer[43];
      return secsSince1900 - 2208988800UL + timeZone * SECS_PER_HOUR;
    }
  }
  Serial.println("No NTP Response :-(");
  return 0; // return 0 if unable to get the time
}

// send an NTP request to the time server at the given address
void sendNTPpacket(IPAddress &address)
{
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12] = 49;
  packetBuffer[13] = 0x4E;
  packetBuffer[14] = 49;
  packetBuffer[15] = 52;
  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer, NTP_PACKET_SIZE);
  Udp.endPacket();
}

Wednesday 25 April 2018

The SPI who came in from the cold

The film was based on John Le Carre's novel of the same name
“Don't give it to them all at once, make them work for it. Confuse them with detail, leave things out, go back on your tracks" -  John le CarrĂ©The Spy Who Came In from the Cold

I had always wanted a wall clock which self-corrects from the Internet using NTP. A few weeks ago I bought a cheap MAX7219 8-digit 7-segment display. It seems a perfect match for the ESP8266, which will access NTP from my home WiFi. The MAX7219 uses SPI which should link to the ESP-12E and can then be scaled up to wall clock size.

Straightforward project, perfect for IoT. It started well- I linked two ESP-12E using SPI and the sample programs in Arduino IDE. That worked a treat. I may not even need to learn the Lua programming language. Then real life intervened.

It did not work. The Lua sample code had worked when I used ESP-12Es as master and slave. Did I get a bad MAX7219? It certainly was a cheap clone from a no-name China supplier. Was it the level translator, the Lua program or the ESP-12E? 

It was weird- at times it seems to start working (the display comes on) only when I power it off.It seemed to work even when the MAX7219 did not have power - the MAX7219 seemed to be able to leech enough power from its signal lines. Randomly disconnecting a signal cable sometimes produced a fleeting glimpse of a working display.

But after three days, I had to admit- I was stuck. But what is really needed is an admission that the easy way failed. 
I needed to know whether it is a problem with the MAX7129 or the software.One way was to get the MAX7219 working with different software. The other would be to buy another MAX7219 module, preferably from a different supplier. To rule out the ESP-12E's flakey start I need a different controller. 

I had SPI source code for the PIC18F14K50 CPU from Microchip- it came with their development kit. I even have a finished PIC18F14K50 prototype PCB I designed myself some years ago but had not gotten round to testing it.


Testing the SPI PIC18F14K50. Being USB devices they came up in Linux as /dev/ttyACM0 and /dev/ttyACM1

The SPI PIC18F14K50 and their software (sample code from the Microchip MCC18 compiler) did not work either. Time to back off SPI. The PIC18F14K50's SPI pins can be reprogrammed to use the I2C protocol. Maybe I will have better luck - at least it will prove I have working PIC18F14K50.

Fell back to I2C. Did not work either! It is starting to look like I am going backwards. Time to bring up the Death Star, my oscilloscope. Nothing gets by the oscilloscope. I love my scope, but just setting it up takes one day. It forces me to take things real slow, step by steady step, soldering test points, checking voltages pin by pin. Oh, and you now have to read the datasheets. Ugh. Buzzkill. Time to RTFM

And true enough, the scope showed problem is the MCC18 code. It does not work. That is quickly fixed, and now I2C between two PIC18F14K50 work. The SPI worked soon after, but the datasheets said there is no real standard for SPI, there can be 4 variants. The timing cycles for the PIC18F14K50 and the MAX7219 needs to be carefully examined for a match.

The 4 SPI modes supported by the PIC18F14K50
Luckily, there is a match: the second mode in the PIC18F14K50 matches:

MAX7219 Timing Diagram. 

Set up the PIC18F14K50 with the MAX7219. It did not work! In fact the symptoms are much the same as the ESP-12E, the display flashes working on power down, the MAX7219 starting up without power. But this time I have the scope. Turned out the PCB labelling was wrong- SPI MISO and MOSI were swapped round. And more MCC18 code needed to be replaced, but eventually it worked:



MAX7219 working with the PIC18F14K50

It's time to get back to the source of my grief- the software in the ESP-12E sample code. Now this is not a reflection on the software quality or the author- SPI simply had too many variants for the code to work right off the bat. Plus the author was interfacing to his SPI device using 32-bit SPI cycles. The MAX7219 needed 16-bit cycles.


Bottom: working SPI  on ESP-12E and MAX7219. Top: the mighty Death Star

You can get lucky most times and have your project work just using Internet resources. Indeed, you should and must use the optimistic approach first. But sometimes you just have to get out the Death Star and do the slow but steady steamroller. 



Later, I found the LEDControl already has working software for the ESP8266. It had not come up with my Google searches becasue it did not mention SPI, just the MAX7219.

The ESP-12E SPI interface to MAX7219 has had twists and turns worthy of a John Le Carre movie. Such is life. Happy Trails. 

Tuesday 24 April 2018

Using SPI to interface the ESP-12E to the MAX7219 LED display (updated 2019-03-24)

From top: ESP-12E Nodemcu, middle: Level Converter, bottom: MAX7219 8-digit 7-segment display
You will need an ESP-12E NodeMCU ESP8266, a 4-channel level converter and a MAX7219 7-segment LED display.

2019-03-24 update: it was difficult to start the ESP-12E program, and the problem is that SS, or D8 or GPIO 15 must be pulled low on power up. The ESP-12E has a voltage divider doing this but when the voltage level converter is plugged in the this did not work well. To fix this, I used GPIO 12 or D6 instead of D8. Now D8 is usually used as SPI  MISO but the MAX7219 does not use it.

 The modified wiring is now as follows:
   GPIO    NodeMCU   Name  |   MAX7219
   ===================================
     12       D6                MISO  |   CS
     13       D7                MOSI  |   DIN
     15       D6                SS        |   Not connected
     14       D5                SCK    |   CLK

Since the MAX7219 is a 5V device and the ESP-12E a 3.3V you will need a level converter in between. I used one based on 4 MOSFETs, covered in some detail here.

The MAX7219 draws quite a bit of power so it is advisable to have a separate power source from the ESP-12E. I used a 3A D-Link USB 3.0 hub.

I based my program on the sample code in the NodeMCU Lua sample code 'SPISlave_SafeMaster'  that came with the Arduino IDE.

With the level converter connected, the ESP-12E would not program, so I ended up having to unplug the 3.3V side of the level converter before programming.

The program merely initializes the MAX7219, displays '12345678', then triggers the MAX7219's built-in test function (lights up everything with maximum brightness). The built-in test requires a relatively high-ampere 5V supply.

The program is a Lua sketch, and requires the Arduino IDE. I used a custom-built Slackware Linux version, but there are stock Linux and Windoes versions that work the same.

The program is:

    SPI Safe Master Demo Sketch
    Connect the SPI Master device to the following pins on the esp8266:

*/
#include <SPI.h>

class ESP_MAX7219
{
private:
    uint8_t _ss_pin;
    void SS_High()
    {
        digitalWrite(_ss_pin, HIGH);
    }
    void SS_Low()
    {
        digitalWrite(_ss_pin, LOW);
    }
 
public:
    ESP_MAX7219(uint8_t pin):_ss_pin(pin) {}
    void begin()
    {
        pinMode(_ss_pin, OUTPUT);
        SS_High(); // Default to high
    }

    void writeData(const uint8_t MSB, const uint8_t LSB)
    {
        SS_Low();
        SPI.transfer(MSB);      
        SPI.transfer(LSB);
        SS_High();
    }
};

ESP_MAX7219 esp(12); // 2019-03-24 changed from ESP_MAX7219 esp(SS);

 
void send(const uint8_t msb, const uint8_t lsb)
{
    esp.writeData(msb, lsb);
    delay(10);
    // Serial.print("Slave: ");
    // Serial.println(esp.readData());
    // Serial.println();
}

void setup()
{
    Serial.begin(115200);
    SPI.begin();
    esp.begin();
    delay(1000);
    send(0x0C, 0x01); // Normal Operation
}

void loop()
{
    delay(1000);
    send(0x0C, 0x01); // Normal Operation
    Serial.println("Shutdown off");
    delay(1000);
    send(0x09, 0xFF);
    Serial.println("BCD Mode all 8 digits");
    delay(1000);
    send(0x0A, 0x03);
    Serial.println("Quarter brightness");
    delay(1000);
    send(0x0B, 0x07);
    Serial.println("8 Digit Scan");
    delay(1000);
    send(0x0F, 0x01);
    Serial.println("Test mode on");
    delay(1000);
    send(0x0F, 0x00);
    Serial.println("Test mode off");
    delay(1000);
    send(0x01, 0x08);
    Serial.println("Digit 8");
    send(0x02, 0x07);
    Serial.println("Digit 7");
    send(0x03, 0x06);
    Serial.println("Digit 6");
    send(0x04, 0x05);
    Serial.println("Digit 5");
    send(0x05, 0x04);
    Serial.println("Digit 4");
    send(0x06, 0x03);
    Serial.println("Digit 3");
    send(0x07, 0x02);
    Serial.println("Digit 2");
    send(0x08, 0x01);
    Serial.println("Digit 1");

    /* send(0x0C, 0x00);
    delay(3000);
    Serial.println("Shutdown on"); */
 
    send(0x0C, 0x01);
    Serial.println("Shutdown off");
    delay(3000);
}

There are plenty of other projects out there using the MAX7219, but I could not find one that uses the ESP-12E (or ESP8266). The MAX7219 uses SPI which seems to be gradually being replaced by I2C. This has been tested and worked for me, but turned out to be harder than I thought, but that is another blog post.

Happy Trails.

Sunday 8 April 2018

The rtlizer: low-cost spectrum analyzer using RTL2832U SDR

The Equalizer: 1985 TV Series
"Got a problem? Odds against you? Call the Equalizer. 212 555 4200." - The Equalizer

A spectrum analyzer is a serious piece of kit for an electronics engineer. You can easily spend over RM10,000 on one, although you can get generic ones online for RM4000. The RTL2832U DVB-T USB dongle SDR can be used to make a basic spectrum analyzer for RM39.

It is the great equalizer (after the 1985 TV Series), leveling the playing field for students, hobbyists and Third World engineers.

OZ9AEC has made an everyman's spectrum analyzer using a Raspberry Pi and the RTL2832U dongle. Happily, his design runs without modification on my Slackware 14.2 laptop.

rtlizer required rtl-sdr, which I had previously installed. Here we just need to tell rtlizer where to find it:

$export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig

And add '/usr/local/lib' to the file /etc/ld.so.conf. If you had just installed rtl-sdr it helps to also do:

$ldconfig

I downloaded rtlizer, and built it thus:

$unzip rtlizer-master.zip
$rtlizer/rtlizer-master/src$make

If necessary, unload (or even better, blacklist) the dongle's kernel drier:
$modprobe -vr dvb_usb_rtl28xxu
rmmod dvb_usb_rtl28xxu
rmmod dvb_usb_v2
rmmod rc_core

And it runs, right out of the box (with a complaint or two):

$./rtlizer 1024x720

(rtlizer:22436): Gtk-WARNING **: gtk_window_parse_geometry() called on a window w
ith no visible children; the window should be set up before gtk_window_parse_geom
etry() is called.
window size: 1024x720 pixels
SCALE: 7.20 / Y0: 216 / TXTMARG: 21
Found Fitipower FC0012 tuner

rtlizer responds to the the arrow keys. And wneh I centered rtlyzer on 330MHz, my autogate remote's frequency, I got the spike in the center, with the upper and lower sidebands when the button was pressed:

rtlizer centered on 330MHz
There you have it: an RM39 spectrum analyzer, courtesy of OZ9AEC. Happy Trails.

Software Defined Radio: installing rtl-sdr in Slackware 14.2

RTL2832U 
I got a generic no-name RTL2832U TVB-T USB dongle for just RM39. Plug it into a Slackware 14.2 with a recent kernel (I have 4.16.0) and if you did a 'dmesg -T':

[Fri Mar 30 00:12:58 2018] usb 1-2: new high-speed USB device number 57 using xhci_hcd
[Fri Mar 30 00:12:58 2018] usb 1-2: New USB device found, idVendor=0bda, idProduct=2838
[Fri Mar 30 00:12:58 2018] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Fri Mar 30 00:12:58 2018] usb 1-2: Product: RTL2838UHIDIR
[Fri Mar 30 00:12:58 2018] usb 1-2: Manufacturer: Realtek
[Fri Mar 30 00:12:58 2018] usb 1-2: SerialNumber: 00000001
[Fri Mar 30 00:12:59 2018] usb 1-2: dvb_usb_v2: found a 'Realtek RTL2832U reference design' in warm state
[Fri Mar 30 00:12:59 2018] usb 1-2: dvb_usb_v2: will pass the complete MPEG2 transport stream to the software demuxer
[Fri Mar 30 00:12:59 2018] dvbdev: DVB: registering new adapter (Realtek RTL2832U
 reference design)
[Fri Mar 30 00:12:59 2018] i2c i2c-11: Added multiplexed i2c bus 12
[Fri Mar 30 00:12:59 2018] rtl2832 11-0010: Realtek RTL2832 successfully attached
[Fri Mar 30 00:12:59 2018] usb 1-2: DVB: registering adapter 0 frontend 0 (Realtek RTL2832 (DVB-T))...
[Fri Mar 30 00:12:59 2018] i2c i2c-12: fc0012: Fitipower FC0012 successfully identified
[Fri Mar 30 00:12:59 2018] lirc_dev: IR Remote Control driver registered, major 237
[Fri Mar 30 00:12:59 2018] IR LIRC bridge handler initialized
[Fri Mar 30 00:12:59 2018] Registered IR keymap rc-empty
[Fri Mar 30 00:12:59 2018] rc rc0: Realtek RTL2832U reference design as /devices/
pci0000:00/0000:00:14.0/usb1/1-2/rc/rc0
[Fri Mar 30 00:12:59 2018] input: Realtek RTL2832U reference design as /devices/p
ci0000:00/0000:00:14.0/usb1/1-2/rc/rc0/input132
[Fri Mar 30 00:12:59 2018] rc rc0: lirc_dev: driver ir-lirc-codec (dvb_usb_rtl28xxu) registered at minor = 0
[Fri Mar 30 00:12:59 2018] usb 1-2: dvb_usb_v2: schedule remote query interval to
 200 msecs
[Fri Mar 30 00:13:00 2018] usb 1-2: dvb_usb_v2: 'Realtek RTL2832U reference design' successfully initialized and connected
[Fri Mar 30 00:13:00 2018] usbcore: registered new interface driver dvb_usb_rtl28xxu

            This means the device is working and ahs been recognized by the kernel:
root$ls -lt /dev/dvb
total 0
drwxr-xr-x 2 root root 120 Apr  2 09:33 adapter0

The SDR software of choice for Linux Slackware 14.2 has got to be rtl-sdr. I simply followed the steps for Linux listed in their 'Quick Start Guiide', and are relisted here as tested steps for Slackware 14.2.

My download was from osmocom.

~/sdr/rtl-sdr$unzip rtl-sdr-master.zip
~/sdr/rtl-sdr/rtl-sdr-master$autoreconf -i
~/sdr/rtl-sdr/rtl-sdr-master$./configure
~/sdr/rtl-sdr/rtl-sdr-master$make
~/sdr/rtl-sdr/rtl-sdr-master$su -c "make install"

Since rtl-sdr is mainly a library; at this point it will list some instructions on linking to it:

----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

~/sdr/rtl-sdr/rtl-sdr-master$su -c "ldconfig"

Now you will need to remove (or better yet blacklist) the kernel driver for RTL2832U:

root@aspireF15:/home/heong/sdr$modprobe -vr dvb_usb_rtl28xxu
rmmod dvb_usb_rtl28xxu
rmmod dvb_usb_v2
rmmod rc_core

A quick test (Ctrl-C to exit):

$rtl_test
Found 1 device(s):
  0:  Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM
Found Fitipower FC0012 tuner
Supported gain values (5): -9.9 -4.0 7.1 17.9 19.2
Sampling at 2048000 S/s.

Info: This tool will continuously read from the device, and report if
samples get lost. If you observe no further output, everything is fine.

Reading samples in async mode...
Signal caught, exiting!
^CSignal caught, exiting!

User cancel, exiting...
Samples per million lost (minimum): 0

The next command to use is rtl_power. Aim it at some known frequency, perhaps your local FM radio station. In my case it aimed it at my autogate remote at 330MHz:

$rtl_power -f 320000000:340000000:1000 -e 10m -g 40 -i 1s persona.csv
Number of frequency hops: 8
Dongle bandwidth: 2500000Hz
Downsampling by: 1x
Cropping by: 0.00%
Total FFT bins: 32768
Logged FFT bins: 32768
FFT bin size: 610.35Hz
Buffer size: 16384 bytes (3.28ms)
Reporting every 1 seconds
Found 1 device(s):
  0:  Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM
Found Fitipower FC0012 tuner
Tuner gain set to 19.20 dB.
Exact sample rate is: 2500000.107620 Hz

User cancel, exiting...

This produces a massive data file persona.csv.

Now we need to visualize the data. keenerd has a simple python script, heatmap.py. Download it, rename it to heatmap.py and make it executable:
$chmod +x heatmap.py

We aim it at persona.csv:
$./heatmap.py --offset 1000 --ytick 0.001s --palette charolastra persona.csv persona.png

The output is an image file persona.png, which looked something like this:

x-axis is frequence, y-axis is time. The 8 red smudges correspond to two buttons on the autogate remote, 4 presses each

There you have it: instant gratification SDR. 

Happy Trails.

Friday 6 April 2018

Software Radio Ga Ga

Philips transistor radio, circa 1970

"I'd sit alone and watch your light

My only friend through teenage nights
And everything I had to know
I heard it on my radio ...


You had your time, you had the power
You've yet to have your finest hour
Radio, radio" - Queen, Radio Ga Ga 

Back in 1974, Mum bought a transistor radio, and it was a technical marvel. Older radios used vacuum tubes, took minutes to warm up, failed often and were the size of a coffee table. The new radio lasted over 10 years despite being knocked off the mantelpiece more than once.

Eventually I learned in Uni this marvel is of a new superheterodyne design. And such a satisfying word it was- it rolls off your tongue guaranteed to impress those 1960s era dinosaurs still clinging to their relay logic computers.

Transistor Radio
  

Superheterodyne receiver
Radio signal at various stages
Then in 1992 there was this incredible IEEE paper "Software Radio: Survey, Critical Analysis and Future Directions" by Joe Mitola, describing a radio largely implemented in software, a Software Defined Radio, and everything changed again.
Software allowed the audio signal to be digitally encrypted; security is now much stronger. The military was an enthusiastic early adopter. It was also more resistant to jamming, used its radio spectrum much more efficiently, and more than one radio protocol can be supported. This was a great boon to cellphones, especially the base stations. 

But SDR was not really my specialty - it was far removed from the superheterodyne of yore, and so for the next 20 years SDR remained an exotic technical curiosity, extolled in the IEEE. 

In 1998 Digital Terrestrial Television DVB-T started replacing regular analog TV worldwide. Realtek, a Taiwan IC maker introduced the RTL2832U, and IC meant for DVB-T. In 2010 Eric Fry, while writing a Linux driver for the RTL2832U discovered that it was possible to read raw I/Q data from it. This meant the RTL2832U-based TVB-T USB dongle can be used as a cheap general-purpose SDR. 

The RM39 RTL2832-based TVB-T USB Dongle au naturel


And everything changed again. Like the superheterodyne, SDR is suddenly within reach.

In the next few posts I shall show how to install SDR software for Slackware 14.2. Some of the uses for SDR are a low-cost spectrum analyzer, lightning detector and autogate remote hacker. All far better things to do than watching TV with it. 

Life is good- Happy trails.