Monday, 6 August 2018

RS-485 Modbus IoT Gateway using ESP8266 NodeMCU ESP-12E: TCP/IP Slave Part 2 of 3

Clockwise from top: USB3 externally-powered hub, RS-485 dongle, RS-485 to TTL serial PCB and ESP-12E ESP8266 NodeMCU module

Modbus TCP/IP to RS-485 passthrough gateway for RM25.


Part 1 describes an ESP-12E Modbus RTU Master using RS-485 interface. It can read and write RS-485 Modbus devices but it uses the ESP-12E debug serial port to do so. This works if the ESP-12E itself is the host controller, but Modbus masters usually have a lot more horsepower.



To be really useful, we can also make the ESP-12E a TCP/IP Modbus Slave. It still can work as a host, but this will make it a Modbus TCP/IP to RS485 "passthrough" gateway. A real Modbus host, say a desktop or Industrial PC can then orchestrate a whole bunch of Modbus devices to control a whole buildings' services.

A real TCP/IP to RS-485 Modbus gateway is some RM680

Of course the ESP-12E is not as powerful as a regular Modbus TCP/IP to RS485 passthrough but at RM25 it is an ideal way of retrofitting IoT functionality to Modbus devices.

If we replace the desktop with a cloud-based server we can scale  
As usual someone has already provided the required code. yaacov's ModbusSlaveTCP made an excellent template. I downloaded it as a zip file and copied yaacov's files from ArduinoModbusSlaveTCP-master/src into my Arduino IDE directory <your Linux account>/Arduino/libraries/ModbusSlaveTCP/

Then I expanded the sketch in Part 1:

#include <ESP8266WiFi.h>
#include <ModbusSlaveTCP.h>

const char* ssid = "YourAccessPoint";
const char* pass = "StrongPassword";
IPAddress staticIP(10,0,0,100);
IPAddress gateway(10,0,0,1);
IPAddress subnet(255,255,255,0);


// slave id = 1, rs485 control-pin = 8, baud = 9600
#define SLAVE_ID 1
// Modbus object declaration
ModbusTCP slave(SLAVE_ID);

#include <ModbusMaster232.h>
#include <SoftwareSerial.h>  // Modbus RTU pins   D7(13),D8(15)   RX,TX
// MAX485 half duplex control lines
#define not_RE 14 // D5. Enable receiver, active low
#define DE 12 // D6  Enable Transmitter, active high

// Instantiate ModbusMaster object as slave ID 1
  ModbusMaster232 node(1);

void setup() {
  pinMode(not_RE, OUTPUT);
  pinMode(DE, OUTPUT);
  // default to transmit mode to reduce noise
  digitalWrite(not_RE, HIGH); // disable receiver
  digitalWrite(DE, HIGH); // enable transmitter
 
  Serial.begin(9600);
  delay(100);
  node.begin(9600);  // Modbus RTU
  delay(100);
 
   
    /* Connect WiFi to the network
     */
    Serial.print("Connecting to ");
    Serial.println(ssid);
    WiFi.disconnect();
    WiFi.hostname("ModbusTCPslaveRS485master");
    WiFi.mode(WIFI_STA);
    WiFi.begin(ssid, pass);
    WiFi.config(staticIP, gateway, subnet); // Static IP. Not required for dhcp

    int wifi_loop = 0;
    while (WiFi.status() != WL_CONNECTED) {
        delay(1000);
        Serial.print(".");
        /*
        if (wifi_loop++ == 10)
        {
            Serial.println("Reconnecting ...");
            WiFi.disconnect();
            WiFi.mode(WIFI_STA);
            WiFi.begin(ssid, pass);
            delay(1000);
            wifi_loop = 0;
        }
        */
    }
    Serial.println(WiFi.localIP());
    /* register handler functions
     * into the modbus slave callback vector.
     */
    slave.cbVector[CB_WRITE_COIL] = writeDigitlOut;
    slave.cbVector[CB_READ_DISCRETE_INPUT] = readDigitalIn; //    
    slave.cbVector[CB_READ_COILS] = readDigitalIn;
    slave.cbVector[CB_READ_REGISTERS] = readAnalogIn;
    slave.cbVector[CB_WRITE_MULTIPLE_REGISTERS] = writeAnalogOut; // cmheong
     
    /* start slave and listen to TCP port 502
     */
    slave.begin();
   
    // log to serial port
    Serial.println("");
    Serial.print("Modbus ready, listen on ");
    Serial.print(WiFi.localIP());
    Serial.println(" : 502");
}

int loop_i = 0;
uint16_t readDiscreteInputs[10];
int Mdelay = 10; // from 5


void loop() {

  /*
  node.readDiscreteInputs(loop_i, 1);
  readDiscreteInputs[loop_i] = node.getResponseBuffer(0);
  node.clearResponseBuffer();
  Serial.print("[");
  Serial.print(loop_i);
  Serial.print("] ");
  Serial.print(readDiscreteInputs[loop_i]);
  if (++loop_i >= 10)
  {
    Serial.println("");
    loop_i = 0;
  }
  */
  // delay(Mdelay); // no need for delay(5) since we print 5 char at 9600

    /* listen for modbus commands con serial port
     *
     * on a request, handle the request.
     * if the request has a user handler function registered in cbVector
     * call the user handler function.
     */
  slave.poll();
}

/**
 * Handel Force Single Coil (FC=05)
 * set digital output pins (coils) on and off
 */
void writeDigitlOut(uint8_t fc, uint16_t address, uint16_t status) {
    digitalWrite(address, status);
}

/**
 * Handel Read Input Status (FC=02/01)
 * write back the values from digital in pins (input status).
 *
 * handler functions must return void and take:
 *      uint8_t  fc - function code
 *      uint16_t address - first register/coil address
 *      uint16_t length/status - length of data / coil status
 */
void readDigitalIn(uint8_t fc, uint16_t address, uint16_t length)
{
    int data = 0;
 
    // read digital input
    Serial.printf("digital input bytes fc %02x at address %04x length %d data ", fc, address, length);
    node.readDiscreteInputs(address, length);

    for (int i = 0; i <= (length-1)/8; i++) // cmheong 2018-08-06
    {
      data = node.getResponseBuffer(i);
      slave.writeCoilsToBuffer(i, (uint8_t) data); // digitalRead(address + i));
      Serial.printf(" %x", node.getResponseBuffer(i));
    }

    delay(Mdelay);
   
    Serial.println("");
    node.clearResponseBuffer();
}

/**
 * Handel Read Input Registers (FC=04/03)
 * write back the values from analog in pins (input registers).
 */
void readAnalogIn(uint8_t fc, uint16_t address, uint16_t length) {
    // read analog input
    for (int i = 0; i < length; i++) {
        slave.writeRegisterToBuffer(i, analogRead(address + i));
    }
}

// cmheong 2018-07-31 write_registers()
void writeAnalogOut(uint8_t fc, uint16_t address, uint16_t length)
{
    Serial.printf("analog output bytes at address %04x length %d data ", address, length);
    for (int i = 0; i < length; i++)
    {
        Serial.printf("%x ", slave.readRegisterFromBuffer(i));
        Serial.println("");
        node.send(slave.readRegisterFromBuffer(i));
        // node.writeSingleRegister(address, slave.readRegisterFromBuffer(i));
        delay(Mdelay);  
    }
    node.writeMultipleRegisters(address, length);
}
  
I only tested 'Read Discrete Registers' (function code 2) and 'Write Multiple Registers' (function code 16) on a real Modbus RTU device, but you get the idea. yaakov's code did not process function code 2 properly, so I modifiled ModbusSlaveTCP.cpp of his library:

        case FC_READ_DISCRETE_INPUT: // read input state (digital in)
            address = word(bufIn[MLEN + 2], bufIn[MLEN + 3]); // coil to set.
            length = word(bufIn[MLEN + 4], bufIn[MLEN + 5]);

            // sanity check.
            if (length > MAX_BUFFER) return 0;

            // check command length.
            if (lengthIn != (MLEN + 6)) return 0;

            // build valid empty answer.
            lengthOut = MLEN + 3 + (length - 1) / 8 + 1; // cmheong 2018-08-06
            bufOut[MLEN + 2] = length;  // cmheong 2018-08-06

            // clear data out.
            memset(MLEN + bufOut + 2, 0, bufOut[2]);  // cmheong 2018-08-06

            if (cbVector[CB_READ_DISCRETE_INPUT]) // cmheong 2018-08-02
            {
                cbVector[CB_READ_DISCRETE_INPUT](fc, address, length);
            }
            break;

And added a new function:

void ModbusTCP::writeCoilsToBuffer(int offset, uint8_t state)
{
    int address = MLEN + 3 + offset;

    bufOut[address] = state;
}

 The ESP-12E will connect to your WiFi and use a fixed IP address (change it to suit your own address assignments) 10.0.0.100. To test it, I used my laptop to connect to the same WiFi access point. I then modified pymodbus's excellent synchronous-client.py thus:

from pymodbus.client.sync import ModbusTcpClient as ModbusClient
client = ModbusClient('10.0.0.100', method='rtu', port=502) # 2018-07-29

The test code is:
rr = client.read_discrete_inputs(1,1,unit=0x01)
if rr != None :
    print "\nread discrete inputs from", coils, rr.bits, '\n'

rq = client.write_registers(0x1001, [0x001f]*1, unit=0x01)
if rq != None :
    print "\write holding_registers from", 10, rq, '\n'

A sample working output is:

root@aspireF15:/home/heong/EMS/pymodbus/pymodbus-master/examples/current$python ./esp8266-tcpipclient.py 0x01
DEBUG:pymodbus.transaction:Current transaction state - IDLE
DEBUG:pymodbus.transaction:Running transaction 1
DEBUG:pymodbus.transaction:SEND: 0x0 0x1 0x0 0x0 0x0 0x6 0x1 0x2 0x0 0x1 0x0 0x1
DEBUG:pymodbus.client.sync:New Transaction state 'SENDING'
DEBUG:pymodbus.transaction:Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
DEBUG:pymodbus.transaction:Changing transaction state from 'WAITING FOR REPLY' to 'PROCESSING REPLY'
DEBUG:pymodbus.transaction:RECV: 0x0 0x1 0x0 0x0 0x0 0x4 0x1 0x2 0x1 0x7
DEBUG:pymodbus.framer.socket_framer:Processing: 0x0 0x1 0x0 0x0 0x0 0x4 0x1 0x2 0x1 0x7
DEBUG:pymodbus.factory:Factory Response[ReadDiscreteInputsResponse: 2]
DEBUG:pymodbus.transaction:Adding transaction 1
DEBUG:pymodbus.transaction:Getting transaction 1
DEBUG:pymodbus.transaction:Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'

read discrete inputs from 1 [True, True, True, False, False, False, False, False] 

DEBUG:pymodbus.transaction:Current transaction state - TRANSCATION_COMPLETE
DEBUG:pymodbus.transaction:Running transaction 2
DEBUG:pymodbus.transaction:SEND: 0x0 0x2 0x0 0x0 0x0 0x9 0x1 0x10 0x10 0x1 0x0 0x1 0x2 0x0 0x1f
DEBUG:pymodbus.client.sync:New Transaction state 'SENDING'
DEBUG:pymodbus.transaction:Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
DEBUG:pymodbus.transaction:Changing transaction state from 'WAITING FOR REPLY' to 'PROCESSING REPLY'
DEBUG:pymodbus.transaction:RECV: 0x0 0x2 0x0 0x0 0x0 0x6 0x1 0x10 0x10 0x1 0x0 0x1
DEBUG:pymodbus.framer.socket_framer:Processing: 0x0 0x2 0x0 0x0 0x0 0x6 0x1 0x10 0x10 0x1 0x0 0x1
DEBUG:pymodbus.factory:Factory Response[WriteMultipleRegistersResponse: 16]
DEBUG:pymodbus.transaction:Adding transaction 2
DEBUG:pymodbus.transaction:Getting transaction 2
DEBUG:pymodbus.transaction:Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'
Modbus device replied!

There you have it, a Modbus TCP/IP to RS485 passthrough gateway for less than RM25. Slap on an AWS or Google Cloud server and you are ready for a free docker microservice demon Modbus host.

Happy Trails!

Friday, 27 July 2018

RS-485 Modbus IoT Gateway using ESP8266 NodeMCU ESP-12E Part 1 of 3

Clockwise from top: USB 3.0 powered hub, RS-485 USB dongle, Arduino RS-485 to TTL module and NodeMCU ESP-12E ESP8266
Many remote devices, especially for industrial use come with the RS-485 serial interface and use the Modbus protocol. These industrial remote devices are often easily converted to IoT devices using for example, an IoT Gateway that also has RS-485 interfaces.

The Schneider PM1200 3-phase Power Meter has an RS-485 interface and would make a great IoT device.

Schneider PM1200 Power Meter


Fuji FRENIC 15KW Inverter

The ESP8266 is incredibly tempting as such  an IoT gateway. It is low-cost, low-power and the WiFi module allows for a much safer non-galvanic, isolated data connection. Similar devices like the monster 10-30KW Inverters used in escalators, cranes, etc can be monitored safely from the cloud this way.

As usual I used a low-cost (RM3.38) Arduino RS-485 to serial TTL module. Now this is a 5V device and since the ESP8266 is strictly a 3.3V device, we should really use a logic level converter, but at a pinch it should work at 3.3V, especially if I powered  it from the ESP-12E.

I also used a modified version of the Trialcommand ESP8255 Master Modbus RTU (RS232) sketch. This in turn use the SoftwareSerial library, which essentially determined the wiring diagram.

RS-485 to 5V TTL module 

MOSFET-based Logic Level Converter

  The wiring diagram (sans level converter) is:

       ESP12-E    RS-485 PCB
            3V3
            GND
            D5 ------- *RE
            D6 ------- DE
            D7 ------- RO
            D8 ------- DI

NodeMCU ESP-12E Pinout

The wiring really makes sense when you look at the MAX485E IC datasheet:

Notice the MAX485E is a half-duplex device
I made a little ribbon cable with Moles 0.1" pitch Molex connectors:

Cable assembly. Right, top: data & control, right bottom: rs-485 power. Left: ESP-12E
When assembled the RS-485 module looked like this:


Download the libraries and copy them into your Arduino IDE libraries folder.

$ls -lt /root/Arduino/libraries
total 16
drwxr-xr-x 4 root  root  4096 Apr 26 20:44 Time-master
-rw-r--r-- 1 root  root    87 Feb 27 07:20 readme.txt
drwxr-xr-x 2 heong users 4096 Mar 15  2017 ModBusMaster232
drwxr-xr-x 3 heong users 4096 Mar 15  2017 SoftwareSerial

Since the ESP8266-Modbus-RTU-Master library uses full duplex TTL RS-232 I put in a tiny patch to convert it to half duplex required by the MAX485E. The file is ./libraries/ModBusMaster232/ModbusMaster232^Cpp and the change is in boldface at line 735:

#define not_RE 14 // D5. Enable receiver, active low
#define DE 12 // D6  Enable Transmitter, active high

  u16CRC = 0xFFFF;
  for (i = 0; i < u8ModbusADUSize; i++)
  {
  //Function  crc16  for ESP8266   - PDAControl
    u16CRC = _crc16_update2(u16CRC, u8ModbusADU[i]);
  }
  u8ModbusADU[u8ModbusADUSize++] = lowByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize++] = highByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize] = 0;

  // transmit request
  digitalWrite(not_RE, HIGH); // cmheong Set max485 to transmit
  digitalWrite(DE, HIGH);
  for (i = 0; i < u8ModbusADUSize; i++)
  {
                swSer.print(char(u8ModbusADU[i]));
                delay(2); // cmheong 2018-07-26
  }
  u16CRC = 0xFFFF;
  for (i = 0; i < u8ModbusADUSize; i++)
  {
  //Function  crc16  for ESP8266   - PDAControl
    u16CRC = _crc16_update2(u16CRC, u8ModbusADU[i]);
  }
  u8ModbusADU[u8ModbusADUSize++] = lowByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize++] = highByte(u16CRC);
  u8ModbusADU[u8ModbusADUSize] = 0;

  // transmit request
  digitalWrite(not_RE, HIGH); // cmheong Set max485 to transmit
  digitalWrite(DE, HIGH);
  for (i = 0; i < u8ModbusADUSize; i++)
  {
                swSer.print(char(u8ModbusADU[i]));
                delay(2); // cmheong 2018-07-26
  }


I modified the Trialcommand sketch (patches in boldface):

#include <ModbusMaster232.h> 
#include <SoftwareSerial.h>  // Modbus RTU pins   D7(13),D8(15)   RX,TX
// MAX485 half duplex control lines
#define not_RE 14 // D5. Enable receiver, active low
#define DE 12 // D6  Enable Transmitter, active high

// Instantiate ModbusMaster object as slave ID 1
  ModbusMaster232 node(1);

void setup()
{
  pinMode(not_RE, OUTPUT);
  pinMode(DE, OUTPUT);
  // default to transmit mode to reduce noise
  digitalWrite(not_RE, HIGH); // disable receiver
  digitalWrite(DE, HIGH); // enable transmitter
  
  Serial.begin(9600);
  delay(100);  
  node.begin(9600);  // Modbus RTU
  delay(100);
  Serial.println("Connected "); 
  Serial.println("Modbus RTU Master Online");
  
}


void loop()
{
//Website http://trialcommand.com
///////// Holding Register [0]  A [9]   = 10 Holding Registers Escritura
///////// Holding Register [0] A [9] = 10 Holding Registers Writing

int Mdelay = 500; // from 5 

node.readDiscreteInputs(0, 1); 
Serial.print("[0] ");
Serial.print(node.getResponseBuffer(0));
node.clearResponseBuffer();
delay(Mdelay);
  
node.writeSingleRegister(0x1001, 0xff); 
delay(Mdelay);
  
}

There you have it: the ESP8266 NodeMCU ESP-12E will now be able to request digital input from the modbus device as well as output to it. Having obtained the data it will now need to relay it to the Internet via its webpage, which will be shown in Part 2.

Happy Trails.

Thursday, 5 July 2018

A Long Goodbye for Slackware


I first heard of Linux in the winter of 1995. I was writing a device driver for Microsoft's Windows NT intending to knock it into shape for industrial use. And complaining about the poor quality of the Windows code. Like all the time. I think just to shut me up, my friend (and boss) Wiljan Derks told me about this Finnish student who was writing a UNIX operating system all on his own. He called it Linux, just because his name was Linus Torvalds.

Linus Torvalds 1995 Amsterdam

I escaped on home leave back to balmy Malaysia in February 1996, and there in the sleepy backwater of Ipoh was this book on Linux, 'Slackware Unleashed'. At the back was glued a CDROM with Slackware release 1.1. It was an omen: I bought the only copy immediately.

And it has been Slackware for 22 years From the first install using 12 floppy disks (cdrom drives were hard to find then) on a suitcase-sized Zeos 80486DX2 souped up to a dizzying 66MHz 12MB DRAM to a tiny unassuming Raspberry Pi Zero W blazing along at 1GHz and 512MB.

Patrick Volkerding, Slackware's Benevolent Dictator for Life
Incredibly, through all these years, just like Linux, Slackware is run by one person: Patrick Volkerding.

Initially I ran a number of operating systems: Windows NT, Slackware and SuSE. But by 2004 I dropped Windows, which I had used since 1984. When Novell bought SuSE I dropped that too, and it had only been Slackware since then.

Slackware has always been slow to release new versions. The emphasis has always been stability, which is perfect for me: I had always been using a mainline Linux as an embedded device, and Slackware was a lot less work.

True it required a much bigger footprint, but hardware got better all the time and since elevator projects had a development time of two years anyway, all it needed was a leap of faith: start development immediately using a desktop and bet on the embedded hardware being able to run Slackware by the time I needed it.

Happily, in 14 years Slackware and Moore's Law has not let me down. Slackware progressed from an Advantech Industrial PC to the Via EPIA to the Intel-based fanless boxes with aplomb.

Advantech IPC
Via EPIA
Quanmax Qbox: Intel-based fanless CPU

But then came the ARM-based CPUs like the Beagleboard and the Raspberry Pi. Now a mainline Linux can fit in the palm of your hand.
Beagleboard
Raspberry Pi 3

All this required a fair amount of updates. Despite its manual nature we could cope with Slackware updates, until now: browser vulnerabilities sometimes needed three or more upgrades a year. Browsers like Firefox and Chrome were huge applications and a real pain to upgrade manually, so Debian's siren call beckoned.

When the last two installs of Raspbian went without a hitch, it is time to test my main development laptops on Debian. I started with a spare laptop: an ancient Acer Aspire 5050. Debian 9 'Stretch' installed and ran with great ease. 

Debian 9 'Stretch' on an ancient Acer Aspire 5050

True the speakers did not work and the graphics were slower than my tiny Raspberry Pi Zero W's, but Firefox is up to to the minute, so I think this is the start of Slackware's long goodbye.


On second thought, it is more like au revoir, Slackware.
 

Monday, 2 July 2018

Give your old audio amplifiers a new lease of life with this RM11.90 bluetooth audio receiver


Chinese no-name Bluetooth Audio Receiver
Over the years we kept upgrading the living room audio systems: first turntable gave way to the cassette tape deck, which then changed to the CD player, then mp3 player. This resulted in a few orphaned amplifiers and speakers, especially if they have odd impedances like 4 or 6 Ohms.

These days we play most of our music from smartphones; why not convert them to bluetooth speakers? I used this RM11.90 sgrobot bluetooth audio receiver.

Note the LED, resistor and capacitor
It came with some loose parts: an LED, 100 Ohm resistor and an electrolytic capacitor. Somehow it reminded me of excess bluetooth audio receiver modules off a PCB mainboard for a bluetooth speaker or somesuch now sold off cheaply. The IC markings C7THN5004 did not come up on an Internet search; they look like custom markings. Never mind, better this than discarding it in the municipal landfill.

gameinstance.com has a good write-up on it:




I soldered the parts on it; it literally took one minute.


The pinouts are printed on the PCB, but here they are anyway:



3rd pin from top: left speaker, 4th pin: right speaker
I powered it from my trusty D-link USB3 hub, which puts out a whopping 2A at 5V.

To test, I connected it to yet another one of my orphaned PC analog speakers with integrated audio amplifier. I used my Raspberry Pi Zero W with a brand-new version of Raspbian (Debian 'stretch') and omxplayer. It worked without fuss.  It came up as WIN-668 and paired without asking for a PIN.

I had less luck with my Raspberry Pi B+. It had built-in analog and HDMI audio and omxplayer just would not work out of the box. I'm sure that is fixable, but that is another blog post.

As usual, if you can't be bothered with this DIY malarkey, you can buy it ready-made for RM16.90:
RM29: Bluetooth audio receiver. The USB connector is for power only
Once the speakers are bluetooth-enabled, especially combined with the Raspberry Pi Zero W, it becomes and Internet of Things (IoT) device. Now applications like your very own DIY Google Home becomes possible.

Not bad for something one step from the rubbish heap. Happy Trails.

Wednesday, 27 June 2018

Raspberry Pi Zero W as WiFi Repeater

No extra WiFi dongles used. The dongle shown is a wireless keyboard used for development only
The ESP-01S WiFi Repeater works well for IoT use. It can be daisy-chained maybe 5 times for a maximum range of 200m, but I found it a little slow for normal use. In particular youtube stuttered even on an old Rolling Stones video in black and white.

It would be nice to see how the Raspberry Pi Zero W fared as a WiFi repeater. At RM40 (the microUSB cable is RM2) from cytron, it is about double the price of the ESP-01S Repeater and close to the RM52 retail price for a D-Link DMG-112A.

Even so the Raspberry Pi WiFi Repeater will come out ahead if it can also be used as an IoT device, such as a WiFi autogate remote controller.

Albert Chaharbakshi: had last word on the Raspi WiFi Repeater


I could not find anything to add to Albert Chaharbakhshi's classy post.

Well, except this youtube video to prove his post really works. Even the Raspberry Pi website's version requires another WiFi interface or at least a wired Ethernet interface to work.

Youtube video of Raspi Zero W WiFi repeater feeding a laptop with a video stream
The Pi is shown connected to a monitor and keyboard but is set for "headless" operation with a static IP address. A laptop was able to play a reasonable youtube video (Porcelain Black's Naughty Naughty) at 480p with no difficulty.

There you have it, a practically commercial grade WiFi Repeater. Happy Trails.


Tuesday, 19 June 2018

ESP8266 IoT Low-cost WiFi Repeater

Left: NodeMCU EPS-12E, right: ESP-01S with CH340 adapter
The ESP8266, especially the ESP-01S board can be used as a low-cost WiFi repeater. Add a CH340 USB serial port adapter(RM6.50) for the ESP-01S (RM14.90) and it is still only RM21.40.

A WiFi Range Extender like the D-Link DMG-112A can be bought for RM45, but the ESP8266 is especially cost-effective if the WiFi repeater software can be included in your ESP8266 IoT device. Such an IoT device can be meshed with others to form a WiFi net, extending the IoT cloud's reach.

D-Link DMG-112A WiFi Range Extender is only RM45
Hasten then to Martin Ger's superlative site.

I used Expressive's esptool.py successfully with all three of my programmers, the CH340 ESP-01S adapter, my makeshift ESP-01S programmer, as well as the built-in programmer in the NodeMCU ESP-12E Lua.

I had already upgraded my Slackware 14.2's python to version 2.7.11. This made installing esptools.py easy. Just do:

# pip install esptool

Download's Martin Ger's firmware/0x00000.bin and firmware/0x10000.bin from here. I started with the NodeMCU ESP-12E:

$esptool.py --port /dev/ttyUSB0 write_flash -fs 4MB -ff 80m -fm dio 0x00000 ./0x00000.bin 0x10000 ./0x10000.bin
esptool.py v2.4.1
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: 60:01:94:70:54:1b
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Flash params set to 0x024f
Compressed 37072 bytes to 25809...
Wrote 37072 bytes (25809 compressed) at 0x00000000 in 2.3 seconds (effective 126
.9 kbit/s)...
Hash of data verified.
Compressed 262916 bytes to 178828...
Wrote 262916 bytes (178828 compressed) at 0x00010000 in 15.8 seconds (effective
133.0 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

And that was it; it reset the ESP-12E and the firmware started without fuss.

My ESP-01S took a lot longer as the programming baud rate is only 9600. Remember to put the ESP-01S into 'program' mode manually:

$esptool.py --baud 9600 --port /dev/ttyUSB1 write_flash -fs 4MB -ff 80m -fm dio 0x00000 ./0x00000.bin 0x10000 ./0x10000.bin
esptool.py v2.4.1
Serial port /dev/ttyUSB1
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: ec:fa:bc:1c:c6:0d
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Flash params set to 0x024f
Compressed 38048 bytes to 26400...
Wrote 38048 bytes (26400 compressed) at 0x00000000 in 27.8 seconds (effective 11
.0 kbit/s)...
Hash of data verified.
Compressed 265860 bytes to 181152...
Wrote 265860 bytes (181152 compressed) at 0x00010000 in 190.5 seconds (effective
 11.2 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

Power-cycle (or reset) the ESP-01S to start the firmware running.

I had no problems accessing the WiFi repeater Access Point. It is a little slow, and my test youtube video stuttered a bit, but it played to completion. For IoT use, especially the built-in MQTT, it should be sufficient.

Happy Trails.


Tuesday, 12 June 2018

Raspberry Pi Zero W as IoT Bluetooth Gateway Part3: Replacing Slackware with Raspbian

Something's missing ... yes the USB bluetooth dongle

Nowadays I use the iterative development method. Following Part 1 and Part 2, we used the Raspberry Pi Zero W as an IoT Bluetooth Gateway daily for about six months. It was more secure than using the smartphone to directly pair to the autogate bluetooth module HC-06 - the PIN is only 4 digits and easily brute-forced.. Using a WiFi repeater, its range was much better. We could now reliably operate the autogate without getting out of the car.

In addition the Pi Zero W gateway hogged the HC-06 all the time, and prevented any intruder from pairing to it. To gain access the intruder would now have to brute-force the WPA2 passphrase in the WiFi repeater, a much tougher opponent.

It also got turned on and off a lot to avoid being damaged by lightning, and sometimes it would fail to start up properly. This is usually due to HC-06 connectivity issues- perhaps it was not unpaired properly or was later that usual being discovered. This allowed the startup to be adjusted using my bash script in /etc/rc.d/rc.local.

Now that it was stable enough it was time to improve it further. From Part 2, Slackware 14.2 did not recognize the Raspberry Pi Zero W's builtin bluetooth module. Perhaps the Raspbian kernel I used was not current enough or perhaps my Slackware installation lacked something, but having to use an RM32 bluetooth dongle with an RM42.20 Raspberry Pi Zero W rankled.

I downloaded the Raspberry Pi NOOBS, and following the installation guide, installed the latest Raspbian OS into by Pi Zero W. Make sure to set up the WiFi connection to your broadband. The builtin bluetooth controller worked first time and had no trouble working with the HC-06 in the autogate. I repeated the procedure in Part 1, and upgraded the firmware/BIOS in the Slackware 14.1 image, but the builtin bluetooth controller did not come up.

Now I could have gone further and upgraded the Linux kernel as well, but perhaps it is time to work with Raspbian/Debian for a while. At least I would not have to keep up with the updates.

To use Raspbian for the IoT Bluetooth Gateway, I needed pymodbus, apache, and php. In Slackware, pymodbus needed upgrades to python 2.7, pip and pysetuptools. Checked in Raspbian:

root@raspi-0-w-2:/root# pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
root@raspi-0-w-2:/root# python --version
Python 2.7.13

Now that was a pleasant surprise: no upgrades necessary. Now for pysetuptools: Make sure your WiFi connection is working before you try this:

root@raspi-0-w-2:/root# apt-get upgrade python-setuptools
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-setuptools is already the newest version (33.1.1-1).
python-setuptools set to manually installed.
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Again, no upgrades necessary. Might as well go ahead with pymodbus:

root@raspi-0-w-2:/root# pip install -U pymodbus

Installed successfully, no fuss. Now my Modbus server program needed twisted, so:

root@raspi-0-w-2:/root# pip install twisted

No problems there. Also required is cryptography:

root@raspi-0-w-2:/root# pip install cryptography

Now for apache. Before that just to be sure I checked for upgrades to Raspbian:

root@raspi-0-w-2:/root# apt-get update

No upgrades necessary- I am beginning to like Debian. Now for apache:

root@raspi-0-w-2:/root# apt-get install apache2 -y

No fuss, apache ran right out of the box (check using http://localhost/index.html):

root@raspi-0-w-2:/root# find /var -name index.html
/var/www/html/index.html

root@raspi-0-w-2:/root# ps -ef
root     20567     2  0 14:14 ?        00:00:00 [kworker/0:3]
root     21396     1  0 14:16 ?        00:00:00 /usr/sbin/apache2 -k start
www-data 21398 21396  0 14:16 ?        00:00:00 /usr/sbin/apache2 -k start
www-data 21399 21396  0 14:16 ?        00:00:00 /usr/sbin/apache2 -k start
root     21723  1705  0 14:18 pts/1    00:00:00 ps -ef

Similarly, php just worked, striaght from the box:

root@raspi-0-w-2:/root# apt-get install libapache2-mod-php php

The web server was linked to pymodbus using the same method as before. Now Raspbian defaults to using dynamic IP, and ssh disabled, which is not very useful if you are using the Pi in "headless" (ie without monitor, mouse or keyboard). ssh is easily enabled via the Raspbian desktop or using raspi-config. To get it to use static IP simply add the following lines to /etc/dhcpcd.conf:

interface wlan0
static ip_address=192.168.1.2/24
static routers=192.168.1.1
static domain_name_servers=8.8.8.8

Lastly Raspbian uses the much improved bluetoothctl, which is an interactive program and not easily included in a bash script like bluez. Luckily Linux has just the thing we need: 'expect' can be used to fool bluetoothctl into thinking it is interacting with a human via the keyboard.

root@raspi-0-w-2:/root# apt-get install expect

Put it in a little bash script and you have a much more robust startup script than simple-agent:

root@raspi-0-w-2:/root# cat ./hc-06.sh
#!/usr/bin/expect -f

set prompt "#"
set address [lindex $argv 0]

spawn sudo bluetoothctl -a
expect -re $prompt
send "remove $address\r"
sleep 1
expect -re $prompt
send "scan on\r"
send_user "\nSleeping\r"
sleep 5
send_user "\nDone waiting for controller\r"
expect "Controller"
send_user "\nSleeping ... waiting for autogate\r"
sleep 5
send_user "\nDone waiting for autogate\r"
expect "HC-06"
send "scan off\r"
send "trust $address\r"
sleep 2
send "pair $address\r"
sleep 2
send "0000\r"
sleep 3
send_user "\nShould be paired now.\r"
send "quit\r"
expect eof

Note: replace '0000' with your password and launch the script thus:

root@raspi-0-w-2:/root# ./hc-06.sh 12:34:56:78:9A:BC

Lastly bind to your bluetooth device to get the serial port /dev/rfcomm0:
rfcomm bind 0 12:34:56:78:9A:BC 1

And you are ready for the pymodbus program, which I launched in the background and captured its output to a log file:

root@raspi-0-w-2:/root# python ./autogate_server.py >> ./autogate.log 2>> ./autogate.log &

Raspbian turned out to be quite pleasant to use, but then it is Linux, so we expect nothing less. 

Happy Trails.

Wednesday, 30 May 2018

Using the CH340 USB dongle as ESP-01S Programmer

Note the add-on push-button switch

Some time ago I bought a cheap (RM6.50) CH340-based USB to serial 3V3 TTL dongle with a socket for the ESP-01S. It was described as a development board, and I assumed it will also program the ESP-01.


That assumption turned out to be quite unwise, as 'programming' was not mentioned:


It most certainly would not program an ESP-01S. What is needed was a switch to pull GPIO0 low when the ESP-01S is powered up or reset. AndyS19 has a nice writeup on this.

ESP-01S pinout

I happened to have a cheap (RM0.80) small push-button switch very popular here for use in car, autogate or garage remotes. The distance between the leads (ie pitch) almost exactly fits the pins GPIO0 and GND of the CH340 ESP-01S USB dongle, on the solder-side of the PCB.





Push-button switch for remotes


Solder-side of the CH340 ESP-01S adapter

I simply glued the switch to the PCB and soldered the leads onto the GND and GPIO0 header pins, and presto, I have an ESP-01S programmer.



To program, hold down the button as you insert the dongle in your laptop. Release the button and fire up your Arduino sketch. Use the 'ESP8266 Generic' setting. Some ESP-01S I bought would only work with 9600 baud, others are fine with 115200.

It will be especially handy to have an ESP8266 board to run my code on those train rides on my daily commute.

Life is good. Happy Trails.












Tuesday, 8 May 2018

IoT Porch Light using ESP8266 and Arduino Relay PCB

WiFi control of Porch :Light
Warning: this project contains live mains voltages that can be lethal. There can be a fire risk especially if you use generic no-name China parts like I have. Do not use generic parts unless you first test them under load and disassemble them. Some generic parts are under-specified and can be hazardous. Do not mount the lamp on flammable surfaces like wood or plastic.

Following related posts on ESP-01S, the ESP-12E Arduino sketch, this post shows the final assembly of the light panel and the 5V power supply.

The light panel is a generic "Surface LED Down Light Panel Lamp" from cpeelectrical which I bought for RM43, mainly because there is room inside the enclosure to mount the ESP-1S and relay PCB.

Top: ESP-01S with relay module. Middle: 230Vac to 5Vdc power supply. Bottom: 18W LED Driver



For the ESP-01S power supply I used an old smartphone charger. It was marked as 'output 5.7V 800mA' but when used I measured 5.05V which is just as well as the ESP-01S only takes 5V. It is too large to be mounted inside the lamp, so I removed the PCB and mounted it without its enclosure.

Old Android smartphone charger, disassembled. Note the lack of a controller IC
The power supply was extremely simple: all the components were discrete devices (ie no IC) and just an oscillator at the primary side, and a single rectifier diodes at the primary and secondary. Still, it had been used for some 3 years sealed in its plastic enclosure with no ventilation holes and should be reliable enough for this purpose.

5V Charger tested on ESP-01S
If you do not have an old charger, you can buy them here in Malaysia for RM6 or more. It is better to buy branded chargers like Samsung, as generic chargers have been known to catch fire.

The ESP-01S drew 170mA during WiFi access, and 60mA more when operating the relay. In practice it drew somewhat more than 230mA on power up so that 800mA at 5V came in handy. For easy installation I had the relay come on on power up. This also acts as a manual override 'ON' from the light AC mains power switch. 

The final part, the LED Driver came with the lamp. It took in 85-230Vac (any lower and it shuts down) and puts out a stonking 60Vdc at no load, but settled down to 50Vdc at 330mA when the lamp is turned on. 

LED driver 18W 230V


Now the relay can be wired up to switch the DC or AC side of the LED driver. I opted to switch the AC side- hopefully it saves a little power as the primary side will not be always on.

LED Driver. Note the IC controller on the primary side
For enclosures for the ESP-01S and the mains 5V power, I used the casings from 20W LED drivers. The entire LED driver costs only RM6 each and I had a couple of faulty ones left over from other LED lamps. You must enclose them: all 3 boards have 230V mains voltages that must be insulated. Here in Malaysia insects get into electrical fittings, especially light fittings including bulbs.

There you have it: an IoT lamp, switched from your smartphone or desktop browser. Happy Trails.