So, I've been playing around with the Raspberry Pi and home automation devices such as IKEA Trådfri lights, a TP-Link HS100 switch, my Netatmo smart thermostat and a few ESP8366 devices including various varieties of Sonoff switches. Rather than blunder on I thought it would be nice and take stock of what I've done and install everything cleanly from scratch.

Most of the pearls of wisdom herein have been taken from this excellent guide: https://screenzone.eu/arduino-mqtt-sensor-grafana/

Here is the software I want installed just to control the devices:

  • Node-RED
  • Mosquitto MQTT broker

But one of my Sonoff's is a Sonoff POW which provides real time power measurements and I also want to hook up at least one of my ESP8266 to a DHT22 temperature/humidity sensor and measure temperature real time. So some method of graphing the data over time would be nice. Some more software will be required for this:

  • InfluxDB
  • Grafana

That's the shopping list for now so let's get started.

Raspian

The Pi will be running purely as a server so I don't need the graphics interface. But I do need wireless.

So, install Raspian Stretch Lite as normal, add a HDMI connection and keyboard, just for now and login. To add wireless we need to go edit the wpa-supplicant file:

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

and add the following lines:
network={
ssid="myWifiSsid"
psk="myWifiPassword"
}

with the appropriate ssid and password. Note the password is in the clear - we can generate the hashed password but I didn't know of an easy way of copying it and pasting it into the editor - but once we have SSH running this is easy from another computer so edit raspi-config

sudo raspi-config

to change the password, expand storage, set the Hostname, and make sure SSH is enabled. And since we are not using graphics we can drop the memory split to use the minimum 16MB for graphics.

Reboot, check wifi is working and remember the IP address. Now we can ssh into the Pi and generate a hashed wifi key

wpa_passphrase "myWifiSsid" "myWifiPassword"

which will produce output like
network={
ssid="MyWifiSsid"
#psk="MyWifiPassword"
psk=ab058a266ce8fe3b5cdc92a6ab6edd35ce13f88797614e21b8af5ba0e4afb784
}

so we can re-edit wpa_supplicant.conf and replace the original psk definition which was out wifi password in the clear with the new hashed version.

All that's left to do is ensure the Pi has all the latest packages:

sudo apt-get update && sudo-apt-get upgrade

Node-RED

Node-REDd comes pre-installed with Raspberry Pi full install images but not with the Lite versions. The documentation and installation of Node-RED is excellent - check https://nodered.org/docs/hardware/raspberrypi and all we need is

bash <(curl -sL https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered)

Now add Node-Red to systemd so it auto-starts:

sudo systemctl enable nodered.service

reboot and test. Use any browser on your home LAN and navigate to Pi's ip address:1880

Security

Before we can edit the ~/.node-red/setting.js file we need to generate a hashed password. Install node-red-admin:

sudo npm install -g  node-red-admin

Now we can generate the password with:

node-red-admin hash-pw

Enter your desired admin password at the prompt and copy the resultant string.Now edit settings.js file and uncomment the adminAuth section and replace the password with the one you have just copied. Restart node-RED (or reboot) to activate the change.

MQTT

There are a lot of websites explaining that MQTT is broken on Debian Stretch but it looks like the repositories now have a slightly back-level version of MQTT which has all the dependencies met so install is the simple:

sudo apt-get install mosquitto mosquitto-clients

While we're installing let's install Python support for MQTT too, just in case - this is a simple python pip install but ... the Lite version of Raspian doesn't have python pip installed so firstly

sudo apt-get install python-pip
sudo apt-get install python3-pip

then we can install paho:

sudo pip install paho-mqtt

To test MQTT simply subscribe to a topic by:

mosquitto_sub -h localhost -t "test"

Leave this running and start a different shell (I simply used SSH) and publish something to that test topic:

mosquitto_pub -h localhost -t "test" -m "test msg"

Back on your subscribe shell the message payload test msg should appear and all is well.

Security

The Screenzone Guide suggests it might be a good idea to add some security to MQTT in the form of requiring a Userid and Password to publish or subscribe to messages.

Firstly we create an authorisation file:

sudo nano /etc/mosquitto/conf.d/auth.conf

and add the lines:
allow_anonymous false
password_file /etc/mosquitto/passwd

Then create a user homeauth and supply a password

sudo mosquitto_passwd -c /etc/mosquitto/passwd homeauth

Restart mosquitto (or reboot) for the changes to take effect:

sudo systemctl restart mosquitto

Now let's try our tests again - the simple sub and pub tests above both fail to connect because of an authorisation error - so let's add authorisation:

mosquitto_sub -h localhost -t "test" -u "homeauth" -P "mysecret"

and

mosquitto_pub -h localhost -t "test" -m "test msg" -u "homeauth" -P "mysecret"

and confirm they work as previously.

InfluxDB

https://bentek.fr/influxdb-grafana-raspberry-pi/

curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -

echo "deb https://repos.influxdata.com/debian stretch stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

sudo apt-get update

sudo apt-get install influxdb

Grafana

curl https://bintray.com/user/downloadSubjectPublicKey?username=bintray | sudo apt-key add -

echo "deb https://dl.bintray.com/fg2it/deb stretch main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

sudo apt-get update

sudo apt-get install grafana
sudo nano /etc/grafana/grafana.ini

[server]
# Protocol (http, https, socket)
protocol = http

# The ip address to bind to, empty will bind to all interfaces
;httpaddr =

# The http port to use
httpport = 3000

sudo service grafana-server restart
sudo systemctl enable grafana-server.service 

Configuring Node-Red

Let's use a simple example from a Tasmota flashed Sonoff-POW

The MQTT message is:
"{"Time":"2018-08-07T12:04:27", "Total":31.918, "Yesterday":0.229, "Today":0.003, "Period":0, "Power":1, "Factor":0.00, "Voltage":253, "Current":9.362}"

First step is to use the influx CL to create a database
'create database power'

Then a simple function is needed to convert this to a payload into the requisite influx payload format - we are probably only interested in power, current and voltage so simply converting the payload to JSON and taking the appropriate parts works:

node.status({fill:"blue",shape:"dot",text: msg.payload});

tmp=JSON.parse(msg.payload);

msg.payload = {
	power: tmp.Power,
	voltage: tmp.Voltage,
	current: tmp.Current,
	sensor:"Sonoff-POW1",
    };
    
return msg;

*The first line just output the incoming message payload as status under the function box in Node-Red for debugging
*

Just 3 Node-Red nodes are needed an MQTT output node to pass the published message to the Function node and the output from that is simply passed to an Influxdb input and data is written to the database.

Using the influx CL
use power
to select the appropriiate database

show measurements should show the measurement name you specified in the Influxdb Node-Red node - in my case it is "Sonoff" so

select * from Sonoff
shows

time                current power sensor      voltage
----                ------- ----- ------      -------
1533633288497843297 0       0     Sonoff-POW1 254
1533633318505875412 0       0     Sonoff-POW1 254
1533633348715756458 0       0     Sonoff-POW1 255
1533633378731317810 0       0     Sonoff-POW1 254
1533633408891859566 0       0     Sonoff-POW1 254
1533633438910263696 0       0     Sonoff-POW1 254
1533633469053749624 0       0     Sonoff-POW1 254
1533633499139945397 0       0     Sonoff-POW1 254
1533633529244007464 0       0     Sonoff-POW1 254
1533633559250354927 9.44    1     Sonoff-POW1 254
1533633589398467843 9.44    2     Sonoff-POW1 255
1533633619458944619 0       0     Sonoff-POW1 255
1533633649730510602 9.474   1     Sonoff-POW1 256

*Yes, I didn't have a load on my Sonoff-POW at the time!"