It all started with my friend Tim watching this video by Matt Parker

The problem Matt was trying to solve was to find 5 5-letter words (as in the Wordle dictionary) that between them contained 25 different letters of the alphabet, i.e. none of the five words had any letters in common.

I sat and thought for a while and came up with - 24 letters with WALTZ, FJORD, NYMPH, QUICK, and BEGS - close but no cigar, as the saying goes. Time to write some code. I'd not seen the video at the time so didn't know that Matt had written some code that took almost two months to run. I set myself a target of under an hour (on my 12 year old first gen i7 desktop).

The problem space is huge, there are 12972 words in the original, pre-NY Times and so there are 12972*12971*12970*12969*12968/120 different combinations - that's 3,058,571,387,612,503,040 which might take more than an hour (more than a year, more than a decade...) to work out and check all of them.

One way to reduce that is to ignore all the words that have two or more of the same letter in them - they automatically exclude themselves from the solution. Removing them reduces the list size to a mere 8322 words. But we can go one stage further and remove all the anagrams of the remaining words (remembering them so if any are in the solution we can add in the extra anagram versions of the solution). Amazingly that drops the word count to 5182.

Now that seems like a huge reduction, over 50% but simply checking every combination would require 5182*5181*5180*5179*5178/120 =310,190,561,767,076,760 which is still a HUGE number. Checking every combination simply won't work.

I spent a long time thinking about how to loop over the combinations breaking out of the loops immediately I found a fail, but my brain simply wasn't big enough to work out the complex code. Even worse, a natural solution might involve recursion. To be honest I stalled, but a chance conversation down the pub with another friend, Charlie and he sent me a list of solutions the following day - done using a spreadsheet! Amazing!

Then I had an idea, rather than stop iterating every time there was a clash, why not filter the iterations so that clashes could never happen. So the first iteration I would iterate all the way through the entire 5182 words but for each word (word1, say) I would then build a list of all words that didn't have any common letters and iterate through this (word2) say to build a list of word3 letters and if this existed I'd build a list of word 4 letters (this list wouldn't happen too often and would likely be quite small). If I did have a word4 list (i.e. words that had no letters in common with word1, word2, word3, and word4 then I'd try and build a list of any words that also had nothing in common with any of these. The rare occasion this word5 list actually succeeded then word1, word2, word3, word4, and any word5s would be the solution to the problem.

This approach is very Pythonic too as Python's list comprehension makes building the filtered lists pretty trivial. There were some further optimisations, the main one being that if I'm some way down that 5182 long list of word1 words I need only start building the word2 list from the next word (e.g. if I'm at 100 words down the original 5182 word list for word1, I can start filtering the word 2 list at word 101). Why? Lets take the 20th word as an example. If I compare the 100th word in the list as word1 and the 20th word in the list as word2 that's exactly the same as comparing the 20th word in the list as word1 with the 100th word in the list as word2 - which I've already done. There's also a very minor tweak that means I can cut the word1 list 4 words (i.e. at 5178) before the end, the word2 list 3 words before the end, etc.

Assuming the filtering is fast (it is) then the number of iterations shrinks dramatically. That list of word2 words is generally a fraction of the original 5182 and the word3 list is much, much, smaller. If any word4s exist at all they will be pretty scarce and, on the 10 occasions we actually have a list of word5s then it is a list of length 1!  The problem with this approach is that, since the filtered word lists are being built dynamically it is not possible to calculate their length in advance, and so work out how many iterations the code might take. So I had to temporarily add some counts and was somewhat amazed that this approach takes just 63,515,889 - a relatively small number, especially when compared to 3,729,483,737,997,660,720

A real life example, I set the debugger to break at word 2000 out of the 5182 words, which happened to be 'carts'. So the second list of words will be all the words from position 2001 to the end of the list (i.e. 2182 of them) but filtered to exclude any with any of the letters 'c', 'a', 'r', 't', or 's' - this reduces that 2182 to just 413 words to check. The first of these happened to be at the first position in the 413 words list and was  'defog'. So now we check the 412 words in the list from position 2 to the end - but exclude any with letters in 'defog' (we don't need to exclude any 'carts' letters this time as they've already been filtered). Filtering the 412 words leads to just 27 words in the third list to check, and the first 7 of these meant that no further 4th words could be found, but the 8th in the list 'jumby' did give a couple of options for word 4 - 'plink' and 'whilk'. But neither of those lead to a fifth word and solution.

The next thing was to optimise the code, even just 63 and a half million comparisons of 5 digit strings might take some time. So I converted each of those unique lettered, anagrams removed words to 26 bit binary number - simply if there was an 'a' in the word then bit 25 would be set, a 'b' meant bit 24 ... and a 'z' bit 0. Now working with these, comparing two is simply a Boolean AND and if that result is 0 the binary numbers have no bits in common, i.e. the original words have no letters in common - Boolean ANDs tend to be very fast computer operations.

Here's the algorithm:

    answers = []
    # Get first candidate word (bword0) by interating through the entire list
    # Note -  don't need last 4 words, they will be checked in the later iterations
    for i1, bword1 in enumerate(binwords1[:-4]): # Binwords1 is the entire list of words
        # For each word in the forst word list build the second word list,
        # starting at the current position in the list + 1
        # but filtering out all words that have any letters in common with bword1
        # and then iterate through this list of second words (bword2)
        binwords2 = [bw for bw in binwords1[i1 + 1:] if not (bw & bword1)]
        if binwords2:  # The code works without checking that the list exists but the check optimises the code slightly!
            for i2, bword2 in enumerate(binwords2[:-3]): # Again, last 3 words will be checked later
                # Build a third word list in a similar fashion and iterate through
                # Note since the list binword2 is filtered not to contain bits common with bword1 we need only check ...
                # commonality with bword2 (i.e. no need to accumulate a bitmap for all previous words to check against)
                binwords3 = [bw for bw in binwords2[i2 + 1:] if not (bw & bword2)]
                if binwords3:
                    for i3, bword3 in enumerate(binwords3[:-2]):
                        # Now try to build a fourth wordlist of words with letters not in words1, 2, and 3
                        binwords4 = [bw for bw in binwords3[i3 + 1:] if not (bw & bword3)]
                        if binwords4:
                            for i4, bword4 in enumerate(binwords4[:-1]):
                                # We have 4 words - try and build a list of 5th words that don't have letter matches
                                # If this list is non-empty we have solutions!
                                binwords5 = [bw for bw in binwords4[i4 + 1:] if not (bw & bword4)]
                                if binwords5:
                                    for bword5 in binwords5:
                                        answers.append([bword1, bword2, bword3, bword4, bword5])

    return answers # List of all the possible Wordle Pangrams (excluding anagrams)

The above code certainly shows the power of Python list comprehension and slicing in the one-liner for building the filtered lists.

binwords2 = [bw for bw in binwords1[i1 + 1:] if not (bw & bword1)]

And that's about it. There's obviously some code to remove words with duplicate letters - but again Python helps :

if len(set(aword)) == len(aword)

is a simple check for words of all unique letters and, of course, converting the words to binary numbers makes detecting anagrams as simple as checking the binary number is unique. There's also some horrible kludge code to scan the solutions checking if any of the words are anagrams and, if so, creating new, alternative solutions.   Believe it or not, one of the words in one of the solutions actually has an anagram!

Oh, if you are interested - here's the complete list of Wordle Pangrams:

fjord, nymph, waltz, gucks, vibex
fjord, chunk, waltz, gymps, vibex
prick, glent, jumby, vozhd, waqfs
brick, jumpy, glent, vozhd, waqfs
jumpy, bling, treck, vozhd, waqfs
bemix, clunk, grypt, vozhd, waqfs
blunk, cimex, grypt, vozhd, waqfs
brung, cylix, kempt, vozhd, waqfs
brung, xylic, kempt, vozhd, waqfs
clipt, jumby, kreng, vozhd, waqfs
jumby, pling, treck, vozhd, waqfs

Oh, yes, I also forgot - how long does this code take? Well on my 12 year old i7 it takes 2 mins 20 secs - I beat my hour target by a margin that surprised even me. It's almost a minute longer on my MacBook Air which shows my aging Linux desktop is still a pretty good development machine!

Here's the complete code:

import WordleDict
import string
import time


def time_usage(func):
    """
    Simple decorator function to calculate and print time taken for any function
    """

    def wrapper(*args, **kwargs):
        begin_ts = time.time()
        retval = func(*args, **kwargs)
        end_ts = time.time()
        print(F"Function - {func.__name__}: Elapsed time: {(end_ts - begin_ts)}")
        return retval

    return wrapper


def binariseword(word):
    """
    Convert any word into 26-bit bitmap a=bit25 set, b= bit24 et .. z = bit 0 set
    NOTE - although this was written for 5 char Wordle words, it will work withwords of any length and either ...
    Upper or Lowr case, or a mix thereof

    :param word: A character string containing alphabetic only characters
    :return bitmap: the binary representation of the word
    """

    bitmap = 0
    for letter in word.lower():
        bit = 1 << string.ascii_lowercase.index(letter)  # Set the bit depending on position of letter in alphabet
        bitmap = bitmap | bit  # and accumulate into a bitmap of all letters in the word
    return bitmap


# @time_usage
def uniqueletters_noanagrams_bin(wordlist):
    """
    Filter a list of words removing any words that contain duplicate letters and or anagrams
    Also convert any such words to a binary bitmap equivalent (see binarise word)
    :param wordlist: list of character string words
    :return bwdict: A dictionary of with the filtered binarised words as keys with the char equivalent values
    :return anagdict: A dictionary of binarised words with an array of anagrams - the word as in bwdict is the first
                      element in this array - this is only created if there are actually anagrams
    """

    # Should be separate functions but for a large wordlist combining into one saves an iteration
    bwdict = {}
    anagdict = {}
    bitset = set()
    for word in wordlist:
        # Is the set of unique letters in the word the same length as the number of letters in the word?
        if len(set(word)) == len(word):  # If so the word doesn't have duplicate letters
            bitmap = binariseword(word)  # so convert the word to its binary bitmap equivalent
            # Does the set of bitmaps so far contain this bitmap
            if bitmap not in bitset:  # No - so it is not an anagram of an existing word (which would have same bitmap)
                bwdict[bitmap] = word
                bitset.add(bitmap)  # and the the new binarised word into the set of all such so far
            else:  # Yes - we already have this bitmap so the word is an anagram of an existing word
                if bitmap not in anagdict:  # Do we already have other anagrams of the word?
                    anagdict[bitmap] = [bwdict[bitmap],
                                        word]  # No - create a dictionary entry for the original word ...
                else:  # Yes - add it to the existling list of anagrams                  ... and its newly found anagram
                    anagdict[bitmap].append(word)
    return bwdict, anagdict


@time_usage
def get_pangrams(binwords1):
    """
    Find all five word panagrams in the list of 5 letter Wordle words (which have been binarised)
            - i.e. a set of words comprising 25 distinct letters (or 25 distinct bits in the binarised version)

    :param binwords: A list of binarised words to search and check for pangrams
    :return answers: A list of 5 entry lists containing binarised words that comprise the pangram
    """

    # This module is the core of this code and finding an efficient algorithm is necessary
    # There are 12972 words in the Wordle list (original version pre-NY Times). After filtering out words with ...
    # duplicate letters this is still 5182 unique character strings. So there are are 5182!/5177! or ...
    # 3.7359635e+18 total combinations!!

    # This code reduces the iterations substantially (to just 74582079 iterations)
    # It iterates through the entire list of words and for each word produces a (smaller) filtered list of all ...
    # words which do not share common letters with that word. It then iterates through this list for the second word ...
    # and for each second word it produces a (now much smaller) list of filtered third words, For each word in this ...
    # third list a fourth word list of all words not sharing letters with words1, 2 or 3 is produced ...
    # (if any such exist) and should iterating this produce any list of fifth words we have a solution!

    # The iteration loop for the second word need only start after the current position of the current initial word ...
    # iteration e.g. if we are at iteration 100 in word 0 we can start building the filtered list for the second ...
    # word starting at position 101. This is (again for example) word 10 on the second iteration compared with word ...
    # 100 on the first iteration is the same as word 10 on the first iteration compared with word 100 on the second ...
    # which has already been checked.

    # The code is further optimised by using the binarised equivalent version of the words so a simply Boolean AND ...
    # suffices to check commonality of bits (i.e. letters in the original word)

    answers = []
    # Get first candidate word (bword0) by interating through the entire list
    # Note -  don't need last 4 words, they will be checked in the later iterations
    for i1, bword1 in enumerate(binwords1[:-4]): # Binwords1 is the entire list of words
        # For each word in the forst word list build the second word list,
        # starting at the current position in the list + 1
        # but filtering out all words that have any letters in common with bword1
        # and then iterate through this list of second words (bword2)
        binwords2 = [bw for bw in binwords1[i1 + 1:] if not (bw & bword1)]
        if binwords2:  # The code works without checking that the list exists but the check optimises the code slightly!
            for i2, bword2 in enumerate(binwords2[:-3]): # Again, last 3 words will be checked later
                # Build a third word list in a similar fashion and iterate through
                # Note since the list binword2 is filtered not to contain bits common with bword1 we need only check ...
                # commonality with bword2 (i.e. no need to accumulate a bitmap for all previous words to check against)
                binwords3 = [bw for bw in binwords2[i2 + 1:] if not (bw & bword2)]
                if binwords3:
                    for i3, bword3 in enumerate(binwords3[:-2]):
                        # Now try to build a fourth wordlist of words with letters not in words1, 2, and 3
                        binwords4 = [bw for bw in binwords3[i3 + 1:] if not (bw & bword3)]
                        if binwords4:
                            for i4, bword4 in enumerate(binwords4[:-1]):
                                # We have 4 words - try and build a list of 5th words that don't have letter matches
                                # If this list is non-empty we have solutions!
                                binwords5 = [bw for bw in binwords4[i4 + 1:] if not (bw & bword4)]
                                if binwords5:
                                    for bword5 in binwords5:
                                        answers.append([bword1, bword2, bword3, bword4, bword5])

    return answers



# @time_usage
def print_with_angrams(bw_solutions, anagdict):
    """
    Print all the pangram solutions including all anagram variants

    :param bw_solutions: A list of lists of 5 letter binarised word pangrams
    :param anagdict: A dictionary mapping the binarised word to all anagrams thereof
    :return NONE:
    """
    # This is horrible KLUDGE code. Converting the binarised word lists to words and printing is trivial but ...
    # folding in all anagrams is horrible, especially since more than one word in a solution may have one or more ...
    # anagrams

    solutions = []
    for bw_solution in bw_solutions: # Get a single binary solution
        solution = []
        for bword in bw_solution:
            solution.append(bwdict[bword])  # Convert the binarised word back to its char equivalent
        solutions.append(solution)          # and add the list of char pangrams into a total solutions list
        for bword in bw_solution:
            if bword in anagdict:           # iterate through the binarised words again looking for anagrams
                for i in range(len(anagdict[bword]) - 1): # there may be more than one so need to iterate anagrams
                    oldword = anagdict[bword][i]          # get the original word, or previous anagram
                    newword = anagdict[bword][i + 1]      # and get its newly found anagram
                    for oldsolution in solutions:         # go back through all the existing lists of pangrams
                        if oldword in oldsolution:        # and if the old word is in the list replace it with the new
                            newsolution = list(map(lambda x: x.replace(oldword, newword), oldsolution))
                            solutions.append(newsolution) # dynamically extend the list of solutions so if other ...
                                                          # words in this pangram are anagrammed all combinations ...
                                                          # will be produced.

    for solution in solutions: # Now simply print all the solutions which now includes all anagram variants
        print(', '.join([word for word in solution]))




# END OF THE FUNCTIONS - MAIN CODE STARTS HERE

if __name__ == "__main__":
    bwdict, anagdict = uniqueletters_noanagrams_bin(WordleDict.all)  # Filter out anagrams and duplicate letter words
    solutions = get_pangrams(list(bwdict.keys()))  # Use the binary equivalents of the words to find all pangrams
    print_with_angrams(solutions, anagdict)        # Print the solutions along with any anagram variants

Part 1 showed how to implement a Python Flask server and use WhoIsXMLAPI and the browser user-agent information to find some basic information about the client.

Proxying an existing production server to the flask server

We now need to make that accessible via the web. Generally this means creating a subdomain for the server, requesting SSL certificates etc. However, since I already have a Technicals website (you are reading it at the moment) it will be easier to hang it off this site, and then subdomain registration and SSL registration is already taken care off.
So we need to create a new URL route, /mike on this site and reverse proxy this to the flask server running on 192.168.1.15:5000

    location /mike/ {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host;
        proxy_pass http://192.168.1.15:5000/;

    }

is all that is needed.

Now browsing to
/mike
will redirect from the normal Technicals server machine to the flask server.

One further tweak - we are restricted to 1000 WhoIsXMLAPI lookups a month and it would be easy to exceeed that during heavy testing, or if the website fell foul to an attack of any sort, so the flask server expects a parameter on the URL, if it is present and with the correct value then the code will do the WhoIsXMLAPI lookup, if not it will use a saved pre-canned one. This requires no implementation changes as it is handled entirely in the flask but it does mean that to actually get the code to implement a real browser lookup it needs to be called with the parameter - e.g.

/mike?type=xxxx

[Oh, if that doesn't work for you, it'll be because the parameter isn't really 'type=xxxx' :-) ]

Making the flask service a system service

The code works and is now available on the external internet via the URL above but currently it needs to be started manually and will need manual restarting should the 192.168.1.15 machine reboot.

We will create a system service to automatically start the flask server at startup.

sudo nano /etc/systemd/system/internetprobe.service

and add the following

[Unit]
Description=Demo internet credentials check for Mike Eacott
After=network.target

[Service]
User=pi
WorkingDirectory=/home/pi/internetprobe
ExecStart=/home/pi/internetprobe/security1.py
Restart=always

[Install]
WantedBy=multi-user.target

We can start it straight away to test it

sudo systemctl daemon-reload
sudo systemctl start internetprobe

and check it is running

sudo systemctl status internetprobe

before enabling it so that it will startup a boot time

sudo systemctl enable internetprobe

That should be it!

My good friend Mike is a local Cyber champion and often gives talks to various organisations, generally retired folks. His next talk is to be on the subject of Internet Safety and he wanted to demonstrate the sort of information that is available to any website you visit.

For example - how do sites like this, know about you?
https://www.whatsmyip.org/more-info-about-you/

So I took up the challenge to see if I could write something to provide similar information and just how simple it might be.

The information comes from 3 separate sources

• Your Browser header information
• The WHOIS entry corresponding to your IP address
• A collated database which is possibly updated from sites to whom you have given permission to access your location.

The Browser headers contain information about your Browser (obviously), Operating System, machine type and model, and screen size.

The WHOIS entry has the details of your ISP and connection type.

The database entries should be able to make a reasonable stab at your geo-location although my be well out (up to 200km!). I suspect accuracy depends on how many websites, in the past, have been granted access to your location. Also some database repositories appear more accurate than others.

For this exercise I chose WhoisXMLAPI - https://ip-geolocation.whoisxmlapi.com/api because their geolocation of my own IP was fairly accurate and they had a generous free 1000 lookups per month without the need for credit card registration. They also do the IP lookup (otherwise I would need to do a request.remote_addr), the WhoIs lookup, and even do a DNS lookup to list domains registered to this IP.

Let's look at the Python code - it really is pretty minimal - just get and parse the User-Agent and then call SimpleGeoIP for all the IP, ISP, and geo information. A little bit of code saves wasting Simple GeoIP calls when testing, by reusing a 'Blue Peter' call that I made earlier.The results from both the User-Agent and SimpleGeoIp calls (both Python dictionary structures) are then simply passed onto the Jinja2 template presenting the page on the Flask server.

Here's the ENTIRE Python code:

#! /usr/bin/python3
from flask import Flask, request, render_template
from flask_simple_geoip import SimpleGeoIP
from user_agents import parse

app= Flask(__name__)  # define the flask web server
app.config["GEOIPIFY_API_KEY"] = "xxxxxxxxxxxxxxxxxxxxxxxx" # my API key for SimpleGeoIP

@app.route('/') #Define the / (i.e. index) page of the website
def index():
# Add a "type=xxxx" parameter to the URL so that testing doesn't waste SimpleGeoIP lookups 
    production = True if request.args.get('type') == 'xxxx' else False
    
    UAHeaders = request.headers.get('User-Agent') # Get the HTTP user_agent
    UAparsed = parse(UAHeaders) # Parse it nicely
    
    if production:
        simple_geoip = SimpleGeoIP(app)  # set up the SimpleGeoIp API
        geoip_data = simple_geoip.get_geoip_data() # call SimpleGeoipto get IP info
        runtype = "Production"
    else: # if just testing use precanned information  
        geoip_data = {'ip': '81.174.xx.xx', 'location': {'country': 'GB', 'region': 'England', 'city': "xxxxxxxxxx", 'lat': xxxxxxxx, 'lng': -1.39905, 'postalCode': '', 'timezone': '+00:00', 'geonameId': 7287933}, 'domains': ['two-drifters.co.uk'], 'as': {'asn': 6871, 'name': 'Plusnet plc', 'route': '81.174.128.0/17', 'domain': 'http://www.plus.net', 'type': 'Cable/DSL/ISP'}, 'isp': 'Plusnet', 'connectionType': 'broadband'
        runtype = "Test"

    return render_template('index.html',runtype=runtype, ua=UAparsed, geoip=geoip_data) # render the webpage
   
if __name__ == '__main__':  # main program code
    app.run(host='192.168.1.15', debug=True) # simply run our flask webserver app on this machine

In the HTML a couple of Javascript functions get the screen and viewport sizes (thanks to Andy Langton for this code)

<script type="text/javascript">
      document.getElementById("screensize").innerHTML =
        screen.width + "x" + screen.height;
    </script>

    <script type="text/javascript">
      var wh = getViewportSize();
      var w = wh[0];
      var h = wh[1];

      function getViewportSize() {
        var viewportwidth;
        var viewportheight;
        if (typeof window.innerWidth != "undefined") {
          (viewportwidth = window.innerWidth),
            (viewportheight = window.innerHeight);
        } else if (
          typeof document.documentElement != "undefined" &&
          typeof document.documentElement.clientWidth != "undefined" &&
          document.documentElement.clientWidth != 0
        ) {
          (viewportwidth = document.documentElement.clientWidth),
            (viewportheight = document.documentElement.clientHeight);
        } else {
          (viewportwidth = document.getElementsByTagName("body")[0]
            .clientWidth),
            (viewportheight = document.getElementsByTagName("body")[0]
              .clientHeight);
        }
        return [viewportwidth, viewportheight];
      }

      document.getElementById("viewportsize").innerHTML = w + "x" + h;
    </script>

And there is a function directly lifted from Here's developer documentation to display the map - the only difference being that the lat/lon coordinates are passed into the function from the SimpleGeoIP call.

<html>
  <head>
  ...
     <script
      src="https://js.api.here.com/v3/3.1/mapsjs-core.js"
      type="text/javascript"
      charset="utf-8"
    ></script>
    <script
      src="https://js.api.here.com/v3/3.1/mapsjs-service.js"
      type="text/javascript"
      charset="utf-8"
    ></script>
  ...  
  </head>

  <body style="margin: 0">
  ...   
    <script>
        var platform = new H.service.Platform({
          'apikey': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
        });

      var lat = {{ geoip['location']['lat']}};
      var long = {{geoip['location']['lng']}};
      var maptypes = platform.createDefaultLayers();
       var map = new H.Map(
          document.getElementById('mapContainer'),
          maptypes.vector.normal.map,
          {
            zoom: 13,
            center: { lat: lat, lng: long }
          });
    </script>
  ...  
  </body>
</html>

The corresponding CSS is very basic and the only tricksy bit as a nice two column formatter.

And that's it! As long as things are in the correct sub-directories, (templates/ for the html and static/css/ for the CSS) it just works. Running it and browsing to 192.168.1.15:5000 gives a page similar to this:

That was run via my VPN. My normal home location does actually include my City details, but not my Postcode.

Basic Install and fix Update problem

One of the problems with the new Jetson Nano 2GB board is that a lot of the tutorials and install guides are outdated and lead to problems. So here's my take based on Jetpack 4.4.1 Jan 2021.

Install - simply download the system image from the Nvidia site, use Balena Etcher to write an SD card on your main machine; boot the Nano and follow the normal Ubuntu install instructions.

Update with the normal

sudo apt-get update && sudo apt-get install

DO NOT REBOOT despite being told one is necessary

Clear the old files

sudo apt-get autoremove

The update process breaks the lightdm greeter screen and if you reboot you will be presented with a blank screen. No problems, you can simply SSH into the Nano and perform the following, but, of course, it is simpler to do this whilst still logged into the Nano, before rebooting.

Add Nano editor if you don't like Vim

sudo apt-get install nano

Get the lightdm gtk greeter

sudo apt-get install lightdm-gtk-greeter

and tell lightdm to use this greeter:

sudo nano /etc/lighdm/lightdm.conf.d/50-nvidia.conf

add the following line to the end

greeter-session=lightdm-gtk-greeter

You should now get a login screen on reboot and continue as normal

Install Microsoft VS Code

Follow this instructions here: https://www.jetsonhacks.com/2020/11/28/visual-studio-code/

Basically clone the git repository and run

./installVSCode.sh

from within it.

Fix OpenCV code completion and pylint errors in VS Code

Big thanks to Paul McWhorter at TopTechBoy who has an excellent  set of Jetson Nano, opencv and AI tutorials on Youtube. Beware, though, the earlier episodes were written for previous versions of Jetpack and are out of date - following the instructions on this website is better for Jetpack 4.4.1. BTW Paul, in turn, thanked David Westmorland for coming up with this fix

Firstly check the location of your python opencv install:

dpkg -L libopencv-python

It is probably at

/usr/lib/python3.6/dist-packages/cv2/python-3.6

Start VS Code and go to preferences with

ctrl-shift-p

Search for

Preferences: Open Settings (JSON)

If you haven't configured VS Code then this file is probably empty (save a couple of braces), add the following two lines

"python.linting.pylintArgs": ["--generate-members", "--extension-pkg-whitelist=cv2"],

"python.autoComplete.extraPaths": ["/usr/lib/python3.6/dist-packages/cv2/python-3.6"]

Note: The path in the second expression must be the same as the path as shown by that previous dpgk -L command.

Save that preferences file and exit VS Code. On restart opencv pylint errors should have gone and code completion/prompting should work for opencv

Install matplotlib

I wanted to play with matplotlib so simply installed via

sudo apt-get install python3-matplotlib

A complicated little beast. 2 PLLs which can be integer (preferable, less jitter) or fractional multiples of the reference crystal oscillator. Max freq for a PLL is 900MHz with max crystal multipliers being in the range 15-90

After setting the PLL the output clocks are either integer or fractional divisors of the selected PLL. I believe the range is 4 to 900. If fractional then parameters need to be supplied in the format multiplier, numerator, denominator where numerator and denominator are limited to 20 bits (i.e. 1048575). Fortunately Python makes this pretty straightforward,

Here's some simple but crude code to calculate a spot frequency from a given PLL frequency:

from fractions import Fraction

def calc_freq(pll_freq, freq):
    mult,rem = divmod(pll_freq, freq) 
    if mult < 4 or mult >900:
        raise Exception("Target frequency produced invalid divider for the given PLL frequency")
    frac=Fraction(rem/freq).limit_denominator(1048575)
    return int(mult), frac.numerator, frac.denominator

def set_freq(clock,pll,freq):
    m,n,d = calc_freq(pll.frequency,freq)
    if d == 1:
        clock.configure_integer(pll, m)
    else:
        clock.configure_fractional(pll,m,n,d )
        
        
 set_freq(si5351.clock_1,si5351.pll_a,9.0e6) #sets clock1 to 9.0MHz       

To divide down further there is an additional R-divide which can be applied to a specific clock, values are multiples of 2 from 1 to 128.

So, I've been playing around with the Raspberry Pi and home automation devices such as IKEA Trådfri lights, a TP-Link HS100 switch, my Netatmo smart thermostat and a few ESP8366 devices including various varieties of Sonoff switches. Rather than blunder on I thought it would be nice and take stock of what I've done and install everything cleanly from scratch.

Most of the pearls of wisdom herein have been taken from this excellent guide: https://screenzone.eu/arduino-mqtt-sensor-grafana/

Here is the software I want installed just to control the devices:

• Node-RED
• Mosquitto MQTT broker

But one of my Sonoff's is a Sonoff POW which provides real time power measurements and I also want to hook up at least one of my ESP8266 to a DHT22 temperature/humidity sensor and measure temperature real time. So some method of graphing the data over time would be nice. Some more software will be required for this:

• InfluxDB
• Grafana

That's the shopping list for now so let's get started.

Raspian

The Pi will be running purely as a server so I don't need the graphics interface. But I do need wireless.

So, install Raspian Stretch Lite as normal, add a HDMI connection and keyboard, just for now and login. To add wireless we need to go edit the wpa-supplicant file:

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

and add the following lines:
network={
ssid="myWifiSsid"
psk="myWifiPassword"
}
with the appropriate ssid and password. Note the password is in the clear - we can generate the hashed password but I didn't know of an easy way of copying it and pasting it into the editor - but once we have SSH running this is easy from another computer so edit raspi-config

sudo raspi-config

to change the password, expand storage, set the Hostname, and make sure SSH is enabled. And since we are not using graphics we can drop the memory split to use the minimum 16MB for graphics.

Reboot, check wifi is working and remember the IP address. Now we can ssh into the Pi and generate a hashed wifi key

wpa_passphrase "myWifiSsid" "myWifiPassword"

which will produce output like
network={
ssid="MyWifiSsid"
#psk="MyWifiPassword"
psk=ab058a266ce8fe3b5cdc92a6ab6edd35ce13f88797614e21b8af5ba0e4afb784
}
so we can re-edit wpa_supplicant.conf and replace the original psk definition which was out wifi password in the clear with the new hashed version.

All that's left to do is ensure the Pi has all the latest packages:

sudo apt-get update && sudo-apt-get upgrade

Node-RED

Node-REDd comes pre-installed with Raspberry Pi full install images but not with the Lite versions. The documentation and installation of Node-RED is excellent - check https://nodered.org/docs/hardware/raspberrypi and all we need is

bash <(curl -sL https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered)

Now add Node-Red to systemd so it auto-starts:

sudo systemctl enable nodered.service

reboot and test. Use any browser on your home LAN and navigate to Pi's ip address:1880

Security

Before we can edit the ~/.node-red/setting.js file we need to generate a hashed password. Install node-red-admin:

sudo npm install -g  node-red-admin

Now we can generate the password with:

node-red-admin hash-pw

Enter your desired admin password at the prompt and copy the resultant string.Now edit settings.js file and uncomment the adminAuth section and replace the password with the one you have just copied. Restart node-RED (or reboot) to activate the change.

MQTT

There are a lot of websites explaining that MQTT is broken on Debian Stretch but it looks like the repositories now have a slightly back-level version of MQTT which has all the dependencies met so install is the simple:

sudo apt-get install mosquitto mosquitto-clients

While we're installing let's install Python support for MQTT too, just in case - this is a simple python pip install but ... the Lite version of Raspian doesn't have python pip installed so firstly

sudo apt-get install python-pip
sudo apt-get install python3-pip

then we can install paho:

sudo pip install paho-mqtt

To test MQTT simply subscribe to a topic by:

mosquitto_sub -h localhost -t "test"

Leave this running and start a different shell (I simply used SSH) and publish something to that test topic:

mosquitto_pub -h localhost -t "test" -m "test msg"

Back on your subscribe shell the message payload test msg should appear and all is well.

Security

The Screenzone Guide suggests it might be a good idea to add some security to MQTT in the form of requiring a Userid and Password to publish or subscribe to messages.

Firstly we create an authorisation file:

sudo nano /etc/mosquitto/conf.d/auth.conf

and add the lines:
allow_anonymous false
password_file /etc/mosquitto/passwd

Then create a user homeauth and supply a password

sudo mosquitto_passwd -c /etc/mosquitto/passwd homeauth

Restart mosquitto (or reboot) for the changes to take effect:

sudo systemctl restart mosquitto

Now let's try our tests again - the simple sub and pub tests above both fail to connect because of an authorisation error - so let's add authorisation:

mosquitto_sub -h localhost -t "test" -u "homeauth" -P "mysecret"

and

mosquitto_pub -h localhost -t "test" -m "test msg" -u "homeauth" -P "mysecret"

and confirm they work as previously.

InfluxDB

https://bentek.fr/influxdb-grafana-raspberry-pi/

curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -

echo "deb https://repos.influxdata.com/debian stretch stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

sudo apt-get update

sudo apt-get install influxdb

Grafana

curl https://bintray.com/user/downloadSubjectPublicKey?username=bintray | sudo apt-key add -

echo "deb https://dl.bintray.com/fg2it/deb stretch main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

sudo apt-get update

sudo apt-get install grafana

sudo nano /etc/grafana/grafana.ini

[server]
# Protocol (http, https, socket)
protocol = http
# The ip address to bind to, empty will bind to all interfaces
;httpaddr =
# The http port to use
httpport = 3000

sudo service grafana-server restart

sudo systemctl enable grafana-server.service 

Configuring Node-Red

Let's use a simple example from a Tasmota flashed Sonoff-POW

The MQTT message is:
"{"Time":"2018-08-07T12:04:27", "Total":31.918, "Yesterday":0.229, "Today":0.003, "Period":0, "Power":1, "Factor":0.00, "Voltage":253, "Current":9.362}"

First step is to use the influx CL to create a database
'create database power'

Then a simple function is needed to convert this to a payload into the requisite influx payload format - we are probably only interested in power, current and voltage so simply converting the payload to JSON and taking the appropriate parts works:

node.status({fill:"blue",shape:"dot",text: msg.payload});

tmp=JSON.parse(msg.payload);

msg.payload = {
	power: tmp.Power,
	voltage: tmp.Voltage,
	current: tmp.Current,
	sensor:"Sonoff-POW1",
    };
    
return msg;

*The first line just output the incoming message payload as status under the function box in Node-Red for debugging
*

Just 3 Node-Red nodes are needed an MQTT output node to pass the published message to the Function node and the output from that is simply passed to an Influxdb input and data is written to the database.

Using the influx CL
use power
to select the appropriiate database

show measurements should show the measurement name you specified in the Influxdb Node-Red node - in my case it is "Sonoff" so

select * from Sonoff
shows

time                current power sensor      voltage
----                ------- ----- ------      -------
1533633288497843297 0       0     Sonoff-POW1 254
1533633318505875412 0       0     Sonoff-POW1 254
1533633348715756458 0       0     Sonoff-POW1 255
1533633378731317810 0       0     Sonoff-POW1 254
1533633408891859566 0       0     Sonoff-POW1 254
1533633438910263696 0       0     Sonoff-POW1 254
1533633469053749624 0       0     Sonoff-POW1 254
1533633499139945397 0       0     Sonoff-POW1 254
1533633529244007464 0       0     Sonoff-POW1 254
1533633559250354927 9.44    1     Sonoff-POW1 254
1533633589398467843 9.44    2     Sonoff-POW1 255
1533633619458944619 0       0     Sonoff-POW1 255
1533633649730510602 9.474   1     Sonoff-POW1 256

*Yes, I didn't have a load on my Sonoff-POW at the time!"

The UK Digital Economy Act of 2017 brings into force the strong possibility that the UK government will introduce age verification to certain websites (i.e. porn). This, in itself, is worrying and something I strongly disagree with. Sadly the genie is out of the bottle with internet content. If working with 8-10 year old kids at Code Club has taught me anything it is the fact that they are much more computer savvy than we might expect; I foresee kids using TOR and the Dark Net as a simple way of circumventing such measures. Driving them underground makes as much sense as criminalising drugs, and so driving people who seek them into the hands of an underworld selling products of dubious prevenance.

But even more worrying is that the act grans Ofcom new powers that will allow them to collect or demand more information from broadband ISPs and mobile operators. Whilst my internet browsing is fairly innocous it is something which I would prefer the option as to what I share and with whom rather than acceeding to a government's carte blanche spying.

So, time for that VPN I've been considering from some years. A quick survey of recommendations suggests that NordVPN is a good candidate for my needs. Even better it has an offer on currently with 3 years service for $99 which works at at a penny or two over £2 a month. Even better it has a (not well advertised) 3 day free-trial; no sign-up required, no credit card, just an email address registration.

It comes with applications for Windos, Mac, Android etc but nothing for Linux. I won't detail the steps here because the NordVPN documentation does a god job but for Linux:

1. Install OpenVPN
2. Download the NordVPN server configuration server files
3. Configure Network Manager to use one of the server configs

Each step was remarkably simple and up and running in just a couple of minutes. Download speeds drop from 34Mbs to 30Mbs but I can live with that.

An even easier install is the browser plug-in so that geo-location can be switched with a click. Works well, I could listen to the Leonard Cohen tribute concert via Canadian CBC replay - this was rejected yesterday using both Opera's built-in VPN proxy and the Hola VPN plug-in for Chrome. Also tested was switching to the USA and, sure enough, BBC iPlayer content was no longer available.

Android is a simple app, installed and one click turns on or off the VPN. All is good.

Snowball Fi-i-i-ight!

You are going to throw a snowball at your classmates. How many times will YOU get SPLATTED!

Step 1. An introduction to Radio on the micro:bit

The micro:bit has a radio chip that can be used to send and receive messages. You can set each micro:bit to a 'radio group' so that it will only send and receive messages within that group. This is very handy - imagine you and your friend are playing a game on two micro:bits sending messages to each other. If another pair of people are playing the same game then your micro:bits might receive their messages as well and this would spoil the game. But if each group of players is on a different radio group then they will only send and receive messages in their own group and so won't interfere with nearby players and so, all the games will work.

We are going to take advantage of that by setting every micro:bit to a different radio group and then when we want to send a message to someone else we will temporarily change our group so it matches the person we want to send to and then we can send them a message. That way we know only one person will receive our message.

Step 2. Setting up the basic variables

Let's write some start code that sets up a different number for each player and also says how many people are playing - we'll need that later.

STOP - Did you just copy that code?

Remember we said earlier that we are going to assign each micro:bit to a different radio group. Well, if we all just copied the code above, what has happened? - everyone has set their radio to group 0.

Everyone needs to have a different value for mynumber. So, agree who will be 0, 1, 2 ,3 etc. - the numbers should start with 0 and be in order with no gaps.

Also note the variable:

• this holds the number of people in our snowball fight. I've guessed there will be 7 or us here today (so the last player will be number 6 - remember we started at 0) If there isn't 7 players then change this. It should be the same value for everyone.

We'll add a long 5 second pause to make sure we can all check we have the correct player number.

If we are going to have a snowball fight - someone needs a snowball. We could give everyone a snowball but chaos would happen. So let's give just one player a snowball to start with so he can throw it to someone else.

We will give it to player 0 and we will also set a variable to say that he currently has the snowball. For everyone else we will tell the code that they don't have the snowball. We will do this using something called a Boolean variable (named after a Victorian called George Boole who first thought of this sort of thing). A Boolean variable is just like any other variable but it doesn't hold a number or a string, it holds either the value True or the value False. So we will make a variable called ihavethesnowball and player 0 will set it to true and everyone else will set it to false.

We could write different code for player 0 to set ihavethesnowball to true. But it is easier if we all have the same code - so let's test the value of the mynumber variable and make the code to be different for player 0. Let's also give player 0 a great fat snowball on their screen, and since no-one else has a snowball we'll clear their screens.

Here's the complete on start code:

Remember your code may have a different value of mynumber and if there aren't 7 of us here today then lastplayer will be different too.

Step 3. Throwing the snowball

We will shake the microbit to throw the snowball. But you can't throw the snowball if you don't have it, so we will make the on shake code do something only for the player who currently has the snowball - like this:

Note. Does that if test look strange?

Normally when we do an if test we need to have an = or > or similar sign in, like "if score=10" or "if tries > 2". Because the ihavethesnowball variable is a Boolean variable that can only have the values true or false we could say "if ihavethesnowball = true" here and this would work, in fact it would be exactly the same. But I think the simpler "if ihavethesnowball" makes slightly more sense. What do you think? If you prefer "if ihavethesnowball = true" then you can change the code.

We need to decide who to throw it to. For now we can select a random person. This is easy we can just use:

But there is a problem?

STOP - Can you see the problem?

That code can pick any player - including the player who is throwing the snowball. Now, not only is it stupid to throw the snowball and splat yourself but the micro:bit radio code doesn't allow messages to go to the player who sent them, so the message will just get lost and the snowball disappear into space.

We need to write some code to pick a random number but make sure it isn't our own number.

STOP - how would you do this?

There are several way but the way we'll use is to pick a random number and it is our own player number we'll pick another one and keep doing so until we get a different player.

One way to write this in micro:bit code is to deliberately set the throwto variable to our own number (the very thing we don't want!) and then loop round picking random numbers until we get one which isn't our number. Like this:

So we have someone to throw the snowball to (the player number indicated by the variable throwto)

When we are ready to throw the snowball we will change our radio group to match the radio group of the person we are throwing to. Since they are the only other person in that radio group when we send a message only they will get it.

Once we've thrown the snowball (that is, sent the message) we will change our radio group back to our own player number.

Also in the message we send we'll put our own player number, so they know who threw the snowball at them.

Of course, once we have thrown the snowball we don't have it any more! So we better remove the snowball off our screen and tell our code that we don't have it.

So the code is going to be like this

This is quite complicated so let's just check what that code does - if we don't have the snowball it does nothing. Easy! If we do have the snowball we pick another player and switch to their radio group so we can throw the snowball at them (i.e. send them a radio message). Then we set the radio group back to the one we assigned at start. This is important because it now means we are back in a radio group with just us in it, so we are ready for any snowballs being thrown our way in the future. And finally - we don't have the snowball anymore so we clear the screen and tell the code.

Step 4. Getting hit!

We know we've just been hit by the snowball when we get a radio message. Remember the code is designed so we are the only one that will get a message. The message contains the number of the player who threw the snowball but we won't use that information just yet, we'll just react to being hit (that is receiving a radio message)

The code is really easy - when we receive a radio message we just plop a big snowball on the screen and tell our code we have the snowball (by setting the variable ihavethesnowball to true). Just to slow things down a little we'll also add a pause, otherwise the snowball will zip around the group a bit fast.

Here's the code:

Let's have a SNOWBALL FI-I-IGHT!

Challenge 1: A better SPLAT!

Let's put some simple animation to look like the snowball is coming to get us. Here's my code - can you make something that looks better?

Challenge 2: Counting the splats!

Let's count how many times each of us get hit and as soon as someone gets 6 hits end the game. We just need to count each time we received a snowball.

Let's do the count code first - when do we know we've been splatted?

The answer is every time we receive a radio message - so in the on Radio Received code we can just add one to the count. First we need to set the count to 0 and the best place to do that will be On Start. We will also set a variable maxsplats to say how many splats a player can get before the game ends

Note - setting a variable like maxsplats makes tidier code

We could just test for the number 6 in the code, but if we wanted to make the game longer or shorter we would have to hunt around in the code and find this number and change it. By having a variable with an obvious name right at the top of our On Start code it makes it much easier to change things in the future

and then just add 1 when we receive a snowball splat

So now all we need to is check when splats = maxsplats and end the game.

STOP - can you think of a problem about ending the game?

It is easy to end the game on our micro:bit, we can just check if we've been splatted the maximum number of times and put out a message. Like this:

But how do we tell everyone else to stop?

The only way we can communicate with them is to send them another radio message. Let's send them an unusual number - like 99, to say the game has ended. We need to send this message to every other radio group. It's a bit easier than the random code. Can you think what the code is like?

This code will work:

We loop around for all the player numbers and for every number that is not our own we set the radio group to that number and send a 99. So every other player should receive a 99 radio message.

Now we have to write some code to handle that special radio message telling us the game has ended.

We can simple check when we receive a radio message whether it is 99 and so we have to end the game, if it isn't then it must be an incoming snowball and we get splatted as usual. So our radio receive code now starts like this:

And that's it!

Not quite - the player who last got a Snowball still has it, and so if he shakes then it will get thrown and the game will continue. This is a nasty code bug which, fortunately is easy to fix. All we need do is tell the player when he gets splatted the maximum number of times that he no longer has the snowball - the game is over and the snowball has melted!

We'll add that into the end of the radio receive code which will now look like:

Now that really is it?

STOP - can you still see a potential bug?

What would happen if there were more than 99 players in this game? The snowball could never get thrown from player 99 - if it did then the receiver would get a radio message with the thrower's number 99 and the code would tell that player it is the end of the game.

In reality we are not likely to play this game with 99 players or more so our code is probably OK but we need to bear this in mind as a possible problem. Often in computer programs bugs occur because the person who wrote the code though "Oh, this could never happen, so I won't bother writing a lot of code just in case it does".

I think we are safe in assuming our Code Club won't get 99 members so -

Let's go and Snowball Fi-i-i-ght!

Complete Code

Download the code here:

https://makecode.microbit.org/_ccjDyAAtH4kR

Expansion (Teacher's note)

A really nice expansion for this would be to change the randow throw to allow the player with the snowball to point the micro:bit at the person they want to splat! If the players were in a circle and of known compass orientation then the micro:bit could use the compass bearing to know in which direction it was pointing before being shaken and so calculate which player gets splatted!

Fill It Up

You are going to make a game where the object is to light all the LEDs on the Micro:bit in the fastest possible time. One LED will flash and you will tilt the Micro:bit to roll the flashing LED around the screen. Every time it rolls to a new LED the new LED will light it up.

All you have to do is light up all 25 LEDs as fast as possible. Easy?

Step1: Rolling an LED around the screen

We're going to use the LED screen coordinate system to control the movement of the LED. The screen has two coordinates: X, running from left to right and Y running from top to bottom. Each starts at 0 so the top left hand corner is X = 0 and Y=0 and the bottom right hand corner is X=4 and Y=4

Let's set an LED in the middle of the screen. This is where X=2 and Y=2. Since we will be changing these values we need to create 2 variables, let's call them 'cur_x' and 'cur_y'.
('cur' is short for 'current' and is often used in variable names to indicate that this is the current value of the variable but it may change)

That doesn't do much so let's light up the LED at that point

We can use the accelerometer function of the Micro:bit to detect when the Micro:bit is being tilted. To start with we will just detect movement along the X-axis and light up the next LED when we detect a tilt

We'll need to do this in a Forever loop. Every time the Micro:bit is tilted to the right the X acceleration will become greater than 0 and so in that case we will increase the value of cur_x so that the next LED to the right lights up. Similarly if the Micro:bit is tilted to the left we need to decrease cur_x so the LED to the left is lit up.

Try it. It's very sensitive and hard to get just the next LED lit up. Slow it down by adding a pause statement (of say 100ms) after the plot statement in the forever loop

We can do the same thing for moving LEDs up and down, so let's add similar code for the Y axis - up and down movement

Step2:Fixing the bug!

Can you roll the LED round the screen. I suspect you are probably having difficulty and sometimes the LED just doesn't seem to be moving.

STOP - Can you think why?

Remember what we said earlier that the screen goes from X=0, Y=0 in the top left corner to X=4, Y=4 at the bottom right corner.

Got it yet? Whenever we detect any acceleration we increase or decrease the value of cur_x or cur_y. So cur_x or cur_y may have values bigger than 4 or less than 0. When we use a plot statement with values outside the screen such as "plot x -2 y 7" then nothing gets plotted. So we only see LEDs being plotted when cur_x and cur_y have values between 0 and 4 inclusive.

To fix this we need to put some test into our code so that if cur_x or cur_y are already 0 or 4 we don't change them and make them go out of range.

Here's the code to stop cur_x being >4

Can you write the code to make sure cur_x and cur_y are always within the range 0 to 4?

Well, that might be working, it's hard to tell because we can't see where we are. Let's add an unplot so that we can see that we are just moving around the LEDs correctly.

That's great - we can move the LED around, just tilting the Micro:bit and it always has a LED lit. However, before we added that unplot we almost had a complete game, the square of LEDs was filling up nicely.

What's the difference? Well, now the last thing in our Forever loop is an unplot which turns the LED off, so when we move to a new position the previous LED is turned off. Previously our loop ended with a plot and so the LEDs stayed on. So having the plot at the end of the loop is good. What do you think will happen if we do the unplot first and then the plot?

Let's try it, adding a delay

Wow - that works! The screen fills up nicely and the current position of the LED we tilt is shown by flashing - perfect. We have a game.

Almost!

Step 3: Checking for Game End

How do you think we know when the game ends?

That's right, all 25 LEDs are lit up. So we need a variable that counts every time we light up an LED and when it reaches 25 we know we have finished.

What happens when we move back over an LED that is already lit? We definitely don't want to count it again. So we need to make sure we only count LEDs that are unlit before we tilt onto them for the first time whicxh is what makes them become lit

So first of all we need a variable, and it is going to start off with the value 0 so no LEDs are lit. We can put this in our On Start code

Now after we have just moved (i.e. all those "if acceleration" statements) we need to check if the point we have moved to is unlit and if it is we can add one to our count

OK so now we know how many LEDs are lit up and we know we want the game to end when 25 LEDs are lit. How are we going to use this? Also can we make the game start again when we've finished?

Instead of it starting the game automatically we'll make it only start when button A is pressed. And instead of a Forever loop we'll code a While loop that continues for as long as there are LEDs still needing lighting - that is as long as LEDScount is less than 25.

That works. As some as the screen fills up we put put the message "Finished"

Step 4: Adding a timer

Great, now if we could just replace that finished message with a message that says how long it took to fill the screen we would have a good game.

The Micro:bit has a clock (runninngtime) that says how long it has been running (in milliseconds). So we can tell how long we have taken to complete the game by looking at the time when we start the game and the time when we end the game and subtracting.

So at the beginning of the On Button A block before the While loop let's add another variable to record the starttime.

And at the end of the game we can work out the time taken in milliseconds. If we then divide this by 1000 we get seconds which is better to tell the user. We also need to report this

That's it

Almost!

If we press button A we get a new game but the screen is full up so we can't play. Simple to fix, we just need to a a Clear Screen instruction everytime we start the game.

Step 5: Making it easier

The game is very difficult to control. That is because whenever we notice that the Micro:bit's acceleration is not 0 we move the LED. This makes the game very 'twitchy' - it would be a lot less sensitive if we only moved the LED if the accereration was greater than a value bigger than 0 - the bigger the value we choose, the less sensitive the game becomes. Let's try a value of 50 and so we want to replace all of the code that says if acceleration(mg)x > 0 with if acceleration(mg)x > 50 and the same for acceleration(mg)y. We also want to replace if acceleration(mg)x < 0 with if acceleration(mg)x < -50 and again for acceleration(mg)y.

We can easily make these changes, but what happens if we find it is still too sensitive and we want to try a sensitivity of 100 rather than 50? We have to changes the code in lots of places. Whenever we change code in several places there is always a chance we will miss one and so introduce a bug into our code. How can we avoid this?

It is good coding practice to use a variable such as max_accel which we set once and once only in our On Start code and then do tests such as if acceleration(mg)x > max_accel. Now if we want to make a change we just need to change the single line of code where we give max_accel a value and everywhere in the code will use the new value. No chance of missing anywhere and introducing a bug.

So we will set max_accel in oue On Start block and also to allow for the less than tests we will define another variable min_accel as -1*max_accel (i.e. if we set max_accel to 50 then min_accel will be -50, just what we need for the if acceleration(mg)x < ... tests.

Another way of tweaking the speed of the code is to adjust the delay every time we plot and unplot the flashing LED. Currently we have this as a fixed 100 milli-second delay but we might want to explore changing this to make the game easier or harder so let's also replace those pause(ms) 100 lines to test a variable called slowness.

Here's the On Start code:

Complete Code

Download the code here:

https://makecode.microbit.org/_agRexb6g35sf

Installing Ghost is so easy using ghost-cli and the installer does everything. Well, almost everything. Three things left for me to do on my Technicals blog:

1. Set up mail to allow resetting forgotten passwords
2. Adding Disqus commenting
3. Adding Prism code highlighting

1. Setting up mail

The Ghost documentation covers this here - https://docs.ghost.org/v1.0.0/docs/mail-config

Basically if you have a mailgun account then you simply edit the config.production.json (or config.development.json, as appropriate) and replace the "mail section" with something like the following

"mail": {
    "from": "'Test Ghost Blog' <test@two-drifters.co.uk>",
    "transport": "SMTP",
    "options": {
        "service": "Mailgun",
        "auth": {
            "user": "postmaster@mg.two-drifters.co.uk",
            "pass": "********************************"
        }
    }
},

Restart Ghost and head on over to the Labs section of the Ghost admin page to send a test email.

One minor annoyance is that I get such emails flagged with an 'unencrypted' icon within Gmail, despite mailgun using TLS security. This seems to be some interaction with GoDaddy's secureserver.net - I have an open problem with GoDaddy and will report back if and when resolved. It's more an annoyance than problem.

2. Adding Disqus commenting

If you use the default Casper theme this is already coded but commented out. Navigate to the content/themes folder and select your current theme. Then edit post.hbs and scroll down to the Disqus section and uncomment it -(the easiest way is simple to move the --}} line from the end of the section to the line before the <section class="post-full-comments"> line - so just the explanatory documentation is commented out.

There is one further change - locate the line s.src = 'https://test-apkdzgmqhj.disqus.com/embed.js'; and replace the test URL identifier with the Disqus identifier for your blog.

3. Adding Prism code highlighting

This is only necessary if you want to include code snippets in your blog. This Technicals blog will almost certainly do so. So, head on over to Prism http://prismjs.com/download.html, select the theme, languages and extensions you want to enable and download both the javascript and css files.

Now, navigate to your themes content/themes/ theme /assets folder and copy the downloaded javascript file into the js folder. Similarly copy the css file into the css folder.

To activate Prism you need to edit your theme's default.hbs file and add the following two lines to the appropriate section:

{{! Styles'n'Scripts }}
<link rel="stylesheet" type="text/css" href="{{asset "css/prism.css"}}"/>

{{!-- jQuery + Fitvids, which makes all video embeds responsive --}}
<script type="text/javascript" src="{{asset "js/prism.js"}}"></script>

Yeah, that seems to work!

Now we have a development blog let's turn our thoughts to production. A production blog needs to restart automatically at reboot or in the event of a failure and really ought use SSL.

Colour me impressed - ghost-cli does both of these things out of the box without any further work from the installer. It configures systemd to provide automatic failure restarting and it acquires certificates from letsencrypt.org and automatically configures nginx to use these to provide SSL security. This means that post-install configuration of Ghost is now much simpler than the previous 0.x versions.

One decision needs to be made before installation. Do you want ot run with an sqlite3 database or do you need the robustness that mysql offers. Personally I think that a Raspberry Pi is best suited to sqlite3 but if you prefer mysql you will need to install it prior to installing Ghost. I didn't so don't have install instructions here.

Additionally, since Ghost is going to set up SSL keys via Letsencrypt then you must have all of the DNS entries set up correctly in advance so that Letsencrypt can find DNS entries for your blog.

To create a production system then follow the steps in the previous post detailing creating a Test, Development Blog except for the final ghost install local step. Of course, if you've actually created such a blog on your server already then you don't need to re-install the software again, just start at the point where you need to create a var/www folder for your new blog.

Once you have the folder, with the revised owner and have navigated to it then you can simply install Ghost with

ghost install --db sqlite3

if you decided to use mysql then you just need ghost install
And that's it

The installation process will warn that you are not running Ubuntu 16.04 but you can simply ignore that warning. It will then install Ghost which takes quite a while and ask do you want nginx, systemd and SSL configured before finally asking whether you want Ghost started. Be aware that generating the keys for SSL takes a long time, over 30 minutes on my un-overclocked Raspberry Pi 3. Also be aware that the response from Ghost starting occurs several seconds before Ghost has successfully started - so should you received a 502 Bad Gateway error it may just be that we were a little eager in trying out your new system. Wait a few seconds before retrying.

This summer Ghost underwent major changes when it moved up to version 1.0. One of these changes was a new command line interface which also installed Ghost itself.

The font of Ghost installation knowledge on a Raspberry Pi is https://ghostpi.pro who had a post dated August where it was suggested that Ghost 1.0 no longer installs on a Raspberry Pi running Raspbian but needs Ubuntu 16.04. I thought I'd explore the problem and see where the errors lay with little hope that I'd be able to fix them - but 4 months is a long time in software and the latest Ghost 1.19.0 (at least) installs on a Raspberry Pi without problem. In fact, the ghost-cli installer makes the whole process even easier than previously.

Let's do an install from a fresh raspian-stretch-lite image (in my case it just happened to be the image dated 29-11-2017

After installing the image and whilst still on the install computer as opposed to the Raspberry Pi I added an empty file ssh to the boot partition to allow for SSH to the Raspberry Pi immediately - so I didn't need to hook up a monitor and keyboard to the Raspberry Pi. I also edited the file /etc/dhcpcd.conf to give the Raspberry Pi a static IP address. Much easier to find it than using dhcp.

Then SSH to the Raspberry Pi, login and make sure all the software is up-to-date

sudo apt-get update && sudo apt-get upgrade

and change the default password, reduce the amount of graphics memory and expand the file system using raspi-config

sudo raspi-config

which needs a reboot to take effect.
 

We'll use Sqlite3 rather then MySQL so need to install it:

sudo apt-get install sqlite3

Next, install nginx

sudo apt-get install nginx

nginx needs a tweak for ghost.
Edit /etc/nginx/nginx.conf and uncomment
server_names_has_bucket_size 64;

Now install node.js - 6.9.5 is the recommended version for Ghost.

wget https://nodejs.org/dist/v6.9.5/node-v6.9.5-linux-armv7l.tar.gz
sudo mv node-v6.9.5-linux-armv7l.tar.gz /opt
cd /opt
sudo tar -xzf node-v6.9.5-linux-armv7l.tar.gz
sudo mv node-v6.9.5-linux-armv7l nodejs
sudo rm node-v6.9.5-linux-armv7l.tar.gz
sudo ln -s /opt/nodejs/bin/node /usr/bin/node 
sudo ln -s /opt/nodejs/bin/npm /usr/bin/npm

Once node.js is installed we can install ghost-cli

sudo npm i -g ghost-cli@latest
sudo ln -s /opt/nodejs/bin/ghost /usr/bin/ghost

We probably don't need yarn but the node.js install suggested it might be useful so why not install it now

curl -o- -L https://yarnpkg.com/install.sh | bash

We're now ready to install ghost itself, so let's make the directory for the blog and ensure we have all the permissions we need.

sudo mkdir /var/www/ghost
sudo chown -R pi:pi /var/www/ghost

and install ghost in to it - NOTE this takes quite a while, be patient!

cd /var/www/ghost
ghost install local

And that's it. If you want to access the test Ghost from a different computer on your LAN you need to edit config.development.json. Change the first "url" line from localhost to the actual IP-Address of your Raspberry Pi Ghost server and similarly change the "host" line a couple of lines lower to also refer to thesame IP-Address rather than 127.0.0.1

Restart Ghost with

ghost restart

and you should be able to access yor blog from within your LAN on IP-Address:Port

Just a couple of notes because this was trickier than expected, especially since all my experiences with Arduinos and GPSs have been fairly straightforward.

I bought a cheap USB GPS which turns out to be a Neo-7 - dmesg giving

[   28.770396] usb 1-1.5: New USB device found, idVendor=1546, idProduct=01a7
[   28.770426] usb 1-1.5: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   28.770443] usb 1-1.5: Product: u-blox 7 - GPS/GNSS Receiver
[   28.770459] usb 1-1.5: Manufacturer: u-blox AG - www.u-blox.com
[   28.771952] cdc_acm 1-1.5:1.0: ttyACM0: USB ACM device

So, 'simples' thinks I: install gpsd and gpsd-clients and then start gpsd with sudo gpsd /dev/ttyACM0 -n -F /var/run/gpsd.sock and running cgps -s will give me my location.

Not so fast there. cgps is giving me no fix and timing out.

First check the gps is working by simply looking at the serial output stream with
cat /dev/ttyACM0

Yep that works - there are the NMEA sentences and I can see the valid fix data. SO the problem must be with gpsd.

Long story short, after looking at syslog and realising it must be a socket problem I Googled around and found this page https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=138711 where I owe a big THANK YOU to Per Linderholm for his post suggesting changing /lib/systemd/system/gpsd.socket

Here's the relevant bit of his post

I think the problem is in how systemd is configured.

Try to change /lib/systemd/system/gpsd.socket
from

ListenStream=127.0.0.1:2947

which listens to localhost only
to

ListenStream=0.0.0.0:2947

which listens to "all" , and restart gpsd (or reboot).

Thanks Per that fixed it and now I get all my location data working nicely with cgps.

Alas, too soon I spoke Today I decided to rebuild the Raspberry Pi (don't ask!) and the above steps failed. I then remembered that I had previously tried something else from that link - delorian's hint that the socket needed deleting via:

sudo systemctl stop gpsd.socket
sudo systemctl disable gpsd.socket

Ah, now looking at syslog I can see gpsd is starting correctly and, yes indeed, cgps works!

My first tablet was a 1st gen Nexus 7. Probably as good as they got at the time. But then Android 5 came along and the Nexus 7 2012 version became unusable - performance plummeted and it was just too frustrating to attempt anything other than power-off.

I read recently that Android 6 and 7 have given it a new lease of life, but, of course, Google abandoned it years ago so any revitalisation would require a manually installed 3rd-party ROM. So let's do it!

Here's a checklist:

• Install and run Kingroot to get root access
• Install Root Checker to confirm rooted OK
• Install and run SuperSuMe to switch root access from Kingroot to SuperSu (Kingroot has problems with some apps - SuperSu is more reliable but harder to install from scratch)
• Install ClockworkRecoveryMod (CWM) to allow booting into Recovery Mode
• Download new Android ROM [Android 7.1 (Nougat)] and and appropriate Google Apps (Gapps)
• Boot into Recovery Mode
• Data reset, clear cache and Dalvik cache and install Android ROM
• Reboot/Poweroff and boot back into Recovery
• Data reset, clear caches and install Gapps

Tablet now in factory clean state and just need to set everything up.

Actually it was all fairly straightforward once Kingroot eventually installed. It took several attempts before I could get it to root. Wasn't it Einstein who defined "Insanity: doing the same thing over and over again and expecting different results" - yet Kingroot was insane, subsequent runs kept failing differently until after about 5 attempts root was successful.

The hardest part of the exercise was finding all the components - back 3 or 4 years ago the web was awash with Nexus 7 firmware hack sites but today there are broken links and links to out-of-date versions of Kingroot and, in particular, CWM - enough not to give you confidence that anything found is actually likely to work!

Anyway a new Nougat Nexus 7 2012 and, indeed all is sprightly. Happy campers (literally, the tablet is for the campervan).

Not so fast there! After performing flawlessly all weekend we left it without charge and the battery depleted. Now, no matter how long I charged it the device would not power-up. I tried all the tricks: power button for > 30 secs, power + vol down, charge for 156 mins and remove the cable before powering, remove back case, disconnect and reconnect battery - nothing!

Today I dug out my handy-dandy USB voltage and current measurer and started charging the device. It was drawing a charge current of 0.0A - i.e. not charging. Ah, that's the problem - it won't power on because the battery is flat and no end of pressing various buttons for eons can possibly wake it up until I can get some juice into it. Tried several cables and eventually one cable showed a charge of 0.44A and after 30 mins there was enough charge to actually attempt power-on, which worked first time without any of the hold for 30 secs palaver. But 0.44A is not a particularly fast charge rate, especially when attached to a 2A 5V PSU. The problem appears to be a worn USB socket - wiggling the cable readily switches back to not charging and there is definitely a lot of play of the plug in the socket.

A quick poke around the web - and yes - this is a very common problem. So common that USB connector assemblies are rife on Ebay for just over £3 a time, or if I want to solder I can just buy a compatible connector for half that price.

The saving isn't worth the effort of fiddly soldering - an assembly is on order. Watch this space.

Chuffed and flushed with success I tried the new websites and yes typing http://two.drifters-co.uk into Chrome automatically presents the home page as secure HTTPS. All the sub-domains work too.

The article that lead me to SSL informed me that Opera had a built-in VPN. So I tend to use Opera for much of my browsing now. I tried http://two-drifters.co.uk in Opera and, shock, horror Opera deemed it insecure

Opera checks HTTPS pages to ensure that no internal links are HTTP. Of course, my home page had all my sub-domain links as HTTP relying on the server to redirect to HTTPS. I changed these but still the site was deemed insecure. A little further investigation of the source and there are a couple of links to Cloudflare and Google Fonts that also need changing to HTTPS (because I'm sure Cloudflare and Google will support it) and, at last ...

Opera is Happy.