A bluetooth 'chorded' keyboard

Since way back in the 90s when a friend showed me details of a 'wearable' computer, as a programmer and electronics graduate, I was intrigued that a computer could be small enough to be carried around on your belt, that the screen could be tiny and attached to some large black rimmed glasses, and that the keyboard could be held and operated in one hand using only a few buttons. At the time this was research work going on in US universities and there was no way I was ever going to get my hands on that kind of equipment.
More recently with the easy availability of micro-controllers and smaller footprint computers, like the Raspberry Pi, this is far more accessible. However it's not really interested me because I can pretty much do everything I want on my smartphone, so why bother carrying another computer? What has stuck in the back of my mind though was that one handed keyboard, so I did some research.
The original Twiddler that I'd seen are still made, and have been updated, but for me there's no fun in buying one, I should be able to build my own. These devices are referred to as chorded keyboards, as you play a 'chord', similar to a musical instrument, with the switches in order to generate a key press.
What did I need?
  • A controller that could be easily programmed
  • Battery power
  • Bluetooth capability
  • Some switches
That should do it. Please note this is not intended to be a complete step by step to building a Chorded keyboard but an overview to help others get started with ESP32 and BLE. I'll write up the step by step at a later date.
So, on to the process. With an initial request for #makersHelp on Twitter, I got the suggestion from Wayne Keenan (@wkeenan) to try the ESP32 Dev board. The Adafruit feather was another option but the ESP32 was a lot cheaper. It also has a huge list of features including:
  • Wifi
  • Bluetooth
  • Digital and Analog IO pins
  • I2C
  • Capacitive touch

and on and on. I expect to be building and writing about many more projects based around this board.

I split the project into manageable pieces, each a working step towards the completed whole:
  • Setup for programming the ESP32
  • Configuring the ESP32 as a bluetooth keyboard
  • Switch input as trigger for keys
  • Create the case to hold the electronics and fit in my hand
Setup for programming the ESP32
There are a lot of details on the Esspressif website about how to configure the development environment for ESP32 but I didn't want to start with a completely new set of tools. I'd previously used the Arduino IDE for programming Arduinos and Wemos boards, so was familiar with it and had it installed. Another quick Google and I found a link with some build instructions:
What this didn't say, but I would highly recommend, is if you already have the IDE installed but are using an earlier version than 1.8.5 then uninstall it first and the then install atleast 1.8.5. I didn't and after my first run through I couldn't actually load any code onto the ESP32.
Another thing I found was not all ESP32 Dev boards are created equal. The Doit board I had uploaded new code fine from the IDE. However the low cost ebay ones I purchased required that the Boot button be held down on the board for the upload to take place.
Once I'd got Blink running though, I knew I was on my way.
Configuring the ESP32 as a bluetooth keyboard
It must be possible, I'd read and been told it could be done, how hard could it be?
Well, once you have the RIGHT instructions actually it's not too bad. There is a lot of information out there on Bluetooth LE, a little on presenting the correct interface to make a device think you are a keyboard, and after that it's pretty much up to you!
First I needed to understand about BLE, and for that I found these tutorials by Andreas Spiess:
That got me an understanding of how to get BLE working and be visible from my phone, the nRF Connect app that Andreas mentions is invaluable.
For this to work though I needed the device to present itself as a Human Interface Device (HID), specifically a keyboard, and to then send key press values to whatever it was connected to. Details of how to do that I found in this thread: https://github.com/nkolban/esp32-snippets/issues/230
Fortunately that was some time back and these are now all available in the ESP32 BLE Arduino library that can be installed in the Arduino IDE under Sketch->Manage Libraries.

Switch input
As I'd used the Arduino IDE, configuring the input was straightforward. I configured the ports to which the switches were connected as input, wired in pull up resistors and then could read values from them. Oddly if pin <> was connected when I uploaded code then the upload failed. I got around it by connecting to a different pin, and I may come back and investigate the root cause at some point. I also then did some more reading and found that similar to the Adruino I could configure internal pull up resistors on the ESP32 to save some wiring.
Initially I combined the switch/key mappings from the Sparkfun code, and the keyboard example from the BLE library. This then gave me a working device! I could press the first button and get an 'a', the second and get a 'b', both together and get a 'c'.
However there were a number of issues here:
  • My device was constructed from 5 switches hot glued to a piece of cardboard
  • The switch presses were inconsistent and didn't always trigger the right character

More investigation required, and I needed a better case.

Creating a case
My initial prototype had just used switches that I found in the trove at Makespace but they were designed to be mounted on circuit boards rather then in a case. For the next iteration I selected these Adafruit ones from The Pi Hut https://thepihut.com/products/adafruit-16mm-panel-mount-momentary-pushbutton-red.
They were the right size, fairly cheap and could easily be mounted as I wanted. I also knew from my experimentation with the prototype that using a straight binary mapping from the switches the the alphabet was going to make typing hard. I mentioned this to some makers and was asked if this worked like a Micro Writer, having never head of it I headed back to Google.
Turns out that was exactly what I needed. The Microwriter was a chorded keyboard, invented in the UK in the 1970's, more details here: https://en.wikipedia.org/wiki/Microwriter. It used 5 buttons to map the the keyboard and another 'control' button to change input modes and allow for things like upper case letters, numbers, punctuation and editing. So, I ordered 6 buttons, with the control one being a different colour.
I am definitely not a designer, but I can create boxes in Fusion360 and add fillets to round off the corners - hence the utilitarian nature of the case currently.
I also know that my 3D printer is not the most accurate out there, but that doesn't matter as once I have a working version I can always print a higher quality one at Makespace. The first thing was fitting the switches and I didn't want to wait 5 hours to see on a completed box so just printed the face with the buttons in it. For the first version the holes were a little small, so I increased them and printed a box. The box comprised of two sections, one which housed 4 of the buttons, to be pressed by my fingers, and the other which housed the fifth button and the control, which were positioned so that they could be pressed separately, or together, using my thumb. I also added a power switch so that I didn't have to keep disconnecting the battery.
Nine hours of printing later and I had my new case. Although it easily fitted into my hand I added a velcro strap to hold it in place, and then fitted the switches, wiring, battery and controller. Which is when I discovered my next mistake - I had not taken into account the depth of the switches and with the battery in place the case didn't close! So, I remeasured, adjusted the models in Fusion360, another nine hours later I had a case that fitted.
Having read up on the Microwriter I decided it would be a good idea to follow their lead on the key mappings, as well as the button configuration. They appeared to have spent a lot of time thinking about the key configurations, how frequently letters are used, and also helping people learn to type using the buttons. So I updated the code to map the same configurations of buttons as the Microwriter to the same keys. I was fine for the alphabet but struggled with the other character mappings until I found this document: https://cdn.sparkfun.com/datasheets/Wireless/Bluetooth/RN-HID-User-Guide-v1.0r.pdf There is a table at the end which maps each of the characters to a numerical value which needs to be passed.
The Microwriter also had an interesting take on WHEN to read the keyboard input. I was reading the switches as they were pressed, which meant that if they were not all preassed at the same time I sometimes got the wrong character. The Microwriter triggers when the first switch is released, so all of the others are still in place. Once I'd changed the code to work this way the output was far more consistent with what I was trying to type.
My bluetooth chorded keyboard was finally working, I could connect to my phone and type emails or texts - excellent! It's not yet as fast as my thumbs, but that isn't really where I intend to use it. I started playing Elite Dangerous in VR some time back but struggled with the 'chat' window, as I had to remove my headset to see the keyboard. Once I learn to type with my knew keyboard that problem will be solved, and I'll be off to explore the universe!
Next steps:
  • Improve the Bluetooth - currently I have to re-pair the device every time I switch the power off and on
  • Improve the case - it's a box, it works, but I'd love some form or organic hand shaped case
  • Practice - I'm currently at about one word every 5 mins, that needs to improve!
Have fun.

Experience on Demand, by Jeremy Bailenson

I've been meaning for some time to start blogging the books I read that I think others will find useful, or interesting. Plus, I wanted to investigate how the Amazon affiliate scheme works - buyers pay the same for the book but if enough people buy via the link I get an Amazon voucher occasionally! Time to get started.

So, here's the first one.

I've read a number of VR books in the last year, it's something I've always been interested in and now the technology is reaching a point where it is actually available, and useable. Unlike previous books though Jeremy doesn't go into the detail of specific technologies, this book is more about what can and is being done.

VR's uses in a whole range of areas are covered, including:

  • Sports Training
  • Empathy Training - in areas such as:Pollution awareness, refugee crisis and racism
  • Pain relief - which surprised me
  • Social communication
  • Education

Jeremy's practical experiences as founding director of the Virtual Human Interaction Lab at Stanford University enable him to talk about what really works and what doesn't. All of this is then backed up with examples and data from the work that he and his colleagues have done.

If you want to understand more about what the point of VR is and where it's going, read this.


Low cost 3D printer build

Ever since starting at Makespace in Cambridge I've loved the fact that the tools available today mean that you can design and build almost anything you want. Makespace has some great equipment but for me it's a trip in and too far away when the job that you want to do can take hours to complete. The process of production with a 3D printer is not an exact science and requires multiple iterations, traveling somewhere to then wait for hours is painful, so it was time to get myself a printer.
It didn't need to be anything special as when I need a fine finish I could always use the printers at Makespace, I had some Amazon vouchers that I was itching to use and there seemed to be some bargains available but would they be up to the job?

Time to do some research.

I read reviews, watched Youtube videos and talked to makers about the printers I was looking to buy. I liked the look of the Prusa i3 for price, functionality and the fact that lots and lots had been built at home by makers, in exactly the same way I was planning. However eventually it came down to price, and when I found what I wanted for £150 on Amazon that was it. I decided that if I spent that even if had to replace a lot of the parts I would learn a lot and get value for money - decision made.

I wanted to be sure I had it in time for the Christmas holidays, so ordered in plenty of time, and a week later my parcel arrived.

Now plans are great but I don't have the most patience and within a couple of days I decided I couldn't wait for Christmas. I justified this to myself with the reasoning that if I did start construction over Christmas and there were pieces missing I'd never be able to get the parts delivered - So I started the build.

The Chinese/English instructions were amusing/confusing and in a couple of cases incorrect but I already knew this would be the case from the articles I'd read. I also knew from the videos I'd watched roughly how to put it together and these two pretty much covered it:
The build itself was straightforward and I could work out any confusing pieces from pictures on the website or from other builds. The only problem I had was when it came to connecting electrical elements to the controller board. Here the instructions were just plan wrong, the picture of the board was incorrectly labeled and the written instructions referred to connecting two different components to the same sockets on the board.

Fortunately these days many people have done this and a lot of information is available on the web. The board I had said GT2560 and although this was clearly a copy it was fortunately fairly accurate. The board details on the Geeetech site http://www.geeetech.com/wiki/index.php/GT2560 told me which connections were which and I could then complete construction.

Once the build was done that was it! No, really, I fed in the cheap red PLA filament that I got with it, triggered the heating of the bed and hotend for PLA and then leveled the bed. I created a simple rectangle shape in Fusion360 and imported it into Cura. Took the SD card from the printer and saved the gcode to it from Cura. Then selected 'Print from SD card', chose my rectangle and away it went. The result was surprisingly good, accurate as far as my measurement with a ruler was concerned.

Brilliant, for £150 I now had the ability to design and fabricate pretty much anything I needed in plastic.

Wall of Enlightenment v3 - the rebirth

Having given out my details and promising I'd blog at the Pi Birthday bash this weekend, I'd better crack on!
Given the previous issues I needed a rethink on the design, key points:
  • Build wall mounted from the start, to avoid issues when I finally came to hang it
  • Weatherproof
  • Improved frame
  • Tensioned strips to keep them straight
  • Short, heavy duty power cables
  • Improved signal cables
  • Connectors, connectors and connectors
A quote from Brian Corteil was my mantra for this build: "If in doubt over engineer it, it'll be easier to build initially and far better for reuse show after show".
So firstly the construction, thin conduit didn't work and I wanted the strips to be tensioned so hanging vertically was the best option. To keep the weight down but still make the construction easy having a box of some sort with the strips hanging down from it seemed ideal. I settled on electrical trunking and 75mm square gave me plenty of room to play with. Ideally I wanted this to be black so that it would not stand out but the requirement for that was a huge order! So, white and a liberal application of matte black spray paint resolved that.
Next hanging the strips vertically - this meant that I'd have to modify the code to rotate the image by 90 degrees which was part of the reason I'd dismissed it originally. A review of the Fadecandy documentation though gave me the answer very quickly, Micah had already solved this for me - with a simple config setting - BRILLIANT. I also learnt a lot from the example on the Fadecandy site: https://learn.adafruit.com/1500-neopixel-led-curtain-with-raspberry-pi-fadecandy/overview with regard to constructing the screen.
The LED strips are in a waterproof covering and I used hot glue to seal the ends with a 10mm glass ball. Unfortunately the only place I could find 10mm glass balls was Alibaba an the minimum order was 10,000 - so I have about 9000 spare if anyone needs any?
For the connection to the trunking, which also needed to be waterproof, I investigated options and costs with a number of suppliers and ended up with a plastic screw threaded version I found on Alibaba. After a few emails backwards and forwards we agreed on the details and the supplier sent me comprehensive design documents. A few weeks later and the finished articles arrived exactly as I'd requested. The sockets bolted directly into the trunking and the plugs screwed into them. I soldered the strips to these and again sealed them with hot glue and heat shrink tubing. (Having now attended a few shows bayonet fittings would have been far easier).
The Fadecandy cards as controllers for the strips were still the right way to go but with the previous version I had two problems related to the wiring, the first was the power. To get the 5v from the power supply box to the trunking was going to need some decent sized cables. Given that part of the problem I had with the initial version I wanted to make sure that this time was a success. The second issue was the wiring for the data lines to the addressable strips. On advice from Micah(fadecandy designer) I replaced the previous twisted pair with ribbon cable with alternate connections being Ground and also used 16 way ribbon connectors with header pins soldered to the cards. This makes it easy to swap cards out if they are faulty or I wanted to use them elsewhere while the Wall is not in use.
I grouped the strips into 8 and tie wrapped them onto laser cut perspex rectangles. This logically grouped the strips per fadecandy card and also made them easier to carry when detatched. Attaching them to a flat surface at the top and bottom of each strip meant that they were held facing forwards to get the best light from the LEDs.
Finally into the trunking went the Raspberry Pi as before with the Fadecandy server installed. Unfortunately at this point I realised my next mistake with the power - with all of the LEDs on there was insufficient power to run the Pi! This was easily solved by running a mains lead into the trunking and adding an adapter specifically for the Pi. Longer term I'll change that.
So finally I had a working screen - only half the size of the clients requirement but that would be easily remedied by building another duplicate 'half' and connecting the two toether when needed.
I exhibited at the UK Makerfaire in Newcastle and it looked pretty good.

'Wall of Enlightenment' v2 - the next instalment

It's taken a while (the original post was Feb!) but it's time to tell the story of the full size screen, which I've named 'The Wall of Enlightenment'. As a reminder the aim was to build a 2 metre wide, 1 metre high screen which could be hung on the side of the Pitt building in Cambridge as part of Science week.

Having created the initial small prototype my intention was to use the same materials, and scale it up. The prototype went down well at the Guildhall exhibition and I was given the green light to build the full size screen. Ordering 75m of LED strips from China was an interesting experience but once I'd got through the complexities of sending the money they arrived within a week. Construction could then be started on the floor of my home office.
70m of LEDs was a great box to open

The Wall is based around a Raspberry Pi which then uses Fadecandy cards to control the addressable LED strips. The maximum length of these before any lag is visible is estimated to be 64, from details on the Fadecandy site. With 32 LED per metre strips that gave me the 2 metre width I was looking for, so I started by cutting the strips and constructing the frame around them from 25mm black plastic conduit.
Initial construction

Using the same aspect ration on the vertical meant that I needed 32 rows and with 8 strips per Fadecandy that was 4 cards. As these were small my intention was to hide them inside the trunking but to get things running initially I wired them up outside. I also added a length of nylon line to the end of each strip and sealed the end with hit glue and heat shrink tubing. This would make it waterproof and the line could be used to tension the strips so that they didn't sag once the screen was hanging in position.
Once I had the first 8 lines connected I tested to ensure things were working as expected. I used the fadecandy server on the pi and streamed a video from my laptop. At a friends suggestion I used the ipod video advert as it was great for motion and colours. The result was great and can be seen here
Once I knew that worked I was confident that I could complete the rest the same way and after many long nights of soldering, sticking and threading wiring the construction was complete.
Construction complete

So, time to fire up this gigantic beast and enjoy the fruits of my labours, again the same video with the result here. However, something wasn't right. There was way more 'noise' on the display and the colours were not consistent. Something was wrong and this wouldn't be good enough for the display. I started to look at the wiring, the connections, the power levels to determine where the issue was. I also posted on the fadecandy google group and got some great feedback. Unfortunately the more I did the worse it got, with complete strips starting to die and more and more noise.
Here's the last test with some scrolling text: https://www.youtube.com/watch?v=wWNaVCvFfiM
At this point time wise it was the first day that the screen was due to be on display but there was no way it was up to that, it was dead on the office floor.
Dead on the office floor

This wasn't going to beat me. I needed a rethink and some new ideas, while continuing with the design I had may have worked there were some issues that had arisen that really needed to be addressed:
  • Needed to be weatherproof, there were holes all over the tubing for the strips and the nylon tensioners which would have been likely to leak
  • The frame itself was too weak to be rigid when transported or when hung on the wall
  • The strips needed to be tensioned to maintain the integrity of the image
  • The power cables needed to be thick and short to transfer the 5v from the power supply to the strips
  • Construction on the floor meant I didn't think about handing it or the support required
  • Thin conduit was what I had, worked well in the prototype, was thin and looked good but was not strong enough or large anough to contain all of the cabling
  • Electrical interference appeared to be due to the way it was wired, using twisted pair for signal lines
  • It was hugely difficult feeding the wires through the tubes
  • I'd soldered the fadecandy cards directly which made it very difficult to change their configuration or position

So I sat down with some friends at Makespace later that week and we designed version 3.