Jul 292017
 
Serverless Computing AKA FaaS

While looking into what “Serverless Computing” means as it looks like an oxymoron, I created a presentation for my workplace’s “Knowledge Sharing” series. Total of 140 people attended. Last year’s Docker presentation was about 200.

I removed the counter and translation example for cost reasons. The example code is on GitHub for everyone to play with. Here the slides.

 

Jun 262017
 

Reading all this nice stuff about TDD. Using the Espruino GUI suddenly feels limiting. It’s great for testing and small things less than 1000 lines, but once you want modules and testing, it fails as it’s simply not made for that.

But there’s a fix, at least for me: https://github.com/haraldkubota/esp32

In short:

  • browserify to make a single JavaScipt file out of modules
  • Jest for testing
  • make to bundle it all together (ok, I do use npm too, but make it nicer IMHO)
  • netcat to upload the code

ESP32 with BME280, showing its measured data with sparklines

The sparkline as well as the circular buffer code it uses was attempted to be written via TDD. I think I failed partially here, but I still have plenty tests. When restructuring the code I was happy about the tests as they confirmed all is still working! Hooray for tests!

Jun 042017
 
Google Cloud, Authentication, Storage and NodeJS

How to use Cloud Storage from NodeJS? It’s actually easy once you found the libraries to use and a sample which works. But finding this was a long journey which took most of a day. Here was the most useful article of Sebastien having had the same problem. Shortly later I found this: https://googlecloudplatform.github.io/google-cloud-node/#/docs/storage/1.1.0/storage which helped a lot.

The short summary here in the hope that someone might read this and get a simple working example of NodeJS+Google Cloud Storage.

Pre-Requisite: Access

On the Google Cloud Console, go to IAM & Admin, create a Service Account, and create a key for it. Keep this key. It’s the JSON key in the keyFilename. This plus the projectID plus permissions is all you need to get access.

Under IAM you change the role of the new service account to Storage Admin to see all buckets. if you only want to connect to one bucket, use Storage Object Admin. Or less depending on what you need.

BTW updating permissions is almost instantly.

Task 1: See Buckets

This is the NodeJS program running on my home PC. I split it into 3 parts, but it’s really one file

The service accounts needs Storage Object Admin permississions for this.

var gcs = require('@google-cloud/storage')({
 projectId: 'some-thing-12345',
 keyFilename: './KEY.json'  // absolute path ok too
});
// GOOGLE_APPLICATION_CREDENTIALS=$keyFilename was not needed
// contrary to some documentation

gcs.getBuckets(function(err, buckets) {
 if (!err) {
  buckets.forEach(function(value){
  console.log("name: "+value.metadata.name);
  console.log("location: "+value.metadata.location);
  }); 
 } else {
  console.log(err);
 }
});

2. Look Inside Buckets

Service account needs Storage Object Viewer (read-only) resp. Creator (read/write).

// List contents of bucket 'some-test-1'

var wt1 = gcs.bucket('some-test-1');

wt1.getFiles(function(err, files) {
 if (!err) {
  files.forEach(x=>{console.log("Name: "+x.name)});
  // files is an array of File objects.
 } else {
  console.log(err);
 }
});

3. Access Files in a Bucket

Service account needs Storage Object Viewer (read-only) resp. Creator (read/write)

// Download a file (resp. create a read-stream)

var fs = require('fs');
var remoteFile = wt1.file('f1/a1.sh');
var localFilename = 'test-a1.sh';

remoteFile.createReadStream()
 .on('error', function(err) {
  console.log(err);
  })
 .on('response', function(response) {
 // Server connected and responded with the specified status and headers.
  })
 .on('end', function() {
  // The file is fully downloaded.
  })
 .pipe(fs.createWriteStream(localFilename));

 

May 282017
 

To program the ESP32 is usually a bit of luck and sometimes pressing the reset button if you’re less lucky.

RTS and/or DTR should be able to reset the ESP32. Once it’s reliably reset, a download via netcat works. If I only had a program to reset RTS and/or DTR!

I found one here, changed it to toggle DTR instead of RTS as toggling RTS did not do anything. Now I can reset the ESP32 reliably. Full automation is achieved!

Here the DTR toggle program:

#include <stdio.h>
#include <stdlib.h>
#include <termios.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>


static struct termios oldterminfo;


void closeserial(int fd)
{
 tcsetattr(fd, TCSANOW, &oldterminfo);
 if (close(fd) < 0)
 perror("closeserial()");
}


int openserial(char *devicename)
{
 int fd;
 struct termios attr;

 if ((fd = open(devicename, O_RDWR)) == -1) {
 perror("openserial(): open()");
 return 0;
 }
 if (tcgetattr(fd, &oldterminfo) == -1) {
 perror("openserial(): tcgetattr()");
 return 0;
 }
 attr = oldterminfo;
 attr.c_cflag |= CRTSCTS | CLOCAL;
 attr.c_oflag = 0;
 if (tcflush(fd, TCIOFLUSH) == -1) {
 perror("openserial(): tcflush()");
 return 0;
 }
 if (tcsetattr(fd, TCSANOW, &attr) == -1) {
 perror("initserial(): tcsetattr()");
 return 0;
 }
 return fd;
}


int setSerial(int fd, int bitmask, int level)
{
 int status;

 if (ioctl(fd, TIOCMGET, &status) == -1) {
 perror("setSerial(): TIOCMGET");
 return 0;
 }
 if (level)
 status |= bitmask;
 else
 status &= ~bitmask;
 if (ioctl(fd, TIOCMSET, &status) == -1) {
 perror("setSerial(): TIOCMSET");
 return 0;
 }
 return 1;
}


int main(int argc, char *argv[])
{
 int fd;
 char *serialdev = "/dev/ttyUSB0";
 int bitmask;

 if (argc == 2) {
 serialdev = argv[1];
 }

 // printf("device: %s\n", serialdev);


 fd = openserial(serialdev);
 if (!fd) {
 fprintf(stderr, "Error while initializing %s.\n", serialdev);
 return 1;
 }

 bitmask = TIOCM_DTR; // alternatively TIOCM_RTS | TIOCM_DTR

 setSerial(fd, bitmask, 0);
 sleep(1); /* pause 1 second */
 setSerial(fd, bitmask, 1);

 closeserial(fd);
 return 0;
}

And the program to upload to the ESP (via netcat):

#!/bin/bash 
# 
# Upload JS file to ESP32 via wifi 
# 
# Usage: 
# uploadesp.sh JAVASCRIPTFILE IPADDRESS 
 
if [[ $# -ne 2 ]] ; then 
        echo "Usage: $0 JAVASCRIPTFILE IPADDRESS" 
        exit 20 
fi 
 
if [[ ! -f "$1" ]] ; then 
        echo "ERROR: Cannot read file \"$1\"" 
        exit 10 
fi 
 
dtr-toggle 
 
TMP="$(mktemp).js" 
espruino -n -o $TMP $1 
echo "Uploading to $2:23" 
nc $2 23 < $TMP >/dev/null 
 
# rm $TMP 
exit 0

 

May 282017
 
ESP32 and Espruino

Those ESP modules and Espruino are a very attractive combination. The latest one I ordered is particularily nice: ESP32 with plenty RAM (512 KB), 2 cores, with a 128×64 pixel OLED, plus Espruino. Sweet!

./esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 921600 \
  --before default_reset --after hard_reset write_flash \
  -z --flash_mode qio --flash_freq 80m --flash_size detect \
  0x1000 /tmp/espruino_1v92.71_esp32/bootloader.bin \
  0x8000 /tmp/espruino_1v92.71_esp32/partitions_espruino.bin \
  0x10000 ../Espruino/espruino_esp32.bin

The bootloader.bin and partitions_espruino.bin come from the Travis cutting-edge builds. The binary I compiled from https://github.com/espruino/Espruino

Connecting via Web IDE works usually. In case of problems, use minicom to connect (115200 baud). Pushing the reset button always works. At this point the Web IDE will connect too.

Uploading code via Web IDE works. 100% reliable as far as I can say. But I wanted a command line uploads to do things like TypeScript and use make or Jenkins for build, test, upload cycles.

https://www.npmjs.com/package/espruino is a NPM modules for NodeJS which makes this possible. However direct uploads via serial port (/dev/ttyUSB0) does not work reliable (<10% success). Turns out there’s a better way:

#!/bin/bash
# Usage: uploadesp.sh JAVASCRIPT_FILE.js IP_OF_ESP32 
TMP=$(mktemp)
espruino -n -o $TMP $1
nc $2 23 < $TMP >/dev/null

That’s 100% working so far and much, much faster to upload. Note that the espruino command (actually espruino-cli.js) needs some changes to recognize the -n option to not talk to the serial interface.

Update: dio and 40MHz works fine too.

May 212017
 

ESP8285

Espruino does not need more than 1 MB of Flash memory, so using a single chip solution with 1 MB Flash integrated like the ESP8285 seesm to be a sensible choice. And it works with Espruino just like the ESP8266 does.

The only special part is how to flash it:

esptool.py --port /dev/ttyUSB0 --baud 115200 write_flash \
--flash_mode dout --flash_size 8m \
0x0000 boot_v1.6.bin \
0x1000 espruino_esp8266_user1.bin \
0xFC000 esp_init_data_default.bin 0xFE000 blank.bin

There’s no difference when it comes to save(): only very small programs can be saved. I hope the ESP32 is better in this regard.

Mar 252017
 
STM32F4 Discovery and Espruino

Some time ago I installed Espruino on the STM32F4 Discovery board.

Here my notes:

sudo aptitude install cmake libusb-1.0-0-dev
git clone https://github.com/texane/stlink.git
cd stlink
cmake .
make
# install into /usr/local/
sudo make install
sudo ldconfig -v

How you should be able to see the STM32F4 Discovery board:

$ sudo st-info --probe 
Found 1 stlink programmers 
 serial: 563f6b06493f51521236213f 
openocd: "\x56\x3f\x6b\x06\x49\x3f\x51\x52\x12\x36\x21\x3f" 
  flash: 1048576 (pagesize: 16384) 
   sram: 196608 
 chipid: 0x0413 
  descr: F4 device

and now flash:

 
$ sudo st-flash write espruino_1v91_stm32f4discovery.bin 0x08000000                                
st-flash 1.3.1-11-g78ced6a 
2017-03-25T20:41:03 INFO src/common.c: Loading device parameters.... 
2017-03-25T20:41:03 INFO src/common.c: Device connected is: F4 device, id 0x10016413 
2017-03-25T20:41:03 INFO src/common.c: SRAM size: 0x30000 bytes (192 KiB), Flash: 0x100000 bytes (1024 KiB) in pages of 16
384 bytes 
2017-03-25T20:41:03 INFO src/common.c: Attempting to write 298872 (0x48f78) bytes to stm32 address: 134217728 (0x8000000) 
Flash page at addr: 0x08040000 erasedEraseFlash - Sector:0x6 Size:0x20000  
2017-03-25T20:41:09 INFO src/common.c: Finished erasing 7 pages of 131072 (0x20000) bytes 
2017-03-25T20:41:09 INFO src/common.c: Starting Flash write for F2/F4/L4 
2017-03-25T20:41:09 INFO src/flash_loader.c: Successfully loaded flash loader in sram 
enabling 32-bit flash writes 
size: 32768 
size: 32768 
size: 32768 
size: 32768 
size: 32768 
size: 32768 
size: 32768 
size: 32768 
size: 32768 
size: 3960 
2017-03-25T20:41:13 INFO src/common.c: Starting verification of write complete 
2017-03-25T20:41:16 INFO src/common.c: Flash written and verified! jolly good!

How you should be able to see the STM32F4 Discovery board:

And here a small program which reads out the accelerometer LIS302DL and the most upper LED will be lit up. Be aware that on newer F4 Discovery boards this model has changed to LIS3DSH, which is not compatibel.

function onInit() {
 SPI1.setup({sck: A5, miso:A6, mosi:A7});
 SPI1.send([0x20,0b01000111], E3);
}

var avrx=0.0, avry=0.0;
function getAcc() {
  var accx = SPI1.send([0xA9,0], E3)[1];
  var accy = SPI1.send([0xAB,0], E3)[1];
  if (accx>127) accx-=256;
  if (accy>127) accy-=256;
  avrx = 0.1*accx + 0.9*avrx;
  avry = 0.1*accy + 0.9*avry;
  digitalWrite(LED1, avrx > 32);
  digitalWrite(LED4, avrx < -32);
  digitalWrite(LED2, avry > 32);
  digitalWrite(LED3, avry < -32);
}
onInit();setInterval(getAcc, 10);

Mar 202017
 

Bluetooth and its LE variant are new areas for me and, but the learning curve is not too steep thanks to Puck.js. Making  it do what I want is not easy though since I don’t know what it can do.

The most important documents are:

Here an example:

// Have a service to show temperature and battery level
// This is only visible when you are connecting to the Puck

var currentTemperature=E.getTemperature().toFixed(2)*100;

console.log("Temp: "+currentTemperature);

NRF.setServices({
 0x1809 : { // Health Thermometer
  0x2A6E: { // Temperature
   readable: true,
   notify: true,
   value : [ currentTemperature % 256, currentTemperature / 256 ]
  }
 },
 0x180f : { // Battery Level
  0x2a19 : { // Percentage
   readable : true,
   notify: true,
   value: [ Puck.getBatteryPercentage() ],
  }
 }
});

// Updating the temperature

setInterval(function() {
 currentTemperature += 1; // For debugging: increase temp by 0.01 degree per update
 NRF.updateServices({
  0x1809 : {
   0x2A6E : {
    value : [ currentTemperature % 256, currentTemperature / 256 ],
    notify: true
   }
  }
 });
}, 2000);

Mar 112017
 

My Espruino has colored I/O pins and since I saw this, I thought “That’s neat!”

Playing with the Orange Pi Zero and its 26 pin expansion port made me really want those colored pins as connecting things wrong can destroy ports quickly: 5V and 3.3V mixed up? Magic smoke escapes. Connected GND instead of I/O pin? 50% chance of a dead output port. Miss-counted pins and used pin 11 instead of 13…happy debugging!

The Asus Tinkerboard has it too:

but it seems this is not available without the rest of the Tinkerboard.

I had to build one from coloured 40 pin single row pins I got from AliExpress:

And here is the result:

Red is +5V, yellow is +3.3V, black is GND, blue is I²C and green is SPI. I don use UARTs a lot, so I did not bother to colour those.

Took about 15min of work incl. soldering. Totally worth it.

Mar 052017
 
Espruino, ESP8266 and the I2C OLED

Small Sunday afternoon project: Make the ESP8266 with Espruino work with the OLED display I have. NodeMCU has no issues. Time to try Espruino+JavaScript instead of NodeMCU+Lua.

Ingredients:

~/.local/bin/esptool.py --port /dev/ttyUSB0 --baud 115200 write_flash 
--flash_freq 80m --flash_mode qio --flash_size 32m \
 0x0000 "boot_v1.6.bin" \
0x1000 espruino_esp8266_user1.bin \
0x3FC000 esp_init_data_default.bin \
0x37E000 blank.bin
  • and then this little program will use it (if you have configured everything correctly and I2C works as it should):
I2C1.setup({sda: NodeMCU.D2, scl: NodeMCU.D1});

function start(){
 // write some text
 g.drawString("Hello World!",2,2);
 // write to the screen
 g.flip(); 
}

var g = require("SSD1306").connect(I2C1, start, {height:64});

 

Took me some tries to get it working and very helpful was the ability to scan the I2C bus:

function isDeviceOnBus(i2c,id) {
  try {
    return i2c.readFrom(id,1);
  }
  catch(err) {
    return -1;
  }
}

function detect(i2c,first, last) {
  first = first | 0;
  last = last | 0x77;
  var idsOnBus = Array();
  for (var id = first; id <= last; id++) {
    if ( isDeviceOnBus(i2c,id) != -1) {
      idsOnBus.push("0x"+id.toString(16));
    }
  }
  return idsOnBus;
}

I2C1.setup({sda: NodeMCU.D2, scl: NodeMCU.D1});
console.log('I2C detect as array:',detect(I2C1));

Note that I could not make the Wemos OLED work as its size (64×48) seems not supported. The 128×64 display I have worked out of the box. 128×32 is supposed to work too (see here).