Moving Containers from CA to TK

From CA to TK

Moving Docker container is supposed to be easy, but when doing a move, why not clean up, modernize and improve? Which of course makes such a move as difficult as any non-Docker move.

I moved several containers/services by literally copying the directory with the docker-compose.yml file in it.  That same directory has all the mount points for the Docker images, so moving is as simple as

On the old VM:

ssh OLD_HOST 'tar cf - DIR_NAME' | tar xfv -

which, if you got the permissions, works like a charm. If you don't have the permissions to tar up the old directory (e.g. root owned files which are only root-readable, e.g. private keys). If you don have the permissions, then execute this (the tar as well as the un-tar) as root.

Then a

docker-compose up -d

and all is running now and will continue to run in case of a reboot.

Mail

For mail I wanted to go away from the home-made postfix-dovecot container I created a long time ago: with the constant thread of security issues, maintenance and updates are getting mandatory. Also I had no spam filter included which back then was less of a problem than it is now. So I was looking for a simpler to maintain mail solution. I would not have minded to pay for a commercial one. Most commercial email hosting companies are totally oversized for my needs though, but at the same time I have to host 2 or 3 DNS domains which often is not part of the smallest offering.

My requirements were modest:

  1. 2 or 3 DNS domains to host, with proper MX records
  2. IMAP4 and SMTP
  3. web mailer frontend for those times I cannot use my phone
  4. TLS everywhere with no certificate warnings (e.g. self-signed certificates) for SMTP, IMAP4 and webmail
  5. 2 users minimum, unlikely ever more than 5
  6. Aliases from the usual suspects (info, postmaster)
  7. Some anti-spam solution

In the end I decided to do self-hosting again, if only to not forget how this all works. Here is the docker-compose.yml file:

version: '3'

services:
  mailserver:
    image: analogic/poste.io
    volumes:
      - /home/USER_NAME/mymailserver/data:/data
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "25:25"
      - "8080:80"
      - "110:110"
      - "143:143"
      - "8443:443"
      - "465:465"
      - "587:587"
      - "993:993"
      - "995:995"
    restart: always

You will have to configure the users and domains once incl. uploading the certificate (one certificate with two alternative names for 2 DNS domains). Also DKIM records (handled by poste.io), SPF (manual) and updating the MX records. It worked flawlessly!

Updating the Let's Encrypt certificate is not difficult: since all files are in the /data directory, updating those from outside the container is simple. It does need a restart of the container though.

One issue though:

As you can see, quite a lot of memory is used: 27.6% of a 2 GB RAM VM. The small VM I started with had only 1 GB RAM, and while all was running, it was very low on free memory and had to use swap. That's the only drawback of this Docker image: you cannot turn off ClamAV. However maybe that's ok since viruses and malware are a real problem and this helps to contain it.

Comments

AWS Snippets

Find Latest AMI

Find latest Amazon Linux 2 image in us-east-1:

aws --region=us-east-1 ec2 describe-images --owners amazon --filters \
'Name=name,Values=amzn2-ami-hvm-*-x86_64-gp2' \
'Name=state,Values=available' | \
jq -r '.Images | sort_by(.CreationDate) | last(.[]).ImageId'

To verify or generally check out an AMI:

aws --region=us-east-1 ec2 describe-images --image-ids ami-XXXXXXX | jq .

Find latest Amazon Linux 2 images in all regions

regions=$(aws ec2 describe-regions --query 'Regions[].{Name:RegionName}' --output=text | sort)
for i in $regions ; do
  echo -n "$i "
  aws --region=$i ec2 describe-images --owners amazon \
  --filters 'Name=name,Values=amzn2-ami-hvm-*-x86_64-gp2' 'Name=state,Values=available' | \
  jq -r '.Images | sort_by(.CreationDate) | last(.[]).ImageId'
done

List all Regions

aws ec2 describe-regions --query 'Regions[].{Name:RegionName}' --output=text | sort
# same as
aws ec2 describe-regions | jq -r '.Regions | sort_by(.RegionName) | .[].RegionName'

See also https://github.com/haraldkubota/aws-stuff for some more examples using NodeJS instead of the AWS CLI.

Comments

Static WDS on Mikrotik's RouterOS

Mikrotik's RouterOS and WDS

WDS is a bit of a mystery: when it works, it's great, especially in fully automatic meshed mode: add a node, extend your network. So much the theory. But when it won't work, it's causing trouble. Then it's time to take out the big guns: static WDS connections.

As so often, it's easy to do once you know how to do it:

1. For the WLAN interface to use, set WDS mode to static:

2. Define a static WDS connection. Needs to be done on both sides. You'll need to add the WLAN MAC address of the counterpart (remote) WDS endpoint:

Done. It should connect now and be used.

Comments

Comparing JPEG, x265, AV1 and RAW

Here a nice link where you can compare visually the differences in JPEG, x265, AV1 and RAW image format: https://people.xiph.org/~tdaede/av1stilldemo/

Comments

Updating Lambda Functions for this Blog

This blog has a neat surprisingly monolothic install procedure. I like that it works when it works. It's not good at updating specific items though and I find myself updating some Lambda functions several times a day. One reason is to try to use Node.js 6.10 instead of the original 4.3 environment. Or reducing memory to 256MB instead of 512MB. Or adding webpack with tree-shaking (AKA dead-code elimination) to make those Lambda functions not 40k lines long (but 'only' 2k lines).

Anyway, doing this by deleting everything and re-installing everything is fine until the day there are blog entries and re-installing might unexpectedly wipe our your blag articles. Thus I'd rather be able to refresh the Lambda functions knowing that nothing else will be modified.

So here the simple steps for a more granular update procedure, mainly for for Lambda functions:

List them all

aws --region us-east-1 lambda list-functions \
| jq -r '.Functions[] | select( .FunctionName | startswith("blog_")) | .FunctionName'

Delete them all

aws --region us-east-1 lambda list-functions \
| jq -r '.Functions[] | select( .FunctionName | startswith("blog_")) | .FunctionName' \
| xargs -n 1 --replace={} aws --region us-east-1 lambda delete-function --function-name={}

Redeploy them all

Actually deploy what's missing.

node upload_lambda.js

A good way would be to have two different releases which would make a roll-back very simple too.

Comments