Want to buy a HDD in Akihabara? Bad Timing.

Getting a new HDD was always easy enough to do. Until recently with the Thailand flooding. Now it’s hard to get a new one, and if you do, the price increase quite a bit. For those who can read Japanese: http://akiba-pc.watch.impress.co.jp/hotline/20111105/etc_hdd.html and for those who don’t here an English translation.

Prices increased to about 15000 Yen/TB, which is about 3 times as much as before the flooding. Seagate works again but Western Digital is in worse shape it seems.

I was lucky: bought a 750GB 2.5″ HDD for (compared to now) cheap just about 2 weeks ago.

 
Moving Servers

Moving servers from Strato (can’t complain, good enough service, nothing ever broke, few but regular planned outages, and not expensive either), to Linode (quite a bit more expensive for the same performance, but less performance is ok too, so it’s about the same price). The main advantage of Linode is the location: Tokyo.

Main differences:

  • Location Germany vs Japan
  • Ping round trip from home decreased from 310ms to 30ms (and 18ms is my ADSL line already)
  • CPU changed from 1 core Opteron 2347 HE to 4 core Xen L5520 (but in reality it’s only 1 core anyway, however the new one is about 3 times faster according to http://www.webhostingtalk.com/showthread.php?s=&threadid=308055)
  • RAM decreased from 2GB to 0.5GB (but I only use about 0.4GB anyway)
  • Disk size decreased from 80GB incl. backup plus 80GB ftp, to 20GB and no backups
  • Data traffic decreased from 3TB to 200GB (but I never crossed 100GB)
  • Virtualizer changed from Virtuozzo to Xen
  • I now have some transparency about oversubscription of the underlying server
  • DNS is totally different: Strato uses a simple wildcard scheme: anything with *.studiokubota.com is pointing to the server. Linode uses a more traditional scheme where I need to define those DNS entries manually. But now I have control about it and can point hosts in my domain to any IP I like.
  • IP addresses decreased from 2 to 1 (but I used the extra IP only for VPN)
  • I need to use another registrar (I actually don’t need to, but it’s much cheaper if I do)

As you can see, it’s a mixed bag, but I am, generally ok although I’ll lose the ability to do proxy services in Germany, which allowed me to see some content some companies like to prohibit me from watching (streaming movies).

As a reminder to myself, here the additional commands/configs I had to do to move the server with all stuff I need:

# This is pretty much copy & paste. Run as root on the new server.
OLD=the.old.server
NEW=the.new.server
MYSQLPASSWORD=THEMYSQLPASSWORD

# User accounts

# Add encrypted password
echo >>/etc/passwd "harald:*:2000:100:Harald Kubota:/home/harald:/bin/bash"
# Copy root .ssh directory from old server

cd ~root

rsync -av $OLD:~root/.ssh .
# Install packages
aptitude update
aptitude full-upgrade
aptitude install locate
aptitude install apache2 imagemagick openvpn  gcc rsnapshot
aptitude install libapache2-mod-perl2 libapache2-mod-php5 libapache2-mod-perl2-doc
aptitude install phpmyadmin bsd-mailx
aptitude install postfix spamassassin procmail dovecot-common dovecot-imapd
# Configure postfix as Internet site with its local name as hostname

aptitude install mysql-server

# Set mysql "root" password, do not initialize the DB beside creating it

# Installing MT5 support files

aptitude install php5

aptitude install bsd-mailx exim4 libsoap-lite-perl libdbi-perl libimage-size-perl \
libmime-charset-perl libmime-encwords-perl perlmagick libxml-libxslt-perl \
 libxml-simple-perl libxml-libxml-perl libmime-tools-perl
aptitude install libgd-gd2-perl libio-compress-zlib-perl libarchive-any-perl
# copy /var/www

rsync -avz --progress $OLD:/var/www/ /var/www/

# For test purposes, create a symbolic link to one of the existing web pages

cd /var/www
ln -s harald.studiokubota.com $NEW

# Copy apache config

rsync -av $OLD:/etc/apache2/ /etc/apache2/

# Copy backup job from /etc/cron.daily/

rsync -av $OLD:/etc/cron.daily/mysqldump /etc/cron.daily/

# Mail: postfix, dovecot, spamassassin

rsync -av $OLD:/etc/postfix /etc/
# Modify main.cf to contain all domain names and fix the hostname too
# Also add virtual names and rebuild via
postmap /etc/postfox/virtual

rsync -av $OLD:/etc/dovecot /etc/
rsync -av $OLD:/etc/spamassassin /etc/
rsync -ac $OLD:/etc/default/spamassassin /etc/default/

mkdir /var/log/spamassassin
groupadd spamd
useradd -d /var/log/spamassassin -M -g spamd -s /bin/false spamd
chown spamd:spamd /var/log/spamassassin

# To learn ham/spam by putting spam in Spam and ham in Ham, add those to your user crontab:
#10 19 * * * sudo sa-learn --spam -u spamd --dir /home/harald/Maildir/.Spam/* -D
#10 20 * * * sudo sa-learn --ham -u spamd --dir /home/harald/Maildir/.Ham/* -D

# On both servers stop spamassassin and copy the DB over
ssh $OLD "service spamassassin stop"
service spamassassin stop
rsync -av $OLD:/var/log/spamassassin .
# Start spamassassin again
ssh $OLD "service spamassassin start"
service spamassassin start
service postfix restart
service dovecot restart

# Restore mysql DB

d=`date +%d`
cat >/tmp/dump.sh <<EOF
FN="/var/tmp/dbdump-${d}.bz2"
rm -f "\$FN"
mysqldump -p$MYSQLPASSWORD --all-databases | bzip2 -9 >\$FN
EOF
scp /tmp/dump.sh $OLD:/var/tmp/
ssh $OLD "bash /var/tmp/dump.sh"

# Restore on NEW server

rsync -av --progress $OLD:/var/tmp/dbdump-${d}.bz2 /var/tmp/
bunzip2 </var/tmp/dbdump-${d}.bz2 | mysql -u root -p
mysqladmin -p$MYSQLPASSWORD flush-privileges

# Other things

rsync -av --progress $OLD:/etc/openvpn /etc/
rsync -av --progress $OLD:/etc/default/openvpn /etc/default/

Files to modify manually later:

  • /etc/openvpn/openvpn.conf
  • some more files in /etc/openvpn to create/configure (it’s obvious), and adjust the files on the VPN client too
 
Eclipse Indigo and AVR Plugin

How to add the AVR Plugin into Eclipse 4.0 AKA Indigo:

Done. Without the first link you’ll get an error which does not really help a lot.

 

 
Kubuntu 11.10 - Upgrade and Reinstall

Usually I am careful when it comes to upgrading. I had my share of Gentoo updates which caused severe problems when updating files it should not replace. So I usually wait some weeks until new releases are sufficiently tested by enough people as I rarely do I need features right way. The last Kubuntu upgrades though went pretty smooth. No big issues, and only cosmetic ones. So I dared to upgrade on the day after Ubuntu/Kubuntu were released. And hey, no big issues!

On my Dell 3700 everything continued to work. I had to remove my home directory’s .kde and some other directories, and Flash on Firefox was awfully quiet. And that was it already. Dual screen worked fine after re-configuring (I actually put back the old xorg.conf). Network was ok from the start.

After testing and finding the low volume in Flash very annoying, I tried a fresh re-install. But I could have saved those 2h: no difference. which means the upgrade worked just fine.

Some small issues though, but not a fault of Kubuntu:

  1. Eclipse is upgraded to 4.0 (Galileo). The AVR Eclipse plugin is not yet updated to handle this though, so I cannot install it.
  2. Flash audio is quiet. In Firefox and Chrome.
 
16 Segment LED

16 Segment LEDs are (obviously) somewhere between 7 segment LED and graphical displays. Quite micro-controller friendly as they do not need a lot of memory or CPU performance or external hardware, but they look better and can display the full range of alphanumerical characters.

For those using these type of displays, a lot of time is spent trying to find “bitmap” data for characters. Some special ICs contain those already, but that I have none of those. After searching for font data and coming up empty-handed, I created my own:

// Segment bit order is A1 A2 B C D1 D2 E F G1 G2 H I J K L M

const unsigned int uiCharacterMap[0x40] =
	{
//	0	1	2	3	4	5	6	7   8   9
	0xff09, 0x3008, 0xec41, 0xdc48, 0x01d2, 0xcd84, 0x1fc0, 0xc00a,	0xffc0, 0xf1c0,
	0x7712, 0x3300, 0x6742, 0x7740, 0x3350, 0x5750, 0x1752, 0x7300, 0x7752, 0x7350
};

Those are digits 0 to 19 (yes, not just 0 to 9). Unfortunately 20 does not work, and neither do 22 or 23. Here a video of the 20 numbers: http://www.youtube.com/watch?v=pI8WTVAadiA

 
I2C fun

I realized quickly that the MultiAVR library is only the core Arduino library. What is missing are the non-core ones, like the I²C library (AKA TWI or “Wire”). Expecting a serious re-write, I was pleasantly surprised to see that I had to make exactly one change:

*** /usr/share/arduino/libraries/Wire/utility/twi.c     2011-01-10 14:28:07.000000000 +0900
--- /home/harald/workspace/BlinkSanguino/utility/twi.c  2011-09-24 22:38:53.283676108 +0900
***************
*** 69,74 ****
--- 69,77 ----
      // as per note from atmega8 manual pg167
      sbi(PORTC, 4);
      sbi(PORTC, 5);
+   #elif defined(__AVR_ATmega644P__) || defined(__AVR_ATmega1284P__)
+     sbi(PORTC, 0);
+     sbi(PORTC, 1);
    #else
      // activate internal pull-ups for twi
      // as per note from atmega128 manual pg204

and that was it.

In one day I was able to talk to:

  1. the LCD (16×2 characters)
  2. an PCF8574A (8 bit I/O)
  3. an PCF8583 (RTC)
  4. an MCP23017 (16 bit I/O)
The last one has a strange behaviour: Port A and B seem to be swapped.

 

 
Upgraded Arduino

Sanguino is a project which needed a slightly more powerful CPU, so they made Arduino libraries work on the ATmegaxx4 series work. It’s popular as it has slightly more of everything: FLASH, RAM, EEPROM, I/O pins and UARTs. Unfortunately that was when Arduino was at release 18. Current is 22. I could not make this even compile release 18 under Eclipse/avr-gcc 4.3.5.

Luckily someone was not too happy about Arduino only supporting their CPUs (ATmega8/168/328 and ATmega1280 and 2560) either. AVR has very similar CPUs with slightly different I/O configuration, but they are all pretty compatible. So they adopted the Arduino core libraries to run on pretty much any AVR CPU which looks sensible, including the ATmegaxx4 series.

Those libraries did not compile out-of-the box either, but minor modifications concerning the definition of port_to_mode_PGM and related fixed this:

-const uint16_t PROGMEM port_to_mode_PGM[] =
+volatile uint8_t * PROGMEM port_to_mode_PGM[] =
-const uint16_t PROGMEM port_to_output_PGM[] =
+volatile uint8_t * PROGMEM port_to_output_PGM[] =
-const uint16_t PROGMEM port_to_input_PGM[] =
+volatile uint8_t * PROGMEM port_to_input_PGM[] =
and in pins_arduino.h:
-extern const uint16_t PROGMEM port_to_mode_PGM[];
-extern const uint16_t PROGMEM port_to_input_PGM[];
-extern const uint16_t PROGMEM port_to_output_PGM[];
+extern volatile uint8_t * PROGMEM port_to_mode_PGM[];
+extern volatile uint8_t * PROGMEM port_to_input_PGM[];
+extern volatile uint8_t * PROGMEM port_to_output_PGM[];

Compiling the traditional blink program worked as expected.

For reference and for all those who want to use Eclipse, Arduino libraries and a ATmega644P resp. ATmega1284P, here the archives of the Eclipse projects: MultiAVR20.zip and BlinkSanguino.zip.



 

 
Programming AVRs

Programming an AVR microcontroller can be done by many ways. If you have an Arduino bootloader like this, it can be done via serial interface. If you use the one and only serial interface though, that’s not the best option.

One method is to use an existing Arduino, which is neat if you have one or want to use one dedicated as programmer.  Or you could get a dedicated programmer or something more generic. Or an FT245R. Which I happen to have flying around.

The instructions are simple to follow and worked out-of-the-box as long as the tested version of AVRDUDE and libftdi is used.

I installed this special version into /usr/local. Here some example invocations:

  • Setting fuses (default frequency of an 644P is 1MHz, so you must use -B 1 to slow the programming down):
    sudo avrdude-ftdi -v -p m644p -c ft245r -P ft0 -B 1 -U lfuse:w:0xE2:m -U hfuse:w:0×99:m -U efuse:w:0xFD:m
  • Reading all 64KB FLASH from the 644P (takes 13s):
    sudo avrdude-ftdi -v -p m644p -c ft245r -P ft0 -U flash:r:/tmp/644flash.hex

Small update: /etc/avrdude.conf must contain this:

programmer
  id    = "ftdi";
  desc  = "SparkFun FTDI Basic Breakout";
  type  = ft245r;
  miso  = 1;  # CB0
  sck   = 0;  # CB1
  mosi  = 2;  # CB2
  reset = 4;  # CB4
;

 

 
AVR minus Arduino

Arduino’s hardware is great and plentiful available, but for bare bones, the single microcontroller is all you need.

So I got myself some ATmega328P chips (250 Yen a piece) to set up a minimal Arduino like described here. All I need is an Arduino to program the blank chip. Got that, so I was hopeful.

First step is to burn a bootloader via the ISP interface of the 328P. Worked flawlessly. Infact, I tried all variations of bootloaders: with external clock and without, and the normal bootloader as well as OptiBoot. All of them looked ok: after reset an LED connected to digital pin 13 blinks with the usual “there is no program” sequence. But all attempts to program something using the serial connection failed with

avrdude: stk500_recv(): programmer is not responding

errors. I tried all possible board types and I tried to use avrdude using all possible baud rates. Nothing worked.

After spending countless hours trying to find a solution I gave up and skipped the Arduino interface part and its abstraction. The benefit is that I know exactly what is happening and the serial port now is free and available for other uses beside programming.

I found a good (and simple) example here and here I found how to use avrdude correctly:

avrdude -v -p m328p -c avrisp -P /dev/ttyUSB0 -b 19200 -e -U flash:w:demo.hex

This is using an Arduino as programmer acting as an avrisp programmer to program the ATmega328P. Worked on the first try.

Some more examples:

  • And to read the config and fuse bits:
avrdude -v -p m328p -c avrisp -P /dev/ttyUSB0 -b 19200
  • To program the fuse bits (fuse bit values taken from here) and erase the flash memory (-e):
avrdude -v -p m328p -c avrisp -P /dev/ttyUSB0 -b 19200 -e -U lfuse:w:0xE2:m -U hfuse:w:0xDA:m -U efuse:w:0xFD:m

On an unrelated note: During the next days, so time permits, I’ll check out this and this. This looks helpful too. Then I can get rid of the Arduino IDE, but keep the libraries.

 
More Air for Dell 3700

I bought a nice Dell Vostro 3700 some time ago. While generally I am still happy, it clearly has a temperature problem: When you keep all cores busy and they run at their maximum frequency, it gets so hot, it shuts down due to overheating.

This is usually not a problem as most tasks are not using all cores all the time, but when you do a large compile, and for the sake of efficiency try to do a “make -j”, make starts as many compiles concurrently as possible. If those take long time to compile, all cores are busy for an extended period of time.

A not-so-great fix is to simply not keep more than 1 core busy for extended periods of time. Or lower the maximum clock frequency to 1.197 GHz down from 1.596 GHz, so even 4 busy cores will not overheat the CPU. You can find the list of usable frequencies via a simple

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies

and to set the max and min frequency:

for i in /sys/devices/system/cpu/cpu[0-9]* ; do
  echo $highest  >$i/cpufreq/scaling_max_freq
  echo $lowest  >$i/cpufreq/scaling_min_freq
done

but of course this is not more than a workaround and not a fix.

The root cause is (beside the CPU creating too much heat) is the insufficient cooling system. The airflow is…poor. It’s not a problem of dust collecting anywhere (visible): I opened the back of the notebook and it’s clean. If there is dust, it is hiding really well.  The air vents on the back of the notebook are small. The fan is too weak. In the end, the airflow should be higher.

So one of those notebook coolers might be what I can get, but the results seem to vary if you read the reviews on (e.g.) Amazon. It cools notebooks down some few (about 5) degrees is the most common reply.

Paying 1000-8000 Yen for that little cooling…I felt I can do better myself.

Typical design look like this. It is essentially a USB powered fan and some plastic/aluminium. Either the notebook is lifted considerable or if not, then the spinning fan has a hard time to get air flowing since it’s then too close to the table.

Another problem is the limited power the fan can use: 5V @ 500mA max. That is not a lot, even if the fan is quite efficient.

But when I saw this it looked much better as it can freely pull air and push under the notebook. Except for two flaws: too small fans (read: either poor air flow or noisy) and air is pushed below the notebook, creating a considerable air flow left and right below the notebook. Right side is where my mouse is. I don’t need air flow there. But the idea was good, so I started to build a prototype.

It starts with a PWM controlled fan I bought some time ago. 12cm large to generate a generous air flow. PWM controlled to keep it quiet when not needed. To control it, you need a (approx.) 25 kHz positive PWM signal (TTL compatible). My Arduino can do that. All I needed now is a duct to push air below the notebook.

For a prototype I used the white cardboard from empty milk boxes. Very durable, quite stiff, easy to cut and we have a lot of them. Other people already found out it’s very useful and I myself used itbefore for an air duct to remove a useless 3cm fan in a small power supply unit.

Took 1h in total to build. Works quite well.

The fan is controlled manually via an Arduino with a small modification to make the PWM run at 32kHz as 500Hz, the default, makes a sound in the fan. 16kHz makes a higher and weaker sound. 32hKz makes no audible sound. The needed code is found on the Arduino Playground. The rest is trivial. Took about 2h to program.

Update: Since I wrote this entry, I build a wooden housing using 2mm MDF  board. Looks much better. I can highly recommend MDF: easy to saw and glue.

© 2011 Harald's Random Stuff Suffusion theme by Sayontan Sinha