tips & tricks

Encrypting your entire hard disk (almost)


I have a small netbook that I use when I travel, one of the original Asus EeePC’s, the 900.  It has a 9″ screen and a 16GB flash drive.  It runs Linux, and it’s just about right for accessing email, some light surfing, and doing small tasks like writing blog posts and messing with my checkbook.  And since it runs Linux, I can do a lot of nice network stuff with it, like SSH tunneling, VPN’s, and I can even make it act like a wireless access point.

However, the idea of leaving my little PC in a hotel room while I am out having fun leaves me a little uneasy.  I am not concerned with the hardware… it’s not worth much.  But I am concerned about my files, and the temporary files like browser cookies and cache.  I’d hate for someone to walk away with my EeePC and also gain access to
countless other things with it.

So this week, I decided to encrypt the main flash drive.  Before, the entire flash device was allocated as one device:

partition 1  –  16GB  –  the whole enhilada

Here’s how I made my conversion.

(0) What you will need:

  • a 1GB or larger USB stick (to boot off of)
  • an SD card or USB drive big enough to back up your root partition

(1) Boot the system using a “live USB stick” (you can create one in Ubuntu by going to “System / Administration / Startup Disk Creator”.  Open up a terminal and do “sudo -i” to become root.

ubuntu@ubuntu:~$ sudo -i
root@ubuntu:~$ cd /

(2) Install some tools that you’ll need… they will be installed in the Live USB session in RAM, not on your computer.  We’ll install them on your computer later.

root@ubuntu:/$ apt-get install cryptsetup

(3) Insert an SD card and format it. I formatted the entire card.  Sometimes, you might want to make partitions on it and format one partition.

root@ubuntu:/$ mkfs.ext4 /dev/sdb
root@ubuntu:/$ mkdir /mnt/sd
root@ubuntu:/$ mount /dev/sdb /mnt/sd

(4) Back up the main disk onto the SD card. The “numeric-owner” option causes the actual owner and group numbers to be stored in the tar file, rather than trying to match the owner/group names to the names from /etc/passwd and /etc/group (remember, we booted from a live USB stick).

root@ubuntu:/$ tar --one-file-system --numeric-owner -zcf /mnt/sd/all.tar.gz .

(5) Re-partition the main disk. I chose 128MB for /boot.  The rest of the disk will be encrypted.  The new layout looks like this:

partition 1  –  128MB  –  /boot, must remain unencrypted
partition 2  –  15.8GB  –  everything else, encrypted

root@ubuntu:/$ fdisk -l

Disk /dev/sda: 16.1 GB, 16139354112 bytes
255 heads, 63 sectors/track, 1962 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002d507

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        1          17      136521   83  Linux
/dev/sda2           18        1962    15623212+  83  Linux

(6) Make new filesystems on the newly-partitioned disk.

root@ubuntu:/$ mkfs.ext4 /dev/sda1
root@ubuntu:/$ mkfs.ext4 /dev/sda2

(7) Restore /boot to sda1. It will be restored into a “boot” subdirectory, because that’s the way it was on the original disk.  But since this is a stand-alone /boot partition, we need to move the files to that filesystem’s root.

root@ubuntu:/$ mkdir /mnt/sda1
root@ubuntu:/$ mount /dev/sda1 /mnt/sda1
root@ubuntu:/$ cd /mnt/sda1
root@ubuntu:/mnt/sda1$ tar --numeric-owner -zxf /mnt/sd/all.tar.gz ./boot
root@ubuntu:/mnt/sda1$ mv boot/* .
root@ubuntu:/mnt/sda1$ rmdir boot
root@ubuntu:/mnt/sda1$ cd /
root@ubuntu:/$ umount /mnt/sda1

(8) Make an encrypted filesystem on sda2. We will need a label, so I will call it “cryptoroot”.  You can choose anything here.

root@ubuntu:/$ cryptsetup luksFormat /dev/sda2

This will overwrite data on /dev/sda2 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: ********
Verify passphrase: ********
root@ubuntu:/$ cryptsetup luksOpen /dev/sda2 cryptoroot
root@ubuntu:/$ mkfs.ext4 /dev/mapper/cryptoroot

(9) Restore the rest of the saved files to the encrypted filesystem that lives on sda2.  We can remove the extra files in /boot, since that will become the mount point for sda1.  We need to leave the empty /boot directory in place, though.

root@ubuntu:/$ mkdir /mnt/sda2
root@ubuntu:/$ mount /dev/mapper/cryptoroot /mnt/sda2
root@ubuntu:/$ cd /mnt/sda2
root@ubuntu:/mnt/sda2$ tar --numeric-owner -zxf /mnt/sd/all.tar.gz
root@ubuntu:/mnt/sda2$ rm -rf boot/*
root@ubuntu:/mnt/sda2$ cd /

(10) Determine the UUID’s of the sda2 device and the encrypted filesystem that sits on top of sda2.

root@ubuntu:/$ blkid
/dev/sda1: UUID="285c9798-1067-4f7f-bab0-4743b68d9f04" TYPE="ext4"
/dev/sda2: UUID="ddd60502-87f0-43c5-aa28-c911c35f9278" TYPE="crypto_LUKS"   << [UUID-LUKS]
/dev/mapper/root: UUID="a613df67-3179-441c-8ce5-a286c16aa053" TYPE="ext4"   << [UUID-ROOT]
/dev/sdb: UUID="41745452-3f89-44f9-b547-aca5a5306162" TYPE="ext3"

Notice that you’ll also see sda1 (/boot) and sdb (the SD card) as well as some others, like USB stick.  Below, I will refer to the actual UUID’s that we read here as [UUID-LUKS] and [UUID-ROOT].

(11) Do a “chroot” inside the target system. A chroot basically uses the kernel from the Live USB stick, but the filesystem from the main disk.  Notice that when you do this, the prompt changes to what you usually see when you boot that system.

root@ubuntu:/$ mount /dev/sda1       /mnt/sda2/boot
root@ubuntu:/$ mount --bind /proc    /mnt/sda2/proc
root@ubuntu:/$ mount --bind /dev     /mnt/sda2/dev
root@ubuntu:/$ mount --bind /dev/pts /mnt/sda2/dev/pts
root@ubuntu:/$ mount --bind /sys     /mnt/sda2/sys
root@ubuntu:/$ chroot /mnt/sda2

(12) Install cryptsetup on the target.

root@enigma:/$ apt-get install cryptsetup

(13) Change some of the config files on the encrypted drive’s /etc so it will know where to find the new root filesystem.

root@enigma:/$ cat /etc/crypttab
cryptoroot  UUID=[UUID-LUKS]  none  luks
root@enigma:/$ cat /etc/fstab
proc  /proc  proc  nodev,noexec,nosuid  0  0
# / was on /dev/sda1 during installation
# UUID=[OLD-UUID-OF-SDA1]  /  ext4  errors=remount-ro  0  1
UUID=[UUID-ROOT]  /  ext4  errors=remount-ro  0  1
/dev/sda1  /boot  ext4  defaults  0  0
# RAM disks
tmpfs   /tmp       tmpfs   defaults   0  0
tmpfs   /var/tmp   tmpfs   defaults   0  0
tmpfs   /var/log   tmpfs   defaults   0  0
tmpfs   /dev/shm   tmpfs   defaults   0  0

(14) Rebuild the GRUB bootloader, since the files have moved from sda1:/boot to sda1:/ .

root@enigma:/$ update-grub
root@enigma:/$ grub-install /dev/sda

(15) Update the initial RAM disk so it will know to prompt for the LUKS passphrase so it can mount the new encrypted root filesystem.

root@enigma:/$ update-initramfs -u -v

(16) Reboot.

root@enigma:/$ exit
root@ubuntu:/$ umount /mnt/sda2/sys
root@ubuntu:/$ umount /mnt/sda2/dev/pts
root@ubuntu:/$ umount /mnt/sda2/dev
root@ubuntu:/$ umount /mnt/sda2/proc
root@ubuntu:/$ umount /mnt/sda2/boot
root@ubuntu:/$ umount /mnt/sda2
root@ubuntu:/$ reboot

When it has shut down the Live USB system, you can remove the USB stick and let it boot the system normally.  If all went well, you will be prompted for the LUKS passphrase a few seconds into the bootup process.

A Tech Trifecta


Dropbox + KeePass + VeraCrypt

I would like to introduce three software packages that are profoundly useful in their own rights, but then go a step further and show how these three tools can be used together to revolutionize how you keep track of your secrets & finances in the digital age.


The first tool is Dropbox, a cloud service where you can store a folder of your own personal files. They offer a free service where you can keep 2GB of files, and you can pay for more storage.

You can access the files from anywhere using a web interface. Or better, you can install the Dropbox program, and it will automatically keep a local copy of all of your stuff up to date in a folder on your computer (Windows, Mac or Linux). If you’re offline for a bit, no problem, because you still have your local copy of the files.

The real magic happens when you connect multiple computers to the same Dropbox account — say, your home PC and your office PC. Dropbox keeps the files in sync for you.

There is even a Dropbox app for your iPhone, so you can access your files on the go.


If I asked you how many passwords you have, you might think that you have a dozen or so. One day, I decided to make a list of every password I had (including multiple accounts where I happened to use the same password). I was shocked to find that I had more than 100.

PRO TIP – Most people use the same five passwords over and over… they don’t bother remembering which ones they used for a particular web site… they just try all five of their “not so important” passwords. That means that if I want to steal something from you, I can just set up a service and invite you to log in. I make your next login attempt fail, so you’ll get confused and enter all five of your “not so important” passwords, and possibly your “good” ones, too. Then I can go to eBay and PayPal and Amazon with your username and all of your passwords, and I have a pretty good chance of logging into your account.

For this reason, you should use a different password at EVERY web site.

KeePass is an open source program that stores your passwords. The program stores all of that information in a single data file that you lock using a single “master password”. This should be the only password you ever remember. The rest of them should be long strings of gibberish characters.

Keepass has a nice “generate” feature which will come up with really good passwords like “an@aiph5Ph”.

KeePass is available for Windows, Mac and Linux (KeePassX), and there are a couple of iPhone variants (I recommend “MyKeePass”).

Keepass stores four main fields: the browser URL address, user name, password, and notes. That means that I can store an entry like this for my bank:

name = alan_porter
password = uG7pi~ji9a
notes = security question, pet’s name = teabag

These days, it’s really easy to get confused about the main web address that you’re supposed to use to log in to your bank. They make things confusing by using fancy URL’s like “” instead of “”. If you have not had your coffee one morning and you type “” by mistake (“Me” instead of “You”), you’ll find yourself logging into a phishing site, which is trying to steal your password.

With keepass, there is no room for this sort of error. You double click on the URL and a browser opens on that site. You double click on the user name and it copies the user name to the clipboard. Paste it in the browser. Similarly, double click on the password and paste it in the browser. If your bank uses “security questions”, you can store your answers in the notes section.

PRO TIP – Don’t use truthful answers for “security” questions… treat these as extended passwords, just like you’d see in the old spy movies (he says “the sky looks like rain”, and you answer “that helps the bananas grow”). The answers do not even need to make sense… they just need to be remembered… in your head (bad) or in your KeePass file (good).

And no, I have never had a pet named “teabag”.


The last tool I would like to talk about is VeraCrypt*, a free package that allows you to create an encrypted volume on a file or a disk.

NOTE: the original version of this blog post recommended TrueCrypt, which has been discontinued — VeraCrypt is a drop-in replacement with several improvements, most notably it is open source and it has undergone very close scrutiny and security audits.

You can create a VeraCrypt file with a name like “” that you store on your desktop. You give it a size and a password. Then when you open that file, you’ll have to enter the password, and then it will appear as a Windows drive letter. On a Mac or Linux, it will show up as folder in /Volumes or in /media. When you close it, the drive letter is gone, and your secrets are stored, encrypted, in the “” file.

You can store this “” file anywhere you want… on your desktop, on a USB flash drive, or even on your Dropbox account.

You might also want to use VeraCrypt to encrypt an entire device, like a USB flash drive or SD card — nice.


Using these three tools together, suddenly we have a very secure way to do our online banking.

Regardless of what computer you’re using — Mac, Windows, Linux, desktop, laptop, at home, at work, whatever — you now have access to your passwords, and all of the other files that you need: Quicken or GnuCash, the PDF files of your bills, personal files, etc.


A typical online bill-paying session might look like this:

  • Look in your Dropbox for your KeePass password file, say D:\mypasswords.kdb. Open it with KeePass.
  • Look in your Dropbox for your VeraCrypt volume, say D:\ Open it with VeraCrypt. It will be mounted as drive V.
  • Use your normal accounting package (Quicken or Gnucash) to open your checkbook, say V:\checkbook.qdf or V:\checkbook.gc.
  • Find your bank password in KeePass. Click on the safe URL, copy the username and password from KeePass to the browser.
  • When you need to save a statement or a bill, save into the VeraCrypt volume on the V drive, like in V:\statements.


Say you’re out and about, and you want to use your iphone to log into a web site that you don’t know the password for (of course you don’t… because you use GOOD and UNIQUE passwords like “taeYi#c6We”, right?).

  • First, you use the iPhone Dropbox app to locate your KeePass file. Click on the filename, and the app will say “I don’t know what kind of file this is”. But it will show you a little “chain link” icon in the bottom. Click this icon to get a web URL for that one file. The URL will be copied to your clipboard.
  • Next, run MyKeePass and click “+” to add a new KeePass file. Select “download from WWW” and paste the URL that you just got from Dropbox.

Now, you can open your KeePass file from anywhere! You’ll have to enter the file’s master password that you set up earlier. After that, you’ll be able to browse the information in the KeePass file.

Note that you can only READ the file using the “download from WWW” method. If you need to save a new password or some other information, you’ll need to write it down on a piece of paper until you are back at your PC (no biggie — that’s pretty rare).


Some readers might note that we just shared our KeePass password file with the entire world, when we told Dropbox to give us a URL. Someone else could use that same (random-ish) URL to download our KeePass file. Since this file is encrypted using the master password, it will look like gibberish to anyone but you. For this reason, we should make sure that we use a VERY STRONG password for the KeePass file… not your dog’s name (remember, mine is “teabag”).

Using SSH to tunnel web traffic


Every once in a while, I learn something new and magical that SSH can do.  For a while now, I have been using its “dynamic port forwarding” capability, and I wanted to share that here.

Dynamic port forwarding is a way to create a SOCKS proxy that listens on your local machine, and forwards all traffic through the SSH tunnel to the remote side, where it will be passed along to the remote network.

What this means is that you can tell your web browser to send some – or all – of your web requests through the tunnel, and the SSH daemon on the other side will send those requests out to the internet for you, and then it will send the responses back through the tunnel to your browser.

There are several practical uses for SSH tunneling.  Here are a few real-world examples of where I have used tunnels:

  • You may be behind a firewall that blocks web sites that you want to visit. I was at a company that had a web proxy that blocked some “hacking” web sites, when I really needed to look up some information on computer vulnerabilities, to make sure that their systems were properly patched.  Using the SSH tunnel bypassed that company’s web proxy.
  • You may not trust a hotel’s internet service. In fact, you SHOULD NOT trust a hotel’s internet service.  They keep records of sites that you visit, ostensibly to keep a customer profile of your interests.  Some also do deep packet inspection to observe words in the web pages and emails that you read.  Intrusive marketing!
  • You may want to work from home, and access work lab resources from home. If you can SSH into one machine at work, then you can access the web resources inside that network.
  • Working on a cluster of machines, where only one of them is directly accessible from your desk. It stinks to have to do your work in a noisy lab.  Why not shell into the gateway machine and tunnel your web traffic through that SSH connection?
  • You may want to access a web interface to something at home, like a printer… or an oven. You should open the SSH service to your router, and then you can SSH in and easily access web interfaces to everything in your home.
  • You may want a web interface on some service on an internet-facing machine, but you don’t want to expose it to everyone. Maybe you want phpMyAdmin on a server, but you don’t want everyone to see it. So run the web service on the local ( network only, and use an SSH tunnel to get to it from wherever you are.
  • Your ISP throttles web traffic from a particular site. In my case, it was an overseas movie streaming site, and my ISP would limit my traffic to 300kbps for 30 seconds, followed by nothing for 30 seconds (they do this to discourage you from watching movies over the internet).  Sending the traffic through an SSH tunnel means the ISP only sees an encrypted stream of something on port 22 — they have no idea that the content is actually a movie streamed from China on port 80.
  • You may live in India, but want to watch a show on Hulu (which only serves the US market).  By tunneling through a US-based server, it will appear to Hulu that you’re in the US.
  • You may be visiting China, and you want to bypass the Great Firewall of China. Believe it or not, my humble domain was not accessible directly from Shanghai.
  • You may get a “better” route by going through a proxy. This is speculation on my part… but I did this once when I was in a free-for-all online ticket purchase.  I wondered if I might get a faster route from my remote data center than I would get directly from my ISP.

So, how do you create a tunnel?  It’s super easy.

Create an SSH session with a tunnel.

On Linux or a Mac, open a shell and type “ssh -D 10000“.

On Windows*, you can run Putty (an SSH client).  In the “connection / SSH / tunnels” menu, click on “dynamic”, then enter source port = 10000, then click “add”.  You’ll see a “D10000” in the box that lists your forwarded ports.  (* Putty is also available for Linux.)

That’s it… you’re done.

Tell the browser to use the tunnel.

The easiest way to get started with SOCKS proxies is to set it in the “preferences / advanced / network / connection settings” menu in Firefox (or the “preferences / under the hood / change proxy settings” menu in Google Chrome).  In both cases, you want to set “manual proxy configuration” and SOCKS host = “localhost” and SOCKS port = 10000.  If there is a selection for SOCKS v5, then check it.

Now, all web requests will go to a proxy that is running on your local machine on port 10000.  If you do a “netstat -plnt | grep 10000”, you’ll see that your SSH or Putty session is listening on port 10000.  SSH then forwards the web request to the remote machine, where SSHD will send it on out to the internet.  Web responses will come back along that same path.

Anyone on your local network will only be able to see one connection coming from your machine: an encrypted SSH session to the remote.  They will not see the web requests or responses.  This makes it a great tool for using on public networks, like coffee shops or libraries.

Test it

Point your browser to a site that tells you your IP address, such as  It should report the IP address of the remote server that you are connecting through, and not your local IP address.

Advanced – quickly selecting proxies using FoxyProxy or Proxy Switchy!

It can be a hassle to go into the preferences menu to turn the proxy on and off.  So there are browser plug-ins that allow you to quickly switch between a direct connection and using a SOCKS proxy.

For Firefox, there is the excellent “FoxyProxy“.  For Chrome, you’ll want “Proxy Switchy!“.  Both plug-in’s allow you to select a proxy using a button on the main browser window.

But there’s more… they also allow you build a list of rules to automatically switch based on the URL that you are browsing.  For example, at work, I set up a rule to route all web requests for the lab network through my SSH session that goes to the lab gateway machine.

It works like this.  First, I set up a tunnel to the lab gateway machine:

  • ssh -D 10001 alan@ (lab gateway)

Then I set up the following FoxyProxy rules:

  • 192.168.100.* – SOCKS localhost:10001 (SSH to lab)
  • anything else – direct

Now, I can get to the lab machines using the lab proxy, and any other site that I access uses the normal corporate network.

Closing thoughts

If you like Putty or FoxyProxy or Proxy Switchy!, please consider donating to the projects to support and encourage the developers.

How to resize a VirtualBox disk


I primarily run Linux, but occasionally need to run something in Windows.  So I use the open source VirtualBox program to run Windows XP in a box (where it belongs).

Most of the time, I create a new VM for a specific task.  For example, around tax time, I might create a VM that has nothing in it except WIndows XP and Turbo Tax or Tax Cut — or better yet — Tax Act.  To make this a little easier, I create a VM that has a fresh copy of Windows XP on a 10GB hard disk, and then I made a compressed copy of the disk image file.  It compresses nicely down to 700MB, just about right to fit on a CD.  So step one of doing my taxes is to copy this disk image file, uncompress it, and create a new VM that uses that disk.

That is, I treat the Windows OS as disposable — create a VM, use it, save my files somewhere else and then throw the VM away.

Every once in a while, though, I need more than the 10GB that I originally allocated to the VM.  For example, iTunes usually needs a lot more storage than that.  So I need to resize the Windows C: drive.  It turns out that this is very easy to do with the open source tool “gparted“.  This comes included in Ubuntu Live CD’s, so I just boot the VM into an Ubuntu Live CD session and prepare the new, larger disk.

Here’s the step-by-step, gleefully stolen from the VirtualBox support forums.

  • Make sure you have an Ubuntu Live CD, or a “System Rescue D“.
  • Create a new hard disk image (*.vdi file) using Virtual Disk Manager.  In VirtualBox, go to File / Virtual Disk Manager.
  • Set your current VM to use the new disk image as it’s second hard disk and the Ubuntu Live CD (or System Rescue CD) ISO file as it’s CDROM device.
  • Boot the VM from the CDROM.
  • If you’re using the System Rescue CD, start a graphical “X-windows” session by typing startx at the command prompt.
  • When your X-windows starts up, open up a terminal and type gparted.
  • You’ll need to create a partition on the new disk.  So in gparted, select the new disk and the Device / Create Partition Table.
  • Then select the windows partition and choose copy.
  • Select the second hard disk, right click on the representation of the disk and click paste.
  • Gparted will prompt you for the size of the disk, drag the slider to the max size.
  • Click apply and wait…
  • Important – when it is done, right click on the disk and choose Manage Flags, and select Boot.
  • Exit gparted and power off the VM.
  • Change the VM settings to only have one disk (the new bigger disk) and un-select the ISO as the CDROM.
  • Boot the VM into your windows install on it’s new bigger disk!  The first time it boots up, Windows may do a disk check and reboot.

Once you’re happy with the new larger disk, you might want to delete the old, smaller one.

This method should work the same, regardless of whether the host OS is Linux, Windows or Mac OS.

Keep that Ubuntu Live CD around — it really comes in handy!

Disabling Linux user accounts – passwd and chage


On internet-facing machines, I like to disable the root password so that you can only log in as root using an SSH key.  This is done by setting the seemingly-scary option PermitRootLogin without-password in /etc/ssh/sshd_config .  This option means that you can ONLY use a key to log in as root… a password will never be accepted, and so you can not guess it by trial-and-error.

While I am at it, I go ahead and disable the root password completely.  That’ll be one less thing for me to remember, and one less thing to keep secret.

First, be sure that you have at least one “sudoer” user, or at least one SSH key in ~root/.ssh/authorized_keys . Otherwise, you’ll realize in an “ignisecond” that you’ve just locked yourself out.

Then, use the passwd command to “lock” the account.

$ passwd -l root

This command will put a ! character in the password field of the /etc/shadow file so that no password hash will ever match the string in the shadow file.

But on SOME SYSTEMS, it also does a second thing — it may change the account expiration date to January 2, 1970. This will prevent SSH access, even with a key.

While researching this article, I had this “world-shifting-under-me” feeling, as I distinctly remember having to work around this issue. However, on every system I tried, I could not reproduce the expiration-date-changing behavior. Then I found that it was an issue that flip-flopped in the Debian community between 2006 and 2008. In 2006, they made the passwd -l command lock the password and also expire the user account. But in 2008, they decided that the change affected too many people, where most had grown accustomed to the passwd command affecting the password only, and not touching the account expiration date.

If you want to change the account’s expiration date, you should use the chage command.

$ chage -E -1 root  # never expire
$ chage -E 1 root  # expire on Jan 2, 1970
$ chage -E 0 root  # not recommended, undefined behavior
$ chage -E 2010-12-25  # expire on a specific date

So here I was, all set to share a nugget of wisdom with the world, and instead, it ends up being a trip down memory lane. However, in the process, I learned a couple of things.

A very safe way to disable a user account’s password, while keeping the account open to SSH access is like this: passwd -l user && chage -E -1 user . The chage part is unnecessary on modern systems, but it does not hurt anything.

A quick way to check on whether a password is locked, or a user account is expired, is to use passwd -S user, like this.

$ passwd -S user
user P 09/11/2007 0 99999 7 -1

This says that user‘s password is set (P), it was changed way back 2007, there is no minimum age (restriction on how often they can change their password), there’s a very long maximum age (time when they are forced to change their password), the warning period is one week, and the account is not inactive/expired.

So there you go, two for one, a Linux tip and a history lesson!



When it comes to searching, there seems to be two battling camps: the ones that prefer to index stuff in the middle of the night, and the ones that just want to search when you need to search. The problem is that, many times, “in the middle of the night” does not end up being “when you’re not using the computer”. The other problem is that this sort of indexing operation can often completely cripple a machine, by using a lot of RAM and completely slamming disk I/O.

As far back as Windows 3.1, with it’s FindFast disk indexing tool, I have been annoyed by indexing processes that wake up and chew your hard disk to shreds… just in case you might want to search for something later.

What a stupid idea.

The latest culprit in Ubuntu is apt-xapian-index, which digs through your package list information, assembling some treasure trove of information that was apparently already on the disk, if you ever needed to ask for it.


sudo apt-get remove apt-xapian-index

A better long-term solution:

If you have information that you would like to be indexed for faster retrieval later, do the indexing upon insertion, not periodically. That is, when you apt-get install a package, set a trigger to update the relevant bits of your package index at that time.

Photo utility – renrot


When I take pictures with my digital cameras, they name the image files something like this:

  • Wife’s Nikon camera – dscn0115.jpg
  • My iPhone – img_0367.jpg
  • My Panasonic camera – p1070126.jpg

I tend to let the images pile up on the cameras for a while, and then I copy them all onto our file server. I start off by dumping them all into one big folder. But then I sort them into folders based on the date and the event, with names like “pictures/y2010/2010-02-14_chinese_new_year” for events and “pictures/y2010/2010-02” for the random shots. I don’t mind this process, and it’s actually kind of fun to review them as I am copying them to our file server.

I should note as an aside, when I worked for Ericsson’s research lab in Singapore, I was talking to one of the researchers who had studied the many ways that people could organize and categorize photographs, both paper and digital. It turns out that a huge majority of people they studied tended to associate photo sets together, based on relative dates. That is, if you asked for a particular photo, they would think “that was about the same time as Bob’s birthday party, so it must have been in August”.

I know that there are newer ways to organize photos, with databases and “tags” and what-not. Mac people really love to let the computer take care of those details. But I still like the idea of using folders with dates… old school.

Since I am dealing with dates and times of photos, it seems a little silly that all of my photos are named using dumb serial numbers. I find myself looking at the image’s EXIF properties, the information that the camera stores about the image — when and where the image was taken, what the camera settings were, etc. This seems a little tedious.

I recently found a utility called “renrot”. It’s primary job is to read the EXIF data from a photo, and rotate the image to match the EXIF rotation flag. That is, it rewrites the image so that it will load from top-to-bottom, which makes it more compatible with less-than-intelligent viewers — like some digital picture frames. But while it’s doing that, it also renames the file based on the time and date of when the photo was taken.

So now I can start by going into the directory that contains my big pile of photos and doing this:

renrot --name-template %Y%m%d-%H%M%S --extension jpg *.JPG

If I wanted to be careful not to mix the iPhone pictures with the Nikon pictures, I could do this:

renrot --name-template iphone-%Y%m%d-%H%M%S --extension jpg IMG_*.JPG
renrot --name-template nikon-%Y%m%d-%H%M%S --extension jpg DSCN*.JPG
renrot --name-template pan-%Y%m%d-%H%M%S --extension jpg P*.JPG

If I want to be a purist, and just rename the photos without actually rotating the image, I can do that, too.

renrot --no-rotate --name-template %Y%m%d-%H%M%S --extension jpg *.JPG

Pretty cool.

When ‘sudo’ pauses…


On my Ubuntu 9.10 server at home, I had been having a hard-to-diagnose problem where the “sudo” command will pause for 20 seconds before getting on with its business.

What made this problem so hard to track down is that it would happen once, and then the log jam would be cleared for a while. I would usually see it the first time I issued a sudo command, but never again in that session. The next day, it would do it again.

Last night, I finally tracked the problem down.

What helped the most was the discovery that I could do “sudo -K” to make sudo “forget” my earlier authentication. When I re-tried to run a sudo command, it would prompt for a password and then delay 20 seconds… every time.

So now I had a way to test out theories. I just needed some theories to test.

I saw many reports on the internet about Fedora users seeing a similar issue. Their problem turned out to be in the /etc/hosts file — there were problems if “localhost” and “localhost.localdomain” and even the machine’s given hostname were not listed there. But this was not the case for me. My hosts file was fine.

Instead, I started tracing what happens when sudo is called (unfortunately, you can’t just “strace sudo somecommand“, because strace does not like to trace a setuid program).

I looked at PAM, the pluggable authentication modules. In the /etc/pam.d directory, there was a file called “sudo”. This did not have anything interesting in it. But it did include a couple of other files: “common-auth” and “common-account”. It turns out that the last line in the common-auth file was the culprit:

auth  optional  unwrap

This line is supposed to decrypt the user’s home directory if is encrypted, so it can read the files as part of sudo’s startup. But I don’t have any encrypted home directories. So for me, this is unnecessary.

Commenting out this line made the 20-second delay go away.

Plugging external commands into ‘gqview’


I have been taking digital photographs since 1998, and so one piece of software which I use quite a bit is my photo organizer. I use a tool called “gqview”. It’s a very nice browser that shows folders and thumbnails and a large image on the screen at the same time. And it does a good job showing slide shows.

One nice feature of gqview is that you can plug external programs into the ‘edit’ menu. When you install it (at least on Ubuntu systems), a few of the plug-in’s are populated already:

  • edit in GIMP (external package: gimp)
  • edit in xpaint (external package: xpaint)
  • rotate clockwise (external package: libjpeg-progs)
  • rotate counter-clockwise (external package: libjpeg-progs)

I wanted to add a plug-in to shrink images a certain percentage. I often find that I want to send a bunch of pictures to friends via email, and I really don’t want to send a 50 MB email, and I don’t want the pictures to be so big that the reader had to scroll to see it.

It’s pretty easy to add the plug-in. The actual shrinking can be handled easily with the ‘convert’ command, which is part of the excellent ‘imagemagick’ set of graphics tools. Plugging convert into gqview is as easy as this:

%vif convert %p -scale 25% %p_tmp ; then
    mv %p_tmp %p ; else rm %p_tmp ; fi

I didn’t make this up myself — I simply copied the “rotate” plug-in’s and changed what I needed to. I am not sure what the %v is for, but it’s pretty clear that %p is the full path of the image file.

Now I can highlight a whole bunch of photos and shrink them all with a single command.

Surgical web page editing with “stylish”


Have you noticed that some web sites have ads that are somewhat… unpredictable? Facebook is a good example of this. The ads that appear on the right can sometimes contain trashy-looking content. I’d like to hide them, if I could.

I found a Firefox plugin that does a good job of slicing out selected bits of content. It’s called “stylish“.

Just install the plugin, and then set up a script that intercepts the stylesheet for a particular site and does a little on-the-fly modification. The script I use for Facebook looks like this:

@namespace url(;
@-moz-document domain("") {
   .emu_ad, .UIStandardFrame_SidebarAds, #home_sponsor {
      display: none !important;

I don’t claim to fully understand the script, but I do know that it looks for a specific block in the Facebook HTML and style sheet, and it sets the style for the sidebar ad box to “display: none”. That makes it disappear.

Back to the Future


A few days ago, I learned a very important lesson about filesystems and snapshots. I learned that a complete copy is not always a Good Thing™.

I help manage a server for our local Linux Users Group. We have about 250 users on the system, and all of our system administration is done by volunteers.

A few months ago, I made a complete backup of our /home partition using the guidelines that have been told to me by Smart People™:

  • make a snapshot volume of /home (called home-snap)
  • make a new empty volume (called home-backup)
  • use ‘dd‘ to copy from home-snap to home-backup
  • remove the home-snap snapshot volume

All was fine, until a few months later, when we decided to reboot.

When the machine rebooted, it mounted the WRONG copy of /home. It looked in /etc/fstab to see what to mount, read the UUID, and started looking for that filesystem among the logical volumes.

Here’s a list of the available filesystems and their UUID’s.

root@pilot:~# blkid
/dev/mapper/vg01-home: UUID="1a578e6f-772b-4892-86e3-1181aadda119" TYPE="ext3" SEC_TYPE="ext2"
/dev/mapper/vg01-home-backup: UUID="1a578e6f-772b-4892-86e3-1181aadda119" TYPE="ext3" SEC_TYPE="ext2"
/dev/mapper/vg01-swap: TYPE="swap" UUID="303f2743-da69-466b-a200-40a1a369fa1c"
/dev/mapper/vg01-u804: UUID="b5689a93-b7ad-4011-a0f9-ffaf2d68bf6f" TYPE="ext3"
/dev/sdb: UUID="Uh0TI1-pxD4-M1Pm-5kP3-zU1a-IRgm-bD0JAq" TYPE="lvm2pv"
/dev/sda: UUID="9oZhBo-3DPP-1eay-kgGM-fd06-yuJB-c2eCo7" TYPE="lvm2pv"
/dev/sdc1: UUID="5c15308e-a81b-4fd9-b2c2-7ef3fe39ce0b" SEC_TYPE="ext2" TYPE="ext3"
/dev/sdc2: TYPE="swap" UUID="08c55fa5-3379-4f6a-b798-4b8f3ead6790"
/dev/sdc3: UUID="5a544a7f-90ed-474c-b096-1b5929c83109" SEC_TYPE="ext2" TYPE="ext3"

Notice anything goofy? Yes, the UUID for the home volume is the same as the UUID for the home-backup volume! Of course it is… I used ‘dd‘ to copy the entire volume!

So our machine booted up, looked for a filesystem whose UUID was ‘1a578e6f-772b-4892-86e3-1181aadda119’ and it mounted it on /home. Unfortunately, it found the home-backup volume before it found the real home volume, and so our 250 users took a step back in time for the evening.

All of the files in our home directories looked like they did back in May.

On the surface, this does not seem like such a Bad Thing™. But over the course of the next few hours, users started receiving email, and logging IRC chats, and doing all of the other things that users do. These new emails and log files were written to home-backup instead of home, and so now we were starting to mix old and new files.

This is a lot like the movie “Back to the Future”, when Marty’s mom tries to kiss him. Except the characters involved here are not as good-looking.

The fix was quick and painless. I simply generated a new UUID for the home-backup volume, and then rebooted. The magic command is simply:

 tune2fs -U random /dev/mapper/vg01-home-backup

But the cleanup would come later. If someone were interested in the emails or log files that were mistakenly written to the wrong volume (their “past life”), then they would need to look on that volume for “new” files. Pretty easy work.

find /mnt/home-backup/porter -mtime -7

This will show all files in my “backup” home directory that are less than a week old. Since the backup was made four months ago, I would expect all files in that directory to either be more than four months old, or just one day old. This command will show you the new files.

So I am revising the backup procedure as follows:

  • make a snapshot volume of /home (called home-snap)
  • make a new empty volume (called home-backup)
  • use ‘dd‘ to copy from home-snap to home-backup
  • remove the home-snap snapshot volume
  • change the UUID on home-backup ◄— new

In fact, now that we already have a base to work with, I might just use rsync to copy files instead of dd to copy the entire volume. This will leave the backup with its own UUID, and will avoid collisions like the one we saw.

iPhone VPN


One of the coolest features of the iPhone is the way it uses the best data network that it can find. If you’re at home or at work, or even at a coffee shop, it will use the local wifi network. But if you’re out of range of any suitable wifi networks, it will use AT&T’s “3G” (UMTS) network. And if it can’t find a UMTS network, it’ll fall back to EDGE. Phone companies call this hybrid approach “ABC”, or “always best connection”.

Now that I have an internet device in my pocket, I find myself using public (or otherwise open) wifi connections quite a bit. And this carries with it some unintended consequences. That is… everything I type and everything I read is transmitted in the clear, unencrypted.

I try to make a habit of encrypting my data traffic whenever possible. My mail server is set up to only allow SSL connections. So no matter where I check my mail from, I am forced to use an encrypted connection. Similarly, banks and commerce web sites usually force you to switch to HTTPS before you start entering information. But there are a lot of applications on the iPhone that do not use encryption at all.

You might ask yourself why bother to encrypt your Twitter connection, since what you type is going to be blasted out to the world anyway. But the point is…

If you encrypt everything, then nothing is left to chance.

So I decided to explore a VPN option on the iPhone. It supports three flavors of VPN: L2TP, PPTP and IPSec. I was disappointed (but not surprised) that “openvpn” was not an option, since I already use this excellent open source SSL-based VPN package.

So I decided to give PPTP a try.

Setting up the PPTP server

On my Ubuntu 8.04 LTS server, I installed a PPTP server called, appropriately enough, “pptpd“. Configuration was very easy. Most of the setup was done for me after I did the standard apt-get install pptpd. I simply needed to pick a private subnet that would be used for my VPN clients, and an IP address in that subnet to use for the server. I chose the subnet and for the server (these addresses are part of a private network address space, defined by RFC 1918, just like 192.168.x.x and 10.x.x.x addresses).

My /etc/pptp.conf configuration file for the pptp daemon looks like this:

option /etc/ppp/pptpd-options

I also needed to tell the daemon to give out some DNS addresses when a client connects, so in the /etc/ppp/pptpd-options file, I added the two “ms-dns” lines below:

name pptpd
ms-dns  #
ms-dns  #

Finally, I needed to add an entry into the /etc/ppp/chap-secrets file that would contain my password. Mine looks like this:

alan pptpd MyHardToGuessPassword *

At this point, the PPTP server was completely configured, so I restarted it with service pptpd restart.

Setting up the iPhone

On the iPhone, I needed to set up a VPN client. This is very easy. On the settings screen, go to general / network / VPN and “Add VPN Configuration…”. Then just fill in the blanks.

  • choose “PPTP”
  • enter a description
  • your server’s IP address
  • the username (from above)
  • RSA SecurID=OFF
  • the password (from above)
  • encryption level = Auto
  • “Send All Traffic” = ON
  • Proxy = OFF

Click on “Save” and you will see a switch in the network tab and also in the main settings tab to turn the VPN on and off.

For now, I am leaving it off unless I am on a public network. I am not sure, but I think that keeping the VPN alive might use a lot of battery. So I do not use it unless I need it.


For me to get this VPN on the internet, I had to do two more things: punch a hole in my firewall for the PPTP traffic, and forward traffic from my VPN out to the rest of the world.

For my server, both of these tasks were handed by the same tool: shorewall.

I added a “masquerade” rule to /etc/shorewall/masq to NAT all of the traffic from 172.16.4.x out through my main network interface.

eth0    # OpenVPN and PPTP

And then I added two rules to /etc/shorewall/rules to allow the PPTP traffic in.

ACCEPT  net  fw   tcp  1723  # PPTP
ACCEPT  net  fw   gre        # PPTP

When shorewall starts, it will generate the iptables rules that are used by the kernel to filter packets. If you’re using hand-written iptables rules, then you will need some rules that look something like this:

# accept "gre" protocol traffic (PPTP tunnel traffic)
iptables -A INPUT -p gre -j ACCEPT
iptables -A OUTPUT -p gre -j ACCEPT
# accept PPTP control traffic to TCP port 1723
# (my server IP is
iptables -A INPUT -p tcp --sport 1723 -s -j ACCEPT
iptables -A OUTPUT -p tcp --dport 1723 -d -j ACCEPT
# masquerade/NAT internet traffic out of interface eth0
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
# turn on packet forwarding
echo "1" > /proc/sys/net/ipv4/ip_forward


Now, when I am in a coffee shop, I can turn on the VPN easily by flipping the switch in the iPhone’s main settings screen. It will make a TCP connection to my server, negotiate a few things, and then send all further network traffic through an encrypted tunnel directly to my server, which relays it out to the internet.

You can test this by going to a web site like If the VPN is working, it will show your server’s IP address. If not, it’ll show the coffee shop’s IP address.

Once I am using the VPN, anyone in the coffee shop who happens to be sniffing traffic on the wireless network will only see a single connection from my iPhone to my server, but the contents of this connection will be scrambled.

Go to Top