free software
Linux, open source software, tips and tricks.
Disabling Linux user accounts – passwd and chage
0On internet-facing machines, I like to disable the root password so that you can only log in as root using an SSH key. This is done by setting the seemingly-scary option PermitRootLogin without-password in /etc/ssh/sshd_config . This option means that you can ONLY use a key to log in as root… a password will never be accepted, and so you can not guess it by trial-and-error.
While I am at it, I go ahead and disable the root password completely. That’ll be one less thing for me to remember, and one less thing to keep secret.
First, be sure that you have at least one “sudoer” user, or at least one SSH key in ~root/.ssh/authorized_keys . Otherwise, you’ll realize in an “ignisecond” that you’ve just locked yourself out.
Then, use the passwd command to “lock” the account.
$ passwd -l root
This command will put a ! character in the password field of the /etc/shadow file so that no password hash will ever match the string in the shadow file.
But on SOME SYSTEMS, it also does a second thing — it may change the account expiration date to January 2, 1970. This will prevent SSH access, even with a key.
While researching this article, I had this “world-shifting-under-me” feeling, as I distinctly remember having to work around this issue. However, on every system I tried, I could not reproduce the expiration-date-changing behavior. Then I found that it was an issue that flip-flopped in the Debian community between 2006 and 2008. In 2006, they made the passwd -l command lock the password and also expire the user account. But in 2008, they decided that the change affected too many people, where most had grown accustomed to the passwd command affecting the password only, and not touching the account expiration date.
If you want to change the account’s expiration date, you should use the chage command.
$ chage -E -1 root # never expire $ chage -E 1 root # expire on Jan 2, 1970 $ chage -E 0 root # not recommended, undefined behavior $ chage -E 2010-12-25 # expire on a specific date
So here I was, all set to share a nugget of wisdom with the world, and instead, it ends up being a trip down memory lane. However, in the process, I learned a couple of things.
A very safe way to disable a user account’s password, while keeping the account open to SSH access is like this: passwd -l user && chage -E -1 user . The chage part is unnecessary on modern systems, but it does not hurt anything.
A quick way to check on whether a password is locked, or a user account is expired, is to use passwd -S user, like this.
$ passwd -S user user P 09/11/2007 0 99999 7 -1
This says that user‘s password is set (P), it was changed way back 2007, there is no minimum age (restriction on how often they can change their password), there’s a very long maximum age (time when they are forced to change their password), the warning period is one week, and the account is not inactive/expired.
So there you go, two for one, a Linux tip and a history lesson!
apt-xapian-index
2When it comes to searching, there seems to be two battling camps: the ones that prefer to index stuff in the middle of the night, and the ones that just want to search when you need to search. The problem is that, many times, “in the middle of the night” does not end up being “when you’re not using the computer”. The other problem is that this sort of indexing operation can often completely cripple a machine, by using a lot of RAM and completely slamming disk I/O.
As far back as Windows 3.1, with it’s FindFast disk indexing tool, I have been annoyed by indexing processes that wake up and chew your hard disk to shreds… just in case you might want to search for something later.
What a stupid idea.
The latest culprit in Ubuntu is apt-xapian-index, which digs through your package list information, assembling some treasure trove of information that was apparently already on the disk, if you ever needed to ask for it.
Solution:
sudo apt-get remove apt-xapian-index
A better long-term solution:
If you have information that you would like to be indexed for faster retrieval later, do the indexing upon insertion, not periodically. That is, when you apt-get install a package, set a trigger to update the relevant bits of your package index at that time.
Installing Ubuntu on LVM
0I am a big fan of LVM, the Linux Logical Volume Manager. It allows me to create, resize and delete disk partitions very easily. That comes in handy when you’re trying new things out. And it also has a nice snapshot feature, which I use when I am making a backup of a partition… while it is still in use.
If you use the Ubuntu alternate install CD, it gives you menu options for using and configuring Logical Volume Manager. But if you’re using the normal Live installation CD, you have to do it manually. Fortunately, this is not too hard. Below, I will tell how I do it.
- Boot the Live CD and open a shell.
- Install the LVM package in the Live CD environment:
apt-get install lvm2
- Tell LVM to look for existing volumes, and make them active:
vgchange -ay
- You may need to mark one or more physical volumes as being LVM pv’s:
pvcreate /dev/sdX1
- You may need to create a volume group:
vgcreate vglaptop /dev/sdX1
- Create the volumes that you want to use:
lvcreate vglaptop --name=ubuntu --size=10G
- Put a filesystem on the new volume:
mkfs.ext4 /dev/vglaptop/ubuntu
- Run the installer and do “manual partitioning”.
- Use the menus to match up the newly formatted filesystems with the different mount points(root, home, etc).
- When the installer finishes, tell it to “keep on testing”. Do not reboot.
- Mount the new root filesystem:
cd /mnt ; mkdir u ; mount /dev/vglaptop/ubuntu u
- Mount any other partitions below that:
mount /dev/sda1 u/boot ; mount /dev/vglaptop/home u/home
(actually, you only need the root filesystem and /boot — you won’t need /home and others)
- Mount the “special” filesystems:
mount --bind /dev u/dev ; mount -t proc proc u/proc
- Start up a “change root”:
chroot u
A new shell will start, using the new root filesystem as its root.
- Install the LVM package inside the chroot:
apt-get install lvm2
- Re-build the initial RAM disk:
update-initramfs -u -k all
(this step may be done for you when you install LVM)
- You may need to update grub:
update-grub
- Exit the chroot shell:
exit
- Cleanly unmount everything:
umount u/dev ; umount u/proc ; umount u/home ; umount u
- Reboot:
reboot
And when it boots up, it will load the kernel and initramfs, and then it will mount the root filesystem, which is on your new logical volume!
Photo utility – renrot
0When I take pictures with my digital cameras, they name the image files something like this:
- Wife’s Nikon camera –
dscn0115.jpg - My iPhone –
img_0367.jpg - My Panasonic camera –
p1070126.jpg
I tend to let the images pile up on the cameras for a while, and then I copy them all onto our file server. I start off by dumping them all into one big folder. But then I sort them into folders based on the date and the event, with names like “pictures/y2010/2010-02-14_chinese_new_year” for events and “pictures/y2010/2010-02” for the random shots. I don’t mind this process, and it’s actually kind of fun to review them as I am copying them to our file server.
I should note as an aside, when I worked for Ericsson’s research lab in Singapore, I was talking to one of the researchers who had studied the many ways that people could organize and categorize photographs, both paper and digital. It turns out that a huge majority of people they studied tended to associate photo sets together, based on relative dates. That is, if you asked for a particular photo, they would think “that was about the same time as Bob’s birthday party, so it must have been in August”.
I know that there are newer ways to organize photos, with databases and “tags” and what-not. Mac people really love to let the computer take care of those details. But I still like the idea of using folders with dates… old school.
Since I am dealing with dates and times of photos, it seems a little silly that all of my photos are named using dumb serial numbers. I find myself looking at the image’s EXIF properties, the information that the camera stores about the image — when and where the image was taken, what the camera settings were, etc. This seems a little tedious.
I recently found a utility called “renrot”. It’s primary job is to read the EXIF data from a photo, and rotate the image to match the EXIF rotation flag. That is, it rewrites the image so that it will load from top-to-bottom, which makes it more compatible with less-than-intelligent viewers — like some digital picture frames. But while it’s doing that, it also renames the file based on the time and date of when the photo was taken.
So now I can start by going into the directory that contains my big pile of photos and doing this:
renrot --name-template %Y%m%d-%H%M%S --extension jpg *.JPG
If I wanted to be careful not to mix the iPhone pictures with the Nikon pictures, I could do this:
renrot --name-template iphone-%Y%m%d-%H%M%S --extension jpg IMG_*.JPG renrot --name-template nikon-%Y%m%d-%H%M%S --extension jpg DSCN*.JPG renrot --name-template pan-%Y%m%d-%H%M%S --extension jpg P*.JPG
If I want to be a purist, and just rename the photos without actually rotating the image, I can do that, too.
renrot --no-rotate --name-template %Y%m%d-%H%M%S --extension jpg *.JPG
Pretty cool.
When ‘sudo’ pauses…
1On my Ubuntu 9.10 server at home, I had been having a hard-to-diagnose problem where the “sudo” command will pause for 20 seconds before getting on with its business.
What made this problem so hard to track down is that it would happen once, and then the log jam would be cleared for a while. I would usually see it the first time I issued a sudo command, but never again in that session. The next day, it would do it again.
Last night, I finally tracked the problem down.
What helped the most was the discovery that I could do “sudo -K” to make sudo “forget” my earlier authentication. When I re-tried to run a sudo command, it would prompt for a password and then delay 20 seconds… every time.
So now I had a way to test out theories. I just needed some theories to test.
I saw many reports on the internet about Fedora users seeing a similar issue. Their problem turned out to be in the /etc/hosts file — there were problems if “localhost” and “localhost.localdomain” and even the machine’s given hostname were not listed there. But this was not the case for me. My hosts file was fine.
Instead, I started tracing what happens when sudo is called (unfortunately, you can’t just “strace sudo somecommand“, because strace does not like to trace a setuid program).
I looked at PAM, the pluggable authentication modules. In the /etc/pam.d directory, there was a file called “sudo”. This did not have anything interesting in it. But it did include a couple of other files: “common-auth” and “common-account”. It turns out that the last line in the common-auth file was the culprit:
auth optional pam_ecryptfs.so unwrap
This line is supposed to decrypt the user’s home directory if is encrypted, so it can read the files as part of sudo’s startup. But I don’t have any encrypted home directories. So for me, this is unnecessary.
Commenting out this line made the 20-second delay go away.
Plugging external commands into ‘gqview’
0I have been taking digital photographs since 1998, and so one piece of software which I use quite a bit is my photo organizer. I use a tool called “gqview”. It’s a very nice browser that shows folders and thumbnails and a large image on the screen at the same time. And it does a good job showing slide shows.
One nice feature of gqview is that you can plug external programs into the ‘edit’ menu. When you install it (at least on Ubuntu systems), a few of the plug-in’s are populated already:
- edit in GIMP (external package: gimp)
- edit in xpaint (external package: xpaint)
- rotate clockwise (external package: libjpeg-progs)
- rotate counter-clockwise (external package: libjpeg-progs)
I wanted to add a plug-in to shrink images a certain percentage. I often find that I want to send a bunch of pictures to friends via email, and I really don’t want to send a 50 MB email, and I don’t want the pictures to be so big that the reader had to scroll to see it.
It’s pretty easy to add the plug-in. The actual shrinking can be handled easily with the ‘convert’ command, which is part of the excellent ‘imagemagick’ set of graphics tools. Plugging convert into gqview is as easy as this:
%vif convert %p -scale 25% %p_tmp ; then
mv %p_tmp %p ; else rm %p_tmp ; fi
I didn’t make this up myself — I simply copied the “rotate” plug-in’s and changed what I needed to. I am not sure what the %v is for, but it’s pretty clear that %p is the full path of the image file.
Now I can highlight a whole bunch of photos and shrink them all with a single command.
Surgical web page editing with “stylish”
3Have you noticed that some web sites have ads that are somewhat… unpredictable? Facebook is a good example of this. The ads that appear on the right can sometimes contain trashy-looking content. I’d like to hide them, if I could.
I found a Firefox plugin that does a good job of slicing out selected bits of content. It’s called “stylish“.
Just install the plugin, and then set up a script that intercepts the stylesheet for a particular site and does a little on-the-fly modification. The script I use for Facebook looks like this:
@namespace url(http://www.w3.org/1999/xhtml);
@-moz-document domain("facebook.com") {
.emu_ad, .UIStandardFrame_SidebarAds, #home_sponsor {
display: none !important;
}
}
I don’t claim to fully understand the script, but I do know that it looks for a specific block in the Facebook HTML and style sheet, and it sets the style for the sidebar ad box to “display: none”. That makes it disappear.
Back to the Future
2A few days ago, I learned a very important lesson about filesystems and snapshots. I learned that a complete copy is not always a Good Thing™.
I help manage a server for our local Linux Users Group. We have about 250 users on the system, and all of our system administration is done by volunteers.
A few months ago, I made a complete backup of our /home partition using the guidelines that have been told to me by Smart People™:
- make a snapshot volume of /home (called
home-snap) - make a new empty volume (called
home-backup) - use ‘
dd‘ to copy fromhome-snaptohome-backup - remove the
home-snapsnapshot volume
All was fine, until a few months later, when we decided to reboot.
When the machine rebooted, it mounted the WRONG copy of /home. It looked in /etc/fstab to see what to mount, read the UUID, and started looking for that filesystem among the logical volumes.
Here’s a list of the available filesystems and their UUID’s.
root@pilot:~# blkid /dev/mapper/vg01-home: UUID="1a578e6f-772b-4892-86e3-1181aadda119" TYPE="ext3" SEC_TYPE="ext2" /dev/mapper/vg01-home-backup: UUID="1a578e6f-772b-4892-86e3-1181aadda119" TYPE="ext3" SEC_TYPE="ext2" /dev/mapper/vg01-swap: TYPE="swap" UUID="303f2743-da69-466b-a200-40a1a369fa1c" /dev/mapper/vg01-u804: UUID="b5689a93-b7ad-4011-a0f9-ffaf2d68bf6f" TYPE="ext3" /dev/sdb: UUID="Uh0TI1-pxD4-M1Pm-5kP3-zU1a-IRgm-bD0JAq" TYPE="lvm2pv" /dev/sda: UUID="9oZhBo-3DPP-1eay-kgGM-fd06-yuJB-c2eCo7" TYPE="lvm2pv" /dev/sdc1: UUID="5c15308e-a81b-4fd9-b2c2-7ef3fe39ce0b" SEC_TYPE="ext2" TYPE="ext3" /dev/sdc2: TYPE="swap" UUID="08c55fa5-3379-4f6a-b798-4b8f3ead6790" /dev/sdc3: UUID="5a544a7f-90ed-474c-b096-1b5929c83109" SEC_TYPE="ext2" TYPE="ext3" root@pilot:~#
Notice anything goofy? Yes, the UUID for the home volume is the same as the UUID for the home-backup volume! Of course it is… I used ‘dd‘ to copy the entire volume!
So our machine booted up, looked for a filesystem whose UUID was ‘1a578e6f-772b-4892-86e3-1181aadda119’ and it mounted it on /home. Unfortunately, it found the home-backup volume before it found the real home volume, and so our 250 users took a step back in time for the evening.
All of the files in our home directories looked like they did back in May.
On the surface, this does not seem like such a Bad Thing™. But over the course of the next few hours, users started receiving email, and logging IRC chats, and doing all of the other things that users do. These new emails and log files were written to home-backup instead of home, and so now we were starting to mix old and new files.
This is a lot like the movie “Back to the Future”, when Marty’s mom tries to kiss him. Except the characters involved here are not as good-looking.
The fix was quick and painless. I simply generated a new UUID for the home-backup volume, and then rebooted. The magic command is simply:
tune2fs -U random /dev/mapper/vg01-home-backup
But the cleanup would come later. If someone were interested in the emails or log files that were mistakenly written to the wrong volume (their “past life”), then they would need to look on that volume for “new” files. Pretty easy work.
find /mnt/home-backup/porter -mtime -7
This will show all files in my “backup” home directory that are less than a week old. Since the backup was made four months ago, I would expect all files in that directory to either be more than four months old, or just one day old. This command will show you the new files.
So I am revising the backup procedure as follows:
- make a snapshot volume of /home (called
home-snap) - make a new empty volume (called
home-backup) - use ‘
dd‘ to copy fromhome-snaptohome-backup - remove the
home-snapsnapshot volume - change the UUID on
home-backup◄— new
In fact, now that we already have a base to work with, I might just use rsync to copy files instead of dd to copy the entire volume. This will leave the backup with its own UUID, and will avoid collisions like the one we saw.
iPhone VPN
2One of the coolest features of the iPhone is the way it uses the best data network that it can find. If you’re at home or at work, or even at a coffee shop, it will use the local wifi network. But if you’re out of range of any suitable wifi networks, it will use AT&T’s “3G” (UMTS) network. And if it can’t find a UMTS network, it’ll fall back to EDGE. Phone companies call this hybrid approach “ABC”, or “always best connection”.
Now that I have an internet device in my pocket, I find myself using public (or otherwise open) wifi connections quite a bit. And this carries with it some unintended consequences. That is… everything I type and everything I read is transmitted in the clear, unencrypted.
I try to make a habit of encrypting my data traffic whenever possible. My mail server is set up to only allow SSL connections. So no matter where I check my mail from, I am forced to use an encrypted connection. Similarly, banks and commerce web sites usually force you to switch to HTTPS before you start entering information. But there are a lot of applications on the iPhone that do not use encryption at all.
You might ask yourself why bother to encrypt your Twitter connection, since what you type is going to be blasted out to the world anyway. But the point is…
If you encrypt everything, then nothing is left to chance.
So I decided to explore a VPN option on the iPhone. It supports three flavors of VPN: L2TP, PPTP and IPSec. I was disappointed (but not surprised) that “openvpn” was not an option, since I already use this excellent open source SSL-based VPN package.
So I decided to give PPTP a try.
Setting up the PPTP server
On my Ubuntu 8.04 LTS server, I installed a PPTP server called, appropriately enough, “pptpd“. Configuration was very easy. Most of the setup was done for me after I did the standard apt-get install pptpd. I simply needed to pick a private subnet that would be used for my VPN clients, and an IP address in that subnet to use for the server. I chose the 172.16.4.0/16 subnet and 172.16.4.1 for the server (these addresses are part of a private network address space, defined by RFC 1918, just like 192.168.x.x and 10.x.x.x addresses).
My /etc/pptp.conf configuration file for the pptp daemon looks like this:
option /etc/ppp/pptpd-options logwtmp localip 172.16.4.1 remoteip 172.16.4.2-250
I also needed to tell the daemon to give out some DNS addresses when a client connects, so in the /etc/ppp/pptpd-options file, I added the two “ms-dns” lines below:
name pptpd refuse-pap refuse-chap refuse-mschap require-mschap-v2 require-mppe-128 ms-dns 208.67.222.222 # resolver1.opendns.com ms-dns 208.67.220.220 # resolver2.opendns.com proxyarp nodefaultroute lock nobsdcomp
Finally, I needed to add an entry into the /etc/ppp/chap-secrets file that would contain my password. Mine looks like this:
alan pptpd MyHardToGuessPassword *
At this point, the PPTP server was completely configured, so I restarted it with service pptpd restart.
Setting up the iPhone
On the iPhone, I needed to set up a VPN client. This is very easy. On the settings screen, go to general / network / VPN and “Add VPN Configuration…”. Then just fill in the blanks.
- choose “PPTP”
- enter a description
- your server’s IP address
- the username (from above)
- RSA SecurID=OFF
- the password (from above)
- encryption level = Auto
- “Send All Traffic” = ON
- Proxy = OFF
Click on “Save” and you will see a switch in the network tab and also in the main settings tab to turn the VPN on and off.
For now, I am leaving it off unless I am on a public network. I am not sure, but I think that keeping the VPN alive might use a lot of battery. So I do not use it unless I need it.
Networking
For me to get this VPN on the internet, I had to do two more things: punch a hole in my firewall for the PPTP traffic, and forward traffic from my VPN out to the rest of the world.
For my server, both of these tasks were handed by the same tool: shorewall.
I added a “masquerade” rule to /etc/shorewall/masq to NAT all of the traffic from 172.16.4.x out through my main network interface.
eth0 172.16.4.0/24 # OpenVPN and PPTP
And then I added two rules to /etc/shorewall/rules to allow the PPTP traffic in.
ACCEPT net fw tcp 1723 # PPTP ACCEPT net fw gre # PPTP
When shorewall starts, it will generate the iptables rules that are used by the kernel to filter packets. If you’re using hand-written iptables rules, then you will need some rules that look something like this:
# accept "gre" protocol traffic (PPTP tunnel traffic) iptables -A INPUT -p gre -j ACCEPT iptables -A OUTPUT -p gre -j ACCEPT # accept PPTP control traffic to TCP port 1723 # (my server IP is 11.22.33.44) iptables -A INPUT -p tcp --sport 1723 -s 11.22.33.44 -j ACCEPT iptables -A OUTPUT -p tcp --dport 1723 -d 11.22.33.44 -j ACCEPT # masquerade/NAT internet traffic out of interface eth0 iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE # turn on packet forwarding echo "1" > /proc/sys/net/ipv4/ip_forward
Conclusion
Now, when I am in a coffee shop, I can turn on the VPN easily by flipping the switch in the iPhone’s main settings screen. It will make a TCP connection to my server, negotiate a few things, and then send all further network traffic through an encrypted tunnel directly to my server, which relays it out to the internet.
You can test this by going to a web site like http://www.whatismyip.com/. If the VPN is working, it will show your server’s IP address. If not, it’ll show the coffee shop’s IP address.
Once I am using the VPN, anyone in the coffee shop who happens to be sniffing traffic on the wireless network will only see a single connection from my iPhone to my server, but the contents of this connection will be scrambled.
Custom ring tones for the iPhone
1I spent five years working for Ericsson, devoting all of my professional energy towards making cellular phones better. I developed device drivers, application software, “middleware”, protocol stacks, and device prototypes. Not a day went by where I did not think about cellular phones in some way. It was not just a job, but a total gadget lifestyle.
Even in areas where I was not actively assigned to work, I still found ways to influence our products. Two particular examples come to mind.
In mid-2000, I was invited by our vice president (one of the two top managers in our NC office) to attend an all-day brainstorming session to discuss phones for kids and pre-teens: what it would mean for the kids, for the phone companies, for the parents, and for the content providers. I was a bit shocked when I entered the room — the attendee list was much smaller than I had expected, and I turned out to be the only software developer there. But the session went well, and I shared my (year 2000-era) thoughts on how Ericsson could never write even a small fraction of the applications that our customers would want, and so we would need to include some sort of API or virtual machine. At the time, Java looked promising. Fast forward to today, and see the success of Apple’s app store.
The second story, if you’ll indulge me (it is my blog, after all), is when I made friends with the King of Rings in Sweden. He was responsible for all ring tones that we delivered world-wide. I knew that he was also a Palm PDA user, so I showed him a Palm app that contained some really cool alert tones, and we discussed what made them really good alarm sounds: they did not blend in as background noise, they did not sound like voices or singing, they contained some pure tones of different pitches which would cut through the noise of everyday life. In short, they were alerts, not just sounds. That guy was very cool, and he had a very fun job.
It should be no surprise after hearing my Palm stories, that today I carry an iPhone. It’s everything the Palm aspired to be ten years ago, and a lot more than the Palm never imagined. It should also not be a surprise that I would find it important to install some good non-music ring tones for my iPhone. I was pleased to find that it is pretty easy to put custom ring tones on this device without writing a check to Apple or to AT&T. I dig free, and I really dig open.
On iTunes (we’ll forget about open for a second), I subscribed to a podcast that publishes ring tones. The one that I picked was the MacMost iPhone Ring Tones podcast. Every so often, it dumps a pile of ring tones (m4r files) onto your iPhone.
Some of them were cool, some were trash, and others needed a little bit of work. For example, one of them was a woman’s voice that said “ring ring, ring ring, your iPhone is ringing”. I liked the first half, but I thought the last part was tacky. So I decided to edit that one.
On my Linux machine, I downloaded “X Convert File Audio” (xcfa) and “audacity“. I copied the ring tone from iTunes to my desktop. I changed the file extension from “m4r” to “m4a”, since they really are the same thing, but Apple uses the “r” to distinguish ring tones from regular music files. Then I ran xcfa to convert the file to a common “wav” format. The GUI is a little crude (and some of the text is in French), but it works well enough for a quick conversion. Audacity understands wav files, and so I was able to edit the “your iPhone is ringing” out of my sample, and I cut and pasted until I had a 30-second clip (which worked better than a shorter clip for some reason). Audacity has all of the features you’d want, so you could add echo or reverb or whatever you like. I saved my sound as a wav file, using a new name (and also filling in that new name in the “properties” dialog box that popped up). Then I ran xcfa again to convert the file back to “m4a” format, and renamed it back to “m4r”. Finally, I imported the file back into iTunes.
So that was pretty simple: (1) export from iTunes (2) m4r to m4a (3) m4a to wav (4) edit (5) wav to m4a (6) m4a to m4r (7) import into iTunes.
If you wanted to start with a sound or a song instead of an existing ring tone, you would simply convert it to “wav” format and then continue at step (4).
If you’ll excuse me, my iPhone just farted.