tips & tricks
Plugging external commands into ‘gqview’
0I have been taking digital photographs since 1998, and so one piece of software which I use quite a bit is my photo organizer. I use a tool called “gqview”. It’s a very nice browser that shows folders and thumbnails and a large image on the screen at the same time. And it does a good job showing slide shows.
One nice feature of gqview is that you can plug external programs into the ‘edit’ menu. When you install it (at least on Ubuntu systems), a few of the plug-in’s are populated already:
- edit in GIMP (external package: gimp)
- edit in xpaint (external package: xpaint)
- rotate clockwise (external package: libjpeg-progs)
- rotate counter-clockwise (external package: libjpeg-progs)
I wanted to add a plug-in to shrink images a certain percentage. I often find that I want to send a bunch of pictures to friends via email, and I really don’t want to send a 50 MB email, and I don’t want the pictures to be so big that the reader had to scroll to see it.
It’s pretty easy to add the plug-in. The actual shrinking can be handled easily with the ‘convert’ command, which is part of the excellent ‘imagemagick’ set of graphics tools. Plugging convert into gqview is as easy as this:
%vif convert %p -scale 25% %p_tmp ; then
mv %p_tmp %p ; else rm %p_tmp ; fi
I didn’t make this up myself — I simply copied the “rotate” plug-in’s and changed what I needed to. I am not sure what the %v is for, but it’s pretty clear that %p is the full path of the image file.
Now I can highlight a whole bunch of photos and shrink them all with a single command.
Surgical web page editing with “stylish”
3Have you noticed that some web sites have ads that are somewhat… unpredictable? Facebook is a good example of this. The ads that appear on the right can sometimes contain trashy-looking content. I’d like to hide them, if I could.
I found a Firefox plugin that does a good job of slicing out selected bits of content. It’s called “stylish“.
Just install the plugin, and then set up a script that intercepts the stylesheet for a particular site and does a little on-the-fly modification. The script I use for Facebook looks like this:
@namespace url(http://www.w3.org/1999/xhtml);
@-moz-document domain("facebook.com") {
.emu_ad, .UIStandardFrame_SidebarAds, #home_sponsor {
display: none !important;
}
}
I don’t claim to fully understand the script, but I do know that it looks for a specific block in the Facebook HTML and style sheet, and it sets the style for the sidebar ad box to “display: none”. That makes it disappear.
Back to the Future
2A few days ago, I learned a very important lesson about filesystems and snapshots. I learned that a complete copy is not always a Good Thing™.
I help manage a server for our local Linux Users Group. We have about 250 users on the system, and all of our system administration is done by volunteers.
A few months ago, I made a complete backup of our /home partition using the guidelines that have been told to me by Smart People™:
- make a snapshot volume of /home (called
home-snap) - make a new empty volume (called
home-backup) - use ‘
dd‘ to copy fromhome-snaptohome-backup - remove the
home-snapsnapshot volume
All was fine, until a few months later, when we decided to reboot.
When the machine rebooted, it mounted the WRONG copy of /home. It looked in /etc/fstab to see what to mount, read the UUID, and started looking for that filesystem among the logical volumes.
Here’s a list of the available filesystems and their UUID’s.
root@pilot:~# blkid /dev/mapper/vg01-home: UUID="1a578e6f-772b-4892-86e3-1181aadda119" TYPE="ext3" SEC_TYPE="ext2" /dev/mapper/vg01-home-backup: UUID="1a578e6f-772b-4892-86e3-1181aadda119" TYPE="ext3" SEC_TYPE="ext2" /dev/mapper/vg01-swap: TYPE="swap" UUID="303f2743-da69-466b-a200-40a1a369fa1c" /dev/mapper/vg01-u804: UUID="b5689a93-b7ad-4011-a0f9-ffaf2d68bf6f" TYPE="ext3" /dev/sdb: UUID="Uh0TI1-pxD4-M1Pm-5kP3-zU1a-IRgm-bD0JAq" TYPE="lvm2pv" /dev/sda: UUID="9oZhBo-3DPP-1eay-kgGM-fd06-yuJB-c2eCo7" TYPE="lvm2pv" /dev/sdc1: UUID="5c15308e-a81b-4fd9-b2c2-7ef3fe39ce0b" SEC_TYPE="ext2" TYPE="ext3" /dev/sdc2: TYPE="swap" UUID="08c55fa5-3379-4f6a-b798-4b8f3ead6790" /dev/sdc3: UUID="5a544a7f-90ed-474c-b096-1b5929c83109" SEC_TYPE="ext2" TYPE="ext3" root@pilot:~#
Notice anything goofy? Yes, the UUID for the home volume is the same as the UUID for the home-backup volume! Of course it is… I used ‘dd‘ to copy the entire volume!
So our machine booted up, looked for a filesystem whose UUID was ‘1a578e6f-772b-4892-86e3-1181aadda119’ and it mounted it on /home. Unfortunately, it found the home-backup volume before it found the real home volume, and so our 250 users took a step back in time for the evening.
All of the files in our home directories looked like they did back in May.
On the surface, this does not seem like such a Bad Thing™. But over the course of the next few hours, users started receiving email, and logging IRC chats, and doing all of the other things that users do. These new emails and log files were written to home-backup instead of home, and so now we were starting to mix old and new files.
This is a lot like the movie “Back to the Future”, when Marty’s mom tries to kiss him. Except the characters involved here are not as good-looking.
The fix was quick and painless. I simply generated a new UUID for the home-backup volume, and then rebooted. The magic command is simply:
tune2fs -U random /dev/mapper/vg01-home-backup
But the cleanup would come later. If someone were interested in the emails or log files that were mistakenly written to the wrong volume (their “past life”), then they would need to look on that volume for “new” files. Pretty easy work.
find /mnt/home-backup/porter -mtime -7
This will show all files in my “backup” home directory that are less than a week old. Since the backup was made four months ago, I would expect all files in that directory to either be more than four months old, or just one day old. This command will show you the new files.
So I am revising the backup procedure as follows:
- make a snapshot volume of /home (called
home-snap) - make a new empty volume (called
home-backup) - use ‘
dd‘ to copy fromhome-snaptohome-backup - remove the
home-snapsnapshot volume - change the UUID on
home-backup◄— new
In fact, now that we already have a base to work with, I might just use rsync to copy files instead of dd to copy the entire volume. This will leave the backup with its own UUID, and will avoid collisions like the one we saw.
iPhone VPN
2One of the coolest features of the iPhone is the way it uses the best data network that it can find. If you’re at home or at work, or even at a coffee shop, it will use the local wifi network. But if you’re out of range of any suitable wifi networks, it will use AT&T’s “3G” (UMTS) network. And if it can’t find a UMTS network, it’ll fall back to EDGE. Phone companies call this hybrid approach “ABC”, or “always best connection”.
Now that I have an internet device in my pocket, I find myself using public (or otherwise open) wifi connections quite a bit. And this carries with it some unintended consequences. That is… everything I type and everything I read is transmitted in the clear, unencrypted.
I try to make a habit of encrypting my data traffic whenever possible. My mail server is set up to only allow SSL connections. So no matter where I check my mail from, I am forced to use an encrypted connection. Similarly, banks and commerce web sites usually force you to switch to HTTPS before you start entering information. But there are a lot of applications on the iPhone that do not use encryption at all.
You might ask yourself why bother to encrypt your Twitter connection, since what you type is going to be blasted out to the world anyway. But the point is…
If you encrypt everything, then nothing is left to chance.
So I decided to explore a VPN option on the iPhone. It supports three flavors of VPN: L2TP, PPTP and IPSec. I was disappointed (but not surprised) that “openvpn” was not an option, since I already use this excellent open source SSL-based VPN package.
So I decided to give PPTP a try.
Setting up the PPTP server
On my Ubuntu 8.04 LTS server, I installed a PPTP server called, appropriately enough, “pptpd“. Configuration was very easy. Most of the setup was done for me after I did the standard apt-get install pptpd. I simply needed to pick a private subnet that would be used for my VPN clients, and an IP address in that subnet to use for the server. I chose the 172.16.4.0/16 subnet and 172.16.4.1 for the server (these addresses are part of a private network address space, defined by RFC 1918, just like 192.168.x.x and 10.x.x.x addresses).
My /etc/pptp.conf configuration file for the pptp daemon looks like this:
option /etc/ppp/pptpd-options logwtmp localip 172.16.4.1 remoteip 172.16.4.2-250
I also needed to tell the daemon to give out some DNS addresses when a client connects, so in the /etc/ppp/pptpd-options file, I added the two “ms-dns” lines below:
name pptpd refuse-pap refuse-chap refuse-mschap require-mschap-v2 require-mppe-128 ms-dns 208.67.222.222 # resolver1.opendns.com ms-dns 208.67.220.220 # resolver2.opendns.com proxyarp nodefaultroute lock nobsdcomp
Finally, I needed to add an entry into the /etc/ppp/chap-secrets file that would contain my password. Mine looks like this:
alan pptpd MyHardToGuessPassword *
At this point, the PPTP server was completely configured, so I restarted it with service pptpd restart.
Setting up the iPhone
On the iPhone, I needed to set up a VPN client. This is very easy. On the settings screen, go to general / network / VPN and “Add VPN Configuration…”. Then just fill in the blanks.
- choose “PPTP”
- enter a description
- your server’s IP address
- the username (from above)
- RSA SecurID=OFF
- the password (from above)
- encryption level = Auto
- “Send All Traffic” = ON
- Proxy = OFF
Click on “Save” and you will see a switch in the network tab and also in the main settings tab to turn the VPN on and off.
For now, I am leaving it off unless I am on a public network. I am not sure, but I think that keeping the VPN alive might use a lot of battery. So I do not use it unless I need it.
Networking
For me to get this VPN on the internet, I had to do two more things: punch a hole in my firewall for the PPTP traffic, and forward traffic from my VPN out to the rest of the world.
For my server, both of these tasks were handed by the same tool: shorewall.
I added a “masquerade” rule to /etc/shorewall/masq to NAT all of the traffic from 172.16.4.x out through my main network interface.
eth0 172.16.4.0/24 # OpenVPN and PPTP
And then I added two rules to /etc/shorewall/rules to allow the PPTP traffic in.
ACCEPT net fw tcp 1723 # PPTP ACCEPT net fw gre # PPTP
When shorewall starts, it will generate the iptables rules that are used by the kernel to filter packets. If you’re using hand-written iptables rules, then you will need some rules that look something like this:
# accept "gre" protocol traffic (PPTP tunnel traffic) iptables -A INPUT -p gre -j ACCEPT iptables -A OUTPUT -p gre -j ACCEPT # accept PPTP control traffic to TCP port 1723 # (my server IP is 11.22.33.44) iptables -A INPUT -p tcp --sport 1723 -s 11.22.33.44 -j ACCEPT iptables -A OUTPUT -p tcp --dport 1723 -d 11.22.33.44 -j ACCEPT # masquerade/NAT internet traffic out of interface eth0 iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE # turn on packet forwarding echo "1" > /proc/sys/net/ipv4/ip_forward
Conclusion
Now, when I am in a coffee shop, I can turn on the VPN easily by flipping the switch in the iPhone’s main settings screen. It will make a TCP connection to my server, negotiate a few things, and then send all further network traffic through an encrypted tunnel directly to my server, which relays it out to the internet.
You can test this by going to a web site like http://www.whatismyip.com/. If the VPN is working, it will show your server’s IP address. If not, it’ll show the coffee shop’s IP address.
Once I am using the VPN, anyone in the coffee shop who happens to be sniffing traffic on the wireless network will only see a single connection from my iPhone to my server, but the contents of this connection will be scrambled.
Plant my key
4Here’s a neat little script that I developed at “the oven place”. It “plants” my SSH key on a remote machine, so I can get in later without entering a password.
#!/bin/sh
user_at_machine=$1
# plant my SSH key on the target machine
cat $HOME/.ssh/id_rsa.pub | \
ssh $user_at_machine \
"if [ ! -d .ssh ] ; then mkdir -m 700 .ssh ; fi ; \
cat >> .ssh/authorized_keys"
All of the action takes place in one single line (which I broke up here so it would wrap OK). It takes your public key from the machine you’re on, and it pipes it into an SSH session, which you’ll have to type your password into. On the remote machine, it creates a .ssh directory if it needs to, and then it appends your public key to an authorized_keys file.
The next time you log in to that remote machine, you will not need to enter your password.
Counting down with ‘pv’
0Here’s a neat tool.
Ever start some long task, and wonder how much longer it has to go? There’s a small utility called pv (short for “pipe viewer”) that counts time and bytes through a pipe, and it shows a nice progress bar.
Here’s how I used it to watch a very large file being compressed (note the use of the most excellent lzma compression utility.
$ pv < winxp.vdi | lzma > winxp.vdi.lzma 1.79GB 0:21:50 [1.49MB/s] [===> ] 10% ETA 3:11:02
Note that if I had used a different pipe notation, pv would not have been able to read the input file size, and therefore it could not make estimates of remaining time. So instead, it shows you the “Knight Rider” scanning eye for progress.
$ cat winxp.vdi | pv | lzma > winxp.vdi.lzma 5.48MB 0:00:04 [1.27MB/s] [ <=> ]
Nice tool. Thanks, Andrew!
Save that thought…
1At work, our build process can produce thousands of lines of code, and sometimes the important bits can slide off of the terminal’s scrollback buffer.
So I created an alias that logs all output of the shell. It’s nothing fancy… it’s just a call to script with an argument to put the time and date in the filename. But it goes a step further, actually exec-ing the script command, so you do not have to exit twice when you’re done (once to exit the script, and once to exit the original shell).
alias log='exec script "/home/alan/logs/terminal-$(date +%Y%m%d-%H%M%S)-$$.log"'
Still, this requires you to think about logging before you start your work.
Since I normally launch my shell windows from a Gnome application launcher icon, I decided to modify that launcher so that EVERYTHING is logged.
On Gnome, right-click on the panel at the top and select “+ Add to panel…”. Then select “application launcher” and pick out the gnome-terminal from the menus. You’ll have a little terminal icon on the top Gnome panel.
Right-click on that terminal icon and select “properties”. It should say “gnome-terminal”. Change it to the following:
sh -c "exec gnome-terminal -x script $HOME/logs/terminal-$(date +%%Y%%m%%d-%%H%%M%%S)-$$.log"
The double-percents are so the launcher will not interpret them.
Now, when you click on that application launcher icon, a new terminal window will open, and all output to that terminal will be logged.
Firefox: invalid or unsupported form of compression
0Occasionally, I have been seeing the following error in Firefox (v3.0.11).
The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.
It occurs frequently when I am using a SOCKS proxy. One web site which happens to exhibit the problem more than others is Bruce Schneier’s blog.
The work-around that seems to work for me is to do a “Tools / Clear Private Data…” and select Cache, Cookies and Offline Website Data.
There’s a lively discussion on the Mozilla support forum, but no one seems to know what the source of the problem is.
Two heads are better than one
0At work, I got a new PC with two monitors, and I have really enjoyed having the extra screen real estate.
When I worked from home this week, I decided that I needed to try hooking an external monitor to my HP laptop. I was happy that, unlike the last time I tried this a few years ago, it “Just Worked” ™ without any special calesthenics on my part.
I wondered why Gnome decided to put the panels (the bars at the top and the bottom) on the external monitor instead of on the laptop display. To me, it seemed more natural to have the “start” menu (actually, the “Applications, Places, System” tri-menu) on the external display instead of right in front of me.
It turns out that this is really easy to fix. Gnome has this configuration tool that is roughly equivalent to the Windows registry… yuck. You can access it using either gconf-editor or gconftool-2. Inside that deep mine of settings, there is one jewel that tells where the panels should go. So I wrote a quick script to move the panels from one monitor to the other and back.
#!/bin/bash
m=$1
if [[ ( ! -z "$m" ) && ( ( $m -eq 1 ) || ( $m -eq 0 ) ) ]]
then
menu="/apps/panel/toplevels"
panels=$(gconftool-2 --dump $menu | \
grep '<key>.*/monitor</key>' | \
sed -e 's/^.*<key>//g' -e 's/</key>.*$//g')
for p in $panels
do
gconftool-2 --set "$menu/$p" --type integer "$m"
done
else
echo "usage --> $0 [ 0 | 1 ]"
echo "moves gnome panels to monitor 0 (VGA) or 1 (LCD)"
fi
exit 0
The Quick Lounge
0In my line of work, I occasionally have the luxury of working from home. When that happens, I prefer to run all of my applications on my desktop PC at the office, and I use “NX” to connect remotely. Specifically, I use the NoMachine NX server on my Ubuntu PC at work (although I am considering trying the open source “freenx” server), and at home I use the “qtnx” client on my laptop, which also runs Ubuntu.
This works pretty well, except my desktop environment does not like the stress of changing resolutions back and forth. It seems that the “panels” in Gnome get confused about where to place the different applets and widgets, since at one time it may have a lot of room, and at another time it may be more cramped.
I got tired of scooting my quick-launch icons around one-by-one, and so I was looking for a container where I could keep them together and move them around as a group. I tried the “drawer” applet, which is available in Ubuntu by default, but that was not quite right. I don’t want to open and close the drawer every time I launch something… I just want the icons to always be there in a group.
I found an applet called “quick-lounge-applet” which really fits the bill. One wonders whether the author meant to say “quick launch”, but perhaps had a poor grasp of English. Either way, this little applet does a great job of keeping my quick-launch icons together in a group, and it can be moved around easily.
After installing the applet (using the normal apt-get install quick-lounge-applet, I found that it was not listed in the “+ Add to panel…” menu. Apparently, Gnome needs to be prodded before it recognises newly-installed applets. There is a simple work-around. Simply re-start the service that keeps track of that stuff: killall bonobo-activation-server. The service will re-start, and there will be a new entry in the “+ Add to panel…” menu called “Launchers List”.
Now, if you’ll excuse me, I am in a hurry. I have some quick lounging to do.