tips & tricks

Every once in a while, I learn something new and magical that SSH can do.  For a while now, I have been using its “dynamic port forwarding” capability, and I wanted to share that here.

Dynamic port forwarding is a way to create a SOCKS proxy that listens on your local machine, and forwards all traffic through the SSH tunnel to the remote side, where it will be passed along to the remote network.

What this means is that you can tell your web browser to send some – or all – of your web requests through the tunnel, and the SSH daemon on the other side will send those requests out to the internet for you, and then it will send the responses back through the tunnel to your browser.

There are several practical uses for SSH tunneling.  Here are a few real-world examples of where I have used tunnels:

• You may be behind a firewall that blocks web sites that you want to visit. I was at a company that had a web proxy that blocked some “hacking” web sites, when I really needed to look up some information on computer vulnerabilities, to make sure that their systems were properly patched.  Using the SSH tunnel bypassed that company’s web proxy.
• You may not trust a hotel’s internet service. In fact, you SHOULD NOT trust a hotel’s internet service.  They keep records of sites that you visit, ostensibly to keep a customer profile of your interests.  Some also do deep packet inspection to observe words in the web pages and emails that you read.  Intrusive marketing!
• You may want to work from home, and access work lab resources from home. If you can SSH into one machine at work, then you can access the web resources inside that network.
• Working on a cluster of machines, where only one of them is directly accessible from your desk. It stinks to have to do your work in a noisy lab.  Why not shell into the gateway machine and tunnel your web traffic through that SSH connection?
• You may want to access a web interface to something at home, like a printer… or an oven. You should open the SSH service to your router, and then you can SSH in and easily access web interfaces to everything in your home.
• You may want a web interface on some service on an internet-facing machine, but you don’t want to expose it to everyone. Maybe you want phpMyAdmin on a server, but you don’t want everyone to see it. So run the web service on the local (127.0.0.1) network only, and use an SSH tunnel to get to it from wherever you are.
• Your ISP throttles web traffic from a particular site. In my case, it was an overseas movie streaming site, and my ISP would limit my traffic to 300kbps for 30 seconds, followed by nothing for 30 seconds (they do this to discourage you from watching movies over the internet).  Sending the traffic through an SSH tunnel means the ISP only sees an encrypted stream of something on port 22 — they have no idea that the content is actually a movie streamed from China on port 80.
• You may live in India, but want to watch a show on Hulu (which only serves the US market).  By tunneling through a US-based server, it will appear to Hulu that you’re in the US.
• You may be visiting China, and you want to bypass the Great Firewall of China. Believe it or not, my humble domain was not accessible directly from Shanghai.
• You may get a “better” route by going through a proxy. This is speculation on my part… but I did this once when I was in a free-for-all online ticket purchase.  I wondered if I might get a faster route from my remote data center than I would get directly from my ISP.

So, how do you create a tunnel?  It’s super easy.

Create an SSH session with a tunnel.

On Linux or a Mac, open a shell and type “ssh -D 10000 username@remotehost.com“.

On Windows*, you can run Putty (an SSH client).  In the “connection / SSH / tunnels” menu, click on “dynamic”, then enter source port = 10000, then click “add”.  You’ll see a “D10000” in the box that lists your forwarded ports.  (* Putty is also available for Linux.)

That’s it… you’re done.

Tell the browser to use the tunnel.

The easiest way to get started with SOCKS proxies is to set it in the “preferences / advanced / network / connection settings” menu in Firefox (or the “preferences / under the hood / change proxy settings” menu in Google Chrome).  In both cases, you want to set “manual proxy configuration” and SOCKS host = “localhost” and SOCKS port = 10000.  If there is a selection for SOCKS v5, then check it.

Now, all web requests will go to a proxy that is running on your local machine on port 10000.  If you do a “netstat -plnt | grep 10000”, you’ll see that your SSH or Putty session is listening on port 10000.  SSH then forwards the web request to the remote machine, where SSHD will send it on out to the internet.  Web responses will come back along that same path.

Anyone on your local network will only be able to see one connection coming from your machine: an encrypted SSH session to the remote.  They will not see the web requests or responses.  This makes it a great tool for using on public networks, like coffee shops or libraries.

Test it

Point your browser to a site that tells you your IP address, such as whatismyip.com.  It should report the IP address of the remote server that you are connecting through, and not your local IP address.

Advanced – quickly selecting proxies using FoxyProxy or Proxy Switchy!

It can be a hassle to go into the preferences menu to turn the proxy on and off.  So there are browser plug-ins that allow you to quickly switch between a direct connection and using a SOCKS proxy.

For Firefox, there is the excellent “FoxyProxy“.  For Chrome, you’ll want “Proxy Switchy!“.  Both plug-in’s allow you to select a proxy using a button on the main browser window.

But there’s more… they also allow you build a list of rules to automatically switch based on the URL that you are browsing.  For example, at work, I set up a rule to route all web requests for the lab network through my SSH session that goes to the lab gateway machine.

It works like this.  First, I set up a tunnel to the lab gateway machine:

• ssh -D 10001 alan@192.168.100.1 (lab gateway)

Then I set up the following FoxyProxy rules:

• 192.168.100.* – SOCKS localhost:10001 (SSH to lab)
• anything else – direct

Now, I can get to the lab machines using the lab proxy, and any other site that I access uses the normal corporate network.

Closing thoughts

If you like Putty or FoxyProxy or Proxy Switchy!, please consider donating to the projects to support and encourage the developers.

On internet-facing machines, I like to disable the root password so that you can only log in as root using an SSH key.  This is done by setting the seemingly-scary option PermitRootLogin without-password in /etc/ssh/sshd_config .  This option means that you can ONLY use a key to log in as root… a password will never be accepted, and so you can not guess it by trial-and-error.

While I am at it, I go ahead and disable the root password completely.  That’ll be one less thing for me to remember, and one less thing to keep secret.

First, be sure that you have at least one “sudoer” user, or at least one SSH key in ~root/.ssh/authorized_keys . Otherwise, you’ll realize in an “ignisecond” that you’ve just locked yourself out.

Then, use the passwd command to “lock” the account.

$ passwd -l root

This command will put a ! character in the password field of the /etc/shadow file so that no password hash will ever match the string in the shadow file.

But on SOME SYSTEMS, it also does a second thing — it may change the account expiration date to January 2, 1970. This will prevent SSH access, even with a key.

While researching this article, I had this “world-shifting-under-me” feeling, as I distinctly remember having to work around this issue. However, on every system I tried, I could not reproduce the expiration-date-changing behavior. Then I found that it was an issue that flip-flopped in the Debian community between 2006 and 2008. In 2006, they made the passwd -l command lock the password and also expire the user account. But in 2008, they decided that the change affected too many people, where most had grown accustomed to the passwd command affecting the password only, and not touching the account expiration date.

If you want to change the account’s expiration date, you should use the chage command.

$ chage -E -1 root  # never expire
$ chage -E 1 root  # expire on Jan 2, 1970
$ chage -E 0 root  # not recommended, undefined behavior
$ chage -E 2010-12-25  # expire on a specific date

So here I was, all set to share a nugget of wisdom with the world, and instead, it ends up being a trip down memory lane. However, in the process, I learned a couple of things.

A very safe way to disable a user account’s password, while keeping the account open to SSH access is like this: passwd -l user && chage -E -1 user . The chage part is unnecessary on modern systems, but it does not hurt anything.

A quick way to check on whether a password is locked, or a user account is expired, is to use passwd -S user, like this.

$ passwd -S user
user P 09/11/2007 0 99999 7 -1

This says that user‘s password is set (P), it was changed way back 2007, there is no minimum age (restriction on how often they can change their password), there’s a very long maximum age (time when they are forced to change their password), the warning period is one week, and the account is not inactive/expired.

So there you go, two for one, a Linux tip and a history lesson!

When I take pictures with my digital cameras, they name the image files something like this:

• Wife’s Nikon camera – dscn0115.jpg
• My iPhone – img_0367.jpg
• My Panasonic camera – p1070126.jpg

I tend to let the images pile up on the cameras for a while, and then I copy them all onto our file server. I start off by dumping them all into one big folder. But then I sort them into folders based on the date and the event, with names like “pictures/y2010/2010-02-14_chinese_new_year” for events and “pictures/y2010/2010-02” for the random shots. I don’t mind this process, and it’s actually kind of fun to review them as I am copying them to our file server.

I should note as an aside, when I worked for Ericsson’s research lab in Singapore, I was talking to one of the researchers who had studied the many ways that people could organize and categorize photographs, both paper and digital. It turns out that a huge majority of people they studied tended to associate photo sets together, based on relative dates. That is, if you asked for a particular photo, they would think “that was about the same time as Bob’s birthday party, so it must have been in August”.

I know that there are newer ways to organize photos, with databases and “tags” and what-not. Mac people really love to let the computer take care of those details. But I still like the idea of using folders with dates… old school.

Since I am dealing with dates and times of photos, it seems a little silly that all of my photos are named using dumb serial numbers. I find myself looking at the image’s EXIF properties, the information that the camera stores about the image — when and where the image was taken, what the camera settings were, etc. This seems a little tedious.

I recently found a utility called “renrot”. It’s primary job is to read the EXIF data from a photo, and rotate the image to match the EXIF rotation flag. That is, it rewrites the image so that it will load from top-to-bottom, which makes it more compatible with less-than-intelligent viewers — like some digital picture frames. But while it’s doing that, it also renames the file based on the time and date of when the photo was taken.

So now I can start by going into the directory that contains my big pile of photos and doing this:

renrot --name-template %Y%m%d-%H%M%S --extension jpg *.JPG

If I wanted to be careful not to mix the iPhone pictures with the Nikon pictures, I could do this:

renrot --name-template iphone-%Y%m%d-%H%M%S --extension jpg IMG_*.JPG
renrot --name-template nikon-%Y%m%d-%H%M%S --extension jpg DSCN*.JPG
renrot --name-template pan-%Y%m%d-%H%M%S --extension jpg P*.JPG

If I want to be a purist, and just rename the photos without actually rotating the image, I can do that, too.

renrot --no-rotate --name-template %Y%m%d-%H%M%S --extension jpg *.JPG

Pretty cool.

I have been taking digital photographs since 1998, and so one piece of software which I use quite a bit is my photo organizer. I use a tool called “gqview”. It’s a very nice browser that shows folders and thumbnails and a large image on the screen at the same time. And it does a good job showing slide shows.

One nice feature of gqview is that you can plug external programs into the ‘edit’ menu. When you install it (at least on Ubuntu systems), a few of the plug-in’s are populated already:

• edit in GIMP (external package: gimp)
• edit in xpaint (external package: xpaint)
• rotate clockwise (external package: libjpeg-progs)
• rotate counter-clockwise (external package: libjpeg-progs)

I wanted to add a plug-in to shrink images a certain percentage. I often find that I want to send a bunch of pictures to friends via email, and I really don’t want to send a 50 MB email, and I don’t want the pictures to be so big that the reader had to scroll to see it.

It’s pretty easy to add the plug-in. The actual shrinking can be handled easily with the ‘convert’ command, which is part of the excellent ‘imagemagick’ set of graphics tools. Plugging convert into gqview is as easy as this:

%vif convert %p -scale 25% %p_tmp ; then
    mv %p_tmp %p ; else rm %p_tmp ; fi

I didn’t make this up myself — I simply copied the “rotate” plug-in’s and changed what I needed to. I am not sure what the %v is for, but it’s pretty clear that %p is the full path of the image file.

Now I can highlight a whole bunch of photos and shrink them all with a single command.