{"id":65,"date":"2009-09-19T11:24:59","date_gmt":"2009-09-19T16:24:59","guid":{"rendered":"http:\/\/blog.alanporter.com\/2009-09-19\/back-to-the-future"},"modified":"2009-09-19T11:24:59","modified_gmt":"2009-09-19T16:24:59","slug":"back-to-the-future","status":"publish","type":"post","link":"https:\/\/alanporter.com\/blog\/2009\/09\/19\/back-to-the-future\/","title":{"rendered":"Back to the Future"},"content":{"rendered":"<p>A few days ago, I learned a very important lesson about filesystems and snapshots.  I learned that a <strong>complete copy<\/strong> is not always a <em>Good Thing\u2122<\/em>.<\/p>\n<p>I help manage a server for our local Linux Users Group.  We have about 250 users on the system, and all of our system administration is done by volunteers.<\/p>\n<p>A few months ago, I made a complete backup of our <code>\/home<\/code> partition using the guidelines that have been told to me by <em>Smart People\u2122<\/em>:<\/p>\n<ul>\n<li>make a snapshot volume of \/home (called <code>home-snap<\/code>)<\/li>\n<li>make a new empty volume (called <code>home-backup<\/code>)<\/li>\n<li>use &#8216;<code>dd<\/code>&#8216; to copy from <code>home-snap<\/code> to <code>home-backup<\/code><\/li>\n<li>remove the <code>home-snap<\/code> snapshot volume<\/li>\n<\/ul>\n<p>All was fine, until a few months later, when we decided to reboot.<\/p>\n<p>When the machine rebooted, it mounted the WRONG copy of <code>\/home<\/code>.  It looked  in <code>\/etc\/fstab<\/code> to see what to mount, read the UUID, and started looking for that filesystem among the logical volumes.<\/p>\n<p>Here&#8217;s a list of the available filesystems and their UUID&#8217;s.<\/p>\n<pre>root@pilot:~# blkid\n\/dev\/mapper\/vg01-home: UUID=\"1a578e6f-772b-4892-86e3-1181aadda119\" TYPE=\"ext3\" SEC_TYPE=\"ext2\"\n\/dev\/mapper\/vg01-home-backup: UUID=\"1a578e6f-772b-4892-86e3-1181aadda119\" TYPE=\"ext3\" SEC_TYPE=\"ext2\"\n\/dev\/mapper\/vg01-swap: TYPE=\"swap\" UUID=\"303f2743-da69-466b-a200-40a1a369fa1c\"\n\/dev\/mapper\/vg01-u804: UUID=\"b5689a93-b7ad-4011-a0f9-ffaf2d68bf6f\" TYPE=\"ext3\"\n\/dev\/sdb: UUID=\"Uh0TI1-pxD4-M1Pm-5kP3-zU1a-IRgm-bD0JAq\" TYPE=\"lvm2pv\"\n\/dev\/sda: UUID=\"9oZhBo-3DPP-1eay-kgGM-fd06-yuJB-c2eCo7\" TYPE=\"lvm2pv\"\n\/dev\/sdc1: UUID=\"5c15308e-a81b-4fd9-b2c2-7ef3fe39ce0b\" SEC_TYPE=\"ext2\" TYPE=\"ext3\"\n\/dev\/sdc2: TYPE=\"swap\" UUID=\"08c55fa5-3379-4f6a-b798-4b8f3ead6790\"\n\/dev\/sdc3: UUID=\"5a544a7f-90ed-474c-b096-1b5929c83109\" SEC_TYPE=\"ext2\" TYPE=\"ext3\"\nroot@pilot:~#<\/pre>\n<p>Notice anything goofy?  Yes, the UUID for the <code>home<\/code> volume is the same as the UUID for the <code>home-backup<\/code> volume!  Of course it is&#8230; I used &#8216;<code>dd<\/code>&#8216; to copy the entire volume!<\/p>\n<p>So our machine booted up, looked for a filesystem whose UUID was &#8216;1a578e6f-772b-4892-86e3-1181aadda119&#8217; and it mounted it on <code>\/home<\/code>.   Unfortunately, it found the <code>home-backup<\/code> volume before it found the real <code>home<\/code> volume, and so our 250 users took a step back in time for the evening.<\/p>\n<p>All of the files in our home directories looked like they did back in May.<\/p>\n<p>On the surface, this does not seem like such a <em>Bad Thing\u2122<\/em>.  But over the course of the next few hours, users started receiving email, and logging IRC chats, and doing all of the other things that users do.  These new emails and log files were written to <code>home-backup<\/code> instead of <code>home<\/code>, and so now we were starting to mix old and new files.<\/p>\n<p>This is a lot like the movie &#8220;Back to the Future&#8221;, when Marty&#8217;s mom tries to kiss him.  Except the characters involved here are not as good-looking.<\/p>\n<p>The fix was quick and painless.  I simply generated a new UUID for the <code>home-backup<\/code> volume, and then rebooted.  The magic command is simply:<\/p>\n<pre> tune2fs -U random \/dev\/mapper\/vg01-home-backup<\/pre>\n<p>But the cleanup would come later.  If someone were interested in the emails or log files that were mistakenly written to the wrong volume (their &#8220;past life&#8221;), then they would need to look on that volume for &#8220;new&#8221; files.  Pretty easy work.<\/p>\n<pre>find \/mnt\/home-backup\/porter -mtime -7<\/pre>\n<p>This will show all files in my &#8220;backup&#8221; home directory that are less than a week old.  Since the backup was made four months ago, I would expect all files in that directory to either be more than four months old, or just one day old.  This command will show you the new files.<\/p>\n<p>So I am revising the backup procedure as follows:<\/p>\n<ul>\n<li>make a snapshot volume of \/home (called <code>home-snap<\/code>)<\/li>\n<li>make a new empty volume (called <code>home-backup<\/code>)<\/li>\n<li>use &#8216;<code>dd<\/code>&#8216; to copy from <code>home-snap<\/code> to <code>home-backup<\/code><\/li>\n<li>remove the <code>home-snap<\/code> snapshot volume<\/li>\n<li><strong>change the UUID on <code>home-backup<\/code><\/strong> \u25c4\u2014 new<\/li>\n<\/ul>\n<p>In fact, now that we already have a base to work with, I might just use <code>rsync<\/code> to copy <strong>files<\/strong> instead of <code>dd<\/code> to copy the entire volume.  This will leave the backup with its own UUID, and will avoid collisions like the one we saw.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A few days ago, I learned a very important lesson about filesystems and snapshots. I learned that a complete copy is not always a Good Thing\u2122. I help manage a server for our local Linux Users Group. We have about 250 users on the system, and all of our system administration is done by volunteers. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,15,16],"tags":[],"class_list":["post-65","post","type-post","status-publish","format-standard","hentry","category-software","category-tips-tricks","category-trilug","count-0","even alt","author-alan","last"],"_links":{"self":[{"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/posts\/65","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/comments?post=65"}],"version-history":[{"count":0,"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/posts\/65\/revisions"}],"wp:attachment":[{"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/media?parent=65"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/categories?post=65"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/alanporter.com\/blog\/wp-json\/wp\/v2\/tags?post=65"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}