The Art of Tech

Ed: This post was originally written on 2006/08/03, but for some reason I missed publishing it. As I believe the information is useful, I'm publishing it now.

On my home workstation, I used LVM for the one 160GB hard drive. When I first built the box (in May of 2004), I installed Fedora Core 2 for x86_64; it's a dual AMD Opteron 242 (1.6GHz). I created four partitions:

1. 100MB /boot/
2. 2GB swap (there's 1GB RAM, but I can expand to 8GB)
3. remainder (~96GB) LVM
4. 60GB Windows XP Professional

For a couple of weeks, I have been unable to install any updates to FC5. When I run yum update, I get a series of error messages telling me that yum needs additional disk space on the root partition in order to install packages. Strangely enough, almost none of those packages have files that end up in the root partition.

The root filesystem was a 512MB ext3 LVM logical volume. Sure, I could make it bigger, but I shouldn't have to. In fact, I should be able to have it be only about 256MB and still have a completely usable system. I have /home/, /tmp/, /usr/, /var/ and even /opt/ all split off onto their own LVs, so there's not that much stuff left on the root volume. In fact, running du -sx / shows that there is just barely 200MB of data present. However, running df / shows that it is 99% full (about 508MB used). Clearly, ext3 is not an efficient choice for a root partition when the bulky filesystems are split out like this.

So, I decided to "convert" to another filesystem type. I chose to use XFS for the root volume. Converting the root partition can be a little bit tricky, as the initrd needs to have the right drivers to work with it. Here's my step-by-step (assuming that the volume group is named system):

1. Create a new volume and format it with the new filesystem type (the size could be smaller, if desired):

   # lvcreate -L 512MB -n newroot system
   # mkfs.xfs /dev/system/newroot
   

2. Mount the new root volume and remount the current root volume read-only:

   # mkdir /mnt/temp
   # mount /dev/system/newroot /mnt/temp/
   # mount -o remount,ro /
   

3. Copy the contents of just the root volume to the new root volume (the -x switch is very important):

   # cp -ax /* /mnt/temp/
   

4. Unmount the new root volume:

   # umount /mnt/temp/
   

5. Run uname -r and record the value. This will be needed later when running mkinitrd.
6. Reboot the system into a rescue environment. In order to be able to boot with the new root partition, it is necessary to recreate the initrd. You could hand edit the existing one(s) to fix things up, but this will be quite a bit easier. Use a CD or a network bootable rescue environment for your distro (FC5 in my case).
7. If you let the setup for the rescue environment mount your partitions/volumes, that's fine, just make sure to unmount everything so you can switch root partitions. If you don't let it mount up the parts, that's fine too; just use the LVM commands for your rescue environment to activate LVM.
8. Rename the old root volume out of the way and rename the new one into place (this only works while the volumes are not mounted):

   # lvrename system root oldroot
   # lvrename system newroot root
   

9. Mount the (new) root volume and at least /boot/ and /usr/ on top of that:

# mkdir /mnt/system/
# mount /dev/system/root /mnt/system/
# mount /dev/sda1 /mnt/system/boot/
# mount /dev/system/usr /mnt/system/usr/

10. It's time to recreate the initrd. This is easiest when done chroot'ed into the system environment found on your hard drive (assuming your new root volume was mounted at /mnt/system/):

    # chroot /mnt/system/
    

11. Run mkinitrd. Depending on which distribution is installed, this could require quite a different set of switches than what I'm showing you here. In my case, it was Fedora, so I used the mkinitrd command found in the /sbin/new-kernel-pkg script, which is run as part of the post-install scripts from every Red Hat kernel RPM (just open up /sbin/new-kernel-pkg and find the right command to run, replacing "something" with the correct kernel version string recorded earlier from the uname -r command):

    # /sbin/mkinitrd --allow-missing -f /boot/initrd-2.6.something 2.6.something
    

12. Leave the chroot environment and unmount everything (yes, if you just cleanly exit the rescue environment, it should cleanly unmount everything, but let's just be sure it happens right, shall we?):

    # exit
    # umount /mnt/system/usr/
    # umount /mnt/system/boot/
    # umount /mnt/system/
    

13. Reboot. The system should come up with your root volume in use with a new filesystem type.

That's it. That wasn't so difficult, was it?

The entire thing could be done from within the rescue environment, including copying the contents of the root volume to the new root volume, if you like.

I've had a Network Time Protocol (NTP) server running in my home network for many years. The two workstation dual boot Windows and Linux. When running Linux, the ntpd daemon keeps them in sync with my NTP server. When running Windows, I haven't bothered, as there isn't anything running under windows that cares about time at all, and since they run Linux most of the time.

Today, I decided to figure out how to keep Windows client's clock synchronized with NTP. A quick Google search located an NIST PDF document titled, "Configuring Windows 2000 and Windows XP to use NIST Time Servers." There is also a link in the introductory pages of that document to the freely available download from Microsoft to enable Windows NT systems to be configured as NTP clients.

As it turns out, it's really easy to configure Windows 2000 and Windows XP Professional, as they both have native, out-of-the-box support to run as a NTP clients. So, to boil down the NIST's 12 page document, open a "DOS prompt" command shell as a user with admin rights and run:

> net /setsntp:servername
> net stop w32time
> net start w32time

That's it.

You could also use the Time/Date control panel. When I tried it (first) on one of my boxes, it had the default configuration of using a Microsoft time server already configured. When I changed it in that dialog box to use my time server, it griped that it wouldn't use the time I was providing as the server's stratum was lower than the host's. I wonder what goofy things Microsoft's time server is doing, as my time server is stratum 2. Anyway, I ran those three commands and it successfully switched over to using my NTP server.

You would think that the fact that Windows 2000 and Windows XP have the NTP client built-in would be one of those things that I would have already "just known." Well, I didn't, but now I do.

About three weeks ago, I ran rug update on my notebook (II'm mainly using openSUSE 10.2) and got the Adobe Flashplayer 9 for Linux update installed.

That was easy.

I last checked about 12 days ago, but there wasn't an RPM for Fedora Core 6 (the current release), available yet.

Yesterday, I was working on a new DNS server I was setting up for someone, when I noticed that I hadn't setup rndc on my home DNS server. After fixing that, I added a couple of zones to my home DNS server to have it serve (for now) as the secondary server for some new domains, but the zone transfers weren't working. It was immediately obvious that I needed to alter the firewall configuration to allow the DNS server to initiate zone transfers, so I ended up looking at a couple parts of my firewall configuration.

It was an easy change, but in the course of all of this, I had made a typo that caused me a problem with one of my VLAN connections. When I tried to bring the link down so I could reinitialize it with some fixed configuration, it hung. Running kill -9 didn't work, so I decided to just reboot. It had been several months since my last boot and there was a newer kernel that I wanted to be using, anyway, so it wasn't a big deal.

But then, DHCP (which is on the same box) no longer worked. I double-checked that dhcpd was running, did a little sniffing and discovered that the DHCP segments weren't actually getting past the firewall configuration. This is odd, since a Netfilter firewall configuration that is completely locked down (i.e. a policy of DROP on all built-in chains and no rules) has always still permitted DHCP traffic to get through. However, I decided to add a couple of rules to the firewall, such as:

-A INPUT -p udp --sport 68 --dport 67 -m state --state new -j ACCEPT

With those new DHCP rules loaded up, it started working again. It would appear that the latest kernel for Red Hat Enterprise Linux 4 and CentOS 4 (which that server runs), has made a change in Netfilter so that it now affects DHCP renewals.

Perhaps a little explanation of DHCP is order, here (skip down a little if you already know this part, in order to get the rest of the story).

I received an email this morning from U.S. Airways, updating me on my mileage status. Here's a clipping from it:

Account Balance Update

Dear LAMONT R PETERSON,

Tier Level: {% if ( = " " ) %}Dividend Miles{% endif %} {% if ( = "S" ) %}Silver Preferred{% endif %} {% if ( = "G" ) %}Gold Preferred{% endif %} {% if ( = "P" ) %}Platinum Preferred{% endif %} {% if ( = "C" ) %}Chairman's Preferred{% endif %}
Dividend Miles #: 1234567890

Welcome to the new Dividend Miles:

You now have access to over 842 worldwide destinations at your fingertips and mileage-earning opportunities with over 100 partners such as hotels, rental car and financial service companies, just to name a few.

. . . snip . . .

Apparently, the marketing department gets more proofreading resources than the programmers.

I forgot to mention the use of GPG keys with yum in my recent post about using the mplug repository to install Adobe Flash Player.

In the .repo file for the mplug Yum repository, there are two lines about GPG; gpgcheck=1 turns on the feature that GPG signatures must be present and valid for packages downloaded from that repo or yum will not install them. gpgkey=http://macromedia.mplug.org/FEDORA-GPG-KEY provides the location where the GPG key file can be found. Simply download the key and run rpm --import FEDORA-GPG-KEY (as root, of course) to install the key into RPM.

There is older documentation on the Internet that will tell you to run gpg --import keyfile either prior to or in stead of rpm --import keyfile. That's for way older versions of RPM and no longer applies; it is only necessary (and desireable) to install the key into RPM.

In a recent post of mine, I talked about using the mplug yum repository to install the Adobe Flash Player web browser plugin (Adobe bought Macromedia in 2005).

I also talked a little about the future of Flash Player on Linux. The main thing is that Flash 8 will probably never come to Linux, but 8.5 would. Today, I came across this post by Emmy Huang which talks about Flash Player 9 for Linux. There should be a Linux beta late this year for public testing with the final release sometime in early 2007, according to Huang.

Also, there is a new Penguin.SWF blog at Adobe, run my "Mike M." Go there for all the latest on Flash for Linux.

Well, it finally happened. Over this past weekend, while working to finish migrating data from a failing drive to a new one, I found the end of the rope. I have been unable to recover enough data from the old drive to be able to do anything useful with it and since it was still part of the LVM Volume Group (or VG), the problems it was causing were just too much to deal with.

So, I wiped all of the hard drives for the data VG. I am rebuilding everything from my backups. Ha, I'm sure I got some of you. You thought that perhaps I was going through all those gyrations trying to save my data because I didn't have any backups. Well, I think I'm a little smarter than that :) .

Seriously, though, I had backed up all the stuff that would be difficult to replace/reconstruct as well as all the irreplaceable data. I had things like several Linux distributions' files and .iso images, to make it easy to burn discs and to do network installs (either whole installs or just to have the packages handy). There were mirrors of updates and other software. All of that is easy to replace, though, so I didn't bother backing it up.

The next thing I would like to do for my file server is to pick up another 320GB drive so that I can pair it with the one I have now and migrate everything into an LVM on RAID1 set. Later, I'll add more 320GB drives and convert the RAID1 to RAID5 or RAID6. In the meantime, the 45GB drive is now sitting in the target pile, waiting for the next target shooting outing and the 120GB drive is going to be an online backup store in the file server. That leaves me with only 299GB of storage on my file server.

How will I ever get by? :)

The Linux Logical Volume Management system (LVM) is a wonderful piece of technology. In a nutshell, you take disparate storage devices and place them under the control of LVM in a container called a Volume Group (or VG for short). You can then create Logical Volumes (or LV for short) in the VG, which can be formatted, mounted and used, just like any block device. The LVM code will manage everything.

Sometimes, when I'm first introducing students to LVM, they ask me something like, "Why would I ever want to do this? Add an extra layer of complexity and overhead, just to get some fancy looking partitions? No thanks; I'll stick to plain partitions."

First of all, LVM is very efficient, introducing almost zero overhead for most normal operations. Yes, there is a little, but it can be difficult to meassue, let alone notice. Second, there are many irritating storage management tasks that are downright child's play to accomplish with LVM in place but quite difficult with just plain partitions.

As you may have read in my most recent entries, I am updating my home file server, replacing the two existing ATA data drives with a new larger, faster SATA drive.

The old drives are 45GB & 120GB in size, which works out to approximately 41GB and 111GB formatted capacity, for just over 152GB storage. The new drive is a 320GB model, which works out to just over 299GB formatted capcity.

Since the old drives are nearly full, I'm going to be moving a lot of data. As I'm using LVM, I can migrate the data to the new drive with ease, while I have everything online and in use, but I'll cover that later. But before I move the data over, I would like to know if the new drive is good. Although this technique won't tell you for sure (as in life, sometimes there is no certainty), but it's a good indication that things are OK and it gives the new drive a little bit of a workout.

I simply used the badblocks command to test the entire drive, like so: