April 2006 Archives

The market for Linux training is heating up and we are looking for more talented folks to become Guru instructors. We recently filled a position (big wave to Clint), but we have several more spots open.

We have traditionally hired via word-of-mouth.

Job details follow...

=======================
Job title: Guru
Location: Bountiful, Utah and North America (see description below)
Type: full-time position with benefits
Salary: Negotiable depending on experience
Description: You would be joining an extremely talented team of Linux
enthusiasts. Core job responsibilities include teaching Linux classes
locally and abroad, and developing and updating Linux course materials.
Requirements: Strong Linux administration skills with additional broad topic knowledge in security, networking, scripting, and programming very beneficial. The ability
to communicate effectively is required with an outgoing personality a plus. The willingness and
ability to travel also required.

Pros:
* No dealing with pointy haired bosses, corporate red-tape, hostile
workplace politics, etc.
* We are a small company that has been in business for 7 profitable
years. Owned 100% by employees with no outside funding allows us
to operate without external pressures.
* Guru Labs environment and culture foster rapid skills development.
You will be constantly working with smart people on cutting edge Linux
projects. Teaching Linux will add a whole new depth to what you thought
you already knew.
* Gurus are actively encouraged to participate in the open source community.
* Fun travel at least 2 weeks a month. We teach Linux classes (both
public enrollment, and private corporate on-site) all over the US and
beyond. Spend your evenings enjoying local attractions and culinary
wonders.

Cons:
* Expensed meals from the best restaurants in the country will soon
make all the restaurants in Utah seem a bit dull.
* Being surrounded by geeks with tech gadgets may give you "gadget
envy" and lead to increased electronics purchases.
* Your ego may take a hit as you discover that having your laptop
multi-boot between 5 Linux distros sporting custom kernels that you
built and packaged is suddenly the norm. :)

Send resumes to jobs@gurulabs.com

When installing and/or upgrading packages using the /usr/bin/rpm command you have several choices depending on the exact outcome desired and the pre-existing situation.

First one should be aware of one of the RPM rules that is the main factor in this choice, namely "A file can only be 'owned' by a single RPM package". As with all rules in the UNIX/Linux it is possible to override this rule, but you get to keep all the pieces when stuff breaks.

Because of this rule, for the vast majority of packages you might install, you can only have one version installed.

For example, lets say you have two RPMs for the Apache web server.

httpd-2.0.54-10.3
httpd-2.2.0-5.1.2

(the package names come from Fedora Core v4 and v5 respectively).

You can examining the packages (before installation) and see what files are contained. In this case we'll just look at what files have "sbin" in their path since the complete list is over 300 files.

First the package for Apache version 2.0:

$ rpm -qlp httpd-2.0.54-10.3*rpm | grep sbin
/usr/sbin/apachectl
/usr/sbin/httpd
/usr/sbin/httpd.worker
/usr/sbin/rotatelogs
/usr/sbin/suexec

And for Apache version 2.2:

$ rpm -ql httpd-2.2.0-5.1.2 | grep sbin
/usr/sbin/apachectl
/usr/sbin/htcacheclean
/usr/sbin/httpd
/usr/sbin/httpd.worker
/usr/sbin/httxt2dbm
/usr/sbin/rotatelogs
/usr/sbin/suexec

Out of the 300+ files if a single file is the same you can't have both packages installed at the same time. Really the constraint isn't the two Apache RPMs, but the entire set of RPMs that are installed or will be installed on a box. No files can "conflict" (be the same).

With this rule covered, now what RPM options are available and when would one use them. Pretty much you will always use the "vh" options to get verbose output and hash (#) marks. But of the primary action options are "-i", "-U", and "-F".

* "-i". Performs an installation without removing (aka upgrading) any older version of the package. If you have an older version of the package installed, most likely THE COMMAND WILL FAIL because of overlapping files (see the Apache example above). So, for the most part, you can only use "-i" if you know ahead of time that you don't have an older version of the package already installed.

This begs the question, "When can I have two versions of a package installed simultaneously?". There are two situations, one fairly common and the other not so common.

* With the "kernel" RPM package. It turns out that every file provided by the kernel RPM package has the kernel version string somewhere in the full path, for example:

/lib/modules/2.6.15-1.2054_FC5/kernel/arch/i386/crypto/aes-i586.ko

Because of that you CAN have multiple kernel RPM packages installed at the same time, and you might actually WANT to. You might want to because the kernel is very critical to the operation of the system and if you install a new kernel version, and for whatever reason (bad driver, bug, etc) it doesn't work properly or won't boot, by having your old "known good" kernel installed you can easily recover (reboot and select the old kernel from the GRUB menu).

* The other case is when trying to run an old binary you discover it is requires /usr/lib/libfoo.so.1 and you have /usr/lib/libfoo.so.2 installed. Like the kernel RPM, most (but not all), library packages have the version string embedded in the file name and therefore don't conflict.

By using "-i" you can install libfoo-1.0.18.i386.rpm alongside libfoo-2.0.22.i386.rpm.

Finally, for the sake of completeness another related question "How can I have two packages installed where both are supplying a file with same full path?". Here are some example scenarios:

* sendmail and postfix both trying to provide /usr/sbin/sendmail
* SUN Java, IBM Java, and GCJ trying to provide /usr/bin/java
* CUPS, LPRng both trying to provide /usr/bin/lpr

The answer is (as is often the case in computer science), don't have the files conflict and use an abstraction layer. This was first done by the Debian folks in the creation of the "alternatives" system, it is used very widely in Debian/Ubuntu for lots of different packages. Red Hat adopted it during the 7.x time frame but used it just with MTAs (sendmail, postfix) and printing subsystems. SUSE has now adopted it with version 10.0 but only for Java packages from the jpackage project. The complete discussion of the alternatives system is beyond the scope of this blog post. We do have excellent coverage in our Linux training classes though.

* "-F". Performs an installation and removes (aka upgrades) any older version of the package if and only if you DO have an older version of the package installed. This option I like to call the "upgrade only" option. It is relic of the olden days of updating a Linux box with errata. Back then you update your system with the updates by:

1. Download all available updates (using ftp and mget *rpm) into a local directory.
2. In that directory run "rpm -Fvh *rpm".

This way you wouldn't install any new software packages that happened to have an update and instead you would just update the packages you did have installed.

Today we keep our systems current with smarter methods such as "yum -y update, you, up2date, rug, etc".

* "-U". This option performs an install if you don't have an older version already installed, and an upgrade if you do. I call it the "install or update as needed" option.

So to answer the question, "What RPM option does Dax Kelson use to install or upgrade RPM packages?" the answer is, "I try not to use /usr/bin/rpm unless I'm installing packages I've created myself or manually downloaded." Instead to install software I use a front end that figures out and downloads the dependencies automatically for me. For example:

* yum install packagename ...
* yast -i packagename ...
* up2date packagename ...

In the case when I've created my own RPM or done a manual download of a RPM package I like to use the "-U" option. This way RPM does the right thing (install or upgrade as needed) for me and I don't have to keep track of mentally if I already have the package installed.

Today Novell/SUSE submitted the AppArmor patches to the Linux Kernel Mailing List (LKML). Following the discussion is likely to be interesting.

Red Hat has adopted the SELinux security framework (already accepted into the Linux kernel). The SELinux frameworks plugs into the kernel's LSM subsystem. Some people have complained of the complexity of SELinux. Because of the complexity and interference many people just turn off SELinux. The response from the SELinux folks is that Linux software has complex interactions and *any* solution to properly secure it will be, by definition, at least as complex. Furthermore, the SELinux developers say that they have worked hard on developing a clean foundation that is basically complete now and that all the easy to use front end management software can now appear.

Novell/SUSE has chosen an alternate, less complex security framework, AppArmor. The benefit is well, that is less complex and doesn't "interfere" as much as SELinux so it is less likely to get turned off. The complaint about AppArmor is that it doesn't provide full security and depends on file pathnames, and won't scale well because of required locking. If a file's name changes (hard link, mount, etc) the security goes out the window. Another issue brought up is that the use AppArmor precludes the use of filesystem namespaces support for which has been slowly added to the kernel. The use of namespaces is supposed to usher in a new era of flexible and wonderful abilities that could be very useful for desktop users and virtualization. Today however, nobody is making use of filesystem namespaces in any mainstream distribution.

Personally, as a system administration and user of Linux I encourage the distributions to "un-fork" as much as possible. Thanks to the Linux Standards Base (LSB) and other efforts managing Red Hat boxes and SUSE boxes is, for the most part, the same. So from this stand point I'm pretty disappointed to see this split. It becomes yet another thing I must wrap my brain around and keep up on. Also, from an efficiency and pace of innovation perspective I would have preferred all the resources and development pushing and pulling in the same direction.

At Guru Labs we already have extensive SELinux coverage in our GL550 Linux security training class. When we do the big rev for RHEL5 and SLES10 we will be adding extensive coverage of AppArmor as well.

Being a computer user on a network that uses single sign on (SSO) is very convenient. Another benefit is the "other thing" that users don't generally concern themselves with, increased security. The Kerberos Network Authentication Protocol developed at MIT in the 1980s is the open standard that has been adopted widely.

On your network, the more services that are using SSO authentication, the greater the benefit of SSO. This is commonly called the "Fax Effect" (the more people that own fax machines, the greater the benefit to each fax machine owner). Today many services are able to use Kerberos authentication either directly, or indirectly through GSSAPI or SASL+GSSAPI.

Some of these services include:

* SQL Servers (PostgreSQL, Oracle)
* SMTP (Postfix, Sendmail)
* IMAP (Cyrus-imapd, Dovecot)
* Email clients (Evolution, Thunderbird, Kmail)
* SSH (OpenSSH)
* telnet/ftp/rlogin/rsh
* rsync (via ssh)
* Web Applications (Apache +mod_auth_kerb or IIS plus Mozilla/Firefox/Konqueror/IE)
* File Servers (NFSv3/v4 with "sec=krb5" on Linux, or Samba)
* Print Servers (LPRng or later this year, CUPS)
* Network equipment (Cisco IOS and others)

Here at Guru Labs, we have been on a multi-year mission get every service on our network using Kerberos authentication. Not just with Kerberos, but across the board we try to develop best practices, "dog food" them and then write about them in our Linux courseware and training.

One service we recently Kerberized was our Jabber instant messaging server. Getting Jabber kerberized is very nice, particularly when using Gaim. If you configure Gaim to store your passwords (not the default, but very conveniently tempting), it stores them in plaintext in your ~/.gaim/accounts.xml file.

As of April, 2006 the GSSAPI+Kerberos Jabber landscape is as follows:

There are two open source Jabber server implementation that supports GSSAPI+Kerberos authentication.

* Jabberd v2.0 with Simon Wilkinson's Kerberos/GSSAPI/SASL patch. This is a mature well tested solution. This is what we are using at Guru Labs. The patch has been accepted into the CVS tree and will be in the future v2.1 release.

* The highly regarded and actively developed Java based Wildfire Server is just barely (days ago) starting to work with GSSAPI. Once the rough edges are polished off and a stable release is made with GSSAPI support we are going to strongly consider moving to this server.

On the client front there are patches for Psi and Coccinella and Gaim. I haven't used Psi or Coccinella so I don't know if the patches are current or have been accepted into the official trees.

For gaim v1.5.x there are two patches. Simon Wilkinson developed a SASL-GSSAPI patch that was later modified by Greg Hudson of MIT to support gracefull fallback by prompting for a password if a Kerberos ticket is not obtainable. This is something I wish more client software would do.

The soon to be released gaim v2.0 has Simon's patch integrated, so it will support GSSAPI/Kerberos authentication out-of-the-box. There are plans to add graceful fallback and other features.

Our experiences getting Jabber Kerberized will be rolled into our GL550 "Enterprise Linux Security Administration" training course. The course includes extensive Kerberos coverage both of MIT's implementation and KTH's Heimdal implementation (used on SUSE Linux Enterprise Server 9) as well as best practices for Kerberizing common services (see the list above). It is the only Kerberos training class that I'm aware of.

My laptop has the Intel 2200BG card and uses the ipw2200 driver. By default, when the driver loads, it tries to associate to any network that is open and accessible. This is the physical equivalent of your laptop automatically plugging itself into any network port in the area.

I don't like this behavior, both from a I-want-control-of-my-network-status as well as go-to-jail-or-pay-a-big-fine stand point.

Fortunately driver allows you to turn off this auto-associate behavior with a parameter. In my /etc/modprobe.conf I added the line:

options ipw2200 associate=0

Problem solved.

I take my laptop pretty seriously since I use it as my primary computer both at work and home. I'm picky about the performance, weight, screen and durability. It's the same for most of us at Guru Labs. A major line of work for us is lugging our laptops around the world delivering Linux training. The ThinkPad T series is a common sight around the office.

For years laptop hard drives ran at 4200RPM and were a major bottleneck in mobile performance. Fortunately 5400 and 7200RPM drives brought "desktop like" performance to laptops. Two years ago when I bought my ThinkPad T42p I went for the largest 7200RPM drive available at the time, 60GB. I have really enjoyed the speed and vowed that I wouldn't get anything slower than 7200RPM in my laptop. The only problem is that I have been a bit cramped by the space and, even today, the largest 7200RPM 2.5" laptop hard drive is only modestly larger at 100GB.

Not too long ago the Seagate Momentus 5400.3 ST9160821A 160GB Hard Drive was released and took the crown as the new champion in 2.5" laptop capacity. When I saw that it was a 5400RPM hard drive I was a bit bummed -- however when I found out it was the first hard drive to ship with perpendicular recording technology I was intrigued.

Reviews were hard to come by, and the ones I read didn't have any comparisons against 7200RPM laptop hard drives. I took a chance and bought one with the strong hopes that the high areal density would translate into performance that could match my 7200RPM drive.

Here are what the initial performance numbers (average numbers reported from several hdparm -tT runs) look like:

For my original 60GB 7200RPM drive:
/dev/hda:
Timing cached reads: 2104 MB in 2.00 seconds = 1051.93 MB/sec
Timing buffered disk reads: 114 MB in 3.00 seconds = 37.95 MB/sec

For the new Seagate Momentus 160GB 5400RPM drive:
/dev/hda:
Timing cached reads: 2112 MB in 2.00 seconds = 1055.82 MB/sec
Timing buffered disk reads: 122 MB in 3.00 seconds = 40.61 MB/sec

As you can see it exceeded, not just matched, the performance I have been used too. Additional benefits of the drive are quieter operation and the 5400RPM uses less power to increases my battery life. I'm very pleased.

At the Guru Labs office you can get easily blinded by all the shiny geek toys and I'm afraid I've triggered another round of upgrades. :)