grub features

The primary requirement for GRUB is that it be compliant with the Multiboot Specification, which is described in section `Motivation' in The Multiboot Specification.

The other goals, listed in approximate order of importance, are:

• Basic functions must be straightforward for end-users.
• Rich functionality to support kernel experts and designers.
• Backward compatibility for booting FreeBSD, NetBSD, OpenBSD, and Linux. Proprietary kernels (such as DOS, Windows NT, and OS/2) are supported via a chain-loading function.

Except for specific compatibility modes (chain-loading and the Linux piggyback format), all kernels will be started in much the same state as in the Multiboot Specification. Only kernels loaded at 1 megabyte or above are presently supported. Any attempt to load below that boundary will simply result in immediate failure and an error message reporting the problem.

In addition to the requirements above, GRUB has the following features (note that the Multiboot Specification doesn't require all the features that GRUB supports):

Recognize multiple executable formats

Support many of the a.out variants plus ELF. Symbol tables are also loaded.

Support non-Multiboot kernels

Support many of the various free 32-bit kernels that lack Multiboot compliance (primarily FreeBSD, NetBSD, OpenBSD, and Linux). Chain-loading of other boot loaders is also supported.

Load multiples modules

Fully support the Multiboot feature of loading multiple modules.

Load a configuration file

Support a human-readable text configuration file with preset boot commands. You can also load another configuration file dynamically and embed a preset configuration file in a GRUB image file. The list of commands (see section 13. The list of available commands) are a superset of those supported on the command-line. An example configuration file is provided in 5. Configuration.

Provide a menu interface

A menu interface listing preset boot commands, with a programmable timeout, is available. There is no fixed limit on the number of boot entries, and the current implementation has space for several hundred.

Have a flexible command-line interface

A fairly flexible command-line interface, accessible from the menu, is available to edit any preset commands, or write a new boot command set from scratch. If no configuration file is present, GRUB drops to the command-line.

The list of commands (see section 13. The list of available commands) are a subset of those supported for configuration files. Editing commands closely resembles the Bash command-line (see section `Command Line Editing' in Bash Features), with TAB-completion of commands, devices, partitions, and files in a directory depending on context.

Support multiple filesystem types

Support multiple filesystem types transparently, plus a useful explicit blocklist notation. The currently supported filesystem types are BSD FFS, DOS FAT16 and FAT32, Minix fs, Linux ext2fs, ReiserFS, JFS, XFS, and VSTa fs. See section 11. Filesystem syntax and semantics, for more information.

Support automatic decompression

Can decompress files which were compressed by gzip. This function is both automatic and transparent to the user (i.e. all functions operate upon the uncompressed contents of the specified files). This greatly reduces a file size and loading time, a particularly great benefit for floppies.(2)

It is conceivable that some kernel modules should be loaded in a compressed state, so a different module-loading command can be specified to avoid uncompressing the modules.

Access data on any installed device

Support reading data from any or all floppies or hard disk(s) recognized by the BIOS, independent of the setting of the root device.

Be independent of drive geometry translations

Unlike many other boot loaders, GRUB makes the particular drive translation irrelevant. A drive installed and running with one translation may be converted to another translation without any adverse effects or changes in GRUB's configuration.

Detect all installed RAM

GRUB can generally find all the installed RAM on a PC-compatible machine. It uses an advanced BIOS query technique for finding all memory regions. As described on the Multiboot Specification (see section `Motivation' in The Multiboot Specification), not all kernels make use of this information, but GRUB provides it for those who do.

Support Logical Block Address mode

In traditional disk calls (called CHS mode), there is a geometry translation problem, that is, the BIOS cannot access over 1024 cylinders, so the accessible space is limited to at least 508 MB and to at most 8GB. GRUB can't universally solve this problem, as there is no standard interface used in all machines. However, several newer machines have the new interface, Logical Block Address (LBA) mode. GRUB automatically detects if LBA mode is available and uses it if available. In LBA mode, GRUB can access the entire disk.

Support network booting

GRUB is basically a disk-based boot loader but also has network support. You can load OS images from a network by using the TFTP protocol.

Support remote terminals

To support computers with no console, GRUB provides remote terminal support, so that you can control GRUB from a remote host. Only serial terminal support is implemented at the moment.

1.4 The role of a boot loader

<!--docid::SEC5::-->

The following is a quotation from Gordon Matzigkeit, a GRUB fanatic:

Some people like to acknowledge both the operating system and kernel when they talk about their computers, so they might say they use "GNU/Linux" or "GNU/Hurd". Other people seem to think that the kernel is the most important part of the system, so they like to call their GNU operating systems "Linux systems."

I, personally, believe that this is a grave injustice, because the boot loader is the most important software of all. I used to refer to the above systems as either "LILO"(3) or "GRUB" systems.

Unfortunately, nobody ever understood what I was talking about; now I just use the word "GNU" as a pseudonym for GRUB.

So, if you ever hear people talking about their alleged "GNU" systems, remember that they are actually paying homage to the best boot loader around... GRUB!

We, the GRUB maintainers, do not (usually) encourage Gordon's level of fanaticism, but it helps to remember that boot loaders deserve recognition. We hope that you enjoy using GNU GRUB as much as we did writing it.

naming convention

GRUB Naming convention

The device syntax used in GRUB is a wee bit different from what you may have seen before in your operating system(s), and you need to know it so that you can specify a drive/partition.

Look at the following examples and explanations:

(fd0)

First of all, GRUB requires that the device name be enclosed with `(' and `)'. The `fd' part means that it is a floppy disk. The number `0' is the drive number, which is counted from zero. This expression means that GRUB will use the whole floppy disk.

(hd0,1)

Here, `hd' means it is a hard disk drive. The first integer `0' indicates the drive number, that is, the first hard disk, while the second integer, `1', indicates the partition number (or the PC slice number in the BSD terminology). Once again, please note that the partition numbers are counted from zero, not from one. This expression means the second partition of the first hard disk drive. In this case, GRUB uses one partition of the disk, instead of the whole disk.

(hd0,4)

This specifies the first extended partition of the first hard disk drive. Note that the partition numbers for extended partitions are counted from `4', regardless of the actual number of primary partitions on your hard disk.

(hd1,a)

This means the BSD `a' partition of the second hard disk. If you need to specify which PC slice number should be used, use something like this: `(hd1,0,a)'. If the PC slice number is omitted, GRUB searches for the first PC slice which has a BSD `a' partition.

Of course, to actually access the disks or partitions with GRUB, you need to use the device specification in a command, like `root (fd0)' or `unhide (hd0,2)'. To help you find out which number specifies a partition you want, the GRUB command-line (see section 12.1 The flexible command-line interface) options have argument completion. This means that, for example, you only need to type

root (

followed by a TAB, and GRUB will display the list of drives, partitions, or file names. So it should be quite easy to determine the name of your target partition, even with minimal knowledge of the syntax.

Note that GRUB does not distinguish IDE from SCSI - it simply counts the drive numbers from zero, regardless of their type. Normally, any IDE drive number is less than any SCSI drive number, although that is not true if you change the boot sequence by swapping IDE and SCSI drives in your BIOS.

Now the question is, how to specify a file? Again, consider an example:

(hd0,0)/vmlinuz

This specifies the file named `vmlinuz', found on the first partition of the first hard disk drive. Note that the argument completion works with file names, too.

That was easy, admit it. Now read the next chapter, to find out how to actually install GRUB on your drive.

<!--docid::SEC6::-->

Installation

In order to install GRUB as your boot loader, you need to first install the GRUB system and utilities under your UNIX-like operating system (see section A. How to obtain and build GRUB). You can do this either from the source tarball, or as a package for your OS.

After you have done that, you need to install the boot loader on a drive (floppy or hard disk). There are two ways of doing that - either using the utility grub-install (see section 16. Invoking grub-install) on a UNIX-like OS, or by running GRUB itself from a floppy. These are quite similar, however the utility might probe a wrong BIOS drive, so you should be careful.

Also, if you install GRUB on a UNIX-like OS, please make sure that you have an emergency boot disk ready, so that you can rescue your computer if, by any chance, your hard drive becomes unusable (unbootable).

GRUB comes with boot images, which are normally put in the directory `/usr/share/grub/i386-pc'. If you do not use grub-install, then you need to copy the files `stage1', `stage2', and `*stage1_5' to the directory `/boot/grub'. Hereafter, the directory where GRUB images are initially placed (normally `/usr/share/grub/i386-pc') will be called the image directory, and the directory where the boot loader needs to find them (usually `/boot/grub') will be called the boot directory.

3.1 Creating a GRUB boot floppy
3.2 Installing GRUB natively
3.3 Installing GRUB using grub-install
3.4 Making a GRUB bootable CD-ROM

3.1 Creating a GRUB boot floppy

To create a GRUB boot floppy, you need to take the files `stage1' and `stage2' from the image directory, and write them to the first and the second block of the floppy disk, respectively.

Caution: This procedure will destroy any data currently stored on the floppy.

On a UNIX-like operating system, that is done with the following commands:

# cd /usr/share/grub/i386-pc
# dd if=stage1 of=/dev/fd0 bs=512 count=1
1+0 records in
1+0 records out
# dd if=stage2 of=/dev/fd0 bs=512 seek=1
153+1 records in
153+1 records out
#

The device file name may be different. Consult the manual for your OS.

3.2 Installing GRUB natively

Caution: Installing GRUB's stage1 in this manner will erase the normal boot-sector used by an OS.

GRUB can currently boot GNU Mach, Linux, FreeBSD, NetBSD, and OpenBSD directly, so using it on a boot sector (the first sector of a partition) should be okay. But generally, it would be a good idea to back up the first sector of the partition on which you are installing GRUB's stage1. This isn't as important if you are installing GRUB on the first sector of a hard disk, since it's easy to reinitialize it (e.g. by running `FDISK /MBR' from DOS).

If you decide to install GRUB in the native environment, which is definitely desirable, you'll need to create a GRUB boot disk, and reboot your computer with it. Otherwise, see 3.3 Installing GRUB using grub-install.

Once started, GRUB will show the command-line interface (see section 12.1 The flexible command-line interface). First, set the GRUB's root device(4) to the partition containing the boot directory, like this:

grub> root (hd0,0)

If you are not sure which partition actually holds this directory, use the command find (see section 13.3.11 find), like this:

grub> find /boot/grub/stage1

This will search for the file name `/boot/grub/stage1' and show the devices which contain the file.

Once you've set the root device correctly, run the command setup (see section 13.3.34 setup):

grub> setup (hd0)

This command will install the GRUB boot loader on the Master Boot Record (MBR) of the first drive. If you want to put GRUB into the boot sector of a partition instead of putting it in the MBR, specify the partition into which you want to install GRUB:

grub> setup (hd0,0)

If you install GRUB into a partition or a drive other than the first one, you must chain-load GRUB from another boot loader. Refer to the manual for the boot loader to know how to chain-load GRUB.

After using the setup command, you will boot into GRUB without the GRUB floppy. See the chapter 4. Booting to find out how to boot your operating systems from GRUB.

3.3 Installing GRUB using grub-install

Caution: This procedure is definitely less safe, because there are several ways in which your computer can become unbootable. For example, most operating systems don't tell GRUB how to map BIOS drives to OS devices correctly--GRUB merely guesses the mapping. This will succeed in most cases, but not always. Therefore, GRUB provides you with a map file called the device map, which you must fix if it is wrong. See section 15.3 The map between BIOS drives and OS devices, for more details.

If you still do want to install GRUB under a UNIX-like OS (such as GNU), invoke the program grub-install (see section 16. Invoking grub-install) as the superuser (root).

The usage is basically very simple. You only need to specify one argument to the program, namely, where to install the boot loader. The argument can be either a device file (like `/dev/hda') or a partition specified in GRUB's notation. For example, under Linux the following will install GRUB into the MBR of the first IDE disk:

# grub-install /dev/hda

Likewise, under GNU/Hurd, this has the same effect:

# grub-install /dev/hd0

If it is the first BIOS drive, this is the same as well:

# grub-install '(hd0)'

Or you can omit the parentheses:

# grub-install hd0

But all the above examples assume that GRUB should use images under the root directory. If you want GRUB to use images under a directory other than the root directory, you need to specify the option `--root-directory'. The typical usage is that you create a GRUB boot floppy with a filesystem. Here is an example:

# mke2fs /dev/fd0
# mount -t ext2 /dev/fd0 /mnt
# grub-install --root-directory=/mnt fd0
# umount /mnt

Another example is when you have a separate boot partition which is mounted at `/boot'. Since GRUB is a boot loader, it doesn't know anything about mountpoints at all. Thus, you need to run grub-install like this:

# grub-install --root-directory=/boot /dev/hda

By the way, as noted above, it is quite difficult to guess BIOS drives correctly under a UNIX-like OS. Thus, grub-install will prompt you to check if it could really guess the correct mappings, after the installation. The format is defined in 15.3 The map between BIOS drives and OS devices. Please be quite careful. If the output is wrong, it is unlikely that your computer will be able to boot with no problem.

Note that grub-install is actually just a shell script and the real task is done by the grub shell grub (see section 15. Invoking the grub shell). Therefore, you may run grub directly to install GRUB, without using grub-install. Don't do that, however, unless you are very familiar with the internals of GRUB. Installing a boot loader on a running OS may be extremely dangerous.

3.4 Making a GRUB bootable CD-ROM

GRUB supports the no emulation mode in the El Torito specification(5). This means that you can use the whole CD-ROM from GRUB and you don't have to make a floppy or hard disk image file, which can cause compatibility problems.

For booting from a CD-ROM, GRUB uses a special Stage 2 called `stage2_eltorito'. The only GRUB files you need to have in your bootable CD-ROM are this `stage2_eltorito' and optionally a config file `menu.lst'. You don't need to use `stage1' or `stage2', because El Torito is quite different from the standard boot process.

Here is an example of procedures to make a bootable CD-ROM image. First, make a top directory for the bootable image, say, `iso':

$ mkdir iso

Make a directory for GRUB:

$ mkdir -p iso/boot/grub

Copy the file `stage2_eltorito':

$ cp /usr/share/grub/i386-pc/stage2_eltorito iso/boot/grub

If desired, make the config file `menu.lst' under `iso/boot/grub' (see section 5. Configuration), and copy any files and directories for the disc to the directory `iso/'.

Finally, make a ISO9660 image file like this:

$ mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot \
-boot-load-size 4 -boot-info-table -o grub.iso iso

This produces a file named `grub.iso', which then can be burned into a CD (or a DVD). mkisofs has already set up the disc to boot from the boot/grub/stage2_eltorito file, so there is no need to setup GRUB on the disc. (Note that the -boot-load-size 4 bit is required for compatibility with the BIOS on many older machines.)

You can use the device `(cd)' to access a CD-ROM in your config file. This is not required; GRUB automatically sets the root device to `(cd)' when booted from a CD-ROM. It is only necessary to refer to `(cd)' if you want to access other drives as well.

booting

GRUB can load Multiboot-compliant kernels in a consistent way, but for some free operating systems you need to use some OS-specific magic.

4.1 How to boot operating systems How to boot OSes with GRUB generally
4.2 Some caveats on OS-specific issues Notes on some operating systems

4.1 How to boot operating systems

GRUB has two distinct boot methods. One of the two is to load an operating system directly, and the other is to chain-load another boot loader which then will load an operating system actually. Generally speaking, the former is more desirable, because you don't need to install or maintain other boot loaders and GRUB is flexible enough to load an operating system from an arbitrary disk/partition. However, the latter is sometimes required, since GRUB doesn't support all the existing operating systems natively.

4.1.1 How to boot an OS directly with GRUB
4.1.2 Load another boot loader to boot unsupported operating systems

4.1.1 How to boot an OS directly with GRUB

Multiboot (see section `Motivation' in The Multiboot Specification) is the native format supported by GRUB. For the sake of convenience, there is also support for Linux, FreeBSD, NetBSD and OpenBSD. If you want to boot other operating systems, you will have to chain-load them (see section 4.1.2 Load another boot loader to boot unsupported operating systems).

Generally, GRUB can boot any Multiboot-compliant OS in the following steps:

1. Set GRUB's root device to the drive where the OS images are stored with the command root (see section 13.3.31 root).

2. Load the kernel image with the command kernel (see section 13.3.20 kernel).

3. If you need modules, load them with the command module (see section 13.3.25 module) or modulenounzip (see section 13.3.26 modulenounzip).

4. Run the command boot (see section 13.3.2 boot).

Linux, FreeBSD, NetBSD and OpenBSD can be booted in a similar manner. You load a kernel image with the command kernel and then run the command boot. If the kernel requires some parameters, just append the parameters to kernel, after the file name of the kernel. Also, please refer to 4.2 Some caveats on OS-specific issues, for information on your OS-specific issues.

4.1.2 Load another boot loader to boot unsupported operating systems

If you want to boot an unsupported operating system (e.g. Windows 95), chain-load a boot loader for the operating system. Normally, the boot loader is embedded in the boot sector of the partition on which the operating system is installed.

1. Set GRUB's root device to the partition by the command rootnoverify (see section 13.3.32 rootnoverify):

grub> rootnoverify (hd0,0)

2. Set the active flag in the partition using the command makeactive(6) (see section 13.3.22 makeactive):

grub> makeactive

3. Load the boot loader with the command chainloader (see section 13.3.4 chainloader):

grub> chainloader +1

`+1' indicates that GRUB should read one sector from the start of the partition. The complete description about this syntax can be found in 11.3 How to specify block lists.

4. Run the command boot (see section 13.3.2 boot).

However, DOS and Windows have some deficiencies, so you might have to use more complicated instructions. See section 4.2.6 DOS/Windows, for more information.

4.2 Some caveats on OS-specific issues

Here, we describe some caveats on several operating systems.

4.2.1 GNU/Hurd
4.2.2 GNU/Linux

4.2.1 GNU/Hurd

Since GNU/Hurd is Multiboot-compliant, it is easy to boot it; there is nothing special about it. But do not forget that you have to specify a root partition to the kernel.

1. Set GRUB's root device to the same drive as GNU/Hurd's. Probably the command find /boot/gnumach or similar can help you (see section 13.3.11 find).

2. Load the kernel and the module, like this:

grub> kernel /boot/gnumach root=hd0s1
grub> module /boot/serverboot

3. Run the command boot (see section 13.3.2 boot).

4.2.2 GNU/Linux

It is relatively easy to boot GNU/Linux from GRUB, because it somewhat resembles to boot a Multiboot-compliant OS.

1. Set GRUB's root device to the same drive as GNU/Linux's. Probably the command find /vmlinuz or similar can help you (see section 13.3.11 find).

2. Load the kernel:

grub> kernel /vmlinuz root=/dev/hda1

If you need to specify some kernel parameters, just append them to the command. For example, to set `vga' to `ext', do this:

grub> kernel /vmlinuz root=/dev/hda1 vga=ext

See the documentation in the Linux source tree for complete information on the available options.

3. If you use an initrd, execute the command initrd (see section 13.3.17 initrd) after kernel:

grub> initrd /initrd

4. Finally, run the command boot (see section 13.3.2 boot).

Caution: If you use an initrd and specify the `mem=' option to the kernel to let it use less than actual memory size, you will also have to specify the same memory size to GRUB. To let GRUB know the size, run the command uppermem before loading the kernel. See section 13.3.37 uppermem, for more information.

configuration

You've probably noticed that you need to type several commands to boot your OS. There's a solution to that - GRUB provides a menu interface (see section 12.2 The simple menu interface) from which you can select an item (using arrow keys) that will do everything to boot an OS.

To enable the menu, you need a configuration file, `menu.lst' under the boot directory. We'll analyze an example file.

The file first contains some general settings, the menu interface related options. You can put these commands (see section 13.1 The list of commands for the menu only) before any of the items (starting with title (see section 13.1.5 title)).

#
# Sample boot menu configuration file
#

As you may have guessed, these lines are comments. Lines starting with a hash character (`#'), and blank lines, are ignored by GRUB.

# By default, boot the first entry.
default 0

The first entry (here, counting starts with number zero, not one!) will be the default choice.

# Boot automatically after 30 secs.
timeout 30

As the comment says, GRUB will boot automatically in 30 seconds, unless interrupted with a keypress.

# Fallback to the second entry.
fallback 1

If, for any reason, the default entry doesn't work, fall back to the second one (this is rarely used, for obvious reasons).

Note that the complete descriptions of these commands, which are menu interface specific, can be found in 13.1 The list of commands for the menu only. Other descriptions can be found in 13. The list of available commands.

Now, on to the actual OS definitions. You will see that each entry begins with a special command, title (see section 13.1.5 title), and the action is described after it. Note that there is no command boot (see section 13.3.2 boot) at the end of each item. That is because GRUB automatically executes boot if it loads other commands successfully.

The argument for the command title is used to display a short title/description of the entry in the menu. Since title displays the argument as is, you can write basically anything there.

# For booting GNU/Hurd
title  GNU/Hurd
root   (hd0,0)
kernel /boot/gnumach.gz root=hd0s1
module /boot/serverboot.gz

This boots GNU/Hurd from the first hard disk.

# For booting GNU/Linux
title  GNU/Linux
kernel (hd1,0)/vmlinuz root=/dev/hdb1

This boots GNU/Linux, but from the second hard disk.

# For booting Mach (getting kernel from floppy)
title  Utah Mach4 multiboot
root   (hd0,2)
pause  Insert the diskette now^G!!
kernel (fd0)/boot/kernel root=hd0s3
module (fd0)/boot/bootstrap

This boots Mach with a kernel on a floppy, but the root filesystem at hd0s3. It also contains a pause line (see section 13.3.27 pause), which will cause GRUB to display a prompt and delay, before actually executing the rest of the commands and booting.

downloading OS images from a network

6. Downloading OS images from a network

Although GRUB is a disk-based boot loader, it does provide network support. To use the network support, you need to enable at least one network driver in the GRUB build process. For more information please see `netboot/README.netboot' in the source distribution.

6.1 How to set up your network

GRUB requires a file server and optionally a server that will assign an IP address to the machine on which GRUB is running. For the former, only TFTP is supported at the moment. The latter is either BOOTP, DHCP or a RARP server(7). It is not necessary to run both the servers on one computer. How to configure these servers is beyond the scope of this document, so please refer to the manuals specific to those protocols/servers.

If you decided to use a server to assign an IP address, set up the server and run bootp (see section 13.2.1 bootp), dhcp (see section 13.2.4 dhcp) or rarp (see section 13.2.11 rarp) for BOOTP, DHCP or RARP, respectively. Each command will show an assigned IP address, a netmask, an IP address for your TFTP server and a gateway. If any of the addresses is wrong or it causes an error, probably the configuration of your servers isn't set up properly.

Otherwise, run ifconfig, like this:

grub> ifconfig --address=192.168.110.23 --server=192.168.110.14

You can also use ifconfig in conjuction with bootp, dhcp or rarp (e.g. to reassign the server address manually). See section 13.2.6 ifconfig, for more details.

Finally, download your OS images from your network. The network can be accessed using the network drive `(nd)'. Everything else is very similar to the normal instructions (see section 4. Booting).

Here is an example:

grub> bootp
Probing... [NE*000]
NE2000 base ...
Address: 192.168.110.23    Netmask: 255.255.255.0
Server: 192.168.110.14     Gateway: 192.168.110.1

grub> root (nd)
grub> kernel /tftproot/gnumach.gz root=sd0s1
grub> module /tftproot/serverboot.gz
grub> boot

6.2 Booting from a network

It is sometimes very useful to boot from a network, especially when you use a machine which has no local disk. In this case, you need to obtain a kind of Net Boot ROM, such as a PXE ROM or a free software package like Etherboot. Such a Boot ROM first boots the machine, sets up the network card installed into the machine, and downloads a second stage boot image from the network. Then, the second image will try to boot an operating system actually from the network.

GRUB provides two second stage images, `nbgrub' and `pxegrub' (see section 10. GRUB image files). These images are the same as the normal Stage 2, except that they set up a network automatically, and try to load a configuration file from the network, if specified. The usage is very simple: If the machine has a PXE ROM, use `pxegrub'. If the machine has an NBI loader such as Etherboot, use `nbgrub'. There is no difference between them except their formats. Since the way to load a second stage image you want to use should be described in the manual on your Net Boot ROM, please refer to the manual, for more information.

However, there is one thing specific to GRUB. Namely, how to specify a configuration file in a BOOTP/DHCP server. For now, GRUB uses the tag `150', to get the name of a configuration file. The following is an example with a BOOTP configuration:

.allhost:hd=/tmp:bf=null:\
:ds=145.71.35.1 145.71.32.1:\
:sm=255.255.254.0:\
:gw=145.71.35.1:\
:sa=145.71.35.5:

foo:ht=1:ha=63655d0334a7:ip=145.71.35.127:\
:bf=/nbgrub:\
:tc=.allhost:\
:T150="(nd)/tftpboot/menu.lst.foo":

Note that you should specify the drive name (nd) in the name of the configuration file. This is because you might change the root drive before downloading the configuration from the TFTP server when the preset menu feature is used (see section 8. Embedding a configuration file into GRUB).

See the manual of your BOOTP/DHCP server for more information. The exact syntax should differ a little from the example.

embedding grub configuration file

Embedding a configuration file into GRUB

GRUB supports a preset menu which is to be always loaded before starting. The preset menu feature is useful, for example, when your computer has no console but a serial cable. In this case, it is critical to set up the serial terminal as soon as possible, since you cannot see any message until the serial terminal begins to work. So it is good to run the commands serial (see section 13.2.12 serial) and terminal (see section 13.2.14 terminal) before anything else at the start-up time.

How the preset menu works is slightly complicated:

1. GRUB checks if the preset menu feature is used, and loads the preset menu, if available. This includes running commands and reading boot entries, like an ordinary configuration file.

2. GRUB checks if the configuration file is available. Note that this check is performed regardless of the existence of the preset menu. The configuration file is loaded even if the preset menu was loaded.

3. If the preset menu includes any boot entries, they are cleared when the configuration file is loaded. It doesn't matter whether the configuration file has any entries or no entry. The boot entries in the preset menu are used only when GRUB fails in loading the configuration file.

To enable the preset menu feature, you must rebuild GRUB specifying a file to the configure script with the option `--enable-preset-menu'. The file has the same semantics as normal configuration files (see section 5. Configuration).

Another point you should take care is that the diskless support (see section 6.2 Booting from a network) diverts the preset menu. Diskless images embed a preset menu to execute the command bootp (see section 13.2.1 bootp) automatically, unless you specify your own preset menu to the configure script. This means that you must put commands to initialize a network in the preset menu yourself, because diskless images don't set it up implicitly, when you use the preset menu explicitly.

Therefore, a typical preset menu used with diskless support would be like this:

# Set up the serial terminal, first of all.
serial --unit=0 --speed=19200
terminal --timeout=0 serial

# Initialize the network.
dhcp

protection from cracking

Protecting your computer from cracking

You may be interested in how to prevent ordinary users from doing whatever they like, if you share your computer with other people. So this chapter describes how to improve the security of GRUB.

One thing which could be a security hole is that the user can do too many things with GRUB, because GRUB allows one to modify its configuration and run arbitrary commands at run-time. For example, the user can even read `/etc/passwd' in the command-line interface by the command cat (see section 13.3.3 cat). So it is necessary to disable all the interactive operations.

Thus, GRUB provides a password feature, so that only administrators can start the interactive operations (i.e. editing menu entries and entering the command-line interface). To use this feature, you need to run the command password in your configuration file (see section 13.2.10 password), like this:

password --md5 PASSWORD

If this is specified, GRUB disallows any interactive control, until you press the key p and enter a correct password. The option `--md5' tells GRUB that `PASSWORD' is in MD5 format. If it is omitted, GRUB assumes the `PASSWORD' is in clear text.

You can encrypt your password with the command md5crypt (see section 13.3.24 md5crypt). For example, run the grub shell (see section 15. Invoking the grub shell), and enter your password:

grub> md5crypt
Password: **********
Encrypted: $1$U$JK7xFegdxWH6VuppCUSIb.

Then, cut and paste the encrypted password to your configuration file.

Also, you can specify an optional argument to password. See this example:

password PASSWORD /boot/grub/menu-admin.lst

In this case, GRUB will load `/boot/grub/menu-admin.lst' as a configuration file when you enter the valid password.

Another thing which may be dangerous is that any user can choose any menu entry. Usually, this wouldn't be problematic, but you might want to permit only administrators to run some of your menu entries, such as an entry for booting an insecure OS like DOS.

GRUB provides the command lock (see section 13.3.21 lock). This command always fails until you enter the valid password, so you can use it, like this:

title Boot DOS
lock
rootnoverify (hd0,1)
makeactive
chainload +1

You should insert lock right after title, because any user can execute commands in an entry until GRUB encounters lock.

You can also use the command password instead of lock. In this case the boot process will ask for the password and stop if it was entered incorrectly. Since the password takes its own PASSWORD argument this is useful if you want different passwords for different entries.

grub image file

GRUB image files

GRUB consists of several images: two essential stages, optional stages called Stage 1.5, one image for bootable CD-ROM, and two network boot images. Here is a short overview of them. See section D. Hacking GRUB, for more details.

`stage1'
This is an essential image used for booting up GRUB. Usually, this is embedded in an MBR or the boot sector of a partition. Because a PC boot sector is 512 bytes, the size of this image is exactly 512 bytes.

All `stage1' must do is to load Stage 2 or Stage 1.5 from a local disk. Because of the size restriction, `stage1' encodes the location of Stage 2 (or Stage 1.5) in a block list format, so it never understand any filesystem structure.

`stage2'
This is the core image of GRUB. It does everything but booting up itself. Usually, this is put in a filesystem, but that is not required.

`e2fs_stage1_5'
`fat_stage1_5'
`ffs_stage1_5'
`jfs_stage1_5'
`minix_stage1_5'
`reiserfs_stage1_5'
`vstafs_stage1_5'
`xfs_stage1_5'

These are called Stage 1.5, because they serve as a bridge between `stage1' and `stage2', that is to say, Stage 1.5 is loaded by Stage 1 and Stage 1.5 loads Stage 2. The difference between `stage1' and `*_stage1_5' is that the former doesn't understand any filesystem while the latter understands one filesystem (e.g. `e2fs_stage1_5' understands ext2fs). So you can move the Stage 2 image to another location safely, even after GRUB has been installed.

While Stage 2 cannot generally be embedded in a fixed area as the size is so large, Stage 1.5 can be installed into the area right after an MBR, or the boot loader area of a ReiserFS or a FFS.

`stage2_eltorito'
This is a boot image for CD-ROMs using the no emulation mode in El Torito specification. This is identical to Stage 2, except that this boots up without Stage 1 and sets up a special drive `(cd)'.

`nbgrub'
This is a network boot image for the Network Image Proposal used by some network boot loaders, such as Etherboot. This is mostly the same as Stage 2, but it also sets up a network and loads a configuration file from the network.

`pxegrub'
This is another network boot image for the Preboot Execution Environment used by several Netboot ROMs. This is identical to `nbgrub', except for the format.

filesystem syntax and semantics

Filesystem syntax and semantics

GRUB uses a special syntax for specifying disk drives which can be accessed by BIOS. Because of BIOS limitations, GRUB cannot distinguish between IDE, ESDI, SCSI, or others. You must know yourself which BIOS device is equivalent to which OS device. Normally, that will be clear if you see the files in a device or use the command find (see section 13.3.11 find).

How to specify devices

The device syntax is like this:

(device[,part-num][,bsd-subpart-letter])

`[]' means the parameter is optional. device should be either `fd' or `hd' followed by a digit, like `fd0'. But you can also set device to a hexadecimal or a decimal number which is a BIOS drive number, so the following are equivalent:

(hd0)
(0x80)
(128)

part-num represents the partition number of device, starting from zero for primary partitions and from four for extended partitions, and bsd-subpart-letter represents the BSD disklabel subpartition, such as `a' or `e'.

A shortcut for specifying BSD subpartitions is (device,bsd-subpart-letter), in this case, GRUB searches for the first PC partition containing a BSD disklabel, then finds the subpartition bsd-subpart-letter. Here is an example:

(hd0,a)

The syntax `(hd0)' represents using the entire disk (or the MBR when installing GRUB), while the syntax `(hd0,0)' represents using the first partition of the disk (or the boot sector of the partition when installing GRUB).

If you enabled the network support, the special drive, `(nd)', is also available. Before using the network drive, you must initialize the network. See section 6. Downloading OS images from a network, for more information.

If you boot GRUB from a CD-ROM, `(cd)' is available. See section 3.4 Making a GRUB bootable CD-ROM, for details.

How to specify files

There are two ways to specify files, by absolute file name and by block list.

An absolute file name resembles a Unix absolute file name, using `/' for the directory separator (not `\' as in DOS). One example is `(hd0,0)/boot/grub/menu.lst'. This means the file `/boot/grub/menu.lst' in the first partition of the first hard disk. If you omit the device name in an absolute file name, GRUB uses GRUB's root device implicitly. So if you set the root device to, say, `(hd1,0)' by the command root (see section 13.3.31 root), then /boot/kernel is the same as (hd1,0)/boot/kernel.

How to specify block lists

A block list is used for specifying a file that doesn't appear in the filesystem, like a chainloader. The syntax is [offset]+length[,[offset]+length].... Here is an example:

0+100,200+1,300+300

This represents that GRUB should read blocks 0 through 99, block 200, and blocks 300 through 599. If you omit an offset, then GRUB assumes the offset is zero.

Like the file name syntax (see section 11.2 How to specify files), if a blocklist does not contain a device name, then GRUB uses GRUB's root device. So (hd0,1)+1 is the same as +1 when the root device is `(hd0,1)'.

grub's user interface

GRUB's user interface

GRUB has both a simple menu interface for choosing preset entries from a configuration file, and a highly flexible command-line for performing any desired combination of boot commands.

GRUB looks for its configuration file as soon as it is loaded. If one is found, then the full menu interface is activated using whatever entries were found in the file. If you choose the command-line menu option, or if the configuration file was not found, then GRUB drops to the command-line interface.

The flexible command-line interface

The command-line interface provides a prompt and after it an editable text area much like a command-line in Unix or DOS. Each command is immediately executed after it is entered(8). The commands (see section 13.3 The list of command-line and menu entry commands) are a subset of those available in the configuration file, used with exactly the same syntax.

Cursor movement and editing of the text on the line can be done via a subset of the functions available in the Bash shell:

C-f
PC right key
Move forward one character.

C-b
PC left key
Move back one character.

C-a
HOME
Move to the start of the line.

C-e
END
Move the the end of the line.

C-d
DEL
Delete the character underneath the cursor.

C-h
BS
Delete the character to the left of the cursor.

C-k
Kill the text from the current cursor position to the end of the line.

C-u
Kill backward from the cursor to the beginning of the line.

C-y
Yank the killed text back into the buffer at the cursor.

C-p
PC up key
Move up through the history list.

C-n
PC down key
Move down through the history list.

When typing commands interactively, if the cursor is within or before the first word in the command-line, pressing the TAB key (or C-i) will display a listing of the available commands, and if the cursor is after the first word, the TAB will provide a completion listing of disks, partitions, and file names depending on the context. Note that to obtain a list of drives, one must open a parenthesis, as root (.

Note that you cannot use the completion functionality in the TFTP filesystem. This is because TFTP doesn't support file name listing for the security.

The simple menu interface

The menu interface is quite easy to use. Its commands are both reasonably intuitive and described on screen.

Basically, the menu interface provides a list of boot entries to the user to choose from. Use the arrow keys to select the entry of choice, then press RET to run it. An optional timeout is available to boot the default entry (the first one if not set), which is aborted by pressing any key.

Commands are available to enter a bare command-line by pressing c (which operates exactly like the non-config-file version of GRUB, but allows one to return to the menu if desired by pressing ESC) or to edit any of the boot entries by pressing e.

If you protect the menu interface with a password (see section 9. Protecting your computer from cracking), all you can do is choose an entry by pressing RET, or press p to enter the password.

Editing a menu entry

The menu entry editor looks much like the main menu interface, but the lines in the menu are individual commands in the selected entry instead of entry names.

If an ESC is pressed in the editor, it aborts all the changes made to the configuration entry and returns to the main menu interface.

When a particular line is selected, the editor places the user in a special version of the GRUB command-line to edit that line. When the user hits RET, GRUB replaces the line in question in the boot entry with the changes (unless it was aborted via ESC, in which case the changes are thrown away).

If you want to add a new line to the menu entry, press o if adding a line after the current line or press O if before the current line.

To delete a line, hit the key d. Although GRUB unfortunately does not support undo, you can do almost the same thing by just returning to the main menu.

The hidden menu interface

When your terminal is dumb or you request GRUB to hide the menu interface explicitly with the command hiddenmenu (see section 13.1.3 hiddenmenu), GRUB doesn't show the menu interface (see section 12.2 The simple menu interface) and automatically boots the default entry, unless interrupted by pressing ESC.

When you interrupt the timeout and your terminal is dumb, GRUB falls back to the command-line interface (see section 12.1 The flexible command-line interface).

command-line & menu entry commands

These commands are usable in the command-line and in menu entries. If you forget a command, you can run the command help (see section 15 help).

1 blocklist

Command: blocklist file
Print the block list notation of the file file. See section 11.3 How to specify block lists.

2 boot

Command: boot
Boot the OS or chain-loader which has been loaded. Only necessary if running the fully interactive command-line (it is implicit at the end of a menu entry).

3 cat

Command: cat file
Display the contents of the file file. This command may be useful to remind you of your OS's root partition:

grub> cat /etc/fstab

4 chainloader

Command: chainloader [`--force'] file
Load file as a chain-loader. Like any other file loaded by the filesystem code, it can use the blocklist notation to grab the first sector of the current partition with `+1'. If you specify the option `--force', then load file forcibly, whether it has a correct signature or not. This is required when you want to load a defective boot loader, such as SCO UnixWare 7.1 (see section 4.2.7 SCO UnixWare).

5 cmp

Command: cmp file1 file2
Compare the file file1 with the file file2. If they differ in size, print the sizes like this:

Differ in size: 0x1234 [foo], 0x4321 [bar]

If the sizes are equal but the bytes at an offset differ, then print the bytes like this:

Differ at the offset 777: 0xbe [foo], 0xef [bar]

If they are completely identical, nothing will be printed.

6 configfile

Command: configfile file
Load file as a configuration file.

7 debug

Command: debug
Toggle debug mode (by default it is off). When debug mode is on, some extra messages are printed to show disk activity. This global debug flag is mainly useful for GRUB developers when testing new code.

8 displayapm

Command: displayapm
Display APM BIOS information.

9 displaymem

Command: displaymem
Display what GRUB thinks the system address space map of the machine is, including all regions of physical RAM installed. GRUB's upper/lower memory display uses the standard BIOS interface for the available memory in the first megabyte, or lower memory, and a synthesized number from various BIOS interfaces of the memory starting at 1MB and going up to the first chipset hole for upper memory (the standard PC upper memory interface is limited to reporting a maximum of 64MB).

10 embed

Command: embed stage1_5 device
Embed the Stage 1.5 stage1_5 in the sectors after the MBR if device is a drive, or in the boot loader area if device is a FFS partition or a ReiserFS partition.(9) Print the number of sectors which stage1_5 occupies, if successful.

Usually, you don't need to run this command directly. See section 34 setup.

11 find

Command: find filename
Search for the file name filename in all mountable partitions and print the list of the devices which contain the file. The file name filename should be an absolute file name like /boot/grub/stage1.

12 fstest

Command: fstest
Toggle filesystem test mode. Filesystem test mode, when turned on, prints out data corresponding to all the device reads and what values are being sent to the low-level routines. The format is `<partition-offset-sector, byte-offset, byte-length>' for high-level reads inside a partition, and `[disk-offset-sector]' for low-level sector requests from the disk. Filesystem test mode is turned off by any use of the install (see section 18 install) or testload (see section 35 testload) commands.

13 geometry

Command: geometry drive [cylinder head sector [total_sector]]
Print the information for the drive drive. In the grub shell, you can set the geometry of the drive arbitrarily. The number of cylinders, the number of heads, the number of sectors and the number of total sectors are set to CYLINDER, HEAD, SECTOR and TOTAL_SECTOR, respectively. If you omit TOTAL_SECTOR, then it will be calculated based on the C/H/S values automatically.

14 halt

Command: halt `--no-apm'
The command halts the computer. If the `--no-apm' option is specified, no APM BIOS call is performed. Otherwise, the computer is shut down using APM.

15 help

Command: help `--all' [pattern ...]
Display helpful information about builtin commands. If you do not specify pattern, this command shows short descriptions of most of available commands. If you specify the option `--all' to this command, short descriptions of rarely used commands (such as 35 testload) are displayed as well.

If you specify any patterns, it displays longer information about each of the commands which match those patterns.

16 impsprobe

Command: impsprobe
Probe the Intel Multiprocessor Specification 1.1 or 1.4 configuration table and boot the various CPUs which are found into a tight loop. This command can be used only in the Stage 2, but not in the grub shell.

17 initrd

Command: initrd file ...
Load an initial ramdisk for a Linux format boot image and set the appropriate parameters in the Linux setup area in memory. See also 4.2.2 GNU/Linux.

18 install

Command: install [`--force-lba'] [`--stage2=os_stage2_file'] stage1_file [`d'] dest_dev stage2_file [addr] [`p'] [config_file] [real_config_file]
This command is fairly complex, and you should not use this command unless you are familiar with GRUB. Use setup (see section 34 setup) instead.

In short, it will perform a full install presuming the Stage 2 or Stage 1.5(10) is in its final install location.

In slightly more detail, it will load stage1_file, validate that it is a GRUB Stage 1 of the right version number, install in it a blocklist for loading stage2_file as a Stage 2. If the option `d' is present, the Stage 1 will always look for the actual disk stage2_file was installed on, rather than using the booting drive. The Stage 2 will be loaded at address addr, which must be `0x8000' for a true Stage 2, and `0x2000' for a Stage 1.5. If addr is not present, GRUB will determine the address automatically. It then writes the completed Stage 1 to the first block of the device dest_dev. If the options `p' or config_file are present, then it reads the first block of stage2, modifies it with the values of the partition stage2_file was found on (for `p') or places the string config_file into the area telling the stage2 where to look for a configuration file at boot time. Likewise, if real_config_file is present and stage2_file is a Stage 1.5, then the Stage 2 config_file is patched with the configuration file name real_config_file. This command preserves the DOS BPB (and for hard disks, the partition table) of the sector the Stage 1 is to be installed into.

Caution: Several buggy BIOSes don't pass a booting drive properly when booting from a hard disk drive. Therefore, you will unfortunately have to specify the option `d', whether your Stage2 resides at the booting drive or not, if you have such a BIOS. We know these are defective in this way:

Fujitsu LifeBook 400 BIOS version 31J0103A

HP Vectra XU 6/200 BIOS version GG.06.11

Caution2: A number of BIOSes don't return a correct LBA support bitmap even if they do have the support. So GRUB provides a solution to ignore the wrong bitmap, that is, the option `--force-lba'. Don't use this option if you know that your BIOS doesn't have LBA support.

Caution3: You must specify the option `--stage2' in the grub shell, if you cannot unmount the filesystem where your stage2 file resides. The argument should be the file name in your operating system.

19 ioprobe

Command: ioprobe drive
Probe I/O ports used for the drive drive. This command will list the I/O ports on the screen. For technical information, See section D. Hacking GRUB.

20 kernel

Command: kernel [`--type=type'] [`--no-mem-option'] file ...
Attempt to load the primary boot image (Multiboot a.out or ELF, Linux zImage or bzImage, FreeBSD a.out, NetBSD a.out, etc.) from file. The rest of the line is passed verbatim as the kernel command-line. Any modules must be reloaded after using this command.

This command also accepts the option `--type' so that you can specify the kernel type of file explicitly. The argument type must be one of these: `netbsd', `freebsd', `openbsd', `linux', `biglinux', and `multiboot'. However, you need to specify it only if you want to load a NetBSD ELF kernel, because GRUB can automatically determine a kernel type in the other cases, quite safely.

The option `--no-mem-option' is effective only for Linux. If the option is specified, GRUB doesn't pass the option `mem=' to the kernel. This option is implied for Linux kernels 2.4.18 and newer.

21 lock

Command: lock
Prevent normal users from executing arbitrary menu entries. You must use the command password if you really want this command to be useful (see section 13.2.10 password).

This command is used in a menu, as shown in this example:

title This entry is too dangerous to be executed by normal users
lock
root (hd0,a)
kernel /no-security-os

See also 9. Protecting your computer from cracking.

22 makeactive

Command: makeactive
Set the active partition on the root disk to GRUB's root device. This command is limited to primary PC partitions on a hard disk.

23 map

Command: map to_drive from_drive
Map the drive from_drive to the drive to_drive. This is necessary when you chain-load some operating systems, such as DOS, if such an OS resides at a non-first drive. Here is an example:

grub> map (hd0) (hd1)
grub> map (hd1) (hd0)

The example exchanges the order between the first hard disk and the second hard disk. See also 4.2.6 DOS/Windows.

24 md5crypt

Command: md5crypt
Prompt to enter a password, and encrypt it in MD5 format. The encrypted password can be used with the command password (see section 13.2.10 password). See also 9. Protecting your computer from cracking.

25 module

Command: module file ...
Load a boot module file for a Multiboot format boot image (no interpretation of the file contents are made, so the user of this command must know what the kernel in question expects). The rest of the line is passed as the module command-line, like the kernel command. You must load a Multiboot kernel image before loading any module. See also 26 modulenounzip.

26 modulenounzip

Command: modulenounzip file ...
The same as module (see section 25 module), except that automatic decompression is disabled.

27 pause

Command: pause message ...
Print the message, then wait until a key is pressed. Note that placing ^G (ASCII code 7) in the message will cause the speaker to emit the standard beep sound, which is useful when prompting the user to change floppies.

28 quit

Command: quit
Exit from the grub shell grub (see section 15. Invoking the grub shell). This command can be used only in the grub shell.

29 reboot

Command: reboot
Reboot the computer.

30 read

Command: read addr
Read a 32-bit value from memory at address addr and display it in hex format.

31 root

Command: root device [hdbias]
Set the current root device to the device device, then attempt to mount it to get the partition size (for passing the partition descriptor in ES:ESI, used by some chain-loaded boot loaders), the BSD drive-type (for booting BSD kernels using their native boot format), and correctly determine the PC partition where a BSD sub-partition is located. The optional hdbias parameter is a number to tell a BSD kernel how many BIOS drive numbers are on controllers before the current one. For example, if there is an IDE disk and a SCSI disk, and your FreeBSD root partition is on the SCSI disk, then use a `1' for hdbias.

See also 32 rootnoverify.

32 rootnoverify

Command: rootnoverify device [hdbias]
Similar to root (see section 31 root), but don't attempt to mount the partition. This is useful for when an OS is outside of the area of the disk that GRUB can read, but setting the correct root device is still desired. Note that the items mentioned in root above which derived from attempting the mount will not work correctly.

33 savedefault

Command: savedefault
Save the current menu entry as a default entry. Here is an example:

default saved
timeout 10

title GNU/Linux
root (hd0,0)
kernel /boot/vmlinuz root=/dev/sda1 vga=ext
initrd /boot/initrd
savedefault

title FreeBSD
root (hd0,a)
kernel /boot/loader
savedefault

With this configuration, GRUB will choose the entry booted previously as the default entry. See also 13.1.1 default.

34 setup

Command: setup [`--force-lba'] [`--stage2=os_stage2_file'] [`--prefix=dir'] install_device [image_device]
Set up the installation of GRUB automatically. This command uses the more flexible command install (see section 18 install) in the backend and installs GRUB into the device install_device. If image_device is specified, then find the GRUB images (see section 10. GRUB image files) in the device image_device, otherwise use the current root device, which can be set by the command root. If install_device is a hard disk, then embed a Stage 1.5 in the disk if possible.

The option `--prefix' specifies the directory under which GRUB images are put. If it is not specified, GRUB automatically searches them in `/boot/grub' and `/grub'.

The options `--force-lba' and `--stage2' are just passed to install if specified. See section 18 install, for more information.

35 testload

Command: testload file
Read the entire contents of file in several different ways and compare them, to test the filesystem code. The output is somewhat cryptic, but if no errors are reported and the final `i=X, filepos=Y' reading has X and Y equal, then it is definitely consistent, and very likely works correctly subject to a consistent offset error. If this test succeeds, then a good next step is to try loading a kernel.

36 testvbe

Command: testvbe mode
Test the VESA BIOS EXTENSION mode mode. This command will switch your video card to the graphics mode, and show an endless animation. Hit any key to return. See also 38 vbeprobe.

37 uppermem

Command: uppermem kbytes
Force GRUB to assume that only kbytes kilobytes of upper memory are installed. Any system address range maps are discarded.

Caution: This should be used with great caution, and should only be necessary on some old machines. GRUB's BIOS probe can pick up all RAM on all new machines the author has ever heard of. It can also be used for debugging purposes to lie to an OS.

38 vbeprobe

Command: vbeprobe [mode]
Probe VESA BIOS EXTENSION information. If the mode mode is specified, show only the information about mode. Otherwise, this command lists up available VBE modes on the screen. See also 36 testvbe.

making loadable kernel modules

An LKM lives in a single ELF object file (normally named like "serial.o"). You typically keep all your LKM object files in a particular directory (near your base kernel image makes sense). When you use the insmod program to insert an LKM into the kernel, you give the name of that object file.

For the LKMs that are part of Linux, you build them as part of the same kernel build process that generates the base kernel image. See the README file in the Linux source tree. In short, after you make the base kernel image with a command such as make zImage, you will make all the LKMs with the command

make modules

This results in a bunch of LKM object files (*.o) throughout the Linux source tree. (In older versions of Linux, there would be symbolic links in the modules directory of the Linux source tree pointing to all those LKM object files). These LKMs are ready to load, but you probably want to install them in some appropriate directory. The conventional place is described in Section 5.6. The command make modules_install will copy them all over to the conventional locations.

Part of configuring the Linux kernel (at build time) is choosing which parts of the kernel to bind into the base kernel and which parts to generate as separate LKMs. In the basic question-and-answer configuration (make config), you are asked, for each optional part of the kernel, whether you want it bound into the kernel (a "Y" response), created as an LKM (an "M" response),
or just skipped completely (an "N" response). Other configuration methods are similar.

As explained in Section 2.3, you should have only the bare minimum bound into the base kernel. And only skip completely the parts that you're sure you'll never want. There is very little to lose by building an LKM that you won't use. Some compile time, some disk space, some chance of a problem in the code killing the kernel build. That's it.

As part of the configuration dialog you also must choose whether to use symbol versioning or not. This choice affects building both the base kernel and the LKMs and it is crucial you get it right. See Section 6.

LKMs that are not part of Linux (i.e. not distributed with the Linux kernel) have their own build procedures which I will not cover. The goal of any such procedure, though, is always to end up with an ELF object file.

You don't necessarily have to rebuild all your LKMs and your base kernel image at the same time (e.g. you could build just the base kernel and use LKMs you built earlier with it) but it is always a good idea.

lvm commands

Initializing disks or disk partitions

To use LVM, partitions and whole disks must first be converted into physical volumes (PVs) using the command. For example, to convert pvcreate/dev/hda and /dev/hdb into PVs use the following commands:

pvcreate /dev/hda
pvcreate /dev/hdb

If a Linux partition is to be converted make sure that it is given partition type 0x8E using fdisk, then use pvcreate:

pvcreate /dev/hda1

Creating a volume group

Once you have one or more physical volumes created, you can create a volume group from these PVs using the vgcreate command. The following command:

vgcreate  volume_group_one /dev/hda /dev/hdb

creates a new VG called volume_group_one with two disks, /dev/hda and /dev/hdb, and 4 MB PEs. If both /dev/hda and /dev/hdb are 128 GB in size, then the VG volume_group_one will have a total of 2**16 physical extents that can be allocated to logical volumes.

Additional PVs can be added to this volume group using the vgextend command. The following commands convert /dev/hdc into a PV and then adds that PV to volume_group_one:

pvcreate /dev/hdc
vgextend volume_group_one /dev/hdc

This same PV can be removed from volume_group_one by the vgreduce command:

vgreduce volume_group_one /dev/hdc

Note that any logical volumes using physical extents from PV /dev/hdc will be removed as well. This raises the issue of how we create an LV within a volume group in the first place.

Creating a logical volume

We use the lvcreate command to create a new logical volume using the free physical extents in the VG pool. Continuing our example using VG volume_group_one (with two PVs /dev/hda and /dev/hdb and a total capacity of 256 GB), we could allocate nearly all the PEs in the volume group to a single linear LV called logical_volume_one with the following LVM command:

lvcreate -n logical_volume_one   --size 255G volume_group_one 

Instead of specifying the LV size in GB we could also specify it in terms of logical extents. First we use vgdisplay to determine the number of PEs in the volume_group_one:

vgdisplay volume_group_one | grep "Total PE"

which returns

Total PE   65536

Then the following lvcreate command will create a logical volume with 65536 logical extents and fill the volume group completely:

lvcreate -n logical_volume_one  -l 65536 volume_group_one

To create a 1500MB linear LV named logical_volume_one and its block device special file /dev/volume_group_one/logical_volume_one use the following command:

lvcreate -L1500 -n logical_volume_one volume_group_one

The lvcreate command uses linear mappings by default.

Striped mappings can also be created with lvcreate. For example, to create a 255 GB large logical volume with two stripes and stripe size of 4 KB the following command can be used:

lvcreate -i2 -I4 --size 255G -n logical_volume_one_striped volume_group_one

It is possible to allocate a logical volume from a specific physical volume in the VG by specifying the PV or PVs at the end of the lvcreate command. If you want the logical volume to be allocated from a specific physical volume in the volume group, specify the PV or PVs at the end of the lvcreate command line. For example, this command:

lvcreate -i2 -I4 -L128G -n logical_volume_one_striped volume_group_one /dev/hda /dev/hdb 

creates a striped LV named logical_volume_one that is striped across two PVs (/dev/hda and /dev/hdb) with stripe size 4 KB and 128 GB in size.

An LV can be removed from a VG through the lvremove command, but first the LV must be unmounted:

umount /dev/volume_group_one/logical_volume_one
lvremove /dev/volume_group_one/logical_volume_one

Note that LVM volume groups and underlying logical volumes are included in the device special file directory tree in the /dev directory with the following layout:

/dev// 
so that if we had two volume groups myvg1 and myvg2 and each with three logical volumes named lv01, lv02, lv03, six device special files would be created:

/dev/myvg1/lv01
/dev/myvg1/lv02
/dev/myvg1/lv03
/dev/myvg2/lv01
/dev/myvg2/lv02
/dev/myvg2/lv03

Extending a logical volume

An LV can be extended by using the lvextend command. You can specify either an absolute size for the extended LV or how much additional storage you want to add to the LVM. For example:

lvextend -L120G /dev/myvg/homevol

will extend LV /dev/myvg/homevol to 12 GB, while

lvextend -L+10G /dev/myvg/homevol

will extend LV /dev/myvg/homevol by an additional 10 GB. Once a logical volume has been extended, the underlying file system can be expanded to exploit the additional storage now available on the LV. With Red Hat Enterprise Linux 4, it is possible to expand both the ext3fs and GFS file systems online, without bringing the system down. (The ext3 file system can be shrunk or expanded offline using the ext2resize command.) To resize ext3fs, the following command

ext2online /dev/myvg/homevol

will extend the ext3 file system to completely fill the LV, /dev/myvg/homevol, on which it resides.

The file system specified by device (partition, loop device, or logical volume) or mount point must currently be mounted, and it will be enlarged to fill the device, by default. If an optional size parameter is specified, then this size will be used instead.

Differences between LVM1 and LVM2

The new release of LVM, LVM 2, is available only on Red Hat Enterprise Linux 4 and later kernels. It is upwardly compatible with LVM 1 and retains the same command line interface structure. However it uses a new, more scalable and resilient metadata structure that allows for transactional metadata updates (that allow quick recovery after server failures), very large numbers of devices, and clustering. For Enterprise Linux servers deployed in mission-critical environments that require high availability, LVM2 is the right choice for Linux volume management. Table 1. A comparison of LVM 1 and LVM 2 summarizes the differences between LVM1 and LVM2 in features, kernel support, and other areas.

lvm -help

dumpconfig- Dump active configuration
formats- List available metadata formats
help--Display help for commands
lvchange- Change the attributes of logical volume(s)
lvconvert- Change logical volume layout
lvcreate- Create a logical volume
lvdisplay- Display information about a logical volume
lvextend- Add space to a logical volume
lvmchange- With the device mapper, this is obsolete and does nothing.
lvmdiskscan-List devices that may be used as physical volumes
lvmsadc- Collect activity data
lvmsar- Create activity report
lvreduce- Reduce the size of a logical volume
lvremove- Remove logical volume(s) from the system
lvrename- Rename a logical volume
lvresize- Resize a logical volume
lvs-- Display information about logical volumes
lvscan- List all logical volumes in all volume groups
pvchange- Change attributes of physical volume(s)
pvresize- Resize physical volume(s)
pvck--Check the consistency of physical volume(s)
pvcreate- Initialize physical volume(s) for use by LVM
pvdata- Display the on-disk metadata for physical volume(s)
pvdisplay- Display various attributes of physical volume(s)
pvmove- Move extents from one physical volume to another
pvremove- Remove LVM label(s) from physical volume(s)
pvs-- Display information about physical volumes
pvscan- List all physical volumes
segtypes- List available segment types
vgcfgbackup-Backup volume group configuration(s)
vgcfgrestore Restore volume group configuration
vgchange- Change volume group attributes
vgck--Check the consistency of volume group(s)
vgconvert- Change volume group metadata format
vgcreate- Create a volume group
vgdisplay- Display volume group information
vgexport- Unregister volume group(s) from the system
vgextend- Add physical volumes to a volume group
vgimport- Register exported volume group with system
vgmerge- Merge volume groups
vgmknodes- Create the special files for volume group devices in /dev
vgreduce- Remove physical volume(s) from a volume group
vgremove- Remove volume group(s)
vgrename- Rename a volume group
vgs-- Display information about volume groups
vgscan- Search for all volume groups
vgsplit- Move physical volumes into a new or existing volume group
version- Display software and driver version information

cryptsetup-luks on fedora: step 2, 3

cryptsetup-luks on fedora: step  2,  3
 
Remove the user whose Home directory we just backed up
We will be recreating the unpriviledged user (jmaher) after we have encrypted and re-mounted our /home directory, so we should clean things up first and remove that account:
# userdel jmaher
 
Step 3: Get the correct cryptsetup version
 
You need the version of cryptsetup with luks enabled. You can determine if the correct version of cryptsetup is install using the command:
 
# cryptsetup --help
 
You should see "cryptsetup-luks" displayed near the top of the output.
If you do not have cryptsetup, you can install it using yum (assuming yum has been properly configured):
 
# yum -y install cryptsetup-luks
 
The -y option will assume that the answer is yes to any question that would be presented during the execution of yum.

cryptsetup-luks on fedora: step 4

cryptsetup-luks on fedora: step 4

Step 4a: (Optional) Check the hard disk for errors and fill it with random data.

(Much of the following was shamelessly lifted from William Owen Smith's HOWTO: EncryptedDeviceUsingLUKS.)

It's probably a good idea to check for errors on the partition where /home will be mounted. Not only is this good practice, but modern hard disks contain a few 'spare' sectors, and if they detect errors in reading, they can silently replace the bad sector with a backup sector (this is invisible to the OS). So writing and reading the entire disk before you start should allow this to happen.

We will now use the information in /etc/fstab to determine the partition that is currently mounted on /home. (This partition may be represented as a logical volume, as it is in the example below.) You should see a line in /etc/fstab that is similar to the following:

/dev/vg0/home /home ext3 defaults 1 2

The above line indicates that partition /dev/vg0/home, which is actually a logical volume, is currently mounted on /home. We will use this partition as our physical device. We need to unmount this device in order the proceed with the next steps:

# umount /home

It's good to fill an encrypted disk with initial random data. This makes breaking the passphrase so much harder. The below method is sufficient for a casual attack but is not 'random enough' to defeat sophisticated cryptographers. If you need protection against sophisticated cryptanaylsis, use the '/dev/urandom' method shown in Step 4b.

The following command will perform a disk check and fill the disk with random data at the same time. Read the man page for more details on this command:

# /sbin/badblocks -c 10240 -s -w -t random -v /dev/vg0/home
(wait several hours...)
Checking for bad blocks in read-write mode
From block 0 to 295360984
done
Reading and comparing: done
Pass completed, 0 bad blocks found.
#

The -c option will test 10,240 blocks at a time. The -s option will show the progress of the scan by writing out the block numbers as they are checked. The -w option scans for bad blocks by writing some patterns (0xaa, 0x55, 0xff, 0x00) on every block of the device, reading every block and comparing the contents. The -t options specifies that each scanned block should be filled with a random bit pattern. The -v option indicates verbose mode. The physical device (logical volume) is /dev/vg0/home.

This will take some time. On William Owen Smith's USB-attached 300Gb disk it took around 8 hours. Phase 1 will write random data to the disk, phase 2 will read it back and verify it.

Step 4b: (Optional) Fill the disk with random data

If you didn't do Step 4a (or you left out the -t random option), do Step 4b. This will take a long time, because generating good quality random data is very CPU intensive. However, this method is 'more random' (and more secure) than the primitive random number generator included in 'badblocks', above.

One minor advantage to the method in Step xa above is that it has a progress indicator, while "dd" only shows its progress when a USR1 signal is sent to it ("kill -USR1 `pidof dd`").

# dd if=/dev/urandom of=/dev/vg0/home
(wait several hours...)

cryptsetup-luks on fedora: step 5, 6

cryptsetup-luks on fedora: step 5, 6

Step 5: Initialize a LUKS partition and set the initial key

This step establishes the mapping between physical partitions and logical partitions.

In this HOWTO, our physical partition will actually be a logical volume. By default, when installing Fedora Core 5, a volume group and logical volumes within the volume group are created. The volume group is called VolGroup00?, and the logical volumes are called LogVol00?, LogVol01?, etc, for each of the partitions. However, in this HOWTO, our volume group will be called vg0, and our logical volume that will eventually get mounted to /home will be called home. So, the full path of the physical partition that will be mounted on /home (when we are done) is /dev/vg0/home. (Your device path will likely be different, but you need to identify the device that is currently mounted to /home.)

With that said, let's use the following command to initialize a LUKS partition and set the initial key using a passphrase (note, this will wipe out all data on the /home partition):

# cryptsetup --verbose --verify-passphrase luksFormat /dev/vg0/home

WARNING!
========
This will overwrite data on /dev/vg0/home irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: (enter your passphrase, and write it down somewhere!)
Verify passphrase: (repeat passphrase)

Step 6: Create a mapping between physical and logical partitions

# cryptsetup luksOpen /dev/vg0/home home
Enter LUKS passphrase:
#

If all is well, you now have a special file called /dev/mapper/home. This is what you will mount on /home. Verify that the file was created:

# ls -l /dev/mapper/

total 0
crw------- 1 root root 10, 63 May 24 06:52 control
brw-rw---- 1 root disk 253, 4 May 24 10:54 home
brw-rw---- 1 root disk 253, 1 May 24 06:52 vg0-home
brw-rw---- 1 root disk 253, 0 May 24 10:53 vg0-root
brw-rw---- 1 root disk 253, 2 May 24 06:52 vg0-swap

Notice the other logical volumes (vg0-home, vg0-root, and vg0-swap) that were created when Fedora Core 5 was installed. (Note, the names of these volumes were changed by me during the installation. The were originally VolGroup00-LogVol00?, VolGroup00-LogVol01?, etc.) The fact that you are using logical volumes (like /dev/vg0/home) as physical devices can be confusing. It may help to remember that when we refer to physical devices we use devices located in the volume group directory (example: /dev/vg0), and when we refer to logical devices we use devices located in /dev/mapper (i.e., they have been mapped are are ready to use). (Okay, yes, it's confusing that the physical devices in /dev/vg0 are also listed as logical devices in /dev/mapper. Try to ignore them.)

cryptsetup-luks on fedora: step 7, 8

cryptsetup-luks on fedora: step 7, 8

Step 7: Create a filesystem on the new logical partition

For this HOWTO, we make an ext3 file system on /dev/mapper/home using the following commands:

# /sbin/mkfs.ext3 -j -m 1 /dev/mapper/home

(wait several minutes...)
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
36634624 inodes, 73258400 blocks
732584 blocks (1.00%) reserved for the super user
First data block=0
2236 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
#

(Note, the above output was borrowed from William Owen Smith's HOWTO: "EncryptedDeviceUsingLUKS".)

Step 8: Mount the filesystem

Mount your new logical device /dev/mapper/home to /home.

# mount /dev/mapper/home /home

View the file system's disk usage to verify that it worked:

# df -h /dev/mapper/home
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/home 4.0G 80M 3.8G 3% /home

cryptsetup-luks on fedora: step 9, 10

cryptsetup-luks on fedora: step 9, 10

Step 9: Restore the user's Home directory

Re-create the unpriviledged user:

# useradd -m jmaher
# passwd jmaher
Changing password for user jmaher.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
#

The -m option create's the user's home directory using the files and directories in /etc/skel as a template.

Now we need to copy MOST of the user's backed-up files back to the user's Home directory. I say MOST because I have found that copying all of the files back to the user's Home directory will break the use of the Home directory for that user. I have not investigated this, so someone else may want to comment as to the reason for this. Basically, I found it safe to copy all non-hidden files and directories back to the /home/jmaher using the following command:

# /bin/cp -r --preserve /root/jmaher/* /home/jmaher

The -r options allows recursion of subdirectories to occur, and the --preserve option preserves permissions and ownership of the files and directories.

I would recommend selectively copying hidden files and directories for those applications you find most important. For example, I really wanted my Thunderbird, Firefox, and ssh settings to be restored, so I used the following commands:

# /bin/cp -r --preserve /root/jmaher/.thunderbird /home/jmaher
# /bin/cp -r --preserve /root/jmaher/.mozilla /home/jmaher
# /bin/cp -r --preserve /root/jmaher/.ssh /home/jmaher

If you had previously modified .bashrc, .bash_profile, or .bash_logout, then you may want to copy those files as well.

Don't reboot yet, but you should now be able to test your actions and log on as the unpriviledged user (jmaher) using the following command:

# su - jmaher

After confirming that you can log on as the unpriviledged user without errors indicating that the user's environment in /home is missing, log off as the unpriviledged user to return to root.

# logout

Step 10: Modify /etc/fstab

Some aspects of the boot sequence need to be changed, because the physical volume (/dev/vg0/home) that gets mounted to /home is encrypted and is no longer a recognizable file system as far as /bin/mount is concerned. Of course, if cryptsetup is used to open the device (using the command cryptsetup luksOpen /dev/vg0/home), then /bin/mount could see that the device has an ext3 file system, and the device can be mounted.

So here are the steps to do that.

Change the line in /etc/fstab that mounted the Home directory so that:
(a) the first field refers to /dev/mapper/home rather than /dev/vg0/home
(b) the fifth field no longer indicates that this device should be accessed by the dump command
(c) the six field no longer indicates that fsck should check this device at boot time.

In short, change the line that looks similar to this:

/dev/vg0/home /home ext3 defaults 1 2

and change it to this:

/dev/mapper/home /home ext3 defaults 0 0

cryptsetup-luks on fedora: step 11, 12 & 13

cryptsetup-luks on fedora: step 11, 12 & 13

Step 11: Create and modify luksopen script

Copy the wonderful script called luksopen (created by embro and modified by johnny) from http://www.saout.de/tikiwiki/tiki-index.php?page=luksopen, and paste it into a new file called /sbin/luksopen.

Modify the script as follows:

a. Change devArray variable from:
devArray=(/dev/hda7 /dev/hda10 /dev/hda11 /dev/hda13)
to:
devArray=(/dev/vg0/home)
(Remember, this is the physical device used for /home. Yours is probably different.)

b. Delete the entire mapArray variable line

c. Change mntArray variable from:
mntArray=(/tmp /mnt/bergen /mnt/trondheim /mnt/oslo)
to:
mntArray=(/home)

d. Replace the line that reads:
map=${mapArray[$i]}
with:
# assign last directory name of device name to $map variable
map_elements=`echo ${devArray[$i]} | sed -e 's/^\///' -e 's/\// /g'`
for e in $map_elements ; do map=$e ; done

e. Add ' answer' (no quotes) to the following line:
read -p "Next device in list is \"$dev\". Do you want to open and mount it? (y/N): "
so that it looks like this:
read -p "Next device in list is \"$dev\". Do you want to open and mount it? (y/N): " answer

f. Delete (or comment out) the three lines at the bottom of the script that immediately preceed the final 'done' command. So, if you choose to comment out those lines, they would look like this:
#if $j -le 0 || ! mount "/dev/mapper/$map" "$mnt" ; then
# echo "Failed to mount \"/dev/mapper/$map\" on \"$mnt\"" >&2
#fi

Step 12: Edit /etc/rc.d/rc.sysinit

During a boot, we now need to open (unlock) our encrypted partition (/dev/vg0/home) and map it to our logical volume (/dev/mapper/home) before the logical volume can be mounted on /home. Because we would like to have /home mounted during the boot (so we can log on as an unpriviledged user), we need to be prompted for the LUKS passphrase (established in Step 5) before any attempts are made to mount /dev/mapper/home to /home (as configured in /etc/fstab).

Now, one could assume that simply adding /sbin/luksopen to /etc/rc.d/rc.local would result in the necessary prompting for the LUKS passphrase; however, when booting to runlevel 5 I have experienced problems getting prompted. Therefore, modifying /etc/rc.d/rc.sysinit as follows will solve the prompting problem.

Modify /etc/rc.d/rc.sysinit to incude the following code . . .

# Run /sbin/luksopen to use cryptsetup to map /dev/vg0/home to /dev/mapper/home.
if -x /sbin/luksopen ; then
/sbin/luksopen
fi

. . . immediately before the code where /usr/bin/rhgb is called. In Fedora Core 5 you can place the above code immediately before the comment line that reads:

# Start the graphical boot, if necessary; /usr may not be mounted yet, so we
# may have to do this again after mounting

Step 13: REBOOT

What You Can Expect

The boot process will be essentially the same as before, but this time you will be prompted with the following text shortly after the boot begins:

Next device in list is /dev/vg0/home. Do you want to open and mount it? (y/N):

You need to type y <ENTER>, and you will then be prompted to enter your passphrase. If you enter your passphrase correctly, the device (/dev/vg0/home) that you encrypted and mapped in Steps 5 and 6 above will be mapped to /dev/mapper/home and mounted to /home. The boot process will complete, and you can log on as your unpriviledged user (jmaher).

(Written by John Maher, 24 May 2006)

cryptsetup-luks on fedora: 1 page

Take a Fedora Core 5 system and encrypt (using dm-crypt and LUKS) the partition that gets mounted on /home. Note that /home needs to be on its own partition, not on the / partition. Also, in words similar to those from night-shade, I have tested this with LVM2 devices containing nothing important. It worked for me but you are advised to have current working backups if the data matters to you. Because we are dealing with the /home partition, these instructions will also explain how to ensure that the /home partition is mounted during a boot.

Step 0: Log on as root

Because you will need to unmount /home, you must log on as root rather than su to root from an unpriveledged user account.

-=Step 1: Backup /home Presumably you would like to return to the same Home environment that you started with before you encrypted your /home partition. Therefore, you need to backup the contents of /home. (Be aware that these instructions will not necessary restore your Home environment EXACTLY as it was before you encrypted /home. Please read all of these instructions before proceeding, so that you are sure that this solution will work for you.) In this HOWTO, we will assume there is only one unpriveledged user (jmaher) on the system, so only /home/jmaher needs to be backed up. One way to back up this folder is to use the following commands:

# mkdir /root/jmaher
# /bin/cp -a /home/jmaher/.* /root/jmaher

The -a option is for archiving files and directories. It uses recursion and preserves the permissions of the files and directories.

Step 2: Remove the user whose Home directory we just backed up

We will be recreating the unpriviledged user (jmaher) after we have encrypted and re-mounted our /home directory, so we should clean things up first and remove that account:

# userdel jmaher

Step 3: Get the correct cryptsetup version

You need the version of cryptsetup with luks enabled. You can determine if the correct version of cryptsetup is install using the command:

# cryptsetup --help

You should see "cryptsetup-luks" displayed near the top of the output.

If you do not have cryptsetup, you can install it using yum (assuming yum has been properly configured):

# yum -y install cryptsetup-luks

The -y option will assume that the answer is yes to any question that would be presented during the execution of yum.

Step 4a: (Optional) Check the hard disk for errors and fill it with random data.

(Much of the following was shamelessly lifted from William Owen Smith's HOWTO: EncryptedDeviceUsingLUKS.)

It's probably a good idea to check for errors on the partition where /home will be mounted. Not only is this good practice, but modern hard disks contain a few 'spare' sectors, and if they detect errors in reading, they can silently replace the bad sector with a backup sector (this is invisible to the OS). So writing and reading the entire disk before you start should allow this to happen.

We will now use the information in /etc/fstab to determine the partition that is currently mounted on /home. (This partition may be represented as a logical volume, as it is in the example below.) You should see a line in /etc/fstab that is similar to the following:

/dev/vg0/home /home ext3 defaults 1 2

The above line indicates that partition /dev/vg0/home, which is actually a logical volume, is currently mounted on /home. We will use this partition as our physical device. We need to unmount this device in order the proceed with the next steps:

# umount /home

It's good to fill an encrypted disk with initial random data. This makes breaking the passphrase so much harder. The below method is sufficient for a casual attack but is not 'random enough' to defeat sophisticated cryptographers. If you need protection against sophisticated cryptanaylsis, use the '/dev/urandom' method shown in Step 4b.

The following command will perform a disk check and fill the disk with random data at the same time. Read the man page for more details on this command:

# /sbin/badblocks -c 10240 -s -w -t random -v /dev/vg0/home
(wait several hours...)
Checking for bad blocks in read-write mode
From block 0 to 295360984
done
Reading and comparing: done
Pass completed, 0 bad blocks found.
#

The -c option will test 10,240 blocks at a time. The -s option will show the progress of the scan by writing out the block numbers as they are checked. The -w option scans for bad blocks by writing some patterns (0xaa, 0x55, 0xff, 0x00) on every block of the device, reading every block and comparing the contents. The -t options specifies that each scanned block should be filled with a random bit pattern. The -v option indicates verbose mode. The physical device (logical volume) is /dev/vg0/home.

This will take some time. On William Owen Smith's USB-attached 300Gb disk it took around 8 hours. Phase 1 will write random data to the disk, phase 2 will read it back and verify it.

Step 4b: (Optional) Fill the disk with random data

If you didn't do Step 4a (or you left out the -t random option), do Step 4b. This will take a long time, because generating good quality random data is very CPU intensive. However, this method is 'more random' (and more secure) than the primitive random number generator included in 'badblocks', above.

One minor advantage to the method in Step xa above is that it has a progress indicator, while "dd" only shows its progress when a USR1 signal is sent to it ("kill -USR1 `pidof dd`").

# dd if=/dev/urandom of=/dev/vg0/home
(wait several hours...)

Step 5: Initialize a LUKS partition and set the initial key

This step establishes the mapping between physical partitions and logical partitions.

In this HOWTO, our physical partition will actually be a logical volume. By default, when installing Fedora Core 5, a volume group and logical volumes within the volume group are created. The volume group is called VolGroup00?, and the logical volumes are called LogVol00?, LogVol01?, etc, for each of the partitions. However, in this HOWTO, our volume group will be called vg0, and our logical volume that will eventually get mounted to /home will be called home. So, the full path of the physical partition that will be mounted on /home (when we are done) is /dev/vg0/home. (Your device path will likely be different, but you need to identify the device that is currently mounted to /home.)

With that said, let's use the following command to initialize a LUKS partition and set the initial key using a passphrase (note, this will wipe out all data on the /home partition):

# cryptsetup --verbose --verify-passphrase luksFormat /dev/vg0/home

WARNING!
========
This will overwrite data on /dev/vg0/home irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: (enter your passphrase, and write it down somewhere!)
Verify passphrase: (repeat passphrase)

Step 6: Create a mapping between physical and logical partitions

# cryptsetup luksOpen /dev/vg0/home home
Enter LUKS passphrase:
#

If all is well, you now have a special file called /dev/mapper/home. This is what you will mount on /home. Verify that the file was created:

# ls -l /dev/mapper/

total 0
crw------- 1 root root 10, 63 May 24 06:52 control
brw-rw---- 1 root disk 253, 4 May 24 10:54 home
brw-rw---- 1 root disk 253, 1 May 24 06:52 vg0-home
brw-rw---- 1 root disk 253, 0 May 24 10:53 vg0-root
brw-rw---- 1 root disk 253, 2 May 24 06:52 vg0-swap

Notice the other logical volumes (vg0-home, vg0-root, and vg0-swap) that were created when Fedora Core 5 was installed. (Note, the names of these volumes were changed by me during the installation. The were originally VolGroup00-LogVol00?, VolGroup00-LogVol01?, etc.) The fact that you are using logical volumes (like /dev/vg0/home) as physical devices can be confusing. It may help to remember that when we refer to physical devices we use devices located in the volume group directory (example: /dev/vg0), and when we refer to logical devices we use devices located in /dev/mapper (i.e., they have been mapped are are ready to use). (Okay, yes, it's confusing that the physical devices in /dev/vg0 are also listed as logical devices in /dev/mapper. Try to ignore them.)

Step 7: Create a filesystem on the new logical partition

For this HOWTO, we make an ext3 file system on /dev/mapper/home using the following commands:

# /sbin/mkfs.ext3 -j -m 1 /dev/mapper/home

(wait several minutes...)
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
36634624 inodes, 73258400 blocks
732584 blocks (1.00%) reserved for the super user
First data block=0
2236 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
#

(Note, the above output was borrowed from William Owen Smith's HOWTO: "EncryptedDeviceUsingLUKS".)

Step 8: Mount the filesystem

Mount your new logical device /dev/mapper/home to /home.

# mount /dev/mapper/home /home

View the file system's disk usage to verify that it worked:

# df -h /dev/mapper/home
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/home 4.0G 80M 3.8G 3% /home

Step 9: Restore the user's Home directory

Re-create the unpriviledged user:

# useradd -m jmaher
# passwd jmaher
Changing password for user jmaher.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
#

The -m option create's the user's home directory using the files and directories in /etc/skel as a template.

Now we need to copy MOST of the user's backed-up files back to the user's Home directory. I say MOST because I have found that copying all of the files back to the user's Home directory will break the use of the Home directory for that user. I have not investigated this, so someone else may want to comment as to the reason for this. Basically, I found it safe to copy all non-hidden files and directories back to the /home/jmaher using the following command:

# /bin/cp -r --preserve /root/jmaher/* /home/jmaher

The -r options allows recursion of subdirectories to occur, and the --preserve option preserves permissions and ownership of the files and directories.

I would recommend selectively copying hidden files and directories for those applications you find most important. For example, I really wanted my Thunderbird, Firefox, and ssh settings to be restored, so I used the following commands:

# /bin/cp -r --preserve /root/jmaher/.thunderbird /home/jmaher
# /bin/cp -r --preserve /root/jmaher/.mozilla /home/jmaher
# /bin/cp -r --preserve /root/jmaher/.ssh /home/jmaher

If you had previously modified .bashrc, .bash_profile, or .bash_logout, then you may want to copy those files as well.

Don't reboot yet, but you should now be able to test your actions and log on as the unpriviledged user (jmaher) using the following command:

# su - jmaher

After confirming that you can log on as the unpriviledged user without errors indicating that the user's environment in /home is missing, log off as the unpriviledged user to return to root.

# logout

Step 10: Modify /etc/fstab

Some aspects of the boot sequence need to be changed, because the physical volume (/dev/vg0/home) that gets mounted to /home is encrypted and is no longer a recognizable file system as far as /bin/mount is concerned. Of course, if cryptsetup is used to open the device (using the command cryptsetup luksOpen /dev/vg0/home), then /bin/mount could see that the device has an ext3 file system, and the device can be mounted.

So here are the steps to do that.

Change the line in /etc/fstab that mounted the Home directory so that:
(a) the first field refers to /dev/mapper/home rather than /dev/vg0/home
(b) the fifth field no longer indicates that this device should be accessed by the dump command
(c) the six field no longer indicates that fsck should check this device at boot time.

In short, change the line that looks similar to this:

/dev/vg0/home /home ext3 defaults 1 2

and change it to this:

/dev/mapper/home /home ext3 defaults 0 0

Step 11: Create and modify luksopen script

Copy the wonderful script called luksopen (created by embro and modified by johnny) from http://www.saout.de/tikiwiki/tiki-index.php?page=luksopen, and paste it into a new file called /sbin/luksopen.

Modify the script as follows:

a. Change devArray variable from:
devArray=(/dev/hda7 /dev/hda10 /dev/hda11 /dev/hda13)
to:
devArray=(/dev/vg0/home)
(Remember, this is the physical device used for /home. Yours is probably different.)

b. Delete the entire mapArray variable line

c. Change mntArray variable from:
mntArray=(/tmp /mnt/bergen /mnt/trondheim /mnt/oslo)
to:
mntArray=(/home)

d. Replace the line that reads:
map=${mapArray[$i]}
with:
# assign last directory name of device name to $map variable
map_elements=`echo ${devArray[$i]} | sed -e 's/^\///' -e 's/\// /g'`
for e in $map_elements ; do map=$e ; done

e. Add ' answer' (no quotes) to the following line:
read -p "Next device in list is \"$dev\". Do you want to open and mount it? (y/N): "
so that it looks like this:
read -p "Next device in list is \"$dev\". Do you want to open and mount it? (y/N): " answer

f. Delete (or comment out) the three lines at the bottom of the script that immediately preceed the final 'done' command. So, if you choose to comment out those lines, they would look like this:
#if $j -le 0 || ! mount "/dev/mapper/$map" "$mnt" ; then
# echo "Failed to mount \"/dev/mapper/$map\" on \"$mnt\"" >&2
#fi

Step 12: Edit /etc/rc.d/rc.sysinit

During a boot, we now need to open (unlock) our encrypted partition (/dev/vg0/home) and map it to our logical volume (/dev/mapper/home) before the logical volume can be mounted on /home. Because we would like to have /home mounted during the boot (so we can log on as an unpriviledged user), we need to be prompted for the LUKS passphrase (established in Step 5) before any attempts are made to mount /dev/mapper/home to /home (as configured in /etc/fstab).

Now, one could assume that simply adding /sbin/luksopen to /etc/rc.d/rc.local would result in the necessary prompting for the LUKS passphrase; however, when booting to runlevel 5 I have experienced problems getting prompted. Therefore, modifying /etc/rc.d/rc.sysinit as follows will solve the prompting problem.

Modify /etc/rc.d/rc.sysinit to incude the following code . . .

# Run /sbin/luksopen to use cryptsetup to map /dev/vg0/home to /dev/mapper/home.
if -x /sbin/luksopen ; then
/sbin/luksopen
fi

. . . immediately before the code where /usr/bin/rhgb is called. In Fedora Core 5 you can place the above code immediately before the comment line that reads:

# Start the graphical boot, if necessary; /usr may not be mounted yet, so we
# may have to do this again after mounting

Step 13: REBOOT

What You Can Expect

The boot process will be essentially the same as before, but this time you will be prompted with the following text shortly after the boot begins:

Next device in list is /dev/vg0/home. Do you want to open and mount it? (y/N):

You need to type y <ENTER>, and you will then be prompted to enter your passphrase. If you enter your passphrase correctly, the device (/dev/vg0/home) that you encrypted and mapped in Steps 5 and 6 above will be mapped to /dev/mapper/home and mounted to /home. The boot process will complete, and you can log on as your unpriviledged user (jmaher).

(Written by John Maher, 24 May 2006)

introduction to secure sockets layer/private key infrastructure

PKI is an asymmetric key system which consists of a public key (which is sent to clients) and a private key (stays local on the server). PKI differs from a symmetric key system in which both the client and server use the same key for encryption/decryption.

2.1 Responsibilities of SSL/PKI

SSL sets out to fulfill requirements that make it acceptable for use in the transmission of even the most sensitive of transactions, such as credit card information, medical records, legal documents, and e-commerce applications. Each application can choose to utilize some or all of the following criteria depending on the sensitivity and value of the transactions it will be processing.

Privacy

Let's say that a message is to be coded for transmission from A to B. A uses B's public key to encrypt the message. In this way B will be the only person who can decode and read this message using his private key. We cannot however be sure that A is the person who he claims to be.

Authenticity

In order to be sure that A is the person who he claims to be, we want guaranteed authenticity. This requires a slightly more complex coding process. In this case, A's message to B is first encrypted with A's private key and then with B's public key. B now has to decrypt it first with his private key and then with A's public key. Now B can be sure that A is who he claims to be as nobody else could create a message encrypted with his private key. SSL achieves this with the use of certificates (PKI). A certificate is issued by a neutral third party - such as a certificate authority (CA) - and includes a digital signature and/or a time stamp in addition to the public key of the certified party. A self-signed digital certificate can be created by anyone with the correct SSL tools, but self-signed certificates lack the weight of validation performed by a mutually respected neutral third party.

integrity

In SSL, integrity is guaranteed by using a MAC (Message Authentication Code) with the necessary hash table functions. Upon generation of a message, the MAC is obtained by applying a hash function and the result is then added to the message. After the message has been received, validity is then checked by comparing the message's embedded MAC with a new MAC computed from the received message. This would immediately reveal messages that have been altered by a third party.

Non-Repudiation

Non-repudiation protects both parties from each other during online transactions. It prevents one or the other from saying that they did not send a particular piece of information. Non-repudiation does not allow either party to alter the transaction after it has been made. Digital non-repudiation is the equivalent of signing a contract, in the traditional sense.

 

2.2 How SSL Works

The SSL protocol includes two sub-protocols: the SSL record protocol and the SSL handshake protocol. The SSL record protocol defines the format used to transmit data. The SSL handshake protocol involves using the SSL record protocol to exchange a series of messages between an SSL-enabled server and an SSL-enabled client when they first establish an SSL connection. This exchange of messages is designed to facilitate the following actions:

• Authenticate the server to the client. The server certificate is signed by a Certificate Authority to insure that it is not corrupted and establishes a chain of trust.
• Allow the client and server to select the cryptographic algorithms, or ciphers, that they both support.
• Optionally authenticate the client to the server.
• Use public-key encryption techniques to generate shared secrets.
• Establish an encrypted SSL connection.

SSL Handshake Protocol

The Handshake Protocol is used to co-ordinate the state of the client and the server. During the handshake, the following events take place:

• Certificates are exchanged between the client and server (asymmetric keys). The server sends its public key to the client. If the server is set to verify client authentication via a certificate, the client sends its public key to the server. The validity dates on the certificates are verified and they are checked for the digital signature of a trusted certificate authority. If the validity date and/or digital signature are not correct, the browser will issue a warning to the user. The user is then given the option to trust the certificate holder.
• The client then generates a random key (symmetric key). These will be used for encryption and for calculating MACs. They are encrypted using the server's public key and sent to the server. Only the server has the ability to decrypt the new random key. The new symmetric key is used for encrypting the data that is sent between client and server.

  Note: The use of a symmetric key after server-browser authentication greatly enhances subsequent throughput performance.

• A message encryption algorithm and a hash function for integrity are negotiated. This negotiation process could be carried out such that the client presents a list of supported algorithms to the server, which, in turn, selects the strongest cipher available to both of them. Identifiers for the chosen encryption algorithm and hash function are stored in the cipher spec field of the current state for use by the record protocol.
• All of the following fields are set during handshaking: Protocol Version, Session ID, Cipher Suite, Compression Method and two random values ClientHello.random and ServerHello.random.

Note: An IP address is required for each SSL connection. Name based virtual hosts are resolved during the application layer. Remember Secure Sockets Layer resides below the application layer.

Session Key(Symmetric Code)

• 40-bit, originally used only for export purposes
• 56-bit, used by DES
• 64-bit key - used by CAST, 256 times stronger than 56-bit
• 80-bit key - used by CAST, 16 million times stronger than 56-bit (infeasible to break with current technology)
• 128-bit key - used by CAST or RC2, exhaustive key search impossible now and for the foreseeable future

Public/Private Key Pair(Asymmetric Code)

• 512-bit
• 768-bit
• 1024-bit
• 2048-bit

2.3 How PKI Works

The client and the server each have a public key and a private key (the client's browser randomly creates a key pair for the SSL session, unless, a client certificate is held by the client and requested by the server).

The sender uses their private key to encrypt a message. This act authenticates the source of the message. The resulting cipher is encrypted once more with the receiving party's public key. This action provides confidentiality because only the receiving party is able to do the initial decryption of the message using their private key. The receiver uses the sender's public key to further decrypt the encrypted message. Because only the sender has access to their private key, the receiver is assured that the encrypted message originated from the sender.

A message digest is used to verify that neither party or a third party has tampered with or changed the message in any way. A message digest is obtained by applying a hash function (part of the private key known as the fingerprint) to the message. The digest (which is now known as the signature) is attached or appended to the message. The signature's length is constant (no matter how large the file is) and depends on what type of message digest the private key contains (md5 - 128 bit, sha1 - 160 bit, etc). Changing even one bit in the message will change the length of the signature and thus prove that the message has been tampered with.

2.4 Certificates(x509 Standard)

Digital certificates make it possible to trust an entity on the Internet. A digital certificate contains the user's credentials, which have been verified by a neutral third-party certificate authority.

A mathematical algorithm and a value (key) are used to encrypt data into an unreadable form. A second key is used to decrypt the data, using a complementary algorithm and a related value. The two keys must contain a related value and are known as a key pair.

Note: ITU-T Recommendation X.509 [CCI88c] specifies the authentication service for X.500 directories, as well as the X.509 certificate syntax. The certificate is signed by the issuer to authenticate the binding between the subject (user's) name and the user's public key. SSLv3 was adopted in 1994. The major difference between versions 2 and 3 is the addition of the extensions field. This field grants more flexibility as it can convey additional information beyond just the key and name binding. Standard extensions include subject and issuer attributes, certification policy information, and key usage restrictions.

An X.509 certificate consists of the following fields:

• Version
• serial number
• signature algorithm ID
• issuer name
• validity period
• subject (user) name
• subject public key information
• issuer unique identifier (version 2 and 3 only)
• subject unique identifier (version 2 and 3 only)
• extensions (version 3 only)
• signature on the above fields

2.5 Digital Certificate Private Key

The private key is not embedded within a digital certificate. The private key does not include any server information. It contains encryption information and a fingerprint. It is generated locally on your system and should remain in a secure environment. If the private key is compromised, a perpetrator essentially has the code to your security system. The transmissions between client and server can be intercepted and decrypted. This type of vulnerability is why it is recommended to create a private key that is encrypted using triple DES technology. The file is then encrypted and password protected making it all but impossible to use without the correct pass phrase.

The security of a transaction is dependent on its private key. Should this key fall into the wrong hands then anyone can easily duplicate it and use it to compromise security. A compromised key could lead to messages meant for the server to be intercepted and manipulated by unscrupulous hackers. A fully secure system must be able to detect impostors and prevent the duplication of keys.

2.6 Digital Certificate Public Key

The public key is embedded in a digital certificate, which is sent by the server to a client when a secure connection is requested. This process identifies the server using the certificate. The public key validates the integrity, authenticity, and is also used to encrypt data to create a private data transmission.

2.7 Certificate Signing Request(CSR)

A CSR contains the information required by a certificate authority to create the certificate. The CSR contains an encrypted version of the private key's complimentary algorithm, common value, and information that identifies the server. This information includes, but is not limited to, country, state, organization, common name (domain name), and contact information.

working with certificates

The following section covers the steps involved in creating the private key file, certificate signing request, and a self-signed certificate. If you plan to obtain a certificate signed by a certificate authority, you will need to create a certificate signing request (CSR). Otherwise, you can create a self-signed certificate.

3.1 Create a Private Key

To create a private key, you must have the OpenSSL toolkit installed and configured with Apache. The following examples use the OpenSSL command line tool which is located in the /usr/local/ssl/bin directory by default. The examples assume that the directory containing the OpenSSL command line tool has been added to the $PATH.

To create a private key using the triple des encryption standard (recommended), use the following command:

openssl genrsa -des3 -out filename.key 1024

You will be prompted to enter and re-enter a pass phrase. If you choose to use triple des encryption, you will be prompted for the password each time you start the SSL server from a cold start. (When using the restart command, you will not be prompted for the password). Some of you may find this password prompt to be a nuisance, especially if you need to boot the system during off-hours. Or, you may believe that your system is already sufficiently secure. So, if you choose not to have a password prompt (hence no triple des encryption), use the command below. If you would rather create just a 512-bit key, then omit the 1024 at the end of the command and OpenSSL will default to 512 bits. Using the smaller key is slightly faster, but it is also less secure.

To create a private key without triple des encryption, use the following command:

openssl genrsa -out filename.key 1024

To add a password to an existing private key, use the following command:

openssl -in filename.key -des3 -out newfilename.key

To remove a password from an existing private key, use the following command:

openssl -in filename.key -out newfilename.key

Note: Your private key will be created in the current directory unless otherwise specified. There are 3 easy ways to deal with this. If OpenSSL is in your path, you can run it from the directory that you have designated to store your key files in (default is /etc/httpd/conf/ssl.key if you installed Apache using the RPM or /usr/local/apache/conf/ssl.key if you installed Apache using the source files). Another solution is to copy the files from the directory where they were created to the correct directory. And, last but not least, you can specify the path when running the command (eg. openssl genrsa -out /etc/httpd/conf/ssl.key/filename.key 1024). Doesn't matter how you do it as long as it gets done before you proceed.

For more information on the OpenSSL toolkit check out: OpenSSL Website.

3.2 Create a Certificate Signing Request

To obtain a certificate signed by a certificate authority, you will need to create a Certificate Signing Request (CSR). The purpose is to send the certificate authority enough information to create the certificate without sending the entire private key or compromising any sensitive information. The CSR also contains the information that will be included in the certificate, such as, domain name, locality information, etc.

• Locate the private key that you would like to creat a CSR from. Enter the following command:
  

  openssl req -new -key filename.key -out filename.csr
  

• You will be prompted for Locality information, common name (domain name), organizational information, etc. Check with the CA that you are applying to for information on required fields and invalid entries.
• Send the CSR to the CA per their instructions.
• Wait for your new certificate and/or create a self-signed certificate. A self-signed certificate can be used until you receive your certificate from the certificate authority.

Note: Use the following command to create a private key and request at the same time.

openssl genrsa -des3 -out filename.key 1024

3.3 Creating a Self-Signed Certificate

It is not necessary to create a self-signed certificate if you are obtaining a CA-signed certificate. However, creating a self-signed certificate is very simple. All you need is a private key and the name of the server (fully qualified domain name) that you want to secure. You will be prompted for information such as locality information, common name (domain name), organizational information, etc. OpenSSL gives you a great deal of freedom here. The only required field for the certificate to function correctly is the common name (domain name) field. If this is not present or incorrect, you will receive a Certificate Name Check warning from your browser.

To create a self-signed certificate:

openssl req -new -key filename.key -x509 -out filename.crt

3.4 Installing your Web Server Certificate

If you followed these instructions so far you shouldn't have any problems at this point. If you sent your CSR to a certificate authority and you have not gotten your certificate back yet, you can take a break now! If you are using a self-signed certificate, or you have received your certificate, you may continue.

• Ensure that the private key file is in the directory that you have chosen to use. The following examples will be based on the RedHat RPM installation default of /etc/httpd/conf/ssl.key.
• Ensure that the CA-signed or self-signed certificate is in its designated location. Again, I will be using the RPM default of /etc/httpd/conf/ssl.crt. If it is not there already, put it there.
• If there is an intermediate (root) certificate to be installed, copy it to the /etc/httpd/conf/ssl.crt directory, also.
• Now, you will be required to edit the httpd.conf file. Make a back-up of this file before you proceed to the next step, Configuring your Apache Server.

configuring your apache server

The Apache server must be configured with supplementary API modules in order to support SSL. There are many SSL software packages available. My examples are based on Apache configured with ModSSL and OpenSSL. There are countless mailing lists and newsgroups available to support these products. You may find these instructions helpful for some commercial SSL software packages that are based on the Apache web server.

A few things to keep in mind: You can have multiple virtual hosts on the same server. You can have numerous name-based virtual hosts on the same IP address. You can also have numerous name-based virtual hosts and one (1) secure virtual host on the same IP. But - you cannot have multiple secure virtual hosts on the same IP. The question that so many ask: Why? The answer is: SSL works below the application layer. Name based hosts are not defined until the application layer.

Specifically, you cannot have multiple secure virtual hosts on the same SOCKET (IP address + port). By default, a secure host will use port 443. You can change configure your virtual host to use a different port number with the same IP, thus creating another socket. There are many disadvantages to this approach. The most obvious disadvantage is that if you are not using the default port, your URL must also contain the port number to access the secure site.

Example:

• Site using default port - www.something.com - would be accessed as https://www.something.com
• A site using port 8888 would be accessed as https://www.something.com:8888

Another disadvantage is that if you introduce more ports, you will be providing more opportunities for port sniffing hackers. Last, if you select a port that is used by something else, you will create conflict problem.

4.1 Define a Secure Virtual Host

Setting up virtual hosts is fairly straightforward. I will go through the basics of setting up a secure virtual host.

In these examples, I use the .crt and .key file extensions. That is my personal way of avoiding confusion with the various files. With Apache, you can use any extension you choose - or no extension at all.

All of your secure virtual hosts should be contained within <IfDefine SSL> and </IfDefine SSL>, usually located towards the end of the httpd.conf file.

An example of a secure virtual host:

<VirtualHost 172.18.116.42:443>
DocumentRoot /etc/httpd/htdocs
ServerName www.somewhere.com
ServerAdmin someone@somewhere.com
ErrorLog /etc/httpd/logs/error_log
TransferLog /etc/httpd/logs/access_log
SSLEngine on
SSLCertificateFile /etc/httpd/conf/ssl.crt/server.crt
SSLCertificateKeyFile /etc/httpd/conf/ssl.key/server.key
SSLCACertificateFile /etc/httpd/conf/ssl.crt/ca-bundle.crt
<Files ~ "\.(cgi|shtml)$">
      SSLOptions +StdEnvVars
</Files>
<Directory "/etc/httpd/cgi-bin">
      SSLOptions +StdEnvVars
</Directory>
SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown
CustomLog /etc/httpd/logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>

The directives that are the most important for SSL are the SSLEngine on, SSLCertificateFile, SSLCertificateKeyFile, and in many cases SSLCACertificateFile directives.

SSL Engine

"SSLEngine on" - this is ModSSL's command to start SSL.

SSLCertificateFile

SSLCertificateFile Tells Apache where to find the certificate file and what it is named. The example above shows "server.crt" as the certificate file name. This is the default that is added when you configure ModSSL with Apache. I personally don't recommend using the default names. Save yourself some frustration and name your certificates as servername.crt (domainname.crt). You may also decide to use an alternative directory than the default /etc/httpd/conf/ssl.crt or /usr/local/apache/conf/ssl.crt. Just remember to make the necessary changes to the path.

SSLCertificateKeyFile

SSLCertificateKeyFile tells Apache the name of the private key and where to find it. The directory defined here should have read/write permissions for root only. No one else should have access to this directory.

SSLCACertificateFile

The SSLCACertificateFile directive tells Apache where to find the Intermediate (root) certificate. This directive may or may not be necessary depending on the CA that you are using. This certificate is essentially a ring of trust.

Intermediate Certificate - A Certificate Authority obtains a certificate in much the same way as you. This is known as an intermediate certificate. It basically says that the holder of the intermediate certificate is whom they say they are and is authorized to issue certificates to customers. Web browsers have a list of "trusted" certificate authorities that is updated with each release. If a Certificate authority is fairly new, its intermediate certificate may not be in the browser's list of trusted CA's. Combine this with the fact that most people don't update their browsers very often; it could take years before a CA is recognized as trusted automatically. The solution is to install the intermediate certificate on the server using the SSLCACertificateFile directive. Usually, a "trusted" CA issues the intermediate certificate. If it is not, then you may need to use the SSLCertificateChainFile directive, although this is unlikely.

4.2 Certificate Examples

Server Certificate File

-----BEGIN CERTIFICATE-----
MIIC8DCCAlmgAwIBAgIBEDANBgkqhkiG9w0BAQQFADCBxDELMAkGA1UEBhMCWkEx
FTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMR0wGwYD
VQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UECxMfQ2VydGlmaWNhdGlv
biBTZXJ2aWNlcyBEaXZpc2lvbjEZMBcGA1UEAxMQVGhhd3RlIFNlcnZlciBDQTEm
MCQGCSqGSIb3DQEJARYXc2VydmVyLWNlcnRzQHRoYXd0ZS5jb20wHhcNOTkwNTI1
MDMwMDAwWhcNMDIwNjEwMDMwMDAwWjBTMQswCQYDVQQGEwJVUzEbMBkGA1UEChMS
RXF1aWZheCBTZWN1cmUgSW5jMScwJQYDVQQDEx5FcXVpZmF4IFNlY3VyZSBFLUJ1
c2luZXNzIENBLTIwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMYna8GjS9mG
q4Cb8L0VwDBMZ+ztPI05urQb8F0t1Dp4I3gOFUs2WZJJv9Y1zCFwQbQbfJuBuXmZ
QKIZJOw3jwPbfcvoTyqQhM0Yyb1YzgM2ghuv8Zz/+LYrjBo2yrmf86zvMhDVOD7z
dhDzyTxCh5F6+K6Mcmmar+ncFMmIum2bAgMBAAGjYjBgMBIGA1UdEwEB/wQIMAYB
Af8CAQAwSgYDVR0lBEMwQQYIKwYBBQUHAwEGCCsGAQUFBwMDBgorBgEEAYI3CgMD
BglghkgBhvhCBAEGCCsGAQUFBwMIBgorBgEEAYI3CgMCMA0GCSqGSIb3DQEBBAUA
A4GBALIfbC0RQ9g4Zxf/Y8IA2jWm8Tt+jvFWPt5wT3n5k0orRAvbmTROVPHGSLw7
oMNeapH1eRG5yn+erwqYazcoFXJ6AsIC5WUjAnClsSrHBCAnEn6rDU080F38xIQ3
j1FBvwMOxAq/JR5eZZcBHlSpJad88Twfd7E+0fQcqgk+nnjH
-----END CERTIFICATE-----

Contents of the Certificate File

Certificate:
   Data:
     Version: 3 (0x2)
     Serial Number: 1516 (0x5ec)
     Signature Algorithm: md5WithRSAEncryption
     Issuer: C=US, O=Equifax Secure Inc, CN=Equifax Secure E-Business CA
     Validity
       Not Before: Jul 12 15:21:01 2000 GMT
       Not After : Jun  2 22:42:34 2001 GMT
     Subject: C=us, ST=ga, L=atlanta, O=Equifax, OU=Rick, CN=172.18.116.44/Email=richard.sigle@equifax.com
     Subject Public Key Info:
       Public Key Algorithm: rsaEncryption
       RSA Public Key: (1024 bit)
           Modulus (1024 bit):
             00:c8:eb:93:26:97:ca:00:ce:4c:e4:f3:fd:43:31:
             cd:53:ed:b4:8a:ad:93:84:dc:7a:48:39:b5:28:57:
             03:7f:a9:ac:3e:58:6a:7a:e3:52:3e:1e:52:58:a2:
             6f:23:ad:bb:84:d8:88:ed:6d:a5:da:08:6b:c8:6c:
             a5:4c:34:67:d8:46:1c:ca:20:50:b0:e8:54:7f:ca:
             5e:ef:09:ff:6e:8d:a6:2b:02:f5:54:0f:c2:d0:45:
             12:ad:66:e7:8b:dd:68:be:64:a4:9b:69:bd:a4:1a:
             5e:ef:09:ff:6e:8d:a6:2b:02:f5:54:0f:c2:d0:45:
             12:ad:66:e7:8b:dd:68:be:64:a4:9b:69:bd:a4:1a:
             5a:2f:3b:6e:73:84:d8:d6:17:bd:12:39:34:fa:3d:
             d8:a9:e8:59:3c:c2:61:c5:b3
           Exponent: 65537 (0x10001)
     X509v3 extensions:
       X509v3 Key Usage: critical
          Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment
       Netscape Cert Type:
          SSL Server
       X509v3 Authority Key Identifier:
          keyid:5B:E0:A8:75:1C:78:02:47:71:AB:CE:27:32:E7:24:88:42:28:48:56
   Signature Algorithm: md5WithRSAEncryption
     87:53:74:e9:e1:a6:10:56:8c:fa:63:0e:7b:72:ff:76:4b:79:
     0e:49:2a:58:ed:71:7a:bf:77:61:fa:e8:74:04:37:8c:d3:6a:
     9a:3d:80:76:7a:c3:64:30:e7:1b:40:25:4e:2a:81:8b:e5:ac:
     76:a4:38:67:cc:3f:93:43:e1:1d:c3:8d:ba:ed:cc:d7:aa:a4:
     ab:d3:84:77:7c:8f:26:f6:dd:ba:3b:6a:99:81:e1:9e:7e:0f:
     ca:a6:ff:c0:c3:59:6e:dc:a6:03:23:bf:8f:24:ff:15:ad:ac:
     0d:85:fc:38:bf:d1:24:2d:1a:d3:72:55:12:95:5f:65:f0:60:
     df:b1

Private Key File

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,124F61450D85A480

ELz64SV+tFSRybsHjY9NH7CP7yDHXP6xcd9FY6MVgQykTkq2h0n7j+tmpfUPbStT
6jCgm/dTYM9mpkQ3jYZBALiVD5JNJ9t1dWisxQXY/nsak8LSTN7LhUtZSfk5xSmV
Zsl4gwQS20UdBzFiJ+4qDajP/pzocSdSuQvxIHq7UzNwJsW8UYxR3I1qrDgyNXKS
db41BWH4QdNtE0p+pi9VndDzXktqZGHEvtrQTV+39DV/dwOdnGBpYBETljMO5X6t
D42xcVs0Doa1vZ6PiMCkwFNPXsPlKHZtHwEL4I3CQdiH4E0oYh3klBzlXBY4YldN
A+s4xU44FpXp5xwt9nnVPUKHPo+NpdaRK7dAcRNO3GN3+ek1ggzvEjjuWKes3RQh
PlHPuF7VWo4KeaTfTIwJWfGxz4nvwlVByPJ6Z73Mn0VcDXCkVm6+h3PLlYL0FMqM
baUyQPpw6bhfW71FO/IIQxz3R1EqkxW7OHv74uuYl8kjHXf3S6qRZEGUG/zOGLGr
mI5s2qnU69HlBObFkc6WQq0QxMq4PiUi7HhCLMkH8+wBsNNMnb75+7lQKkEhdOeE
iUMKe5kgQqfd9w8jsBH5nu+J/nCfvPdp0isQW+P3/Rrh6YMwdKnlVfNZWdGiTzpQ
ngThAGq5lit4uf4zdTIYYrs+T9I5ltjj0KgCUD4VL5/7OfnR3gcphpbHXQf0E2cz
Qwq7q7ppKwCf/x92pHi8oVevlV5Dx9NQbGhEOA5pooqD6S2xZBbPLzkUKWDEO2il
oBZ5L1jClR5jjdF2U61w7aRrL0t6luDU/aRv/fcoYes=
-----END RSA PRIVATE KEY-----

Contents of the Private Key

read RSA key
Enter PEM pass phrase:
Private-Key: (1024 bit)
modulus:
    00:c8:eb:93:26:97:ca:00:ce:4c:e4:f3:fd:43:31:
    cd:53:ed:b4:8a:ad:93:84:dc:7a:48:39:b5:28:57:
    03:7f:a9:ac:3e:58:6a:7a:e3:52:3e:1e:52:58:a2:
    6f:23:ad:bb:84:d8:88:ed:6d:a5:da:08:6b:c8:6c:
    a5:4c:34:67:d8:46:1c:ca:20:50:b0:e8:54:7f:ca:
    5e:ef:09:ff:6e:8d:a6:2b:02:f5:54:0f:c2:d0:45:
    12:ad:66:e7:8b:dd:68:be:64:a4:9b:69:bd:a4:1a:
    5a:2f:3b:6e:73:84:d8:d6:17:bd:12:39:34:fa:3d:
    d8:a9:e8:59:3c:c2:61:c5:b3
publicExponent: 65537 (0x10001)
privateExponent:
    00:b6:57:7d:3b:58:24:1e:a9:1b:85:e9:9c:9e:5f:
    d3:3d:69:0c:21:93:37:bf:2b:2c:da:e1:6c:74:48:
    cb:c7:0f:60:5f:50:74:8a:44:45:be:54:5c:5d:4e:
    45:58:f6:f1:a8:b5:af:46:f2:ec:c2:bc:43:bd:28:
    44:b7:ad:13:d3:ca:de:59:24:e8:fa:f8:e5:5f:45:
    38:2c:a0:a3:de:98:13:d8:80:38:e1:47:53:4c:ea:
    e4:66:c3:82:93:89:c3:90:83:44:e1:13:4f:74:76:
    e2:c0:89:97:77:5f:33:d8:7d:27:21:52:55:c2:d7:
    dc:01:f9:bc:21:8d:a3:f5:c1
prime1:
    00:e3:2d:6b:5e:05:6b:e1:46:e6:ab:ae:f3:8b:d0:
    5f:94:5c:6f:f5:47:46:1d:4e:66:d3:7e:98:18:e0:
    2c:0d:08:ca:b7:29:72:af:53:62:30:ec:be:26:1f:
    cc:5a:ed:65:62:65:70:1e:18:19:61:e3:77:00:a7:
    3a:9e:4e:12:93
prime2:
    00:e2:69:56:78:e8:39:ff:17:db:cc:39:d7:7f:70:
    41:dc:c5:59:43:16:c1:84:4c:ae:e7:5d:8a:c5:4b:
    da:88:8e:03:99:7c:88:f2:8a:13:31:57:44:e0:b5:
    c8:0a:60:b0:05:de:f6:9e:f2:00:ec:37:21:8d:3b:
    dc:8e:c9:d4:61
exponent1:
    1a:ad:6a:be:4f:c4:ab:5f:b8:16:d1:24:a8:76:7f:
    c2:dc:58:09:65:a5:46:2b:be:c7:77:46:45:25:8e:
    06:b9:d1:94:50:b9:b6:fd:03:ba:db:12:39:47:e2:
    a7:8a:d9:2d:04:dc:75:ac:3e:ce:cf:f7:59:8c:49:
    c5:ed:45:21
exponent2:
    2d:4e:fd:32:06:ef:0c:40:7f:08:d8:8e:6a:7f:51:
    7e:d7:b3:6c:3c:92:8f:62:35:22:31:d3:02:76:92:
    8d:ff:35:73:32:bb:c9:25:9e:7f:a2:42:33:61:cd:
    5d:5e:49:fb:72:ca:11:b6:c6:3e:7f:2d:e4:b0:95:
    0b:b2:12:21
coefficient:
    50:52:09:22:cb:fb:b2:b8:58:85:ab:1d:82:b9:6e:
    d0:f6:dc:e8:ce:a6:5d:a1:ff:c8:4d:3b:2b:1c:19:
    64:f0:c4:4a:bc:b2:1d:2b:2d:09:59:83:a3:9a:89:
    f8:db:2c:2c:8a:bd:fd:a3:16:51:76:aa:ce:ea:85:
    6b:1c:9f:f7

4.3 Restart the Web Server

The script to restart the webserver may be located in /usr/local/sbin, /usr/sbin, (where the script is called httpd) or /usr/local/apache/bin (where the script is called apachectl). If you are not running the server with SSL enabled, you will need to stop and start the server. You may also write your own customized scripts to start, restart, and stop your server. As long as it starts the SSL engine, you should be OK.

The commands are:

httpd stop
httpd startssl
httpd restart

or

apachectl stop
apachectl startssl
apachectl restart

building a secure redhat apache server - troubleshooting

Here are some common problems that may arise.

5.1 Server Appears to start, but you cannot access the secure site

Check the error_log file. If you did not set your virtual host to write to an error log, you may want to reconsider. The example SSL virtual host writes to an error log file. Most likely you will have a few warnings and an error at the end of the log that basically say that the private key does not match the certificate. Example:

[Tue Nov 21 09:09:02 2000] [notice] Apache/1.3.14 (Unix) mod_ssl/2.7.1
OpenSSL/0.9.6 configured -- resuming normal operations
[Tue Nov 21 09:09:16 2000] [notice] caught SIGTERM, shutting down
[Tue Nov 21 14:39:54 2000] [notice] Apache/1.3.14 (Unix) mod_ssl/2.7.1
OpenSSL/0.9.6 configured -- resuming normal operations
[Tue Nov 21 14:40:31 2000] [notice] caught SIGTERM, shutting down
[Tue Nov 21 14:43:53 2000] [error] mod_ssl: Init: (esi.fin.equifax.com:443)
Unable to configure RSA server private key (OpenSSL library error follows)
[Tue Nov 21 14:43:53 2000] [error] OpenSSL: error:0B080074:x509 certificate
routines:X509_check_private_key:key values mismatch

If you get the error messages above, chances are the key and certificate do not match. Make sure you aren't using the default server.key file. You should also check the httpd.conf file to make sure that the directives are pointing to the correct private key and certificate.

You can check to make sure that you your private key and certificate are in the correct format and match each other. To do this, give the commands below to decrypt the private key in one terminal window and decrypt the certificate in the other. What you will be comparing are the Modulus and the Exponent of each key. If the modulus and exponent from the key matches the set from the certificate, you have just confirmed that your certificate and key are correctly paired.

If all else fails, create a new private key, CSR or self-signed certificate. Before you do this, check your CA's re-issue policy. You may be charged for a re-issue.

To view the contents of the certificate:

openssl x509 -noout -text -in filename.crt

To view the contents of the private key:

openssl rsa -noout -text -in filename.key

5.2 Certificate Name Check Warning is issued by the client's browser

The most common cause for this is omitting the "www" at the beginning of the domain name when creating the CSR. The name defined by the "ServerName" directive for that virtual host must match the domain name presented by the certificate exactly or the browser will let the client know. The exception is a wild card certificate. A wild card certificate's domain name field would look like *.somedomain.com. This enables you to use one certificate for any number of sub-domains of somedomain.com (e.g. host1.somedomain.com and host2.somedomain.com).

5.3 Certificate was Signed by an Untrusted Certificate Authority Warning is issued by the client's browser

If you are using a self-signed certificate, you will get this warning. Your clients will be given the option to trust your certificate or not. If you have a CA signed certificate and are getting the untrusted warning, you probably need to install their intermediate (root) certificate.

5.4 SSLEngine on is an un-recognized command (when starting Apache)

This error message is issued if you do not have ModSSL compiled with Apache. Some SSL packages use a different directive to start SSL within a virtual host. If you are using a package that does use a different directive, you will also receive this error message.

5.5 You have forgotten your "PEM Passphrase" and you would like to know how to reset it

There is no way to reset this passphrase. The only solution is to remember the passphrase or create a new private key. You will then need to obtain a new certificate or create a new self-signed certificate.

Installing apache and tomcat on redhat enterprise linux 4

1.0 Introduction

Java servlets are a powerful tool for building websites and web based applications. One skill that every Java web developer should have is the ability to install and configure the Tomcat servlet engine. Many thanks to the Apache Software Foundation for providing this mature, stable, open source software. It was recently voted the Best Application Server of 2003 by InfoWorld readers.

This article discusses how to integrate Tomcat with the Apache web server on Red Hat Linux 9 or Red Hat Enterprise Linux 3. The goal is to provide a simple, stable configuration that will allow users to gain confidence using Tomcat in a development environment. Setting up a production Tomcat server is outside the scope of this article.

Please note the following conventions:

• All commands are issued as root unless otherwise noted.
• {YOUR_DOMAIN} and {YOUR_APPLICATION} are placeholder values that should be customized to your setup. For example, {YOUR_DOMAIN} might be "localhost.test" and {YOUR_APPLICATION} might be "test".

2.0 Installing Apache

I chose to install Apache using the Red Hat RPM. Using the RPM instead of compiling Apache from source simplifies system administration in the following ways:

• Updates and bug fixes can be installed automatically from the Red Hat Network.
• Startup and shutdown scripts are already configured and available.

I recommend using the Red Hat up2date command line utility to install Red Hat RPMs. It eliminates a multitude of headaches by ensuring the software you install is the correct version and you have the right dependencies installed on your system.

Install the following Red Hat RPMs if they are not already installed:

• httpd: the Apache web server
• httpd-devel: development tools that will be needed to create the mod_jk connector

To install these packages using up2date, make sure you are connected to the Internet, and enter the following:

up2date -i httpd httpd-devel

You should now be able to start/stop/restart Apache as follows:

service httpd start
service httpd stop
service httpd restart

Verify that Apache is working by starting Apache and typing http://localhost/ into your browser. You should see the default Apache install page with links to documentation.

3.0 Installing Tomcat

The only requirements to run Tomcat are that a Java Development Kit (JDK), also called a Java Software Development Kit (SDK), be installed and the JAVA_HOME environment variable be set.

3.1 Java SDK

I chose to install Sun's Java 2 Platform, Standard Edition, which can be downloaded from http://java.sun.com/j2se/). I chose the J2SE v1.4.2 SDK Linux self-extracting binary file.

Change to the directory where you downloaded the SDK and make the self-extracting binary executable:

chmod +x j2sdk-1_4_2-linux-i586.bin

Run the self-extracting binary:

./j2sdk-1_4_2-linux-i586.bin

There should now be a directory called j2sdk1.4.2 in the download directory. Move the SDK directory to where you want it to be installed. I chose to install it in /usr/java. Create /usr/java if it doesn't exist. Here is the command I used from inside the download directory:

mv j2sdk1.4.2 /usr/java

Set the JAVA_HOME environment variable, by modifying /etc/profile so it includes the following:

JAVA_HOME="/usr/java/j2sdk1.4.2"
export JAVA_HOME

/etc/profile is run at startup and when a user logs into the system, so you will need to log out and log back in for JAVA_HOME to be defined.

exit
su -

Check to make sure JAVA_HOME is defined correctly using the command below. You should see the path to your Java SDK.

echo $JAVA_HOME

3.2 Tomcat Account

You will install and configure Tomcat as root; however, you should create a dedicated group and user account for Tomcat to run under as follows:

groupadd tomcat
useradd -g tomcat tomcat

3.3 Download Tomcat

Download the latest release binary build from http://www.apache.org/dist/jakarta/tomcat-5/. Since Tomcat runs directly on top of a standard JDK, I cannot think of any reason to building it from source. There are a number of different download formats. I chose the gnu zipped tar file (jakarta-tomcat-5.0.28.tar.gz).

3.4 Tomcat Standalone

Unzip Tomcat by issuing the following command from your download directory:

tar xvzf jakarta-tomcat-5.0.28.tar.gz

This will create a directory called jakarta-tomcat-5.0.28. Move this directory to wherever you would like to install Tomcat. I chose /usr/local. Here is the command I issued from inside the download directory:

mv jakarta-tomcat-5.0.28 /usr/local/

The directory where Tomcat is installed is referred to as CATALINA_HOME in the Tomcat documentation. In this installation CATALINA_HOME=/usr/local/jakarta-tomcat-5.0.28.

I recommend setting up a symbolic link to point to your current Tomcat version. This will save you from having to make changes to startup and shutdown scripts each time you upgrade Tomcat. It also allows you to keep several versions of Tomcat on your system and easily switch amongst them. Here is the command I issued from inside /usr/local to create a symbolic link called /usr/local/jakarta-tomcat that points to /usr/local/jakarta-tomcat-5.0.28:

ln -s jakarta-tomcat-5.0.28 jakarta-tomcat

Change the group and owner of the /usr/local/jakarta-tomcat and /usr/local/jakarta-tomcat-5.0.28 directories to tomcat:

chown tomcat.tomcat /usr/local/jakarta-tomcat
chown -R tomcat.tomcat /usr/local/jakarta-tomcat-5.0.28

It is not necessary to set the CATALINA_HOME environment variable. Tomcat is smart enough to figure out CATALINA_HOME on its own.

You should now be able to start and stop Tomcat from the CATALINA_HOME/bin directory by typing ./startup.sh and ./shutdown.sh respectively. Test that Tomcat is working by starting it and typing http://localhost:8080 into your browser. You should see the Tomcat welcome page with links to documentation and sample code. Verify Tomcat is working by running some of the examples.

4.0 Installing the Connector

4.1 Connector Benefits

At this point, Apache and Tomcat should be working separately in standalone mode. You can run Tomcat in standalone mode as an alternative to Apache. In fact, in some cases, it is said that Tomcat standalone is faster than serving static content from Apache and dynamic content from Tomcat. However, there are the following compelling reasons to use Apache as the front end:

1. You can use Apache to buffer slow connections. Tomcat uses java.io, which uses a thread for each request, so Tomcat can run out of connections as the number of slow requests grows. This could be an issue if your application supports a large number of dial-up users.
2. You can use a connector such as mod_jk to load balance amongst several Tomcat instances.
3. You can take advantage of Apache features such as cgi and PHP.
4. You can take advantage of Apache modules such as mod_rewrite, mod_headers, and mod_expire.
5. You can isolate virtual hosts in their own Tomcat instances.

The increased functionality obtained by using Apache on the front end can outweigh the effort required to install and configure a connector.

4.2 Selecting a Connector

Development on the mod_jk2 connector was discontinued on 11/15/2004; therefore, you no longer have to decide between the mod_jk and mod_jk2 connectors. Use the mod_jk connector. It has been around a long while and is very stable.

4.3 Building the mod_jk Connector

The mod_jk connector is the communication link between Apache and Tomcat. It listens on a defined port for requests from Apache and forwards those requests to Tomcat.

Install the following Red Hat RPMs if they are not already installed:

• libtool
• automake
• autoconf

Download the jk connector source from http://www.apache.org/dist/jakarta/tomcat-connectors/jk/. I used jakarta-tomcat-connectors-1.2.8-src.tar.gz.

Unzip the contents of the file into your download directory as follows:

tar xvzf jakarta-tomcat-connectors-1.2.8-src.tar.gz

This will create a folder called jakarta-tomcat-connectors-1.2.8-src. Move this folder to wherever you store source files on your system. I chose /usr/src. Here is the command I issued from inside the download directory:

mv jakarta-tomcat-connectors-1.2.8-src /usr/src

I refer to the folder where the connector source is installed as CONN_SRC_HOME. In my case CONN_SRC_HOME = /usr/src/jakarta-tomcat-connectors-1.2.8-src.

Change to directory CONN_SRC_HOME/jk/native and run the buildconf.sh script as follows:

./buildconf.sh

This will create the CONN_SRC_HOME/jk/native/configure file. Run the configure script with the path to the apxs file on your system. If you installed Apache using the Red Hat RPM, apxs should be located at /usr/sbin/apxs.

./configure --with-apxs=/usr/sbin/apxs

Build mod_jk with the following command:

make

If you see missing object errors, try this alternate command:

make LIBTOOL=/etc/httpd/build/libtool

If all went well, the mod_jk.so file was successfully created. Manually copy it to Apache's shared object files directory:

cp CONN_SRC_HOME/jk/native/apache-2.0/mod_jk.so /etc/httpd/modules

5.0 Creating the Application Directories

Both Apache and Tomcat application files will be installed under the /home/tomcat/webapps directory. This setup will allow you to easily upgrade Tomcat and back up your application files.

Create your application directories as follows:

mkdir /home/tomcat/webapps
mkdir /home/tomcat/webapps/{YOUR_DOMAIN}
mkdir /home/tomcat/webapps/{YOUR_DOMAIN}/logs
mkdir /home/tomcat/webapps/{YOUR_DOMAIN}/{YOUR_APPLICATION}
mkdir /home/tomcat/webapps/{YOUR_DOMAIN}/{YOUR_APPLICATION}/WEB-INF
mkdir /home/tomcat/webapps/{YOUR_DOMAIN}/{YOUR_APPLICATION}/WEB-INF/classes

Set directory permissions as follows:

chmod 755 /home/tomcat/webapps
chmod 755 /home/tomcat/webapps/{YOUR_DOMAIN}
chmod 755 /home/tomcat/webapps/{YOUR_DOMAIN}/{YOUR_APPLICATION}

6.0 Configuring Tomcat

6.1 workers.properties

The workers.properties file contains information so mod_jk can connect to the Tomcat worker processes.

Place the following workers.properties file in the /etc/httpd/conf directory:

# workers.properties - ajp13
#
# List workers
worker.list=wrkr
#
# Define wrkr
worker.wrkr.port=8009
worker.wrkr.host=localhost
worker.wrkr.type=ajp13
worker.wrkr.cachesize=10
worker.wrkr.cache_timeout=600
worker.wrkr.socket_timeout=300

Notes

1. There is an example workers.properties file located in the CONN_SRC_HOME/jk/conf directory. The example file provides a lot of useful information and insight into the workers.properties file, but it contains so much information that it can be confusing. I recommend using it as a learning tool but creating your own workers.properties file from scratch.
2. The configuration above assumes Apache and Tomcat are located on the same box and requests are forwarded to Tomcat using type ajp13 workers. Type ajp13 workers forward requests to out-of-process Tomcat workers using the ajpv13 protocol over TCP/IP sockets.
3. The name of the worker in the JkMount directive in httpd.conf must match the name of the worker in worker.list ("wrkr" in the configuration above).

6.2 server.xml

The CATALINA_HOME/conf/server.xml file contains Tomcat server configuration information. The default server.xml is great for verifying that Tomcat works in standalone mode and for viewing the examples that come with the application, but it contains so much information that it can be confusing. I recommend saving it for future reference and creating a new server.xml.

Save the default server.xml as follows:

mv CATALINA_HOME/conf/server.xml CATALINA_HOME/conf/server.xml.orig

Copy the following into a new server.xml file:

<Server port="8005" shutdown="0fbb9aebcbfbef203eca71b6be367859" debug="0">

	<Service name="Tomcat-Apache">

		<Connector address="127.0.0.1"
			port="8009"
			minProcessors="5"
			maxProcessors="75"
			enableLookups="false"
			protocol="AJP/1.3"
			debug="0"/>

		<Engine name="appserver"
			debug="0"
			defaultHost="{YOUR_DOMAIN}">

			<Host name="{YOUR_DOMAIN}"
				appBase="/home/tomcat/webapps"
				autoDeploy="false"
				deployOnStartup="false"
				unpackWARs="false"
				deployXML="true"
				debug="0"/>

		</Engine>

	</Service>

</Server>

If you do keep the default server.xml, make sure you comment out any other connectors besides mod_jk that are listening on port 8009. The default file comes with the Coyote/JK2 connector enabled for the Tomcat-Standalone service. This will conflict with the mod_jk connector in your Tomcat-Apache service. You should comment this connector out. It isn't needed when you connect directly to Tomcat in standalone mode (port 8080), so I'm not sure why this connector is enabled by default.

The Connector address defines the interface that Tomcat will listen on for mod_jk requests from Apache. In my configuration, Apache and Tomcat reside on the same box, so I have set the address to the loopback address. The default is for Tomcat to listen on all interfaces, so restricting it to the loopback interface improves security.

The Server shutdown property is the text string that is sent over a socket connection to stop Tomcat. The default value is "SHUTDOWN". The shutdown port is always on the loopback interface, which provides host-level protection. However, there is the possibility that the host could be compromised and someone could send the command SHUTDOWN to all ports and knock Tomcat offline. To prevent this, replace the default value with one that is difficult to guess. Do not use the example string above. Create your own by feeding random bytes into md5sum as follows:

head -1024c /dev/random | md5sum

Change the permissions on server.xml to prevent unprivileged users from reading the shutdown string:

chmod 600 CATALINA_HOME/conf/server.xml

6.3 Configuring the Context

It is recommended that contexts be defined in separate files, not in server.xml. The context configuration directory has the same name as the Engine in server.xml.

Create the context configuration directory as follows:

mkdir CATALINA_HOME/conf/appserver

The host context configuration directory has the same name as the corresponding Host in server.xml.

Create the context configuration directory for your domain as follows:

mkdir CATALINA_HOME/conf/appserver/{YOUR_DOMAIN}

Create the context configuration file as follows:

touch CATALINA_HOME/conf/appserver/{YOUR_DOMAIN}/{YOUR_APPLICATION}.xml

Copy the following text into {YOUR_APPLICATION}.xml:

<Context path=""
	docBase="{YOUR_APPLICATION}"
	reloadable="true"
	debug="0"/>

Setting the Context reloadable property to true tells Tomcat to automatically load new and changed application class files found in /WEB-INF/classes and /WEB-INF/lib. This feature is very useful during development. However, it is recommended to set reloadable to false in production environments because monitoring class file changes requires significant server resources.

6.4 log4j

Download the latest log4j and install log4j.jar in CATALINA_HOME/common/lib.

Download the latest commons logging and install commons-logging.jar in CATALINA_HOME/common/lib.

Create a file called log4j.properties as follows and place it in the CATALINA_HOME/common/classes directory:

log4j.rootLogger=WARN, R
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=/usr/local/jakarta-tomcat/logs/tomcat.log
log4j.appender.R.MaxFileSize=10MB
log4j.appender.R.MaxBackupIndex=10
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%d{DATE} - %p %c - %m%n
log4j.logger.org.apache.catalina=WARN, R

7.0 Configuring Apache

Apache is configured with directives placed in the main Apache configuration file, /etc/httpd/conf/httpd.conf. In addition, Apache 2 has configuration files for perl, php, and ssl located in /etc/httpd/conf.d/.

Rename the /etc/httpd/conf.d/ssl.conf file to ssl.conf.bak. The default Red Hat Apache 2 installation comes with ssl support enabled. If ssl is needed, you can re-enable it after you have successfully integrated Apache and Tomcat.

7.1 httpd.conf

You will notice that there are three sections labeled in the httpd.conf file supplied by Red Hat: (1) Global Environment, (2) Main Server Configuration, and (3) Virtual Hosts.

Add the following to the bottom of the existing LoadModule directives in the Global Environment section:

LoadModule jk_module modules/mod_jk.so

Add the following to the bottom of the Main Server Configuration section:

JkWorkersFile "/etc/httpd/conf/workers.properties"
JkLogFile "/etc/httpd/logs/mod_jk.log"
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

Set up a Virtual Host directive in the Virtual Hosts section of httpd.conf. Below is an example of how to set up the {YOUR_DOMAIN} website so Tomcat handles all jsp pages and requests with "servlet" in the path:

NameVirtualHost 127.0.0.1:80

<VirtualHost 127.0.0.1:80>
	ServerAdmin webmaster@{YOUR_DOMAIN}
	ServerName {YOUR_DOMAIN}
	DocumentRoot /home/tomcat/webapps/{YOUR_DOMAIN}/{YOUR_APPLICATION}
	ErrorLog /home/tomcat/webapps/{YOUR_DOMAIN}/logs/error_log
	CustomLog /home/tomcat/webapps/{YOUR_DOMAIN}/logs/access_log common
	JkMount /*.jsp wrkr
	JkMount /servlet/* wrkr
	# Deny direct access to WEB-INF
	<LocationMatch ".*WEB-INF.*">
		AllowOverride None
		deny from all
	</LocationMatch>
</VirtualHost>

The argument for the NameVirtualHost directive must match exactly the argument for the VirtualHost directive (127.0.0.1:80).

The JkMount directive specifies url patterns of requests that will be forwarded to Tomcat for processing.

You can test your Apache configuration by typing the following:

httpd -t -D DUMP_VHOSTS

You should get something like the following response:

127.0.0.1:80           is a NameVirtualHost
         default server {YOUR_DOMAIN} (/etc/httpd/conf/httpd.conf:1056)
         port 80 namevhost {YOUR_DOMAIN} (/etc/httpd/conf/httpd.conf:1056)
Syntax OK

8.0 Setting Up {YOUR_DOMAIN}

{YOUR_DOMAIN} does not need to be a domain name with a DNS entry. For testing purposes, you can set up any domain you want in the /etc/hosts file of the machine that you will be using to access your application.

The example below shows the entry for {YOUR_DOMAIN} when running Apache and Tomcat on a single machine, typical for a development computer.

127.0.0.1	{YOUR_DOMAIN}

9.0 Testing Apache

We will create and install a simple Hello World html page so we can test to make sure Apache handles requests for static html pages.

9.1 Hello World HTML

Copy the following text into a file called HelloWorld.html and install the file in the /home/tomcat/webapps/{YOUR_DOMAIN}/{YOUR_APPLICATION} directory.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Hello World HTML!</title>
</head>
<body>
<h1>Hello World HTML!</h1>
</body>
</html>

If Apache has not been restarted since you added your virtual host, do so as follows:

service httpd restart

You should now be able to type http://{YOUR_DOMAIN}/HelloWorld.html into your browser and see the always-exciting "Hello World" message.

10.0 Testing Tomcat

We will create and install a simple Hello World jsp page and servlet so we can test to make sure Apache forwards servlet requests to Tomcat for handling.

10.1 Hello World JSP

Copy the following text into a file called HelloWorld.jsp and install the file in the /home/tomcat/webapps/{YOUR_DOMAIN} directory.

<%@ page contentType="text/html;charset=WINDOWS-1252"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
<title>Hello World JSP</title>
</head>
<body>
<h1><% out.println(" Hello World JSP!"); %></h1>
</body>
</html>

10.2 Hello World Servlet

Copy the following into a file called HelloWorld.java:

import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
public class HelloWorld
    extends HttpServlet {
    public void doGet(HttpServletRequest request, 
                       HttpServletResponse response)
                throws IOException, ServletException {
		
		response.setContentType("text/html");
		PrintWriter out = response.getWriter();
		
		out.println("Hello World Servlet!");

	}

}

Compile the source into a class file as follows:

javac -classpath /usr/local/jakarta-tomcat/common/lib/servlet.jar HelloWorld.java

This will create a file called HelloWorld.class. Copy the HelloWorld.class file to the /home/tomcat/webapps/{YOUR_DOMAIN}/{YOUR_APPLICATION}/WEB-INF/classes directory.

10.3 Tomcat Application

The web.xml file is where a servlet name is mapped to a URL pattern so Tomcat can run your servlet when requested. Below is the web.xml file that runs the HelloWorld servlet whenever the URL http://{YOUR_DOMAIN}/servlet/HelloWorld is entered in the browser:

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
    "http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>

	<servlet>
		<servlet-name>HelloWorld</servlet-name>
		<servlet-class>HelloWorld</servlet-class>
	</servlet>
	<servlet-mapping>
		<servlet-name>HelloWorld</servlet-name>
		<url-pattern>/servlet/HelloWorld</url-pattern>
	</servlet-mapping>
                
</web-app>

Restart Tomcat as follows:

/CATALINA_HOME/bin/shutdown.sh
/CATALINA_HOME/bin/startup.sh

Restart Apache as follows:

service httpd restart

You should now be able to type the following into your browser and see the always-exciting "Hello World" message:
http://{YOUR_DOMAIN}/HelloWorld.jsp
http://{YOUR_DOMAIN}/servlet/HelloWorld

11.0 Advanced Configuration

The following steps are not mandatory, but are suggested for a better, tighter Tomcat installation.

11.1 Tomcat Startup Script

If you want to automatically start Tomcat when your system boots and manage it using the service command as we do Apache, you must create an initialization script.

Copy the following text into a file called tomcat and install the file in the /etc/rc.d/init.d directory.

#!/bin/sh
#
# Startup script for Tomcat Servlet Engine
#
# chkconfig: 345 86 14
# description: Tomcat Servlet Engine
# processname: tomcat
# pidfile: /usr/local/jakarta-tomcat/bin/tomcat.pid
#

# User under which tomcat will run
TOMCAT_USER=tomcat

RETVAL=0

# start, debug, stop, and status functions
start() {
    # Start Tomcat in normal mode
    SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
    if [ $SHUTDOWN_PORT -ne 0 ]; then
        echo "Tomcat already started"
    else
        echo "Starting tomcat..."
        chown -R $TOMCAT_USER:$TOMCAT_USER /usr/local/jakarta-tomcat/*
        chown -R $TOMCAT_USER:$TOMCAT_USER /home/tomcat/*
        su -l $TOMCAT_USER -c '/usr/local/jakarta-tomcat/bin/startup.sh'
	SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
        while [ $SHUTDOWN_PORT -eq 0 ]; do
            sleep 1
            SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
        done	
        RETVAL=$?
        echo "Tomcat started in normal mode"
        [ $RETVAL=0 ] && touch /var/lock/subsys/tomcat
    fi
}

debug() {
    # Start Tomcat in debug mode
    SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
    if [ $SHUTDOWN_PORT -ne 0 ]; then
        echo "Tomcat already started"
    else
        echo "Starting tomcat in debug mode..."
        chown -R $TOMCAT_USER:$TOMCAT_USER /usr/local/jakarta-tomcat/*
        chown -R $TOMCAT_USER:$TOMCAT_USER /home/tomcat/*
        su -l $TOMCAT_USER -c '/usr/local/jakarta-tomcat/bin/catalina.sh jpda start'
	SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
        while [ $SHUTDOWN_PORT -eq 0 ]; do
            sleep 1
            SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
        done	
        RETVAL=$?
        echo "Tomcat started in debug mode"
        [ $RETVAL=0 ] && touch /var/lock/subsys/tomcat
    fi
}

stop() {
    SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
    if [ $SHUTDOWN_PORT -eq 0 ]; then
        echo "Tomcat already stopped"
    else
        echo "Stopping tomcat..."
        su -l $TOMCAT_USER -c '/usr/local/jakarta-tomcat/bin/shutdown.sh'
	SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
        while [ $SHUTDOWN_PORT -ne 0 ]; do
            sleep 1
            SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
        done
	RETVAL=$?
        echo "Tomcat stopped"
        [ $RETVAL=0 ] && rm -f /var/lock/subsys/tomcat /usr/local/jakarta-tomcat/bin/tomcat.pid
    fi
}

status() {
    SHUTDOWN_PORT=`netstat -vatn|grep LISTEN|grep 8005|wc -l`
    if [ $SHUTDOWN_PORT -eq 0 ]; then
        echo "Tomcat stopped"
    else
        MODE="normal"
        JPDA_PORT=`netstat -vatn|grep LISTEN|grep 8000|wc -l`
        if [ $JPDA_PORT -ne 0 ]; then
            MODE="debug"
        fi
	echo "Tomcat running in $MODE mode"
    fi
}

case "$1" in
  start)
        start
        ;;
  debug)
        debug
        ;;
  stop)
        stop
        ;;
  restart)
        stop
        start
        ;;
  redebug)
        stop
        debug
        ;;
  status)
  	status
	;;
  *)
	echo "Usage: $0 {start|debug|stop|restart|redebug|status}"
	exit 1
esac

exit $RETVAL

Add the startup script to your system as follows:

chkconfig --add tomcat

The path of the file that contains the pid of the catalina startup java process can be set with the CATALINA_PID environment variable. CATALINA_HOME/bin/catalina.sh automatically calls a file called setenv.sh if it exists, so this is a good place to set environment variables.

Create setenv.sh as follows:

cd CATALINA_HOME/bin
touch setenv.sh
chmod 644 setenv.sh

Copy the following text into setenv.sh:

CATALINA_PID=/usr/local/jakarta-tomcat/bin/tomcat.pid

Now you will be able to start/stop/restart/status Tomcat using the following commands:

service tomcat start
service tomcat stop
service tomcat restart
service tomcat status

If you want Tomcat to start automatically when your system boots, you need to add tomcat to your runlevel as follows:

chkconfig --level 5 tomcat on

Runlevel 5 is the X Window System, typical for a development computer. Runlevel 3 is typical for a dedicated web server.

Apache and Tomcat can be started and restarted in any order. In the past (specifically with the 1.2.5 connector), if Tomcat was restarted, Apache would have to be restarted. This was because the AJP13 protocol maintains open sockets between Apache and Tomcat, and when Tomcat was restarted the connections would be hung in CLOSE_WAIT status until Apache was restarted. This has been fixed starting with the 1.2.6 connector.

11.2 Development Setup

During development, you will need access to your tomcat application directory. Add the user account under which you will be doing development to the tomcat group in /etc/group. For example, this is what the tomcat entry might look like in /etc/group if you do development under the yourname account:

tomcat:x:502:yourname

Make sure the tomcat group has permission to publish files (e.g. using ant) to your Tomcat application in /home/tomcat/webapps/{YOUR_DOMAIN}. Issue the following commands as root:

chmod g+x /home/tomcat
chmod -R g+rw /home/tomcat

12.0 Troubleshooting

12.1 Log Files To Watch

/home/tomcat/webapps/{YOUR_DOMAIN}/logs/error_log

Look here for clues to Apache httpd.conf configuration issues, for example VirtualHost setup.

CATALINA_HOME/logs/catalina.out

Look here for clues to Tomcat server.xml configuration issues. This file is written to when Tomcat starts and stops. It also catches System.out and System.err.

CATALINA_HOME/logs/tomcat.log

Look here for clues to all Tomcat issues. This is the main Tomcat log file.

/etc/httpd/logs/mod_jk.log

Look here for clues to mod_jk configuration issues.

12.2 Monitoring Connections

The following command can be used to monitor the Apache, Tomcat, and mod_jk connections:

netstat -vatn | grep 80

Below is output from running this command. Line numbers have been added to the beginning of each line for discussion purposes.

1 tcp        0      0 127.0.0.1:8005          0.0.0.0:*               LISTEN
2 tcp        0      0 127.0.0.1:8009          0.0.0.0:*               LISTEN
3 tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN
4 tcp        0      0 127.0.0.1:8009          127.0.0.1:34449         ESTABLISHED
5 tcp        0      0 127.0.0.1:34449         127.0.0.1:8009          ESTABLISHED

Notes

1. Line 1 shows Tomcat listening on port 8005 for the shutdown command.
2. Line 2 shows Tomcat listening on port 8009 for requests from Apache.
3. Line 3 shows Apache listening on port 80 for user requests.
4. Line 4 shows the Tomcat end of a mod_jk connection.
5. Line 5 shows the Apache end of a mod_jk connection.

13.0 Disclaimer

This document is provided as is without any express or implied warranties. While every effort has been taken to ensure the accuracy of the information contained in this document, the author assumes no responsibility for errors, omissions, or damages resulting from the use of the information contained herein. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

apache tomcat logging tutorial