Linux System Duplication

Andrew McGill, ledge.co.za

Revision 0.3; $Id: systemduplication.sgml,v 1.10 2006/11/16 07:58:08 andrewm Exp $

Step by step instructions for duplicating a Linux system by copying the files from one hard disk to another.

1. Introduction

You may need to duplicate your Linux system onto another hard disk for any number of reasons, including:

You have just replaced your hard disk with a newer, bigger or more reliable one
You have a new computer
You tried it with Ghost, and it messed up. You would like to mess it up yourself.
You want to change the filesystem type you are using (e.g. reiserfs to ext3fs)

1.1 Assumptions

The following assumptions have been used in this document to simpify things (for the author, and not so much for the reader):

The original, working system resides on a single IDE disk (e.g. hda)
A second disk is available for copying the system to (e.g. hdc)
You can type commands fairly accurately (no dyslexia)
You can think and adapt if your system is a little different
You want to duplicate your system

1.2 Introduction to the method

Do some research (mount; fdisk -p /dev/hda)
Set up second system's drive on /dev/hdc
Create similar partitions on the new disk hdc (fdisk /dev/hdc)
Format the new partitions (mke2fs / mkreiserfs / mkswap /dev/hdc99)
Mount the new disk on /mnt (mount /dev/hdc3 /mnt) and create mount directories in /mnt and mount the new file systems there
Copy the files (cp -vax /. /mnt ... or something similar) (don't try cp -vax /* /mnt - it's not the same)
Shutdown the system
Install the hard disk (hdc) in a new system (it won't work yet)
Boot the new system using a rescue disk (or something)
Run lilo, reboot and test

1.3 Disclaimer

You will destroy your data. Don't blame me.

2. Research

Short version: Make notes of the partitions on your existing disk

Before you can copy a system, you need to know how big the various partitions are. Write down the output of one of the commands below. (fdisk is more useful if your new disk is the same as the old, and df is more useful at other times). The output of grep will show where your partitions should be mounted.

fdisk -p /dev/hda
df -hl
mount
grep hd /etc/fstab

You may have a disk with SCSI or SATA drives. In this case the drives are named /dev/sda, /dev/sdb etc, and the partitions are named /dev/sda1, /dev/sda2, /dev/sda3, etc.

On my system with a 12G drive, the output looks like this:

tonto:~ # fdisk -l /dev/hda

Disk /dev/hda: 255 heads, 63 sectors, 1467 cylinders
Units = cylinders of 16065 * 512 bytes

  Device Boot Start   End   Blocks   Id  System
/dev/hda1   *      1   255  2048256    c  Win95 FAT32
/dev/hda2        256   264    72292+  82  Linux swap
/dev/hda3        265   776  4112640   83  Linux
/dev/hda4        777  1467  5550457+   5  Extended
/dev/hda5        777   777     8001   83  Linux
/dev/hda6        778  1467  5542393+  83  Linux

tonto:~ # df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda3             3.9G  3.7G  262M  94% /
/dev/hda5             7.6M  5.7M  1.5M  79% /boot
/dev/hda6             5.3G  4.7G  700M  88% /data
shmfs                  58M     0   57M   0% /dev/shm
/dev/hda1             1.9G  1.6G  399M  80% /windows/C

tonto:~ # mount

/dev/hda3 on / type reiserfs (rw)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
/dev/hda5 on /boot type ext2 (rw)
/dev/hda6 on /data type reiserfs (rw)
shmfs on /dev/shm type shm (rw)
usbdevfs on /proc/bus/usb type usbdevfs (rw)
/dev/hda1 on /windows/C type vfat (rw,noexec)

tonto:~ # grep hd /etc/fstab

/dev/hda1       /windows/C      vfat    noauto,user 0 0
/dev/hda5       /boot   ext2    defaults 1 2
/dev/hda2       swap    swap    defaults 0 2
/dev/hda3       /       reiserfs        defaults 1 1
/dev/hda6       /data   reiserfs        defaults 1 1

(it's a wonderfully bad example, since it includes a Windows filesystem). The thing to take note of is the size of the partitions (how many megabytes M or gigabytes G) and their locations. You will be creating identical partitions on your new disk.

3. Playing with hardware I

Short version: Plug in new hard disk into old machine

This howto works on the assumption that your system is configured something like the following: (or that you are smart enough to figure out what to do)

Primary IDE channel:   Master : /dev/hda    Original system
                      Slave  : /dev/hdb    CDROM or unused
Secondary IDE channel: Master : /dev/hdc    Duplicate system
                      Slave  : /dev/hdd    Unused

If you plug the duplicate hard disk into /dev/hdb instead of /dev/hdc, then you will need to set the jumpers on the hard disk for it to act as a "Slave" and not as a "Master". Copying data between disks on separate interfaces could make things work a little faster though.

4. Create similar partitions

Short version: fdisk /dev/hdc; (d)elete existing partitions and make (n)ew paritions using (t)ype 82 and 83.

Fdisk is a tool that edits the first part of the disk, which is called the partition table. This determines how the space on the disk is allocated.

The resources for using fdisk are the fdisk man page, and the output of the help command when you run fdisk:

man fdisk
fdisk /dev/hdc
Command (m for help): m

Here's the procedure:

Make sure you are using the correct disk (/dev/hdc, not /dev/hda)
```
fdisk /dev/hdc
```
Test how to escape - press Ctrl+C to stop fiddling the partition table.
Delete the existing partitions
```
p               # Print the existing partition table
d               # Delete a partition
```
You may have to delete the `logical' partitions before you delete the extended partition

Create new partitions of the correct sizes. Use the same numbers as before. To create partitions with a number greater than 4, create an extended partition, and then create additional "logical" partitions.

n               # new partition
Command (m for help): n
Command action
  l   logical (5 or over)
  p   primary partition (1-4)
p               # primary partition
Partition number (1-4): 1
First cylinder (1-1467, default 1): 1
Last cylinder or +size or +sizeM or +sizeK
  (1-255, default 255): 255

Set the type of the new partitions you have created. For Linux partitions, you will want to set the type to Linux (83) for the data partitions, and to Linux swap (82) for the swap partition(s).
```
Command (m for help): t
Partition number (1-6): 1
Hex code (type L to list codes): 83
Changed system type of partition 1 to 83 (Linux)
```
Have a look at the new partition table (it should look something like the one on the original disk)
```
p
```
If you are happy with what you see, write it to the disk:
```
w
```
If fdisk complains that it cannot convince the kernel to reload the new partition table, you will have to reboot the system. This is normally a bad sign though, because there is no reason you will see this on an unused disk:
```
Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error
16: Device or resource busy.  The kernel still uses the
old table.  The new table will be used at the next reboot.

Syncing disks.
```
If you inadvertently overwrote your original system's partition table, you should be able to recover by using the tool gpart (guess partitions) which is found on most rescue disks. Alternatively you can simply re-enter the same partition information as you had previously. Note that some systems have a menu driven program called cfdisk. If you like menu driven programs, you can give it a try, but don't blame me. When all is done, if it is done correctly, then these commands should produce similar output:
```
fdisk -p /dev/hda
fdisk -p /dev/hdc
```

5. Format the new partitions

Short version: mke2fs or mkreiserfs and mkswap

If the output of

grep hd /etc/fdtab

looks like this:

/dev/hda1       /windows/C      vfat    noauto,user 0 0
/dev/hda5       /boot   ext2    defaults 1 2
/dev/hda2       swap    swap    defaults 0 2
/dev/hda3       /       reiserfs        defaults 1 1
/dev/hda6       /data   reiserfs        defaults 1 1

Then these are the commands that you will use to `format' the new partitions: There is an IMPORTANT change here! All of the partitions are being created on the NEW drive /dev/hdc, not on the old drive. Don't get it wrong, or you will delete the system you are trying to duplicate.

mkfs.ext2 /dev/hdc4
mkswap /dev/hdc2
mkfs.reiserfs /dev/hdc3
mkfs.reiserfs /dev/hdc6

There is also mkfs.ext3 which makes ext3 format filesystems.

We won't need the Windows partion on the new system :) ... but if we did for some reason, it would be safer to get Windows to do it with its own format command.

6. Mount partitions

Mount the root partition of the new system on the /mnt point:

mount /dev/hdc3 /mnt

Create directories in this system for each of the mount points it will use. On my system, this means three directories (although you probably won't have all of these)

mkdir -p /mnt/boot
mkdir -p /mnt/windows/C
mkdir -p /mnt/data

Now mount the partitions on the directory mount points you have made:

mount /dev/hdc5 /mnt/boot
mount /dev/hdc3 /mnt/
mount /dev/hdc6 /mnt/data

Check your work. The output of `mount' should show similar entries for /dev/hda (point points based at /) and /dev/hdc (based at /mnt/).

7. Copy files

For each partition, copy all the files (see GOTCHA in endnotes). The copy command make exact(ish) archive (-a) copies, has a handy verbose mode (-v), and a `don't cross filesystems' option (-x), which we will use. Copy each of the partitions from the old system to the new system:

cp -vax /boot/.  /mnt/boot
cp -vax /. /mnt
cp -vax /data/. /mnt/data

Alternatively, you can use tar to do the same thing. This is a better idea if you have sparse files (but if you are reading this howto, you might not know what those are...)

cd /
tar clS / /data /boot | tar xv -C /mnt

You have to specify each mount point, since the `l' option above tells tar not to cross between file-systems (similar to the `-x' option in cp).

Did I mention that if you have a database server or a mail server running you should stop it during the time that you copy its files? If you fail to do this on a relatively busy server, you may copy files which are in a particularly nonsensical state which they pass though during updates.

To be (relatively) sure that the data has been copied to the disk:

sync

That's it. Well, almost.

7.1 If you changed device names and file systems ...

You're one of those people who is doing this whole thing because you're migrating from ext3 to jfs (crazy!), from SCSI to IDE, and now suddenly your system doesn't boot because of kernel panics or something similar.

You need to edit files:

vi /mnt/etc/fstab -- edit the filesystem table that says which devices should be put where.
vi /mnt/etc/lilo -- if your system is using LILO to boot, then make sure that the line root=/dev/sda8 points to the device your main partition (/.) is on. If you've changed filesystem types, then reiserfs may change to ext2 and jfs so that the device names correspond again with the partition types they contain. After changing this file, run chroot /mnt and lilo again as described above.
vi /mnt/boot/grub/menu.lst -- if you're using GRUB as your boot loader, then this is where you need to change your device names.

If you're a little in the dark about vi, here's a three point tutorial:

Press i to insert, and Escape when you're finished inserting.
Type ZZ (capitals) to save and exit, or ZQ to exit without saving.
If in doubt, quit vi and run the command vimtutor to learn how to do it.

7.2 If you changed your disk controller ...

If you are using a different disk controller on the target system (e.g. sata_nv to piix), for most distributions, you will have to make a new initial root disk, using the mkinitrd command. If you need to do this and you don't do this, you will get a kernel panic "cannot mount root filesystem".

You need to:

Find out which module(s) the disk requires
Tell mkinitrd to include it (or them)
Run mkinitrd on the target system to make a new initrd file for booting.

To find out which module to use, run lspci and lsmod. As the kernel becomes more and more modular, it becomes harder to know which is your disk controller module. You can use modinfo piix to see what a module says about itself:

modinfo piix
 filename:       /lib/modules/2.6.15-23-386/kernel/drivers/ide/pci/piix.ko
 author:         Andre Hedrick, Andrzej Krzysztofowicz
 description:    PCI driver module for Intel PIIX IDE
 license:        GPL
 vermagic:       2.6.15-23-386 preempt 486 gcc-4.0

If you don't know, guess. It can't hurt that much.

Now tell mkinitrd to include the module:

Debian/Ubuntu: vi /etc/mkinitrd/modules, then run update-initrd.
SuSE: vi /etc/sysconfig/kernel, then run mkinitrd.
Redhat ... erk ... suggestions welcome

8. Playing with hardware II

Now you have to get the new system to be bootable.

8.1 Method 1 (easy) (using rescue disk)

Shutdown the original system.
Remove the duplicate hard disk.
Put the duplicate disk in the new system.
Boot up the duplicate system using the rescue disk or rescue boot option provided by your Linux distribution. This should dump you at a shell prompt.
Mount the root partition:
```
mount /dev/hda3 /mnt
```
Enter the new system, mount additional filesystems (e.g. /boot), and run lilo to make the system bootable.
```
chroot /mnt
mount -a
lilo
```
If you are using the GRUB boot loader, then instead of lilo, you type grub-install /dev/hda to install GRUB on the MBR.
On some distributions, grub-install is broken. For these you have to ...
```
grub
 root (hd0,0)     # assuming /boot is /dev/hda1
 setup (hd0)
 quit
```
Now you can exit and shutdown:
```
umount -a
sync
exit
umount /mnt
reboot
```

Remove the disk and reboot. The system should boot as a duplicate.

8.2 Method 2 (best for GRUB) (hard, but you don't need a rescue disk)

If your system boots with GRUB, you can set up the second disk to be ready to boot when it becomes the first disk:

Boot up the original system with the duplicate hard disk in (probably you are here already)
Chroot into the new system.
Lie to GRUB about where its hard disks are
Install GRUB (on the second disk)
Tell GRUB the truth again for posterity

Here's the detail:

Mount the duplicated root partition:
```
mount /dev/hdc3 /mnt
```
Chroot into the new system, mount additional filesystems (e.g. /boot):
```
chroot /mnt
mount -a
```
Now edit /boot/grub/device.map, and tell it that the first disk is your duplicate disk (/dev/hdc):
```
cd /boot/grub
cp device.map device.map.foo
vi device.map.foo
```
Change the contents of device.map from this ...
```
(hd0)   /dev/hda
```
... to this...
```
(hd0)   /dev/hdc
```
Now, tell GRUB to install itself. GRUB's device naming is funny, but you have to give the partition number-1. If your duplicated root partition is /dev/hdc7, then the name to give GRUB is (hd0,6). Here's how you run grub and get it to install: you start GRUB, using the device map file that says what the BIOS drive number will be after we swap the disks ...
```
grub --device-map=device-map.foo
root (hd0,6)
setup (hd0)
quit
```

I think that's it. Now swop the disks, and boot up the duplicate system.

8.3 Method 3 (best for LILO) (hard, but you don't need a rescue disk)

If your system boots with LILO, you can set up the second disk to be ready to boot when it becomes the first disk:

Boot up the original system with the duplicate hard disk in (probably you are here already)
Chroot into the new system.
Lie to LILO about where its hard disks are
Install LILO (on the second disk)
Tell LILO the truth again for posterity

Here's the detail:

Mount the duplicated root partition:
```
mount /dev/hdc3 /mnt
```
Chroot into the new system, mount additional filesystems (e.g. /boot):
```
chroot /mnt
mount -a
```
Now edit /etc/lilo.conf, and tell it that the first disk is your duplicate disk (/dev/hdc). You need to add this data:
```
disk=/dev/hdc
 bios=0x80
disk=/dev/hda
 bios=0x80
```
Now, tell LILO to install itself on the new hard disk.
```
lilo -b /dev/hdc
```

I think that's it. Now swop the disks, and boot up the duplicate system. You may want to remove the modifications from lilo.conf on the new disk, but you could happily leave them there too.

8.4 Method 4 (easier, but more likely to fail) (actually, 100% likely)

This method involves booting up off a floppy disk. The only problem is that you cannot use an initial root disk (initrd), which may make scsi and reiserfs and ext3 systems fail. Actually, it's not very likely to work at all Actually, it's not very likely to work at all. It won't work. Do this if you're desparate.

Create a boot disk containing the Linux kernel:

dd if=/boot/vmlinuz of=/dev/fd0 bs=18k

Set the root partition

rdev /dev/fd0 /dev/hda3

Shutdown the original system.
Remove the duplicate hard disk.
Put the duplicate disk in the new system.
Boot up the duplicate system using the kernel disk you made.
It should boot as a duplicate of the original system.
Login as root and install lilo:
```
lilo
```

The system should now be bootable. If it is not, you can repeat this step.

9. History

Version 0.1: Thanks to Brett Geer for pointing out that cp -vax /* /mnt is wrong. That was not pretty.

Version 0.2: Added some GRUB data ... and a bit of fstab and menu.lst notes.

Did I mention you will destroy your data?

Someone from Spain says there's a "mistake mounting partitions", but didn't elaborate. So be careful there.

Gotcha: Newer systems may mount a temporary filesystem over /dev for devfs or udev. This causes cp -a to be incomplete, and the result is an unbootable system (missing /dev/console). To avoid this, the root filesystem of the source must be remounted somewhere where the real contents of /dev are not hidden. That's a TODO.

Stagira's Blog - Tips and Tricks

Monday, July 30, 2007

Linux Disc Duplication