Thursday, July 23, 2015

RedHat Enterprise Linux Server Backup and Restore for Disaster Recovery Purpose

http://unixadminschool.com/blog/2015/07/redhat-enterprise-linux-server-backup-and-restore-for-disaster-recovery-purpose


This article is to explain the procedure to backup and restore a RHEL server for the purpose of Disaster recovery situation. For the purpose of simplification we will  be backing of the entire server data into an additional disk of the same server. But in real time scenarios, the backup should be taken to a remote server / tape/ backup device. 
Assumptions that we made for this procedure:
1. We are going to backup and restore the data to same server with same disk configuration.
2. We are not making any change to partition table or MBR ( Master Boot Record) , and so we are not taking backup of the MBR.
3. If you want to restore data to a complete new disk, you must recreate the partition table as per the original server, otherwise the server wont’ boot.
4. We are assuming that our Linux server having only non-LVM root partition. If your server is having LVM root file system, then the procedure will change a little bit before you start the restore.

Backup Procedure

Step 1:  Take note of below below configuration information and files,

# cat /etc/fstab
LABEL=/ / ext3 defaults 1 1
LABEL=/boot1 /boot ext3 defaults 1 2
LABEL=/data /data ext3 defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SWAP-sda5 swap swap defaults 0 0
/dev/sda6 /backup-files ext3 defaults 0 0
# fdisk -l
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1925 15358140 83 Linux
/dev/sda3 1926 3200 10241437+ 83 Linux
/dev/sda4 3201 4864 13366080 5 Extended
/dev/sda5 3201 3391 1534176 82 Linux swap / Solaris
/dev/sda6 3392 4864 11831841 83 Linux

Step  2: Backup all the partitions of RHEL server

Note: In this example, we will use the /dev/sda6 partition to save backup files, and we assume that /dev/sda6 is mounted on /backup-files.

Step 2a: Bootup your system into Single User Mode

Single user mode can be accessed by appending an “S“, “s“, or “single” to the kernel command line in GRUB. Likewise, a “3” can be used to boot to runlevel 3. To do this, restart the system and when the GRUB splash screen presents itself:
  1. Select/highlight the desired kernel using the up/down arrow keys.
  2. Press the letter a to modify the kernel line.
  3. On the new screen, press the spacebar to add a space, then type the letter s and press Enter.
This will boot the system into single user mode, i.e. the boot process will stop immediately after execution of rc.sysinit and present a root BASH shell.

Step 2b; Use “dump” command to backup the contents of the partitions

If the system has been running for a long time, it is advisable to run e2fsck on the partitions before backup.  Please note that “dump command ”  should) not be used at heavily loaded and mounted filesystem as it could backup corrupted version of files.
If you want to take the Backups into Local Disk i.e . /dev/sda6
# dump -0uf /backup-files/sda1.dump /dev/sda1
# dump -0uf /backup-files/sda2.dump /dev/sda2
# dump -0uf /backup-files/sda3.dump /dev/sda3
If you want to the backups into remote server
Start the network services in single user mode using the below command
#/sbin/service network restart
Dump the local disk date to remote server i.e. bkpserver.igurkul.com
# dump -0u -f – /dev/sda1 | ssh root@bkpserver.igurkul.com dd of=/tmp/sda1.dump

Simulate the Disk Failure Scenario 


             We can simulate a disk failure by using fdisk to delete the partitions that have been backed up.

Restore the Backup data and test the Server

Step 1:  Bootup your system into Rescue Mode using below procedure

  • Get the first disc of installation media for the major version of the installed Red Hat Enterprise Linux (or the binary DVD). 
  • Note: boot.iso can be used for booting instead of the installation media, but the installation tree (including rescue programs) is still needed in HTTP server or anywhere.
Follow the instructions given by the rescue boot process:
  • Boot the system from boot disc
  • once the system has successfully booted from the ISO image and Red Hat Enterprise Linux boot screen will appear.     
      Type: linux rescue without the quotes, and hit enter at the prompt.
    [F1-Main] [F2-Options] [F3-General] [F4-Kernel] [F5-Rescue]

    boot: linux rescue
  • When prompted for language, keyboard, and network, provide the pertinent information for the system
  • Select: Continue when prompted to allow the rescue environment to mount Red Hat Enterprise Linux installation under /mnt/sysimage directory.
  • Run the command chroot /mnt/sysimage and then chroot to your system image.

Step 2: Rebuild sda1/sda2/sda3/sda4/sda5 by using the fdisk command

Notice: When rebuild all the partitions, please use the same “Start” and “End” number as before.  

Step 3: Create file systems

Create New filesystem on top of sda1, sda2, sda3 and sda5 by using the mkfs command, as shown below.
Note: DO NOT format /dev/sda6 because it saves backup files.
# mkfs.ext3 /dev/sda1
# mkfs.ext3 /dev/sda2
# mkfs.ext3 /dev/sda3

Step 4: Re-label all the partitions

So they match how they were before. This step is important for the restore to work.
# e2label /dev/sda1 /boot1
# e2label /dev/sda2 /
# e2label /dev/sda3 /data
# mkswap -L SWAP-sda5 /dev/sda5

Step 5: Prepare the working directories and mount the target filesystems

# mkdir /mnt/sda1
# mount -t ext3 /dev/sda1 /mnt/sda1
# mkdir /mnt/sda2
# mount -t ext3 /dev/sda2 /mnt/sda2
# mkdir /mnt/sda3
# mount -t ext3 /dev/sda3 /mnt/sda3
# mkdir /backup-files
# mount -t ext3 /dev/sda6 /backup-files

Step 6: Restore the data

To Restore data from local disks:
# cd /mnt/sda1
# restore -rf /backup-files/sda1.dump
# cd /mnt/sda2
# restore -rf /backup-files/sda2.dump
# cd /mnt/sda3
# restore -rf /backup-files/sda3.dump
To restore data from Remote Server  i.e. bkpserver.igurkul.com
login into bkpserver.igurkul.com, and restore sda1 from sda1.dump file:
# cd /mnt/sda1
# /usr/bin/ssh restore -r -f bkpserver.igurkul.com:/tmp/sda1.dump

Step 7: Reboot the server to verify that it is booting properly


Procedure to backup and Restore the Linux Partition Table:

Procedure A:    Using  the “sfdisk” command:

Run the command below to backup the partition table on device /dev/sda to /root/partition-sda.img:
# sfdisk -d /dev/sda > /root/partition-sda.img
You should then copy /root/partition-sda.img to some other storage, for example a portable USB disk.
If the partition is damaged and needs to be restored from backup, please connect your USB disk to the server and boot the server with installation media and enter rescue mode.
Do not mount the root partition at this time – select “Skip” when the system asks if you want to mount the root partition on /mnt/sysimage.
Create a temporary directory, for example /mnt/temp and mount the filesystem of your USB device which contains your backup. For example:
# mkdir /mnt/temp
# mount /dev/sdb1 /mnt/temp/
# cd /mnt/temp/
where /dev/sdb1 is the file system of the USB disk. After that, run:
# sfdisk /dev/sda < /mnt/temp/partition-sda.img
To verify that the partition table has been restored, run:
fdisk -l /dev/sda

Procedure B: Using  the “dd” command:

The MBR (Master Boot Record) occupies the first 446 bytes of the disk while the partition table occupies the next 64 bytes. We can use “dd” to dump the range from 447 – 510 bytes in the first sector.

 For example, if the harddisk is /dev/sda, then run the command below:
# dd if=/dev/sda of=/root/partition-sda.img bs=1 count=64 skip=446
You should then copy /root/partition-sda.img to some other storage, for example a portable USB disk.  If the partition is damaged and needs to be restored from backup, please connect your USB disk to the server and boot the server with installation media and enter rescue mode. 
Do not mount the root partition at this time – select “Skip” when the system asks if you want to mount the root partition on /mnt/sysimage.
Create a temporary directory, for example /mnt/temp and mount the filesystem of your USB device which contains your backup. For example:
# mkdir /mnt/temp
# mount /dev/sdb1 /mnt/temp/
# cd /mnt/temp/
where /dev/sdb1 is the file system of the USB disk. After that, run:
# dd if=/mnt/temp/partition-sda.img of=/dev/sda bs=1 count=64 seek=446
To verify that the partition table has been restored, run:
# fdisk -l /dev/sda