The system.
# cat /etc/issue /etc/debian_version Debian GNU/Linux 6.0 \n \l 6.0.7
Install the MD devices --Linux Software RAID-- Manager
# apt-get install mdadm
The System is installed on a ~250GB hard drive.
# fdisk -l Disk /dev/sda: 250.1 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00096f89 Device Boot Start End Blocks Id System /dev/sda1 * 1 29933 240430080 83 Linux /dev/sda2 29933 30402 3766273 5 Extended /dev/sda5 29933 30402 3766272 82 Linux swap / Solaris
Shutdown the system, add one more ~250GB disk, and boot
What partitions does the system see?
cat /proc/partitions major minor #blocks name 8 0 244198584 sda 8 1 240430080 sda1 8 2 1 sda2 8 5 3766272 sda5 8 16 244198584 sdbYupe, the second drive --sdb is identical to the first drive --sda.
List all active md devices and their status
# cat /proc/mdstat Personalities : unused devices:
Copy the sda partition scheme to sdb.
# sfdisk -d /dev/sda | sfdisk --force /dev/sdb
Check the partition tables with sfdisk.
# sfdisk -l Disk /dev/sdb: 30401 cylinders, 255 heads, 63 sectors/track Warning: extended partition does not start at a cylinder boundary. DOS and Linux will interpret the contents differently. Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End #cyls #blocks Id System /dev/sdb1 * 0+ 29932- 29933- 240430080 83 Linux /dev/sdb2 29932+ 30401- 469- 3766273 5 Extended /dev/sdb3 0 - 0 0 0 Empty /dev/sdb4 0 - 0 0 0 Empty /dev/sdb5 29932+ 30401- 469- 3766272 82 Linux swap / Solaris Disk /dev/sda: 30401 cylinders, 255 heads, 63 sectors/track Warning: extended partition does not start at a cylinder boundary. DOS and Linux will interpret the contents differently. Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End #cyls #blocks Id System /dev/sda1 * 0+ 29932- 29933- 240430080 83 Linux /dev/sda2 29932+ 30401- 469- 3766273 5 Extended /dev/sda3 0 - 0 0 0 Empty /dev/sda4 0 - 0 0 0 Empty /dev/sda5 29932+ 30401- 469- 3766272 82 Linux swap / Solaris
Check the partition tables with fdisk.
# fdisk -l Disk /dev/sdb: 250.1 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xdb0cbcd3 Device Boot Start End Blocks Id System /dev/sdb1 * 1 29933 240430080 83 Linux /dev/sdb2 29933 30402 3766273 5 Extended /dev/sdb5 29933 30402 3766272 82 Linux swap / Solaris Disk /dev/sda: 250.1 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00096f89 Device Boot Start End Blocks Id System /dev/sda1 * 1 29933 240430080 83 Linux /dev/sda2 29933 30402 3766273 5 Extended /dev/sda5 29933 30402 3766272 82 Linux swap / Solaris
Change the sdb1 and sdb5 partitions Ox ID to 'Linux raid auto' ( fd )
# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): t
Partition number (1-5): 1
Hex code (type L to list codes): L
0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris
1 FAT12 39 Plan 9 82 Linux swap / So c1 DRDOS/sec (FAT-
2 XENIX root 3c PartitionMagic 83 Linux c4 DRDOS/sec (FAT-
3 XENIX usr 40 Venix 80286 84 OS/2 hidden C: c6 DRDOS/sec (FAT-
4 FAT16 <32M 41 PPC PReP Boot 85 Linux extended c7 Syrinx
5 Extended 42 SFS 86 NTFS volume set da Non-FS data
6 FAT16 4d QNX4.x 87 NTFS volume set db CP/M / CTOS / .
7 HPFS/NTFS 4e QNX4.x 2nd part 88 Linux plaintext de Dell Utility
8 AIX 4f QNX4.x 3rd part 8e Linux LVM df BootIt
9 AIX bootable 50 OnTrack DM 93 Amoeba e1 DOS access
a OS/2 Boot Manag 51 OnTrack DM6 Aux 94 Amoeba BBT e3 DOS R/O
b W95 FAT32 52 CP/M 9f BSD/OS e4 SpeedStor
c W95 FAT32 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi eb BeOS fs
e W95 FAT16 (LBA) 54 OnTrackDM6 a5 FreeBSD ee GPT
f W95 Ext'd (LBA) 55 EZ-Drive a6 OpenBSD ef EFI (FAT-12/16/
10 OPUS 56 Golden Bow a7 NeXTSTEP f0 Linux/PA-RISC b
11 Hidden FAT12 5c Priam Edisk a8 Darwin UFS f1 SpeedStor
12 Compaq diagnost 61 SpeedStor a9 NetBSD f4 SpeedStor
14 Hidden FAT16 <3 63 GNU HURD or Sys ab Darwin boot f2 DOS secondary
16 Hidden FAT16 64 Novell Netware af HFS / HFS+ fb VMware VMFS
17 Hidden HPFS/NTF 65 Novell Netware b7 BSDI fs fc VMware VMKCORE
18 AST SmartSleep 70 DiskSecure Mult b8 BSDI swap fd Linux raid auto
1b Hidden W95 FAT3 75 PC/IX bb Boot Wizard hid fe LANstep
1c Hidden W95 FAT3 80 Old Minix be Solaris boot ff BBT
1e Hidden W95 FAT1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): t
Partition number (1-5): 5
Hex code (type L to list codes): fd
Changed system type of partition 5 to fd (Linux raid autodetect)
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Check sdb partitions.
# fdisk -l /dev/sdb Disk /dev/sdb: 250.1 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xdb0cbcd3 Device Boot Start End Blocks Id System /dev/sdb1 * 1 29933 240430080 fd Linux raid autodetect /dev/sdb2 29933 30402 3766273 5 Extended /dev/sdb5 29933 30402 3766272 fd Linux raid autodetect
Zero-out any remains of RAID arrays on /dev/sdb
# mdadm --zero-superblock /dev/sdb1 # mdadm --zero-superblock /dev/sdb5
Create the RAID arrays using the older 0.90 superblock format.
# mdadm --create /dev/md0 --level=1 --metadata=0 --raid-disks=2 missing /dev/sdb1 mdadm: array /dev/md0 started. # mdadm --create /dev/md1 --level=1 --metadata=0 --raid-disks=2 missing /dev/sdb5 mdadm: array /dev/md1 started."The version-0.90 superblock limits the number of component devices within an array to 28, and limits each component device to a maximum size of 2TB on kernel version <3.1 and 4TB on kernel version >=3.1."
List MD devices and MD Status.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdb5[1]
3766208 blocks [2/1] [_U]
md0 : active raid1 sdb1[1]
240430016 blocks [2/1] [_U]
unused devices:
Create File Systems on MD devices.
# mkfs.ext3 -v -m .1 -b 4096 -E stride=32,stripe-width=64 /dev/md0 # mkswap /dev/md1
Set /etc/mdadm/mdadm.conf
# cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf.0Delete ARRAY definitions in /etc/mdadm/mdadm.conf
# mdadm --examine --scan >> /etc/mdadm/mdadm.conf
Mount md0
# mkdir /mnt/md0 # mount /dev/md0 /mnt/md0
Set
/dev/md0 to mount at /
and
/dev/md1 as swap
in /etc/fstab for now
# mkdir /etc/bak # cp /etc/fstab /etc/bak/fstab.0 # vi /etc/fstab # cat /etc/fstab |grep md /dev/md0 / ext3 errors=remount-ro 0 1 /dev/md1 none swap sw 0 0
Configure Grub (2).
# cp /etc/grub.d/40_custom /etc/grub.d/09_raid1 # uname -r 2.6.32-5-amd64Set /etc/grub.d/09_raid1
# vi /etc/grub.d/09_raid1
# cat /etc/grub.d/09_raid1
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64 raid1' --class debian --class gnu-linux --class gnu --class os {
insmod raid
insmod mdraid
insmod part_msdos
insmod ext2
set root='(md0)'
echo 'Loading Linux 2.6.32-5-amd64 ....'
linux /boot/vmlinuz-2.6.32-5-amd64 root=/dev/md0 ro quiet
echo 'Loading initial ramdisk ... .'
initrd /boot/initrd.img-2.6.32-5-amd64
}
Uncomment GRUB_DISABLE_LINUX_UUID=true in /etc/default/grub
# vi /etc/default/grub
Back up /etc/mtab
# cp /etc/mtab /etc/bak/mtab.0Replace /dev/sda1 with /dev/md0 in /etc/mtab.
Update Grub
# grub-mkconfig -o /boot/grub/grub.cfgUpdate the initramfs image
# update-initramfs -u
Copy the system to /dev/md0
# cp -dpRx / /mnt/md0/
Install grub on both drives.
# grub-install /dev/sdb # grub-install /dev/sda
Reboot
# shutdown -r now
Check if md is used.
# df -h Filesystem Size Used Avail Use% Mounted on /dev/md0 226G 1.6G 213G 1% / tmpfs 943M 0 943M 0% /lib/init/rw udev 936M 152K 936M 1% /dev tmpfs 943M 0 943M 0% /dev/shm
Set the Ox partition IDs on /dev/sda to "Linux RAID auto" ( fd ).
# for part in 1 5;do sfdisk --change-id /dev/sda $part fd;done
Add /dev/sda5 to the /dev/md1 RAID array
# mdadm --add /dev/md1 /dev/sda5 mdadm: added /dev/sda5Check
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda5[2] sdb5[1]
3766208 blocks [2/1] [_U]
[>....................] recovery = 3.3% (124800/3766208) finish=7.7min speed=7800K/sec
md0 : active raid1 sdb1[1]
240430016 blocks [2/1] [_U]
unused devices:
Add /dev/sda1 to /dev/md0
# mdadm --add /dev/md0 /dev/sda1 mdadm: added /dev/sda1
Check MD Status again.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda5[2] sdb5[1]
3766208 blocks [2/1] [_U]
[=>...................] recovery = 6.2% (234368/3766208) finish=55.2min speed=1064K/sec
md0 : active raid1 sda1[2] sdb1[1]
240430016 blocks [2/1] [_U]
resync=DELAYED
unused devices:
Wait until the 're'sync is done.
Come on!
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda5[0] sdb5[1]
3766208 blocks [2/2] [UU]
md0 : active raid1 sda1[2] sdb1[1]
240430016 blocks [2/1] [_U]
[===========>.........] recovery = 59.1% (142333696/240430016) finish=23.1min speed=70622K/sec
unused devices:
Finally.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda5[0] sdb5[1]
3766208 blocks [2/2] [UU]
md0 : active raid1 sda1[0] sdb1[1]
240430016 blocks [2/2] [UU]
unused devices:
Install Grub2 everywhere and reboot.
# grub-install /dev/sdb # grub-install /dev/sda # grub-install /dev/md0 # reboot
Test! What would happen in a hard drive failure?
Shutdown the system and pull out a hard disk drive.
# shutdown -h nowPull out /dev/sdb and Boot.
The system came up with one drive and a degraded Array.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active (auto-read-only) raid1 sda5[0]
3766208 blocks [2/1] [U_]
md0 : active raid1 sda1[0]
240430016 blocks [2/1] [U_]
unused devices:
Let's test how it would come up if the /dev/sda fails.
# shutdown -h nowPut in /dev/sdb, pull out /dev/sda, and boot the system.
Great, the system came up again with one drive on a degraded Array.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active (auto-read-only) raid1 sda5[1]
3766208 blocks [2/1] [_U]
md0 : active raid1 sda1[1]
240430016 blocks [2/1] [_U]
unused devices:
# shutdown -h nowPut in /dev/sda again and boot the system.
The system comes up fine.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active (auto-read-only) raid1 sda5[0] sdb5[1]
3766208 blocks [2/2] [UU]
md0 : active raid1 sdb1[1]
240430016 blocks [2/1] [_U]
unused devices:
Oups, sda1 is missing from the RAID 1 Array md0.
# mdadm -a /dev/md0 /dev/sda1 mdadm: re-added /dev/sda1Check
# cat /proc/mdstat
Personalities : [raid1]
md1 : active (auto-read-only) raid1 sda5[0] sdb5[1]
3766208 blocks [2/2] [UU]
md0 : active raid1 sda1[2] sdb1[1]
240430016 blocks [2/1] [_U]
[>....................] recovery = 0.6% (1466496/240430016) finish=88.4min speed=45012K/sec
unused devices:
Of course in case of a real hard drive failure the new drive entered is not going to have the same partition table with the system nor the superblock information, nor grub2 installed. If the new drive entered is identical to the one used by the RAID Array the steps describing how to add a drive to the RAID 1 drive Array should work fine, eg:
# sfdisk -d /dev/OLD-GOOD-DRIVE | sfdisk --force /dev/NEW-REPLACEMENT-DRIVE # mdadm --zero-superblock /dev/NEW-REPLACEMENT-DRIVE-PARTITIONS # mdadm -a /dev/RAID-DEVICE /dev/NEW-REPLACEMENT-DRIVE-PARTITIONS # grub-install /dev/NEW-REPLACEMENT-DRIVE
If the new hard drive is not an identical drive you will have to work a bit more creating partitions that match the ones used by the system.
Well, it is not a good idea to use a RAID 1 Array on a swap md ... And you can get away without creating a RAID 0 array for swap. You could use /dev/sda5 and /dev/sdb5 as 2 different swap partitions.
References:
RAID superblock formats
Raid Setup
Swapping_on_RAID
Software RAID 1 on Debian GNU Linux
