Software RAID on Linux

Linux Software RAID - An unorganized and random comments page which may or may not be helpful to you. *Created: Mon Mar 29 18:28:01 CST 2004

A RAID 5 Example

/boot on the boot partition cannot be RAID 5. This is a limitation in Linux software raid. For this example I have 3 IDE harddrives set up for RAID level five and a seperate raid partition for the following:

/boot assigned to md0 with RAID level 1 
/tmp  assigned to md2 with RAID level 5 
/     assigned to md3 with RAID level 5 
/home assigned to md4 with RAID level 5 

and the linux swap partition on md1 RAID level 5.

Three harddrives are hda, hdb, and hdc.

md0 is a raid partition, consisting of hda1, hdb1, and hdc1. A complete table of raid partitions with drive assignments is as follows:

hda1 + hdb1 + hdc1 assigned to md0 (/boot)
hda2 + hdb2 + hdc2 assigned to md4 (/home)
hda3 + hdb3 + hdc3 assigned to md3 (/)
hda5 + hdb5 + hdc5 assigned to md2 (/tmp)
hda6 + hdb6 + hdc6 assigned to md1 (swap)

(Q). Did everything load ok with RAID when I booted?

Take a look at the boot messages and see for yourself. Type the following command at the console:

'dmesg|less'

(Q). Hows RAID doing now? or Has a drive died while its running?

Take a look at the raid status by typing the following command at the console:

'cat /proc/mdstat'

(Q). What does it mean when I see "kicking hda1" or "kicking" whatever in the dmesg and RAID / the system will not start?

Looks like you have a bad drive or the drive is fine but RAID seems to think there is a problem. See which drive was kicked and try using the following command to add it back into the array: 'raidhotadd /dev/md0 /dev/hda1' replacing md0 with the array that the drive was kicked from and hda1 with the actual drive that was kicked from that array. Be careful not to accidently use raidhotadd to add a drive partition into the wrong array. Linux/RAID will let you do this but you will regret it.

raidhotadd and raidhotremove will only work on a running array. If only 1 out of the 3 drive partitions are functioning within an array, then raidhotadd and raidhotremove are useless.

You can usually tell what drive belongs to what array by looking at your RAID configuration file, which is /etc/raidtab.

(*). Useful command summary:

vi /etc/raidtab
dmesg
cat /proc/mdstat
raidhotadd /dev/md? /dev/hd??
raidhotadd
raidhotremove
lsraid -a /dev/md0
lsraid -A -d /dev/hdc2

(man). lsraid usage examples

• lsraid -A -a /dev/md0

Display a short listing of the md0 device.

• lsraid -A -d /dev/sda1

Display a short listing of the array that sda1 belongs to.

• lsraid -A -f -a /dev/md0

Display the failed devices belonging to the md0 device.

• lsraid -D -l -a /dev/md0

Display a long dump of the on-disk md superblock of every disk in

md0.

• lsraid -D -a /dev/md0 -d /dev/sda1

Display a short discription of the disks in md0 as well as a short

description of the disk sda1. sda1 will only be described once if

it belongs to md0.

• lsraid -R -a /dev/md0 -a /dev/md1 -a /dev/md2

Display a description of the arrays in an output format suitable

for using in raidtab(5) files. Note that if md0 and md1 are raid0

arrays and md2 is a raid1 created from md0 and md1, this command

will output the information in the correct order.

• lsraid -R -p

Scan all block devices in /proc/partitions and display all discov-

ered md devices in a format suitable for using in raidtab(5) files.

Command Reference by Example

vi /etc/raidtab                 

dmesg                           

cat /proc/mdstat                

raidhotadd /dev/mdN /dev/hdXN   

raidhotremove                   

lsraid -a /dev/mdN              

lsraid -A -d /dev/hdcN          

raidstop                        

raidsetfaulty                   

mkraid -f (dangerous!)          

badblocks                       

mount /proc /proc -t proc