Thursday, June 6, 2013

Re-create corrupted ASM disks

Problem description:

Following 3 disks can see via OS but not ASM:
OCRVOTED01
OCRVOTED02
OCRVOTED03

# multipath -l|grep OCR
shared_asm_OCRVOTED01 (1HITACHI_770140123364) dm-24 HITACHI,DF600F
shared_asm_OCRVOTED02 (1HITACHI_770140123365) dm-25 HITACHI,DF600F
shared_asm_OCRVOTED03 (1HITACHI_770140123366) dm-26 HITACHI,DF600F
shared_asm_OCRVOTED04 (1HITACHI_770140123367) dm-27 HITACHI,DF600F
shared_asm_OCRVOTED05 (1HITACHI_770140123368) dm-28 HITACHI,DF600F

# /etc/init.d/oracleasm listdisks |grep OCR
OCRVOTED04
OCRVOTED05

Cause:
Disks are corrupted

Action Plan:

1. Try to recreate asm disk 

/etc/init.d/oracleasm createdisk OCRVOTED01 /dev/mapper/shared_asm_OCRVOTED01
Marking disk "OCRVOTED01" as an ASM disk: [FAILED]

found following message from /var/log/oracleasm
Device "/dev/mapper/shared_asm_OCRVOTED01" is already labeled for ASM disk ""


2. Overwrite the ASM header information on the disk. This can be achieved with the UNIX command dd. The following command will write 100x1024b blocks to the specified raw device:
dd if=/dev/zero of=/dev/mapper/shared_asm_OCRVOTED01 bs=1024 count=100
dd if=/dev/zero of=/dev/mapper/shared_asm_OCRVOTED02 bs=1024 count=100
dd if=/dev/zero of=/dev/mapper/shared_asm_OCRVOTED03 bs=1024 count=100

3. Recreate asm disks
/etc/init.d/oracleasm createdisk OCRVOTED01 /dev/mapper/shared_asm_OCRVOTED01
/etc/init.d/oracleasm createdisk OCRVOTED02 /dev/mapper/shared_asm_OCRVOTED02
/etc/init.d/oracleasm createdisk OCRVOTED03 /dev/mapper/shared_asm_OCRVOTED03 

4. Run oracleasm scandisks on both nodes
/etc/init.d/oracleasm scandisks

5. Verify
# /etc/init.d/oracleasm listdisks |grep OCR
OCRVOTED01
OCRVOTED02
OCRVOTED03
OCRVOTED04
OCRVOTED05

No comments:

Post a Comment