Tuesday, May 21, 2013

Deleted trace files still holding by Oracle process

Problem description

delete an big trace file under bdump , but storage was not released

Symptoms
$ du -sh /u02/oracle/rac
31M

$ df -h /u02/oracle/rac
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/rac3--vg01-u02_rac
                       16G   15G  316M  98% /u02/oracle/rac

$/usr/sbin/lsof /u02/oracle/niids
COMMAND   PID   USER   FD   TYPE  DEVICE        SIZE   NODE NAME
COMMAND   PID   USER   FD   TYPE  DEVICE        SIZE   NODE NAME
oracle   1177 oracle    2w   REG 253,239         798  81929 /u02/oracle/rac/logs/bdump/RAC3_ora_1177.trc
oracle   1177 oracle    5w   REG 253,239         893  17213 /u02/oracle/rac/logs/udump/RAC3_ora_25913.trc (deleted)
oracle   1177 oracle    6w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   1177 oracle    8w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   1949 oracle    5w   REG 253,239         893  17213 /u02/oracle/rac/logs/udump/RAC3_ora_25913.trc (deleted)
oracle   1949 oracle    6w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   1949 oracle    8w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   7147 oracle    2w   REG 253,239         743  89415 /u02/oracle/rac/logs/bdump/RAC3_nsv0_7147.trc (deleted)
oracle   7147 oracle    5w   REG 253,239         893  17213 /u02/oracle/rac/logs/udump/RAC3_ora_25913.trc (deleted)
oracle   7147 oracle    6w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   7147 oracle    8w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   7147 oracle    9w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   9497 oracle    2w   REG 253,239      229376  89437 /u02/oracle/rac/logs/bdump/RAC3_lns1_9497.trc
oracle   9497 oracle    5w   REG 253,239         893  17213 /u02/oracle/rac/logs/udump/RAC3_ora_25913.trc (deleted)
oracle   9497 oracle    6w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   9497 oracle    8w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle   9497 oracle    9w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
ksh     13250 oracle  cwd    DIR 253,239        4096      2 /u02/oracle/rac
oracle  14578 oracle    5w   REG 253,239         893  17213 /u02/oracle/rac/logs/udump/RAC3_ora_25913.trc (deleted)
oracle  14578 oracle    6w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle  14578 oracle    8w   REG 253,239        2565  81922 /u02/oracle/rac/logs/bdump/alert_RAC3.log
oracle  17116 oracle    2w   REG 253,239 15650070528  89291 /u02/oracle/rac/logs/bdump/RAC3_j000_17116.trc (deleted)
oracle  17116 oracle    5w   REG 253,239         893  17213 /u02/oracle/rac/logs/udump/RAC3_ora_25913.trc (deleted)

Cause:
There is still oracle process holding deleted files


Solution:

restart the instance

srvctl stop instance -d RAC -i RAC3
srvctl start instance -d RAC -i RAC3

No comments:

Post a Comment