The System Resource Verifier (aka sysresv) is an Oracle utility that gives you the status and OS resource information for a database instance. It shows you information on the shared memory and semaphore usage of the database and is extremely useful when you need to remove orphaned Oracle memory segments and avoiding a costly and time consuming reboot.

Overview

Oracle uses shared memory segments and semaphores from the OS for the SGA, PGA and other database related resources. When the database starts, it reserves contiguous memory segments and semaphores according the database parameters in the spfile or parameter file. There are times when a database has crashed and leaves memory segments and/or semaphores orphaned. If you have a large amount of memory in the machine, Oracle may still be able to start the database because it can still find the additional memory that it needs. But, if your memory usage is high and the database cannot allocate the memory that it wants, the database will not start.

Sysresv has been available since 8i and is installed as a part of the RDBMS installation in $ORACLE_HOME/bin. It is available on most platforms (except for Windows) and runs as the Oracle user so it does not require elevated privileges to work.

The tool is great with a single database on a server, but I think that where it is most useful is when you have multiple databases on a server. Sysresv looks at all memory segments, not just those that are associated with instances running from its Oracle home. That means when you run the command it looks at the entire machine, even ignoring the RDBMS version. It allows you to confidently and safely remove segments without impacting other running instances.

Let’s take a quick look at how it works, and then we can take a look at an example of real world usage.

Running the command

First, we will run the command by itself. The level of detail given does differ depending on which version of the utility you run:

Version 11g

$ sysresv

IPC Resources for ORACLE_SID "DH11204A" :
Shared Memory:
ID              KEY
33947655        0x00000000
33980424        0x00000000
34013193        0x6cd3407c
Semaphores:
ID              KEY
622597          0xd2e3fa10
Oracle Instance alive for sid "DH11204A"

Version 12c

$ sysresv

IPC Resources for ORACLE_SID "DH12102A" :
Maximum shared memory segment size (shmmax): 4398046511104 bytes
Total system shared memory (shmall): 4398046511104 bytes
Total system shared memory count (shmmni): 4096
*********************** Dumping ipcs output ********************

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status      
0x00000000 296747011  oracle     640        3715072    169                     
0x00000000 296779780  oracle     640        2130706432 85                      
0x00000000 296812549  oracle     640        13062144   85                      
0x68d8048c 296845318  oracle     640        12288      85                      
0x00000000 715751431  oracle     640        16777216   27                      
0x00000000 715784200  oracle     640        2130706432 27                      
0x6cd3407c 715816969  oracle     640        2097152    27                      

------ Semaphore Arrays --------
key        semid      owner      perms      nsems     
0xafe3c6e0 294915     oracle     640        152       
0xafe3c6e1 327684     oracle     640        152       
0xafe3c6e2 360453     oracle     640        152       
0xd2e3fa10 884742     oracle     640        154       

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages    

*********************** End of ipcs command dump **************
 

***************** Dumping Resource Limits(s/h) *****************
core file size                         0 KB/UNLIMITED         
data seg size                     UNLIMITED/UNLIMITED         
scheduling priority                    0 KB/0 KB              
file size                         UNLIMITED/UNLIMITED         
pending signals                      250 KB/250 KB            
max locked memory                    128 GB/128 GB            
max memory size                   UNLIMITED/UNLIMITED         
open files                            64 KB/64 KB             
POSIX message queues                 800 KB/800 KB            
real-time priority                     0 KB/0 KB              
stack size                            32 MB/32 MB             
cpu time                          UNLIMITED/UNLIMITED         
max user processes                    16 KB/16 KB             
virtual memory                    UNLIMITED/UNLIMITED         
file locks                        UNLIMITED/UNLIMITED         

***************** End of Resource Limits Dump ******************
Total /dev/shm size: 16776966144 bytes, used: 245760 bytes
Shared Memory:
ID              KEY
296779780       0x00000000
296812549       0x00000000
296747011       0x00000000
296845318       0x68d8048c
Semaphores:
ID              KEY
294915          0xafe3c6e0
327684          0xafe3c6e1
360453          0xafe3c6e2
Oracle Instance alive for sid "DH12102A"

There are only a few optional parameters with the utililty:

-i Prompt before automatically removing orphaned resources. (I recommend using this)

-f Automatically remove the orphaned resources.

-d on|off If this is on (default), resource information is listed for running instances

-l SID Specify SID(s) to trace in memory. Separate multiple SIDs with spaces. The current SID is the default.

 

When it runs, it checks to see if the memory and semaphores are in use with a running instance. If the segments are not associated with a running instance it will remove the segments from memory automatically if you don’t add –i or will prompt you to remove them.

An example

Now, let’s take a look at an actual use of sysresv from an issue I had last year!

Connecting to the database shows that you are connecting to an idle instance

$ sqlplus / as sysdba
 
SQL*Plus: Release 12.1.0.2.0 Production on Wed Nov 18 17:41:54 2015
 
Copyright (c) 1982, 2014, Oracle.  All rights reserved.
 
Connected to an idle instance.

Attempting to start the instance fails with a shared memory error, so let’s exit

SQL> startup nomount;
ORA-27100: shared memory realm already exists
Linux-x86_64 Error: 17: File exists
Additional information: 3760
Additional information: 73728
 SQL> exit
Disconnected

Run the sysresv command to find the orphaned memory segment. It gives you several lines of memory and semaphore information. The program will give you option to remove the orphaned segments.

$ $ORACLE_HOME/bin/sysresv -i
 
*********************** Dumping process map ********************
00400000-00403000 r-xp 00000000 fd:1b 2755976                            /orasw/app/oracle/product/12.1.0/db_121021/bin/sysresv
00602000-00603000 rw-p 00002000 fd:1b 2755976                            /orasw/app/oracle/product/12.1.0/db_121021/bin/sysresv
0205e000-0207f000 rw-p 00000000 00:00 0                                  [heap]
a0000000-a0012000 rw-s 00000000 00:04 263553035                          /SYSV0ec9c084 (deleted)
.
.
. truncated for brevity
.
.
3113039         0xcdbdc4a6
3178577         0xcdbdc4a7
3244115         0xcdbdc4a8
Oracle Instance not alive for sid "D12RAC2"
Remove ipc resources for sid "D12RAC2" (y/n)?y
Done removing ipc resources for sid "D12RAC2"

Log into SQLPlus again and restart the instance

$ sqlplus / as sysdba
 
SQL*Plus: Release 12.1.0.2.0 Production on Wed Nov 18 18:15:26 2015
 
Copyright (c) 1982, 2014, Oracle.  All rights reserved.
 
Connected to an idle instance.
 SQL> startup
ORACLE instance started.

Total System Global Area 5016387584 bytes
Fixed Size                  2934696 bytes
Variable Size            1040189528 bytes
Database Buffers         3959422976 bytes
Redo Buffers               13840384 bytes
Database mounted.
Database opened.

Sysresv is not a tool that you will use every day. But those times that you need it, you will be very glad that you have it.