Monday, August 13, 2018

"Found 3 configured voting files but 2 voting files are required" after upgrade to 12.2.0.1.180717

After upgrade Grid Infrastructure from 12.1 to 12.2 we obtained the message:

CRS-1705: Found 3 configured voting files but 2 voting files are required, terminating to ensure data integrity; details at (:CSSNM00021:) in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc


The all steps of upgrade were successful. There were no errors on the screen and in the log files.
No database soft was touched, no OS software was touched, the only GI was upgraded.
But all 4 virtual machines upgraded from 12.1 to 12.2.0.1.180817 were in bad state: ClusterWare not started automatically. The Linux restart did not help.

 Previous upgrade of 5th VM few day ago was successful and we obtained the working configuration.

The CW alert log is:
[CSSDMONITOR(121305)]CRS-8500: Oracle Clusterware CSSDMONITOR process is starting with operating system process ID 121305
[CSSDAGENT(121338)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 121338
[OHASD(224984)]CRS-2878: Failed to restart resource 'ora.storage'
[ORAROOTAGENT(228831)]CRS-5021: Check of storage failed: details at "(:CLSN00117:)" in "/u01/app/oracle/diag/crs/mr01vm03/crs/trace/ohasd_orarootagent_root.trc"
[CSSDAGENT(123152)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 123152
[OCSSD(123165)]CRS-8500: Oracle Clusterware OCSSD process is starting with operating system process ID 123165
[OCSSD(123165)]CRS-1713: CSSD daemon is started in hub mode
[OCSSD(123165)]CRS-1705: Found 3 configured voting files but 2 voting files are required, terminating to ensure data integrity; details at (:CSSNM00021:) in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc
[OCSSD(123165)]CRS-1656: The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc
[OCSSD(123165)]CRS-1652: Starting clean up of CRSD resources.
[OCSSD(123165)]CRS-1653: The clean up of the CRSD resources failed.
[OCSSD(123165)]CRS-8503: Oracle Clusterware process OCSSD with operating system process ID 123165 experienced fatal signal or exception code 6.
Errors in file /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc  (incident=729): CRS-8503 [] [] [] [] [] [] [] [] [] [] [] []
Incident details in: /u01/app/oracle/diag/crs/mr01vm03/crs/incident/incdir_729/ocssd_i729.trc


Very strange message: "Found 3 configured voting files but 2 voting files are required,"

SOLUTION:  move voting files to other ASM diskgroup.

The commands log:

[root@mr01vm03 ~]# cd /u01/app/12.2.0.1/grid/bin/
[root@mr01vm03 bin]# ./crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.crf' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.diskmon' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.evmd' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'mr01vm03'
CRS-2677: Stop of 'ora.drivers.acfs' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.crf' on 'mr01vm03' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'mr01vm03'
CRS-2677: Stop of 'ora.gpnpd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.evmd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.cssdmonitor' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'mr01vm03' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'mr01vm03' has completed
CRS-4133: Oracle High Availability Services has been stopped.
 

[root@mr01vm03 bin]# ./crsctl start crs -excl
 
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'mr01vm03'
CRS-2672: Attempting to start 'ora.mdnsd' on 'mr01vm03'
CRS-2676: Start of 'ora.mdnsd' on 'mr01vm03' succeeded
CRS-2676: Start of 'ora.evmd' on 'mr01vm03' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'mr01vm03'
CRS-2676: Start of 'ora.gpnpd' on 'mr01vm03' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'mr01vm03'
CRS-2672: Attempting to start 'ora.gipcd' on 'mr01vm03'
CRS-2676: Start of 'ora.cssdmonitor' on 'mr01vm03' succeeded
CRS-2676: Start of 'ora.gipcd' on 'mr01vm03' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'mr01vm03'
CRS-2672: Attempting to start 'ora.diskmon' on 'mr01vm03'
CRS-2676: Start of 'ora.diskmon' on 'mr01vm03' succeeded
CRS-2676: Start of 'ora.cssd' on 'mr01vm03' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'mr01vm03'
CRS-2672: Attempting to start 'ora.ctssd' on 'mr01vm03'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'mr01vm03'
CRS-2676: Start of 'ora.crf' on 'mr01vm03' succeeded
CRS-2676: Start of 'ora.ctssd' on 'mr01vm03' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'mr01vm03' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'mr01vm03'
CRS-2676: Start of 'ora.asm' on 'mr01vm03' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'mr01vm03'
CRS-2676: Start of 'ora.storage' on 'mr01vm03' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'mr01vm03'
CRS-2676: Start of 'ora.crsd' on 'mr01vm03' succeeded





CW 12.2 log:
[OHASD(126854)]CRS-8500: Oracle Clusterware OHASD process is starting with operating system process ID 126854
[OHASD(126854)]CRS-0714: Oracle Clusterware Release 12.2.0.1.0.
[OHASD(126854)]CRS-2112: The OLR service started on node mr01vm03.
[OHASD(126854)]CRS-1301: Oracle High Availability Service started on node mr01vm03.
[OHASD(126854)]CRS-8017: location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred
Create Relation ADR_CONTROL_AUX
Create Relation DFW_PURGE
[CSSDAGENT(126987)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 126987
[ORAROOTAGENT(126973)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 126973
[CSSDMONITOR(126992)]CRS-8500: Oracle Clusterware CSSDMONITOR process is starting with operating system process ID 126992
[ORAAGENT(126983)]CRS-8500: Oracle Clusterware ORAAGENT process is starting with operating system process ID 126983
[ORAAGENT(127057)]CRS-8500: Oracle Clusterware ORAAGENT process is starting with operating system process ID 127057
[EVMD(127078)]CRS-8500: Oracle Clusterware EVMD process is starting with operating system process ID 127078
[MDNSD(127076)]CRS-8500: Oracle Clusterware MDNSD process is starting with operating system process ID 127076
[GPNPD(127143)]CRS-8500: Oracle Clusterware GPNPD process is starting with operating system process ID 127143
Create Relation DFW_PURGE_ITEM
[CSSDMONITOR(127195)]CRS-8500: Oracle Clusterware CSSDMONITOR process is starting with operating system process ID 127195
[GIPCD(127197)]CRS-8500: Oracle Clusterware GIPCD process is starting with operating system process ID 127197
[GPNPD(127143)]CRS-2328: GPNPD started on node mr01vm03.
[CSSDAGENT(127235)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 127235
[OCSSD(127253)]CRS-8500: Oracle Clusterware OCSSD process is starting with operating system process ID 127253
[OCSSD(127253)]CRS-1713: CSSD daemon is started in hub mode
[OCSSD(127253)]CRS-1707: Lease acquisition for node mr01vm03 number 1 completed
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.5;192.168.10.6/DATAC3_CD_02_mrceladm01; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.7;192.168.10.8/DATAC3_CD_02_mrceladm02; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.9;192.168.10.10/DATAC3_CD_02_mrceladm03; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1601: CSSD Reconfiguration complete. Active nodes are mr01vm03 .
[OCSSD(127253)]CRS-1720: Cluster Synchronization Services daemon (CSSD) is ready for operation.
[OSYSMOND(127434)]CRS-8500: Oracle Clusterware OSYSMOND process is starting with operating system process ID 127434
[OCTSSD(127440)]CRS-8500: Oracle Clusterware OCTSSD process is starting with operating system process ID 127440
[OCTSSD(127440)]CRS-2403: The Cluster Time Synchronization Service on host mr01vm03 is in observer mode.
[OCTSSD(127440)]CRS-2407: The new Cluster Time Synchronization Service reference node is host mr01vm03.
[OCTSSD(127440)]CRS-2401: The Cluster Time Synchronization Service started on host mr01vm03.
[ORAAGENT(127057)]CRS-5011: Check of resource "ora.asm" failed: details at "(:CLSN00006:)" in "/u01/app/oracle/diag/crs/mr01vm03/crs/trace/ohasd_oraagent_oracle.trc"
[CRSD(127895)]CRS-8500: Oracle Clusterware CRSD process is starting with operating system process ID 127895
[CRSD(127895)]CRS-1012: The OCR service started on node mr01vm03.


[root@mr01vm03 bin]# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
1. ONLINE   5789f55a211a4fadbf851c10d7d2f56d (o/192.168.10.7;192.168.10.8/DATAC3_CD_02_mrceladm02) [DATAC3]
2. ONLINE   00a2d7ffaa294f77bf5e1941f5f3f9b3 (o/192.168.10.5;192.168.10.6/DATAC3_CD_02_mrceladm01) [DATAC3]
3. ONLINE   7179ec74f6474f96bfd0cbace76f4e6e (o/192.168.10.9;192.168.10.10/DATAC3_CD_02_mrceladm03) [DATAC3]
Located 3 voting disk(s).
 

[root@mr01vm03 bin]# ./crsctl replace votedisk +RECOC3
 

Successful addition of voting disk 2d8f371e02d54ffabf33e189ed5c42c7.
Successful addition of voting disk dbba5f46190a4f1dbf73753552e2e82b.
Successful addition of voting disk d6caa92e49234f9ebfe6fe749ea7dac9.
Successful deletion of voting disk 5789f55a211a4fadbf851c10d7d2f56d.
Successful deletion of voting disk 00a2d7ffaa294f77bf5e1941f5f3f9b3.
Successful deletion of voting disk 7179ec74f6474f96bfd0cbace76f4e6e.
Successfully replaced voting disk group with +RECOC3.
CRS-4266: Voting file(s) successfully replaced


CW 12.2. log:
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.7;192.168.10.8/RECOC3_CD_02_mrceladm02; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.5;192.168.10.6/RECOC3_CD_04_mrceladm01; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.9;192.168.10.10/RECOC3_CD_02_mrceladm03; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1604: CSSD voting file is offline: o/192.168.10.5;192.168.10.6/DATAC3_CD_02_mrceladm01; details at (:CSSNM00069:) in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1604: CSSD voting file is offline: o/192.168.10.7;192.168.10.8/DATAC3_CD_02_mrceladm02; details at (:CSSNM00069:) in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1604: CSSD voting file is offline: o/192.168.10.9;192.168.10.10/DATAC3_CD_02_mrceladm03; details at (:CSSNM00069:) in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1626: A Configuration change request completed successfully
[OCSSD(127253)]CRS-1601: CSSD Reconfiguration complete. Active nodes are mr01vm03 .

[root@mr01vm03 bin]# ./crsctl replace votedisk +DATAC3
 

Successful addition of voting disk 7493e8b7db5d4f5cbfe5a4113b260499.
Successful addition of voting disk 93a9cb21b7954fc5bf099cde51b6d71c.
Successful addition of voting disk c1026a740b244f12bf650d5727f5313d.
Successful deletion of voting disk 2d8f371e02d54ffabf33e189ed5c42c7.
Successful deletion of voting disk dbba5f46190a4f1dbf73753552e2e82b.
Successful deletion of voting disk d6caa92e49234f9ebfe6fe749ea7dac9.
Successfully replaced voting disk group with +DATAC3.
 

CW 12.2 log:
CRS-4266: Voting file(s) successfully replaced

[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.7;192.168.10.8/DATAC3_CD_02_mrceladm02; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.5;192.168.10.6/DATAC3_CD_02_mrceladm01; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1605: CSSD voting file is online: o/192.168.10.9;192.168.10.10/DATAC3_CD_02_mrceladm03; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1604: CSSD voting file is offline: o/192.168.10.7;192.168.10.8/RECOC3_CD_02_mrceladm02; details at (:CSSNM00069:) in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(127253)]CRS-1626: A Configuration change request completed successfully
[OCSSD(127253)]CRS-1601: CSSD Reconfiguration complete. Active nodes are mr01vm03 .

[root@mr01vm03 bin]# ./crsctl query css votedisk
 

##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
1. ONLINE   7493e8b7db5d4f5cbfe5a4113b260499 (o/192.168.10.7;192.168.10.8/DATAC3_CD_02_mrceladm02) [DATAC3]
2. ONLINE   93a9cb21b7954fc5bf099cde51b6d71c (o/192.168.10.5;192.168.10.6/DATAC3_CD_02_mrceladm01) [DATAC3]
3. ONLINE   c1026a740b244f12bf650d5727f5313d (o/192.168.10.9;192.168.10.10/DATAC3_CD_02_mrceladm03) [DATAC3]
Located 3 voting disk(s).
 

[root@mr01vm03 bin]# ./crsctl stop crs
 

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.crsd' on 'mr01vm03'
CRS-2677: Stop of 'ora.crsd' on 'mr01vm03' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.crf' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'mr01vm03'
CRS-2677: Stop of 'ora.drivers.acfs' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.crf' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.storage' on 'mr01vm03' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'mr01vm03'
CRS-2677: Stop of 'ora.mdnsd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.asm' on 'mr01vm03' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'mr01vm03'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'mr01vm03' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.evmd' on 'mr01vm03'
CRS-2677: Stop of 'ora.ctssd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.evmd' on 'mr01vm03' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'mr01vm03'
CRS-2677: Stop of 'ora.cssd' on 'mr01vm03' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'mr01vm03'
CRS-2673: Attempting to stop 'ora.gipcd' on 'mr01vm03'
CRS-2677: Stop of 'ora.gipcd' on 'mr01vm03' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'mr01vm03' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'mr01vm03' has completed
 

CW 12.2 log:
CRS-4133: Oracle High Availability Services has been stopped.
[GPNPD(127143)]CRS-2329: GPNPD on node mr01vm03 shut down.
[MDNSD(127076)]CRS-5602: mDNS service stopping by request.
[MDNSD(127076)]CRS-8504: Oracle Clusterware MDNSD process with operating system process ID 127076 is exiting
[OCTSSD(127440)]CRS-2405: The Cluster Time Synchronization Service on host mr01vm03 is shutdown by user
[OCTSSD(127440)]CRS-8504: Oracle Clusterware OCTSSD process with operating system process ID 127440 is exiting
[OCSSD(127253)]CRS-1603: CSSD on node mr01vm03 has been shut down.
[OCSSD(127253)]CRS-1660: The CSS daemon shutdown has completed
[OCSSD(127253)]CRS-8504: Oracle Clusterware OCSSD process with operating system process ID 127253 is exiting
[ORAROOTAGENT(126973)]CRS-5822: Agent '/u01/app/12.2.0.1/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) {0:1:6} in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ohasd_orarootagent_root.trc.

[root@mr01vm03 bin]# ./crsctl start crs

CRS-4123: Oracle High Availability Services has been started.


CW 12.2 log:
[OHASD(132769)]CRS-8500: Oracle Clusterware OHASD process is starting with operating system process ID 132769
[OHASD(132769)]CRS-0714: Oracle Clusterware Release 12.2.0.1.0.
[OHASD(132769)]CRS-2112: The OLR service started on node mr01vm03.
[OHASD(132769)]CRS-1301: Oracle High Availability Service started on node mr01vm03.
[OHASD(132769)]CRS-8017: location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred
[ORAROOTAGENT(132854)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 132854
[CSSDAGENT(132870)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 132870
[CSSDMONITOR(132882)]CRS-8500: Oracle Clusterware CSSDMONITOR process is starting with operating system process ID 132882
[ORAAGENT(132866)]CRS-8500: Oracle Clusterware ORAAGENT process is starting with operating system process ID 132866
[ORAAGENT(132932)]CRS-8500: Oracle Clusterware ORAAGENT process is starting with operating system process ID 132932
[MDNSD(132950)]CRS-8500: Oracle Clusterware MDNSD process is starting with operating system process ID 132950
[EVMD(132954)]CRS-8500: Oracle Clusterware EVMD process is starting with operating system process ID 132954
[GPNPD(133002)]CRS-8500: Oracle Clusterware GPNPD process is starting with operating system process ID 133002
[GPNPD(133002)]CRS-2328: GPNPD started on node mr01vm03.
[GIPCD(133059)]CRS-8500: Oracle Clusterware GIPCD process is starting with operating system process ID 133059
[CSSDMONITOR(133145)]CRS-8500: Oracle Clusterware CSSDMONITOR process is starting with operating system process ID 133145
[CSSDAGENT(133161)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 133161
[OCSSD(133176)]CRS-8500: Oracle Clusterware OCSSD process is starting with operating system process ID 133176
[OCSSD(133176)]CRS-1713: CSSD daemon is started in hub mode
[OCSSD(133176)]CRS-1707: Lease acquisition for node mr01vm03 number 1 completed
[OCSSD(133176)]CRS-1605: CSSD voting file is online: o/192.168.10.5;192.168.10.6/DATAC3_CD_02_mrceladm01; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(133176)]CRS-1605: CSSD voting file is online: o/192.168.10.7;192.168.10.8/DATAC3_CD_02_mrceladm02; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(133176)]CRS-1605: CSSD voting file is online: o/192.168.10.9;192.168.10.10/DATAC3_CD_02_mrceladm03; details in /u01/app/oracle/diag/crs/mr01vm03/crs/trace/ocssd.trc.
[OCSSD(133176)]CRS-1601: CSSD Reconfiguration complete. Active nodes are mr01vm03 .
[OCSSD(133176)]CRS-1720: Cluster Synchronization Services daemon (CSSD) is ready for operation.
[OCTSSD(133414)]CRS-8500: Oracle Clusterware OCTSSD process is starting with operating system process ID 133414
[OCTSSD(133414)]CRS-2403: The Cluster Time Synchronization Service on host mr01vm03 is in observer mode.
[OCTSSD(133414)]CRS-2401: The Cluster Time Synchronization Service started on host mr01vm03.
[OCTSSD(133414)]CRS-2407: The new Cluster Time Synchronization Service reference node is host mr01vm03.
[OSYSMOND(133806)]CRS-8500: Oracle Clusterware OSYSMOND process is starting with operating system process ID 133806
[CRSD(133818)]CRS-8500: Oracle Clusterware CRSD process is starting with operating system process ID 133818
[OLOGGERD(133919)]CRS-8500: Oracle Clusterware OLOGGERD process is starting with operating system process ID 133919
[CRSD(133818)]CRS-1012: The OCR service started on node mr01vm03.
[CRSD(133818)]CRS-1201: CRSD started on node mr01vm03.
[ORAAGENT(134062)]CRS-8500: Oracle Clusterware ORAAGENT process is starting with operating system process ID 134062
[ORAROOTAGENT(134088)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 134088
[ORAAGENT(134062)]CRS-5011: Check of resource "dbm03" failed: details at "(:CLSN00007:)" in "/u01/app/oracle/diag/crs/mr01vm03/crs/trace/crsd_oraagent_oracle.trc"
[CRSD(133818)]CRS-2772: Server 'mr01vm03' has been assigned to pool 'Generic'.
[CRSD(133818)]CRS-2772: Server 'mr01vm03' has been assigned to pool 'ora.dbm03'.

[root@mr01vm03 bin]# ./crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       mr01vm03                 STABLE
ora.DATAC1.dg
               OFFLINE OFFLINE      mr01vm03                 STABLE
ora.DATAC3.dg
               ONLINE  ONLINE       mr01vm03                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       mr01vm03                 STABLE
ora.RECOC1.dg
               OFFLINE OFFLINE      mr01vm03                 STABLE
ora.RECOC3.dg
               ONLINE  ONLINE       mr01vm03                 STABLE
ora.chad
               ONLINE  OFFLINE      mr01vm03                 STABLE
ora.net1.network
               ONLINE  ONLINE       mr01vm03                 STABLE
ora.ons
               ONLINE  ONLINE       mr01vm03                 STABLE
ora.proxy_advm
               OFFLINE OFFLINE      mr01vm03                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       mr01vm03                 169.254.47.1 192.168
                                                             .10.15 192.168.10.16
                                                             ,STABLE
ora.asm
      1        ONLINE  ONLINE       mr01vm03                 Started,STABLE
ora.cvu
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.dbm03.db
      1        ONLINE  OFFLINE      mr01vm03                 STARTING
ora.mgmtdb
      1        ONLINE  OFFLINE      mr01vm03                 Instance Shutdown,ST
                                                             ARTING
ora.mr01vm03.vip
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.qosmserver
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       mr01vm03                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       mr01vm03                 STABLE
--------------------------------------------------------------------------------
[root@mr01vm03 bin]#







1 comment:

  1. I think problem with incorrect upgrading gpnp profile, that break ID of voting files. You are not alone with this problem: Exadata : 12c CRS Clusterware Cannot Start Due To Missing Voting Disks "CRS-1637: Unable to locate configured voting file with ID", "CRS-1705: Found 1 configured voting files but 2 voting files are required" (Doc ID 2294135.1)

    Dmitry Gavrilychev

    ReplyDelete

Note: Only a member of this blog may post a comment.

How to disable/setup autostart parameters for specified instance ?

Q: We have a 4-node RAC. I need to disable autostart of the DB on one node only.    How to do it and how to see autostart parameters, confir...