No great genius has ever existed without some touch of madness.

Monday 31 July 2017

ORA-27506 ORA-27300 ORA-27301 ORA-27302 ORA-27303 SKGXP IPC libraries must be the same version. [local: UDP,remote: RDS]

Found errors in /u01/app/oracle/admin/*****tst/bdump/alert_*****tst1.log
=====
ORA-27506: IPC error connecting to a port
ORA-27300: OS system dependent operation:proto mismatch failed with status: 0
ORA-27301: OS failure message: Error 0
ORA-27302: failure occurred at: skgxpcon
ORA-27303: additional information: Protocol of this IPC does not match remote (***.***.***.85). SKGXP IPC libraries must be the same version. [local: UDP,remote: RDS]
ORA-27506: IPC error connecting to a port
ORA-27300: OS system dependent operation:proto mismatch failed with status: 0
ORA-27301: OS failure message: Error 0
ORA-27302: failure occurred at: skgxpcon
ORA-27303: additional information: Protocol of this IPC does not match remote (***.***.***.85). SKGXP IPC libraries must be the same version. [local: UDP,remote: RDS]


Errors in file /u01/app/oracle/diag/rdbms/*****tst/*****tst2/trace/*****tst2_lmon_23968.trc
=====
ORA-27550: Target ID protocol check failed. tid vers=1, type=1, remote instance number=1, local instance number=2
LMON (ospid: 23968): terminating the instance due to error 481
LMON caught an error 27550 in the main loop

  
Issue was with 1st node using UDP and 2nd Node using RDS. However, both should use UDP. Confirmed this through +ASM1 and +ASM2 alert logs --


+ASM1

+ASM2

Solution - revert back 2nd node to use UDP instead of RDS (after shutting down database)
Reference -
Oracle Clusterware and RAC Support for RDS over InfiniBand (Doc ID 751343.1)