Tuesday, July 15, 2014

Exadata X4 multipathing with active-active infiniband

The X4 Exadata has the new feature "active-active" ports on the IB HCA.
As a result the v$asm_disk.PATH shows the 2 IP for the each disk path:

SQL> select path from v$asm_disk;

PATH
-------------------------------------------------------
o/192.168.11.9;192.168.11.10/RECOC1_CD_06_ed02celadm03
o/192.168.11.9;192.168.11.10/DATAC1_CD_09_ed02celadm03
o/192.168.11.9;192.168.11.10/DBFS_DG_CD_04_ed02celadm03
o/192.168.11.9;192.168.11.10/DBFS_DG_CD_03_ed02celadm03
o/192.168.11.9;192.168.11.10/DATAC1_CD_00_ed02celadm03
o/192.168.11.9;192.168.11.10/RECOC1_CD_02_ed02celadm03



[oracle@ed02dbadm01 ~]$ /sbin/ifconfig -a

ib0       Link encap:InfiniBand  HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.1 


ib1       Link encap:InfiniBand  HWaddr 80:00:00:49:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.2


As you can see above the DB node host 'ed02dbadm01' has the IPs 11.1 and 11.2, but disks have IPs 11.9 and 11.10. Therefore the disks are addressed with cell IP addresses, in our case 'ed02celadm03'. Let look its ifconfig:
[root@ed02celadm03 ~]# ifconfig -a

ib0       Link encap:InfiniBand  HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.9

ib1       Link encap:InfiniBand  HWaddr 80:00:00:49:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.10


Yes,

And let's look to cellip.ora on DB nodes, it now has 2 IPs because active-active ports (old version has 1 IP because active-passive ports):

[root@ed02dbadm01 ~]# cat /etc/oracle/cell/network-config/cellip.ora
cell="192.168.11.5;192.168.11.6"
cell="192.168.11.7;192.168.11.8"
cell="192.168.11.9;192.168.11.10"


No comments:

Post a Comment

ZDLRA. ra_install.jar. Exception in thread "main" java.sql.SQLException: ORA-28040

If you run the  Oracle Backup module ra_install.jar and got the message: Exception in thread "main" java.sql.SQLException: ORA-2...