Tuesday, July 15, 2014

Exadata X4 multipathing with active-active infiniband

The X4 Exadata has the new feature "active-active" ports on the IB HCA.
As a result the v$asm_disk.PATH shows the 2 IP for the each disk path:

SQL> select path from v$asm_disk;

PATH
-------------------------------------------------------
o/192.168.11.9;192.168.11.10/RECOC1_CD_06_ed02celadm03
o/192.168.11.9;192.168.11.10/DATAC1_CD_09_ed02celadm03
o/192.168.11.9;192.168.11.10/DBFS_DG_CD_04_ed02celadm03
o/192.168.11.9;192.168.11.10/DBFS_DG_CD_03_ed02celadm03
o/192.168.11.9;192.168.11.10/DATAC1_CD_00_ed02celadm03
o/192.168.11.9;192.168.11.10/RECOC1_CD_02_ed02celadm03



[oracle@ed02dbadm01 ~]$ /sbin/ifconfig -a

ib0       Link encap:InfiniBand  HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.1 


ib1       Link encap:InfiniBand  HWaddr 80:00:00:49:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.2


As you can see above the DB node host 'ed02dbadm01' has the IPs 11.1 and 11.2, but disks have IPs 11.9 and 11.10. Therefore the disks are addressed with cell IP addresses, in our case 'ed02celadm03'. Let look its ifconfig:
[root@ed02celadm03 ~]# ifconfig -a

ib0       Link encap:InfiniBand  HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.9

ib1       Link encap:InfiniBand  HWaddr 80:00:00:49:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
          inet addr:192.168.11.10


Yes,

And let's look to cellip.ora on DB nodes, it now has 2 IPs because active-active ports (old version has 1 IP because active-passive ports):

[root@ed02dbadm01 ~]# cat /etc/oracle/cell/network-config/cellip.ora
cell="192.168.11.5;192.168.11.6"
cell="192.168.11.7;192.168.11.8"
cell="192.168.11.9;192.168.11.10"


No comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Does DEALLOCATE UNUSED or SHRINK SPACE will free space occupied by LOB segment?

Lets check how it works. My env is DB 19.20@Linux-x64 1) I created the table with 4 LOB columns of 4 different LOB types: BASICFILE BLOB, BA...