How to Add Exadata Storage Servers

Как известно, Оракл разрешает расширять Экзадату путем добавления новых серверов хранения. Вот попалась нота на этот счет:

How to Add Exadata Storage Servers to an Existing Database Machine (Doc ID 1476336.1)

Самое интересное в ней – правила:

• Within a rack, you can only add the same kinds of disks as what already exists.  E.g., if you already have high capacity (HC) disks, then you can only add HC disks and similarly if you already have high performance (HP) disks you can only add HP disks
• When adding an additional rack or storage expansion rack, you may add either HC or HP disks, but if the new disks are a different type from the existing ones, then you must add them to a new diskgroup containing only that disk type. E.g., your existing disks are HP and you add an HC storage expansion rack; the new disks must go into a new diskgroup

В переводе на русский:

• В пределах одной стойки все диски должны быть одного типа.Если нужны диски другого типа, то добавляй новую стойку.
• Если добавились диски другого типа, то из них надо создавать отдельную дисковую группу. Иными словами, в одной дисковой группе нельзя смешивать диски разного типа.

How to enable the Flash Cache Compression ?

Prereqs:
- should i shutdown databases ?
- should i shutdown clusterware ?

My experiment say - no, i didn't shutdown DB soft or clusterware (CW). I just connected to cells with terminal program and made my operations. The DB and ASM alert logs showed no errors during this operation.


If you make the "list cell detail" before this operation then you don't see the
"flashCacheCompress" parameter. It is absent.

And a size of FlashCache is:

[root@exa2celadm01 ~]# cellcli -e list flashcache detail|grep -i Size
     effectiveCacheSize:     2978.75G
     size:                   2978.75G


The "alter flashcache all flush" command may take a very long time, for example, 1+ hour. So run it on the all cells simultaneously some times before your operation.


My commands:

CellCLI> alter flashcache all flush
Flash cache ed02celadm01_FLASHCACHE altered successfully

CellCLI> drop flashcache all       
Flash cache ed02celadm01_FLASHCACHE successfully dropped

CellCLI> drop flashlog all         
Flash log ed02celadm01_FLASHLOG successfully dropped

CellCLI> drop celldisk all flashdisk
CellDisk FD_00_ed02celadm01 successfully dropped
CellDisk FD_01_ed02celadm01 successfully dropped
CellDisk FD_02_ed02celadm01 successfully dropped
CellDisk FD_03_ed02celadm01 successfully dropped
CellDisk FD_04_ed02celadm01 successfully dropped
CellDisk FD_05_ed02celadm01 successfully dropped
CellDisk FD_06_ed02celadm01 successfully dropped
CellDisk FD_07_ed02celadm01 successfully dropped
CellDisk FD_08_ed02celadm01 successfully dropped
CellDisk FD_09_ed02celadm01 successfully dropped
CellDisk FD_10_ed02celadm01 successfully dropped
CellDisk FD_11_ed02celadm01 successfully dropped
CellDisk FD_12_ed02celadm01 successfully dropped
CellDisk FD_13_ed02celadm01 successfully dropped
CellDisk FD_14_ed02celadm01 successfully dropped
CellDisk FD_15_ed02celadm01 successfully dropped

CellCLI>

CellCLI> alter cell FlashCacheCompress=true
Cell ed02celadm01 successfully altered

CellCLI>

CellCLI> create celldisk all flashdisk
CellDisk FD_00_ed02celadm01 successfully created
CellDisk FD_01_ed02celadm01 successfully created
CellDisk FD_02_ed02celadm01 successfully created
CellDisk FD_03_ed02celadm01 successfully created
CellDisk FD_04_ed02celadm01 successfully created
CellDisk FD_05_ed02celadm01 successfully created
CellDisk FD_06_ed02celadm01 successfully created
CellDisk FD_07_ed02celadm01 successfully created
CellDisk FD_08_ed02celadm01 successfully created
CellDisk FD_09_ed02celadm01 successfully created
CellDisk FD_10_ed02celadm01 successfully created
CellDisk FD_11_ed02celadm01 successfully created
CellDisk FD_12_ed02celadm01 successfully created
CellDisk FD_13_ed02celadm01 successfully created
CellDisk FD_14_ed02celadm01 successfully created
CellDisk FD_15_ed02celadm01 successfully created

CellCLI> create flashlog all size=512M
Flash log ed02celadm01_FLASHLOG successfully created

CellCLI> create flashcache all
Flash cache ed02celadm01_FLASHCACHE successfully created

CellCLI>

CellCLI> list cell detail      
     name:                   ed02celadm01
     bbuChargeThreshold:     800
     bbuStatus:              normal
     bbuTempThreshold:       60
     bmcType:                IPMI
     cellVersion:            OSS_12.1.1.1.0_LINUX.X64_131219
     cpuCount:               24
     diagHistoryDays:        7
     fanCount:               8/8
     fanStatus:              normal
     flashCacheMode:         WriteBack
     flashCacheCompress:     TRUE
     id:                     1340NM502N
     interconnectCount:      3
     interconnect1:          ib0
     interconnect2:          ib1
     iormBoost:              0.0
     ipaddress1:             192.168.11.5/22
     ipaddress2:             192.168.11.6/22
     kernelVersion:          2.6.39-400.128.1.el5uek
     locatorLEDStatus:       off
     makeModel:              Oracle Corporation SUN SERVER X4-2L High Capacity
     metricHistoryDays:      7
     notificationMethod:     snmp
     notificationPolicy:     critical,warning,clear
     offloadGroupEvents:
     offloadEfficiency:      481,755.6
     powerCount:             2/2
     powerStatus:            normal
     releaseVersion:         12.1.1.1.0
     releaseTrackingBug:     16980054
     snmpSubscriber:         host=ed02dbadm02.distr.fors.ru,port=3872,community=public
                             host=ed02dbadm01.distr.fors.ru,port=3872,community=public
     status:                 online
     temperatureReading:     24.0
     temperatureStatus:      normal
     upTime:                 50 days, 18:13
     cellsrvStatus:          running
     msStatus:               running
     rsStatus:               running


And size of FlashCache in our X4-2 after modification is about 2 times bigger:

[root@ed02celadm01 ~]# cellcli -e list flashcache detail|grep -i Size
     effectiveCacheSize:     5959G
     size:                   5959G