Deinstall ORACLE_HOME

After upgrade to the new GI (from 19.15 to 19.16) spend enough time, all env variables points to the new GI home, so has become the time to delete the old GI home. 

As we know, the GI home has the file $GI_HOME/deinstall/deinstall which do all the work instead of us. You should run this script as grid owner. New OS environment variables don't confuse the deinstall script, it checks what it remove.

How it work:

[root@rac1 deinstall]# ./deinstall
You must not be logged in as root to run ./deinstall.
Log in as Oracle user and rerun ./deinstall.

[grid@rac1 deinstall]$ ./deinstall
Checking for required files and bootstrapping ...
Please wait ...
Location of logs /u01/app/oraInventory/logs/

############ ORACLE DECONFIG TOOL START ############


######################### DECONFIG CHECK OPERATION START #########################
## [START] Install check configuration ##


Checking for existence of the Oracle home location /u01/app/19.15
Oracle Home type selected for deinstall is: Oracle Grid Infrastructure for a Cluster
Oracle Base selected for deinstall is: /u01/app/grid
Checking for existence of central inventory location /u01/app/oraInventory
Checking for existence of the Oracle Grid Infrastructure home /u01/app/19.16/grid
The following nodes are part of this cluster: rac1
Checking for sufficient temp space availability on node(s) : 'rac1'

## [END] Install check configuration ##

Traces log file: /u01/app/oraInventory/logs//crsdc_2023-08-14_09-39-54-AM.log

Network Configuration check config START

Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_check2023-08-14_09-39-54AM.log

Specify all RAC listeners (do not include SCAN listener) that are to be de-configured. Enter .(dot) to deselect all. [ASMNET1LSNR_ASM,LISTENER,LISTENER_SCAN3,LISTENER_SCAN2,LISTENER_SCAN1]:.

Network Configuration check config END

Asm Check Configuration START

ASM de-configuration trace file location: /u01/app/oraInventory/logs/asmcadc_check2023-08-14_09-39-54AM.log

ASM was not detected in the Oracle Home
Database Check Configuration START

Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_check2023-08-14_09-39-54AM.log

Oracle Grid Management database was not found in this Grid Infrastructure home

Database Check Configuration END

######################### DECONFIG CHECK OPERATION END #########################


####################### DECONFIG CHECK OPERATION SUMMARY #######################
Oracle Grid Infrastructure Home is: /u01/app/19.16/grid
The following nodes are part of this cluster: rac1
The cluster node(s) on which the Oracle home deinstallation will be performed are:rac1
Oracle Home selected for deinstall is: /u01/app/19.15
Inventory Location where the Oracle home registered is: /u01/app/oraInventory
The home being deconfigured is NOT a configured Grid Infrastructure home (/u01/app/19.16/grid)
ASM was not detected in the Oracle Home
Oracle Grid Management database was not found in this Grid Infrastructure home
Do you want to continue (y - yes, n - no)? [n]: y
A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_09-39-53-AM.out'
Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_09-39-53-AM.err'

######################## DECONFIG CLEAN OPERATION START ########################
Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_clean2023-08-14_09-39-54AM.log
ASM de-configuration trace file location: /u01/app/oraInventory/logs/asmcadc_clean2023-08-14_09-39-54AM.log
ASM Clean Configuration END

Network Configuration clean config START

Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_clean2023-08-14_09-39-54AM.log

De-configuring Naming Methods configuration file on all nodes...
Naming Methods configuration file de-configured successfully.

De-configuring Local Net Service Names configuration file on all nodes...
Local Net Service Names configuration file de-configured successfully.

De-configuring Directory Usage configuration file on all nodes...
Directory Usage configuration file de-configured successfully.

De-configuring backup files on all nodes...
Backup files de-configured successfully.

The network configuration has been cleaned up successfully.

Network Configuration clean config END


---------------------------------------->

The deconfig command below can be executed in parallel on all the remote nodes. Execute the command on  the local node after the execution completes on all the remote nodes.

Run the following command as the root user or the administrator on node "rac1".

/u01/app/19.15/crs/install/rootcrs.sh -unlock -crshome /u01/app/19.15 -paramfile "/tmp/deinstall2023-08-14_09-39-33AM/response/deinstall_OraGI19Home2.rsp"

Press Enter after you finish running the above commands

<----------------------------------------
Here i switched to next window and run rootcrs.sh:

[root@rac1 deinstall]# /u01/app/19.15/crs/install/rootcrs.sh -unlock -crshome /u01/app/19.15 -paramfile "/tmp/deinstall2023-08-14_09-39-33AM/response/deinstall_OraGI19Home2.rsp"
Using configuration parameter file: /tmp/deinstall2023-08-14_09-39-33AM/response/deinstall_OraGI19Home2.rsp
The log of current session can be found at:
  /u01/app/oraInventory/logs/crsunlock_rac1_2023-08-14_09-43-43AM.log
2023/08/14 09:43:48 CLSRSC-347: Successfully unlock /u01/app/19.15
<----------------------------------------

######################### DECONFIG CLEAN OPERATION END #########################


####################### DECONFIG CLEAN OPERATION SUMMARY #######################
There is no Oracle Grid Management database to de-configure in this Grid Infrastructure home
The home being deconfigured is NOT a configured Grid Infrastructure home (/u01/app/19.16/grid)
Oracle Clusterware was successfully unlocked on node "rac1".
#######################################################################


############# ORACLE DECONFIG TOOL END #############

Using properties file /tmp/deinstall2023-08-14_09-39-33AM/response/deinstall_2023-08-14_09-39-53-AM.rsp
Location of logs /u01/app/oraInventory/logs/

############ ORACLE DEINSTALL TOOL START ############





####################### DEINSTALL CHECK OPERATION SUMMARY #######################
A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_09-39-53-AM.out'
Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_09-39-53-AM.err'

######################## DEINSTALL CLEAN OPERATION START ########################
## [START] Preparing for Deinstall ##
Setting LOCAL_NODE to rac1
Setting CLUSTER_NODES to rac1
Setting CRS_HOME to false
Setting oracle.installer.invPtrLoc to /tmp/deinstall2023-08-14_09-39-33AM/oraInst.loc
Setting oracle.installer.local to false

## [END] Preparing for Deinstall ##

Setting the force flag to false
Setting the force flag to cleanup the Oracle Base
Oracle Universal Installer clean START

Detach Oracle home '/u01/app/19.15' from the central inventory on the local node : Done

Failed to delete the directory '/u01/app/19.15/gpnp/gpnp_bcp__2023_6_3_91739'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/u01/app/19.15/gpnp'. The directory is not empty.
Failed to delete the directory '/u01/app/19.15'. The directory is not empty.
Delete directory '/u01/app/19.15' on the local node : Failed <<<<

The Oracle Base directory '/u01/app/grid' will not be removed on local node. The directory is in use by Oracle Home '/u01/app/19.16/grid'.

Oracle Universal Installer cleanup was successful.

Oracle Universal Installer clean END


## [START] Oracle install clean ##


## [END] Oracle install clean ##


######################### DEINSTALL CLEAN OPERATION END #########################


####################### DEINSTALL CLEAN OPERATION SUMMARY #######################
Successfully detached Oracle home '/u01/app/19.15' from the central inventory on the local node.
Failed to delete directory '/u01/app/19.15' on the local node due to error : Either user has no permission to delete or file is in use.
Review the permissions and manually delete '/u01/app/19.15' on local node.
Oracle Universal Installer cleanup was successful.

Review the permissions and contents of '/u01/app/grid' on nodes(s) 'rac1'.
If there are no Oracle home(s) associated with '/u01/app/grid', manually delete '/u01/app/grid' and its contents.
Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################


############# ORACLE DEINSTALL TOOL END #############

[grid@rac1 deinstall]$

On 2nd node:

[grid@rac2 deinstall]$ cd /u01/app/19.15/deinstall/
[grid@rac2 deinstall]$ ./deinstall
Checking for required files and bootstrapping ...
Please wait ...
Location of logs /u01/app/oraInventory/logs/

############ ORACLE DECONFIG TOOL START ############


######################### DECONFIG CHECK OPERATION START #########################
## [START] Install check configuration ##

Remote nodes on which old homes will be deleted:

Checking for existence of the Oracle home location /u01/app/19.15
Oracle Home type selected for deinstall is: Oracle Grid Infrastructure for a Cluster
Oracle Base selected for deinstall is: /u01/app/grid
Checking for existence of central inventory location /u01/app/oraInventory
Checking for existence of the Oracle Grid Infrastructure home /u01/app/19.16/grid
The following nodes are part of this cluster: rac2,rac1
Checking for sufficient temp space availability on node(s) : 'rac2'

## [END] Install check configuration ##

Traces log file: /u01/app/oraInventory/logs//crsdc_2023-08-14_10-32-34-AM.log

Network Configuration check config START

Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_check2023-08-14_10-32-34AM.log

Network Configuration check config END

Asm Check Configuration START

ASM de-configuration trace file location: /u01/app/oraInventory/logs/asmcadc_check2023-08-14_10-32-34AM.log

ASM was not detected in the Oracle Home
Database Check Configuration START

Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_check2023-08-14_10-32-34AM.log

Oracle Grid Management database was not found in this Grid Infrastructure home

Database Check Configuration END

######################### DECONFIG CHECK OPERATION END #########################


####################### DECONFIG CHECK OPERATION SUMMARY #######################
Oracle Grid Infrastructure Home is: /u01/app/19.16/grid
The following nodes are part of this cluster: rac2,rac1
The cluster node(s) on which the Oracle home deinstallation will be performed are:rac2
Oracle Home selected for deinstall is: /u01/app/19.15
Inventory Location where the Oracle home registered is: /u01/app/oraInventory
The home being deconfigured is NOT a configured Grid Infrastructure home (/u01/app/19.16/grid)
ASM was not detected in the Oracle Home
Oracle Grid Management database was not found in this Grid Infrastructure home
Do you want to continue (y - yes, n - no)? [n]: y
A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_10-32-21-AM.out'
Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_10-32-21-AM.err'

######################## DECONFIG CLEAN OPERATION START ########################
Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_clean2023-08-14_10-32-34AM.log
ASM de-configuration trace file location: /u01/app/oraInventory/logs/asmcadc_clean2023-08-14_10-32-34AM.log
ASM Clean Configuration END

Network Configuration clean config START

Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_clean2023-08-14_10-32-34AM.log

De-configuring Naming Methods configuration file on all nodes...
Naming Methods configuration file de-configured successfully.

De-configuring Local Net Service Names configuration file on all nodes...
Local Net Service Names configuration file de-configured successfully.

De-configuring Directory Usage configuration file on all nodes...
Directory Usage configuration file de-configured successfully.

De-configuring backup files on all nodes...
Backup files de-configured successfully.

The network configuration has been cleaned up successfully.

Network Configuration clean config END


---------------------------------------->

The deconfig command below can be executed in parallel on all the remote nodes. Execute the command on  the local node after the execution completes on all the remote nodes.

Run the following command as the root user or the administrator on node "rac2".

/u01/app/19.15/crs/install/rootcrs.sh -unlock -crshome /u01/app/19.15 -paramfile "/tmp/deinstall2023-08-14_10-32-13AM/response/deinstall_OraGI19Home2.rsp"

Press Enter after you finish running the above commands

<----------------------------------------
[root@rac2 ~]# /u01/app/19.15/crs/install/rootcrs.sh -unlock -crshome /u01/app/19.15 -paramfile "/tmp/deinstall2023-08-14_10-32-13AM/response/deinstall_OraGI19Home2.rsp"
Using configuration parameter file: /tmp/deinstall2023-08-14_10-32-13AM/response/deinstall_OraGI19Home2.rsp
The log of current session can be found at:
  /u01/app/oraInventory/logs/crsunlock_rac2_2023-08-14_10-35-28AM.log
2023/08/14 10:35:36 CLSRSC-347: Successfully unlock /u01/app/19.15
[root@rac2 ~]#

<----------------------------------------

######################### DECONFIG CLEAN OPERATION END #########################


####################### DECONFIG CLEAN OPERATION SUMMARY #######################
There is no Oracle Grid Management database to de-configure in this Grid Infrastructure home
The home being deconfigured is NOT a configured Grid Infrastructure home (/u01/app/19.16/grid)
Oracle Clusterware was successfully unlocked on node "rac2".
#######################################################################


############# ORACLE DECONFIG TOOL END #############

Using properties file /tmp/deinstall2023-08-14_10-32-13AM/response/deinstall_2023-08-14_10-32-21-AM.rsp
Location of logs /u01/app/oraInventory/logs/

############ ORACLE DEINSTALL TOOL START ############





####################### DEINSTALL CHECK OPERATION SUMMARY #######################
A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_10-32-21-AM.out'
Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2023-08-14_10-32-21-AM.err'

######################## DEINSTALL CLEAN OPERATION START ########################
## [START] Preparing for Deinstall ##
Setting LOCAL_NODE to rac2
Setting CLUSTER_NODES to rac2
Setting CRS_HOME to false
Setting oracle.installer.invPtrLoc to /tmp/deinstall2023-08-14_10-32-13AM/oraInst.loc
Setting oracle.installer.local to false

## [END] Preparing for Deinstall ##

Setting the force flag to false
Setting the force flag to cleanup the Oracle Base
Oracle Universal Installer clean START

Detach Oracle home '/u01/app/19.15' from the central inventory on the local node : Done

Failed to delete the directory '/u01/app/19.15/gpnp/gpnp_bcp__2023_6_3_93314'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/u01/app/19.15/gpnp'. The directory is not empty.
Failed to delete the directory '/u01/app/19.15'. The directory is not empty.
Delete directory '/u01/app/19.15' on the local node : Failed <<<<

The Oracle Base directory '/u01/app/grid' will not be removed on local node. The directory is in use by Oracle Home '/u01/app/19.16/grid'.

Oracle Universal Installer cleanup was successful.

Oracle Universal Installer clean END


## [START] Oracle install clean ##


## [END] Oracle install clean ##


######################### DEINSTALL CLEAN OPERATION END #########################


####################### DEINSTALL CLEAN OPERATION SUMMARY #######################
Successfully detached Oracle home '/u01/app/19.15' from the central inventory on the local node.
Failed to delete directory '/u01/app/19.15' on the local node due to error : Either user has no permission to delete or file is in use.
Review the permissions and manually delete '/u01/app/19.15' on local node.
Oracle Universal Installer cleanup was successful.

Review the permissions and contents of '/u01/app/grid' on nodes(s) 'rac2'.
If there are no Oracle home(s) associated with '/u01/app/grid', manually delete '/u01/app/grid' and its contents.
Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################


############# ORACLE DEINSTALL TOOL END #############

[grid@rac2 deinstall]$

 Files in Grid Home:

 

CRS-6706: Oracle Clusterware Release patch level ('nnn') does not match Software patch level ('mmm')

 One day restarting clusterware i obtained CRS-6706:

[root@rac2 ~]# crsctl start cluster
CRS-6706: Oracle Clusterware Release patch level ('...') does not match Software patch level ('...').
Oracle Clusterware cannot be started.
CRS-4000: Command Start failed, or completed with errors.

Ups!

Yes, i have patched the ClusterWare some days ago, but GI/CW successfully started after node reboot. I used the new Zero Downtime GI Patching procedure as described in great article of Daniel Overby Hansen: https://dohdatabase.com/2023/03/10/how-to-patch-oracle-grid-infrastructure-19c-using-zero-downtime-oracle-grid-infrastructure-patching

and may do something wrong ...  

At first I thought it was an error after the patching procedure.

I tried to start GI/CW from other node an obtained the same: "CRS-6706: Oracle Clusterware Release patch level ('...') does not match Software patch level ('...')". This confirmed my assumption of error is somewhere in the cluster software, maybe in cluster registry ...

Google was quick to suggest a suitable note with many clever recommendations:  CRS-6706: Oracle Clusterware Release patch level ('nnn') does not match Software patch level ('mmm') (Doc ID 1639285.1) and some other interesting articles, they recommendto do:

$ORACLE_HOME/bin/clscfg -localpatch
$ORACLE_HOME/crs/install/rootcrs.sh -lock

[root@rac1 ~]# $ORACLE_HOME/bin/clscfg -localpatch
clscfg: EXISTING configuration version 0 detected.
Cannot proceed with the command execution while local OHASD is running.
Error initializing OLR for local patch operation.


[root@rac1 ~]# $ORACLE_HOME/crs/install/rootcrs.sh -lock
Using configuration parameter file: /u01/app/19.15/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/rac1/crsconfig/crslock_rac1_2023-07-26_06-30-09AM.log
Failure in execution (rc=-1, 0, No such file or directory) for command /usr/sbin/semanage fcontext --add -e /bin /u01/app/19.15/bin
2023/07/26 06:30:14 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'

 Nothig helped !

I won't torture you with my investigation. The mistake is mine and it is very easy: the wrong value of ORACLE_HOME variable. :)

For convenience, I have specify a value for ORACLE_HOME in .bash_profile for user root in order to run GI commands from root user (don't setup GI variables each time i did "su - root"). And, of course, this variable was pointing to the wrong path:

Old GI home is /u01/app/19.15/grid
New GI home is /u01/app/19.16/grid

But my variable was ORACLE_HOME=/u01/app/19.15
Nothing more!

After obvious correction CW started successfully.

You may easy reproduce this error spoiling ORACLE_HOME ...

Exadata X10M

Появились данные о том, как выглядит Экзадата Х10М-2 (то, что она уже есть - несомненно, вероятно пока работает только в Oracle Cloud):

DB node Unit -------- CPU ---------- ------------ RAM -------------- -------- PCIe- ---------- Ethernet 
E5-2L     2  2*AMD EPYC 9334 32-Core 384|512|768|1536|1920|3072|6144 x16 8GTs|16GTs|32GTs v5.0 6*25/10/1
E5-2L     2  2*AMD EPYC 9J14 96-Core 384|512|768|1536|1920|3072|6144 x16 8GTs|16GTs|32GTs v5.0 6*25/10/1
Cell                                                                                           H.Disk Flash
X10-2L HC 2  2*AMD EPYC 9334 32-Core 128|192|256|768|1536            x16 8GTs|16GTs|32GTs v5.0 12*22T  6 *7.68T 
X10-2L EF 2  2*AMD EPYC 9334 32-Core                 1536            x16 8GTs|16GTs|32GTs v5.0    -   10 *7.68T  

Основные изменения: переход на процессоры AMD (вероятно потому, что у AMD больше ядер). 

 Переход на AMD не удивляет - ещё весной 2022 года стало известно, что в своём облаке Оракл использует Экзадаты X9M на процессорах AMD, однако в on-premise Оракл эту модель не предлагает:
https://blogs.oracle.com/database/post/ ... ucture-x9m
https://www.oracle.com/a/ocom/docs/engi ... x9m-ds.pdf 

DB node Unit -------- CPU ---------- ------------ RAM -------------- -------- PCIe- ---------- Ethernet 
E4-2c     1  2*AMD EPYC 7J13 64-Core             1536|2048           x16 8GTs|16GTs v4.0       2*25/10/1  << X9M-2

 

Оракл считает, что достоинством AMD является большое количество ядер. В своём облаке Оракл назначил цену за физическое AMD-ядро в два раза меньше, чем за ядро Intel.

Основный изменения в X10M-2:

Серверы БД получат по 192 физических ядра (2 процессора по 96 ядер) и 6Т оперативной памяти, шина PCIe 5 поколения, 6 портов Ethernet 25Гбит/10Гбит/1Гбит. Некоторые серверы БД станут двух-юнитовыми.
Вызывает вопросы модель "E5-2L 2*AMD EPYC 9334 32-Core" - вероятно это сервер для тех, кому не требуется много ядер/лицензий на СУБД Оракл? Непонятная модель, поживём -увидим.

Селлы (ячейки хранения) получат шпиндельные диски по 22Т. Для компенсации возросшего объема шпиндельных дисков флеш-карт станет 6 штук, объем каждой по 8Т. Ядер и оперативной памяти в каждом селле также станет больше. В Х10М-2 исчез РМЕМ (потому что PMEM - это технология Intel). Вместо PMEM ожидаем память CXL.

Минимальная версия Экза-софта, способная работать на X10M-2 - это версия 23.1. Версия 23.1 работает на Oracle Linux 8.7 and UEK6 (5.4.17).

Поддерживаемые версии Oracle Database и Oracle Grid Infrastructure (GI): 19.15 и 21.6.
Иными словами: если ваша база ниже, чем 19.15 или 21.6, то на Х10М-2 такая БД работать не сможет. БД версий 11.2 и 12.1 и 12.2 также не смогут работать на X10M-2. Поэтому прежде, чем поставить 23.1 на свои Х5,Х6,Х7 и т.д. - предварительно обновите ваши БД до 19.5 или выше.

Обновиться на 23.1 можно только с версии 21.2.10 (March 2022) или более поздней.

Ну и несколько слов о перспективах:
- Seagate выпустит HDD емкостью 50 Тбайт в 2026 году https://servernews.ru/1088153
- AMD представила 128-ядерные EPYC Bergamo https://servernews.ru/1088342, https://servernews.ru/1088121

В общем, шпиндельные диски еще поживут.
Ядер в процессорах тоже станет больше.

 

История поиска самой лучшей СУБД от компании Uber

История метаний поисков Uber :
2013 год - Migraing Uber from MySQL to PostgreSQL
2016 год - Migrating Uber from PostgreSQL to MySQL
2023 год - Migrating Uber to Oracle Cloud

Что показывает этот случай?
При росте объемов и серьёзности бизнеса - альтернативы нет. 

Самое подробное описание, которое удалось найти:

Uber Technologies подписала два крупных облачных семилетних контракта и планирует полностью закрыть собственный центр обработки данных, который был приобретён у Microsoft в 2015 году вместе со 100 сотрудниками. В настоящее время более 95 процентов ИТ-ресурсов размещены в собственном ЦОД Uber.

Оценка облачных ЦОД заняла у Uber 11 месяцев. Оценивая облачные компании в Uber решили, что наличие нескольких провайдеров снижает риски и позволяет компании воспользоваться преимуществами различных облачных провайдеров.

С Google, помимо основного контракта на облачные вычисления, Uber будет использовать картографический сервис Google для маршрутизации своих транспортных средств и рекламный продукт Google для своего молодого рекламного бизнеса.

Что касается Oracle, Uber будет использовать облачную систему планирования корпоративных ресурсов компании и другие продукты базы данных для своего грузового бизнеса.

В своем первоначальном публичном предложении в 2018 году Uber заявила, что в 2018 году она потратила 221 миллион долларов на аренду офисов и центров обработки данных, а также на облачные вычисления Google Cloud и Amazon Web Services.

Развертывание колокации было подробно описано главой Uber Compute Дином Нельсоном, который на DCD>London 2018 объяснил, что компания планирует арендовать 576-стоечный центр обработки данных мощностью 5 МВт.

«Каждый сервер имеет 25-гигабитную сеть, — сказал Нельсон -16 стоек образуют модуль. Формируем 30 модулей, что составляет 480 шкафов».

К 480 стойкам добавляются 32 стойки для сети и 64 стойки для различных дополнений, «потому что мы никогда не знаем, что произойдет», в результате чего в общей сложности получается 576 стоек. Компания использует четыре типа стоек: вычислительная, база данных, хранилище (многоуровневое хранилище с «теплым» и «холодным») и графические процессоры для машинного обучения.

https://www.datacenterdynamics.com/en/news/uber-picks-oracle-and-google-for-7-year-cloud-contracts-closing-its-own-data-centers/