Configuring Oracle Transparent Data Encryption (TDE)
on Oracle 12c multitenant architecture
To configure TDE on Oracle 12c multitenant architecture we need to execute some steps in order to
be able to create encrypted tablespaces on Oracle, for example.
1) Adjust the sqlnet.ora file to refer your wallet path
[oracle@db1 admin]$ cat sqlnet.ora
ENCRYPTION_WALLET_LOCATION=
(SOURCE=
(METHOD=FILE)
(METHOD_DATA=
(DIRECTORY=/u01/app/oracle/CDB1)
)
)
2) Create the folder on OS side
[oracle@db1 admin]$ mkdir -p /u01/app/oracle/CDB1
3) Log into database using the role syskm, create the keystore and the auto login keystore
[oracle@db1 admin]$ sqlplus / as syskm
SQL*Plus: Release 12.1.0.2.0 Production on Tue Jul 30 13:42:13 2019
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
SQL> administer key management create keystore '/u01/app/oracle/CDB1' identified by oracle;
keystore altered.
SQL>
SQL>
SQL> ADMINISTER KEY MANAGEMENT CREATE AUTO_LOGIN KEYSTORE FROM KEYSTORE '/u01/app/oracle/CDB1' IDENTIFIED BY oracle;
keystore altered.
SQL> select * from v$encryption_wallet;
WRL_TYPE WRL_PARAMETER STATUS WALLET_TYPE WALLET_OR FULLY_BAC CON_ID
------------ --------------------------------- ---------------------- -------------- ----------------------- -----------
FILE /u01/app/oracle/CDB1/ OPEN_NO_MASTER_KEY AUTOLOGIN SINGLE UNDEFINED 0
4) Bounce the database to see if AUTO_LOGIN keystore is working correctly
SQL> connect / as sysdba
Connected.
SQL> shut immediate;
Database closed.
Database dismounted.
startup
ORACLE instance shut down.
SQL> ORACLE instance started.
Total System Global Area 1660944384 bytes
Fixed Size 2925072 bytes
Variable Size 1040190960 bytes
Database Buffers 603979776 bytes
Redo Buffers 13848576 bytes
Database mounted.
Database opened.
SQL> select * from v$encryption_wallet;
WRL_TYPE WRL_PARAMETER STATUS WALLET_TYPE WALLET_OR FULLY_BAC CON_ID
------------ --------------------------------- ---------------------- -------------- ----------------------- -----------
FILE /u01/app/oracle/CDB1/ OPEN_NO_MASTER_KEY AUTOLOGIN SINGLE UNDEFINED 0
Obs.: The first time, you need to set the masterkey to avoid this status and errors with the key not open properly.
SQL> alter pluggable database all open;
Pluggable database altered.
SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 PDB2 READ WRITE NO
4 PD3 READ WRITE NO
5) Let’s setup the masterkey on the keystore
SQL> conn / as sysdba
Connected.
SQL> grant dba,syskm to C##dba identified by oracle container=ALL;
Grant succeeded.
SQL> administer key management set keystore close container=all;
keystore altered.
SQL> administer key management set keystore open identified by oracle container=all;
keystore altered.
6) On pluggable database, connected with common user C##DBA, let’s set the key for all pdbs.
SQL> conn c##dba/oracle@db1:1521/PD3 as syskm;
Connected.
SQL>
SQL> conn / as sysdba
Connected.
SQL> administer key management set key identified by oracle with backup using 'OMK' container=ALL;
keystore altered.
SQL> select * from v$encryption_wallet;
WRL_TYPE WRL_PARAMETER STATUS WALLET_TYPE WALLET_OR FULLY_BAC CON_ID
------------ --------------------------------- ---------------------- -------------- ----------------------- -----------
FILE /u01/app/oracle/CDB1/ OPEN PASSWORD SINGLE NO 0
As you can see above, now the status is OPEN (by password)
7) Connected on PD3 as common user C##DBA and now we can create our tablespace
SQL> conn c##dba/oracle@db1:1521/PD3;
Connected.
SQL> create tablespace OCM_CRYP
2 datafile '/u01/app/oracle/oradata/CDB1/8DE664DD797442E9E055000000000001/datafile/OCM.DBF'
3 size 10m
4 encryption using 'AES256'
5 default storage(ENCRYPT);
Tablespace created.
So, this is the new way for 12c to management keys for encryption on database.
I hope this helps you!!!
This article was written by Andre Ontalba e Rodrigo Mufalani
Disclaimer: “The postings on this site are my own and don’t necessarily represent may actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications was removed to allow reach generic audience and to be useful.”
Exadata – Advanced Intrusion Detection Environment (AIDE)
Few days ago, we have upgrade one Exadata environment to 19c version. One of new features named AIDE – Advanced Intrusion Detection Environment was introduced on this release of Exadata software, it came pre-configured. This feature is really awesome, and call my attention, because it tracks if the system files were modified and then helps the DMA (Database Machine Administrator) to find issues and security breaches on their environments. AIDE run crontab jobs daily and monitor if files changes in specific directories. If some files changes, it raise an alert and generate logs on /var/log/aide/aide.log.
On Oracle documentation:
exadataAIDE Syntax
The utility is located at /opt/oracle.SupportTools/exadataAIDE.
1 | |
Description of syntax options:
-s[tatus]: Print the current status of the AIDE daily cron job-e[nable]: Enable the AIDE daily cron job-d[isable]: Disable the AIDE daily cron job-u[pdate]: Update the AIDE database metadata and run the daily scan-h[elp]: Print the command syntax and help information
Get the current status of the
aidecron job.exadataAIDE –statusDisable the daily AIDE scan.
exadataAIDE –disableEnable the daily AIDE scan.
exadataAIDE –enableUpdate the AIDE database after making changes to the system.
exadataAIDE –update
Let’s take a look on the “warning file” generated by AIDE.
[root@myexa01 ~]# cat /var/log/aide/aide.log AIDE 0.15.1 found differences between database and filesystem!! Start timestamp: 2019-07-21 19:57:20 Summary: Total number of files: 55788 Added files: 0 Removed files: 0 Changed files: 2 Changed files: changed: /home/oracle/monitor/check_compliance changed: /home/oracle/monitor/check_oracle Detailed information about changes: File: /home/oracle/monitor/check_listener SHA256 : AnuG4ldrP0yB15r/3nOGN+nrnW18391+ , wqPj5hnOIl1HGK7f85/E+0tWGYvEKbH5 File: /home/oracle/monitor/check_oracle SHA256 : t6zk+jufuBfia/YCNHS1WqJt/GPW8xSX , Xk3ZkzP/YpT2jbKvjORsRF3TfoCteY8Z
After that alert, for our environment we need to add an exception for a specific directory by the end of the file:
[root@myexa01 ~]# vi /etc/aide.conf #Add one Exception for monitoring directory !/home/oracle/monitor/
Then we need to rebuild the database for AIDE, the DB is located at /var/lib/aide/aide.db.gz
[root@myexa01 ~]# /opt/oracle.SupportTools/exadataAIDE -u AIDE: database update request accepted.
Then, we need to clen the alerts raised. To cleanup we use dbmcli utility
[root@myexa01 ~]# dbmcli
DBMCLI> LIST ALERTHISTORY
1_1 2019-07-20T18:24:29+02:00 warning "Advanced Intrusion Detection Environment (AIDE) detected potential changes to software on this system. The changes are in /var/log/aide/aide.log "
1_2 2019-07-22T09:14:55+02:00 clear "Advanced Intrusion Detection Environment (AIDE) violation has been cleared."
DBMCLI> drop alerthistory 1_1, 1_2
Alert 1_1 successfully dropped
Alert 1_2 successfully droppedTo check more this secutiry framework on Exadata, please refer:
https://docs.oracle.com/en/engineered-systems/exadata-database-machine/dbmsq/exadata-security-practices.html#GUID-74FF1D0C-59F2-4F02-B648-34048AC02C31
All the Best,
Rodrigo Mufalani
Disclaimer: “The postings on this site are my own and don’t necessarily represent may actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications was removed to allow reach generic audience and to be useful.”
ODA REIMAGE
The idea to reimage ODA is to refresh the environment without the need to jump from one by one to reach the last available version, or even rescue the system from S.O. failure/crash. The process to do a reimage can be check in the official documentation but unfortunately can be very tricky because the information (the order and steps) are not 100% clear. The idea is to show you how to reimage using version 18 (18.3 in this example), that represents the last available.
In resume the process is executed in the order:
ILOM: Boot the ISO
Prepare to create the appliance
Upload GI and DB base version to the repository
Linux: Create the appliance
Firmware and patch
Create Oracle Homes and the databases
Finish and clean the install
The environment in this scenario is:
ODA HA: Oda HA version with two nodes and with SSD disk available.
Reimage with 18 version: 18.3 version at the moment that reimages was done, but can use any 18.x version
Databases: 11, 12, 18 for Oracle Home and available versions
Before even start the reimage you have two steps. The first is to verify the hardware against failures. Disks, memory, CPU, MB, whatever error that exists need to be clean before the start. If no, you can hit error during the process. To check about hw error, you can log in the ILOM in the browser and verify for some incidents/errors (this vary from version to version of ODA, but usually the errors are visible in the first page of ILOM). Or you can log in in the console for ILOM and execute the command “show faulty”. Whatever the mode, check in both nodes. If you have a connection with the system execute smartctl –a /dev/DISK and query about errors and look in the column “Total uncorrected errors”. Another option is check if the disk is in warning state using the command smartctl -q errorsonly -H -l xerror /dev/DISK; if you have the report will be similar to “SMART Health Status: WARNING: ascq=0x97 [asc=b, ascq=97]”. For the errors, open SR and wait for the maintenance from Oracle HW team.
The second is save/backup every config that you need. If you did not lose the system backup from /etc folder it is a good start. Another hint is execute “ifconfig” and “route” to have one output from IP’s that you have and the routing table (vlan list it is important too if used). For Oracle files, backup tnsnames.ora, listener.ora, and sqlnet.ora for every OH or GI that you have. These files are example, depend on every system and needs. Adjust if you need something additional.
ILOM
The process to reimage start putting the ISO in the ilom and booting both nodes with that. After the boot, the install runs automatically, and you have a Linux to create the appliance.
Before starting the reimage one explanation since the machine was already in use it needs to cleanup the disks. The first option is cleanup before the reimage, using the software from the currently running linux/oda/oak. Or running the cleanup after the reimage finish and before creating the appliance (the option that I will use here).
The steps to reimage from ILOM are done in this order:
Login ILOM both nodes using the browser
Navigate to “Host Management” menu and choose “Host Control” submenu.
Select the option “CDROM” in the list and after click in SAVE.
Navigate to “Remote Control” menu and choose “Redirection” submenu.
Click on the option “Launch Remote Control Console”
You can receive some confirmation to execute some java (depend on how your machine has java configured) or ask to agree about the certificate. Always click yes and agree or continue.
In the newly opened window select the menu “KVMS” and after the submenu “Storage”.
In the “Storage Devices” do:
Click on the “Add”
Select the ISO file and after click in the option “Select”
After, verify if the ISO appeared in the “Path” and the “Device Type” is “ISO Image” in the Storage Devices menu. Select the line with the ISO and after click in the option “Connect”.
After you click in the “Connect” the information marked in red will appear in the screen and the menu now will appear as “Disconnect”. You just need to click in the option “OK”.
DO NOT CLONE the Console.
Go back to the Browser and click in the menu “Host Management” and after in the submenu “Power Control”
Select the option “Power Cycle” and after click in the “Save” option.
Maybe you will receive on a question if you are sure about the reboot. Just click Ok.
You will see in the console:
Reboot
The process will boot the ISO
Install the new ODA Image
In the end, you have the Linux console
All the steps above you can see in the Image gallery below. All the images are linked with the step mentioned.
Remember to do this in both nodes. You can do this both at the same time. The installation time depends more from the network because involve uploading the ISO from machine to ODA. The postscript part during the installation tends to take a lot of time.
Please does not close the Remote Console because this kills the ISO mounting device and the installation will crash. There is no problem if the browser login receives timeout, the installation will continue.
One important thing from this part is that does not update the ILOM SP or even other firmware for HW. This will be executed in the next steps.
Recently Oracle changed the ODA base page and removed the links to ISO files. But the patch 27604623 contains the Oracle Database Appliance Bare-metal ISO Image to download. Just to be clear, I am reimaging for baremetal (not OVM).
Prepare to create the appliance
After you “reimage” you need to configure and create the appliance. But before we need to execute some steps. Now, to connect in the Linux console we have two options: same java console opened to do the reimage, or, ssh to ilom ip and start the console (start /SP/console). I will use the last option because allow to work (copy and paste commands) more smoothly. Example (root/welcome1 are user/password that is the default for reimaged ODA):
Oracle(R) Integrated Lights Out Manager
Version 3.2.9.23 r116695
Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
Warning: HTTPS certificate is set to factory default.
Hostname: odak1-ilom
-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y
Serial console started. To stop, type ESC (
Oracle Linux Server release 6.10
Kernel 4.1.12-124.18.6.el6uek.x86_64 on an x86_64
test0 login: root
[root@test0 ~]#
Cleanup
As told before, we need to clean the headers for ODA. I start checking the headers with command “cleanup.pl -checkHeader”:
[root@test0 ~]# /opt/oracle/oak/onecmd/cleanup.pl -checkHeader
Command may take few minutes...
OS Disk Disk Type OAK Header ASM Header(p1) ASM Header(p2)
/dev/sda HDD Found Found Found
/dev/sdb HDD Found Found Found
/dev/sdaa HDD Found Found Found
/dev/sdab HDD Found Found Found
/dev/sdac HDD Found Found Found
/dev/sdad HDD Found Found Found
/dev/sdae HDD Found Found Found
/dev/sdaf HDD Found Found Found
/dev/sdag HDD Found Found Found
/dev/sdah HDD Found Erased Erased
/dev/sdai HDD Found Found Found
/dev/sdaj HDD Found Found Found
/dev/sdak HDD Found Found Found
/dev/sdal HDD Found Found Found
/dev/sdam HDD Found Found Found
/dev/sdan HDD Found Found Found
/dev/sdao SSD Found Found p2 not created
/dev/sdap SSD Found Found p2 not created
/dev/sdaq SSD Found Found p2 not created
/dev/sdar SSD Found Found p2 not created
/dev/sdas SSD Found Found p2 not created
/dev/sdat SSD Found Found p2 not created
/dev/sdau SSD Found Found p2 not created
/dev/sdav SSD Found Found p2 not created
[root@test0 ~]#
Look above that some ASM Header was found. To clean we execute “/opt/oracle/oak/onecmd/cleanup.pl” with option “-erasedata”. The cleanup script executes more than just erase disk headers, it cleans all the configurations (hostname, ips, users) and because of this needs to be executed in both nodes from HA env. This equalizes all nodes with the same config and avoid an error when you create the appliance. Example:
[root@test0 ~]# /opt/oracle/oak/onecmd/cleanup.pl -erasedata
INFO: *******************************************************************
INFO: ** Starting process to cleanup provisioned host oak1 **
INFO: *******************************************************************
WARNING: Secure Erase is an irrecoverable process. All data on the disk
WARNING: will be erased, and cannot be recovered by any means. On X3-2,
WARNING: X4-2, and X5-2 HA, the secure erase process can take more than
WARNING: 10 hours. If you need this data, then take a complete backup
WARNING: before proceeding.
Do you want to continue (yes/no) : yes
INFO:
....
....
INFO: Executing </opt/oracle/oak/bin/odaeraser.py -a -f -v >
Start erasing disks on the system
On some platforms, this will take several hours to finish, please wait
Do you want to continue (yes|no) [yes] ? yes
Number of disks are processing: 0
Disk Vendor Model Erase method Status Time(seconds)
e0_pd_00 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_01 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_02 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_03 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_04 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_05 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_06 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_07 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_08 HGST H7280A520SUN8.0T SCSI crypto erase success 0
e0_pd_09 HGST H7280A520SUN8.0T SCSI cryp[ 777.976350] reboot: Restarting system
...
...
After node reboot you can check the disk readers (look that the cleanup will change the hostname to “oak”):
[root@oak2 ~]# /opt/oracle/oak/onecmd/cleanup.pl -checkHeader
Command may take few minutes...
OS Disk Disk Type OAK Header ASM Header(p1) ASM Header(p2)
/dev/sda HDD Erased Erased UnKnown
/dev/sdb HDD Erased Erased UnKnown
/dev/sdaa HDD Erased Erased UnKnown
/dev/sdab HDD Erased Erased UnKnown
/dev/sdac HDD Erased Erased UnKnown
/dev/sdad HDD Erased Erased UnKnown
/dev/sdae HDD Erased Erased UnKnown
/dev/sdaf HDD Erased Erased UnKnown
/dev/sdag HDD Erased Erased UnKnown
/dev/sdah HDD Erased Erased UnKnown
/dev/sdai HDD Erased Erased UnKnown
/dev/sdaj HDD Erased Erased UnKnown
/dev/sdak HDD Erased Erased UnKnown
/dev/sdal HDD Erased Erased UnKnown
/dev/sdam HDD Erased Erased UnKnown
/dev/sdan HDD Erased Erased UnKnown
/dev/sdao SSD Erased Erased UnKnown
/dev/sdap SSD Erased Erased UnKnown
/dev/sdaq SSD Erased Erased UnKnown
/dev/sdar SSD Erased Erased UnKnown
/dev/sdas SSD Erased Erased UnKnown
/dev/sdat SSD Erased Erased UnKnown
/dev/sdau SSD Erased Erased UnKnown
/dev/sdav SSD Erased Erased UnKnown
First network
After the cleanup we can create the basic network. To do that (need to do in both nodes), just execute the script “/opt/oracle/dcs/bin/odacli” with parameter “configure-firstnet”. Look:
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli configure-firstnet
Select the Interface to configure the network on (bond0 bond1) [bond0]:
Configure DHCP on bond0 (yes/no) [no]:
INFO: You have chosen Static configuration
Use VLAN on bond0 (yes/no) [no]:yes
Configure VLAN on bond0, input VLAN ID [2 - 4094] 2999
INFO: using network interface bond0.2999
Enter the IP address to configure : 200.200.67.100
Enter the Netmask address to configure : 255.255.255.0
Enter the Gateway address to configure[200.200.67.1] :
INFO: Restarting the network
Shutting down interface bond0: [ OK ]
Shutting down interface bond1: [ OK ]
Shutting down interface ib0: [ OK ]
Shutting down interface ib1: [ OK ]
Shutting down interface ibbond0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface bond0: [ OK ]
Bringing up interface bond1: [ OK ]
Bringing up interface ibbond0: Determining if ip address 192.168.16.24 is already in use for device ibbond0...
[ OK ]
Bringing up interface bond0.2999: Determining if ip address 200.200.67.100 is already in use for device bond0.2999...
[ OK ]
INFO: Restarting the DCS agent
initdcsagent stop/waiting
initdcsagent start/running, process 23750
[root@oak1 ~]#
And in the second node:
[root@oak2 ~]# /opt/oracle/dcs/bin/odacli configure-firstnet
Select the Interface to configure the network on (bond0 bond1) [bond0]:
Configure DHCP on bond0 (yes/no) [no]:
INFO: You have chosen Static configuration
Use VLAN on bond0 (yes/no) [no]:yes
Configure VLAN on bond0, input VLAN ID [2 - 4094] 2999
INFO: using network interface bond0.2999
Enter the IP address to configure : 200.200.67.101
Enter the Netmask address to configure : 255.255.255.0
Enter the Gateway address to configure[200.200.67.1] :
INFO: Restarting the network
Shutting down interface bond0: [ OK ]
Shutting down interface bond1: [ OK ]
Shutting down interface ib0: [ OK ]
Shutting down interface ib1: [ OK ]
Shutting down interface ibbond0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface bond0: [ OK ]
Bringing up interface bond1: [ OK ]
Bringing up interface ibbond0: Determining if ip address 192.168.16.25 is already in use for device ibbond0...
[ OK ]
Bringing up interface bond0.2999: Determining if ip address 200.200.67.101 is already in use for device bond0.2999...
[ OK ]
INFO: Restarting the DCS agent
initdcsagent stop/waiting
initdcsagent start/running, process 10775
[root@oak2 ~]#
The most important is to define correctly the vlan (if used, as in this case). You can rerun the command if needed. After that, you can exit the ILOM console and do ssh to the ODA node.
Upload GI and DB base version to the repository
After you configure the first network you need to upload the files that will fill the internal ODA repository and allow the “create appliance script” do initiate the ASM and base Oracle Home. There are some several steps, but you will execute in just one node.
Again, Oracle recently changed the ODA base page (https://support.oracle.com/epmos/faces/DocContentDisplay?id=888888.1) and removed the links to files with clone version for OH and GI. But the patch numbers are patch 27604593 for GI and patch 27604558 for RDBMS. Bot are for baremetal image, if you are using VM installation, they are others (but you can find then patch search inside Oracle MOS.
The first step is to upload the files “p27604558_183000_Linux-x86-64.zip”and “p27604593_183000_Linux-x86-64.zip” to the first node and unzip. Remember that the file name will change if you are reimage using another version. Here I used in “/tmp”
[root@oak1 ~]# mkdir /tmp/deploy
[root@oak1 ~]# cd /tmp/deploy
[root@oak1 deploy]# ###### EXECUTE THE SCP OR COPY FILES TO THE FOLDER########
[root@oak1 deploy]# unzip -qa p27604558_183000_Linux-x86-64.zip
[root@oak1 deploy]# unzip -qa p27604593_183000_Linux-x86-64.zip
replace README.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
[root@oak1 deploy]#
[root@oak1 deploy]#
[root@oak1 deploy]# ls -l
total 20559868
-rw-r--r-- 1 root root 4322872013 Dec 18 08:24 odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip
-rwxr-xr-x 1 root root 6193475789 Dec 18 08:03 odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip
-rw-r--r-- 1 root root 4322873299 May 22 08:37 p27604558_183000_Linux-x86-64.zip
-rw-r--r-- 1 root root 6193477129 May 22 08:35 p27604593_183000_Linux-x86-64.zip
-rw-r--r-- 1 root root 2044 Dec 18 08:05 README.txt
[root@oak1 deploy]#
After that we can upload these file to the internal ODA repository:
[root@oak1 deploy]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/deploy/odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip,/tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip
{
"jobId" : "b66269aa-e007-4304-8074-7ba7e9c8c5c2",
"status" : "Created",
"message" : "/tmp/deploy/odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip,/tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip",
"reports" : [ ],
"createTimestamp" : "May 22, 2019 08:51:14 AM UTC",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "May 22, 2019 08:51:14 AM UTC"
}
[root@oak1 deploy]#
Look that one job will be created, and you can check the progress for the job with the “/opt/oracle/dcs/bin/odacli describe-job -i” passing as parameter the jobId:
[root@oak1 deploy]# /opt/oracle/dcs/bin/odacli describe-job -i b66269aa-e007-4304-8074-7ba7e9c8c5c2
/opt/oracle/dcs/bin/odacli describe-job -i b66269aa-e007-430
Job details
----------------------------------------------------------------
ID: b66269aa-e007-4304-8074-7ba7e9c8c5c2
Description: Repository Update
Status: Running
Created: May 22, 2019 8:51:14 AM UTC
Message: /tmp/deploy/odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip,/tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace May 22, 2019 8:51:14 AM UTC May 22, 2019 8:51:14 AM UTC Success
Setting up ssh equivalance May 22, 2019 8:51:15 AM UTC May 22, 2019 8:51:15 AM UTC Success
Copy BundleFile May 22, 2019 8:51:15 AM UTC May 22, 2019 8:51:15 AM UTC Running
[root@oak1 deploy]#
[root@oak1 deploy]# /opt/oracle/dcs/bin/odacli describe-job -i b66269aa-e007-4304-8074-7ba7e9c8c5c2
Job details
----------------------------------------------------------------
ID: b66269aa-e007-4304-8074-7ba7e9c8c5c2
Description: Repository Update
Status: Success
Created: May 22, 2019 8:51:14 AM UTC
Message: /tmp/deploy/odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip,/tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace May 22, 2019 8:51:14 AM UTC May 22, 2019 8:51:14 AM UTC Success
Setting up ssh equivalance May 22, 2019 8:51:15 AM UTC May 22, 2019 8:51:15 AM UTC Success
Copy BundleFile May 22, 2019 8:51:15 AM UTC May 22, 2019 8:52:21 AM UTC Success
Validating CopiedFile May 22, 2019 8:52:24 AM UTC May 22, 2019 8:52:48 AM UTC Success
Unzip bundle May 22, 2019 8:52:48 AM UTC May 22, 2019 8:54:17 AM UTC Success
Unzip bundle May 22, 2019 8:54:17 AM UTC May 22, 2019 8:55:47 AM UTC Success
Delete PatchBundles May 22, 2019 8:55:47 AM UTC May 22, 2019 8:55:48 AM UTC Success
Removing ssh keys May 22, 2019 8:55:48 AM UTC May 22, 2019 8:55:49 AM UTC Success
[root@oak1 deploy]#
Create appliance
After you clean the disk headers, prepare the first network and upload the files in the repository you can create the appliance. This means that you will configure the operational system, groups, users, folders, GI, ASM, and Oracle database. There are two ways to create, one is using the web interface (this replace the old java app that creates the XML file), and the other is from CLI. Unfortunately, until I wrote this, if you use VLAN you can’t create the appliance with the web interface, just with CLI.
The CLI uses the script “/opt/oracle/dcs/bin/odacli create-appliance” that receive a JSON file as a parameter. There is not some much information the parameter for this JSON file, just the readme with some examples and one web page with other examples. But in both, no real good explanation to each option, look:
https://docs.oracle.com/en/engineered-systems/oracle-database-appliance/18.3/cmtxd/create-appliance-using-json-file.html#GUID-42250FD2-EA91-4457-9ED7-CA3A2A863B40
Fortunately, most of the options are self-explained, other parts I will explain below. First, the full JSON file that I used:
[root@oak1 ~]# cat odas.json
{
"instance" : {
"name" : "odas",
"instanceBaseName" : "odas",
"systemPassword" : null,
"dbEdition" : "EE",
"timeZone" : "Europe/Luxembourg",
"ntpServers" : [ "200.200.13.125" ],
"dnsServers" : [ "200.200.1.32", "200.200.1.33" ],
"domainName" : "xxxx.xxxx.xxx",
"isRoleSeparated" : true,
"osUserGroup" : {
"groups" : [ {
"groupId" : 1001,
"groupName" : "oinstall",
"groupRole" : "oinstall"
}, {
"groupId" : 1003,
"groupName" : "dbaoper",
"groupRole" : "dbaoper"
}, {
"groupId" : 1002,
"groupName" : "dba",
"groupRole" : "dba"
}, {
"groupId" : 1006,
"groupName" : "asmadmin",
"groupRole" : "asmadmin"
}, {
"groupId" : 1005,
"groupName" : "asmoper",
"groupRole" : "asmoper"
}, {
"groupId" : 1004,
"groupName" : "asmdba",
"groupRole" : "asmdba"
} ],
"users" : [ {
"userId" : 1001,
"userName" : "oracle",
"userRole" : "oracleUser"
}, {
"userId" : 1000,
"userName" : "grid",
"userRole" : "gridUser"
} ]
},
"objectStoreCredentials" : null
},
"nodes" : [ {
"nodeNumber" : "0",
"nodeName" : "odas1",
"localDisk" : null,
"network" : [ {
"name" : null,
"nicName" : "bond0.2999",
"vlanId" : null,
"ipAddress" : "200.200.67.100",
"subNetMask" : "255.255.255.0",
"gateway" : "200.200.67.1",
"interfaceType" : null,
"networkType" : [ "Public" ],
"isDefaultNetwork" : true
} ],
"ilom" : null
}, {
"nodeNumber" : "1",
"nodeName" : "odas2",
"localDisk" : null,
"network" : [ {
"name" : null,
"nicName" : "bond0.2999",
"vlanId" : null,
"ipAddress" : "200.200.67.101",
"subNetMask" : "255.255.255.0",
"gateway" : "200.200.67.1",
"interfaceType" : null,
"networkType" : [ "Public" ],
"isDefaultNetwork" : true
} ],
"ilom" : null
} ],
"grid" : {
"diskGroup" : [ {
"diskGroupName" : "DATA",
"redundancy" : "NORMAL",
"disks" : null,
"diskPercentage" : 90
}, {
"diskGroupName" : "RECO",
"redundancy" : "NORMAL",
"disks" : null,
"diskPercentage" : 10
}, {
"diskGroupName" : "REDO",
"redundancy" : "HIGH",
"disks" : null,
"diskPercentage" : null
}, {
"diskGroupName" : "FLASH",
"redundancy" : "NORMAL",
"disks" : null,
"diskPercentage" : null
} ],
"scan" : {
"scanName" : "odas-scan",
"ipAddresses" : [ "200.200.67.104", "200.200.67.105" ]
},
"vip" : [ {
"nodeNumber" : "0",
"vipName" : "odas1-vip",
"ipAddress" : "200.200.67.102"
}, {
"nodeNumber" : "1",
"vipName" : "odas2-vip",
"ipAddress" : "200.200.67.103"
} ],
"language" : "en",
"enableAFD" : "TRUE",
"gridVersion" : null
},
"database" : {
"dbName" : "mfod",
"databaseUniqueName" : "mfod_ora",
"dbVersion" : "18.3.0.0.180717",
"dbHomeId" : null,
"instanceOnly" : false,
"registerOnly" : null,
"isCdb" : true,
"pdBName" : "pdb1",
"pdbAdminuserName" : "pdbuser",
"enableTDE" : false,
"adminPassword" : "WWeeeeee99##",
"dbType" : "RAC",
"dbTargetNodeNumber" : null,
"dbClass" : "OLTP",
"dbShape" : "odb1",
"dbStorage" : "ASM",
"dbRedundancy" : null,
"dbCharacterSet" : {
"characterSet" : "AL32UTF8",
"nlsCharacterset" : "AL16UTF16",
"dbTerritory" : "AMERICA",
"dbLanguage" : "AMERICAN"
},
"dbConsoleEnable" : false,
"backupConfigId" : null,
"backupConfigName" : null,
"dbOnFlashStorage" : true,
"dbEdition" : "EE",
"rmanBkupPassword" : null,
"dbDomainName" : null,
"enableFlashCache" : null,
"level0BackupDay" : null
},
"asr" : null
}
[root@oak1 ~]#
The file starts with a basic config where you define the name parameter and the instanceBaseName that will define some basic internal identification parameters. One example is the time zone.
The first that needs attention is isRoleSeparated and osUserGroup. If you choose the first as false, you don’t need the specify grid users. Otherwise, is needed to specify it.
For nodes option, you specify the hostname, ips, and gateway. Be careful with the nickname that is the same name that you configured the first network. Some steps above. The ilom parameter, since I am not changing now (and I think that in reimage scenario you do not change at this moment), remains null.
The grid parameter defines the disk division. You can choose arbitrary values for diskPercentage and disks that you will select for each one if needed. For disk, left undefined that ODA will divide automatically and accordingly. The key point here is that even, by default, FLASH and REDO diskgroups are created with default values (check that I even defined disk percentage for each one), if you forgot to specify, they are not created and the disk will not be used. I already wrote about one problem in this particular part and you can read here: http://www.fernandosimon.com/blog/oda-json-and-flash/
The database parameter defines the information for database creation. You have the basic like database name, unique name, cdb (or no), pdb name. The most tricky here are dbVersion (that need to be the same from that you uploaded in the repository), adminPassword (that will be used for root password after create the appliance), dbShape (that is the same – you choose one – for ODA available shapes), and dbStorage (that you choose between ASM and ACFS, 11 just support ACFS). Again, for ASM and ACFS I already wrote about that here: http://www.fernandosimon.com/blog/oda-acfs-and-asm-dilemma/
Important hint, if you want to create a database in another version that 18c you need to upload the files/clone images in the repository before.
To create just execute:
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli create-appliance -r odas.json
{
"instance" : {
"name" : "odas",
"instanceBaseName" : "odas",
"systemPassword" : null,
"dbEdition" : "EE",
"timeZone" : "Europe/Luxembourg",
"ntpServers" : [ "200.200.13.125" ],
"dnsServers" : [ "200.200.1.32", "200.200.1.33" ],
"domainName" : "xxxx.xxxx.xxx",
"isRoleSeparated" : true,
"osUserGroup" : {
"groups" : [ {
"groupId" : 1001,
"groupName" : "oinstall",
"groupRole" : "oinstall"
}, {
"groupId" : 1003,
"groupName" : "dbaoper",
"groupRole" : "dbaoper"
}, {
"groupId" : 1002,
"groupName" : "dba",
"groupRole" : "dba"
}, {
"groupId" : 1006,
"groupName" : "asmadmin",
"groupRole" : "asmadmin"
}, {
"groupId" : 1005,
"groupName" : "asmoper",
"groupRole" : "asmoper"
}, {
"groupId" : 1004,
"groupName" : "asmdba",
"groupRole" : "asmdba"
} ],
"users" : [ {
"userId" : 1001,
"userName" : "oracle",
"userRole" : "oracleUser"
}, {
"userId" : 1000,
"userName" : "grid",
"userRole" : "gridUser"
} ]
},
"objectStoreCredentials" : null
},
"nodes" : [ {
"nodeNumber" : "0",
"nodeName" : "odas1",
"localDisk" : null,
"network" : [ {
"name" : null,
"nicName" : "bond0.2999",
"vlanId" : null,
"ipAddress" : "200.200.67.100",
"subNetMask" : "255.255.255.0",
"gateway" : "200.200.67.1",
"interfaceType" : null,
"networkType" : [ "Public" ],
"isDefaultNetwork" : true
} ],
"ilom" : null
}, {
"nodeNumber" : "1",
"nodeName" : "odas2",
"localDisk" : null,
"network" : [ {
"name" : null,
"nicName" : "bond0.2999",
"vlanId" : null,
"ipAddress" : "200.200.67.101",
"subNetMask" : "255.255.255.0",
"gateway" : "200.200.67.1",
"interfaceType" : null,
"networkType" : [ "Public" ],
"isDefaultNetwork" : true
} ],
"ilom" : null
} ],
"grid" : {
"diskGroup" : [ {
"diskGroupName" : "DATA",
"redundancy" : "NORMAL",
"disks" : null,
"diskPercentage" : 90
}, {
"diskGroupName" : "RECO",
"redundancy" : "NORMAL",
"disks" : null,
"diskPercentage" : 10
}, {
"diskGroupName" : "REDO",
"redundancy" : "HIGH",
"disks" : null,
"diskPercentage" : null
}, {
"diskGroupName" : "FLASH",
"redundancy" : "NORMAL",
"disks" : null,
"diskPercentage" : null
} ],
"scan" : {
"scanName" : "odas-scan",
"ipAddresses" : [ "200.200.67.104", "200.200.67.105" ]
},
"vip" : [ {
"nodeNumber" : "0",
"vipName" : "odas1-vip",
"ipAddress" : "200.200.67.102"
}, {
"nodeNumber" : "1",
"vipName" : "odas2-vip",
"ipAddress" : "200.200.67.103"
} ],
"language" : "en",
"enableAFD" : "TRUE",
"gridVersion" : null
},
"database" : {
"dbName" : "mfod",
"databaseUniqueName" : "mfod_ora",
"dbVersion" : "18.3.0.0.180717",
"dbHomeId" : null,
"instanceOnly" : false,
"registerOnly" : null,
"isCdb" : true,
"pdBName" : "pdb1",
"pdbAdminuserName" : "pdbuser",
"enableTDE" : false,
"adminPassword" : "WWeeeeee99##",
"dbType" : "RAC",
"dbTargetNodeNumber" : null,
"dbClass" : "OLTP",
"dbShape" : "odb1",
"dbStorage" : "ASM",
"dbRedundancy" : null,
"dbCharacterSet" : {
"characterSet" : "AL32UTF8",
"nlsCharacterset" : "AL16UTF16",
"dbTerritory" : "AMERICA",
"dbLanguage" : "AMERICAN"
},
"dbConsoleEnable" : false,
"backupConfigId" : null,
"backupConfigName" : null,
"dbOnFlashStorage" : true,
"dbEdition" : "EE",
"rmanBkupPassword" : null,
"dbDomainName" : null,
"enableFlashCache" : null,
"level0BackupDay" : null
},
"asr" : null
}
{
"jobId" : "f07961fa-4192-489b-9a55-53165a012376",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "May 22, 2019 09:03:36 AM UTC",
"resourceList" : [ ],
"description" : "Provisioning service creation",
"updatedTime" : "May 22, 2019 09:03:36 AM UTC"
}
[root@oak1 ~]#
Look that you will receive one jobId if everything is fine. To follow just describe the job:
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli describe-job -i f07961fa-4192-489b-9a55-53165a012376
Job details
----------------------------------------------------------------
ID: f07961fa-4192-489b-9a55-53165a012376
Description: Provisioning service creation
Status: Running
Created: May 22, 2019 11:03:36 AM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
networks updation May 22, 2019 11:03:37 AM CEST May 22, 2019 11:03:42 AM CEST Success
updating network May 22, 2019 11:03:37 AM CEST May 22, 2019 11:03:42 AM CEST Success
Setting up Vlan May 22, 2019 11:03:38 AM CEST May 22, 2019 11:03:42 AM CEST Success
networks updation May 22, 2019 11:03:42 AM CEST May 22, 2019 11:03:47 AM CEST Success
updating network May 22, 2019 11:03:42 AM CEST May 22, 2019 11:03:47 AM CEST Success
Setting up Vlan May 22, 2019 11:03:43 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'asmdba'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'asmoper'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'asmadmin'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'dba'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'dbaoper'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'oinstall'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS user 'grid'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:48 AM CEST Success
OS user 'oracle'creation May 22, 2019 11:03:48 AM CEST May 22, 2019 11:03:48 AM CEST Success
SSH equivalance setup May 22, 2019 11:03:48 AM CEST May 22, 2019 11:03:48 AM CEST Success
Grid home creation May 22, 2019 11:03:49 AM CEST May 22, 2019 11:03:49 AM CEST Running
Creating GI home directories May 22, 2019 11:03:49 AM CEST May 22, 2019 11:03:49 AM CEST Success
Cloning Gi home May 22, 2019 11:03:49 AM CEST May 22, 2019 11:03:49 AM CEST Running
[root@oak1 ~]#
And after some time:
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli describe-job -i f07961fa-4192-489b-9a55-53165a012376
Job details
----------------------------------------------------------------
ID: f07961fa-4192-489b-9a55-53165a012376
Description: Provisioning service creation
Status: Success
Created: May 22, 2019 11:03:36 AM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
networks updation May 22, 2019 11:03:37 AM CEST May 22, 2019 11:03:42 AM CEST Success
updating network May 22, 2019 11:03:37 AM CEST May 22, 2019 11:03:42 AM CEST Success
Setting up Vlan May 22, 2019 11:03:38 AM CEST May 22, 2019 11:03:42 AM CEST Success
networks updation May 22, 2019 11:03:42 AM CEST May 22, 2019 11:03:47 AM CEST Success
updating network May 22, 2019 11:03:42 AM CEST May 22, 2019 11:03:47 AM CEST Success
Setting up Vlan May 22, 2019 11:03:43 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'asmdba'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'asmoper'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'asmadmin'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'dba'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'dbaoper'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS usergroup 'oinstall'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:47 AM CEST Success
OS user 'grid'creation May 22, 2019 11:03:47 AM CEST May 22, 2019 11:03:48 AM CEST Success
OS user 'oracle'creation May 22, 2019 11:03:48 AM CEST May 22, 2019 11:03:48 AM CEST Success
SSH equivalance setup May 22, 2019 11:03:48 AM CEST May 22, 2019 11:03:48 AM CEST Success
Grid home creation May 22, 2019 11:03:49 AM CEST May 22, 2019 11:08:00 AM CEST Success
Creating GI home directories May 22, 2019 11:03:49 AM CEST May 22, 2019 11:03:49 AM CEST Success
Cloning Gi home May 22, 2019 11:03:49 AM CEST May 22, 2019 11:06:36 AM CEST Success
Cloning Gi home May 22, 2019 11:06:36 AM CEST May 22, 2019 11:07:58 AM CEST Success
Updating GiHome version May 22, 2019 11:07:58 AM CEST May 22, 2019 11:08:00 AM CEST Success
Updating GiHome version May 22, 2019 11:07:58 AM CEST May 22, 2019 11:08:00 AM CEST Success
Storage discovery May 22, 2019 11:08:00 AM CEST May 22, 2019 11:19:03 AM CEST Success
Grid stack creation May 22, 2019 11:19:03 AM CEST May 22, 2019 11:53:45 AM CEST Success
Configuring GI May 22, 2019 11:19:03 AM CEST May 22, 2019 11:20:08 AM CEST Success
Running GI root scripts May 22, 2019 11:20:08 AM CEST May 22, 2019 11:32:18 AM CEST Success
Running GI config assistants May 22, 2019 11:37:22 AM CEST May 22, 2019 11:40:04 AM CEST Success
Setting AUDIT SYSLOG LEVEL May 22, 2019 11:50:06 AM CEST May 22, 2019 11:50:06 AM CEST Success
Post cluster OAKD configuration May 22, 2019 11:53:45 AM CEST May 22, 2019 11:59:11 AM CEST Success
Disk group 'RECO'creation May 22, 2019 11:59:20 AM CEST May 22, 2019 11:59:38 AM CEST Success
Disk group 'REDO'creation May 22, 2019 11:59:38 AM CEST May 22, 2019 11:59:47 AM CEST Success
Disk group 'FLASH'creation May 22, 2019 11:59:47 AM CEST May 22, 2019 11:59:58 AM CEST Success
Volume 'commonstore'creation May 22, 2019 11:59:58 AM CEST May 22, 2019 12:00:48 PM CEST Success
ACFS File system 'DATA'creation May 22, 2019 12:00:48 PM CEST May 22, 2019 12:01:13 PM CEST Success
Database home creation May 22, 2019 12:01:13 PM CEST May 22, 2019 12:05:28 PM CEST Success
Validating dbHome available space May 22, 2019 12:01:13 PM CEST May 22, 2019 12:01:13 PM CEST Success
Validating dbHome available space May 22, 2019 12:01:13 PM CEST May 22, 2019 12:01:13 PM CEST Success
Creating DbHome Directory May 22, 2019 12:01:13 PM CEST May 22, 2019 12:01:13 PM CEST Success
Extract DB clones May 22, 2019 12:01:14 PM CEST May 22, 2019 12:04:12 PM CEST Success
Clone Db home May 22, 2019 12:04:12 PM CEST May 22, 2019 12:05:11 PM CEST Success
Enable DB options May 22, 2019 12:05:11 PM CEST May 22, 2019 12:05:27 PM CEST Success
Run Root DB scripts May 22, 2019 12:05:27 PM CEST May 22, 2019 12:05:27 PM CEST Success
Provisioning service creation May 22, 2019 12:05:28 PM CEST May 22, 2019 12:23:12 PM CEST Success
Database Creation May 22, 2019 12:05:28 PM CEST May 22, 2019 12:20:27 PM CEST Success
Change permission for xdb wallet files May 22, 2019 12:20:27 PM CEST May 22, 2019 12:20:27 PM CEST Success
Add Startup Trigger to Open all PDBS May 22, 2019 12:20:27 PM CEST May 22, 2019 12:20:28 PM CEST Success
SqlPatch upgrade May 22, 2019 12:21:53 PM CEST May 22, 2019 12:23:08 PM CEST Success
Running dbms_stats init_package May 22, 2019 12:23:08 PM CEST May 22, 2019 12:23:11 PM CEST Success
updating the Database version May 22, 2019 12:23:11 PM CEST May 22, 2019 12:23:12 PM CEST Success
users tablespace creation May 22, 2019 12:23:12 PM CEST May 22, 2019 12:23:15 PM CEST Success
Install TFA May 22, 2019 12:23:16 PM CEST May 22, 2019 12:28:08 PM CEST Success
[root@oak1 ~]#
You can follow the log too if you want to see more detailed (a lot) info. To do just go to directory /opt/oracle/dcs/log/ and check the file dcs-agent.log. Everything will be there, and it is really importing if some step fails. Here you can see one example of the output.
After the end of the job you can check the details for the appliance:
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli describe-system
Appliance Information
----------------------------------------------------------------
ID: 6ce235cf-effc-4748-9d0f-ac246e9ee819
Platform: X5-2-HA
Data Disk Count: 24
CPU Core Count: 36
Created: May 22, 2019 11:03:36 AM CEST
System Information
----------------------------------------------------------------
Name: odas
Domain Name: xxxx.xxxx.xxx
Time Zone: Europe/Luxembourg
DB Edition: EE
DNS Servers: 200.200.1.32 200.200.1.33
NTP Servers: 200.200.13.125
Disk Group Information
----------------------------------------------------------------
DG Name Redundancy Percentage
------------------------- ------------------------- ------------
Data Normal 90
Reco Normal 10
Redo High 100
Flash Normal 100
[root@oak1 ~]#
[root@oak1 ~]# odaadmcli show diskgroup
DiskGroups
----------
DATA
FLASH
RECO
REDO
[root@oak1 ~]#
[root@oak1 ~]# su - grid
[grid@odas1 ~]$ asmcmd
ASMCMD> lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL N 512 512 4096 4194304 108019712 108007188 6751232 50627978 0 Y DATA/
MOUNTED NORMAL N 512 512 4096 4194304 1525760 1518672 381440 568616 0 N FLASH/
MOUNTED NORMAL N 512 512 4096 4194304 11997184 11995736 749824 5622956 0 N RECO/
MOUNTED HIGH N 512 512 4096 4194304 762880 749908 190720 186396 0 N REDO/
ASMCMD> exit
[grid@odas1 ~]$
[grid@odas1 ~]$ logout
[root@oak1 ~]#
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli list-dbstorages
ID Type DBUnique Name Status
---------------------------------------- ------ -------------------- ----------
d26bf34f-9379-4872-839b-ae0af6926aae Asm mfod_ora Configured
[root@oak1 ~]#
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli describe-dbstorage -i d26bf34f-9379-4872-839b-ae0af6926aae
Database Storage details
----------------------------------------------------------------
ID: d26bf34f-9379-4872-839b-ae0af6926aae
DB Name: mfod
DBUnique Name: mfod_ora
DB Resource ID: 938b616a-99ed-4466-8d76-820d3489517d
Storage Type: ASM
DATA:
Location: +FLASH/mfod_ora
Used Space: 3.21GB
Free Space: 741.53GB
REDO:
Location: +REDO/mfod_ora
Used Space: 4.06GB
Free Space: 244.11GB
RECO:
Location: +RECO/mfod_ora
Used Space: 334.0MB
Free Space: 5.72TB
State: ResourceState(status=Configured)
Created: May 22, 2019 11:03:36 AM CEST
UpdatedTime: May 22, 2019 12:01:13 PM CEST
[root@oak1 ~]#
After that you reboot both nodes to have a fresh start with all the configuration up and running:
[root@oak1 ~]# reboot
Broadcast message from root@odas1
(/dev/ttyS0) at 12:37 ...
The system is going down for reboot NOW!
[root@oak1 ~]#
Oracle Linux Server release 6.10
Kernel 4.1.12-124.18.6.el6uek.x86_64 on an x86_64
odas1 login: root
Password:
Last login: Wed May 22 11:10:13 from odap1.xxxx.xxxx.xxx
[root@odas1 ~]#
As an example, you can add vlan to ODA if needed (after the reboot):
[root@odas1 ~]# odaadmcli show vlan
NAME ID INTERFACE CONFIG_TYPE IP_ADDRESS NETMASK GATEWAY NODENUM
Public-network 2999 bond0 Public 200.200.67.100 255.255.255.0 200.200.67.1 0
Public-network 2999 bond0 Public 200.200.67.101 255.255.255.0 200.200.67.1 1
[root@odas1 ~]#
[root@odas1 ~]# cat /proc/net/vlan/config
VLAN Dev name | VLAN ID
Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD
bond0.2999 | 2999 | bond0
[root@odas1 ~]#
[root@odas1 ~]# odaadmcli create vlan VLAN_3001 -vlanid 3001 -if bond0 -node 0 -setuptype database -ip 200.200.131.58 -netmask 255.255.255.0 -gateway 200.200.131.1
Created Vlan : VLAN_3001
[root@odas1 ~]# odaadmcli create vlan VLAN_3010 -vlanid 3010 -if bond0 -node 0 -setuptype database -ip 200.190.3.58 -netmask 255.255.255.0 -gateway 200.190.3.1
Created Vlan : VLAN_3010
[root@odas1 ~]# odaadmcli create vlan VLAN_3030 -vlanid 3030 -if bond0 -node 0 -setuptype management -ip 200.200.192.156 -netmask 255.255.255.0 -gateway 200.200.192.1
Created Vlan : VLAN_3030
[root@odas1 ~]# odaadmcli create vlan VLAN_3002 -vlanid 3002 -if bond1 -node 0 -setuptype backup -ip 200.200.68.128 -netmask 255.255.255.0 -gateway 200.200.68.1
Created Vlan : VLAN_3002
[root@odas1 ~]# odaadmcli create vlan VLAN_3001 -vlanid 3001 -if bond0 -node 1 -setuptype database -ip 200.200.131.59 -netmask 255.255.255.0 -gateway 200.200.131.1
Created Vlan : VLAN_3001
[root@odas1 ~]# odaadmcli create vlan VLAN_3010 -vlanid 3010 -if bond0 -node 1 -setuptype database -ip 200.190.3.59 -netmask 255.255.255.0 -gateway 200.190.3.1
Created Vlan : VLAN_3010
[root@odas1 ~]# odaadmcli create vlan VLAN_3030 -vlanid 3030 -if bond0 -node 1 -setuptype management -ip 200.200.192.157 -netmask 255.255.255.0 -gateway 200.200.192.1
Created Vlan : VLAN_3030
[root@odas1 ~]# odaadmcli create vlan VLAN_3002 -vlanid 3002 -if bond1 -node 1 -setuptype backup -ip 200.200.68.129 -netmask 255.255.255.0 -gateway 200.200.68.1
Created Vlan : VLAN_3002
[root@odas1 ~]#
[root@odas1 ~]#
[root@odas1 ~]# odaadmcli show vlan
NAME ID INTERFACE CONFIG_TYPE IP_ADDRESS NETMASK GATEWAY NODENUM
Public-network 2999 bond0 Public 200.200.67.100 255.255.255.0 200.200.67.1 0
Public-network 2999 bond0 Public 200.200.67.101 255.255.255.0 200.200.67.1 1
VLAN_3002 3002 bond1 backup 200.200.68.128 255.255.255.0 200.200.68.1 0
VLAN_3002 3002 bond1 backup 200.200.68.129 255.255.255.0 200.200.68.1 1
VLAN_3001 3001 bond0 database 200.200.131.58 255.255.255.0 200.200.131.1 0
VLAN_3001 3001 bond0 database 200.200.131.59 255.255.255.0 200.200.131.1 1
VLAN_3010 3010 bond0 database 200.190.3.58 255.255.255.0 200.190.3.1 0
VLAN_3010 3010 bond0 database 200.190.3.59 255.255.255.0 200.190.3.1 1
VLAN_3030 3030 bond0 management 200.200.192.156 255.255.255.0 200.200.192.1 0
VLAN_3030 3030 bond0 management 200.200.192.157 255.255.255.0 200.200.192.1 1
[root@odas1 ~]#
Firmware and patch
As described in the documentation, the reimage process does not update firmware and we need to this after creating the appliance. This point ins controversy, because there is no information when need to do this, you can run before or after creating the appliance. Here I made after the create, but if you are in really old version and have a lot of old firmware running maybe you can run before the create to avoid issues.
The funny part is that the firmware’s are inside of one the default patch file. Again, you need to search in patch search page because Oracle removed from 88888.1 page these links (look the path number in the example below). So, the procedure to update firmware is the same to apply a patch in ODA and need to download the same patch version. As usual, the first step is to upload the files:
[root@odas1 ~]# cd /tmp/deploy/
[root@odas1 deploy]#
[root@odas1 deploy]# unzip -qa p28864490_183000_Linux-x86-64_1of3.zip
[root@odas1 deploy]# unzip -qa p28864490_183000_Linux-x86-64_2of3.zip
[root@odas1 deploy]# unzip -qa p28864490_183000_Linux-x86-64_3of3.zip
[root@odas1 deploy]#
[root@odas1 deploy]# rm -rf p*zip
[root@odas1 deploy]#
[root@odas1 deploy]# ls -l
total 15441928
-rw-r--r-- 1 root root 2767978208 Dec 13 14:50 oda-sm-18.3.0.0.0-181205-server1of3.zip
-rwxr-xr-x 1 root root 6835610213 Dec 13 15:08 oda-sm-18.3.0.0.0-181205-server2of3.zip
-rwxr-xr-x 1 root root 6193475789 Dec 13 15:22 oda-sm-18.3.0.0.0-181205-server3of3.zip
-rw-r--r-- 1 root root 1155 Dec 13 15:22 README.txt
[root@odas1 deploy]#
[root@odas1 deploy]#
If you want to check what are you running, just execute odacli describe-component (look ilom version as an example):
[root@odas1 ~]# odacli describe-component
System Version
---------------
18.3.0.0.0
System node Name
---------------
odas1
Local System Version
---------------
18.3.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 18.3.0.0.0 up-to-date
GI 18.3.0.0.180717 up-to-date
DB 18.3.0.0.180717 up-to-date
DCSAGENT 18.3.0.0.0 up-to-date
ILOM 3.2.9.23.r116695 4.0.2.26.b.r125868
BIOS 30110000 30130500
OS 6.10 up-to-date
FIRMWARECONTROLLER {
[ c0 ] 4.650.00-7176 up-to-date
[ c1,c2 ] 13.00.00.00 up-to-date
}
FIRMWAREEXPANDER 0018 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] A7E0 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, PAG1 up-to-date
c1d7,c1d8,c1d9,c1d10,c1d11,c1d12,c1d13,
c1d14,c1d15,c2d0,c2d1,c2d2,c2d3,c2d4,
c2d5,c2d6,c2d7,c2d8,c2d9,c2d10,c2d11,
c2d12,c2d13,c2d14,c2d15 ]
[ c1d16,c1d17,c1d18,c1d19,c1d20,c1d21, A29A up-to-date
c1d22,c1d23,c2d16,c2d17,c2d18,c2d19,
c2d20,c2d21,c2d22,c2d23 ]
}
System node Name
---------------
odas2
Local System Version
---------------
18.3.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 18.3.0.0.0 up-to-date
GI 18.3.0.0.180717 up-to-date
DB 18.3.0.0.180717 up-to-date
DCSAGENT 18.3.0.0.0 up-to-date
ILOM 3.2.9.23.r116695 4.0.2.26.b.r125868
BIOS 30110000 30130500
OS 6.10 up-to-date
FIRMWARECONTROLLER {
[ c0 ] 4.650.00-7176 up-to-date
[ c1,c2 ] 13.00.00.00 up-to-date
}
FIRMWAREEXPANDER 0018 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] A7E0 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, PAG1 up-to-date
c1d7,c1d8,c1d9,c1d10,c1d11,c1d12,c1d13,
c1d14,c1d15,c2d0,c2d1,c2d2,c2d3,c2d4,
c2d5,c2d6,c2d7,c2d8,c2d9,c2d10,c2d11,
c2d12,c2d13,c2d14,c2d15 ]
[ c1d16,c1d17,c1d18,c1d19,c1d20,c1d21, A29A up-to-date
c1d22,c1d23,c2d16,c2d17,c2d18,c2d19,
c2d20,c2d21,c2d22,c2d23 ]
}
root@odas1 ~]#
After that you can upload the files in the repository (look the path to the files):
[root@odas1 deploy]# odacli update-repository -f /tmp/deploy/oda-sm-18.3.0.0.0-181205-server1of3.zip,/tmp/deploy/oda-sm-18.3.0.0.0-181205-server2of3.zip,/tmp/deploy/oda-sm-18.3.0.0.0-181205-server3of3.zip
{
"jobId" : "cdfd2e72-777b-4ccb-a670-17b88c7cc102",
"status" : "Created",
"message" : "/tmp/deploy/oda-sm-18.3.0.0.0-181205-server1of3.zip,/tmp/deploy/oda-sm-18.3.0.0.0-181205-server2of3.zip,/tmp/deploy/oda-sm-18.3.0.0.0-181205-server3of3.zip",
"reports" : [ ],
"createTimestamp" : "May 22, 2019 15:01:03 PM CEST",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "May 22, 2019 15:01:03 PM CEST"
}
[root@odas1 deploy]#
[root@odas1 deploy]#
[root@odas1 deploy]# /opt/oracle/dcs/bin/odacli describe-job -i cdfd2e72-777b-4ccb-a670-17b88c7cc102
Job details
----------------------------------------------------------------
ID: cdfd2e72-777b-4ccb-a670-17b88c7cc102
Description: Repository Update
Status: Success
Created: May 22, 2019 3:01:03 PM CEST
Message: /tmp/deploy/oda-sm-18.3.0.0.0-181205-server1of3.zip,/tmp/deploy/oda-sm-18.3.0.0.0-181205-server2of3.zip,/tmp/deploy/oda-sm-18.3.0.0.0-181205-server3of3.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace May 22, 2019 3:01:03 PM CEST May 22, 2019 3:01:03 PM CEST Success
Setting up ssh equivalance May 22, 2019 3:01:03 PM CEST May 22, 2019 3:01:03 PM CEST Success
Copy BundleFile May 22, 2019 3:01:03 PM CEST May 22, 2019 3:02:47 PM CEST Success
Validating CopiedFile May 22, 2019 3:02:49 PM CEST May 22, 2019 3:03:23 PM CEST Success
Unzip bundle May 22, 2019 3:03:23 PM CEST May 22, 2019 3:05:43 PM CEST Success
Unzip bundle May 22, 2019 3:05:43 PM CEST May 22, 2019 3:08:08 PM CEST Success
Delete PatchBundles May 22, 2019 3:08:08 PM CEST May 22, 2019 3:08:11 PM CEST Success
Removing ssh keys May 22, 2019 3:08:11 PM CEST May 22, 2019 3:08:26 PM CEST Success
[root@odas1 deploy]#
[root@odas1 deploy]#
After the success to upload you first update the storage using odacli update-storage command to upload to desired version (the version are in the filename or check what you downloaded from MOS):
[root@odas1 ~]# odacli update-storage -v 18.3.0.0.0 --rolling
{
"jobId" : "18d1e5f8-7c31-4ce7-8241-f7b2ed5673cd",
"status" : "Created",
"message" : "Success of Storage Update may trigger reboot of node after 4-5 minutes. Please wait till node restart",
"reports" : [ ],
"createTimestamp" : "May 22, 2019 15:25:24 PM CEST",
"resourceList" : [ ],
"description" : "Storage Firmware Patching",
"updatedTime" : "May 22, 2019 15:25:24 PM CEST"
}
[root@odas1 ~]#
[root@odas1 ~]# /opt/oracle/dcs/bin/odacli describe-job -i 18d1e5f8-7c31-4ce7-8241-f7b2ed5673cd
Job details
----------------------------------------------------------------
ID: 18d1e5f8-7c31-4ce7-8241-f7b2ed5673cd
Description: Storage Firmware Patching
Status: Success
Created: May 22, 2019 3:25:24 PM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Applying Firmware Disk Patches May 22, 2019 3:25:24 PM CEST May 22, 2019 3:25:45 PM CEST Success
Applying Firmware Controller Patches May 22, 2019 3:25:45 PM CEST May 22, 2019 3:25:58 PM CEST Success
[root@odas1 ~]#
After you can update the server (running odacli update-server). Here, since you are “updating” to the same version that you are running, the steps related to yum does not execute and just firmware part runs:
[root@odas1 ~]# odacli update-server -v 18.3.0.0.0
{
"jobId" : "8bc7cc15-a543-4e26-ade4-8dd578ea67ba",
"status" : "Created",
"message" : "Success of Server Update may trigger reboot of node after 4-5 minutes. Please wait till node restart",
"reports" : [ ],
"createTimestamp" : "May 22, 2019 15:29:15 PM CEST",
"resourceList" : [ ],
"description" : "Server Patching",
"updatedTime" : "May 22, 2019 15:29:15 PM CEST"
}
[root@odas1 ~]#
[root@odas1 ~]# /opt/oracle/dcs/bin/odacli describe-job -i 8bc7cc15-a543-4e26-ade4-8dd578ea67ba
Job details
----------------------------------------------------------------
ID: 8bc7cc15-a543-4e26-ade4-8dd578ea67ba
Description: Server Patching
Status: Running
Created: May 22, 2019 3:29:15 PM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Patch location validation May 22, 2019 3:29:16 PM CEST May 22, 2019 3:29:16 PM CEST Success
Patch location validation May 22, 2019 3:29:16 PM CEST May 22, 2019 3:29:16 PM CEST Success
dcs-controller upgrade May 22, 2019 3:29:19 PM CEST May 22, 2019 3:29:19 PM CEST Success
dcs-controller upgrade May 22, 2019 3:29:19 PM CEST May 22, 2019 3:29:20 PM CEST Success
Patch location validation May 22, 2019 3:29:20 PM CEST May 22, 2019 3:29:21 PM CEST Success
Patch location validation May 22, 2019 3:29:20 PM CEST May 22, 2019 3:29:21 PM CEST Success
dcs-cli upgrade May 22, 2019 3:29:21 PM CEST May 22, 2019 3:29:21 PM CEST Success
dcs-cli upgrade May 22, 2019 3:29:21 PM CEST May 22, 2019 3:29:21 PM CEST Success
Creating repositories using yum May 22, 2019 3:29:29 PM CEST May 22, 2019 3:29:36 PM CEST Success
Creating repositories using yum May 22, 2019 3:29:36 PM CEST May 22, 2019 3:29:36 PM CEST Success
Creating repositories using yum May 22, 2019 3:29:36 PM CEST May 22, 2019 3:29:36 PM CEST Success
Creating repositories using yum May 22, 2019 3:29:36 PM CEST May 22, 2019 3:29:36 PM CEST Success
Creating repositories using yum May 22, 2019 3:29:36 PM CEST May 22, 2019 3:29:36 PM CEST Success
Creating repositories using yum May 22, 2019 3:29:37 PM CEST May 22, 2019 3:29:37 PM CEST Success
Creating repositories using yum May 22, 2019 3:29:37 PM CEST May 22, 2019 3:29:37 PM CEST Success
Updating YumPluginVersionLock rpm May 22, 2019 3:29:37 PM CEST May 22, 2019 3:29:37 PM CEST Success
Applying OS Patches May 22, 2019 3:29:37 PM CEST May 22, 2019 3:31:38 PM CEST Success
Creating repositories using yum May 22, 2019 3:31:38 PM CEST May 22, 2019 3:31:39 PM CEST Success
Applying HMP Patches May 22, 2019 3:31:39 PM CEST May 22, 2019 3:31:39 PM CEST Success
Patch location validation May 22, 2019 3:31:40 PM CEST May 22, 2019 3:31:40 PM CEST Success
Patch location validation May 22, 2019 3:31:40 PM CEST May 22, 2019 3:31:40 PM CEST Success
oda-hw-mgmt upgrade May 22, 2019 3:31:40 PM CEST May 22, 2019 3:31:41 PM CEST Success
oda-hw-mgmt upgrade May 22, 2019 3:31:41 PM CEST May 22, 2019 3:31:41 PM CEST Success
OSS Patching May 22, 2019 3:31:41 PM CEST May 22, 2019 3:31:43 PM CEST Success
Applying Firmware Disk Patches May 22, 2019 3:32:02 PM CEST May 22, 2019 3:32:15 PM CEST Success
Applying Firmware Expander Patches May 22, 2019 3:32:27 PM CEST May 22, 2019 3:32:34 PM CEST Success
Applying Firmware Controller Patches May 22, 2019 3:32:47 PM CEST May 22, 2019 3:32:55 PM CEST Success
Checking Ilom patch Version May 22, 2019 3:32:56 PM CEST May 22, 2019 3:32:59 PM CEST Success
Checking Ilom patch Version May 22, 2019 3:32:59 PM CEST May 22, 2019 3:33:01 PM CEST Success
Patch location validation May 22, 2019 3:33:01 PM CEST May 22, 2019 3:33:02 PM CEST Success
Patch location validation May 22, 2019 3:33:01 PM CEST May 22, 2019 3:33:02 PM CEST Success
Save password in Wallet May 22, 2019 3:33:03 PM CEST May 22, 2019 3:33:04 PM CEST Success
Apply Ilom patch May 22, 2019 3:33:04 PM CEST May 22, 2019 3:45:07 PM CEST Success
Apply Ilom patch May 22, 2019 3:45:07 PM CEST May 22, 2019 3:56:45 PM CEST Success
Copying Flash Bios to Temp location May 22, 2019 3:56:45 PM CEST May 22, 2019 3:56:45 PM CEST Success
Copying Flash Bios to Temp location May 22, 2019 3:56:45 PM CEST May 22, 2019 3:56:45 PM CEST Success
Starting the clusterware May 22, 2019 3:58:41 PM CEST May 22, 2019 4:00:09 PM CEST Success
clusterware patch verification May 22, 2019 4:00:09 PM CEST May 22, 2019 4:00:10 PM CEST Success
clusterware patch verification May 22, 2019 4:00:09 PM CEST May 22, 2019 4:00:10 PM CEST Success
Patch location validation May 22, 2019 4:00:10 PM CEST May 22, 2019 4:00:10 PM CEST Success
Patch location validation May 22, 2019 4:00:10 PM CEST May 22, 2019 4:00:10 PM CEST Success
Opatch updation May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:11 PM CEST Success
Opatch updation May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:11 PM CEST Success
Patch conflict check May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:11 PM CEST Success
Patch conflict check May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:11 PM CEST Success
clusterware upgrade May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:11 PM CEST Success
clusterware upgrade May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:11 PM CEST Success
Updating GiHome version May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:12 PM CEST Success
Updating GiHome version May 22, 2019 4:00:11 PM CEST May 22, 2019 4:00:12 PM CEST Success
Update System version May 22, 2019 4:00:33 PM CEST May 22, 2019 4:00:33 PM CEST Success
Update System version May 22, 2019 4:00:33 PM CEST May 22, 2019 4:00:33 PM CEST Success
preRebootNode Actions May 22, 2019 4:00:33 PM CEST May 22, 2019 4:00:33 PM CEST Running
[root@odas1 ~]#
After, reboot both nodes!
As example check that ilom was updated:
[root@odas1 ~]# odacli describe-component
System Version
---------------
18.3.0.0.0
System node Name
---------------
odas1
Local System Version
---------------
18.3.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 18.3.0.0.0 up-to-date
GI 18.3.0.0.180717 up-to-date
DB 18.3.0.0.180717 up-to-date
DCSAGENT 18.3.0.0.0 up-to-date
ILOM 4.0.2.26.b.r125868 up-to-date
BIOS 30130500 up-to-date
OS 6.10 up-to-date
FIRMWARECONTROLLER {
[ c0 ] 4.650.00-7176 up-to-date
[ c1,c2 ] 13.00.00.00 up-to-date
}
FIRMWAREEXPANDER 0018 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] A7E0 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, PAG1 up-to-date
c1d7,c1d8,c1d9,c1d10,c1d11,c1d12,c1d13,
c1d14,c1d15,c2d0,c2d1,c2d2,c2d3,c2d4,
c2d5,c2d6,c2d7,c2d8,c2d9,c2d10,c2d11,
c2d12,c2d13,c2d14,c2d15 ]
[ c1d16,c1d17,c1d18,c1d19,c1d20,c1d21, A29A up-to-date
c1d22,c1d23,c2d16,c2d17,c2d18,c2d19,
c2d20,c2d21,c2d22,c2d23 ]
}
System node Name
---------------
odas2
Local System Version
---------------
18.3.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 18.3.0.0.0 up-to-date
GI 18.3.0.0.180717 up-to-date
DB 18.3.0.0.180717 up-to-date
DCSAGENT 18.3.0.0.0 up-to-date
ILOM 4.0.2.26.b.r125868 up-to-date
BIOS 30130500 up-to-date
OS 6.10 up-to-date
FIRMWARECONTROLLER {
[ c0 ] 4.650.00-7176 up-to-date
[ c1,c2 ] 13.00.00.00 up-to-date
}
FIRMWAREEXPANDER 0018 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] A7E0 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, PAG1 up-to-date
c1d7,c1d8,c1d9,c1d10,c1d11,c1d12,c1d13,
c1d14,c1d15,c2d0,c2d1,c2d2,c2d3,c2d4,
c2d5,c2d6,c2d7,c2d8,c2d9,c2d10,c2d11,
c2d12,c2d13,c2d14,c2d15 ]
[ c1d16,c1d17,c1d18,c1d19,c1d20,c1d21, A29A up-to-date
c1d22,c1d23,c2d16,c2d17,c2d18,c2d19,
c2d20,c2d21,c2d22,c2d23 ]
}
[root@odas1 ~]#
[root@odas1 ~]#
Create Oracle Homes and the databases
After update all the firmware’s you can upload the clone files for Oracle Homes versions that you want and create it. The procedure is the same than before, upload the files to (and just) ODA node 1 in somewhere and after that upload the repository and create the homes.
Upload the files and update the repository. Look below that I made one for each version:
[root@odas1 ~]# cd /tmp/deploy/
[root@odas1 deploy]#
[root@odas1 deploy]# unzip -qa p23494992_183000_Linux-x86-64.zip
[root@odas1 deploy]# rm p23494992_183000_Linux-x86-64.zip
rm: remove regular file `p23494992_183000_Linux-x86-64.zip'? y
[root@odas1 deploy]#
[root@odas1 deploy]# unzip -qa p23494997_183000_Linux-x86-64.zip
replace README.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
[root@odas1 deploy]# rm p23494997_183000_Linux-x86-64.zip
rm: remove regular file `p23494997_183000_Linux-x86-64.zip'? y
[root@odas1 deploy]#
[root@odas1 deploy]# odacli update-repository -f /tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-12.1.0.2.zip
{
"jobId" : "b88c4922-1107-41f5-b7cd-e1648093051b",
"status" : "Created",
"message" : "/tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-12.1.0.2.zip",
"reports" : [ ],
"createTimestamp" : "May 22, 2019 16:34:28 PM CEST",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "May 22, 2019 16:34:28 PM CEST"
}
[root@odas1 deploy]#
[root@odas1 deploy]# /opt/oracle/dcs/bin/odacli describe-job -i b88c4922-1107-41f5-b7cd-e1648093051b
Job details
----------------------------------------------------------------
ID: b88c4922-1107-41f5-b7cd-e1648093051b
Description: Repository Update
Status: Success
Created: May 22, 2019 4:34:28 PM CEST
Message: /tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-12.1.0.2.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace May 22, 2019 4:34:28 PM CEST May 22, 2019 4:34:28 PM CEST Success
Setting up ssh equivalance May 22, 2019 4:34:28 PM CEST May 22, 2019 4:34:29 PM CEST Success
Copy BundleFile May 22, 2019 4:34:29 PM CEST May 22, 2019 4:34:54 PM CEST Success
Validating CopiedFile May 22, 2019 4:34:54 PM CEST May 22, 2019 4:35:07 PM CEST Success
Unzip bundle May 22, 2019 4:35:07 PM CEST May 22, 2019 4:35:42 PM CEST Success
Unzip bundle May 22, 2019 4:35:46 PM CEST May 22, 2019 4:36:23 PM CEST Success
Delete PatchBundles May 22, 2019 4:36:24 PM CEST May 22, 2019 4:36:25 PM CEST Success
Removing ssh keys May 22, 2019 4:36:25 PM CEST May 22, 2019 4:36:47 PM CEST Success
[root@odas1 deploy]#
[root@odas1 deploy]# odacli update-repository -f /tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-11.2.0.4.zip
{
"jobId" : "6698aa6d-1a44-4b1c-bda1-1e691a41a133",
"status" : "Created",
"message" : "/tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-11.2.0.4.zip",
"reports" : [ ],
"createTimestamp" : "May 22, 2019 16:38:53 PM CEST",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "May 22, 2019 16:38:53 PM CEST"
}
[root@odas1 deploy]#
[root@odas1 deploy]# /opt/oracle/dcs/bin/odacli describe-job -i 6698aa6d-1a44-4b1c-bda1-1e691a41a133
Job details
----------------------------------------------------------------
ID: 6698aa6d-1a44-4b1c-bda1-1e691a41a133
Description: Repository Update
Status: Success
Created: May 22, 2019 4:38:53 PM CEST
Message: /tmp/deploy/odacli-dcs-18.3.0.0.0-180905-DB-11.2.0.4.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace May 22, 2019 4:38:53 PM CEST May 22, 2019 4:38:53 PM CEST Success
Setting up ssh equivalance May 22, 2019 4:38:53 PM CEST May 22, 2019 4:38:53 PM CEST Success
Copy BundleFile May 22, 2019 4:38:54 PM CEST May 22, 2019 4:39:04 PM CEST Success
Validating CopiedFile May 22, 2019 4:39:06 PM CEST May 22, 2019 4:39:14 PM CEST Success
Unzip bundle May 22, 2019 4:39:14 PM CEST May 22, 2019 4:39:32 PM CEST Success
Unzip bundle May 22, 2019 4:39:32 PM CEST May 22, 2019 4:39:54 PM CEST Success
Delete PatchBundles May 22, 2019 4:39:54 PM CEST May 22, 2019 4:39:55 PM CEST Success
Removing ssh keys May 22, 2019 4:39:55 PM CEST May 22, 2019 4:39:55 PM CEST Success
[root@odas1 deploy]#
[root@odas1 deploy]#
After that, you can create the Oracle home for each version using the command odacli create-dbhome. 12c:
[root@odas1 deploy]#
[root@odas1 deploy]# odacli create-dbhome -v 12.1.0.2.180717
Job details
----------------------------------------------------------------
ID: 5ddcdd49-2d1a-404d-b4b8-7ab79c5f9707
Description: Database Home OraDB12102_home1 creation with version :12.1.0.2.180717
Status: Created
Created: May 22, 2019 4:42:58 PM CEST
Message: Create Database Home
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
[root@odas1 deploy]#
[root@odas1 deploy]# /opt/oracle/dcs/bin/odacli describe-job -i 5ddcdd49-2d1a-404d-b4b8-7ab79c5f9707
Job details
----------------------------------------------------------------
ID: 5ddcdd49-2d1a-404d-b4b8-7ab79c5f9707
Description: Database Home OraDB12102_home1 creation with version :12.1.0.2.180717
Status: Success
Created: May 22, 2019 4:42:58 PM CEST
Message: Create Database Home
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Setting up ssh equivalance May 22, 2019 4:42:59 PM CEST May 22, 2019 4:42:59 PM CEST Success
Validating dbHome available space May 22, 2019 4:42:59 PM CEST May 22, 2019 4:42:59 PM CEST Success
Validating dbHome available space May 22, 2019 4:42:59 PM CEST May 22, 2019 4:42:59 PM CEST Success
Creating DbHome Directory May 22, 2019 4:42:59 PM CEST May 22, 2019 4:42:59 PM CEST Success
Extract DB clones May 22, 2019 4:42:59 PM CEST May 22, 2019 4:46:22 PM CEST Success
Clone Db home May 22, 2019 4:46:22 PM CEST May 22, 2019 4:47:32 PM CEST Success
Enable DB options May 22, 2019 4:47:32 PM CEST May 22, 2019 4:47:41 PM CEST Success
Run Root DB scripts May 22, 2019 4:47:41 PM CEST May 22, 2019 4:47:42 PM CEST Success
Removing ssh keys May 22, 2019 4:47:48 PM CEST May 22, 2019 4:47:48 PM CEST Success
[root@odas1 deploy]#
[root@odas1 deploy]#
And 11g:
[root@odas1 deploy]# odacli create-dbhome -v 11.2.0.4.180717
Job details
----------------------------------------------------------------
ID: d4512833-3198-4384-b0d5-3588e3ec8cdb
Description: Database Home OraDB11204_home1 creation with version :11.2.0.4.180717
Status: Created
Created: May 22, 2019 4:50:06 PM CEST
Message: Create Database Home
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
[root@odas1 deploy]#
[root@odas1 deploy]# /opt/oracle/dcs/bin/odacli describe-job -i d4512833-3198-4384-b0d5-3588e3ec8cdb
Job details
----------------------------------------------------------------
ID: d4512833-3198-4384-b0d5-3588e3ec8cdb
Description: Database Home OraDB11204_home1 creation with version :11.2.0.4.180717
Status: Success
Created: May 22, 2019 4:50:06 PM CEST
Message: Create Database Home
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Setting up ssh equivalance May 22, 2019 4:50:07 PM CEST May 22, 2019 4:50:07 PM CEST Success
Validating dbHome available space May 22, 2019 4:50:07 PM CEST May 22, 2019 4:50:07 PM CEST Success
Validating dbHome available space May 22, 2019 4:50:07 PM CEST May 22, 2019 4:50:07 PM CEST Success
Creating DbHome Directory May 22, 2019 4:50:07 PM CEST May 22, 2019 4:50:07 PM CEST Success
Extract DB clones May 22, 2019 4:50:07 PM CEST May 22, 2019 4:51:59 PM CEST Success
Clone Db home May 22, 2019 4:52:00 PM CEST May 22, 2019 4:53:06 PM CEST Success
Enable DB options May 22, 2019 4:53:06 PM CEST May 22, 2019 4:53:11 PM CEST Success
Run Root DB scripts May 22, 2019 4:53:11 PM CEST May 22, 2019 4:53:11 PM CEST Success
Removing ssh keys May 22, 2019 4:53:16 PM CEST May 22, 2019 4:53:17 PM CEST Success
[root@odas1 deploy]#
[root@odas1 deploy]#
And after that you can use the odacli list-dbhomes to see the options for database creation:
ID Name DB Version Home Location Status
---------------------------------------- -------------------- ---------------------------------------- --------------------------------------------- ----------
886021e5-9bbf-4a14-9b50-398ddd00bfd0 OraDB18000_home1 18.3.0.0.180717 /u01/app/oracle/product/18.0.0.0/dbhome_1 Configured
d791001f-b812-49fd-94f5-b11c5f580a30 OraDB12102_home1 12.1.0.2.180717 /u01/app/oracle/product/12.1.0.2/dbhome_1 Configured
d162ff30-dc0d-4c4b-810e-c8f513dc9f62 OraDB11204_home1 11.2.0.4.180717 /u01/app/oracle/product/11.2.0.4/dbhome_1 Configured
[root@odas1 deploy]#
As an example, if you want to delete the database that was created during the creation of appliance just execute odacli delete-database:
[root@odas1 ~]# odacli delete-database --dbName mfod
{
"jobId" : "69adb079-e5e4-4d64-8b20-4eee73c65491",
"status" : "Running",
"message" : null,
"reports" : [ ],
"createTimestamp" : "May 22, 2019 16:17:27 PM CEST",
"resourceList" : [ ],
"description" : "Database service deletion with db name: mfod with id : 938b616a-99ed-4466-8d76-820d3489517d",
"updatedTime" : "May 22, 2019 16:17:27 PM CEST"
}
[root@odas1 ~]#
