Blog – Page 9 – LUXOUG – LUXEMBOURG ORACLE USERS GROUP

Blog

DB_UNIQUE_NAME, PDB, and Data Guard

Category: Database Author: Fernando Simon (Board Member) Date: 5 years ago Comments: 0

DB_UNIQUE_NAME, PDB, and Data Guard

When you change the parameters for the database is possible to specify the db_unique_name and allow more control where you want to apply/use it. This is very useful to limit the scope, but you need to be aware of some collateral effects. Even not present at the official doc, you can use it. But check here some details that you need to take care of.

Unplug and Plug

The environment below is a simple 19c single instance database. There, exists 3 PDB’s and you can see the db_unique_name for this CDB:

SQL> show pdbs




    CON_ID CON_NAME                       OPEN MODE  RESTRICTED

---------- ------------------------------ ---------- ----------

         2 PDB$SEED                       READ ONLY  NO

         3 PDBDG                          READ WRITE NO

         5 PDBMOVE                        READ WRITE NO

SQL> show parameter db_unique_name;




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

db_unique_name                       string      or19dg

SQL>

And we can connect in the PDBMOVE and change the parameters. First, as usual, normal set without any special parameter:

SQL> alter system set open_cursors = 300 scope = spfile;




System altered.




SQL>

But, we can specify the db_unique_name for alter system. ALTER SYSTEM… DB_UNIQUE_NAME = ‘VALUE’. Here you can see that I used the same that as defined for root level.

SQL> alter system set sessions = 100  scope = spfile db_unique_name = 'or19dg';




System altered.




SQL> alter system set cursor_sharing = FORCE scope = spfile db_unique_name = 'or19dg';




System altered.




SQL>

Detail #1 here, if you specify the DB_UNIQUE_NAME the scope can be JUST spfile. Otherwise you will receive error.

SQL> alter system set cursor_sharing = FORCE scope = both db_unique_name = 'or19dg';

alter system set cursor_sharing = FORCE scope = both db_unique_name = 'or19dg'

*

ERROR at line 1:

ORA-65147: DB_UNIQUE_NAME specified without SPFILE scope

SQL>

In this example I set two parameters for PDB level and you can see at PDB_SPFILE$ inside of root cdb that values was set:

SQL> alter session set container = cdb$root;




Session altered.




SQL> select DB_UNIQ_NAME, NAME, VALUE$, PDB_UID from PDB_SPFILE$ where PDB_UID = 2549618825;




DB_UNIQ_NAME                   NAME                           VALUE$                                      PDB_UID

------------------------------ ------------------------------ ---------------------------------------- ----------

*                              open_cursors                   300                                      2549618825

or19dg                         cursor_sharing                 'FORCE'                                  2549618825

or19dg                         sessions                       100                                      2549618825




SQL>

And after the reload of the database the values are set:

[oracle@orcl19p ~]$ srvctl stop database -d or19dg

[oracle@orcl19p ~]$ srvctl start database -d or19dg

[oracle@orcl19p ~]$

[oracle@orcl19p ~]$ sqlplus / as sysdba




SQL*Plus: Release 19.0.0.0.0 - Production on Mon Jun 1 17:33:29 2020

Version 19.5.0.0.0




Copyright (c) 1982, 2019, Oracle.  All rights reserved.







Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.5.0.0.0




SQL> show pdbs




    CON_ID CON_NAME                       OPEN MODE  RESTRICTED

---------- ------------------------------ ---------- ----------

         2 PDB$SEED                       READ ONLY  NO

         3 PDBDG                          READ WRITE NO

         5 PDBMOVE                        READ WRITE NO

SQL> alter session set container = PDBMOVE;




Session altered.




SQL> show parameter cursor_sharing;




NAME                                 TYPE                             VALUE

------------------------------------ -------------------------------- ------------------------------

cursor_sharing                       string                           FORCE

SQL>

UNPLUG

But if I need to unplug the PDB, the values will be exported too, right? Yes, but no as you imagine.

So, doing a simple unplug to XML:

SQL> alter pluggable database PDBMOVE close immediate;




Pluggable database altered.




SQL> alter pluggable database PDBMOVE unplug into '/tmp/pdbmove_par.xml';




Pluggable database altered.




SQL> drop pluggable database PDBMOVE keep datafiles;




Pluggable database dropped.




SQL>

And checking the file for these parameters at generated XML:

[oracle@orcl19p ~]$ cat /tmp/pdbmove_par.xml |grep open_cursors

      <spfile>*.open_cursors=300#HWM:300,</spfile>

[oracle@orcl19p ~]$ cat /tmp/pdbmove_par.xml |grep cursor_sharing

      <spfile>*.cursor_sharing='FORCE'</spfile>

[oracle@orcl19p ~]$ cat /tmp/pdbmove_par.xml |grep sessions

      <spfile>*.sessions=100#HWM:100,flag:1,</spfile>

[oracle@orcl19p ~]$

As you can see here, the parameters was exported, but the parameter changed to “*”. The definition to specific db_unique_name was cleared.

And if we try to plug again the same PDB, we can see that parameter was loaded as “*”:

SQL> create  pluggable database PDBMOVE USING '/tmp/pdbmove_par.xml' NOCOPY TEMPFILE REUSE;




Pluggable database created.




SQL> col name format a30

SQL> col VALUE$ format a40

SQL> set linesize 250

SQL> select CON_ID,DBID,CON_UID,GUID,NAME from v$pdbs;




    CON_ID       DBID    CON_UID GUID                             NAME

---------- ---------- ---------- -------------------------------- ------------------------------

         2  826464235  826464235 9B151A78B6DB533AE0533205A00AFC30 PDB$SEED

         3 1340021208 1340021208 9B155010BEAC661BE0533205A00AF21B PDBDG

         4 2549618825 3364812106 A7082B6C610C5E1DE0533205A00AF7FE PDBMOVE




SQL> show pdbs




    CON_ID CON_NAME                       OPEN MODE  RESTRICTED

---------- ------------------------------ ---------- ----------

         2 PDB$SEED                       READ ONLY  NO

         3 PDBDG                          READ WRITE NO

         4 PDBMOVE                        MOUNTED

SQL> alter pluggable database PDBMOVE open;




Pluggable database altered.




SQL> select DB_UNIQ_NAME, NAME, VALUE$, PDB_UID from PDB_SPFILE$ where PDB_UID = 3364812106;




DB_UNIQ_NAME                   NAME                           VALUE$                                      PDB_UID

------------------------------ ------------------------------ ---------------------------------------- ----------

*                              cursor_sharing                 'FORCE'                                  3364812106

*                              open_cursors                   300                                      3364812106

*                              sessions                       100                                      3364812106




SQL>

DATAGUARD

As you can imagine, using the db_unique_name for an alter system can affect the dataguard environments. If you know, the db_unique_name is different between primary and standby. So, if you define some parameter, depending on the way that you made, after you, switchover/failover can occur that parameter returns to the default value.

It is not the case that this is wrong, maybe you need to set some parameters for one server/side in specific. Maybe because of less memory, less CPU power, or whatever the reason.

In this example, I have gold19c as primary, and golds19c as standby. And I have the GOLD19P as used PDB here:

[oracle@goldpn1 ~]$ dgmgrl sys/oracle@gold19c

DGMGRL for Linux: Release 19.0.0.0.0 - Production on Mon Jun 1 19:13:39 2020

Version 19.7.0.0.0




Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.




Welcome to DGMGRL, type "help" for information.

Connected to "GOLD19C"

Connected as SYSDBA.

DGMGRL> show configuration;




Configuration - gold19c




  Protection Mode: MaxAvailability

  Members:

  gold19c  - Primary database

    golds19c - Physical standby database




Fast-Start Failover:  Disabled




Configuration Status:

SUCCESS   (status updated 72 seconds ago)




DGMGRL> exit

[oracle@goldpn1 ~]$

[oracle@goldpn1 ~]$ sqlplus / as sysdba




SQL*Plus: Release 19.0.0.0.0 - Production on Mon Jun 1 19:16:50 2020

Version 19.7.0.0.0




Copyright (c) 1982, 2020, Oracle.  All rights reserved.







Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.7.0.0.0




SQL> show parameter db_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

db_name                              string      gold19c

SQL> show parameter db_unique_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

db_unique_name                       string      GOLD19C

SQL>

SQL> show pdbs




    CON_ID CON_NAME                       OPEN MODE  RESTRICTED

---------- ------------------------------ ---------- ----------

         2 PDB$SEED                       READ ONLY  NO

         3 GOLD19P                        READ WRITE NO

SQL> col name format a30

SQL> col VALUE$ format a40

SQL> set linesize 250

SQL> select CON_ID,DBID,CON_UID,GUID,NAME from v$pdbs;




    CON_ID       DBID    CON_UID GUID                             NAME

---------- ---------- ---------- -------------------------------- ------------------------------

         2 2661745537 2661745537 A6B13C68753F63D3E0530A07A00A6303 PDB$SEED

         3 2141275600 2141275600 A6B19FBF38306073E0530C07A00ACE20 GOLD19P




SQL>

And to set the parameters I used the same than before, with and without db_unique_name:

SQL> alter session set container = GOLD19P;




Session altered.




SQL> alter system set open_cursors = 300 scope = spfile;




System altered.




SQL> alter system set cursor_sharing = FORCE scope = spfile db_unique_name = 'GOLD19C';




System altered.




SQL>

And as you can see, the parameter was stored.

SQL> alter session set container = cdb$root;




Session altered.




SQL> select DB_UNIQ_NAME, NAME, VALUE$, PDB_UID from PDB_SPFILE$ where PDB_UID = 2141275600;




DB_UNIQ_NAME                   NAME                           VALUE$                                      PDB_UID

------------------------------ ------------------------------ ---------------------------------------- ----------

*                              open_cursors                   300                                      2141275600

GOLD19C                        cursor_sharing                 'FORCE'                                  2141275600




SQL>

But if I made the switchover:

[oracle@goldpn1 ~]$ dgmgrl sys/oracle@gold19c

DGMGRL for Linux: Release 19.0.0.0.0 - Production on Mon Jun 1 19:24:15 2020

Version 19.7.0.0.0




Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.




Welcome to DGMGRL, type "help" for information.

Connected to "GOLD19C"

Connected as SYSDBA.

DGMGRL> switchover to golds19c;

Performing switchover NOW, please wait...

Operation requires a connection to database "golds19c"

Connecting ...

Connected to "GOLDS19C"

Connected as SYSDBA.

New primary database "golds19c" is opening...

Oracle Clusterware is restarting database "gold19c" ...

Connected to "GOLD19C"

Connected to "GOLD19C"

Switchover succeeded, new primary is "golds19c"

DGMGRL>

You can see that now, the value for the parameter cursor_sharing now have the default value because the db_unique_name does not hit what was defined:

[oracle@goldsn1 ~]$ sqlplus / as sysdba




SQL*Plus: Release 19.0.0.0.0 - Production on Mon Jun 1 19:35:01 2020

Version 19.7.0.0.0




Copyright (c) 1982, 2020, Oracle.  All rights reserved.







Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.7.0.0.0




SQL> show parameter db_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

db_name                              string      gold19c

SQL> show parameter db_unique_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

db_unique_name                       string      GOLDS19C

SQL>

SQL> alter session set container = GOLD19P;




Session altered.




SQL> show parameter cursor_sharing;




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

cursor_sharing                       string      EXACT

SQL> show parameter open_cursors;




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

open_cursors                         integer     300

SQL>

But you can see that the parameter is defined at PDB_SPFILE$:

SQL> alter session set container = cdb$root;




Session altered.




SQL>

SQL> col name format a30

SQL> col VALUE$ format a40

SQL> set linesize 250

SQL> select CON_ID,DBID,CON_UID,GUID,NAME from v$pdbs;




    CON_ID       DBID    CON_UID GUID                             NAME

---------- ---------- ---------- -------------------------------- ------------------------------

         2 2661745537 2661745537 A6B13C68753F63D3E0530A07A00A6303 PDB$SEED

         3 2141275600 2141275600 A6B19FBF38306073E0530C07A00ACE20 GOLD19P




SQL> select DB_UNIQ_NAME, NAME, VALUE$, PDB_UID from PDB_SPFILE$ where PDB_UID = 2141275600;




DB_UNIQ_NAME                   NAME                           VALUE$                                      PDB_UID

------------------------------ ------------------------------ ---------------------------------------- ----------

*                              open_cursors                   300                                      2141275600

GOLD19C                        cursor_sharing                 'FORCE'                                  2141275600




SQL>

As showed, the value at standby is not defied, but as told before this can be expected behaviour. Or no. If you want to set the parameter and it became value in both, primary and standby, you don’t need to specify the db_unique_name, otherwise when the switchover/failover occurs the value will not be there. And, this can lead to some unexpected behaviours (at application side as and example) and need to troubleshoot (and until find this about set parameter, can be a long path).

Others points

The db_unique_name option does not check what you specify, so, take care to set the correct value. As you can see below I defined it as SIMON, and was accepted and saved:

SQL> alter session set container = GOLD19P;

Session altered.

SQL> alter system set cursor_sharing = FORCE scope = spfile db_unique_name = 'SIMON';

System altered.

SQL> alter session set container = cdb$root;

Session altered.

SQL> select DB_UNIQ_NAME, NAME, VALUE$, PDB_UID from PDB_SPFILE$ where PDB_UID = 2141275600;

DB_UNIQ_NAME                   NAME                           VALUE$                                      PDB_UID

------------------------------ ------------------------------ ---------------------------------------- ----------

*                              open_cursors                   300                                      2141275600

GOLD19C                        cursor_sharing                 'FORCE'                                  2141275600

SIMON                          cursor_sharing                 'FORCE'                                  2141275600

SQL>

And to reset, the process is similar. As you see below, you can specify the db_unique_name to delete specific. But if you specify nothing, the reset will be done for all.

SQL> alter system reset cursor_sharing scope = spfile db_unique_name = 'GOLD19C';

System altered.

SQL> alter system reset cursor_sharing scope = spfile db_unique_name = 'GOLD19C';

alter system reset cursor_sharing scope = spfile db_unique_name = 'GOLD19C'

*

ERROR at line 1:

ORA-32010: cannot find entry to delete in SPFILE

SQL> alter system set cursor_sharing = FORCE scope = spfile db_unique_name = 'GOLD19C';

System altered.

SQL> alter system reset cursor_sharing scope = spfile;

System altered.

SQL> alter system reset cursor_sharing scope = spfile db_unique_name = 'GOLD19C';

alter system reset cursor_sharing scope = spfile db_unique_name = 'GOLD19C'

*

ERROR at line 1:

ORA-32010: cannot find entry to delete in SPFILE

SQL>

Conclusion

Sometimes a simple definition can lead to some strange behaviors. Think that your application is running ok and after a switchover change completely the way how it works (cursor_sharing as above). And until you dig the solution can take time.

Db_unique_name for set parameter needs to be used carefully to avoid these cases. It is not the case that you never need to use it, sometimes this definition needs to be used. You can use it to prepare everything BEFORE the switchover to avoid some error or to tune the database since the beginning.

References

Initialization parameters in a Multitenant database – FAQ and Examples (Doc ID 2101638.1)

Initialization parameters in a Multitenant database – Facts and additional information (Doc ID 2101596.1)

2.7 Managing Initialization Parameters Using a Server Parameter File

Disclaimer: “The postings on this site are my own and don’t necessarily represent my actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications were removed to allow reach the generic audience and to be useful for the community.”

New Feature – Per-Second Billing for Compute and Autonomous Database.

Category: Cloud Author: Andre Luiz Dutra Ontalba (Board Member) Date: 5 years ago Comments: 0

New Feature - Per-Second Billing for Compute and Autonomous Database.

Oracle this month announced a new form of billing for Compute Instance and Autonomous Database with that the Oracle Cloud has a stronger platform for legacy applications moving to cloud and a new family of cloud native applications that rely on microservices and dynamic scaling.

Compute instances are now billed per second of usage, rather than per hour. This helps you reduce costs when using instances for short amounts of time. Virtual machine (VM) instances have a minimum charge of one minute. Bare metal instances have a minimum charge of one hour. After the first minute (for VMs) or the first hour (for bare metal instances), usage is billed in one-second increments.

With this billing model, usage of Compute and Autonomous Database is billed per-second. All prices continue to be quoted on an hourly basis.

Here are some details about this billing model:

All virtual machine (VM) Compute instances, including those with graphical processing unit (GPU) chips, are now billed per-second with a one-minute minimum.

All bare metal Compute instances, including those with GPU chips, are now billed per-second with a one-hour minimum.

Autonomous Data Warehouse and Autonomous Transaction Processing usage is now billed per-second with a one-minute minimum.

Windows OS images are now billed per-second with a one-minute minimum.

Microsoft SQL Server images available in the Oracle Cloud Marketplace are now billed per-second with a 744-hour (one month) minimum.

The workloads that will see the biggest impact from this change are those that stop and start frequently, and those that run for short durations. Here are some examples of significant benefits:

Application and database development
High-performance computing
Continuous integration/continuous development(CI/CD)
Big boost to Cloud Native applications – Microservices & Dynamic Scaling

I believe that with this, Oracle increasingly invests and provides more resources and different ways for its customers to be able to plan and prepare their environments for the cloud.

Reference: https://docs.cloud.oracle.com/

I hope this short article has helped.

Andre Ontalba

Exadata and ZDLRA, Disable HAIP

Category: Engineer System Author: Fernando Simon (Board Member) Date: 5 years ago Comments: 0

Exadata and ZDLRA, Disable HAIP

HAIP (High Availability IP) is not supported for the Exadata environment but can occur (if you did not create the cluster using OEDA) that HAIP became in use. And this particularity true for ZDLRA. So, during the upgrade from the previous version (12.2) to a higher version, it is needed to remove HAIP.

Usually, when we upgrading from 12.2 to 18c the HAIP is removed from Exadata. If the upgrade is from 12.1, and HAIP is there, it continues and is not removed by the upgrade process. If you are using HAIP and your GI is 12.1, this procedure as-is described here can’t be used (need some adaptation), because of some requirements from ASM+ACFS+DB. But since this is a preliminary step from a GI upgrade, the focus is to disable and remove it from GI.

The HAIP is not needed for Exadata because by architecture the InfiniBand network already defines (per server) two IP’s to avoid the single point of failure. So, it is not needed to create an additional layer (HAIP and virtual IP), that does the same that already exists by network design.

*Image removed from Oracle Presentation: Oracle Real Application Clusters 19c- Best Practices and Internals- EMEA Tour Oct 2019

This procedure can be used for a non-Exadata environment too. But before continue it is important to be aware of some details with ACFS. If you use ACFS, be careful since the HAIP IP is used in the ASM proxy and needs to be updated; and if ACFS is in place, this procedure is valid just for GI 12.2+.

Environment

In this scenario, I am removing HAIP from ZDLRA GI. The HAIP has configured automatically and we are using ACFS.

The focus is to disable HAIP and remove it from CRS. Another detail is that we need to remove the dependencies between the CRS resources (like ASM that need HAIP) too.

One important detail is that since we are removing the HAIP, we need to swap the HAIP IP that it is used by services (like ASM Proxy) for something that exists. So, you need to have high availability IP’s to use. For Exadata, we use the interconnect IP’s, but if you are using in another environment, be careful that your network has the needed requirements (throughput, failover, and others).

GI Upgrade

During the GI upgrade from 12.2 to 19c we need to run the runcluvfy and it will detect if HAIP is enabled:

....

....

Checks did not pass for the following nodes:

        zeroserv02,zeroserv01







Failures were encountered during execution of CVU verification request "stage -pre crsinst".




Verifying Node Connectivity ...FAILED

zeroserv02: PRVG-11068 : Highly Available IP (HAIP) is enabled on the nodes

            "zeroserv01,zeroserv02".




zeroserv01: PRVG-11068 : Highly Available IP (HAIP) is enabled on the nodes

            "zeroserv01,zeroserv02".




Verifying RPM Package Manager database ...INFORMATION

PRVG-11250 : The check "RPM Package Manager database" was not performed because

it needs 'root' user privileges.







CVU operation performed:      stage -pre crsinst

Date:                         Dec 9, 2019 1:38:46 PM

CVU home:                     /u01/app/19.0.0.0/grid/

User:                         oracle

[root@zeroserv01 ~]#

Remove HAIP

The steps need to be executed in order to avoid errors during the procedure. Unfortunately, it is needed to have one maintenance window due to the unavailability of the services. The running databases need to be shutdown due to the CRS restart by an example.

Bellow, the steps are summarized and explained to be followed. Please be aware of the IP’s involved (they will be different in your case).

1 – Shutdown services

So, this first point is to shutdown all databases running in this cluster:

[oracle@zeroserv01 ~]$ srvctl stop database -d zdlras -o immediate

[oracle@zeroserv01 ~]$

2- Check HAIP, ASM Proxy, and ACFS

It is needed to check if HAIP is enabled. Another detail is check for ASM proxy, and ACFS too:

[oracle@zeroserv01 ~]$ $ORACLE_HOME/bin/crsctl stat res ora.cluster_interconnect.haip -t -init

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.cluster_interconnect.haip

      1        ONLINE  ONLINE       zeroserv01               STABLE

--------------------------------------------------------------------------------

[oracle@zeroserv01 ~]$

[oracle@zeroserv01 ~]$ $ORACLE_HOME/bin/crsctl stat res ora.proxy_advm -t

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.proxy_advm

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

--------------------------------------------------------------------------------

[oracle@zeroserv01 ~]$

[oracle@zeroserv01 ~]$ $ORACLE_HOME/bin/crsctl stat res -w "TYPE = ora.acfs.type" -t

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.catalog.raadmin.acfs

               ONLINE  ONLINE       zeroserv01               mounted on /raacfs/r

                                                             aadmin,STABLE

               ONLINE  ONLINE       zeroserv02               mounted on /raacfs/r

                                                             aadmin,STABLE

ora.catalog.raosbadmin.acfs

               ONLINE  ONLINE       zeroserv01               mounted on /osbcat,S

                                                             TABLE

               ONLINE  ONLINE       zeroserv02               mounted on /osbcat,S

                                                             TABLE

--------------------------------------------------------------------------------

[oracle@zeroserv01 ~]$

3 – Check New IP

The next step is to check the new IP that will be used. Since ASM does not use HAIP, we can (at Exadata) pickup the same used IP:

[oracle@zeroserv01 ~]$ export ORACLE_SID=+ASM1

[oracle@zeroserv01 ~]$ export ORACLE_HOME=/u01/app/12.2.0.1/grid

[oracle@zeroserv01 ~]$ export PATH=$ORACLE_HOME/bin:$PATH

[oracle@zeroserv01 ~]$

[oracle@zeroserv01 ~]$ sqlplus / as sysasm




SQL*Plus: Release 12.2.0.1.0 Production on Mon Dec 9 15:00:55 2019




Copyright (c) 1982, 2016, Oracle.  All rights reserved.







Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production




SQL> show parameter cluster_interconnects




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

cluster_interconnects                string      192.168.10.1:192.168.10.2

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

[oracle@zeroserv01 ~]$

[oracle@zeroserv01 ~]$ $ORACLE_HOME/bin/oifcfg getif

bondeth0  10.208.68.0  global  public

ib0  192.168.8.0  global  cluster_interconnect,asm

ib1  192.168.8.0  global  cluster_interconnect,asm

[oracle@zeroserv01 ~]$

In this case, the IP that will substitute HAIP IP is 192.168.10.1/192.168.10.2. And, as you can see above, they are from the interconnect network.

4 – Fixing IP for ASM Proxy

The next step is to fix the IP that is used by the ASM Proxy instance. But default, it pickup the HAIP IP during the startup. The idea here is to force the same IP than the ASM instance for cluster_interconnect parameter.

Connecting at APX instance

Connect at APX instance, check the parameters (and see that are without values). After that we create one init file from memory to have the backup:

[oracle@zeroserv01 ~]$ export ORACLE_HOME=/u01/app/12.2.0.1/grid

[oracle@zeroserv01 ~]$ export PATH=$ORACLE_HOME/bin:$PATH

[oracle@zeroserv01 ~]$ export ORACLE_SID=+APX1

[oracle@zeroserv01 ~]$

[oracle@zeroserv01 ~]$ sqlplus / as sysasm




SQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:20:59 2019




Copyright (c) 1982, 2016, Oracle.  All rights reserved.







Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production




SQL>  show parameter instance_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

instance_name                        string      +APX1

SQL>  show parameter instance_type




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

instance_type                        string      ASMPROXY

SQL> show parameter cluster_interconnects




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

cluster_interconnects                string

SQL> create pfile = '/tmp/pfileapx1' from memory;




File created.




SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

[oracle@zeroserv01 ~]$

Changing the IP

For ASM Proxy instance we need to set the IP for cluster_interconnect parameter with the same IP used by ASM instance.

To do that, we need to set it in init for ASM Proxy. But since the ASM Proxy instance doesn’t have one, we create one at $GI_HOME/dbs with the connect name (look below the folder and instance name). Below I set just the cluster to interconnect parameter and dedicated to the instance running in this node.

[oracle@zeroserv01 ~]$ echo "+APX1.cluster_interconnects='192.168.10.1:192.168.10.2'" > $ORACLE_HOME/dbs/init+APX1.ora

[oracle@zeroserv01 ~]$

[oracle@zeroserv01 ~]$ cat $ORACLE_HOME/dbs/init+APX1.ora

+APX1.cluster_interconnects='192.168.10.1:192.168.10.2'

[oracle@zeroserv01 ~]$

Restart the ASM Proxy

After creating the init, we restart the ASM Proxy in this node.

[root@zeroserv01 ~]# export ORACLE_HOME=/u01/app/12.2.0.1/grid

[root@zeroserv01 ~]# export PATH=$ORACLE_HOME/bin:$PATH

[root@zeroserv01 ~]#

[root@zeroserv01 ~]# $ORACLE_HOME/bin/srvctl stop asm -proxy -node zeroserv01 -force

[root@zeroserv01 ~]#

[root@zeroserv01 ~]# $ORACLE_HOME/bin/srvctl start asm -proxy -node zeroserv01

[root@zeroserv01 ~]#

[root@zeroserv01 ~]# ps -ef |grep APX

oracle   267811      1  0 10:43 ?        00:00:00 apx_pmon_+APX1

oracle   267813      1  0 10:43 ?        00:00:00 apx_clmn_+APX1

oracle   267815      1  0 10:43 ?        00:00:00 apx_psp0_+APX1

oracle   267820      1  1 10:43 ?        00:00:00 apx_vktm_+APX1

oracle   267824      1  0 10:43 ?        00:00:00 apx_gen0_+APX1

oracle   267826      1  0 10:43 ?        00:00:00 apx_mman_+APX1

oracle   267830      1  0 10:43 ?        00:00:00 apx_gen1_+APX1

oracle   267834      1  0 10:43 ?        00:00:00 apx_diag_+APX1

oracle   267836      1  0 10:43 ?        00:00:00 apx_dskm_+APX1

oracle   267838      1  0 10:43 ?        00:00:00 apx_pman_+APX1

oracle   267840      1  0 10:43 ?        00:00:00 apx_dia0_+APX1

oracle   267842      1  0 10:43 ?        00:00:00 apx_lreg_+APX1

oracle   267845      1  0 10:43 ?        00:00:00 apx_pxmn_+APX1

oracle   267847      1  0 10:43 ?        00:00:00 apx_rbal_+APX1

oracle   267849      1  0 10:43 ?        00:00:00 apx_vbg0_+APX1

oracle   267851      1  0 10:43 ?        00:00:00 apx_vdbg_+APX1

oracle   267853      1  0 10:43 ?        00:00:00 apx_vubg_+APX1

root     267979  32720  0 10:43 pts/0    00:00:00 grep --color=auto APX

[root@zeroserv01 ~]#

Look that instance restarted correctly and it is up.

And we can check the parameter if it is OK with the IP:

[oracle@zeroserv01 ~]$ sqlplus / as sysasm




SQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:51:59 2019




Copyright (c) 1982, 2016, Oracle.  All rights reserved.







Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production




SQL> show parameter cluster_interconnects




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

cluster_interconnects                string      192.168.10.1:192.168.10.2

SQL> show parameter instance_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

instance_name                        string      +APX1

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

[oracle@zeroserv01 ~]$

If the instance not startup (or the IP’s are wrong) you need to check the init file that you created in $GIHOME/dbs folder.

Restart ACFS

Since we restarted the ASM Proxy instance, the ACFS mountpoints went down and need to be up again:

[root@zeroserv01 ~]# $ORACLE_HOME/bin/crsctl start res -w "TYPE = ora.acfs.type" -n zeroserv01

CRS-2672: Attempting to start 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01'

CRS-2672: Attempting to start 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01'

CRS-2676: Start of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01' succeeded

CRS-2672: Attempting to start 'ora.catalog.raosbadmin.acfs' on 'zeroserv01'

CRS-2676: Start of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01' succeeded

CRS-2672: Attempting to start 'ora.catalog.raadmin.acfs' on 'zeroserv01'

CRS-2676: Start of 'ora.catalog.raosbadmin.acfs' on 'zeroserv01' succeeded

CRS-2676: Start of 'ora.catalog.raadmin.acfs' on 'zeroserv01' succeeded

[root@zeroserv01 ~]#

I started up using the CRS resource for that.

Other Nodes

After it is Ok for node 1, we can do it for other nodes. Here, since I made for ZDLRA, I have just the second node to fix ASM Proxy. Be careful with the IP’s used, they are specific for this node and remember to use the correct init file.

[root@zeroserv02 ~]# su - oracle

Last login: Tue Dec 10 10:53:02 CET 2019

[oracle@zeroserv02 ~]$ export ORACLE_SID=+ASM2

[oracle@zeroserv02 ~]$ export ORACLE_HOME=/u01/app/12.2.0.1/grid

[oracle@zeroserv02 ~]$ export PATH=$ORACLE_HOME/bin:$PATH

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$ sqlplus / as sysasm




SQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:56:24 2019




Copyright (c) 1982, 2016, Oracle.  All rights reserved.







Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production




SQL> show parameter cluster_interconnects




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

cluster_interconnects                string      192.168.10.3:192.168.10.4

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$ export ORACLE_HOME=/u01/app/12.2.0.1/grid

[oracle@zeroserv02 ~]$ export PATH=$ORACLE_HOME/bin:$PATH

[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX1

[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX2

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$ sqlplus / as sysasm




SQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 10:57:15 2019




Copyright (c) 1982, 2016, Oracle.  All rights reserved.







Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production




SQL>

SQL> show parameter instance_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

instance_name                        string      +APX2

SQL> show parameter cluster_interconnects




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

cluster_interconnects                string

SQL> create pfile = '/tmp/pfileapx2' from memory;




File created.




SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$ ls -l $ORACLE_HOME/dbs/init?APX*

ls: cannot access /u01/app/12.2.0.1/grid/dbs/init?APX*: No such file or directory

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$ echo "+APX2.cluster_interconnects='192.168.10.3:192.168.10.4'" > $ORACLE_HOME/dbs/init+APX2.ora

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$ cat $ORACLE_HOME/dbs/init+APX2.ora

+APX2.cluster_interconnects='192.168.10.3:192.168.10.4'

[oracle@zeroserv02 ~]$

[oracle@zeroserv02 ~]$ exit   

[root@zeroserv02 ~]# export ORACLE_HOME=/u01/app/12.2.0.1/grid

[root@zeroserv02 ~]# export PATH=$ORACLE_HOME/bin:$PATH

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]# $ORACLE_HOME/bin/srvctl stop asm -proxy -node zeroserv02 -force

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]# $ORACLE_HOME/bin/srvctl start asm -proxy -node zeroserv02

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]# $ORACLE_HOME/bin/crsctl start res -w "TYPE = ora.acfs.type" -n zeroserv02

CRS-2672: Attempting to start 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02'

CRS-2672: Attempting to start 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02'

CRS-2676: Start of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02' succeeded

CRS-2672: Attempting to start 'ora.catalog.raosbadmin.acfs' on 'zeroserv02'

CRS-2676: Start of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02' succeeded

CRS-2672: Attempting to start 'ora.catalog.raadmin.acfs' on 'zeroserv02'

CRS-2676: Start of 'ora.catalog.raosbadmin.acfs' on 'zeroserv02' succeeded

CRS-2676: Start of 'ora.catalog.raadmin.acfs' on 'zeroserv02' succeeded

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]# su - oracle

Last login: Tue Dec 10 10:54:10 CET 2019

[oracle@zeroserv02 ~]$ export ORACLE_HOME=/u01/app/12.2.0.1/grid

[oracle@zeroserv02 ~]$ export PATH=$ORACLE_HOME/bin:$PATH

[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX1

[oracle@zeroserv02 ~]$ export ORACLE_SID=+APX2




[oracle@zeroserv02 ~]$ sqlplus / as sysasm




SQL*Plus: Release 12.2.0.1.0 Production on Tue Dec 10 11:00:46 2019




Copyright (c) 1982, 2016, Oracle.  All rights reserved.







Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production




SQL> show parameter cluster_interconnects




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

cluster_interconnects                string      192.168.10.3:192.168.10.4

SQL> show parameter instance_name




NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

instance_name                        string      +APX2

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

You have new mail in /var/spool/mail/oracle

[oracle@zeroserv02 ~]$

5 – ASM Dependency

Since the ASM since the CRS depends on HAIP we need to remove this dependency. Here we have a tricky part. It is needed to completely change the dependencies for ASM, setting it to CRS/CSS directly.

First, at first node we check the current dependency for START and STOP:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stat res ora.asm -p |grep START_DEPENDENCIES=

START_DEPENDENCIES=hard(ora.ASMNET1LSNR_ASM.lsnr,ora.ASMNET2LSNR_ASM.lsnr) weak(ora.LISTENER.lsnr) pullup(ora.ASMNET1LSNR_ASM.lsnr,ora.ASMNET2LSNR_ASM.lsnr) dispersion:active(site:type:ora.asm.type)

[root@zeroserv01 ~]#

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stat res ora.asm -p |grep STOP_DEPENDENCIES=

STOP_DEPENDENCIES=hard(intermediate:ora.ASMNET1LSNR_ASM.lsnr,intermediate:ora.ASMNET2LSNR_ASM.lsnr)

[root@zeroserv01 ~]#

And now we change it. Look at the parameters values. They are completely different, and need to be with these specific values:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl modify res ora.asm -attr "START_DEPENDENCIES='hard(ora.cssd,ora.ctssd) pullup(ora.cssd,ora.ctssd) weak(ora.drivers.acfs)',STOP_DEPENDENCIES='hard(intermediate:ora.cssd)'" -init

[root@zeroserv01 ~]#

If you see closely the HAIP it is not listed there as a dependency, setting with the values, we completely remove HAIP dependency and inheritance.

And we need to do the same for the other node:

[root@zeroserv02 ~]# /u01/app/12.2.0.1/grid/bin/crsctl modify res ora.asm -attr "START_DEPENDENCIES='hard(ora.cssd,ora.ctssd) pullup(ora.cssd,ora.ctssd) weak(ora.drivers.acfs)',STOP_DEPENDENCIES='hard(intermediate:ora.cssd)'" -init

[root@zeroserv02 ~]#

6 – Disable HAIP Resource

The next step is to disable the HAIP resource from startup at CRS. We do this in both nodes.

Node 1:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl modify res ora.cluster_interconnect.haip -attr "ENABLED=0" -init

[root@zeroserv01 ~]#

Node 2:

[root@zeroserv02 ~]# /u01/app/12.2.0.1/grid/bin/crsctl modify res ora.cluster_interconnect.haip -attr "ENABLED=0" -init

[root@zeroserv02 ~]#

But check that we do not stop yet the HAIP resource. It is needed to remain up at this moment.

If we check with the init for CRS, it is still there in node 1:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stat res -t -init

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.cluster_interconnect.haip

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.crf

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.crsd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.cssd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.cssdmonitor

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.ctssd

      1        ONLINE  ONLINE       zeroserv01               OBSERVER,STABLE

ora.diskmon

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.drivers.acfs

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.drivers.oka

      1        OFFLINE OFFLINE                               STABLE

ora.evmd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.gipcd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.gpnpd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.mdnsd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.storage

      1        ONLINE  ONLINE       zeroserv01               STABLE

--------------------------------------------------------------------------------

[root@zeroserv01 ~]#

And for the other node too:

[root@zeroserv02 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stat res -t -init

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.cluster_interconnect.haip

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.crf

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.crsd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.cssd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.cssdmonitor

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.ctssd

      1        ONLINE  ONLINE       zeroserv02               ACTIVE:0,STABLE

ora.diskmon

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.drivers.acfs

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.drivers.oka

      1        OFFLINE OFFLINE                               STABLE

ora.evmd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.gipcd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.gpnpd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.mdnsd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.storage

      1        ONLINE  ONLINE       zeroserv02               STABLE

--------------------------------------------------------------------------------

[root@zeroserv02 ~]#

7 – Restart CRS

After we configure the dependencies for ASM we can restart CRS to shutfown HAIP (and test the changes that we made). I prefer to execute, first, in just one node and if everything goes well, I do for the others.

Stop Cluster

So, first the STOP CLUSTER in the first node:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stop cluster -f

CRS-2673: Attempting to stop 'ora.crsd' on 'zeroserv01'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'zeroserv01'

CRS-2673: Attempting to stop 'ora.catalog.raadmin.acfs' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.catalog.raosbadmin.acfs' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.DELTA.dg' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.CATALOG.dg' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.cvu' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.chad' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.chad' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.qosmserver' on 'zeroserv01'

CRS-2677: Stop of 'ora.DELTA.dg' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.CATALOG.dg' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'zeroserv01'

CRS-2677: Stop of 'ora.cvu' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.asm' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv01'

CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.scan2.vip' on 'zeroserv01'

CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.scan3.vip' on 'zeroserv01'

CRS-2677: Stop of 'ora.catalog.raosbadmin.acfs' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01'

CRS-2677: Stop of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.catalog.raadmin.acfs' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01'

CRS-2677: Stop of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.proxy_advm' on 'zeroserv01'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.zeroserv01.vip' on 'zeroserv01'

CRS-2677: Stop of 'ora.scan3.vip' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.qosmserver' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.scan2.vip' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.zeroserv01.vip' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.proxy_advm' on 'zeroserv01' succeeded

CRS-2675: Stop of 'ora.chad' on 'zeroserv02' failed

CRS-2679: Attempting to clean 'ora.chad' on 'zeroserv02'

CRS-2675: Stop of 'ora.chad' on 'zeroserv01' failed

CRS-2679: Attempting to clean 'ora.chad' on 'zeroserv01'

CRS-2681: Clean of 'ora.chad' on 'zeroserv02' succeeded

CRS-2681: Clean of 'ora.chad' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.mgmtdb' on 'zeroserv01'

CRS-2677: Stop of 'ora.mgmtdb' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.MGMTLSNR' on 'zeroserv01'

CRS-2677: Stop of 'ora.MGMTLSNR' on 'zeroserv01' succeeded

CRS-2672: Attempting to start 'ora.MGMTLSNR' on 'zeroserv02'

CRS-2672: Attempting to start 'ora.qosmserver' on 'zeroserv02'

CRS-2672: Attempting to start 'ora.scan2.vip' on 'zeroserv02'

CRS-2672: Attempting to start 'ora.scan3.vip' on 'zeroserv02'

CRS-2672: Attempting to start 'ora.cvu' on 'zeroserv02'

CRS-2672: Attempting to start 'ora.zeroserv01.vip' on 'zeroserv02'

CRS-2676: Start of 'ora.cvu' on 'zeroserv02' succeeded

CRS-2676: Start of 'ora.scan2.vip' on 'zeroserv02' succeeded

CRS-2676: Start of 'ora.zeroserv01.vip' on 'zeroserv02' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv02'

CRS-2676: Start of 'ora.scan3.vip' on 'zeroserv02' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv02'

CRS-2676: Start of 'ora.MGMTLSNR' on 'zeroserv02' succeeded

CRS-2672: Attempting to start 'ora.mgmtdb' on 'zeroserv02'

CRS-2676: Start of 'ora.qosmserver' on 'zeroserv02' succeeded

CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'zeroserv02' succeeded

CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'zeroserv02' succeeded

CRS-2676: Start of 'ora.mgmtdb' on 'zeroserv02' succeeded

CRS-2672: Attempting to start 'ora.chad' on 'zeroserv02'

CRS-2676: Start of 'ora.chad' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.ons' on 'zeroserv01'

CRS-2677: Stop of 'ora.ons' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'zeroserv01'

CRS-2677: Stop of 'ora.net1.network' on 'zeroserv01' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zeroserv01' has completed

CRS-2677: Stop of 'ora.crsd' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.evmd' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.storage' on 'zeroserv01'

CRS-2677: Stop of 'ora.storage' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'zeroserv01'

CRS-2677: Stop of 'ora.ctssd' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.evmd' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.asm' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'zeroserv01'

CRS-2677: Stop of 'ora.cssd' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.diskmon' on 'zeroserv01'

CRS-2677: Stop of 'ora.diskmon' on 'zeroserv01' succeeded

[root@zeroserv01 ~]#

Stop/Start CRS

And if everything was successful, we can stop CRS. This is needed because HAIP is from init from CRS:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stop crs -f

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.gpnpd' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.crf' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zeroserv01'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'zeroserv01'

CRS-2677: Stop of 'ora.drivers.acfs' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.gpnpd' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.crf' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'zeroserv01'

CRS-2677: Stop of 'ora.mdnsd' on 'zeroserv01' succeeded

CRS-2677: Stop of 'ora.gipcd' on 'zeroserv01' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zeroserv01' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@zeroserv01 ~]#

After that we can start again CRS in this node:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@zeroserv01 ~]#

Check CRS Init

And after some time we can see if the HAIP not restarted (as expected):

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stat res -t -init

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

      1        ONLINE  ONLINE       zeroserv01               Started,STABLE

ora.cluster_interconnect.haip

      1        OFFLINE OFFLINE                               STABLE

ora.crf

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.crsd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.cssd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.cssdmonitor

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.ctssd

      1        ONLINE  ONLINE       zeroserv01               ACTIVE:0,STABLE

ora.diskmon

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.drivers.acfs

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.drivers.oka

      1        OFFLINE OFFLINE                               STABLE

ora.evmd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.gipcd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.gpnpd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.mdnsd

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.storage

      1        ONLINE  ONLINE       zeroserv01               STABLE

--------------------------------------------------------------------------------

[root@zeroserv01 ~]#

As you can see above, the HAIP not started during the init of CRS. If the HAIP still up, please check the topic 5.

Other nodes

After that, we can do the same in the other nodes.

[root@zeroserv02 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stop cluster -f

CRS-2673: Attempting to stop 'ora.crsd' on 'zeroserv02'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'zeroserv02'

CRS-2673: Attempting to stop 'ora.DELTA.dg' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.catalog.raadmin.acfs' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.chad' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.catalog.raosbadmin.acfs' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv02'

CRS-2677: Stop of 'ora.DELTA.dg' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.CATALOG.dg' on 'zeroserv02'

CRS-2677: Stop of 'ora.CATALOG.dg' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'zeroserv02'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.zeroserv02.vip' on 'zeroserv02'

CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.scan1.vip' on 'zeroserv02'

CRS-2677: Stop of 'ora.catalog.raadmin.acfs' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02'

CRS-2677: Stop of 'ora.CATALOG.RAADMIN.advm' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.catalog.raosbadmin.acfs' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02'

CRS-2677: Stop of 'ora.CATALOG.RAOSBADMIN.advm' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.proxy_advm' on 'zeroserv02'

CRS-2677: Stop of 'ora.asm' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv02'

CRS-2677: Stop of 'ora.zeroserv02.vip' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.scan1.vip' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.ASMNET2LSNR_ASM.lsnr' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.proxy_advm' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.chad' on 'zeroserv02' succeeded

CRS-2672: Attempting to start 'ora.scan1.vip' on 'zeroserv01'

CRS-2672: Attempting to start 'ora.zeroserv02.vip' on 'zeroserv01'

CRS-2676: Start of 'ora.zeroserv02.vip' on 'zeroserv01' succeeded

CRS-2676: Start of 'ora.scan1.vip' on 'zeroserv01' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv01'

CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'zeroserv01' succeeded

CRS-2673: Attempting to stop 'ora.ons' on 'zeroserv02'

CRS-2677: Stop of 'ora.ons' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'zeroserv02'

CRS-2677: Stop of 'ora.net1.network' on 'zeroserv02' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'zeroserv02' has completed

CRS-2677: Stop of 'ora.crsd' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.evmd' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.storage' on 'zeroserv02'

CRS-2677: Stop of 'ora.storage' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'zeroserv02'

CRS-2677: Stop of 'ora.ctssd' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.evmd' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.asm' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'zeroserv02'

CRS-2677: Stop of 'ora.cssd' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.diskmon' on 'zeroserv02'

CRS-2677: Stop of 'ora.diskmon' on 'zeroserv02' succeeded

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stop crs -f

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.crf' on 'zeroserv02'

CRS-2673: Attempting to stop 'ora.gpnpd' on 'zeroserv02'

CRS-2677: Stop of 'ora.drivers.acfs' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.crf' on 'zeroserv02' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'zeroserv02'

CRS-2677: Stop of 'ora.mdnsd' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.gpnpd' on 'zeroserv02' succeeded

CRS-2677: Stop of 'ora.gipcd' on 'zeroserv02' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zeroserv02' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@zeroserv02 ~]#

[root@zeroserv02 ~]#

[root@zeroserv02 ~]# /u01/app/12.2.0.1/grid/bin/crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@zeroserv02 ~]#

[root@zeroserv02 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stat res -t -init

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.cluster_interconnect.haip

      1        OFFLINE OFFLINE                               STABLE

ora.crf

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.crsd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.cssd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.cssdmonitor

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.ctssd

      1        ONLINE  ONLINE       zeroserv02               ACTIVE:0,STABLE

ora.diskmon

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.drivers.acfs

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.drivers.oka

      1        OFFLINE OFFLINE                               STABLE

ora.evmd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.gipcd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.gpnpd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.mdnsd

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.storage

      1        ONLINE  ONLINE       zeroserv02               STABLE

--------------------------------------------------------------------------------

[root@zeroserv02 ~]#

8 – Check if everything is UP

After doing the changes in both nodes we can see if everything is up and running:

[root@zeroserv01 ~]# /u01/app/12.2.0.1/grid/bin/crsctl stat res -t

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ob_dbfs

               OFFLINE OFFLINE      zeroserv01               STABLE

               OFFLINE OFFLINE      zeroserv02               STABLE

ora.ASMNET1LSNR_ASM.lsnr

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.ASMNET2LSNR_ASM.lsnr

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.CATALOG.RAADMIN.advm

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.CATALOG.RAOSBADMIN.advm

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.CATALOG.dg

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.DELTA.dg

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.LISTENER.lsnr

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.catalog.raadmin.acfs

               ONLINE  ONLINE       zeroserv01               mounted on /raacfs/r

                                                             aadmin,STABLE

               ONLINE  ONLINE       zeroserv02               mounted on /raacfs/r

                                                             aadmin,STABLE

ora.catalog.raosbadmin.acfs

               ONLINE  ONLINE       zeroserv01               mounted on /osbcat,S

                                                             TABLE

               ONLINE  ONLINE       zeroserv02               mounted on /osbcat,S

                                                             TABLE

ora.chad

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.net1.network

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.ons

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

ora.proxy_advm

               ONLINE  ONLINE       zeroserv01               STABLE

               ONLINE  ONLINE       zeroserv02               STABLE

rep_dbfs

               OFFLINE OFFLINE      zeroserv01               STABLE

               OFFLINE OFFLINE      zeroserv02               STABLE

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.LISTENER_SCAN2.lsnr

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.LISTENER_SCAN3.lsnr

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.MGMTLSNR

      1        ONLINE  ONLINE       zeroserv01               192.168.10.1 192.168

                                                             .10.2,STABLE

ora.asm

      1        ONLINE  ONLINE       zeroserv01               Started,STABLE

      2        ONLINE  ONLINE       zeroserv02               Started,STABLE

ora.cvu

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.mgmtdb

      1        ONLINE  ONLINE       zeroserv01               Open,STABLE

ora.qosmserver

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.scan1.vip

      1        ONLINE  ONLINE       zeroserv02               STABLE

ora.scan2.vip

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.scan3.vip

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.zdlras.db

      1        OFFLINE OFFLINE                               STABLE

      2        OFFLINE OFFLINE                               Instance Shutdown,ST

                                                             ABLE

ora.zeroserv01.vip

      1        ONLINE  ONLINE       zeroserv01               STABLE

ora.zeroserv02.vip

      1        ONLINE  ONLINE       zeroserv02               STABLE

--------------------------------------------------------------------------------

[root@zeroserv01 ~]#

As you can see, everything is up. ASM is online in both nodes, as the listeners for ASM too.

9 – Start Databases

To finish, we need to restart all databases that were stopped at topic 1.

[oracle@zeroserv01 ~]$ srvctl start database -d zdlras

[oracle@zeroserv01 ~]$

Clean up

After you complete the GI upgrade for 19c, you can remove the ASM Proxy init file (from $GI_HOME/dbs) that was created before. It is not needed anymore because the upgrade removes completely the HAIP from CRS because it is not in use.

Conclusion

As told before, HAIP is not supported by the Exadata environment. If you check the definition for HAIP and the hardware design for the Exadata/Engineering System, it is completely redundant. The InfiniBand network and the dual-channel/ports have the same effect (even better)

But unfortunately, HAIP can be up for several reasons (like ZDLRA) and because of the way that ASM Proxy starts and pick up the interconnect IP, HAIP can be selected. Remove HAIP it is a little tricky because of the ASM dependency, we need to set some specific parameters.

As you saw before, there are several steps. But you can use and adapt them to your environment. If it is Exadata/Engineering System (like ZDLRA) they can be used almost as-is (just check the IPs). If it is not, read and adapt.

References:

Oracle E-Business Suite 12i Architecture (Part 2)

Category: E-Business Suite Author: Andre Luiz Dutra Ontalba (Board Member) Date: 5 years ago Comments: 0

Oracle E-Business Suite 12i Architecture (Part 2)

Shared Application System

A traditional multi-node installation of EBS 11i required each application layer to maintain its own file system, which consists of the APPL_TOP file system (APPL_TOP, COMMON_TOP, and some related directories) and application tier technology file stack layer (8.0 .6 ORACLE_HOME and IAS ORACLE_HOME).

Subsequently, this was modified to allow APPL_TOP to be shared between different machines, and subsequently to allow sharing of the entire application layer file system.

Continuing this quick installation strategy, for version 12 it creates a system that shares not only APPL_TOP and the COMMON_TOP file systems, but also the application tier technology stack layer.

Rapid Install sets this setting as the default for nodes running on the same operating system.

These files form the application layer of the file system, and can be shared between application nodes in multiple layers (as long as they are running the same operating system).

Note: Shared file system configuration is not currently supported on application tier nodes servers running Windows.

With a shared application layer file system, all files in this application layer are installed on a single shared disk that is mounted from each application layer node.

Any application layer node can be used to provide standard services, such as a Forms, Web Pages or Concurrent server.

Shared application layer – Example

As well as reducing required disk space, there are several other benefits of setting up shared application levels:

More administrative tasks, patching and maintenance need to be done only once, instead of a single layer application node.

Changes made to the shared file system are immediately accessible on all nodes in the application layer.

Distributes task processing to run in parallel on multiple nodes (Distributed AD).

Reduces general disk requirements.

Add application nodes more easily.

Sharing the File System application between instances.

Capabilities to share the tiered file system application were further extended in version 12.0.4, which introduced the option to share an installation of Oracle E-Business Release 12 with another instance of the database.

An application file system layer installed and configured in this way can be used to access two (or more) database instances.

The restrictions to this are:

All database instances must have the same patches.

Only the application can be shared, the database cannot be shared.

Nota: For more information on features, options and implementation steps, see document 384248.1, Sharing the Application Tier File System in Oracle E-Business Suite Release 12.

Environment Setting

Rapid Install creates environment files to configure the Oracle database, Oracle’s technology suite, Oracle HTTP Server, and Oracle E-Business Suite environments.

The location of these environment files is shown in the following table:

Filename	Location	Path	Environment
<CONTEXT_NAME>.env or CONTEXT_NAME>.cmd	10.2.0.2 ORACLE_HOME	db/tech_st/10.2.0	Oracle Server Enterprise Edition
<CONTEXT_NAME>.env or <CONTEXT_NAME>.cmd	OracleAS 10.1.2 ORACLE_HOME	inst/apps/<context>/ora/10.1.2	Oracle tools technology stack
<CONTEXT_NAME>.env or <CONTEXT_NAME>.cmd	OracleAS 10.1.3 ORACLE_HOME	inst/apps/<context>/ora/10.1.3	Java technology stack
<CONTEXT_NAME>.env or <CONTEXT_NAME>.cmd	APPL_TOP	apps/apps_st/appl	Applications
APPS<CONTEXT_NAME>.env or APPS<CONTEXT_NAME>.cmd	APPL_TOP	apps/apps_st/appl	Consolidated setup file

On UNIX, Oracle E-Business Suite includes a consolidated file called APPS <CONTEXT_NAME> .Env, which establishes both Oracle E-Business Suite and Oracle technology stack environments.

When you install Oracle E-Business Suite, Rapid Install creates this script in the APPL_TOP directory. Many of the parameters are specified during the installation process.

On Windows, the consolidated equivalent environment file is called% APPL_TOP% \ envshell <CONTEXT_NAME> .cmd.

When running it creates a command window with the necessary environment settings for Oracle E-Business Suite. All subsequent operations on APPL_TOP (for example, running adadmin or adpatch) must be performed from this window.

The following table lists the key environment settings in APPS <CONTEXT_NAME> .env.

Parameter	Description
APPLFENV	The name of the environment file, <CONTEXT_NAME>. env. If you rename the environment file, this parameter must be updated.
PLATFORM	The operating system in use. The value (for example, LINUX) must match the value in the APPL_TOP/admin/ adpltfrm.txt file.
APPL_TOP	The main directory for this Oracle E-Business Suite installation.
ADMIN_SCRIPTS_HOME	$ INST_TOP directory that identifies the location of scripts, such as adautocfg.sh, adpreclone.sh, adstrtal.sh and adstpall.sh.
FNDNAM	The name of the ORACLE scheme to which the System Administration responsibility connects. The default is APPS.
GWYUID	The public ORACLE username and password that gives access to the initial Oracle E-Business Suite sign-on form. The default is APPLSYSPUB/PUB.
FND_TOP	The path to the Application Library Object directory. For example, apps /apps_st/appl/fnd/12.0.0.
AU_TOP	The path to the Applications Utilities directory. For example, apps /apps_st/appl/au/12.0.0.
<PROD>_TOP	The path to a product’s top directory. There is an entry for each Oracle E-Business Suite product.
PATH	Sets the directory search path, for example, to FND_TOP and AD_TOP.
APPLDCP	Specifies whether the distributed Concurrent Process is in use. If it is the same it distributes the load in other Concurrent Process in other nodes.
APPCPNAM	Indicates whether the format of the Concurrent Manager log and the output files following the 8.3 file name convention (maximum 8 characters to the left of the dot and 3 to the right, for example, alogfile.log). If this parameter is set to “REQID” (required), Concurrent Manager uses filenames that meet 8.3 naming requirements.
APPLCSF	Identifies the top level directory for Concurrent Manager log and output files. They are consolidated into a single directory for all products. For example, /inst/apps/<context>/logs/appl/conc.
APPLLOG	The subdirectory for Concurrent Manager concurrent log files. The default is extension .log
APPLOUT	The subdirectory for Concurrent Manager’s simultaneous output files. The default is .out
APPLTMP	Identifies the directory of temporary Oracle E-Business Suite files. The default is $ INST_TOP/tmp on UNIX.
APPLPTMP	Identifies the directory for the output temporary PL/SQL files. The possible directory options must be listed in the utl_file_dir parameter init.ora.
INST_TOP	Identifies the top level directory for this instance. For example, inst/apps/<context>. Introduced with Release 12.
NLS_LANG	The language, territory, and character set installed in the database. The default for a new installation is “AMERICAN_AMERICA.US7ASCII”.
NLS_DATE_FORMAT	The National Language Support date format. The default is “DD-MON-RR”, for example, 14-July-19.
NLS_NUMERIC_CHARACTERS	The National Language Support numeric separators. The default is “.” (Semicolon).

Most temporary files are written to the location specified by the APPLTMP environment configuration, which is defined in the Rapid Install.

Oracle E-Business Suite products also create temporary PL/SQL output files used in simultaneous processing. These files are written to a location on the database server node specified by the APPLPTMP environment configuration.

The APPLPTMP directory must be the same directory specified by the utl_file_dir parameter in your database initialization file.

Rapid Install sets both APPLPTMP and the utl_file_dir parameter to the same default directory.

Some Oracle Utilities E-Business Suite use your temporary default operating system directory even if you configure the environment settings listed in the previous paragraph. You must therefore ensure that there is sufficient free disk space in this directory, as well as those indicated by APPLTMP and APPLPTMP.

In a multi-node system, the directory defined by APPLPTMP does not need to exist on application layer servers.

Nota: Temporary files placed in the utl_file_dir directory can be protected from unauthorized access, ensuring that this directory has read and write access to the Oracle database account only.

Other environments files

Several other key environment files are used in an Oracle E-Business Suite system.

O arquivo adovars.env

The adovars.env file, located at $ APPL_TOP/admin, specifies the location of several files, such as Java files, HTML files and the JRE (Java Runtime Environment) files.

It is called from the main application environment file, <CONTEXT_NAME>. Env. The adovars.env file includes comments on the purpose and recommended configuration of each variable. In a 12 release environment, adovars.env is maintained by AutoConfig, and should not be edited manually.

The adovars.env file includes the following parameters:

Parameter	Description
AF_JLIB	Indicates the directory to which all Java archive files are copied. For example, apps /apps_st/COMn/java/lib. Introduced with Release 12.
JAVA_BASE	Indicates the top level of the Java directory. For example, apps/apps_st/COMn/java. Introduced with Release 12.
JAVA_TOP	Indicates the directory to which all Java class files are copied. For example, apps/apps_st/COMn/java/classes. Definition changed with version 12.
OA_JAVA	Indicates the directory to which all Java archive files are copied. For example, apps/apps_st/COMn/java/classes.
OA_JRE_TOP	Indicates the location where the JRE is installed. For example, /local/java/jdk1.5.0_08.
OAH_TOP	Sets the location to which HTML files are copied. For example, apps /apps_st/COMn/webapps/oacore.
OAD_TOP	Defines the locations to which context-sensitive documentation files are copied. For example, apps/apps_st/COMn.
LD_LIBRARY_PATH	Path used on many UNIX platforms to list the directories being scanned for dynamic library files needed at run time.
CLASSPATH	Lists scanned directories and zip files for Java class files needed at run time.

The adconfig.txt file

AD utility programs perform a variety of database and file management tasks. These utilities need to know information about the right configuration to run successfully. This configuration information is specified when Oracle E-Business Suite is installed and subsequently stored in the adconfig.txt file in <APPL_TOP> / admin. Once created, this file is used by other Oracle E-Business Suite utilities.

Nota: adconfig.txt is created with the APPL_TOP file system, and shows the layers that have been configured on a particular node. It is distinct from the config.txt file configured by Rapid Install.

The fndenv.env file

This file defines environment variables used by the application’s object library. For example, it defines APPLBIN as the name of the subdirectory where executable product programs and shell scripts are stored (bin). This file must not be modified: the default values are applicable for all customers. The file is located in the FND_TOP directory.

The devenv.env file

This file defines the variables that allow you to link third-party software and your own custom applications developed with Oracle E-Business Suite.

In version 12, this script is located in $ FND_TOP / usrxit, and is automatically called by fndenv.env. This allows you to compile and link custom forms for Oracle users to outbound and competing programs with Oracle E-Business Suite.

In the next articles, we will continue to understand the structure of the E-Business Suite and guide best practices for installing the product efficiently.

References

Oracle E-Business Suite Release 12 Technology Stack Documentation Roadmap [ID 380482.1]

I hope this help you !!

André Ontalba

Disclaimer: “The postings on this site are my own and don’t necessarily represent may actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications was removed to allow reach generic audience and to be useful.”

« 1 … 7 8 9 10 11 … 32 »

DB_UNIQUE_NAME, PDB, and Data Guard

Unplug and Plug

The environment below is a simple 19c single instance database. There, exists 3 PDB’s and you can see the db_unique_name for this CDB:

And we can connect in the PDBMOVE and change the parameters. First, as usual, normal set without any special parameter:

But, we can specify the db_unique_name for alter system. ALTER SYSTEM… DB_UNIQUE_NAME = ‘VALUE’. Here you can see that I used the same that as defined for root level.

Detail #1 here, if you specify the DB_UNIQUE_NAME the scope can be JUST spfile. Otherwise you will receive error.

In this example I set two parameters for PDB level and you can see at PDB_SPFILE$ inside of root cdb that values was set:

And after the reload of the database the values are set:

UNPLUG

But if I need to unplug the PDB, the values will be exported too, right? Yes, but no as you imagine.

So, doing a simple unplug to XML:

And checking the file for these parameters at generated XML:

As you can see here, the parameters was exported, but the parameter changed to “*”. The definition to specific db_unique_name was cleared.

And if we try to plug again the same PDB, we can see that parameter was loaded as “*”:

DATAGUARD

It is not the case that this is wrong, maybe you need to set some parameters for one server/side in specific. Maybe because of less memory, less CPU power, or whatever the reason.

In this example, I have gold19c as primary, and golds19c as standby. And I have the GOLD19P as used PDB here:

And to set the parameters I used the same than before, with and without db_unique_name:

And as you can see, the parameter was stored.

But if I made the switchover:

You can see that now, the value for the parameter cursor_sharing now have the default value because the db_unique_name does not hit what was defined:

But you can see that the parameter is defined at PDB_SPFILE$:

Others points

The db_unique_name option does not check what you specify, so, take care to set the correct value. As you can see below I defined it as SIMON, and was accepted and saved:

And to reset, the process is similar. As you see below, you can specify the db_unique_name to delete specific. But if you specify nothing, the reset will be done for all.

Conclusion

Sometimes a simple definition can lead to some strange behaviors. Think that your application is running ok and after a switchover change completely the way how it works (cursor_sharing as above). And until you dig the solution can take time.

Db_unique_name for set parameter needs to be used carefully to avoid these cases. It is not the case that you never need to use it, sometimes this definition needs to be used. You can use it to prepare everything BEFORE the switchover to avoid some error or to tune the database since the beginning.

References

New Feature - Per-Second Billing for Compute and Autonomous Database.

Oracle this month announced a new form of billing for Compute Instance and Autonomous Database with that the Oracle Cloud has a stronger platform for legacy applications moving to cloud and a new family of cloud native applications that rely on microservices and dynamic scaling.

With this billing model, usage of Compute and Autonomous Database is billed per-second. All prices continue to be quoted on an hourly basis.

Here are some details about this billing model:

All virtual machine (VM) Compute instances, including those with graphical processing unit (GPU) chips, are now billed per-second with a one-minute minimum.

All bare metal Compute instances, including those with GPU chips, are now billed per-second with a one-hour minimum.

Autonomous Data Warehouse and Autonomous Transaction Processing usage is now billed per-second with a one-minute minimum.

Windows OS images are now billed per-second with a one-minute minimum.

Microsoft SQL Server images available in the Oracle Cloud Marketplace are now billed per-second with a 744-hour (one month) minimum.

The workloads that will see the biggest impact from this change are those that stop and start frequently, and those that run for short durations. Here are some examples of significant benefits:

Continuous integration/continuous development(CI/CD)

Big boost to Cloud Native applications – Microservices & Dynamic Scaling

I believe that with this, Oracle increasingly invests and provides more resources and different ways for its customers to be able to plan and prepare their environments for the cloud.

Reference: https://docs.cloud.oracle.com/

I hope this short article has helped.

Andre Ontalba

Exadata and ZDLRA, Disable HAIP

The HAIP is not needed for Exadata because by architecture the InfiniBand network already defines (per server) two IP’s to avoid the single point of failure. So, it is not needed to create an additional layer (HAIP and virtual IP), that does the same that already exists by network design.

*Image removed from Oracle Presentation: Oracle Real Application Clusters 19c- Best Practices and Internals- EMEA Tour Oct 2019

Environment

In this scenario, I am removing HAIP from ZDLRA GI. The HAIP has configured automatically and we are using ACFS.

The focus is to disable HAIP and remove it from CRS. Another detail is that we need to remove the dependencies between the CRS resources (like ASM that need HAIP) too.

GI Upgrade

During the GI upgrade from 12.2 to 19c we need to run the runcluvfy and it will detect if HAIP is enabled:

Remove HAIP

The steps need to be executed in order to avoid errors during the procedure. Unfortunately, it is needed to have one maintenance window due to the unavailability of the services. The running databases need to be shutdown due to the CRS restart by an example.

Bellow, the steps are summarized and explained to be followed. Please be aware of the IP’s involved (they will be different in your case).

1 – Shutdown services

So, this first point is to shutdown all databases running in this cluster:

2- Check HAIP, ASM Proxy, and ACFS

It is needed to check if HAIP is enabled. Another detail is check for ASM proxy, and ACFS too:

3 – Check New IP

The next step is to check the new IP that will be used. Since ASM does not use HAIP, we can (at Exadata) pickup the same used IP:

In this case, the IP that will substitute HAIP IP is 192.168.10.1/192.168.10.2. And, as you can see above, they are from the interconnect network.

4 – Fixing IP for ASM Proxy

The next step is to fix the IP that is used by the ASM Proxy instance. But default, it pickup the HAIP IP during the startup. The idea here is to force the same IP than the ASM instance for cluster_interconnect parameter.

Connecting at APX instance

Connect at APX instance, check the parameters (and see that are without values). After that we create one init file from memory to have the backup:

Changing the IP

For ASM Proxy instance we need to set the IP for cluster_interconnect parameter with the same IP used by ASM instance.

Restart the ASM Proxy

After creating the init, we restart the ASM Proxy in this node.

Look that instance restarted correctly and it is up.And we can check the parameter if it is OK with the IP:

If the instance not startup (or the IP’s are wrong) you need to check the init file that you created in $GIHOME/dbs folder.

Restart ACFS

Since we restarted the ASM Proxy instance, the ACFS mountpoints went down and need to be up again:

I started up using the CRS resource for that.

Other Nodes

After it is Ok for node 1, we can do it for other nodes. Here, since I made for ZDLRA, I have just the second node to fix ASM Proxy. Be careful with the IP’s used, they are specific for this node and remember to use the correct init file.

5 – ASM Dependency

Since the ASM since the CRS depends on HAIP we need to remove this dependency. Here we have a tricky part. It is needed to completely change the dependencies for ASM, setting it to CRS/CSS directly.

Look that instance restarted correctly and it is up.

And we can check the parameter if it is OK with the IP: