Oracle APPS DBA

September 1, 2017 Troubleshooting AD Controller

APPS DBAErrors/Workarounds-ApplicationsOracle APPS DBA

Fixing a “Failed” worker

Whenever a job fails for the 1st time, the job is deferred at the end of the phase and another job is assigned to that worker.

What If the Job Fails 2nd Time?

If the run time of the job was < 10 min => the job is deferred at the end of the phase and another job is assigned to that Worker.
if the run time of the job was >= 10 min => then the job status will be shown as “Failed”.

What If the job fails 3nd time?

If the Job fail third time the job status will be shown as “Failed”.

Review the Worker log information into

$APPL_TOP/admin/<SID>/log/adworkNNN.log

Example: adwork001.log name for the worker number 1.

After fixing error, start (if is not already started) AD Controller and to use the option 2 “Tell worker to restart a failed job”.

When prompted we have to specify the worker which must be restarted.

If all the workers are failed, we can type all to restart all Workers.

Restarting a Failed Patch Process

During a patch process (or adadmin process) if a job fails and cannot be restarted, then the patch must be restarted.

Here are the steps for doing this:

Tell worker to quit (for all workers) => to manually shutdown/ quit the workers
Tell manager that a worker failed its job
Tell manager that a worker acknowledges quit => the manager will stop, the AutoPatch will restart the patch

PLEASE NOTE: When the patch will restart all the information in the database about this session must be accurate.

How to determine if a process is Hanging or not

We can check the log file to see if some information is added or not to the log
We can determine if the worker process is consuming CPU by issuing below

$ ps -eo pcpu,pid,user,args | grep workerid

3.We check if there are any child processes, which are consuming CPU by issuing following command:

$ ps -eo pcpu,pid,ppid,user,args | grep <Parent Process> | grep -v grep

Restarting a Hanging Worker Process

1. kill at the OS level the processes associated with the Hanging Worker

$ kill -9 ProcesssNumber

2. fix the problem

3. Restart the worker (or the job)

Restart an AD utility after a Node Crash

a. Start AD Controller

b. Choose “4. Tell manager that a worker failed its job“

c. Choose “2. Tell worker to restart a failed job“

d. Restart the AD utility that was running when the node

Shutting down the Manager

1. Start AD Controller

2. Choose “3. Tell worker to quit“

3. Verify that no worker processes are running

September 1, 2017 RapidWiz log files for Oracle Applications Release 12

APPS DBAErrors/Workarounds-ApplicationsInstallations/Configurations-ApplicationsOracle APPS DBA

RapidWiz log files for Oracle Applications Release 12

During installation of Release 12 using RapidWiz, the following log files are created.

In case of any errors during the installation, we can use these log file locations for troubleshooting.

Before the installation starts:
The Rapidwiz Configuration File is saved in three locations,these are follows.

$TMP//conf_.txt
$INST_TOP/conf_.txt
/appsutil/conf_.txt

Database Tier log files are located as follows:

$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/.logRDBMS
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/ApplyDBTechStack_.log
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/ohclone.log
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/make_.log
ORACLE_HOME/appsutil/log/$CONTEXT_NAME/installdbf.log$ORACLE_HOME/appsutil/log/
$CONTEXT_NAME/adcrdb_.log
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/ApplyDatabase_.log
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME//adconfig.log
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME//NetServiceHandler.log

Applications Tier(s) log files are located as follows:

$INST_TOP/logs/.log
$APPL_TOP/admin/$CONTEXT_NAME/log/ApplyAppsTechStack.log
$INST_TOP/logs/ora/10.1.2/install/make_.log
$INST_TOP/logs/ora/10.1.3/install/make_.log
$INST_TOP/admin/log/ApplyAppsTechStack.log
$INST_TOP/admin/log/ohclone.log
$APPL_TOP/admin/$CONTEXT_NAME/log/installAppl.log
$APPL_TOP/admin/$CONTEXT_NAME/log/ApplyAppltop_.log
$APPL_TOP/admin/$CONTEXT_NAME/log/adconfig.log
$APPL_TOP/admin/$CONTEXT_NAME/log/NetServiceHandler.log

Inventory Registration log files are as follows:

/logs/cloneActions.log
/logs/oraInstall.log
/logs/silentInstall.log

September 1, 2017 Oracle AD Controller

APPS DBAInstallations/Configurations-ApplicationsOracle APPS DBA

Oracle AD Controller

ADCTRL is used to manage worker processes that perform a pivotal role in the function of batch processing for patching and many other Applications activities.
ADCTRL is located under $AD_TOP/bin directory with Oracle E-Business Suite.

Procedure to Run AD controller

Step 1 : Login as Applications Tier user & run the environment file.

$cd /prod/ebs/apps/prodappl

$ . ./APPSORA.env

Step 2 : Run the following AD controller command.

$ adctrl

You will be prompted for the location of $APPL_TOP, Password of APPLSYS and APPS. After providing the above information the AD controller menu will appear as shown below.

AD Controller Menu

Show worker status
Tell worker to restart a failed job
Tell worker to quit
Tell manager that a worker failed its job
Tell manager that a worker acknowledges quit
Restart a worker on the current machine
Exit

Checking Status of the Worker

choose the first option “Show worker status“.

Note: If there is no session used by the workers, then the following message will appear:

Error: The FND_INSTALL_PROCESSES table does not exist.

This above given table is used for communication with the worker processes, and if it does not exist, it means that the workers are not running, and the ad utility has not started them yet.

Meaning of Worker Status

STATUS	Description
Waiting	The worker is idle.
Assigned	A job was assigned by the manager to a worker but the worker didn’t start the job.
Running	The worker is running a job.
Failed	The job failed due to an error.
Fixed, Restart	When a jobs restart after the error has been fixed (during this time the worker run the failed job).
Restarted	After the error has been fixed, the worker will have the status “Fixed, Restart” and after that “Restarted”. (The status will not change to “Running”)
Completed	The job was completed and the manager did not yet assigned another job to that worker.

Database Processing Phases Concept

Whenever a Database Patch operation will runs, the tasks are divided into functions. This is done by Oracle when the patch is created.
Suppose a patch will create 2 tables and 2 sequences. In this case the patch driver contains 2 phases, one for tables creation and one for sequences creation.
Because the sequences could be created in the same time, this will be done in parallel by using more workers.

Examples of Database Processing Phases:

seq = create sequence

tab = create tables, synonyms, grants privileges on tables pls = create package specification

plb = create package body

vw = create views

September 1, 2017 ORACLE APPLICATIONS R12 ARCHITECTURE