Saturday, March 5, 2016

ADOP phase abort failed with ORA-20001: Error while calling ad_zd_adop.abortORA-01555: snapshot too old

ADOP pahse=abort is failed

When I was running adop phase=abort, stop with the below error:




Log: /u02/app/oracle/PROD/fs_ne/EBSapps/log/adop/53/adop_20160305_072738.log
[START 2016/03/05 07:27:40] Arguments passed to adop
    phase=abort
[END   2016/03/05 07:27:40] Arguments passed to adop
[STATEMENT] Verifying parameters
[STATEMENT] Sourcing Run edition environment /u02/app/oracle/PROD/fs2/EBSapps/appl/APPSPROD_XXXXX15.env

[EVENT]     Validating system setup.
    [STATEMENT] Checking for required environment
    [PROCEDURE] Running: adpawc -nodisp
    [STATEMENT]  Using 8 workers (Default: 8, Recommended maximum limit: 324)
    [PROCEDURE] [START 2016/03/05 07:28:17] Identifying Admin server node
[EVENT]     Node registry is valid.
    [PROCEDURE] [START 2016/03/05 07:28:17] Performing database sanity checks
[PROCEDURE] [END   2016/03/05 07:28:17] Performing database sanity checkss
[PROCEDURE] [START 2016/03/05 07:28:18] Acquiring lock on sessions table
[STATEMENT] Locking ad_adop_sessions table for XXXXX with wait interval of 60 seconds and number of tries 2

[PROCEDURE] [END   2016/03/05 07:28:18] Acquiring lock on sessions table

[EVENT]     Checking for existing adop sessions.
    [PROCEDURE] [START 2016/03/05 07:28:18] Checking for any pending sessions
    [EVENT]     Checking for pending hotpatch session.
        [PROCEDURE] [START 2016/03/05 07:28:18] Checking active hotpatch session
 [STATEMENT] value of incomplete_session: "0"
        [PROCEDURE] [END   2016/03/05 07:28:18] Checking active hotpatch session
    [EVENT]     Checking for pending cleanup session.
        [PROCEDURE] [START 2016/03/05 07:28:18] Checking for active cleanup session
        [PROCEDURE] [END   2016/03/05 07:28:18] Checking for active cleanup session
    [STATEMENT] Master Session ID: "53"
    Continuing with existing session [Session ID: 53].
    [STATEMENT] Master Node: "XXXXXX"
    [STATEMENT] Check for incomplete patching cycle with session ID 53 on hostname kwshqapp15
    [STATEMENT] SQL output: "0"
    [STATEMENT] Open ADOP Session ID: "53"
    [STATEMENT] Using session ID from existing incomplete patching cycle
    [PROCEDURE] [END   2016/03/05 07:28:18] Checking for any pending sessions
[PROCEDURE] [START 2016/03/05 07:28:18] Unlocking sessions table
[STATEMENT] Unlocking ad_adop_sessions table for XXXXXX with wait interval of 60 seconds and number of tries 2

[PROCEDURE] [END   2016/03/05 07:28:18] Unlocking sessions table
[PROCEDURE] [START 2016/03/05 07:28:18] Running adzdoptl.pl
===========================================================================
ADOP (C.Delta.7)
Session ID: 53
Node: kwshqapp15
Phase: abort
===========================================================================
[STATEMENT] Inside createPatchCtxFile().
[STATEMENT] Patch file system context file exists
[PROCEDURE] [START 2016/03/05 07:28:19] Running abort phase
[PROCEDURE] [START 2016/03/05 07:28:19] Determining abort status
[STATEMENT] abort_status for session ID - 53 : N.
[PROCEDURE] [END   2016/03/05 07:28:19] Determining abort status
[STATEMENT] Status of abort phase: N
[PROCEDURE] [START 2016/03/05 07:28:19] Updating adop session status
[PROCEDURE] [START 2016/03/05 07:28:19] Updating session process ID
[PROCEDURE] [END   2016/03/05 07:28:19] Updating session process ID
[STATEMENT] SQL statement : "  update ad_adop_sessions
  set status='R'
   where adop_session_id = 53 and appltop_id = 195 and node_name='XXXXXX'
"
[PROCEDURE] [END   2016/03/05 07:28:19] Updating adop session status
[PROCEDURE] [START 2016/03/05 07:28:19] Updating session timestamps
[STATEMENT] ABORT Phase START TIME: 05-03-2016 07:28:20
[PROCEDURE] [END   2016/03/05 07:28:20] Updating session timestamps
[PROCEDURE] [START 2016/03/05 07:28:20] Updating session status
[STATEMENT] Updating abort_status=R for session ID : 53
[PROCEDURE] [END   2016/03/05 07:28:20] Updating session status
[PROCEDURE] [START 2016/03/05 07:28:20] Performing abort phase
[STATEMENT] Inside uploadCtxFile().
[STATEMENT] Running getJavaCmd procedure.

[STATEMENT] Running getClasspath procedure.
[STATEMENT] Upload of context file successful

[STATEMENT] LOG FILE: /u02/app/oracle/PROD/fs_ne/EBSapps/log/adop/53/abort_20160305_072738/PROD_kwshqapp15/CtxSynchronizer_uploadCtxFile.log.

[EVENT]     Running abort phase.
    [EVENT]     Log: @ADZDSHOWLOG.sql "2016/03/05 07:28:24"
    [ERROR]     Failed to execute sql statement:
        alter session set events='10624 trace name context forever, level 1';
        declare
        begin
          ad_zd_adop.abort(x_mode=>'ADOP',x_session_id=>53);
        exception
          when others then
            update ad_adop_sessions set status='F'
            where adop_session_id=53;
            commit;
            raise_application_error(-20001,'Error while calling ad_zd_adop.abort' || sqlerrm);
        end;

    [ERROR]     SQLPLUS error: buffer=
SQL*Plus: Release 10.1.0.5.0 - Production on Sat Mar 5 07:28:26 2016
Copyright (c) 1982, 2005, Oracle.  All rights reserved.
SQL> SQL> Connected.
SQL>    declare
*
ERROR at line 1:
ORA-20001: Error while calling ad_zd_adop.abortORA-01555: snapshot too old:
rollback segment number 11 with name "_SYSSMU11_1381604294$" too small
ORA-06512: at line 9
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

    [UNEXPECTED]Error occurred performing abort phase database actions
    [UNEXPECTED]Abort phase has failed.
    [PROCEDURE] [START 2016/03/05 07:57:04] Updating session timestamps
    [STATEMENT] ABORT Phase END TIME: 05-03-2016 07:57:04
    [PROCEDURE] [END   2016/03/05 07:57:04] Updating session timestamps
    [PROCEDURE] [START 2016/03/05 07:57:04] Unlocking sessions table
    [STATEMENT] Unlocking ad_adop_sessions table for XXXXX with wait interval of 60 seconds and number of tries 2

    [PROCEDURE] [END   2016/03/05 07:57:04] Unlocking sessions table
    [PROCEDURE] [START 2016/03/05 07:57:04] Updating adop session status
    [STATEMENT] SQL statement : "  update ad_adop_sessions
  set status='F'
   where adop_session_id = 53 and appltop_id = 195 and node_name='XXXXX'
adop exiting with status = 1 (Fail)

Too overcome this issue.I have added additional datafile to the undotablespace using below:

ALTER TABLESPACE "APPS_UNDOTS2" ADD DATAFILE '/u01/app/oracle/PROD/data/apps_undo05.dbf' SIZE 2000M REUSE AUTOEXTEND OFF

No comments: