Quantcast
Channel: Remote Oracle Database Support
Viewing all 155 articles
Browse latest View live

BR0123E Format error found in line at location arch_last_get-10

$
0
0
Error:

brarchive -u / -c -d util_file -r /oracle/PRD/102_64/dbs/initPRD-logs.utl -s
BR0002I BRARCHIVE 7.00 (40)
BR0006I Start of offline redo log processing: aemjygtf.sve 2013-10-26 14.42.59
BR0484I BRARCHIVE log file: /oracle/PRD/saparch/aemjygtf.sve
BR0477I Oracle pfile /oracle/PRD/102_64/dbs/initPRD.ora created from spfile /oracle/PRD/102_64/dbs/spfilePRD.ora
BR0123E Format error found in line 1830934 of /oracle/PRD/saparch/archPRD.log at location arch_last_get-10
BR0121E Processing of log file /oracle/PRD/saparch/archPRD.log failed
BR0016I 0 offline redo log files processed, total size 0.000 MB
BR0007I End of offline redo log processing: aemjygtf.sve 2013-10-26 14.43.04
BR0280I BRARCHIVE time stamp: 2013-10-26 14.43.04
BR0005I BRARCHIVE terminated with errors


Cause:

1. Archive destination has reached 100% & DBA has removed or zipped the archive files causing mismatch/unavailability of archive sequence for brarchive backup

2. /oracle/PRD/saparch/archPRD.log file is corrupted 

3.  /oracle/PRD/saparch/archPRD.log file has got unwanted characters due to improper editing 

Solution:

Step 1. create a backup copy of arch<SID>.log 

cp /oracle/PRD/saparch/archPRD.log /oracle/PRD/saparch/archPRD.log_bkp

Step 2. check the line number mentioned in the error , go to archPRD.log using vi :1830934 & take appropriate action. Possibly it may have blank line or unwanted characters.

BR0123E Format error found in line 1830934 of /oracle/PRD/saparch/archPRD.log at location arch_last_get-10

Try executing backup again , if still same error then go to step 3

Step 3. It seems archive sequence missing issue , append available archive sequence at below highlighted line at the end of the file to direct brarchive to start backup sequence post 33999. Make sure it is correctly formatted as per similar looking old lines 

#
#* PRD  util_file  aemjntmr svd  2013-10-24 11.24.21  2013-10-24 11.29.04  0  ...........    339316   339376        0        0  ------- 7.00 (40)  @0759153586
#
#* PRD  util_file  aemjozlm svd  2013-10-24 17.24.22  2013-10-24 17.24.32  3  ...........         0        0        0        0  ------- 7.00 (40)  @0759153586

#ARCHIVE.. 339999  /oracle/PRD/oraarch/PRDarch1_339999_759153586.arc  2013-10-19 04.57.39  30245888     7994205640697  1
#SAVED.... aelfptms svd  *VXF1368919944    2013-10-19 05.04.12 ........... ............
#COPIED... ........ ...  ................. .......... ........ ........... ............
#DELETED.. aelfptms svd  2013-10-19 05.04.12#
#* PRD  util_file  aemjyhks sve  2013-10-26 14.50.34  2013-10-26 14.50.36  3  ...........    339998   339999        0        0  ------- 7.00 (40)  @0759153586
#



Try executing backup again , it will show messages like
"BR0017W Offline redo log file '' not found"which is ignorable & same has been skipped due their unavailability as directed by us. 

Backup should execute successfully now & will avoid errorcode 6 "backup failed to back up the requested files"in netbackup logs


brarchive -u / -c -d util_file -r /oracle/PRD/102_64/dbs/initPRD-logs.utl -s
BR0002I BRARCHIVE 7.00 (40)
BR0006I Start of offline redo log processing: aemjyhui.sve 2013-10-26 14.54.44
BR0484I BRARCHIVE log file: /oracle/PRD/saparch/aemjyhui.sve
BR0477I Oracle pfile /oracle/PRD/102_64/dbs/initPRD.ora created from spfile /oracle/PRD/102_64/dbs/spfilePRD.ora


BR0017W Offline redo log file '/oracle/PRD/oraarch/PRDarch1_265934_759153586.arc' not found
BR0017W Offline redo log file '/oracle/PRD/oraarch/PRDarch1_265935_759153586.arc' not found
.

.
.
BR0017W Offline redo log file '/oracle/PRD/oraarch/PRDarch1_339998_759153586.arc' not found
BR0017W Offline redo log file '/oracle/PRD/oraarch/PRDarch1_339999_759153586.arc' not found
 

BR0280I BRARCHIVE time stamp: 2013-10-26 15.18.55
BR0008I Offline redo log processing for database instance: PRD
BR0009I BRARCHIVE action ID: aemjyikw
BR0010I BRARCHIVE function ID: svd
BR0048I Archive function: save_delete
BR0011I 1167 offline redo log files found for processing, total size 33098.799 MB
BR0130I Backup device type: util_file
BR0109I Files will be saved by backup utility
BR0126I Unattended mode active - no operator confirmation required

BR0280I BRARCHIVE time stamp: 2013-10-26 15.18.55
BR0229I Calling backup utility with function 'backup'...
BR0278I Command output of 'backint -u PRD -f backup -i /oracle/PRD/saparch/.aemjyikw.lst -t file -p /oracle/PRD/102_64/dbs/initPRD-logs.utl -c':
WARNING: sort_backup_type not specified in .utl file
WARNING: default sort_backup_type to <size> for backup
********************************************************************************
Program:                backint 7.1
Input File:             /oracle/PRD/saparch/.aemjyikw.lst
Profile:                /oracle/PRD/102_64/dbs/initPRD-logs.utl
Function:               BACKUP
Backup Type:            BACKUP_FILE
********************************************************************************
[16753.01] 15:19:07 INF - Processing /oracle/PRD/oraarch/PRDarch1_340000_759153586.arc
15:19:07.870 [16753] <1> :  Backup of </oracle/PRD/oraarch/PRDarch1_340000_759153586.arc> is in progress.
[16753.01] 15:19:10 INF - Data buffer size = 262144
[16753.01] 15:19:11 INF - Using Media ID @AAAAD for Backup ID SAPDB_1382780941
[16753.01] 15:19:15 INF - Processing /oracle/PRD/oraarch/PRDarch1_340001_759153586.arc
15:19:16.874 [16753] <1> :  Backup of </oracle/PRD/oraarch/PRDarch1_340001_759153586.arc> is in progress.



ORA-12154: TNS: could not resolve the connect identifier specified

$
0
0


ORA-12154: TNS: could not resolve the connect identifier specified

ORA 12154: TNS: Listener does not currently know of service requested

Checklist:

ORA 12154 looks a generic & very much known but getting cause for this error is the trick to resolve it quickly. Below 8 point checklist will ensure all basic's are followed & 9th point will be the final one which will take you closure to the resolution

1. Verify if you have compatible client version with RDBMS db version

10g, 11g or higher oracle version client are compatible with 9i or higher database versions but 8i or lower oracle databases need specific versions of oracle clients only. 

When 32 bit oracle client software is installed on 64 bit windows server it appends its own characters/braces() in connection description and results in a exception.Such issue can be tackled by installing 32 bit client in any other drive like d: , e: but not inside c: drive program files (64)

2. Client Server has only single oracle client binary installed

Mostly seen in windows machines previous oracle clients are not cleanly removed & reflecting multiple client in the same server. Need to remove all old client along with their directory structures correctly if not done by a clean removal method & have a reboot of the system before proceeding with new client installation 

3. tnsnames.ora file contains proper TNS Entry formatting

Having correct TNS entry is the key to resolving ORA-12154 error, make it as simple as below

mydb =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP) (HOST = hostname) (PORT = 1527))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = mydb)
    )
  )

If you suspect there are any missing braces or unknown issue in formatting just use below single liner

mydb=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname)(PORT = 1527)))(CONNECT_DATA = (SERVICE_NAME = mydb)))


and you can easily test its resolution  through TNS alias or using description itself as below

Yes this single liner will work , I am not keen to show any example snaps here as I believe you will test yourself instead of assuming it from my snaps

tnsping "(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname)(PORT = 1527)))(CONNECT_DATA = (SERVICE_NAME = mydb)))


4. Host which you are trying to connect is pingable & added in /etc/hosts

Coming from point 3 one of the basic check should be the resolution of hostname specified in HOST clause of TNS entry which can be an IP address or domain/DNS , LDAP or /etc/hosts resolvable entry

Whichever it is make sure it is the correct one

5. telnet Host Port is working

Checking correctness of hostname specified in HOST, we just have ping & telnet.
If it is pinging then check telnet to DB port on which listener is listening

telnet <IP> <Port>

In most of the cases ping & telnet seems to be working but error does not disappear , don’t lose the battle yet still few more points to be checked

6. DB service/Listener you are trying to connect is up & running on hosted db server

Yes this check looks obvious but do not skip it, may be listener is up on your DB server or although it is up but running with wrong IP (may be cluster IP) or hostname. So verify & compare TNS entry of the server against the client address and confirm its resolution to yourself

7. tnsping <TNS_Alias> is working from client

tnsping is a great tool to troubleshoot oracle net relevant issues , verify if it resolves through correct address description & within stipulated time

8. sqlplus <user>@<TNS_Alias>/<password> is working

Using sqlplus will isolate the issue if ORA-12154 is faced only in third party tools or application . sqlplus is an inbuilt oracle application development tool. Traversing it via tns alias will allow to replicate the issue & can be tested/fixed by yourself only rather than asking user to replicate it for you.

9. Troubleshoot using tracing 

If above all basic check has not helped then use tracing for sqlplus session by following the link, which will surely help to get the cause & can be fixed accordingly 

Before enabling tracing for the session make sure you are able to replicate the error which will get capture in the trace file along with the error around it


Let me know if all above doesn't help !!

ORA-01078 ORA-29701

$
0
0
Error:

SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service


$/rdbms/11gR2/grid/bin/ocssd.bin start
Segmentation Fault - core dumped


Background:

Errors observed while starting up the single instance ASM database post crash happened due to killing of pmon background process or server restart without appropriate service stop

Verifications & Resolution:

ASM instance requires oracle cluster synchronization agent services & daemon in running state.

1. Verify the css process status

$ps -ef | grep -i css
  oracle  2385     1   0 01:23:53 ?           4:44 /rdbms/11gR2/grid/bin/ocssd.bin
  oracle  2373     1   0 01:23:53 ?           0:42 /rdbms/11gR2/grid/bin/cssdagent
  oracle   655   644   0 18:38:17 pts/1       0:00 grep -i css


2. check the status of the cluster resources like diskgroup , asm instance etc.

$crs_stat -t
Name                Type                Target         State        Host
------------- ----------------  -------------  ---------- --------
ora.DG_ARCH.dg ora....up.type    ONLINE    ONLINE    ace-...tuat
ora.DG_DATA.dg ora....up.type    ONLINE    ONLINE    ace-...tuat
ora.DG_INDX.dg ora....up.type     ONLINE    ONLINE    ace-...tuat
ora....ED01.dg   ora....up.type     ONLINE    ONLINE    ace-...tuat
ora....ED02.dg   ora....up.type     ONLINE    ONLINE    ace-...tuat
ora....ED03.dg   ora....up.type     ONLINE    ONLINE    ace-...tuat
ora.asm            ora.asm.type      ONLINE    ONLINE    ace-...tuat
ora.cssd           ora.cssd.type     ONLINE    ONLINE    ace-...tuat
ora.diskmon       ora....on.type     ONLINE    ONLINE    ace-...tuat

3. check alert log of asm for any errors

cat $ORACLE_BASE/log/diag/asm/+asm/<ASM_instance_name>/trace/alert_<ASM_instance_name>.log

Thu Nov 28 01:02:08 2013
Errors in file /rdbms/11gR2/grid/log/diag/asm/+asm/+ASM/trace/+ASM_gmon_17716.trc:
ORA-29702: error occurred in Cluster Group Service operation
ORA-29702: error occurred in Cluster Group Service operation
GMON (ospid: 17716): terminating the instance due to error 29702 


4. verify the existence of parameter file for ASM & check the accessibility of disk paths, other mounted disk

cat $ORACLE_HOME/dbs/init<ASM_instance_name>.ora

asm_diskgroups ='DG_DATA','DG_INDX','DG_ARCH','DG_RED01','DG_RED02','DG_RED03'
asm_diskstring ='/dev/zvol/rdsk/db-dataP*'
instance_type='asm'
large_pool_size=12M
 

$ls -lrt /dev/zvol/rdsk/db-dataP*
total 6
lrwxrwxrwx 1 root root  39 Sep  5  2012 DG_ARCH -> ../../../../devices/pseudo/zfs@0:2c,raw
lrwxrwxrwx 1 root root  39 Sep  5  2012 DG_DATA -> ../../../../devices/pseudo/zfs@0:3c,raw
lrwxrwxrwx 1 root root  39 Sep  5  2012 DG_INDX -> ../../../../devices/pseudo/zfs@0:5c,raw
lrwxrwxrwx 1 root root  39 Sep  5  2012 DG_RED01 -> ../../../../devices/pseudo/zfs@0:6c,raw
lrwxrwxrwx 1 root root  39 Sep  5  2012 DG_RED02 -> ../../../../devices/pseudo/zfs@0:7c,raw
lrwxrwxrwx 1 root root  39 Sep  5  2012 DG_RED03 -> ../../../../devices/pseudo/zfs@0:8c,raw
 


5. if all above checks found ok then try starting up the ASM instance again using srvctl , make sure ORACLE_SID & other environment variables are set correctly

$srvctl start asm

$ps -ef | grep -i pmon
  oracle  2435     1   0 01:24:10 ?           0:00 asm_pmon_+ASM
  oracle  2518  1498   0 01:24:35 pts/2       0:00 grep -i pmon


6. if Step 5 fails then repeat the same after server restart or ask Storage / Unix Admin to check for any issues as server restart may not be feasible in most of the cases


preparing for oracle certification

$
0
0
scope of this document is to highlight the benefits of certification and its preparation , it does not target to specific exam certification code or syllabus but to provide hints to prepare them all.it can be referred by all technology aspiring certified professionals and not just oracle

Oracle certification is an educational program by oracle university to increase the oracle user base in the form of profiles like  dba , developers , architects , business analysts which has following benefits




1.Self pride 
"I am an oracle certified professional/expert/master" these few words itself has so much of weight & efforts behind it. Certification adds on the self pride for sure if done in proper way else it is just an another piece of paper (will explain it why , soon..)

2.Knowledge 
Certified means you have acquired all the professional level knowledge which oracle has decided to share for its product invented component (product is a marketing word & may not upholds the spirit of this composition hence striked it ! ). knowledge is the only precious thing to be earn from certification and rest all like certificate/logo/a line in CV are only matters for impression


3.CV strengthening 
Over 80% of certified DBA's have done this to make their CV unique & impressive .. I was one of them initially. Catch here ! if many of those 80% certified DBA's apply to the same famous company then how it will look unique.Yes it will (read and understand last line of previous paragraph) however rest of the DBA's do prefer to skip the certification and go as they get experience which is also good thing but it needs more hands one exposure to multiple scenarios

4.Interview certainty 
if you are certified then you know almost all the topics on which questions would arise which adds to certainty rather than just following the interview trend and mostly asked questions. Interview trend means setting certain time frame for interview say 2 months and starting appearing for it as the call comes wherein initial few interviews would be equivalent to rejection but will give the insight of questions being asked and same can be prepared for future.With this approach you could loose top most opportunities and need to satisfy on secondary jobs



5.Subject Expertise 
Huge number ofcertifications are available & each of them has a scope to get expertise in it
You can see a Security Oracle DBA , Performance Tuning Expert , Backup/Recovery database administrator or just a simple Oracle DBA (knowing upto certain level but little less than mastery in all topics , which i prefer although people may call "Jack of all trades, master of none" but think your self an all-rounder ! ). While preparing for certification you may build up an interest in certain topics and starts thinking of it more or putting more efforts to it which will make you subject matter expert for the same in future


6.Decision Making 
Knowledge and confidence gained throughout the learning path of certification gives its result in decision making while deciding on technical scenarios like new setup architecture  , backup methodology , performance / scalable approach , cost effectiveness  , upgrade / migration planning and many more. 

I had experienced it in my initial stages and you must read it to get exact meaning of this article.

"A group of managers discussing on migration plan to move a primary & standby database setup to new servers with almost 2 days of business downtime and which seems to be on its final stage of freeze.I was just invited to know my responsibility as a Jr. DBA which was nothing but starting up the database and few post checks after offline cloning from backup to new servers.I was just shocked to know the plan & felt lucky for them to be a part of the late invited discussion.i said 'Being a primary standby database & only server migration without version upgrade it can be done in 30 minutes of downtime'. They said, 'What?How?'. obviously it was simple answer for a certified DBA, 'make use of data guard setup , migrate standby database first to new server then switch over the primary to new standby and repeat the same for old primary, downtime required would be equal to switching over the db & pointing applications to new server/ip's. That's it ! '

A business benefits resides in every smart decisions we make with minimum risk and technical support. knowledge is the key for it which can be achieved through proper certification preparation only.

All above benefits can be enjoyed only if proper path is followed.

Many of the DBA's prefer the fast track approach for certification just by learning the question/answers of examination which makes them certified for sure but they don't enjoy it before , during & after the certification too.

Certification with proper way can be done as below :


   1. Identification
Certification learning path provided on http://education.oracle.com will guide you to select an appropriate certification path and identification of exams , syllabus , passing criteria etc.
This phase mostly matters at the start of the carrier or during switch. Suitable technology with interest to be selected to attain proficiency in future as a long term goal.Certification path includes set of exams which are interdependent according to which associate/professional/expert/master levels can be achieved.Accurate planning is required to attempt all such exams at specific intervals without any distraction till desired level is attained.

 2. Syllabus
Every exam has its syllabus listed with topics & sections.chances of certain questions being asked although it is relevant to syllabus but may look irrelevant due to its scenario based design which demands out of syllabus knowledge as well like UNIX , backup etc.

 3. Study material
Oracle provides the student guide books for its classroom training , same can be referred as a basic by joining oracle partnered institutes. As a low cost approach you may download .pdf of it or borrow a book from your colleague and take printouts from office printer itself (100 pages daily to escape from log capture and print four pages on single page both side which will save trees as well) & do the spiral bind.
In addition to student guide author book/exam guides can be referred ( be aware to purchase the books of standard publications only like McGraw-Hill Oracle press ).

  4. Lab Practice
Youcan have a test setup on set of machines or on a lower environment server to play with.
Reading & practicing the same will be remembered forever.Make sure when you install the binaries on server do not use enterprise edition as it may impact licensing or try to bring down the db once done with the practice 

 5. Reading 
Readingis the best resolution accompanied by understanding which can keep you away from all forms of problems in exam and work.Reading the study material not only boosts your knowledge about the topic but also gives you the power to think along with confidence during crunch time.Reading consistently improves your concentration and yields better results in every good thing you attempt.

 6. Revision

Revisingallthat you have prepared for exam is necessary because the time span in which complete study is done is long enough to forget. Revision helps to recollect everything required before appearing for certification exam and helps to gain confidence at right time.

 7. Exam Questions

Important phase before appearing an exam and must know the previous exam questions & identify the correct answers for the same.One should not jump to this stage skipping above all as it may land up you in crisis. Appearing exam directly after learning only questions and answers will freeze your mind if questions gets twisted and no other option than loosing the marks, having done book reading & practice as a backup will find logical way to the problems and unknown issues.

 8. Exam Schedule
Plan the exam schedule well in advance at least 2-3 weeks before appearing, get enough spare time before appearing the exam to keep yourself calm and prepared. plan within such a period where working environment is stable and no major tasks are assigned.
Before appearing for exam just ask this question to yourself , "Have i done full preparation ? does anything left which will be blamed by myself if i fail?" if any doubt get the exam rescheduled till you get positive answer from yourself.



I have followed above approach without any deviation and did not came across any failure (till now at least and able to share these on completion of my 10th certification exam !)

ORA-01882: timezone region %s not found

$
0
0
Error:  ORA-01882: timezone region %s not found



When: accessing dba_scheduler_jobs view

Where: Pl Sql developer

Cause: Inappropriate values of the time zone for the columns with data type as TIMESTAMP. 

Initial suspect may goes to NLS or TIME_ZONE session parameters as this issue appears in third party tool only

col START_DATE  for a40
col END_DATE for a39
col LAST_START_DATE for a39
col LAST_RUN_DURATION for a39
select owner,JOB_NAME,START_DATE,LAST_START_DATE,LAST_RUN_DURATION,NEXT_RUN_DATE,SCHEDULE_LIMIT,MAX_RUN_DURATION from dba_scheduler_jobs;


OWNER  JOB_NAME START_DATE LAST_START_DATE LAST_RUN_DURATION NEXT_RUN_DATE SCHEDULE_LIMIT MAX_RUN_DURATION
---- ------- ------- --------- ---------- ---------------- -------------- ---------------  ------------- ----------------- 

SYS   PURGE_LOG  28-MAY-10 03.00.00.600000 AM PST8PDT 16-DEC-13 03.00.00.296403 AM PST8PDT +000000000 00:00:01.137385  17-DEC-13 03.00.00.300000 AM PST8PDT
                                                                



PST8PDT TZ found to be causing error


Solution:




1. Use below time zone conversion as a workaround for all columns with TIMESTAMP data type which appeared to be giving problem


selectOWNER,job_name,JOB_ACTION,start_date attimezone'Asia/Calcutta', LAST_START_DATE attimezone'Asia/Calcutta', STATE from dba_scheduler_jobs ;



2.Modify the value to reflect correct timezone value



exec dbms_scheduler.disable('PURGE_LOG');
exec dbms_scheduler.enable('PURGE_LOG');
exec dbms_scheduler.run_job('PURGE_LOG');


Above three activities will reset the column values for START_DATE,NEXT_RUN_DATE but LAST_START_DATE wont get reset. 

To reset LAST_START_DATE value perform below steps :


Identify the ddl for dba_scheduler_jobs view to get the exact internal view being used to populate LAST_START_DATE column

set long 9000
select dbms_metadata.get_ddl('VIEW','DBA_SCHEDULER_JOBS') from dual;


Identify the value from internal view sys.scheduler$_job

select last_start_date from sys.scheduler$_job where  PROGRAM_ACTION like '%PURGE_LOG%';


Update the value & set as null

update sys.scheduler$_job set LAST_START_DATE='' where  PROGRAM_ACTION like '%PURGE_LOG%';









Verify if no column values have wrong time zone & re-try from third party

ORA-01031: insufficient privileges

$
0
0
Error:

ORA-01031: insufficient privileges

When: 

While compiling a procedure/package/function

Solution:

Even after granting privileges like CREATE PROCEDURE / CREATE ANY PROCEDURE or EXECUTE / EXECUTE ANY PROCEDURE , ORA-01031 is faced. 

Such issue is common whenever accessing cross owned objects for modification. 

Here, CSSSAP is accessing CSS owned objects for modification without appropriate privileges

Granting ALTER ANY PROCEDURE helps to over come the insufficient privilege error as below :

SQL>  alter procedure CSS.PROC_ACCESSD compile;

 alter procedure
CSS.PROC_ACCESSD compile
*
ERROR at line 1:
ORA-01031: insufficient privileges

SQL> grant ALTER ANY PROCEDURE to csssap;

Grant succeeded.

SQL> SQL> conn csssap/********

Connected.
SQL> alter procedure CSS.PROC_ACCESSD compile;

Procedure altered.

SQL>

 


ORA-15063: ASM discovered an insufficient number of disks for diskgroup

$
0
0
Error:

ERROR: diskgroup DG02 was not mounted
ORA-15032: not all alterations performed
ORA-15017: diskgroup "DG02" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DG02"
ORA-15080: synchronous I/O operation to a disk failed

ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 5: Input/output error
Additional information: -1
Additional information: 4096
WARNING: Read Failed. group:0 disk:0 AU:0 offset:0 size:4096
ERROR: no read quorum in group: required 2, found 0 disks

ERROR: -9(Error 27061, OS Error (Linux-x86_64 Error: 5: Input/output error


When:

Storage has been disconnected from server or power cycled without stopping ASM or database instances

Analysis & Resolution:

Step1: ASM Alert log has been reviewed to get more details about the error messages & disk involved


Step2:  Queried disk views in running ASM instance to get the mount & header status of the disk , which was closed & candidate respectively suspecting metadata corruption in disk

SQL>select NAME, TOTAL_MB,FREE_MB from v$asm_diskgroup;

no rows selected


SQL>set line 900
SQL>set pages 900
SQL>col PATH for a40
SQL>select GROUP_NUMBER, DISK_NUMBER,INCARNATION,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,OS_MB,FREE_MB,NAME,path from v$asm_disk order by 2;


GROUP_NUMBER DISK_NUMBER INCARNATION MOUNT_S HEADER_STATU MODE_ST STATE         OS_MB    FREE_MB NAME                           PATH
------------ ----------- ----------- ------- ------------ ------- -------- ---------- ---------- ------------------------------ ----------------------------------------
           0           0  3912595095 CLOSED  CANDIDATE    ONLINE  NORMAL       512000          0                                /dev/mapper/mpathe




Step3: Restarted the ASM instance to check if disk group gets mounted & queries on v$ views for corrupted disk path . No luck



SQL> shut immediate;
ORA-15100: invalid or missing diskgroup name
ASM instance shutdown


SQL> startup
ASM instance started
Total System Global Area  283930624 bytes
Fixed Size                  2227664 bytes
Variable Size             256537136 bytes
ASM Cache                  25165824 bytes
ORA-15032: not all alterations performed
ORA-15017: diskgroup "DG02" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DG02"


SQL> select * from v$asm_disk where PATH='/dev/mapper/mpathe';
GROUP_NUMBER DISK_NUMBER COMPOUND_INDEX INCARNATION MOUNT_S HEADER_STATU MODE_ST STATE    REDUNDA LIBRARY                                                              OS_MB    TOTAL_MB    FREE_MB HOT_USED_MB COLD_USED_MB NAME                           FAILGROUP                      LABEL                           PATH                    UDID                                                      PRODUCT                          CREATE_DA MOUNT_DAT REPAIR_TIMER      READS     WRITES  READ_ERRS WRITE_ERRS  READ_TIME WRITE_TIME BYTES_READ BYTES_WRITTEN P HASH_VALUE  HOT_READS HOT_WRITES HOT_BYTES_READ HOT_BYTES_WRITTEN COLD_READS COLD_WRITES COLD_BYTES_READ COLD_BYTES_WRITTEN V SECTOR_SIZE FAILGRO
------------ ----------- -------------- ----------- ------- ------------ ------- -------- ------- ---------------------------------------------------------------- ---------- ---------- ---------- ----------- ------------ ------------------------------ ------------------------------ ------------------------------- ---------------------------------------- 

           0           0              0  3909105280 CLOSED  CANDIDATE    ONLINE  NORMAL   UNKNOWN System                                                               512000          0          0           0            0                                                                                               /dev/mapper/mpathe       0                                                        


Step4: Asked system & storage admin to check from OS & storage end for any misconfiguration or corruption along with OS restart. Still No Luck !



Step5: Performed disk integrity check using grid kfed command. content found to be invalid as highlighted below







$ kfed read /dev/mapper/mpathe
kfbh.endian:                          0 ; 0x000: 0x00
kfbh.hard:                            0 ; 0x001: 0x00
kfbh.type:                            0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt:                          0 ; 0x003: 0x00
kfbh.block.blk:                       0 ; 0x004: blk=0
kfbh.block.obj:                       0 ; 0x008: file=0
kfbh.check:                           0 ; 0x00c: 0x00000000
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
7F0B46922400 00000000 00000000 00000000 00000000  [................]
        Repeat 26 times
7F0B469225B0 00000000 00000000 00000000 00009A1D  [................]
7F0B469225C0 FEEE0001 0001FFFF FFFF0000 00003E7F  [.............>..]
7F0B469225D0 00000000 00000000 00000000 00000000  [................]
        Repeat 1 times
7F0B469225F0 00000000 00000000 00000000 AA550000  [..............U.]
7F0B46922600 20494645 54524150 00010000 0000005C  [EFI PART....\...]
7F0B46922610 61BAE647 00000000 00000001 00000000  [G..a............]
7F0B46922620 3E7FFFFF 00000000 00000022 00000000  [...>....".......]
7F0B46922630 3E7FFFDE 00000000 234EED11 4FB5F1FC  [...>......N#...O]
7F0B46922640 E49A8493 8FF0260A 00000002 00000000  [.....&..........]
7F0B46922650 00000080 00000080 AB54D286 00000000  [..........T.....]
7F0B46922660 00000000 00000000 00000000 00000000  [................]
  Repeat 217 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]



Step6: Disk status re-validated before running repair disk





$ kfod status=TRUE asm_diskstring='/dev/mapper/*' disk=all dscvgroup=TRUE
--------------------------------------------------------------------------------------------
 Disk          Size Header    Path                                     Disk Group   User     Group
========================================================================
 1:     512000 Mb CANDIDATE /dev/mapper/mpathe                       #            grid     oinstall
---------------------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
========================================================================
      +ASM /u01/app/grid/product/11.2.0/grid



Step7: Disk repaired using kfed repair which has restored/repaired the disk header to get metadata back on the disk at desired location


$ kfed repair /dev/mapper/mpathe
 

$ kfed read /dev/mapper/mpathe
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt:                          1 ; 0x003: 0x01
kfbh.block.blk:                       0 ; 0x004: blk=0
kfbh.block.obj:              2147483648 ; 0x008: disk=0
kfbh.check:                  3848140538 ; 0x00c: 0xe55df6fa
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr:         ORCLDISK ; 0x000: length=8
kfdhdb.driver.reserved[0]:            0 ; 0x008: 0x00000000
kfdhdb.driver.reserved[1]:            0 ; 0x00c: 0x00000000
kfdhdb.driver.reserved[2]:            0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]:            0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]:            0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]:            0 ; 0x01c: 0x00000000
kfdhdb.compat:                186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum:                        0 ; 0x024: 0x0000
kfdhdb.grptyp:                        1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname:               DG02_0000 ; 0x028: length=9
kfdhdb.grpname:                    DG02 ; 0x048: length=4
kfdhdb.fgname:                DG02_0000 ; 0x068: length=9
kfdhdb.capname:                         ; 0x088: length=0
kfdhdb.crestmp.hi:             32993836 ; 0x0a8: HOUR=0xc DAYS=0x11 MNTH=0xc YEAR=0x7dd
kfdhdb.crestmp.lo:            632394752 ; 0x0ac: USEC=0x0 MSEC=0x65 SECS=0x1b MINS=0x9
kfdhdb.mntstmp.hi:             32993843 ; 0x0b0: HOUR=0x13 DAYS=0x11 MNTH=0xc YEAR=0x7dd
kfdhdb.mntstmp.lo:           1118950400 ; 0x0b4: USEC=0x0 MSEC=0x75 SECS=0x2b MINS=0x10
kfdhdb.secsize:                     512 ; 0x0b8: 0x0200
kfdhdb.blksize:                    4096 ; 0x0ba: 0x1000
kfdhdb.ausize:                  1048576 ; 0x0bc: 0x00100000
kfdhdb.mfact:                    113792 ; 0x0c0: 0x0001bc80
kfdhdb.dsksize:                  512000 ; 0x0c4: 0x0007d000
kfdhdb.pmcnt:                         6 ; 0x0c8: 0x00000006
kfdhdb.fstlocn:                       1 ; 0x0cc: 0x00000001
kfdhdb.altlocn:                       2 ; 0x0d0: 0x00000002
kfdhdb.f1b1locn:                      2 ; 0x0d4: 0x00000002
kfdhdb.redomirrors[0]:                0 ; 0x0d8: 0x0000
kfdhdb.redomirrors[1]:                0 ; 0x0da: 0x0000
kfdhdb.redomirrors[2]:                0 ; 0x0dc: 0x0000
kfdhdb.redomirrors[3]:                0 ; 0x0de: 0x0000
kfdhdb.dbcompat:              168820736 ; 0x0e0: 0x0a100000
kfdhdb.grpstmp.hi:             32993836 ; 0x0e4: HOUR=0xc DAYS=0x11 MNTH=0xc YEAR=0x7dd
kfdhdb.grpstmp.lo:            632058880 ; 0x0e8: USEC=0x0 MSEC=0x31d SECS=0x1a MINS=0x9
kfdhdb.vfstart:                       0 ; 0x0ec: 0x00000000
kfdhdb.vfend:                         0 ; 0x0f0: 0x00000000
kfdhdb.spfile:                        0 ; 0x0f4: 0x00000000
kfdhdb.spfflg:                        0 ; 0x0f8: 0x00000000
kfdhdb.ub4spare[0]:                   0 ; 0x0fc: 0x00000000
kfdhdb.ub4spare[1]:                   0 ; 0x100: 0x00000000
kfdhdb.ub4spare[2]:                   0 ; 0x104: 0x00000000
kfdhdb.ub4spare[3]:                   0 ; 0x108: 0x00000000
kfdhdb.ub4spare[16]:                  0 ; 0x13c: 0x00000000
kfdhdb.ub4spare[17]:                  0 ; 0x140: 0x00000000
.

.
.
fdhdb.acdb.aba.seq:                  0 ; 0x1d4: 0x00000000
kfdhdb.acdb.aba.blk:                  0 ; 0x1d8: 0x00000000
kfdhdb.acdb.ents:                     0 ; 0x1dc: 0x0000
kfdhdb.acdb.ub2spare:                 0 ; 0x1de: 0x0000
 


Step8:  ASM instance started , DG got mounted & DB Started as well
 
$ sqlplus "/ as sysasm"
SQL*Plus: Release 11.2.0.3.0 Production on Sun Jan 12 19:47:16 2014
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
Connected to an idle instance.
SQL> startup
ASM instance started
Total System Global Area  283930624 bytes
Fixed Size                  2227664 bytes
Variable Size             256537136 bytes
ASM Cache                  25165824 bytes
ASM diskgroups mounted
 

SQL> select NAME, TOTAL_MB,FREE_MB from v$asm_diskgroup;
NAME                             TOTAL_MB    FREE_MB
------------------------------ ---------- ----------
DG02                               512000       9629

SQL>DISK_NUMBER,INCARNATION,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,OS_MB,FREE_MB,NAME,path from v$asm_disk order by 2;GROUP_NUMBER DISK_NUMBER INCARNATION MOUNT_S HEADER_STATU MODE_ST STATE         OS_MB    FREE_MB NAME                           PATH
------------ ----------- ----------- ------- ------------ ------- -------- ---------- ---------- ------------------------------ ----------------------------------------
           1           0  3909403310 CACHED  MEMBER       ONLINE  NORMAL       512000       9629 DG02_0000                      /dev/mapper/mpathe
      


$ sqlplus "/ as sysdba"
SQL*Plus: Release 11.2.0.3.0 Production on Sun Jan 12 19:48:11 2014
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
Connected to an idle instance.
SQL> startup
ORACLE instance started.
Total System Global Area 1720328192 bytes
Fixed Size                  2229064 bytes
Variable Size            1224740024 bytes
Database Buffers          335544320 bytes
Redo Buffers              157814784 bytes
Database mounted.
Database opened.



oracle database corrupt block recovery

$
0
0
Error:

Corrupt block relative dba: 0x1a0975d9 (file 104, block 619993)
Bad check value found during user buffer read
Data in bad block:
 type: 6 format: 2 rdba: 0x1a0975d9
 last change scn: 0x0726.9a3a3fc5 seq: 0x1 flg: 0x06
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x3fc50601
 check value in block header: 0x287e
 computed block checksum: 0xfe81
Reread of rdba: 0x1a0975d9 (file 104, block 619993) found same corrupted data
Tue Feb 18 20:17:59 IST 2014
Corrupt Block Found
         TSN = 47, TSNAME = EVENT_TAB
         RFN = 104, BLK = 619993, RDBA = 436827609
         OBJN = 141322, OBJD = 141322, OBJECT = EVENT_PK, SUBOBJECT =
         SEGMENT OWNER = EVA_ADMIN, SEGMENT TYPE = Index Segment


Background:

Oracle 10g database sizing over 1 TB with no RMAN backup. 
Backup being taken using legacy hot backup image copy to disk/tape.
Block being corrupted belongs to IOT index wherein data resides itself inside index sizing over 500 GB.
Alert log has provided enough information about corruption like tablespace_name  , object_name , object_id , object_type , corrupted block number and more to initiate corruption recovery plan.

Verification:

Important step for any type of corruption in database and should include integrity check for complete database to identify if corruption list includes any more blocks other than that mentioned in alert log

1. Identify possible objects impacted due to corruption based on corrupted block id, 619993 is the corrupted block id appeared in view or alert log asBLK = 619993

SELECT e.file_id,
 e.block_id,
 e.owner,
 e.segment_name,
 e.segment_type
 FROM dba_extents e
 WHERE
 file_id=104 and 619993 BETWEEN 619993 AND 619993 + blocks - 1
 /

2. Perform logical database / datafile integrity check which will populate  v$database_block_corruption


RMAN> run {backup validate datafile 104;}

Starting backup at 19-FEB-14
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=2190 devtype=DISK
channel ORA_DISK_1: starting full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
input datafile fno=00104 name=/data02/EVACP/event_tab_90.dbf
channel ORA_DISK_1: backup set complete, elapsed time: 00:01:15
Finished backup at 19-FEB-14

RMAN>

Get the progress of rman validate command using below ,

select MBYTES_PROCESSED,STATUS,OPERATION from V_$RMAN_STATUS where OPERATION like '%VALIDATE%';

MBYTES_PROCESSED STATUS                  OPERATION
---------------- ----------------------- ---------------------------------
            9216 COMPLETED               BACKUP VALIDATE



Similarly , complete database integrity can also be validated using below rman command.

 backup validate check logical database ;

 3. Corrupt block detected from RMAN validate integrity check will get populated in v$database_block_corruption as well as in alert log

select * from v$database_block_corruption ;
     FILE#     BLOCK#     BLOCKS CORRUPTION_CHANGE# CORRUPTIO
---------- ---------- ---------- ------------------ ---------
       104     619993          1                  0 CHECKSUM 

select file#, block# as blk#, corruption_type as type from v$database_block_corruption;
     FILE#       BLK# TYPE
---------- ---------- ---------
       104     619993 CHECKSUM

4. Offline datafile physical corruption check 

dbv FILE=/corp/data02/EVACP/event_tab_90.dbf FEEDBACK=100


DBVERIFY - Verification complete
Total Pages Examined         : 1179648
Total Pages Processed (Data) : 134144
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 536576
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 8
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 508920
Total Pages Marked Corrupt   : 0
Total Pages Influx           : 0
Highest block SCN            : 378599356 (1831.378599356)
 

Resolution:

Corruption fixes plan depends on type of the recent DB backup being taken before first corruption incident time.

Here , db backup image is a part of hot backup & not RMAN backupset
 
1. Restore all the archivelog files since end backup time to a staging location (/tempfs) or in the same location of current archivelog destination

2. Restore backup of corrupted datafile id 104 to staging location (/tempfs)

3. Catalog archivelog restored to make control file aware of their existence.

RMAN> catalog archivelog '/tempfs/1_54763_668965414.arc';
cataloged archive log
archive log filename=/tempfs/1_54763_668965414.arc recid=70182 stamp=839895943

Make sure all archivelog files are restored since end backup & cataloged to avoid below error during block media recovery

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of blockrecover command at 02/19/2014 00:30:13
RMAN-06053: unable to perform media recovery because of missing log
RMAN-06025: no backup of log thread 1 seq 54769 lowscn 7864522937984 found to restore



4. Catalog backup datafile restored to make control file aware of their existence.


RMAN> catalog datafilecopy '/tempfs/event_tab_90.dbf';
cataloged datafile copy
datafile copy filename=/tempfs/event_tab_90.dbf recid=22055 stamp=839895977



5. Re-verify the current datafile to get corrupted block list

RMAN> run {backup validate datafile 104;}
Starting backup at 19-FEB-14
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=2190 devtype=DISK
channel ORA_DISK_1: starting full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
input datafile fno=00104 name=/data02/EVACP/event_tab_90.dbf
channel ORA_DISK_1: backup set complete, elapsed time: 00:01:15
Finished backup at 19-FEB-14

RMAN>

6. perform block recovery

RMAN> RUN
{
  BLOCKRECOVER DATAFILE 104 BLOCK 619993
  FROM DATAFILECOPY;
}
2> 3> 4> 5>

Starting blockrecover at 19-FEB-14
using channel ORA_DISK_1

channel ORA_DISK_1: restoring block(s) from datafile copy /tempfs/event_tab_90.dbf

starting media recovery

archive log thread 1 sequence 54744 is already on disk as file /tempfs/1_54744_668965414.arc
archive log thread 1 sequence 54745 is already on disk as file /tempfs/1_54745_668965414.arc
archive log thread 1 sequence 54746 is already on disk as file /tempfs/1_54746_668965414.arc
archive log thread 1 sequence 54747 is already on disk as file /tempfs/1_54747_668965414.arc
archive log thread 1 sequence 54748 is already on disk as file /tempfs/1_54748_668965414.arc
archive log thread 1 sequence 54749 is already on disk as file /tempfs/1_54749_668965414.arc
archive log thread 1 sequence 54750 is already on disk as file /tempfs/1_54750_668965414.arc
archive log thread 1 sequence 54751 is already on disk as file /tempfs/1_54751_668965414.arc
archive log thread 1 sequence 54752 is already on disk as file /tempfs/1_54752_668965414.arc
archive log thread 1 sequence 54753 is already on disk as file /tempfs/1_54753_668965414.arc
archive log thread 1 sequence 54754 is already on disk as file /tempfs/1_54754_668965414.arc
archive log thread 1 sequence 54755 is already on disk as file /tempfs/1_54755_668965414.arc
archive log thread 1 sequence 54756 is already on disk as file /tempfs/1_54756_668965414.arc
archive log thread 1 sequence 54757 is already on disk as file /tempfs/1_54757_668965414.arc
archive log thread 1 sequence 54758 is already on disk as file /tempfs/1_54758_668965414.arc
archive log thread 1 sequence 54759 is already on disk as file /tempfs/1_54759_668965414.arc
archive log thread 1 sequence 54760 is already on disk as file /tempfs/1_54760_668965414.arc
archive log thread 1 sequence 54761 is already on disk as file /tempfs/1_54761_668965414.arc
archive log thread 1 sequence 54762 is already on disk as file /tempfs/1_54762_668965414.arc
archive log thread 1 sequence 54763 is already on disk as file /tempfs/1_54763_668965414.arc
archive log thread 1 sequence 54764 is already on disk as file /tempfs/1_54764_668965414.arc
archive log thread 1 sequence 54765 is already on disk as file /tempfs/1_54765_668965414.arc
archive log thread 1 sequence 54766 is already on disk as file /tempfs/1_54766_668965414.arc
archive log thread 1 sequence 54767 is already on disk as file /tempfs/1_54767_668965414.arc
archive log thread 1 sequence 54768 is already on disk as file /tempfs/1_54768_668965414.arc
archive log thread 1 sequence 54769 is already on disk as file /tempfs/1_54769_668965414.arc

media recovery complete, elapsed time: 00:04:06
Finished blockrecover at 19-FEB-14

RMAN>

  

7. Verify the corrupted block status by re-running the verification step 2 & 3. Note immediately querying v$database_block_corruption post block media recovery will reflect the old corrupt block.integrity check needs to be performed again to refresh the corruption view.


SQL> select file#, block# as blk#, corruption_type as type from v$database_block_corruption;

no rows selected



ASM DiskGroup Migration

$
0
0
Step Wise Disk-Group Migration from DG01 to DG02

Scenario:

DBTST is an Oracle11g single instance database on ASM which holds complete database in Disk Group named DG01 spread across 4*50 GB of Disks. Objective is to migrate complete databases from DG01 to DG02 Disk Group. Here DG02 is a new Disk Group on 500 GB disk (/dev/mapper/mpathe)

col NAME for a20
col PATH  for a40
select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,STATE,OS_MB,TOTAL_MB,FREE_MB,NAME,PATH from v$asm_disk;


GROUP_NUMBER DISK_NUMBER MOUNT_S STATE OS_MB   TOTAL_MB  FREE_MB NAME       PATH
------- -------- ------- -------- ---------- ---------- -------- --------------- ----------------------------------------
           1           2 CACHED  NORMAL                 51200    51200         43 DG01_0002    /dev/mapper/mpathc
           1           1 CACHED  NORMAL                 51200    51200         41 DG01_0001    /dev/mapper/mpathb
           1           3 CACHED  NORMAL                 51200    51200         45 DG01_0003    /dev/mapper/mpathd
           1           0 CACHED  NORMAL                 51200    51200         46 DG01_0000    /dev/mapper/mpatha

           0           0 CLOSED  NORMAL                 512000          0          0                      /dev/mapper/mpathe

Migration Steps:

1. Create a New Disk Group DG02

SQL> create diskgroup DG02 external redundancy disk '/dev/mapper/mpathe' attribute 'compatible.asm'='11.2';

Diskgroup created.



SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,STATE,OS_MB,TOTAL_MB,FREE_MB,NAME,PATH from v$asm_disk;

GROUP_NUMBER DISK_NUMBER MOUNT_S STATE OS_MB  TOTAL_MB   FREE_MB NAME      PATH
-------- ----------- ------- -------- ---------- ---------- ---------- -------------------- -------------------------
           1           2 CACHED  NORMAL        51200      51200         43 DG01_0002      /dev/mapper/mpathc
           1           1 CACHED  NORMAL        51200      51200         41 DG01_0001      /dev/mapper/mpathb
           1           3 CACHED  NORMAL        51200      51200         45 DG01_0003      /dev/mapper/mpathd
           1           0 CACHED  NORMAL        51200      51200         46 DG01_0000      /dev/mapper/mpatha
           2           0 CACHED  NORMAL       512000     512000     511944 DG02_0000   /dev/mapper/mpathe

 

 

2. Identify existing schema& DB file locations on DG01

RMAN> report schema;
using target database control file instead of recovery catalog
Report of database schema for database with db_unique_name DBTST

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    2048     SYSTEM               ***     +DG01/DBTST/datafile/system.256.797379227
2    820       SYSAUX               ***     +DG01/DBTST/datafile/sysaux.257.797379227
3    3715     UNDOTBS1           ***     +DG01/DBTST/datafile/undotbs1.258.797379227
4    2048     USERS                 ***     +DG01/DBTST/datafile/users.259.797379227
5    30720    ARSYS                ***     +DG01/DBTST/datafile/arsys.276.812153865
6    30720    ARSYS                ***     +DG01/DBTST/datafile/arsys.275.812153995
7    30720    ARSYS                ***     +DG01/DBTST/datafile/arsys.274.812154063
8    30720    ARSYS                ***     +DG01/DBTST/datafile/arsys.273.812154191
9    10000    ARSYS                ***     +DG01/DBTST/datafile/arsys.266.821749103
10   500       ARSYS                ***     +DG01/DBTST/datafile/arsys.272.828469977
16   7168     SYSAUX               ***     +DG01/DBTST/datafile/sysaux.278.801406735
17   5120     UNDOTBS1           ***     +DG01/DBTST/datafile/undotbs1.279.801406757
18   2048     USERS                 ***     +DG01/DBTST/datafile/users.280.801406775
19   2048     SYSTEM               ***     +DG01/DBTST/datafile/system.281.801406799

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    32767    TEMP                 32767       +DG01/DBTST/tempfile/temp.264.797379307
2    10240    TEMP                 10240       +DG01/DBTST/tempfile/temp.277.801406709
3    10       TEMP                 10          +DG01/DBTST/tempfile/temp.282.802955967
4    10       TEMP                 10          +DG01/DBTST/tempfile/temp.285.802956051


 3. Perform DB file movement to new disk group DG02

Method 1: Copy each db file

e.g. Shown for single datafile

RMAN> backup as copy datafile 19 format '+DG02';

Starting backup at 17-DEC-13
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=10 device type=DISK
channel ORA_DISK_1: starting datafile copy
input datafile file number=00019 name=+DG01/DBTST/datafile/system.281.801406799
output file name=+DG02/DBTST/datafile/system.256.834409133 tag=TAG20131217T121852 RECID=13 STAMP=834409138
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
Finished backup at 17-DEC-13
 


Method 2: Copy complete database (preferred)

RMAN> backup as copy database format '+DG02';

Starting backup at 17-DEC-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile file number=00005 name=+DG01/DBTST/datafile/arsys.276.812153865

output file name=+DG02/DBTST/datafile/arsys.257.834409167 tag=TAG20131217T121927 RECID=14 STAMP=834409255
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:01:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00006 name=+DG01/DBTST/datafile/arsys.275.812153995

output file name=+DG02/DBTST/datafile/arsys.258.834409263 tag=TAG20131217T121927 RECID=15 STAMP=834409350
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:01:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00007 name=+DG01/DBTST/datafile/arsys.274.812154063

output file name=+DG02/DBTST/datafile/arsys.259.834409359 tag=TAG20131217T121927 RECID=16 STAMP=834409447
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:01:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00008 name=+DG01/DBTST/datafile/arsys.273.812154191

output file name=+DG02/DBTST/datafile/arsys.260.834409453 tag=TAG20131217T121927 RECID=17 STAMP=834409553
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:01:45
channel ORA_DISK_1: starting datafile copy
input datafile file number=00009 name=+DG01/DBTST/datafile/arsys.266.821749103

output file name=+DG02/DBTST/datafile/arsys.261.834409559 tag=TAG20131217T121927 RECID=18 STAMP=834409589
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
channel ORA_DISK_1: starting datafile copy
input datafile file number=00016 name=+DG01/DBTST/datafile/sysaux.278.801406735

output file name=+DG02/DBTST/datafile/sysaux.262.834409593 tag=TAG20131217T121927 RECID=19 STAMP=834409614
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:25
channel ORA_DISK_1: starting datafile copy
input datafile file number=00017 name=+DG01/DBTST/datafile/undotbs1.279.801406757

output file name=+DG02/DBTST/datafile/undotbs1.263.834409619 tag=TAG20131217T121927 RECID=20 STAMP=834409633
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00003 name=+DG01/DBTST/datafile/undotbs1.258.797379227

output file name=+DG02/DBTST/datafile/undotbs1.264.834409633 tag=TAG20131217T121927 RECID=21 STAMP=834409647
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00001 name=+DG01/DBTST/datafile/system.256.797379227

output file name=+DG02/DBTST/datafile/system.265.834409649 tag=TAG20131217T121927 RECID=22 STAMP=834409656
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting datafile copy
input datafile file number=00004 name=+DG01/DBTST/datafile/users.259.797379227

output file name=+DG02/DBTST/datafile/users.266.834409663 tag=TAG20131217T121927 RECID=23 STAMP=834409670
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
channel ORA_DISK_1: starting datafile copy
input datafile file number=00018 name=+DG01/DBTST/datafile/users.280.801406775

output file name=+DG02/DBTST/datafile/users.267.834409671 tag=TAG20131217T121927 RECID=24 STAMP=834409677
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
channel ORA_DISK_1: starting datafile copy
input datafile file number=00019 name=+DG01/DBTST/datafile/system.281.801406799

output file name=+DG02/DBTST/datafile/system.268.834409677 tag=TAG20131217T121927 RECID=25 STAMP=834409683
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:08
channel ORA_DISK_1: starting datafile copy
input datafile file number=00002 name=+DG01/DBTST/datafile/sysaux.257.797379227

output file name=+DG02/DBTST/datafile/sysaux.269.834409685 tag=TAG20131217T121927 RECID=26 STAMP=834409687
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting datafile copy
input datafile file number=00010 name=+DG01/DBTST/datafile/arsys.272.828469977

output file name=+DG02/DBTST/datafile/arsys.270.834409689 tag=TAG20131217T121927 RECID=27 STAMP=834409690
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting datafile copy
copying current control file

output file name=+DG02/DBTST/controlfile/backup.271.834409691 tag=TAG20131217T121927 RECID=28 STAMP=834409692
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including current SPFILE in backup set
channel ORA_DISK_1: starting piece 1 at 17-DEC-13
channel ORA_DISK_1: finished piece 1 at 17-DEC-13
piece handle=+DG02/DBTST/backupset/2013_12_17/nnsnf0_tag20131217t121927_0.272.834409695 tag=TAG20131217T121927 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 17-DEC-13

RMAN>

 4. Switch database to New disk groups


RMAN> switch database to copy;

datafile 1 switched to datafile copy "+DG02/DBTST/datafile/system.265.834409649"
datafile 2 switched to datafile copy "+DG02/DBTST/datafile/sysaux.269.834409685"
datafile 3 switched to datafile copy "+DG02/DBTST/datafile/undotbs1.264.834409633"
datafile 4 switched to datafile copy "+DG02/DBTST/datafile/users.266.834409663"
datafile 5 switched to datafile copy "+DG02/DBTST/datafile/arsys.257.834409167"
datafile 6 switched to datafile copy "+DG02/DBTST/datafile/arsys.258.834409263"
datafile 7 switched to datafile copy "+DG02/DBTST/datafile/arsys.259.834409359"
datafile 8 switched to datafile copy "+DG02/DBTST/datafile/arsys.260.834409453"
datafile 9 switched to datafile copy "+DG02/DBTST/datafile/arsys.261.834409559"
datafile 10 switched to datafile copy "+DG02/DBTST/datafile/arsys.270.834409689"
datafile 16 switched to datafile copy "+DG02/DBTST/datafile/sysaux.262.834409593"
datafile 17 switched to datafile copy "+DG02/DBTST/datafile/undotbs1.263.834409619"
datafile 18 switched to datafile copy "+DG02/DBTST/datafile/users.267.834409671"
datafile 19 switched to datafile copy "+DG02/DBTST/datafile/system.268.834409677"


RMAN> report schema;

using target database control file instead of recovery catalog
Report of database schema for database with db_unique_name DBTST

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    2048     SYSTEM               ***     +DG02/DBTST/datafile/system.265.834409649
2    820       SYSAUX               ***     +DG02/DBTST/datafile/sysaux.269.834409685
3    3715     UNDOTBS1           ***     +DG02/DBTST/datafile/undotbs1.264.834409633
4    2048     USERS                 ***     +DG02/DBTST/datafile/users.266.834409663
5    30720    ARSYS                ***     +DG02/DBTST/datafile/arsys.257.834409167
6    30720    ARSYS                ***     +DG02/DBTST/datafile/arsys.258.834409263
7    30720    ARSYS                ***     +DG02/DBTST/datafile/arsys.259.834409359
8    30720    ARSYS                ***     +DG02/DBTST/datafile/arsys.260.834409453
9    10000    ARSYS                ***     +DG02/DBTST/datafile/arsys.261.834409559
10   500       ARSYS                ***     +DG02/DBTST/datafile/arsys.270.834409689
16   7168     SYSAUX               ***     +DG02/DBTST/datafile/sysaux.262.834409593
17   5120     UNDOTBS1           ***     +DG02/DBTST/datafile/undotbs1.263.834409619
18   2048     USERS                 ***     +DG02/DBTST/datafile/users.267.834409671
19   2048     SYSTEM               ***     +DG02/DBTST/datafile/system.268.834409677

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    32767    TEMP                 32767       +DG01/DBTST/tempfile/temp.264.797379307
2    10240    TEMP                 10240       +DG01/DBTST/tempfile/temp.277.801406709
3    10       TEMP                 10          +DG01/DBTST/tempfile/temp.282.802955967
4    10       TEMP                 10          +DG01/DBTST/tempfile/temp.285.802956051
 

5. Control file movement to new disk group DG02

   i. Backup controlfile to new DG02

RMAN> backup as copy current controlfile format '+DG02';

Starting backup at 17-DEC-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
copying current control file
output file name=+DG02/DBTST/controlfile/backup.273.834409791 tag=TAG20131217T122951 RECID=43 STAMP=834409791
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 17-DEC-13


 

 ii. Change DB parameters to point to new DG02

SQL>  show parameter file (o/p truncated)
NAME                     TYPE                             VALUE
-------------  ------------------ ------------------------------
control_files             string           +DG01/DBTST/controlfile/current.260.797379301
db_create_file_dest   string                           +DG01



SQL> alter system set db_create_file_dest='+DG02' scope=spfile;

System altered.


SQL> alter system set control_files='+DG02/DBTST/controlfile/Backup.273.834409791' scope=spfile;

System altered.


SQL> shut immediate;
ORA-01507: database not mounted
ORACLE instance shut down.
SQL> startup nomount;
ORACLE instance started.
Total System Global Area 1720328192 bytes
Fixed Size                  2229064 bytes
Variable Size            1224740024 bytes
Database Buffers          335544320 bytes
Redo Buffers              157814784 bytes

SQL>  show parameter file (o/p truncated)
NAME                     TYPE                             VALUE
-------------  ------------------ ------------------------------
control_files             string         
+DG02/DBTST/controlfile/Backup.273.834409791  
db_create_file_dest   string                           +DG02

SQL> SQL> alter database mount;

Database altered.


6. Perform DB Recovery & Open DB

SQL> recover database using backup controlfile;
ORA-00279: change 7857026695002 generated at 12/17/2013 12:10:29 needed for thread 1
ORA-00289: suggestion : /u01/arc_dest/1_11552_797379303.dbf
ORA-00280: change 7857026695002 for thread 1 is in sequence #11552
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
+DG01/DBTST/onlinelog/redo03.log
Log applied.
Media recovery complete.


SQL> select status, checkpoint_change#, fuzzy,
  2  to_char(checkpoint_time, 'DD-MON-YYYY HH24:MI:SS') as checkpoint_time,
  3  count(*)
  4  from v$datafile_header
group by status, checkpoint_change#, fuzzy, checkpoint_time
  5    6  order by status, checkpoint_change#, fuzzy, checkpoint_time;

STATUS       CHECKPOINT_CHANGE# FUZ CHECKPOINT_TIME                                    COUNT(*)
------- ---------------------- --- ---------------------------------------------------------- -------------
ONLINE         7857026695003 NO  17-DEC-2013 12:10:29                                   14

1 row selected.

SQL> alter database open resetlogs;

Database altered.
7. Re-create Redo Log files

SQL>  select * from v$logfile;


       GROUP# STATUS  TYPE    MEMBER                               IS_
------------- ------- ------- ------------------------------------------- ---
        3          ONLINE  +DG01/DBTST/onlinelog/redo03.log                NO
        2          ONLINE  +DG01/DBTST/onlinelog/redo02.log                NO
        1          ONLINE  +DG01/DBTST/onlinelog/redo01.log                NO

3 rows selected.

SQL> select * from v$log;

GROUP#   THREAD# SEQUENCE#  BYTES     BLOCKSIZE    MEMBERS  ARC STATUS  FIRST_CHANGE# FIRST_TIME          NEXT_CHANGE#  NEXT_TIME
------- ------- ---------- ---------- -------- --------- --- ------- ------------- ------------------- ------------- -------------------
 1      1          1    1073741824       512      1      YES ACTIVE  7857026695004 17-DEC-2013 12:59:03 7857026695398 17-DEC-2013 12:59:52
 3      1          0    1073741824       512      1      YES UNUSED         0                     0
 2      1          2    1073741824       512      1      NO  CURRENT 7857026695398 17-DEC-2013 12:59:52          0

3 rows selected.

SQL> alter database add logfile group 4 ('+DG02') size 1G;

Database altered.

SQL>  select * from v$log;

       GROUP#        THREAD#    SEQUENCE#      BYTES     BLOCKSIZE  MEMBERS ARC STATUS  FIRST_CHANGE# FIRST_TIME NEXT_CHANGE#  NEXT_TIME
------------- ------------- ------------- ------------- ------------- ------------- --- ---------- ------------- ------------ --------- -----------
        1          1        1    1073741824       512      1    YES ACTIVE  7857026695004 17-DEC-2013 12:59:03     7857026695398 17-DEC-2013 12:59:52
        4          1        0    1073741824       512      1    YES UNUSED                 0                         0
        3          1        0    1073741824       512      1    YES UNUSED                 0                         0
        2          1        2    1073741824       512      1    NO  CURRENT      7857026695398 17-DEC-2013 12:59:52         0

4 rows selected.

SQL> select * from v$logfile;

       GROUP# STATUS  TYPE    MEMBER                               IS_
------------- ------- ------- ------------------------------------------------------------ ---
        3          ONLINE  +DG01/DBTST/onlinelog/redo03.log                NO
        2          ONLINE  +DG01/DBTST/onlinelog/redo02.log                NO
        1          ONLINE  +DG01/DBTST/onlinelog/redo01.log                NO
        4          ONLINE  +DG02/DBTST/onlinelog/group_4.274.834411679           NO

4 rows selected.

SQL> alter system checkpoint;

System altered.

SQL> alter database drop logfile group 3;

Database altered.

SQL> select * from v$log;

GROUP#        THREAD#    SEQUENCE#      BYTES     BLOCKSIZE        MEMBERS ARC STATUS    FIRST_CHANGE#   FIRST_TIME    NEXT_CHANGE#  NEXT_TIME
-------- ------------- ------------- ------------- ------------- ------------- --- ---------------- ------------- ------------------- -------------
1          1        1    1073741824       512          1 YES INACTIVE     7857026695004 17-DEC-2013 12:59:03         7857026695398 17-DEC-2013 12:59:52
4          1        0    1073741824       512          1 YES UNUSED                 0                         0
2          1        2    1073741824       512          1 NO    CURRENT      7857026695398 17-DEC-2013 12:59:52         2.8147498E+14


3 rows selected.

SQL> alter database drop logfile group 1;

Database altered.

SQL> select * from v$log;

GROUP#        THREAD#    SEQUENCE#      BYTES     BLOCKSIZE        MEMBERS ARC STATUS    FIRST_CHANGE#   FIRST_TIME    NEXT_CHANGE#  NEXT_TIME
-------- ------------- ------------- ------------- ------------- ------------- --- ---------------- ------------- ------------------- -------------
   4          1        0    1073741824       512          1 YES UNUSED                 0                         0
   2          1        2    1073741824       512          1 NO    CURRENT      7857026695398 17-DEC-2013 12:59:52         2.8147498E+14



2 rows selected.

SQL> alter system switch logfile;

System altered.

SQL> alter system checkpoint;

System altered.

SQL> alter database add logfile group 1 ('+DG02') size 1G;

Database altered.

SQL>  select * from v$log;


GROUP#        THREAD#    SEQUENCE#      BYTES     BLOCKSIZE        MEMBERS ARC STATUS    FIRST_CHANGE#   FIRST_TIME    NEXT_CHANGE#  NEXT_TIME
-------- ------------- ------------- ------------- ------------- ------------- --- ---------------- ------------- ------------------- -------------
  1          1        0    1073741824       512          1 YES UNUSED         0             0
  4          1        3    1073741824       512          1 NO    CURRENT  7857026695467 17-DEC-2013 13:02:28         0
  2          1        2    1073741824       512          1 YES INACTIVE 7857026695398 17-DEC-2013 12:59:52       7857026695467 17-DEC-2013 13:02:28

3 rows selected.

SQL>  alter database drop logfile group 2;

Database altered.

SQL> select * from v$log;

GROUP#        THREAD#    SEQUENCE#      BYTES     BLOCKSIZE        MEMBERS ARC STATUS    FIRST_CHANGE#   FIRST_TIME    NEXT_CHANGE#  NEXT_TIME
-------- ------------- ------------- ------------- ------------- ------------- --- ---------------- ------------- ------------------- -------------
   4          1        3    1073741824       512          1 NO    CURRENT  7857026695467 17-DEC-2013 13:02:28         0
   1          1        0    1073741824       512          1 YES UNUSED                 0             0



2 rows selected.

SQL>  alter database add logfile group2  ('+DG02') size 1G;

Database altered.

SQL> alter database add logfile group 3 ('+DG02') size 1G;

Database altered.

SQL> select * from v$log;

GROUP#        THREAD#    SEQUENCE#      BYTES     BLOCKSIZE        MEMBERS ARC STATUS    FIRST_CHANGE#   FIRST_TIME    NEXT_CHANGE#  NEXT_TIME
-------- ------------- ------------- ------------- ------------- ------------- --- ---------------- ------------- ------------------- -------------
   1          1        0    1073741824       512          1 YES UNUSED                 0                 0
   4          1        3    1073741824       512          1 NO    CURRENT  7857026695467 17-DEC-2013 13:02:28         0
   3          1        0    1073741824       512          1 YES UNUSED                 0                     0
   2          1        0    1073741824       512          1 YES UNUSED                 0             0



4 rows selected.

SQL> alter system switch logfile;

System altered.

SQL> /

System altered.

SQL> select * from v$logfile;

       GROUP# STATUS  TYPE    MEMBER                               IS_
------------- ------- ------- ------------------------------------------------------------ ---
        1          ONLINE  +DG02/DBTST/onlinelog/group_1.275.834411781           NO
        2          ONLINE  +DG02/DBTST/onlinelog/group_2.276.834411805           NO
        3          ONLINE  +DG02/DBTST/onlinelog/group_3.277.834411811           NO
        4          ONLINE  +DG02/DBTST/onlinelog/group_4.274.834411679           NO

4 rows selected.


8. Re-create Temporary Tablespace

SQL> select * from dba_temp_files;

FILE_NAME                                ILE_ID TABLESPACE_NAME       BYTES      BLOCKS STATUS   RELATIVE_FNO AUT    MAXBYTES     MAXBLOCKS    INCREMENT_BY    USER_BYTES   USER_BLOCKS
---------------------------------------- ------ ---------------- ------------- ------------- ------- ------------- --- ------------- ------------- ------------- ------------- -------------
+DG01/DBTST/tempfile/temp.264.797379307   1      TEMP           34358689792     4194176 ONLINE          1 YES   34359721984       4194302          80   34357641216     4194048
+DG01/DBTST/tempfile/temp.277.801406709   2      TEMP          10737418240     1310720 ONLINE          2 NO           0         0           0   10736369664     1310592
+DG01/DBTST/tempfile/temp.282.802955967   3      TEMP          10485760        1280 ONLINE          3 NO           0         0           0       9437184        1152
+DG01/DBTST/tempfile/temp.285.802956051   4      TEMP          10485760        1280 ONLINE          4 NO           0         0           0       9437184        1152


4 rows selected.

SQL> create temporary tablespace TEMP1 tempfile '+DG02' size 20G;

Tablespace created.

SQL> alter database default temporary tablespace TEMP1;

Database altered.

SQL> shut immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.
Total System Global Area    1720328192 bytes
Fixed Size               2229064 bytes
Variable Size            1224740024 bytes
Database Buffers         335544320 bytes
Redo Buffers             157814784 bytes
Database mounted.
Database opened.

SQL> select * from dba_temp_files;

FILE_NAME                           FILE_ID     TABLESPACE_NAME   BYTES      BLOCKS STATUS   RELATIVE_FNO AUT    MAXBYTES     MAXBLOCKS    INCREMENT_BY    USER_BYTES   USER_BLOCKS
-------------------------------------------- ---------- ------------------ ------------- ------------- ------- ------------- --- ------------- ------------- ------------- ------------- -------------
+DG01/DBTST/tempfile/temp.264.797379307       1         TEMP           34358689792     4194176 ONLINE          1 YES   34359721984       4194302          80   34357641216     4194048
+DG02/DBTST/tempfile/temp1.278.834411957      5         TEMP1           21474836480     2621440 ONLINE          1 NO           0         0           0   21473787904     2621312
+DG01/DBTST/tempfile/temp.277.801406709       2         TEMP           10737418240     1310720 ONLINE          2 NO           0         0           0   10736369664     1310592
+DG01/DBTST/tempfile/temp.282.802955967       3         TEMP           10485760        1280 ONLINE          3 NO           0         0           0       9437184        1152
+DG01/DBTST/tempfile/temp.285.802956051       4         TEMP           10485760        1280 ONLINE          4 NO           0         0           0       9437184        1152


5 rows selected.

SQL> drop tablespace TEMP including contents and datafiles;

Tablespace dropped.

SQL>  select * from dba_temp_files;

FILE_NAME                   FILE_ID       TABLESPACE_NAME                 BYTES      BLOCKS STATUS   RELATIVE_FNO AUT    MAXBYTES     MAXBLOCKS    INCREMENT_BY    USER_BYTES   USER_BLOCKS
----------------------------------------- ------------- ------------------------------ ------------- ------------- ------- ------------- --- ------------- ------------- ------------- ------------- -------------
+DG02/DBTST/tempfile/temp1.278.834411957    5          TEMP1                   21474836480     2621440 ONLINE          1 NO           0         0           0   21473787904     2621312


1 row selected.
 

Installing OEM 12c on Linux

$
0
0
Oracle Enterprise Manager 12c cloud control installation


1. Architecture :



2. Background :

Oracle Management Agent (Management Agent/OMA)

The Management Agent is an integral software component that enables you to convert an un-managed host to a managed host in the Enterprise Manager system. The Management Agent works in conjunction with the plug-ins to monitor the targets running on that managed host.

Oracle Management Service (OMS)

The OMS is a Web-based application that orchestrates with the Management Agents and the Plug-Ins to discover targets, monitor and manage them, and store the collected information in a repository for future reference and analysis. The OMS also renders the user interface for Enterprise Manager Cloud Control. The OMS is deployed to the Oracle Middleware home (Middleware home), which is the parent directory that has the Oracle WebLogic Server home, the OMS home, the Management Agent home, the plug-in home, the Java Development Kit (JDK), the OMS instance base directory, the Oracle WT directory, the Oracle Common directory, and other relevant configuration files and directories. While deploying the OMS, the Enterprise Manager Cloud Control Installation Wizard installs Oracle WebLogic Server if it does not already exist in your environment. As a result, a WebLogic Server admin console is also installed.

Oracle Management Repository (Management Repository/OMR)

The Management Repository is a storage location where all the information collected by the Management Agent gets stored. It consists of objects such as database jobs, packages, procedures, views, and tablespaces.

The OMS uploads the monitoring data it receives from the Management Agents to the Management Repository. The Management Repository then organizes the data so that it can be retrieved by the OMS and displayed in the Enterprise Manager Cloud Control console. Since data is stored in the Management Repository, it can be shared between any number of administrators accessing the Enterprise Manager Cloud Control.

At the time of installation, the Enterprise Manager Cloud Control Installation Wizard configures the Management Repository in your existing, certified database. The wizard, however, does not install a new database.

Plug-Ins

Plug-Ins are pluggable entities that offer special management capabilities customized to suit specific target types. Unlike the earlier releases of Enterprise Manager, in Enterprise Manager Cloud Control, the plug-ins work in conjunction with the OMS and the Management Agent to monitor every target in your environment. Therefore, they are deployed to the OMS as well as the Management Agent. In the earlier releases, plug-ins enabled you to monitor only third-party targets, but in Enterprise Manager Cloud Control, plug-ins enable you to monitor all types of targets in your environment.

Plug-ins have independent release cycles, so every time you have a new version of an Oracle product released, you will have a new version of the plug-in released to support monitoring of that new product version in Enterprise Manager Cloud Control. This simplifies things because you no longer have to wait to upgrade your Enterprise Manager system to support a new product version; instead you can upgrade your plug-ins to monitor the new product version.

Enterprise Manager Cloud Control Console

The Enterprise Manager Cloud Control console is the user interface you see after you install Enterprise Manager Cloud Control. With the help of the console, you can monitor and administer your entire computing environment from one location on the network. All the systems and services including enterprise application systems, databases, hosts, middleware application servers, listeners, and so on, are easily managed from one central location.
 3. Pre-requisites OS/DB (OMS/OMR)


OMS Host : xxxxxemccomsdb
OMS version: 12.1.0.4.0
OMS Repository version: 12.1.0.1 (Should be created before OMS Installation)
OS Release : Red Hat Enterprise Linux Server release 6.5 (Santiago)

####
H/W
####

RAM : 16 GB
Swap: 4 GB
CPU : 4 core * 2848 MHz
Storage/Disk Space: 200 GB Total

150 GB :/u01 (OMS Repository/DB)
50  GB :/u02 (Oracle Grid, Weblogic , oracle home binaries)

######################
OS Package - Linux x86_64
######################

 glibc-common-2.3.4-2.43(x86_64)
glibc-devel-2.3.4-2.43(x86_64)
glibc-profile-2.3.4-2.43(x86_64)
glibc-headers-2.3.4-2.43(x86_64)
glibc-utils-2.3.4-2.43(x86_64)
glibc-2.3.4-2.43(x86_64)
make-3.81
binutils-2.20.51.0.2-5.11
gcc-4.4.4
libaio-0.3.107
glibc-common-2.12-1
libstdc++-4.4.4
libXtst-1.0.99.2-3.el6.x86_64.rpm
sysstat-9.0.4
glibc-devel-2.12-1.7.el6.x86_64 (This is a 64-bit package)
glibc-2.12-1.47.0.2

From warning:
glibc-devel-2.12-1.132.el6_5.2.i686

Libraries :
glibc-2.5.12


##################
Kernal Parameters
##################

/proc/sys/net/ipv4/ip_local_port_range  --> 1100 65500
kernel.shmmax parameter to 4294967295

Oracle recommends that you make these settings permanent.
For example, as root, use a text editor to open /etc/sysctl.conf, and add or change to the following: net.ipv4.ip_local_port_range = 11000 65500,
and then restart the network (# /etc/rc.d/init.d/network restart).
For detailed information on how to automate this ephemeral port range alteration on system restarts, refer to your Linux distribution system administration documentation.

##########
User/Group
##########

Groups :  oinstall(501)  dba(502)  asmadmin(504) asmdba(506) asmoper(507)
Users  :  grid(501)  ora11g(502)
Relation:

User      primary Group  Secondary group
----         -------------  ---------------
grid    oinstall       asmadmin,asmdba,asmoper
ora11g  oinstall       dba,asmdba


grid osuser whould have sudo access

############################
Port list between OMS & OMA
############################

11g

Below ports should be opened from OMAgent to OMS

HTTP Console Port   : 7789
HTTPS Console Port  : 7800
HTTP Upload Port    : 4890
HTTPS Upload Port   : 4900
Agent Port          : 3872
Repository Port     : 1527
Enterprise Manager Central Console Port = 4889
Enterprise Manager Central Console Secure Port = 1159



 
 

 12c

Admin Server HTTP SSL Port = 7101 - 7200  = 7102
Enterprise Manager Upload HTTP Port = 4889 - 4898 = 4889
Enterprise Manager Upload HTTP SSL Port = 1159, 4899 - 4908 = 4903
Enterprise Manager Central Console HTTP Port = 7788 - 7798 = 7788
Enterprise Manager Central Console HTTP SSL Port = 7799 - 7809 = 7802
Oracle Management Agent Port = 3872, 1830 - 1849 = 3872
Node Manager HTTP SSL Port = 7401 - 7500 = 7403
Managed Server HTTP Port = 7201 - 7300 = 7202
Managed Server HTTPS Port = 7301 - 7400 = 7301

DB ports = 1527 , 1521 , 1522 , 1526 , 1528
OMS/OMA Ports = 7102 , 4889 , 4903 , 7788 , 7802 , 3872 , 7403 , 7202 , 7301


###########
/etc/hosts
###########

formatting

12.123.123.12 mypc.cn.company.com mypc

## xxxxxemccomsdb

Xxxxxemccomsdb ---- xxxxx244.242.144/25   GW-xxxxx244.242.131     VLAN-841
Xxxxxemccomsdb-bkp ---- 10.132.98.15/23  VLAN-824

########
Weblogic
########

Weblogic Domain Name: GCDomain
Weblogic WebUser name: Weblogic 
pwd: O****3

Node manager User Name: nodemanager
pwd: O****3

https://xxxxxemccomsdb.abcd.co.in:7102/console -- Admin Console user: weblogic
https://xxxxxemccomsdb.abcd.co.in:7301/em -- Manged Server user : sysman

Agent Reachability:

https://xxxxxemccomsdb.abcd.co.in:3872/emd/main/

Lowest Supported OMA target DB is 9.2.0.8


   4. OEM 12c OMR Installation  

Install 12c oracle home & create database as a repository (OMR)






















5. OEM 12c OMS Deployment






ulimit -n 4096 set in .bash_profile





Installed glibc-devel-2.12-1.132.el6_5.2.i686
 
















Lesson for Life 1 : Work Life Balance

$
0
0
Starting with a series of Lesson for Life (LFL) on Lesson Learned page , stories based on personal experience & imaginary characters. LFL will give me opportunity to write something not related to technical skill (which is core of this blog) but will help readers in day-o-day life

                                                       
                Saiket , a computer engineer aged just less than 30 staying with his family including his wife Renuka , son Ansh in rented house in a City. Nearby to his house his parents as well as in-laws were staying. Having parents nearby was good source of support when Ansh was born and growing. Saiket was working for an IT firm over 5 yrs which is considered amongst the top industries.

Saiket & Renuka were awaiting for their possession of New owned house in new city which was due in early months of the coming new year and they were very much excited about the same with bunch of plans in mind.

                Saiket was considered amongst the top high performers of the project and always given high preference to work than anything else. Considering the same he was given added responsibility of transformation project in addition to his current assignment. Being a good opportunity & scope for knowledge and learning he couldn't deny the new responsibility.

Early in start of the new year the project commenced and he has started contributing to his current assignment & new transformation work deliberately. His working hours started increasing as the new project progressed. Saiket has started extending his support to the project throughout day & night. In a day time mostly he use to support his current assignment and in night which is considered as low business hours the transformation planned activity use to take place.

The exercise time got reduced & almost disappear , sleeping hours started reducing & use to get split in any time throughout a day as per project need. He has started becoming weak & thin unknowingly.

                One day in afternoon he got a call from Renuka (wife) informing about the extension of possession date of their new house by more than a year. It was heartbreaking as their have been two extensions happened in past and this year they were assured to get keys.

Devastated by the news Saiket & Renuka has started preparing new plans. Plan was to move to the new city in rented house and nearby location where new owned but awaited possession house was located from next month onward itself to benefit their son Ansh who was entering in his school age from middle of the year.

The new city was far away from Saiket's office & he use to spend over 3 hrs. a day in transportation. Saiket & Renuka took admission for their Son in new city's reputed school which would allowed their kid to learn without interruption due to change in school. School was at equal & nearby distance from their new city's currently rented as well as owned house whose possession  was awaited/extended.

It was less than a month post shifting to new rented house in new city the transformation project missed the deadline and went into the high pressure situation wherein multiple setup to be migrated in a weeks time. Saiket who was not completely settled in new rented house & with his transportation schedule has to work from home to support multiple & ad hoc project's requests.

                The new place were surrounded by ongoing construction site and their new owned house was also one of them & mostly favorable place for mosquito menace. Such conditions were not new for Saiket  & Renuka and they have made all attempts to stay protected from mosquito or water / air born disease.

The new rented house in new city had a network coverage issue for internet service and same was only available in balcony / dry area space.

It was a Thursday evening , Saiket returned to home early after day long hectic work hours not to get himself relaxed but to prepare for critical transformation activity in night. Saiket has spend whole night sitting in balcony (due to n/w coverage issue inside house) and completed activity just prior sunrise. Soon he planned to get a sound sleep almost after 24 hrs. since last sleep as he was suppose to support one more activity next night.

However after 60 min his phone rang reporting a performance issue in migrated services for which he spend complete night. Saiket quickly got onto laptop & started troubleshooting the issue it became afternoon till issue got fixed. He had his lunch in evening itself. Saiket again planned for a sleep as it was over 35 hrs. since he got a last continuous sleep. This time he was unable to sleep because of completely destructive schedule of meal and sleep.

               Saiket again made up his mind and prepared himself for the Friday night activity which was the last activity of project and post which transformation project was getting accomplished. Keeping this moral he has again sat out in balcony with hardly any energy left and with minor back pain , he completed his activity prior sunrise.

Saiket went to sleep fully satisfied with his efforts in last 2-3 days as well over few months that he has spend towards new and ongoing assignment. Mailbox was flooding with appreciations when he was asleep.

After few hours of sleep the minor back pain turned to severe pain along with body temperature reaching to 103F. Saiket got up and thought of going out in living room where he slept for few minutes but the pain & fever turned unbearable. he informed his wife about the same and asked to take him to doctor.

Being new in the city both Saiket & Renuka were only aware of few nearby doctors and they planned to visit the same. Saiket was lack of energy due to last 3-4 days schedule & fever and was too weak to walk too . Renuka took Saiket to nearby clinic however it was closed as doctor was on vacation.

Saiket who was hardly able to sit , asked his wife to search any other clinic nearby and again comeback to the place outside closed clinic where Saiket decided to wait. Renuka found a hospital and took Saiket to the reception , they got the doctor appointment and further checkups. looking at the weakness and constant high fever the doctor suggested to get admitted.

               Renuka & Saiket felt happy when they found this hospital where it was hard to get clinic , however the hospital looked lonely without much patient.

Treatment started and blood report came suggesting nothing clearly but doctor suspected its initial stage of malaria. Saiket & Wife also got convinced as he use to sat down for activity outdoor due to network coverage issue and there was possibility of mosquito bites that his weak health couldn't defend.

Satisfied with the diagnosis , treatment continued over next 24 hrs. but Saiket's health getting worsen. Early morning of Sunday it became unbearable for Saiket and hinted for change in hospital as he was facing issue during breathing and started almost collapsing.

              Saiket's dad & Renuka quickly took discharge , with such worsen condition of Saiket who has started loosing his confidence in health and started whispering to Renuka that "today could be the last day of his life & to take care of Ansh". Threatened by his words Renuka & Saiket's Dad requested for an ambulance from the old city's reputed hospital where they use to leave prior shifting.

Ambulance came in quickly & took Saiket to the old city's hospital on life support. Saiket's office friend ran to support his wife & Dad and took in-charge of all hospital formalities. Next few hours were critical for Saiket who was put on life support in ICCU. Saiket's family , friend awaited over 12 hrs. outside the ICCU with keeping a glance through a small circle glass.

              Doctor has diagnosed the Dengue in early report itself and started the treatment , the blood reports were showing very dip down in blood platelet counts whose function is to stop bleeding by clumping and clogging blood vessel injuries. Saiket was removed from life support post stability and moved to internal patient bed giving relief to family & friends.

Saiket started recovering , every day his family members and office friends use to visit and ask him "What went wrong?" and he has to repeat the whole above story as a cause for his condition.

             Saiket recovered & escaped from horrifying disease. it took over 15 days to get his strength back & started his new life but this time with Work-Life balance !

CLSRSC-218: Failed to install the OHASD startup script CLSRSC-317: Failed to register Oracle OHASD service

$
0
0
Problem:

During Standalone 12c oracle grid installation it has been observed that root.sh is failing on SUSE Linux 11 due to errors CLSRSC-218 & CLSRSC-317

$ /rdbms/app/12.1.0.2/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= ora11g
    ORACLE_HOME=  /rdbms/app/12.1.0.2/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /rdbms/app/12.1.0.2/grid/crs/install/crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'ora11g', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node testlab2 successfully pinned.
2015/09/14 12:11:01 CLSRSC-218: Failed to install the OHASD startup script (error: 25)

2015/09/14 12:11:01 CLSRSC-317: Failed to register Oracle OHASD service

Died at /rdbms/app/12.1.0.2/grid/crs/install/crsohasd.pm line 138.
The command '/rdbms/app/12.1.0.2/grid/perl/bin/perl -I/rdbms/app/12.1.0.2/grid/perl/lib -I/rdbms/app/12.1.0.2/grid/crs/install /rdbms/app/12.1.0.2/grid/crs/install/roothas.pl ' execution failed

/tmp/deinstall2015-09-14_00-38-39PM/perl/bin/perl -I/tmp/deinstall2015-09-14_00-38-39PM/perl/lib -I/tmp/deinstall2015-09-14_00-38-39PM/crs/install /tmp/deinstall2015-09-14_00-38-39PM/crs/install/roothas.pl -force  -deconfig -paramfile "/tmp/deinstall2015-09-14_00-38-39PM/response/deinstall_OraGI12Home1.rsp"
 


Solution :

Mine below logs probably roothas.log in below locations ,

1. <oraInventory_location>/logs/* installation logs
2. <GRID_HOME>/cfgtoollogs/crsconfig/root*.log


Will find problem causing resource service name highlighted below,

>  sles-release-11.2-1.234
>End Command output
2015-09-14 15:23:31: Executing cmd: /bin/rpm -q sles-release
2015-09-14 15:23:31: Command output:
>  sles-release-11.2-1.234
>End Command output
2015-09-14 15:23:31: Executing cmd: /usr/lib/lsb/install_initd /etc/init.d/ohasd
2015-09-14 15:23:31: Command output:
>  insserv: script netbackup.01-28-15.18:48:17: service netbackup already provided!
>  insserv: warning: script 'init.ohasd' missing LSB tags and overrides
>  insserv: Default-Start undefined, assuming default start runlevel(s) for script `init.ohasd'
>  insserv: script netbackup.07-07-14.15:50:05: service netbackup already provided!
>  insserv: There is a loop between service hostagent and init.ohasd if stopped
>  insserv:  loop involving service init.ohasd at depth 2
>  insserv:  loop involving service hostagent at depth 1
>  insserv: Stopping init.ohasd depends on hostagent and therefore on system facility `$all' which can not be true!>  insserv: exiting now without changing boot order!
>End Command output
2015-09-14 15:23:31: Executing cmd: /rdbms/app/oracle/12.1.0.2/db/bin/clsecho -p has -f clsrsc -m 218 "25"
2015-09-14 15:23:31: Command output:
>  CLSRSC-218: Failed to install the OHASD startup script (error: 25)
>End Command output
2015-09-14 15:23:31: Executing cmd: /rdbms/app/oracle/12.1.0.2/db/bin/clsecho -p has -f clsrsc -m 218 "25"
2015-09-14 15:23:31: Command output:
>  CLSRSC-218: Failed to install the OHASD startup script (error: 25)
>End Command output


Here ,  init.ohasd configuration is problematic which can be fixed by below changes in /etc/init.d/init.ohasd

Modify below line of /etc/init.d/init.ohasd from 

#   Required-Start: $all
#   Required-Stop: $all
 


to

#   Required-Start: $network $syslog
#   Required-Stop: $network $syslog


And re-run root.sh

Also refer /rdbms/app/oracle/12.1.0.2/db/crs/init/ohasd.sles i.e. $ORACLE_HOME/crs/init/ohasd.sles incase such changes are required

Resolved:

$sudo /rdbms/app/oracle/12.1.0.2/db/root.sh
ora11g's password:
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= ora11g
    ORACLE_HOME=  /rdbms/app/oracle/12.1.0.2/db

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user:
/rdbms/app/oracle/12.1.0.2/db/perl/bin/perl -I/rdbms/app/oracle/12.1.0.2/db/perl/lib -I/rdbms/app/oracle/12.1.0.2/db/crs/install /rdbms/app/oracle/12.1.0.2/db/crs/install/roothas.pl

To configure Grid Infrastructure for a Cluster execute the following command as ora11g user:
/rdbms/app/oracle/12.1.0.2/db/crs/config/config.szh
This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent operation, and the parameters can be passed through the response file that is available in the installation media.

/tmp/deinstall2015-09-15_11-52-44AM/perl/bin/perl -I/tmp/deinstall2015-09-15_11-52-44AM/perl/lib -I/tmp/deinstall2015-09-15_11-52-44AM/crs/install /tmp/deinstall2015-09-15_11-52-44AM/crs/install/roothas.pl -force  -deconfig -paramfile "/tmp/deinstall2015-09-15_11-52-44AM/response/deinstall_OraGI12Home1.rsp"


Success !?

Reference -  SUSE Linux: root.sh fails with: Failed to install ohasd startup script, error: Inappropriate ioctl for device (Doc ID 1485604.1)

CRS-4123 CRS-4124 CRS-4000

$
0
0

Issue:   cluster start command "crsctl start crs"  fails to start crs services

CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-4124: Oracle High Availability Services startup failed
CRS-4000: Command Start failed, or completed with errors


Cause:

Recent changes to cluster hosts ...

  1. OS patching 
  2. Network subnet changes 
  3. OS reboot  
  4. Files system permission changes 
  5. ASM disk unavailability
  6. private network unreliability 


Possible Solution:


1. Verify if cluster interconnect is reachable : Obtain private interconnect IP from /etc/hosts and check ping response from all cluster nodes

2. Compare ifconfig -a output before & after OS changes to validate if Ethernet naming (e.g. eth0/eth1) has not changed , in case earlier o/p is not available second running node can also be referred


3. Verify ASM disk availability using kfod disks=all 


4. if OS kernal has been upgraded as a recent change take strace o/p as below

sudo strace $GRID_HOME/bin/crsctl start crs

If below errors observed in trace files then possible cause is OHASD process is not able to spawn the crs start
demons and stucked in "ohasd run" , in this scenario either /var/tmp/.oracle , /tmp/.oracle , /usr/tmp/.oracle need to be backed up and cleaned up followed by clean OS and crs restart

sendto(23, "\4", 1, MSG_NOSIGNAL, NULL, 0) = 1

connect(27, {sa_family=AF_LOCAL, sun_path="/var/tmp/.oracle/sprocr_local_conn_0_PROL"}, 110) = -1 E
ioctl(20, FIONBIO, [1])                 = 0

connect(20, {sa_family=AF_LOCAL, sun_path="/var/tmp/.oracle/sOHASD_UI_SOCKET"}, 110) = -1 ENOENT (No such file or directory)

5. if None of the above steps work then need to rollback the recently upgraded kernal or OS patch


6. You may use cluvfy stage -pre crsinst -n node1,node2  as a part of troubleshooting

Refer:

Top 5 Grid Infrastructure Startup Issues (Doc ID 1368382.1)

ORA-01722: invalid number

$
0
0

Issue :

ORA-01722: invalid number


Cause:

I. Attempt has been made to insert a character value in integer datatype column

SQL> create table mytest ( one integer , two varchar2(10));

Table created.

SQL> desc mytest
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ONE                                                NUMBER(38)
 TWO                                                VARCHAR2(10)

SQL> insert into mytest values ( 'test' , 'test');
insert into mytest values ( 'test' , 'test')
                            *
ERROR at line 1:
ORA-01722: invalid number

Solution:

Use integer value to insert into integer datatype column or varchar column

SQL>  insert into mytest values ( 1 , 'test');


1 row created.

SQL> insert into mytest values ( 1 , 1);


1 row created.

SQL> commit;

Commit complete.

SQL> select * from mytest;

       ONE TWO
---------- ----------
         1 test
         1 1

II. Same error may also appear if conversion function is used against inappropriate data type columns 

SQL> select to_number(TWO) from mytest;
select to_number(TWO) from mytest
       *
ERROR at line 1:
ORA-01722: invalid number



SQL> select to_number(ONE) from mytest;

TO_NUMBER(ONE)
--------------
             1
             1




steps to create active data guard in 11g and up-gradation to 12c using manual method

$
0
0
Steps to Create 11 ADG and up-gradation to 12c using manual method

 10 steps will take you through systematic approach to create standby database/ADG (active data guard) followed by activation and up-gradation to 12c using manual method

    1.  11g dummy instance creation for ADG
     2.  Dummy instance cleanup and preparation for rman restore to create standby
     3.  RMAN level 0 restoration initiation
     4.  ADG parameter setting
     5.  Restore archives and apply to standby
     6.  Activating standby
     7.  Pre-requisites of 12c upgrade
     8.  Initiate 12c upgrade
     9.  Time zone upgrade
    10. Post upgrade steps
     A.  Issues Faced

Above approach is best suited to reduce overall downtime involved to application and DB upgrade can be performed within 6 Hrs.

Please refer the setup as below 

testlabdb005a/testlabdb005b -: New two node RAC database hosts to be used for 11g restore/ADG/Upgrade to 12c

testoldlabdb1/testoldlabdb12 -: Old two node RAC database with 11g database TESTDBPRD which acts as a primary db node

TESTDBPRD :- Old 11g Database global name , name to be preserved in ADG
NEWDBPRD :- New 11g ADG database unique name , older name will be preserved as global dbname

 1.   11g dummy instance creation for ADG

We will create a dummy database instance to get db service /ASM/ TNS configuration readily available

[oracle@testlabdb005a ~]$ . ./.bash_profile_11g
[oracle@testlabdb005a ~]$ which sqlplus
/golden/app/oracle/11.2.0.3.0/db/bin/sqlplus
[oracle@testlabdb005a ~]$  export DISPLAY=164.16.18.55:10.0
[oracle@testlabdb005a ~]$ dbca



Oracle RAC Application cluster to be selected  

Supply appropriate physical host name and select all followed by ASM diskgroup selection for Data and FRA location

Set appropriate SGA/PGA and process values



Database instance is available on both hosts now

[oracle@testlabdb005a ~]$ ps -ef | grep -i pmon
oracle    5227     1  0 Mar24 ?        00:04:13 asm_pmon_+ASM1
oracle   15041     1  0 08:23 ?        00:00:00 ora_pmon_NEWDBPRD1
oracle   15479 48767  0 08:23 pts/2    00:00:00 grep --color=auto -i pmon
[oracle@testlabdb005a ~]$ ssh testlabdb005b ps -ef | grep -i pmon
oracle   14788     1  0 Jan27 ?        00:08:15 asm_pmon_+ASM2
oracle   15503     1  0 Jan27 ?        00:06:08 mdb_pmon_-MGMTDB
oracle   17232     1  0 08:23 ?        00:00:00 ora_pmon_NEWDBPRD2


2. Dummy instance cleanup and preparation for rman level 0 restore to create standby


Dummy instance created in step 1 will be cleaned to make way for standby database to be restored from primary 

[oracle@testlabdb005a ~]$ srvctl stop instance -d TESTDBPRD -i NEWDBPRD2

[oracle@testlabdb005a ~]$ srvctl status database -d TESTDBPRD
Instance NEWDBPRD1 is running on node testlabdb005a
Instance NEWDBPRD2 is not running on node testlabdb005b

[oracle@testlabdb005a ~]$ . ./.bash_profile_11g
[oracle@testlabdb005a ~]$ sqlplus "/ as sysdba"
SQL*Plus: Release 11.2.0.3.0 Production on Wed JAN 10 08:26:04 2017
sys@TESTDBPRD> show parameter control
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
control_file_record_keep_time        integer     7
control_files    string  +DATA/TESTDBPRD/controlfile/current.370.943604411, +FRA/testdbprd/controlfile/current.358.943604411
control_management_pack_access       string      DIAGNOSTIC+TUNING

sys@TESTDBPRD> select * from v$log;
 GROUP#  THREAD#  SEQUENCE# BYTES  BLOCKSIZE MEMBERS ARC STATUS FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# NEXT_TIME
------ ------- ------- ------ ------- ------  ------- ----- ------ ---- -------
 1     1   7 52428800  512  2 NO INACTIVE 1026240  10-JAN-17 1031635 10-JAN-17
 2     1   8   52428800  512 2 NO  CURRENT 1031635 10-JAN-17 2.8147E+14
 3     2   1   52428800  512 2 NO  CURRENT 1033793 10-JAN-17 2.8147E+14 10-JAN-17
 4     2   0   52428800  512  2 YES UNUSED    0                     0


sys@TESTDBPRD> show parameter pfile
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +DATA/TESTDBPRD/spfileTESTDBPRD.ora

[oracle@testlabdb005a ~]$ srvctl stop instance -d TESTDBPRD -i NEWDBPRD1
[oracle@testlabdb005a ~]$ srvctl status database -d TESTDBPRD
Instance NEWDBPRD1 is not running on node testlabdb005a
Instance NEWDBPRD2 is not running on node testlabdb005b

[oracle@testlabdb005a ~]$ . ./.bash_profile_asm
[oracle@testlabdb005a ~]$ asmcmd

ASMCMD> lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N 512 4096  1048576 2150400  2148600  0 2148600  0 N  DATA/
MOUNTED  EXTERN  N 512 4096 1048576 307200 306637    0 306637     0 N  FRA/
MOUNTED  EXTERN  N 512 4096  1048576 10240 5611 0 5611 0   Y  OCR_VOTING/
ASMCMD> cd DATA/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type  Redund  Striped  Time             Sys  Name
                                        Y    TESTDBPRD/
                                        Y    NEWDBPR/
                                        N    NEWDBggp/
ASMCMD> cd TESTDBPRD/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type           Redund  Striped  Time             Sys  Name
                                                 Y    CONTROLFILE/
                                                 Y    DATAFILE/
                                                 Y    ONLINELOG/
                                                 Y    PARAMETERFILE/
                                                 Y    TEMPFILE/
PARAMETERFILE  UNPROT  COARSE   JAN 10 08:00:00  N    spfileTESTDBPRD.ora => +DATA/TESTDBPRD/PARAMETERFILE/spfile.280.943604577
ASMCMD> cd DATAFILE/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type      Redund  Striped  Time             Sys  Name
DATAFILE  UNPROT  COARSE   JAN 10 08:00:00  Y    SYSAUX.282.943604343
DATAFILE  UNPROT  COARSE   JAN 10 08:00:00  Y    SYSTEM.281.943604343
DATAFILE  UNPROT  COARSE   JAN 10 08:00:00  Y    UNDOTBS1.283.943604343
DATAFILE  UNPROT  COARSE   JAN 10 08:00:00  Y    UNDOTBS2.279.943604457
DATAFILE  UNPROT  COARSE   JAN 10 08:00:00  Y    USERS.374.943604343
ASMCMD> rm *
You JAN delete multiple files and/or directories.
Are you sure? (y/n) y
ASMCMD> cd +DATA/
ASMCMD> ls -lt
Type  Redund  Striped  Time             Sys  Name
                                        N    NEWDBggp/
                                        Y    NEWDBPR/
                                        Y    TESTDBPRD/
ASMCMD> cd TESTDBPRD/
ASMCMD> ls -lt
Type           Redund  Striped  Time             Sys  Name
                                                 Y    TEMPFILE/
                                                 Y    PARAMETERFILE/
                                                 Y    ONLINELOG/
                                                 Y    CONTROLFILE/
PARAMETERFILE  UNPROT  COARSE   JAN 10 08:00:00  N    spfileTESTDBPRD.ora => +DATA/TESTDBPRD/PARAMETERFILE/spfile.280.943604577

ASMCMD> cd TEMPFILE/
ASMCMD> ls -lt
Type      Redund  Striped  Time             Sys  Name
TEMPFILE  UNPROT  COARSE   JAN 10 08:00:00  Y    TEMP.377.943604415
ASMCMD> rm *
You JAN delete multiple files and/or directories.
Are you sure? (y/n) y


ASMCMD> cd ONLINELOG/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type  Redund  Striped  Time             Sys  Name
                                        Y    TESTDBPRD/
                                        Y    NEWDBPR/
                                        N    NEWDBggp/
ASMCMD> cd TESTDBPRD/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type           Redund  Striped  Time             Sys  Name
                                                 Y    CONTROLFILE/
                                                 Y    ONLINELOG/
                                                 Y    PARAMETERFILE/
PARAMETERFILE  UNPROT  COARSE   JAN 10 08:00:00  N    spfileTESTDBPRD.ora => +DATA/TESTDBPRD/PARAMETERFILE/spfile.280.943604577
ASMCMD> cd ONLINELOG/

ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type       Redund  Striped  Time             Sys  Name
ONLINELOG  UNPROT  COARSE   JAN 10 08:00:00  Y    group_1.371.943604413
ONLINELOG  UNPROT  COARSE   JAN 10 08:00:00  Y    group_2.375.943604415
ONLINELOG  UNPROT  COARSE   JAN 10 08:00:00  Y    group_3.278.943604577
ONLINELOG  UNPROT  COARSE   JAN 10 08:00:00  Y    group_4.424.943604577

ASMCMD> rm *
You JAN delete multiple files and/or directories.
Are you sure? (y/n) y

ASMCMD> cd +DATA/
ASMCMD> cd TESTDBPRD/
ASMCMD> ls -lt
Type           Redund  Striped  Time             Sys  Name
                                                 Y    PARAMETERFILE/
                                                 Y    CONTROLFILE/
PARAMETERFILE  UNPROT  COARSE   JAN 10 08:00:00  N    spfileTESTDBPRD.ora => +DATA/TESTDBPRD/PARAMETERFILE/spfile.280.943604577
ASMCMD> cd CONTROLFILE/
ASMCMD> ls -lt
Type         Redund  Striped  Time             Sys  Name
CONTROLFILE  UNPROT  FINE     JAN 10 08:00:00  Y    Current.370.943604411
ASMCMD> rm *
You JAN delete multiple files and/or directories.
Are you sure? (y/n) y
ASMCMD> lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N  512 4096  1048576  2150400  2150227 0 2150227 0 N  DATA/
MOUNTED  EXTERN  N  512 4096  1048576  307200   306637  0 306637 0  N  FRA/
MOUNTED  EXTERN  N  512 4096  1048576  10240     5611   0  5611 0 Y OCR_VOTING/
ASMCMD> cd +DATA/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'

Type  Redund  Striped  Time             Sys  Name
                                        Y    TESTDBPRD/
                                        Y    NEWDBPR/
                                        N    NEWDBggp/
ASMCMD> cd NEWDBggp/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type  Redund  Striped  Time             Sys  Name
                                        N    onlinelog/
                                        N    tempfile/
ASMCMD> cd ..
ASMCMD> rm -rf NEWDBggp/
ASMCMD> pwd
+DATA
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type  Redund  Striped  Time             Sys  Name
                                        Y    TESTDBPRD/
                                        Y    NEWDBPR/
ASMCMD> cd NEWDBPR/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type  Redund  Striped  Time             Sys  Name
                                        Y    CONTROLFILE/
                                        Y    PARAMETERFILE/
                                        Y    PASSWORD/
ASMCMD> cd CONTROLFILE/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'

Type         Redund  Striped  Time             Sys  Name
CONTROLFILE  UNPROT  FINE     NOV 29 10:00:00  Y    Current.261.925398497
ASMCMD> rm *
You JAN delete multiple files and/or directories.
Are you sure? (y/n) y
ASMCMD> cd ..
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type  Redund  Striped  Time             Sys  Name
                                        Y    PARAMETERFILE/
                                        Y    PASSWORD/
ASMCMD> cd PARAMETERFILE/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'

Type           Redund  Striped  Time             Sys  Name
PARAMETERFILE  UNPROT  COARSE   JAN 27 14:00:00  Y    spfile.268.925398709
ASMCMD> rm *
You JAN delete multiple files and/or directories.
Are you sure? (y/n) y
ASMCMD> cd ..
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'
Type  Redund  Striped  Time             Sys  Name
                                        Y    PASSWORD/
ASMCMD> cd PASSWORD/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'

Type      Redund  Striped  Time             Sys  Name
PASSWORD  UNPROT  COARSE   OCT 16     2016  Y    pwdNEWDBpr.256.925398277
ASMCMD> rm *
You JAN delete multiple files and/or directories.
Are you sure? (y/n) y
ASMCMD> cd ..
ASMCMD-8002: entry 'NEWDBPR' does not exist in directory '+DATA/'
ASMCMD> pwd
+DATA/NEWDBPR/PASSWORD
ASMCMD> cd +DATA
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'

Type  Redund  Striped  Time             Sys  Name
                                        Y    TESTDBPRD/
ASMCMD> cd TESTDBPRD/
ASMCMD> ls -lrt
WARNING:option 'r' is deprecated for 'ls'
please use 'reverse'

Type           Redund  Striped  Time             Sys  Name
                                                 Y    PARAMETERFILE/
PARAMETERFILE  UNPROT  COARSE   JAN 10 08:00:00  N    spfileTESTDBPRD.ora => +DATA/TESTDBPRD/PARAMETERFILE/spfile.280.943604577
ASMCMD> pwd
+DATA/TESTDBPRD
ASMCMD> exit

[oracle@testlabdb005a ~]$ . ./.bash_profile_11g
[oracle@testlabdb005a ~]$ sqlplus "/ as sysdba"
Connected to an idle instance.
idle> startup nomount
ORACLE instance started.
Total System Global Area 4.8103E+10 bytes
Fixed Size                  2239376 bytes
Variable Size            4429186160 bytes
Database Buffers         4.3621E+10 bytes
Redo Buffers               50479104 bytes

idle> show parameter control
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
control_file_record_keep_time        integer     7
control_files                        string      +DATA/TESTDBPRD/controlfile/curr
                                                 ent.370.943604411,
+FRA/testdbprd/controlfile/current.358.943604411
control_management_pack_access       string      DIAGNOSTIC+TUNING


  3.   RMAN level 0 restoration initiation


This step will restore database from primary db node to be converted into standby database

Standby control file backup to be taken from old database


[ora11g@testoldlabdb1 ~]$ . ./.bash_profile_TESTADG1
[ora11g@testoldlabdb1 ~]$ rman target /

Recovery Manager: Release 11.2.0.3.0 - Production on Wed JAN 10 14:05:44 2017
Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.
connected to target database: TESTDBPRD (DBID=1110983454)

RMAN> backup current controlfile for standby format 'stby_ind.cnf';
Starting backup at 10-JAN-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=2285 instance=TESTADG1 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including standby control file in backup set
channel ORA_DISK_1: starting piece 1 at 10-JAN-17
channel ORA_DISK_1: finished piece 1 at 10-JAN-17
piece handle=/oravl01/ora11g/112_RAC/dbs/stby_ind.cnf tag=TAG20170510T140608 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 10-JAN-17
RMAN>

[ora11g@testoldlabdb1 ~]$ scp -P 22 /oravl01/ora11g/112_RAC/dbs/stby_ind.cnf oracle@164.16.18.55:/export/RMANFULLBACK_09JAN17
oracle@164.16.18.55's password:
stby_ind.cnf                100%   64MB   5.3MB/s   00:12


[oracle@testlabdb005a RMANFULLBACK_09JAN17]$ rman target /
Recovery Manager: Release 11.2.0.3.0 - Production on Wed JAN 10 08:46:02 2017
connected to target database: TESTDBPRD (not mounted)

RMAN> restore standby controlfile from '/export/RMANFULLBACK_09JAN17/stby_ind.cnf';
Starting restore at 10-JAN-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=784 instance=NEWDBPRD1 device type=DISK
channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
output file name=+DATA/TESTDBPRD/controlfile/current.256.943606073
output file name=+FRA/TESTDBPRD/controlfile/current.358.943604411
Finished restore at 10-JAN-17
RMAN>

SQL> alter database mount;
Database altered.

[oracle@testlabdb005a RMANFULLBACK_09JAN17]$ rman target /
Recovery Manager: Release 11.2.0.3.0 - Production on Wed JAN 10 08:49:37 2017
connected to target database: TESTDBPRD (DBID=1110983454, not open)
RMAN>

[oracle@testlabdb005a RMANFULLBACK_09JAN17]$ pwd
/export/RMANFULLBACK_09JAN17
[oracle@testlabdb005a RMANFULLBACK_09JAN17]$ ls -lrt
total 305906748
-rw-------. 1 oracle oinstall   137297920 JAN  9 23:51 ctl_TESTDBPRD_obs3rn3c_1_1
-rw-------. 1 oracle oinstall  6895183872 JAN 10 00:03 dbpnps3rfrr_1_1
-rw-------. 1 oracle oinstall  6973827584 JAN 10 00:14 dbpnqs3rfrs_1_1
-rw-------. 1 oracle oinstall  7141895680 JAN 10 00:27 dbpnrs3rfrs_1_1
-rw-------. 1 oracle oinstall  5915882496 JAN 10 00:36 dbpnss3rfrs_1_1
-rw-------. 1 oracle oinstall  6487970816 JAN 10 00:47 dbpnts3rfrs_1_1
-rw-------. 1 oracle oinstall   223815168 JAN 10 00:47 dbpnus3rgds_1_1
-rw-------. 1 oracle oinstall 26752770048 JAN 10 01:30 dbpo0s3rggt_1_1
-rw-------. 1 oracle oinstall 28019957760 JAN 10 02:16 dbpo1s3rggu_1_1
-rw-------. 1 oracle oinstall 29240369152 JAN 10 03:02 dbpo2s3rggu_1_1
-rw-------. 1 oracle oinstall 27524194304 JAN 10 03:46 dbpo3s3rggu_1_1
-rw-------. 1 oracle oinstall 26314809344 JAN 10 04:27 dbpo4s3rggu_1_1
-rw-------. 1 oracle oinstall 26202578944 JAN 10 05:09 dbpo5s3rjcc_1_1
-rw-------. 1 oracle oinstall 27313512448 JAN 10 05:54 dbpo6s3rje4_1_1
-rw-------. 1 oracle oinstall 26819428352 JAN 10 06:37 dbpo7s3rjhd_1_1
-rw-------. 1 oracle oinstall 30237417472 JAN 10 07:29 dbpo8s3rjig_1_1
-rw-------. 1 oracle oinstall 30980997120 JAN 10 08:23 dbpo9s3rjp8_1_1
-rw-------. 1 oracle oinstall    66584576 JAN 10 08:37 stby_ind.cnf
drwxr-xr-x. 3 oracle oinstall        4096 JAN 10 09:04 archive
[oracle@testlabdb005a RMANFULLBACK_09JAN17]$ pwd
/export/RMANFULLBACK_09JAN17
[oracle@testlabdb005a RMANFULLBACK_09JAN17]$ rman target /
Recovery Manager: Release 11.2.0.3.0 - Production on Wed JAN 10 09:49:30 2017
connected to target database: TESTDBPRD (DBID=1110983454, not open)
RMAN> catalog start with '/export/RMANFULLBACK_09JAN17';
Starting implicit crosscheck backup at 10-JAN-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
Crosschecked 1227 objects
Finished implicit crosscheck backup at 10-JAN-17
Starting implicit crosscheck copy at 10-JAN-17
using channel ORA_DISK_1
Crosschecked 2 objects
Finished implicit crosscheck copy at 10-JAN-17
searching for all files in the recovery area
cataloging files...
no files cataloged
searching for all files that match the pattern /export/RMANFULLBACK_09JAN17
List of Files Unknown to the Database
=====================================
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_GG_tts3stid_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_GG_tus3stjk_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_clone_nvs3r8qq_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/GG_NOTREQ/arch_ggg_tjs3r281_1_1.gz
File Name: /export/RMANFULLBACK_09JAN17/archive/GG_NOTREQ/arch_ggg_tks3r2gm_1_1.gz
.
.

File Name: /export/RMANFULLBACK_09JAN17/dbpnts3rfrs_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpnus3rgds_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo0s3rggt_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo1s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo2s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo3s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo4s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo5s3rjcc_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo6s3rje4_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo7s3rjhd_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo8s3rjig_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo9s3rjp8_1_1

Do you really want to catalog the above files (enter YES or NO)? Y
cataloging files...
cataloging done

List of Cataloged Files
=======================
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_clone_nvs3r8qq_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_frs3r0ap_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_fss3r0ni_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_fts3r13l_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_fus3r1gf_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_fvs3r1st_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_g0s3rasb_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_g1s3rb8o_1_1
File Name: /export/RMANFULLBACK_09JAN17/archive/arch_adg_g2s3rbkr_1_1
.
.

File Name: /export/RMANFULLBACK_09JAN17/dbpnus3rgds_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo0s3rggt_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo1s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo2s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo3s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo4s3rggu_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo5s3rjcc_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo6s3rje4_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo7s3rjhd_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo8s3rjig_1_1
File Name: /export/RMANFULLBACK_09JAN17/dbpo9s3rjp8_1_1

RMAN>


    rman_DB_0_level_clone_restore.sh

RUN {
configure device type disk parallelism 5;
ALLOCATE CHANNEL ch01 DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch02  DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch03  DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch04  DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch05  DEVICE TYPE DISK ;
set newname for datafile 1 to '+DATA';
set newname for datafile 2 to '+DATA';
set newname for datafile 3 to '+DATA';
set newname for datafile 4 to '+DATA';
.
.
.
set newname for datafile 97 to '+DATA';
restore database;
switch datafile all;
RELEASE CHANNEL ch01;
RELEASE CHANNEL ch02;
RELEASE CHANNEL ch03;
RELEASE CHANNEL ch04;
RELEASE CHANNEL ch05;
}


o/p:

new RMAN configuration parameters:
CONFIGURE DEVICE TYPE DISK PARALLELISM 5 BACKUP TYPE TO BACKUPSET;
new RMAN configuration parameters are successfully stored

allocated channel: ch01
channel ch01: SID=287 instance=NEWDBPRD1 device type=DISK
allocated channel: ch02
channel ch02: SID=358 instance=NEWDBPRD1 device type=DISK
allocated channel: ch03
channel ch03: SID=431 instance=NEWDBPRD1 device type=DISK
allocated channel: ch04
channel ch04: SID=501 instance=NEWDBPRD1 device type=DISK
allocated channel: ch05
channel ch05: SID=572 instance=NEWDBPRD1 device type=DISK

executing command: SET NEWNAME
executing command: SET NEWNAME
executing command: SET NEWNAME
executing command: SET NEWNAME
executing command: SET NEWNAME
executing command: SET NEWNAME
executing command: SET NEWNAME
..
.
.
.
channel ch01: starting datafile backup set restore
channel ch01: specifying datafile(s) to restore from backup set
channel ch01: restoring datafile 00001 to +DATA
channel ch01: restoring datafile 00018 to +DATA
channel ch01: restoring datafile 00020 to +DATA
channel ch01: restoring datafile 00021 to +DATA
channel ch01: restoring datafile 00022 to +DATA
channel ch01: restoring datafile 00023 to +DATA
channel ch01: restoring datafile 00024 to +DATA
channel ch01: restoring datafile 00029 to +DATA
channel ch01: restoring datafile 00056 to +DATA
channel ch01: restoring datafile 00062 to +DATA
channel ch01: restoring datafile 00074 to +DATA
channel ch01: restoring datafile 00083 to +DATA
channel ch01: restoring datafile 00086 to +DATA
channel ch01: restoring datafile 00091 to +DATA
channel ch01: reading from backup piece /export/RMANFULLBACK_09JAN17/dbpo8s3rjig_1_1
channel ch02: starting datafile backup set restore
channel ch02: specifying datafile(s) to restore from backup set
channel ch02: restoring datafile 00002 to +DATA
channel ch02: restoring datafile 00009 to +DATA
channel ch02: restoring datafile 00034 to +DATA
channel ch02: restoring datafile 00039 to +DATA
channel ch02: restoring datafile 00040 to +DATA
channel ch02: restoring datafile 00051 to +DATA
channel ch02: restoring datafile 00054 to +DATA
channel ch02: restoring datafile 00067 to +DATA
.
.
.
.

input datafile copy RECID=193 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/usr.264.943615617
datafile 91 switched to datafile copy
input datafile copy RECID=194 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/usr.394.943611157
datafile 92 switched to datafile copy
input datafile copy RECID=195 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/usr.260.943611157
datafile 93 switched to datafile copy
input datafile copy RECID=196 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/undotbs2.293.943615617
datafile 94 switched to datafile copy
input datafile copy RECID=197 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/sysaux.290.943615617
datafile 95 switched to datafile copy
input datafile copy RECID=198 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/sysaux.269.943615831
datafile 96 switched to datafile copy
input datafile copy RECID=199 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/users.279.943611155
datafile 97 switched to datafile copy
input datafile copy RECID=200 STAMP=943620733 file name=+DATA/TESTDBPRD/datafile/users.294.943615617
released channel: ch01
released channel: ch02
released channel: ch03
released channel: ch04
released channel: ch05
RMAN>

  4.   ADG parameter setting

This step is not mandatory if you opt to apply archive offline to standby database

On testlabdb005a:

alter database rename file '+FRA/oradata/TESTDBPRD/redo_g1_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g1_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g1_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g1_m02.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g2_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g2_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g2_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g2_m02.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g3_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g3_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g3_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g3_m02.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g4_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g4_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g4_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g4_m02.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g5_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g5_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g5_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g5_m02.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g6_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g6_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g6_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g6_m02.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g7_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g7_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g7_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g7_m02.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g8_m01.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g8_m01.dbf';
alter database rename file '+FRA/oradata/TESTDBPRD/redo_g8_m02.dbf' to '+DATA/TESTDBPRD/onlinelog/redo_g8_m02.dbf';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_9.292.812362187' to '+DATA/TESTDBPRD/onlinelog/group_9.292.812362187';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_9.293.812362187' to '+DATA/TESTDBPRD/onlinelog/group_9.293.812362187';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_10.294.812362321' to '+DATA/TESTDBPRD/onlinelog/group_10.294.812362321';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_10.295.812362323' to '+DATA/TESTDBPRD/onlinelog/group_10.295.812362323';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_11.296.812362331' to '+DATA/TESTDBPRD/onlinelog/group_11.296.812362331';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_11.297.812362333' to '+DATA/TESTDBPRD/onlinelog/group_11.297.812362333';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_12.299.812513579' to '+DATA/TESTDBPRD/onlinelog/group_12.299.812513579';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_12.298.812513581' to '+DATA/TESTDBPRD/onlinelog/group_12.298.812513581';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_13.300.812513589' to '+DATA/TESTDBPRD/onlinelog/group_13.300.812513589';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_13.301.812513591' to '+DATA/TESTDBPRD/onlinelog/group_13.301.812513591';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_14.302.812513721' to '+DATA/TESTDBPRD/onlinelog/group_14.302.812513721';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_14.303.812513723' to '+DATA/TESTDBPRD/onlinelog/group_14.303.812513723';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_15.304.812513735' to '+DATA/TESTDBPRD/onlinelog/group_15.304.812513735';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_15.305.812513737' to '+DATA/TESTDBPRD/onlinelog/group_15.305.812513737';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_16.306.812513745' to '+DATA/TESTDBPRD/onlinelog/group_16.306.812513745';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_16.307.812513747' to '+DATA/TESTDBPRD/onlinelog/group_16.307.812513747';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_17.308.812513765' to '+DATA/TESTDBPRD/onlinelog/group_17.308.812513765';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_17.309.812513767' to '+DATA/TESTDBPRD/onlinelog/group_17.309.812513767';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_18.310.812513775' to '+DATA/TESTDBPRD/onlinelog/group_18.310.812513775';
alter database rename file '+DATA1/TESTDBPRD/onlinelog/group_18.311.812513777' to '+DATA/TESTDBPRD/onlinelog/group_18.311.812513777';

idle> alter system set standby_file_management='AUTO' scope=spfile;
System altered.

idle> alter system set fal_client='NEWDBPRD';
System altered.

idle> alter system set fal_server='TESTADG';
System altered.

idle> alter system set db_create_online_log_dest_1='+DATA' scope=spfile;
System altered.

On testoldlabdb1:


SQL> alter system set log_archive_dest_3='SERVICE=NEWDBPRD ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=NEWDBPRD';

System altered.

SQL> alter system set log_archive_dest_state_3=ENABLE;
System altered.

SQL> alter system set log_archive_config='DG_CONFIG=(TESTDBPRD,TESTADG, NEWDBPRD)' scope=spfile;
System altered.

SQL> alter system set fal_server='TESTDBPRD,NEWDBPRD';
System altered.


 5.   Restore archives and apply to standby


In case real time archive sync not enabled below command can be used to restore archives from primary and applied to standby db

RUN {
configure device type disk parallelism 5;
ALLOCATE CHANNEL ch01 DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch02  DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch03  DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch04  DEVICE TYPE DISK ;
ALLOCATE CHANNEL ch05  DEVICE TYPE DISK ;
restore archivelog from sequence 268574  until sequence 268700 thread 2;
RELEASE CHANNEL ch01;
RELEASE CHANNEL ch02;
RELEASE CHANNEL ch03;
RELEASE CHANNEL ch04;
RELEASE CHANNEL ch05;
}
EOF

   6.   Activating standby


    Following steps can be taken to activate the standby and opening db instance in read write mode

SQL> select name,open_mode,database_role ,SWITCHOVER_STATUS,DATAGUARD_BROKER,GUARD_STATUS from v$database;

NAME      OPEN_MODE   DATABASE_ROLE  SWITCHOVER_STATUS  DATAGUAR GUARD_S
---------------- ---------------- -------------------- -------- -------
TESTDBPRD MOUNTED   PHYSICAL STANDBY NOT ALLOWED          DISABLED NONE

SQL> SELECT THREAD#, LOW_SEQUENCE#, HIGH_SEQUENCE# FROM V$ARCHIVE_GAP;
no rows selected

SQL> alter database recover managed standby database cancel;
Database altered.

SQL> SQL> alter database recover managed standby database finish;
Database altered.

SQL> alter database activate physical standby database;
Database altered.

SQL> select name, open_mode, database_role from v$database;
NAME         OPEN_MODE            DATABASE_ROLE
------------ ----------------- ----------------
TESTDBPRD    MOUNTED              PRIMARY

SQL> alter database open;
Database altered.


 7.   Pre-requisites of 12c upgrade


srvctl stop instance -d TESTDBPRD -i NEWDBPRD2

Complete Checklist for Manual Upgrades to Oracle Database 12c Release 1 (12.1) (Doc ID 1503653.1)

[oracle@testlabdb005a backup]$ srvctl start instance -d TESTDBPRD -i NEWDBPRD2

SQL> select count(1),owner from dba_objects where status <> 'VALID' group by owner;

On new primary testlabdb005a (matching)

  COUNT(1) OWNER
---------- ------------------------------
        15 CORET
         2 ASC_ACE_IP
         5 DMSYS
         1 QS_ADM
        89 PUBLIC
         7 ASC_ACE_TX
        20 NETX
        10 NETXDEV
         1 ASC_SIA
         3 SYSTEM
         2 ASC_ACE_TEMP_TABLE
        16 DATALOAD
         4 OE
        16 SYSMAN
         3 NETXDR
         1 ODM
         7 CORETAQ1
         3 OMA
      1442 SYS
       143 MSPAK
        15 ASC_RF

21 rows selected.

On testoldlabdb1:

  COUNT(1) OWNER
---------- ------------------------------
        15 CORET
         2 ASC_ACE_IP
         5 DMSYS
         1 QS_ADM
        89 PUBLIC
         8 ASC_ACE_TX
        20 NETX
        10 NETXDEV
         2 ASC_SIA
         3 SYSTEM
         2 ASC_ACE_TEMP_TABLE
        16 DATALOAD
         4 OE
        16 SYSMAN
         3 NETXDR
         1 ODM
         7 CORETAQ1
         3 OMA
      1442 SYS
       143 MSPAK
        15 ASC_RF

21 rows selected.

i.   wri$_optstat_histgrm_history purging

select segment_name,bytes/1024/1024 from dba_segments where segment_name in ('WRI$_OPTSTAT_HISTHEAD_HISTORY','WRI$_OPTSTAT_HISTGRM_HISTORY');
SEGMENT_NAME                      BYTES/1024/1024 
-------------------------------- ---------------
WRI$_OPTSTAT_HISTHEAD_HISTORY       7431
WRI$_OPTSTAT_HISTGRM_HISTORY       19379

select count(1) from WRI$_OPTSTAT_HISTHEAD_HISTORY; -- 124808149
select count(1) from WRI$_OPTSTAT_HISTGRM_HISTORY; -- 460134614

select count(1) from dba_tab_partitions where table_name='WRI$_OPTSTAT_HISTGRM_HISTORY';

select count(1) from dba_tab_partitions where table_name='WRI$_OPTSTAT_HISTHEAD_HISTORY';

n  for 15 days older purging
exec dbms_stats.purge_stats(trunc(sysdate-15));

select DBMS_STATS.GET_STATS_HISTORY_RETENTION from dual;

select count(1) from WRI$_OPTSTAT_HISTHEAD_HISTORY;

select count(1) from WRI$_OPTSTAT_HISTGRM_HISTORY;


ii.  Executed sysaux partitioning (to fix sysaux high growth issue)

select 1  from dual;
alter session set "_swrf_test_action" = 72;

iii. Gather dictionary stats

EXECUTE dbms_stats.gather_dictionary_stats;
EXECUTE DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;

iv. Purge dba_recyclebin

sys@TESTDBPRD> purge dba_recyclebin;

DBA Recyclebin purged.


v.  Pre-upgrade script execution


SQL>@/golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/preupgrade_fixups.sql
  $ORACLE_HOME/olap/admin/catnoamd.sql
    to drop olapsys

sys@TESTDBPRD> @/rdbms/app/oracle/12.1.0.2/db/rdbms/admin/preupgrd.sql
Loading Pre-Upgrade Package...
***************************************************************************
Executing Pre-Upgrade Checks in TESTDBPRD...
***************************************************************************
      ************************************************************
                ====>> ERRORS FOUND for TESTDBPRD <<====
 The following are *** ERROR LEVEL CONDITIONS *** that must be addressed
                    prior to attempting your upgrade.
            Failure to do so will result in a failed upgrade.

           You MUST resolve the above errors prior to upgrade
      ************************************************************
      ************************************************************
              ====>> PRE-UPGRADE RESULTS for TESTDBPRD <<====
ACTIONS REQUIRED:

1. Review results of the pre-upgrade checks:
 /golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/preupgrade.log
    
2. Execute in the SOURCE environment BEFORE upgrade:
 /golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/preupgrade_fixups.sql

3. Execute in the NEW environment AFTER upgrade:
 /golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/postupgrade_fixups.sql
 ************************************************************
***************************************************************************
Pre-Upgrade Checks in TESTDBPRD Completed.
***************************************************************************
***************************************************************************

INFORMATION: --> Older Timezone in use
     Database is using a time zone file older than version 18.
     After the upgrade, it is recommended that DBMS_DST package
     be used to upgrade the 11.2.0.3.0 database time zone version
     to the latest version which comes with the new release.
     Please refer to My Oracle Support note number 1509653.1 for details.

sys@TESTDBPRD> SELECT version FROM v$timezone_file;
        14
Note : 1665676.1


 If your current timezone version is lower than 18 (typically 4, 11 or 14) when upgrading from 10.2.0.5, 11.1.0.7, 11.2.0.2, 11.2.0.3 , 11.2.0.4 or 12.1.0.1 to 12.1.0.2 :
If the source database is using a timezone file older than version 18 then the preupgrd.sql (Pre-Upgrade Information Tool) will generate the following warning in the preupgrd.log (this is normal)
Example output:
     Database is using a time zone file older than version 18.
     After the upgrade, it is recommended that DBMS_DST package
     be used to upgrade the -db version here- database time zone version
     to the latest version which comes with the new release.
     Please refer to My Oracle Support note number 977512.1 for details.
The 12.1.0.2 RDBMS DST version after the upgrade to 12.1.0.2 will be the same DST version as used in 10.2.0.5 , 11.1.0.7, 11.2.0.2, 11.2.0.3 ,11.2.0.4 or 12.1.0.2.

After the upgrade to 12.1.0.2 you can:
* (recommended) update the 12.1.0.2 database(s) to DSTv18 (standard DST version of 12.1.0.2) by :
  • or using the upg_tzv_check.sql and upg_tzv_apply.sql scripts in note 1585343.1 Scripts to automatically update the RDBMS DST (timezone) version in an 11gR2 or 12cR1 database .
  • or following note 1509653.1 Updating the RDBMS DST version in 12c Release 1 (12.1.0.1 and up) using DBMS_DST from step 3a) using "18" as (<the new DST version number>) in that note.
* (optional) update to a higher DST than 18 version if this is needed.
The latest DST patch and related note on how to apply this is found in Note 412160.1 Updated Time Zones in Oracle Time Zone File patches under "C) Notes covering the current DST available updates".
Simply follow the note for the latest DST update
vi.                   file status

sys@TESTDBPRD> SELECT * FROM v$recover_file;
no rows selected

sys@TESTDBPRD> SELECT * FROM v$backup WHERE status != 'NOT ACTIVE';

no rows selected

sys@TESTDBPRD> SELECT * FROM dba_2pc_pending;
no rows selected

sys@TESTDBPRD> SELECT username, default_tablespace
     FROM dba_users
     WHERE username in ('SYS','SYSTEM');  2    3

USERNAME                       DEFAULT_TABLESPACE
------------------------------ ------------------------------
SYSTEM                         SYSTEM
SYS                            SYSTEM

SQL> SELECT name FROM v$controlfile;
SQL> SELECT file_name FROM dba_data_files;
SQL> SELECT group#, member FROM v$logfile;

SQL> Alter system set "_system_trig_enabled"=false scope=spfile;
-- to be enabled post upgrade

sys@TESTDBPRD> alter trigger sys.GGS_DDL_TRIGGER_BEFORE disable ;
Trigger altered.


SQL> ALTER SYSTEM SET java_jit_enabled=FALSE;
-- to be enabled post upgrade
System altered.

SQL> alter system set java_pool_size=5G scope=spfile sid='*';
System altered.

SQL>alter system set job_queue_processes=0;

SQL> alter system set db_recovery_file_dest_size=300g;
n  to be set to 10 post upgrade

vi.     Verify free temp/undo space.. Add 20-30G space in TEMP
vii.    Drop DB links post script backup

SELECT 'CREATE '||DECODE(U.NAME,'PUBLIC','public ')||'DATABASE LINK '||CHR(10)
||DECODE(U.NAME,'PUBLIC',Null, 'SYS','',U.NAME||'.')|| L.NAME||chr(10)
||'CONNECT TO ' || L.USERID || ' IDENTIFIED BY "'||L.PASSWORD||'" USING
'''||L.HOST||''''
||chr(10)||';' TEXT
FROM SYS.LINK$ L, SYS.USER$ U
WHERE L.OWNER# = U.USER#;

 
 select OWNER,DB_LINK,USERNAME,HOST from dba_db_links
   drop database link CMSPROD_TEST;
 

viii.     Set job_queue_process to Zero

ix.      SQL>ALTER SYSTEM SET COMPATIBLE = '12.1.0' SCOPE=SPFILE;



Disable archive log & cluster:

sys@TESTDBPRD> shut immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
sys@TESTDBPRD> startup mount
ORACLE instance started.
Total System Global Area 4.8103E+10 bytes
Fixed Size                  2239376 bytes
Variable Size            4563403888 bytes
Database Buffers         4.3487E+10 bytes
Redo Buffers               50479104 bytes
Database mounted.

sys@TESTDBPRD> alter database noarchivelog;
Database altered.

sys@TESTDBPRD> alter database open;
Database altered.

sys@TESTDBPRD> alter system set cluster_database=FALSE scope=spfile;
System altered.

sys@TESTDBPRD> alter system set resumable_timeout=36000 scope=spfile;
--To be removed post upgrade
System altered.


SQL> alter system set db_securefile=PERMITTED scope=spfile;


 8.  Initiate 12c upgrade


i.  Copy current pfile to 12c $ORACLE_HOME/dbs on both nodes
ii. Open db in upgrade in new 12c home

[oracle@testlabdb005a ~]$ . ./.bash_profile_12c
[oracle@testlabdb005a ~]$ sqlplus "/ as sysdba"
SQL*Plus: Release 12.1.0.2.0 Production on Thu JAN 11 08:53:36 2017
Copyright (c) 1982, 2014, Oracle.  All rights reserved.
Connected to an idle instance.
idle> startup upgrade
ORACLE instance started.
Total System Global Area 4.8318E+10 bytes
Fixed Size      4510496 bytes
Variable Size   6174018784 bytes
Database Buffers  4.2010E+10 bytes
Redo Buffers    129703936 bytes
Database mounted.
Database opened.

iii. Run upgrade script

cd $ORACLE_HOME/rdbms/admin
nohup $ORACLE_HOME/perl/bin/perl catctl.pl -n  6 -l $ORACLE_HOME/diagnostics catupgrd.sql & -- leads to high undo / resource contention

Monitor :

/rdbms/app/oracle/12.1.0.2/db/diagnostics/catupgrd0.log


nohup $ORACLE_HOME/perl/bin/perl catctl.pl -n  2 -l $ORACLE_HOME/diagnostics catupgrd.sql &

--validate components

Set line 900 pages 900
col STATUS for a12
col COMP_NAME for a50
select COMP_ID,COMP_NAME,VERSION,STATUS,PROCEDURE,STARTUP from dba_registry;

$sqlplus "/as sysdba" 
SQL> spool fix_cat_logfile.txt
 
SQL> startup upgrade
 
SQL > @?/rdbms/admin/utlrp.sql
SQL > @?/rdbms/admin/catalog.sql
   -- Incase components are still invalid post utlrp
SQL > @?/rdbms/admin/catproc.sql
 

 
Post check:

SQL> startup
SQL> @?/rdbms/admin/utlu121s.sql

 If all components valid then proceed

SQL>@?/rdbms/admin/catuppst.sql

SQL>@?/rdbms/admin/utlrp.sql

iv. Postupgrade scripts


SQL>@/golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrad/postupgrade_fixups.sql


9.  Time zone upgrade

DBMS_DST_scriptsV1.9 setup script to be used


SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
      FROM DATABASE_PROPERTIES
      WHERE PROPERTY_NAME LIKE 'DST_%'
      ORDER BY PROPERTY_NAME;   

PROPERTY_NAME                 VALUE
---------------            ---------------------------------
DST_PRIMARY_TT_VERSION           14
DST_SECONDARY_TT_VERSION         4
DST_UPGRADE_STATE           DATAPUMP(4)


SQL> ALTER SESSION SET EVENTS '30090 TRACE NAME CONTEXT FOREVER, LEVEL 32';

SQL>  exec dbms_dst.unload_secondary;
Session altered.


[oracle@testlabdb005a DBMS_DST_scriptsV1.9]$ pwd
/oradump/software/12c/DBMS_DST_scriptsV1.9
[oracle@testlabdb005a DBMS_DST_scriptsV1.9]$ ls -lrt
total 68
-rw-r--r--. 1 oracle oinstall 31010 Aug 22  2014 upg_tzv_check.sql
-rw-r--r--. 1 oracle oinstall 19502 Aug 22  2014 upg_tzv_apply.sql
-rw-r--r--. 1 oracle oinstall  6294 Jan  8  2015 countstarTSTZ.sql
-rw-r--r--. 1 oracle oinstall  6909 Jan  8  2015 countstatsTSTZ.sql

SQL> @upg_tzv_check.sql
INFO: Starting with RDBMS DST update preparation.
INFO: NO actual RDBMS DST update will be done by this script.
INFO: If an ERROR occurs the script will EXIT sqlplus.
INFO: Doing checks for known issues ...
INFO: Database version is 12.1.0.2 .
INFO: Database RDBMS DST version is DSTv14 .
INFO: No known issues detected.
INFO: Now detecting new RDBMS DST version.
A prepare window has been successfully started.
INFO: Newest RDBMS DST version detected is DSTv18 .
INFO: Next step is checking all TSTZ data.
INFO: It might take a while before any further output is seen ...
A prepare window has been successfully ended.
INFO: A newer RDBMS DST version than the one currently used is found.
INFO: Note that NO DST update was yet done.
INFO: Now run upg_tzv_apply.sql to do the actual RDBMS DST update.
INFO: Note that the upg_tzv_apply.sql script will
INFO: restart the database 2 times WITHOUT any confirmation or prompt.
SQL>

SQL>SHUT IMMEDIATE;
SQL>STARTUP UPGRADE
SQL> @upg_tzv_apply.sql
INFO: If an ERROR occurs the script will EXIT sqlplus.
INFO: The database RDBMS DST version will be updated to DSTv18 .
WARNING: This script will restart the database 2 times
WARNING: WITHOUT asking ANY confirmation.
WARNING: Hit control-c NOW if this is not intended.
INFO: Restarting the database in UPGRADE mode to start the DST upgrade.
Database closed.
Database dismounted.
ORACLE instance shut down.
ORACLE instance started.
Total System Global Area 4.8318E+10 bytes
Fixed Size                  4510496 bytes
Variable Size            1.0737E+10 bytes
Database Buffers         3.7447E+10 bytes
Redo Buffers              129703936 bytes
Database mounted.
Database opened.
INFO: Starting the RDBMS DST upgrade.
INFO: Upgrading all SYS owned TSTZ data.
INFO: It might take time before any further output is seen ...
An upgrade window has been successfully started.
INFO: Restarting the database in NORMAL mode to upgrade non-SYS TSTZ data.
Database closed.
Database dismounted.
ORACLE instance shut down.
ORACLE instance started.
Total System Global Area 4.8318E+10 bytes
Fixed Size                  4510496 bytes
Variable Size            1.0737E+10 bytes
Database Buffers         3.7447E+10 bytes
Redo Buffers              129703936 bytes
Database mounted.
Database opened.
INFO: Upgrading all non-SYS TSTZ data.
INFO: It might take time before any further output is seen ...
INFO: Do NOT start any application yet that uses TSTZ data!
INFO: Next is a list of all upgraded tables:
Table list: "SYSMAN"."MGMT_PROV_OPERATION"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_BOOTSERVER"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_DEFAULT_IMAGE"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_RPM_REP"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_STAGING_DIRS"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_NET_CONFIG"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_IP_RANGE"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_CLUSTER_NODES"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_SUITE_INST_MEMBERS"
Number of failures: 0
Table list: "SYSMAN"."MGMT_CONFIG_ACTIVITIES"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_ASSIGNMENT"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_NOTIFY_QTABLE_S"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_NOTIFY_QTABLE_L"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_LOADER_QTABLE_S"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_LOADER_QTABLE_L"
Number of failures: 0
Table list: "ASC_ACE_TX"."NCCM_ACEDATAMIGRATIONLOG"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_SQL_STATEMENT"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_OPTSTAT_OPERATIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_SCHEDULER_JOBS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_OPTSTAT_USER_PREFS$"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_TAB_STATS_VERSIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_IND_STATS_VERSIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_COL_STATS_VERSIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_HISTGRM_STATS_VERSN"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_WRI$_OPTSTAT_AUX_HISTORY"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_AUTOTASK_CLIENT_HST"
Number of failures: 0
Table list: "GSMADMIN_INTERNAL"."AQ$_CHANGE_LOG_QUEUE_TABLE_S"
Number of failures: 0
Table list: "GSMADMIN_INTERNAL"."AQ$_CHANGE_LOG_QUEUE_TABLE_L"
Number of failures: 0
INFO: Total failures during update of TSTZ data: 0 .
An upgrade window has been successfully ended.
INFO: Your new Server RDBMS DST version is DSTv18 .
INFO: The RDBMS DST update is successfully finished.
INFO: Make sure to exit this sqlplus session.
INFO: Do not use it for timezone related selects.


SQL>  select * from v$timezone_file;

FILENAME                VERSION     CON_ID
-------------------- ---------- --------
timezlrg_18.dat              18          0


10. Post upgrade steps


i.   Set cluster_database to TRUE

Alter system set cluster_database=TRUE scope=spfile;
Srvctl stop instance –d TESTDBPRD –I NEWDBPRD1

Srvctl start instance –d TESTDBPRD –I NEWDBPRD1
Srvctl start instance –d TESTDBPRD –I NEWDBPRD2

ii.  Update profile of DB services with new 12c home

Srvctl upgrade database –d TESTDBPRD –o /rdbms/app/oracle/12.1.0.2/db

A. Issues Faced

i. ORA-00600: internal error code, arguments: [pesldl03_MMap: errno 1 errmsg Operation not permitted

Issue:Alert log notifying below error during utlrp.sql executions

Errors in file /golden/app/oracle/11.2.0.3.0/diag/rdbms/TESTDBPRD/NEWDBPRD1/trace/NEWDBPRD1_ora_62755.trc  (incident=1687133):
ORA-00600: internal error code, arguments: [pesldl03_MMap: errno 1 errmsg Operation not permitted

Cause: /dev/shm was mountable with noexec,nosuid

ORA-600 [pesldl03_MMap: Errno 1 Errmsg Operation Not Permitted] (Doc ID 1625010.1)

Solution:

1. /dev/shm changes
[oracle@testlabdb005a security]$  cat /etc/fstab | grep -i shm
tmpfs                   /dev/shm                tmpfs   defaults,nodev,nosuid,noexec        0 0
should be
[oracle@testlabdb005a security]$  cat /etc/fstab | grep -i shm
tmpfs                   /dev/shm                tmpfs   defaults,nodev       0 0
as per solution note
Ensure that the /dev/shm mount area is of type tmpfs and is mounted with the following options:
With rw and exec permissions set on it
Without noexec or nosuid set on it"
# cat /etc/fstab
...
tmpfs    /dev/shm   tmpfs   defaults     0 0
2. limit changes
/etc/security/limits.conf
* soft nofile 32768
* hard nofile 65535
* soft stack 10240
* soft nproc 131072

oracle soft nproc 131072
oracle hard nproc 131072
3. in /etc/security/limits.d/ <not sure if in 90-nproc.conf or 20-nproc.conf>
*          soft    nproc     131072
oracle  soft    nproc   131072
oracle  hard    nproc   131072

  
ii.  UTLRP.sql hangs with “single task message” wait event

Issue:
Utlrp.sql hangs with wait event “single task message”

Cause:
Many of the procedure/packages dependent on unresolvable db link

Solution: To remove all DB link pointing to invalid DB link


iii. TIME Zone upgrade status issue

Issue: TIMEZONE pre-requisites were not fulfilling for TZ upgrade , DST_UPGRADE_STATE should be NONE however stucked on DATAPUMP(4) state

SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME;  2    3    4

PROPERTY_NAME              VALUE
-----------------       ------------------------------
DST_PRIMARY_TT_VERSION         14
DST_SECONDARY_TT_VERSION        4
DST_UPGRADE_STATE           DATAPUMP(4)

Solution:

Execute below multiple times till DATAPUMP(4) count reduces and reaches to NONE

SQL> ALTER SESSION SET EVENTS '30090 TRACE NAME CONTEXT FOREVER, LEVEL 32';
SQL>  exec dbms_dst.unload_secondary;



How to enable sql session tracing in oracle database

$
0
0
            Enabling session tracing in oracle database


1. Identify sql client session to be traced

For example you are planning to trace the session for module ABC , identify the SPID for session e.g. 13762780

alter session set nls_date_format='dd-mon-yyyy hh24:mi:ss';
set line 1000
col OSUSER for a10
col SID for 999
col USERNAME for a20
col MACHINE for a20
set pagesize 100
SELECT distinct 
   a.sid, 
   a.serial#, 
   b.spid, sq.sql_id,
   b.pid, 
   a.username, 
   a.machine,
   a.status,
a.module,
a.osuser,
a.LAST_CALL_ET/60 "Inac_Min",a.logon_time,c.event, sq.sql_text
FROM 
   v$session a, 
   v$process b,
   v$session_wait c,
v$sqlarea sq
WHERE 
   a.username IS NOT NULL
AND 
   a.paddr=b.addr
and a.username='GENEVA_ADMIN' and a.sql_hash_value=sq.hash_value
and (upper(a.module) like '%ABC%')

and c.sid=a.sid;


2. Enable trace 

For example if spid 13762780 has to be traced

oradebug setospid 13762780
oradebug unlimit
oradebug Event 10046 trace name context forever, level 12

oradebug tracefile_name

3. disable trace

Incase session does not get disconnect the session tracing keep on filing trace file which may lead to panic utilization of file system

oradebug setospid 13762780
oradebug Event 10046 trace name context off

4. Analyzing trace file using tkprof

tkprof trace analyzer will extract the trace file generated to showcase execcution plan of each query executed in session

tkprof testdb_ora_13762780.trc tk_testdb_ora_13762780.out explain=test_mis/test_mis123 waits=yes sort=exeela

5. More methods of tracing

Following command can be executed to enable trace for current session tracing

alter session set tracefile_identifier='abc_trace';
alter session set sql_trace=TRUE;

6. More methods of enabling bulk tracing and analyzing trace with condition

In case you need to enable tracing on complete database , below trigger can be created which will trace each session connected on DB till trigger is in enable state

Note: If trigger is in invalid state then it will decline new connection to DB leading to outage

CREATE OR REPLACE TRIGGER trace AFTER logon ON database begin execute immediate 'Alter session set tracefile_identifier=''ABC_traces''';
execute immediate 'Alter session set events ''10046 trace name context forever, level 12'''; end;

/


One trace files are generated in bulk due to trigger you may use below code to search for ABC@TESTLAB002 in trace and generate tkprof for same 

for i in `ls -lrt | grep -i ABC_traces.trc | awk '{ print $9 }'`
do
#echo $i
if [ `cat $i | grep -i ABC@TESTLAB002 | wc -l | awk '{ print $1 }'` -gt 0 ] ; then
echo $i
tkprof $i $i.out explain=test_mis/test_mis123 waits=yes sort=exeela
fi;

done






Steps to create active data guard in 11g and up-gradation to 12c using manual method - II

$
0
0


<< PREV_PAGE                                                                 Page   1   2   3   4


6. Activating standby


Following steps can be taken to activate the standby and opening db instance in read write mode


SQL> select name,open_mode,database_role ,SWITCHOVER_STATUS,DATAGUARD_BROKER,GUARD_STATUS from v$database;

NAME OPEN_MODE DATABASE_ROLE SWITCHOVER_STATUS DATAGUAR GUARD_S
------------ ------------ -------------------- -------- -------
TESTDBPRD MOUNTED PHYSICAL STANDBY NOT ALLOWED DISABLED NONE



SQL> SELECT THREAD#, LOW_SEQUENCE#, HIGH_SEQUENCE# FROM V$ARCHIVE_GAP;

no rows selected


SQL> alter database recover managed standby database cancel;

Database altered.


SQL> SQL> alter database recover managed standby database finish;

Database altered.


SQL> alter database activate physical standby database;

Database altered.


SQL> select name, open_mode, database_role from v$database;


NAME          OPEN_MODE        DATABASE_ROLE
------------ ----------------- ----------------
TESTDBPRD     MOUNTED           PRIMARY


SQL> alter database open;

Database altered.


7. Pre-requisites of 12c upgrade

srvctl stop instance -d TESTDBPRD -i NEWDBPRD2


Complete Checklist for Manual Upgrades to Oracle Database 12c Release 1 (12.1) (Doc ID 1503653.1)


[oracle@testlabdb005a backup]$ srvctl start instance -d TESTDBPRD -i NEWDBPRD2

SQL> select count(1),owner from dba_objects where status <> 'VALID' group by owner;

On new primary testlabdb005a (matching)


COUNT(1)    OWNER
---------- -------------
15 CORET
2 ASC_ACE_IP
5 DMSYS
1 QS_ADM
89 PUBLIC
7 ASC_ACE_TX
20 NETX
10 NETXDEV
1 ASC_SIA
3 SYSTEM
2 ASC_ACE_TEMP_TABLE
16 DATALOAD
4 OE
16 SYSMAN
3 NETXDR
1 ODM
7 CORETAQ1
3 OMA
1442 SYS
143 MSPAK
15 ASC_RF

21 rows selected.

On testoldlabdb1:

COUNT(1)    OWNER
---------- --------------
15 CORET
2 ASC_ACE_IP
5 DMSYS
1 QS_ADM
89 PUBLIC
8 ASC_ACE_TX
20 NETX
10 NETXDEV
2 ASC_SIA
3 SYSTEM
2 ASC_ACE_TEMP_TABLE
16 DATALOAD
4 OE
16 SYSMAN
3 NETXDR
1 ODM
7 CORETAQ1
3 OMA
1442 SYS
143 MSPAK
15 ASC_RF

21 rows selected.

i. wri$_optstat_histgrm_history purging

select segment_name,bytes/1024/1024 from dba_segments where segment_name in ('WRI$_OPTSTAT_HISTHEAD_HISTORY','WRI$_OPTSTAT_HISTGRM_HISTORY');


SEGMENT_NAME                 BYTES/1024/1024
---------------------------- ---------------
WRI$_OPTSTAT_HISTHEAD_HISTORY 7431
WRI$_OPTSTAT_HISTGRM_HISTORY 19379


select count(1) from WRI$_OPTSTAT_HISTHEAD_HISTORY; -- 124808149

select count(1) from WRI$_OPTSTAT_HISTGRM_HISTORY; -- 460134614

select count(1) from dba_tab_partitions where table_name='WRI$_OPTSTAT_HISTGRM_HISTORY';


select count(1) from dba_tab_partitions where table_name='WRI$_OPTSTAT_HISTHEAD_HISTORY';

-- for 15 days older purging


exec dbms_stats.purge_stats(trunc(sysdate-15));


select DBMS_STATS.GET_STATS_HISTORY_RETENTION from dual;


select count(1) from WRI$_OPTSTAT_HISTHEAD_HISTORY;


select count(1) from WRI$_OPTSTAT_HISTGRM_HISTORY;


ii. Executed sysaux partitioning (to fix sysaux high growth issue)
select 1 from dual;

alter session set "_swrf_test_action" = 72;


iii. Gather dictionary stats


EXECUTE dbms_stats.gather_dictionary_stats;

EXECUTE DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;


iv. Purge dba_recyclebin


sys@TESTDBPRD> purge dba_recyclebin;

DBA Recyclebin purged.


v. Pre-upgrade script execution


SQL>@/golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/preupgrade_fixups.sql

$ORACLE_HOME/olap/admin/catnoamd.sql

to drop olapsys


sys@TESTDBPRD> @/rdbms/app/oracle/12.1.0.2/db/rdbms/admin/preupgrd.sql

Loading Pre-Upgrade Package...
************************************************************

Executing Pre-Upgrade Checks in TESTDBPRD...

************************************************************
************************************************************


====>> ERRORS FOUND for TESTDBPRD <<====
The following are *** ERROR LEVEL CONDITIONS *** that must be addressed
prior to attempting your upgrade.

Failure to do so will result in a failed upgrade.


You MUST resolve the above errors prior to upgrade
************************************************************
************************************************************

====>> PRE-UPGRADE RESULTS for TESTDBPRD <<====
ACTIONS REQUIRED:

1. Review results of the pre-upgrade checks:
/golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/preupgrade.log

2. Execute in the SOURCE environment BEFORE upgrade:

/golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/preupgrade_fixups.sql

3. Execute in the NEW environment AFTER upgrade:

/golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrade/postupgrade_fixups.sql

************************************************************
************************************************************


Pre-Upgrade Checks in TESTDBPRD Completed.


************************************************************
************************************************************

INFORMATION: --> Older Timezone in use
Database is using a time zone file older than version 18.
After the upgrade, it is recommended that DBMS_DST package
be used to upgrade the 11.2.0.3.0 database time zone version
to the latest version which comes with the new release.
Please refer to My Oracle Support note number 1509653.1 for details.

sys@TESTDBPRD> SELECT version FROM v$timezone_file;

14

Note : 1665676.1

If your current timezone version is lower than 18 (typically 4, 11 or 14) when upgrading from 10.2.0.5, 11.1.0.7, 11.2.0.2, 11.2.0.3 , 11.2.0.4 or 12.1.0.1 to 12.1.0.2 :

If the source database is using a timezone file older than version 18 then the preupgrd.sql (Pre-Upgrade Information Tool) will generate the following warning in the preupgrd.log (this is normal)
Example output:

Database is using a time zone file older than version 18.
After the upgrade, it is recommended that DBMS_DST package
be used to upgrade the -db version here- database time zone version
to the latest version which comes with the new release.
Please refer to My Oracle Support note number 977512.1 for details.

The 12.1.0.2 RDBMS DST version after the upgrade to 12.1.0.2 will be the same DST version as used in 10.2.0.5 , 11.1.0.7, 11.2.0.2, 11.2.0.3 ,11.2.0.4 or 12.1.0.2.

After the upgrade to 12.1.0.2 you can:
* (recommended) update the 12.1.0.2 database(s) to DSTv18 (standard DST version of 12.1.0.2) by :
or using the upg_tzv_check.sql and upg_tzv_apply.sql scripts in note 1585343.1 Scripts to automatically update the RDBMS DST (timezone) version in an 11gR2 or 12cR1 database .
or following note 1509653.1 Updating the RDBMS DST version in 12c Release 1 (12.1.0.1 and up) using DBMS_DST from step 3a) using "18" as (<the new DST version number>) in that note.

* (optional) update to a higher DST than 18 version if this is needed.
The latest DST patch and related note on how to apply this is found in Note 412160.1 Updated Time Zones in Oracle Time Zone File patches under "C) Notes covering the current DST available updates".
Simply follow the note for the latest DST update



vi. file status


sys@TESTDBPRD> SELECT * FROM v$recover_file;
no rows selected

sys@TESTDBPRD> SELECT * FROM v$backup WHERE status != 'NOT ACTIVE';

no rows selected

sys@TESTDBPRD> SELECT * FROM dba_2pc_pending;

no rows selected

sys@TESTDBPRD> SELECT username, default_tablespace
FROM dba_users
WHERE username in ('SYS','SYSTEM'); 2 3

USERNAME          DEFAULT_TABLESPACE
---------------- -------------------------
SYSTEM            SYSTEM
SYS               SYSTEM

SQL> SELECT name FROM v$controlfile;
SQL> SELECT file_name FROM dba_data_files;
SQL> SELECT group#, member FROM v$logfile;

SQL> Alter system set "_system_trig_enabled"=false scope=spfile;
-- to be enabled post upgrade

sys@TESTDBPRD> alter trigger sys.GGS_DDL_TRIGGER_BEFORE disable ;

Trigger altered.

SQL> ALTER SYSTEM SET java_jit_enabled=FALSE;
-- to be enabled post upgrade

System altered.

SQL> alter system set java_pool_size=5G scope=spfile sid='*';

System altered.

SQL>alter system set job_queue_processes=0;

SQL> alter system set db_recovery_file_dest_size=300g;


--  to be set to 10 post upgrade

vi. Verify free temp/undo space.. Add 20-30G space in TEMP

vii. Drop DB links post script backup



SELECT 'CREATE '||DECODE(U.NAME,'PUBLIC','public ')||'DATABASE LINK '||CHR(10)||DECODE(U.NAME,'PUBLIC',Null, 'SYS','',U.NAME||'.')|| L.NAME||chr(10)||'CONNECT TO ' || L.USERID || ' IDENTIFIED BY "'||L.PASSWORD||'" USING
'''||L.HOST||''''
||chr(10)||';' TEXT
FROM SYS.LINK$ L, SYS.USER$ U
WHERE L.OWNER# = U.USER#;


select OWNER,DB_LINK,USERNAME,HOST from dba_db_links

drop database link CMSPROD_TEST;


viii. Set job_queue_process to Zero


ix. SQL>ALTER SYSTEM SET COMPATIBLE = '12.1.0' SCOPE=SPFILE;
Disable archive log & cluster:

sys@TESTDBPRD> shut immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

sys@TESTDBPRD> startup mount
ORACLE instance started.
Total System Global Area 4.8103E+10 bytes
Fixed Size 2239376 bytes
Variable Size 4563403888 bytes
Database Buffers 4.3487E+10 bytes
Redo Buffers 50479104 bytes
Database mounted.


sys@TESTDBPRD> alter database noarchivelog;
Database altered.

sys@TESTDBPRD> alter database open;
Database altered.

sys@TESTDBPRD> alter system set cluster_database=FALSE scope=spfile;
System altered.

sys@TESTDBPRD> alter system set resumable_timeout=36000 scope=spfile;
--To be removed post upgrade
System altered.


SQL> alter system set db_securefile=PERMITTED scope=spfile;



>> NEXT_PAGE                                                                        Page   1   2   3   4

Steps to create active data guard in 11g and up-gradation to 12c using manual method - III

$
0
0


<< PREV_PAGE                                                                         Page  1   2   3   4



8. Initiate 12c upgrade


i.   Copy current pfile to 12c $ORACLE_HOME/dbs on both nodes
ii. Open db in upgrade in new 12c home


[oracle@testlabdb005a ~]$ . ./.bash_profile_12c
[oracle@testlabdb005a ~]$ sqlplus "/ as sysdba"
SQL*Plus: Release 12.1.0.2.0 Production on Thu JAN 11 08:53:36 2017
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to an idle instance.
idle> startup upgrade
ORACLE instance started.
Total System Global Area 4.8318E+10 bytes
Fixed Size 4510496 bytes
Variable Size 6174018784 bytes
Database Buffers 4.2010E+10 bytes
Redo Buffers 129703936 bytes
Database mounted.
Database opened.


iii. Run upgrade script

cd $ORACLE_HOME/rdbms/admin

nohup $ORACLE_HOME/perl/bin/perl catctl.pl -n 6 -l $ORACLE_HOME/diagnostics catupgrd.sql & -- leads to high undo / resource contention


Monitor :

/rdbms/app/oracle/12.1.0.2/db/diagnostics/catupgrd0.log


nohup $ORACLE_HOME/perl/bin/perl catctl.pl -n 2 -l $ORACLE_HOME/diagnostics catupgrd.sql &

--validate components


Set line 900 pages 900
col STATUS for a12
col COMP_NAME for a50
select COMP_ID,COMP_NAME,VERSION,STATUS,PROCEDURE,STARTUP from dba_registry;


$sqlplus "/as sysdba"
SQL> spool fix_cat_logfile.txt
SQL> startup upgrade
SQL > @?/rdbms/admin/utlrp.sql
SQL > @?/rdbms/admin/catalog.sql -- Incase components are still invalid post utlrp
SQL > @?/rdbms/admin/catproc.sql

Post check:

SQL> startup
SQL> @?/rdbms/admin/utlu121s.sql

If all components valid then proceed


SQL>@?/rdbms/admin/catuppst.sql
SQL>@?/rdbms/admin/utlrp.sql

iv. Postupgrade scripts


SQL>@/golden/app/oracle/11.2.0.3.0/db/cfgtoollogs/TESTDBPRD/preupgrad/postupgrade_fixups.sql



9. Time zone upgrade


DBMS_DST_scriptsV1.9 setup script to be used


SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME;

PROPERTY_NAME VALUE
--------------- ---------------------------------
DST_PRIMARY_TT_VERSION 14
DST_SECONDARY_TT_VERSION 4
DST_UPGRADE_STATE DATAPUMP(4)

SQL> ALTER SESSION SET EVENTS '30090 TRACE NAME CONTEXT FOREVER, LEVEL 32';


SQL> exec dbms_dst.unload_secondary;
Session altered.


[oracle@testlabdb005a DBMS_DST_scriptsV1.9]$ pwd
/oradump/software/12c/DBMS_DST_scriptsV1.9

[oracle@testlabdb005a DBMS_DST_scriptsV1.9]$ ls -lrt
total 68
-rw-r--r--. 1 oracle oinstall 31010 Aug 22 2014 upg_tzv_check.sql
-rw-r--r--. 1 oracle oinstall 19502 Aug 22 2014 upg_tzv_apply.sql
-rw-r--r--. 1 oracle oinstall 6294 Jan 8 2015 countstarTSTZ.sql
-rw-r--r--. 1 oracle oinstall 6909 Jan 8 2015 countstatsTSTZ.sql


SQL> @upg_tzv_check.sql
INFO: Starting with RDBMS DST update preparation.
INFO: NO actual RDBMS DST update will be done by this script.
INFO: If an ERROR occurs the script will EXIT sqlplus.
INFO: Doing checks for known issues ...
INFO: Database version is 12.1.0.2 .
INFO: Database RDBMS DST version is DSTv14 .
INFO: No known issues detected.
INFO: Now detecting new RDBMS DST version.
A prepare window has been successfully started.
INFO: Newest RDBMS DST version detected is DSTv18 .
INFO: Next step is checking all TSTZ data.
INFO: It might take a while before any further output is seen ...
A prepare window has been successfully ended.
INFO: A newer RDBMS DST version than the one currently used is found.
INFO: Note that NO DST update was yet done.
INFO: Now run upg_tzv_apply.sql to do the actual RDBMS DST update.
INFO: Note that the upg_tzv_apply.sql script will
INFO: restart the database 2 times WITHOUT any confirmation or prompt.
SQL>


SQL>SHUT IMMEDIATE;

SQL>STARTUP UPGRADE

SQL> @upg_tzv_apply.sql

INFO: If an ERROR occurs the script will EXIT sqlplus.
INFO: The database RDBMS DST version will be updated to DSTv18 .
WARNING: This script will restart the database 2 times
WARNING: WITHOUT asking ANY confirmation.
WARNING: Hit control-c NOW if this is not intended.
INFO: Restarting the database in UPGRADE mode to start the DST upgrade.
Database closed.
Database dismounted.
ORACLE instance shut down.
ORACLE instance started.
Total System Global Area 4.8318E+10 bytes
Fixed Size 4510496 bytes
Variable Size 1.0737E+10 bytes
Database Buffers 3.7447E+10 bytes
Redo Buffers 129703936 bytes
Database mounted.
Database opened.
INFO: Starting the RDBMS DST upgrade.
INFO: Upgrading all SYS owned TSTZ data.
INFO: It might take time before any further output is seen ...
An upgrade window has been successfully started.
INFO: Restarting the database in NORMAL mode to upgrade non-SYS TSTZ data.
Database closed.
Database dismounted.
ORACLE instance shut down.
ORACLE instance started.
Total System Global Area 4.8318E+10 bytes
Fixed Size 4510496 bytes
Variable Size 1.0737E+10 bytes
Database Buffers 3.7447E+10 bytes
Redo Buffers 129703936 bytes
Database mounted.
Database opened.
INFO: Upgrading all non-SYS TSTZ data.
INFO: It might take time before any further output is seen ...
INFO: Do NOT start any application yet that uses TSTZ data!
INFO: Next is a list of all upgraded tables:
Table list: "SYSMAN"."MGMT_PROV_OPERATION"

Number of failures: 0

Table list: "SYSMAN"."MGMT_PROV_BOOTSERVER"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_DEFAULT_IMAGE"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_RPM_REP"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_STAGING_DIRS"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_NET_CONFIG"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_IP_RANGE"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_CLUSTER_NODES"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_SUITE_INST_MEMBERS"
Number of failures: 0
Table list: "SYSMAN"."MGMT_CONFIG_ACTIVITIES"
Number of failures: 0
Table list: "SYSMAN"."MGMT_PROV_ASSIGNMENT"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_NOTIFY_QTABLE_S"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_NOTIFY_QTABLE_L"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_LOADER_QTABLE_S"
Number of failures: 0
Table list: "SYSMAN"."AQ$_MGMT_LOADER_QTABLE_L"
Number of failures: 0
Table list: "ASC_ACE_TX"."NCCM_ACEDATAMIGRATIONLOG"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_SQL_STATEMENT"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_OPTSTAT_OPERATIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_SCHEDULER_JOBS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_OPTSTAT_USER_PREFS$"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_TAB_STATS_VERSIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_IND_STATS_VERSIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_COL_STATS_VERSIONS"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_HISTGRM_STATS_VERSN"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_WRI$_OPTSTAT_AUX_HISTORY"
Number of failures: 0
Table list: "SQLTXPLAIN"."SQLT$_DBA_AUTOTASK_CLIENT_HST"
Number of failures: 0
Table list: "GSMADMIN_INTERNAL"."AQ$_CHANGE_LOG_QUEUE_TABLE_S"
Number of failures: 0
Table list: "GSMADMIN_INTERNAL"."AQ$_CHANGE_LOG_QUEUE_TABLE_L"
Number of failures: 0
INFO: Total failures during update of TSTZ data: 0 .
An upgrade window has been successfully ended.
INFO: Your new Server RDBMS DST version is DSTv18 .
INFO: The RDBMS DST update is successfully finished.
INFO: Make sure to exit this sqlplus session.
INFO: Do not use it for timezone related selects.

SQL> select * from v$timezone_file;

FILENAME              VERSION     CON_ID
-------------------- ---------- --------
timezlrg_18.dat       18             0



>> NEXT_PAGE                                                              Page  1   2   3   4

Steps to create active data guard in 11g and up-gradation to 12c using manual method - IV

$
0
0


<< PREV_PAGE                                                                                     Page 1234



10. Post upgrade steps



i. Set cluster_database to TRUE

Alter system set cluster_database=TRUE scope=spfile;

Srvctl stop instance –d TESTDBPRD –I NEWDBPRD1

Srvctl start instance –d TESTDBPRD –I NEWDBPRD1

Srvctl start instance –d TESTDBPRD –I NEWDBPRD2




ii. Update profile of DB services with new 12c home


Srvctl upgrade database –d TESTDBPRD –o /rdbms/app/oracle/12.1.0.2/db


iii. Re verify if all components are in valid state in dba_registry


A. Issues Faced


i. ORA-00600: internal error code, arguments: [pesldl03_MMap: errno 1 errmsg Operation not permitted


Issue: Alert log notifying below error during utlrp.sql executions


Errors in file /golden/app/oracle/11.2.0.3.0/diag/rdbms/TESTDBPRD/NEWDBPRD1/trace/NEWDBPRD1_ora_62755.trc (incident=1687133):


ORA-00600: internal error code, arguments: [pesldl03_MMap: errno 1 errmsg Operation not permitted


Cause: /dev/shm was mountable with noexec,nosuid




ORA-600 [pesldl03_MMap: Errno 1 Errmsg Operation Not Permitted] (Doc ID 1625010.1)

Solution:


1. /dev/shm changes


[oracle@testlabdb005a security]$ cat /etc/fstab | grep -i shm
tmpfs /dev/shm tmpfs defaults,nodev,nosuid,noexec 0 0


should be


[oracle@testlabdb005a security]$ cat /etc/fstab | grep -i shm
tmpfs /dev/shm tmpfs defaults,nodev 0 0



as per solution note


Ensure that the /dev/shm mount area is of type tmpfs and is mounted with the following options:


With rw and exec permissions set on it

Without noexec or nosuid set on it"


# cat /etc/fstab

...

tmpfs /dev/shm tmpfs defaults 0 0


2. limit changes

/etc/security/limits.conf
* soft nofile 32768
* hard nofile 65535
* soft stack 10240
* soft nproc 131072

oracle soft nproc 131072
oracle hard nproc 131072



3. in /etc/security/limits.d/ <not sure if in 90-nproc.conf or 20-nproc.conf>


* soft nproc 131072
oracle soft nproc 131072
oracle hard nproc 131072


ii. UTLRP.sql hangs with “single task message” wait event

Issue:

Utlrp.sql hangs with wait event “single task message”


Cause:

Many of the procedure/packages dependent on unresolvable db link


Solution: To remove all DB link pointing to invalid DB link



iii. TIME Zone upgrade status issue



Issue: TIMEZONE pre-requisites were not fulfilling for TZ upgrade , DST_UPGRADE_STATE should be NONE however stucked on DATAPUMP(4) state


SQL> SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME; 2 3 4


PROPERTY_NAME               VALUE
---------------- ------------------------------
DST_PRIMARY_TT_VERSION       14
DST_SECONDARY_TT_VERSION     4
DST_UPGRADE_STATE DATAPUMP  (4)




Solution:

Execute below multiple times till DATAPUMP(4) count reduces and reaches to NONE

SQL> ALTER SESSION SET EVENTS '30090 TRACE NAME CONTEXT FOREVER, LEVEL 32';

SQL> exec dbms_dst.unload_secondary;



Page 1234

ORA-06512: at "SYS.KUPW$WORKER ORA-39126: Worker unexpected fatal error in KUPW$WORKER

$
0
0
Issue: Unable to export due to ORA-06512 ORA-39126 errors


ORA-39126: Worker unexpected fatal error in KUPW$WORKER.FETCH_XML_OBJECTS [MARKER]
ORA-01157: cannot identify/lock data file 201 - see DBWR trace file

ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95


Solution:

i. export backup taken using below EXPDP command terminated with fatal error


expdp as sysdba" FULL=YES directory=EXP_DMP dumpfile=Mydb_FULL.dmp logfile=Mydb_full.log CONTENT=METADATA_ONLY

[oracle@mytestlaba export_dmp]$ more Mydb_full.log
Export: Release 12.1.0.2.0 - Production on Fri Jun 30 10:52:09 2017
Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.
Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
FLASHBACK automatically enabled to preserve database integrity.
Starting "SYS"."SYS_EXPORT_FULL_03": "/******** AS SYSDBA" FULL=YES directory=EXP_DMP dumpfile=Mydb_full.dmp logfile=Mydb_full.log CONTENT=METADATA_ONLY
Estimate in progress using BLOCKS method...
Processing object type DATABASE_EXPORT/EARLY_OPTIONS/VIEWS_AS_TABLES/TABLE_DATA
Processing object type DATABASE_EXPORT/NORMAL_OPTIONS/TABLE_DATA
Processing object type DATABASE_EXPORT/NORMAL_OPTIONS/VIEWS_AS_TABLES/TABLE_DATA
Total estimation using BLOCKS method: 4.867 MB
Processing object type DATABASE_EXPORT/PRE_SYSTEM_IMPCALLOUT/MARKER
Processing object type DATABASE_EXPORT/PRE_INSTANCE_IMPCALLOUT/MARKER
ORA-39126: Worker unexpected fatal error in KUPW$WORKER.FETCH_XML_OBJECTS [MARKER]
ORA-01157: cannot identify/lock data file 201 - see DBWR trace file
ORA-01110: data file 201: '+FRA/oradata/MYDBPRD/temp_1.dbf'
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: at "SYS.KUPW$WORKER", line 11259
----- PL/SQL Call Stack -----
object line object
handle number name
0xb64fe260 27116 package body SYS.KUPW$WORKER
0xb64fe260 11286 package body SYS.KUPW$WORKER
0xb64fe260 13515 package body SYS.KUPW$WORKER
0xb64fe260 3173 package body SYS.KUPW$WORKER
0xb64fe260 12035 package body SYS.KUPW$WORKER
0xb64fe260 2081 package body SYS.KUPW$WORKER
0xb6a82850 2 anonymous block
In procedure BUILD_OBJECT_STRINGS
In procedure BUILD_SUBNAME_LIST with MARKER:.
In function NEXT_PO_NUMBER
FORALL
FORALL
DBMS_LOB.TRIM
DBMS_LOB.TRIM
DBMS_METADATA.FETCH_XML_CLOB
DBMS_METADATA.FETCH_XML_CLOB
In procedure DETERMINE_FATAL_ERROR with ORA-01157: cannot identify/lock data file 201 - see DBWR trace file
ORA-01110: data file 201: '+FRA/oradata/MYDBPRD/temp_1.dbf'
Processing object type DATABASE_EXPORT/PRE_INSTANCE_IMPCALLOUT/MARKER
ORA-39126: Worker unexpected fatal error in KUPW$WORKER.FETCH_XML_OBJECTS [MARKER]
ORA-01157: cannot identify/lock data file 201 - see DBWR trace file
ORA-01110: data file 201: '+FRA/oradata/MYDBPRD/temp_1.dbf'

ii. Verified 'PROCACT' object status : found valid


[oracle@mytestlaba export_dmp]$ sqlplus "/as sysdba"
SQL*Plus: Release 12.1.0.2.0 Production on Thu Jul 6 08:07:15 2017
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

SQL> select owner,object_type,status,object_name from dba_objects where upper (object_name) like '%PROCACT%';

OWNER      OBJECT_TYPE STATUS  OBJECT_NAME
---------- ----------- ------- ----------------------
SYS        TYPE        VALID   KU$_PROCACT_T
SYS        VIEW        VALID   KU$_PROCACT_SYS_VIEW
SYS        TYPE        VALID   KU$_PROCACT_SCHEMA_T
SYS        VIEW        VALID   KU$_PROCACT_SCHEMA_VIEW
SYS        TYPE        VALID   KU$_PROCACT_INSTANCE_T
SYS        VIEW        VALID   KU$_PROCACT_INSTANCE_VIEW
SYS        VIEW        VALID   KU$_PROCACT_SYS_PKG_VIEW
SYS        VIEW        VALID   KU$_PROCACT_SCHEMA_PKG_VIEW

8 rows selected.

iii. Checked temp file status and files : got error



[oracle@mytestlaba export_dmp]$ sqlplus "/as sysdba"
SQL*Plus: Release 12.1.0.2.0 Production on Thu Jul 6 08:09:34 2017
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

SQL> select * from dba_temp_files;
select * from dba_temp_files
*
ERROR at line 1:
ORA-01157: cannot identify/lock data file 201 - see DBWR trace file
ORA-01110: data file 201: '+FRA/oradata/MYDBPRD/temp_1.dbf'




iv. Unable to locate temp files in ASM diskgroup







SQL> select * from v$tempfile;

FILE# CREATION_CHANGE# CREATION_ TS# RFILE# STATUS ENABLED BYTES
----- ---------------- ---------  -- ------- -------  ---- --- BLOCKS CREATE_BYTES BLOCK_SIZE NAME CON_ID
------------------------------------------------

1 607860 27-MAR-13 3 1 ONLINE READ WRITE
0 0 5242880000 8192
+FRA/oradata/MYDBPRD/temp_1.dbf
0

FILE# CREATION_CHANGE# CREATION_ TS# RFILE# STATUS ENABLED
---------- ---------------- --------- ---------- ---------- ------- ----------
BYTES BLOCKS CREATE_BYTES BLOCK_SIZE
---------- ---------- ------------ ----------
NAME
-----------------------------------------------------------
CON_ID
----------
2 8.5988E+12 17-JUN-17 3 5 ONLINE READ WRITE
2.1475E+10 2621440 2.1475E+10 8192
+DATA/MYDBPRD/TEMPFILE/temp.370.947396711
0


FILE# CREATION_CHANGE# CREATION_ TS# RFILE# STATUS ENABLED
---------- ---------- ------ ---- ------ ---- ------
BYTES BLOCKS CREATE_BYTES BLOCK_SIZE
---------- ---------- ------------ ------
NAME
----------------------------------------------
CON_ID
----------
4 3.0310E+10 08-JUN-13 3 2 ONLINE READ WRITE
0 0 2147483648 8192
+FRA/oradata/MYDBPRD/temp_2.dbf
0


FILE# CREATION_CHANGE# CREATION_ TS# RFILE# STATUS ENABLED
------- --------- ------- ---------- ---------- ------- ----------
BYTES BLOCKS CREATE_BYTES BLOCK_SIZE
---------- ---------- ------------ ----------
NAME
-------------------------------------------------
CON_ID
----------
6 3.8783E+10 27-MAR-14 3 3 ONLINE READ WRITE
1.0737E+10 1310720 1.0737E+10 8192
+DATA/MYDBPRD/TEMPFILE/temp.372.947396713
0

FILE# CREATION_CHANGE# CREATION_ TS# RFILE# STATUS ENABLED
---------- ---------- --------- ---------- ---------- ------
BYTES BLOCKS CREATE_BYTES BLOCK_SIZE
---------- ---------- ------------ ----------
NAME
--------------------------------------------------
CON_ID
----------
7 3.8783E+10 27-MAR-14 3 4 ONLINE READ WRITE
1.0737E+10 1310720 1.0737E+10 8192
+DATA/MYDBPRD/TEMPFILE/temp.371.947396713
0

SQL> set line 900 pages 900
SQL> /


FILE# CREATION_CHANGE# CREATION_ TS# RFILE# STATUS ENABLED BYTES BLOCKS CREATE_BYTES BLOCK_SIZE NAME CON_ID
---------- ---------------- --------- ---------- ---------- ------- ---------- ---------- ---------- ------------ ---------- ------------------------------------------- ----------
1 607860 27-MAR-13 3 1 ONLINE READ WRITE 0 0 5242880000 8192 +FRA/oradata/MYDBPRD/temp_1.dbf 0
2 8.5988E+12 17-JUN-17 3 5 ONLINE READ WRITE 2.1475E+10 2621440 2.1475E+10 8192 +DATA/MYDBPRD/TEMPFILE/temp.370.947396711 0
4 3.0310E+10 08-JUN-13 3 2 ONLINE READ WRITE 0 0 2147483648 8192 +FRA/oradata/MYDBPRD/temp_2.dbf 0
6 3.8783E+10 27-MAR-14 3 3 ONLINE READ WRITE 1.0737E+10 1310720 1.0737E+10 8192 +DATA/MYDBPRD/TEMPFILE/temp.372.947396713 0
7 3.8783E+10 27-MAR-14 3 4 ONLINE READ WRITE 1.0737E+10 1310720 1.0737E+10 8192 +DATA/MYDBPRD/TEMPFILE/temp.371.947396713 0



v. Attempted to reuse the temp file names : got error

SQL> alter tablespace temp add tempfile '+FRA/oradata/MYDBPRD/temp_1.dbf' reuse;
alter tablespace temp add tempfile '+FRA/oradata/MYDBPRD/temp_1.dbf' reuse
*
ERROR at line 1:
ORA-01537: cannot add file '+FRA/oradata/MYDBPRD/temp_1.dbf' - file already part of database

SQL> 



vi. New TEMP tablespace created : successfully





[oracle@mytestlaba export_dmp]$ sqlplus "/as sysdba"
SQL*Plus: Release 12.1.0.2.0 Production on Thu Jul 6 08:12:23 2017
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
SQL> alter tablespace temp add tempfile '+FRA' size 1g;
Tablespace altered.


SQL> select * from dba_temp_files;
select * from dba_temp_files
*
ERROR at line 1:
ORA-01157: cannot identify/lock data file 201 - see DBWR trace file
ORA-01110: data file 201: '+FRA/oradata/MYDBPRD/temp_1.dbf'

SQL> create temporary tablespace TEMP1 tempfile '+FRA' size 1G;
Tablespace created.

 

SQL> alter database default temporary tablespace TEMP1;
Database altered.

SQL> drop tablespace TEMP including contents and datafiles;
Tablespace dropped.
SQL> select * from dba_temp_files;

FILE_NAME
--------------------
FILE_ID TABLESPACE_NAME BYTES BLOCKS STATUS
---------- ------------------- ---------- ---------- -------
RELATIVE_FNO AUT MAXBYTES MAXBLOCKS INCREMENT_BY USER_BYTES USER_BLOCKS
------------ --- ---------- ---------- ---------- -----------
+FRA/MYDBPRD/TEMPFILE/temp1.289.948615227
5 TEMP1 1073741824 131072 ONLINE
1 NO 0 0 0 1072693248 130944

SQL> select file_id,file_name,bytes/1024/1024 from dba_temp_files where tablespace_name='TEMP1';

FILE_ID
----------
FILE_NAME
--------------------------------------------------
BYTES/1024/1024
---------------
5
+FRA/MYDBPRD/TEMPFILE/temp1.289.948615227
1024

SQL> alter database tempfile 5 resize 20G;
Database altered.


SQL> select file_id,file_name,bytes/1024/1024 from dba_temp_files where tablespace_name='TEMP1';

FILE_ID
----------
FILE_NAME
---------------------------------------------------------
BYTES/1024/1024
---------------
5
+FRA/MYDBPRD/TEMPFILE/temp1.289.948615227
20480


vii. Re attempted export : Success

[oracle@mytestlaba export_dmp]$ more MYDB_full.log
;;;
Export: Release 12.1.0.2.0 - Production on Thu Jul 6 08:30:02 2017
Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.
;;;
Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
FLASHBACK automatically enabled to preserve database integrity.
Starting "SYS"."SYS_EXPORT_FULL_05": "/******** AS SYSDBA" FULL=YES directory=EXP_DMP dumpfile=MYDB_FULL.dmp logfile=MYDB_full.log CONTENT=METADATA_ONLY
Estimate in progress using BLOCKS method...
Processing object type DATABASE_EXPORT/EARLY_OPTIONS/VIEWS_AS_TABLES/TABLE_DATA
Processing object type DATABASE_EXPORT/NORMAL_OPTIONS/TABLE_DATA
Processing object type DATABASE_EXPORT/NORMAL_OPTIONS/VIEWS_AS_TABLES/TABLE_DATA
Total estimation using BLOCKS method: 4.867 MB
Processing object type DATABASE_EXPORT/PRE_SYSTEM_IMPCALLOUT/MARKER
Processing object type DATABASE_EXPORT/PRE_INSTANCE_IMPCALLOUT/MARKER
Processing object type DATABASE_EXPORT/TABLESPACE
Processing object type DATABASE_EXPORT/PASSWORD_VERIFY_FUNCTION
Processing object type DATABASE_EXPORT/PROFILE
..
.
.
.

Viewing all 155 articles
Browse latest View live