PCP in RAC environment is providing lots of ORA-20100: File o0000071.tmp

Question

Hi,
We are on 12.0.6 on 2 node RAC and 2 node application
appl1 --all services including concurrent
appl2 --all services  including concurrent with PCP and Hardware loadbalancing
db1 and db2 are two rac nodes. running only db and listener.
but my issue is i have configured PCP, provided
appl1:
====
applprod@erpapp1 $ echo $APPLCSF
/nfs_concurent/ERPPROD_erpapp
applprod@erpapp1 $ echo $APPLPTMP
/nfs_concurent/tmp
applprod@erpapp1 $ echo $APPLTMP
/oracle/applprod/inst/apps/ERPPROD_erpapp1/appltmp
applprod@erpapp1 $
appl2:
====
applprod@erpapp2 $ echo $APPLCSF
/nfs_concurent/ERPPROD_erpapp
applprod@erpapp2 $ echo $APPLPTMP
/nfs_concurent/tmp
applprod@erpapp2 $ echo $APPLTMP
/oracle/applprod/inst/apps/ERPPROD_erpapp2/appltmp
applprod@erpapp2 $
=====
and it is mounted onall the Nodes including dbnode.
we have updated the utlfile path of both the db instances with /nfs_concurent/tmp as first parameter in utl_fil
===========================
SQL> show parameter utl
NAME TYPE VALUE
create_stored_outlines string
utl_file_dir string /nfs_concurent/tmp, /usr/tmp,
/nfs_data/ftp/idmin, /nfs_data
/ftp/idmout, /nfs_data/ftp/scb
out, /nfs_data/ftp/scbin, /nfs
data/ftp/citiin, /nfsdata/ft
p/citiout, /nfs_data/ftp/amsou
t, /nfs_data/ftp/amsin, /u02/E
RPPROD, /u01/app/oracle/produc
t/10/db/appsutil/outbound/ERPP
ROD_erpdb1
SQL>
==================================
The issue is both the db nodes are not creating the *.tmp files in sequence they are
RAC node1:
SQL> select applsys.fnd_temp_files_s.nextval from dual;
NEXTVAL
38562
RAC node 2:
SQL> select applsys.fnd_temp_files_s.nextval from dual;
NEXTVAL
39308
select ORDER_FLAG from dba_sequences where sequence_name='FND_TEMP_FILES_S';
ORDER_FLAG
N
If a node 2 is generating the tmp file *309*.tmp which might already created by the other node then the above error is coming to us,
If you shutdown any one node and cleared all the *tmp files from the /nfs_concurrent/tmp then we will not get the error.
In our issue it couldhave been overwrite the earlier file but the issue is we are not having boththe database user ID is defferent, We are in the process of changing the same but we wanted to change our setup, it should create the new no which was not generated by the other node.
Pl help us in achivingthe same..
oracle@erpdb2 $ id
uid=207(oracle) gid=300(oinstall) groups=301(dba),206(citygrp),207(sbigrp)
oracle@erpdb2 $
oracle@erpdb1 $ id
uid=206(oracle) gid=300(oinstall) groups=301(dba),206(sbigrp),207(citygrp)
oracle@erpdb1 $
We have gone through the note "261693.1Troubleshooting ORA-20100 on Concurrent Processing" to trouble shoot the same,
Remind: application is working fine if you run on the single database node..
Regards
Vasu

Answer 1

Hi Husain,
We have gone through the suggested note, in our case the sequence is not jumping, but in both the nodes the sequence is not the same value..
NodeDB1
====
SQL> Select fnd_concurrent_requests_s.nextval from sys.dual ;
NEXTVAL
631070
SQL> /
NEXTVAL
631072
SQL> select FND_S_FILE_TEMP_ID.nextval from sys.dual;
NEXTVAL
8836
SQL> /
NEXTVAL
8837
SQL> select FND_TEMP_FILES_S.nextval from sys.dual;
NEXTVAL
40151
SQL> /
NEXTVAL
40152
SQL>
NodeDB2
===========
SQL> Select fnd_concurrent_requests_s.nextval from sys.dual ;
NEXTVAL
631069
SQL> /
NEXTVAL
631071
SQL> select FND_S_FILE_TEMP_ID.nextval from sys.dual;
NEXTVAL
8844
SQL> /
NEXTVAL
8845
SQL> select FND_TEMP_FILES_S.nextval from sys.dual;
NEXTVAL
40309
SQL> /
NEXTVAL
40310
SQL>
Pl suggest..
Regards
Vasu