Duplicate File Handling using Adapter Module

Hi All
The Scenario is like this------
XI is picking a files from FTP location.
Duplicate files are also getting picked by XI.
To handle this i have written a module which is finding it out wheather the file is duplicate or not. If the file is not duplicate then it is getting processed.
Now the problem i am facing is -
I dont want to process the file if it is found duplicate, then what code i should write.
What are the ways i can stop the processing of duplicate file.
Regards
Dheeraj Kumar

Hi
I have implemented a module in which i can find out wheather the file is duplicate or not. If the file is not duplicate process the file.
now the problem is --- if file is duplicate then i dont want to process the file.
How can i achieve this?
Regards
Dheeraj Kumar
Edited by: Dheeraj Kumar on Nov 30, 2009 3:21 PM

Similar Messages

Duplicate file handling using Module

Hi All
The Scenario is like this------
XI is picking a files from FTP location.
Duplicate files are also getting picked by XI.
To handle this i have written a module using NWDS which is finding it out wheather the file is duplicate or not. If the file is not duplicate then it is getting processed.
Now the problem i am facing is
I dont want to process the file if it is found duplicate, then what code i should write.
What are the ways i can stop the processing of duplicate file.
Regards
Dheeraj Kumar

Hi
I have implemented a module in which i can find out wheather the file is duplicate or not. If the file is not duplicate process the file.
now the problem is --- if file is duplicate then i dont want to process the file.
How can i achieve this?
Regards
Dheeraj Kumar
Edited by: Dheeraj Kumar on Nov 30, 2009 3:21 PM

Duplicate File Handling Issues - Sender File Adapter - SAP PO 7.31 - Single Stack

Hi All,
We have a requirement to avoid processing of duplicate files. Our system is PI 7.31 Enh. Pack 1 SP 23. I tried using the 'Duplicate File Handling' feature in Sender File Adapter but things are not working out as expected. I processed same file again and again and PO is creating successful messages everytime rather than generating alerts/warnings or deactivating the channel.
I went through the link Michal's PI tips: Duplicate handling in file adapter - 7.31 . I have maintained similar setting but unable to get the functionality achieved. Is there anything I am missing or any setting that is required apart from the Duplicate file handling check box and a threshold count??
Any help will be highly appreciated.
Thanks,
Abhishek

Hello Sarvjeet,
I'd to write a UDF in message mapping to identify duplicate files and throw an exception. In my case, I had to compare with the file load directory (source directory) with the archive directory to identify whether the new file is a duplicate or not. I'm not sure if this is the same case with you. See if below helps: (I used parameterized mapping to input the file locations in integration directory rather than hard-coding it in the mapping)
AbstractTrace trace;
    trace = container.getTrace();
    double archiveFileSize = 0;
    double newFileSizeDouble = Double.parseDouble(newFileSize);
    String archiveFile = "";
    String archiveFileTrimmed = "";
    int var2 = 0;
    File directory = new File(directoryName);
    File[] fList = directory.listFiles();
    Arrays.sort(fList, Collections.reverseOrder());
    // Traversing through all the files
    for (File file : fList){
        // If the directory element is a file
        if (file.isFile()){
                        trace.addInfo("Filename: " + file.getName()+ ":: Archive File Time: "+ Long.toString(file.lastModified()));
                        archiveFile = file.getName();
                      archiveFileTrimmed = archiveFile.substring(20);
                      archiveFileSize = file.length();
                        if (archiveFileTrimmed.equals(newFile) && archiveFileSize == newFileSizeDouble ) {
                                var2 = var2 + 1;
                                trace.addInfo("Duplicate File Found."+newFile);
                                if (var2 == 2) {
                                        break;
                        else {
                                continue;
    if (var2 == 2) {
        var2 = 0;
        throw new StreamTransformationException("Duplicate File Found. Processing for the current file is stopped. File: "+newFile+", File Size: "+newFileSize);
return Integer.toString(var2);
Regards,
Abhishek

Dummy XML not getting generated from empty file by J2EE adapter module

Hi All,
i know when XI gets an empty input text file, it does not generate a send message for it in sender communication channel.
in my scenario, if i get a file with data, i have to generate an XML message for it using file content conversion - this i have done...........
but if i get an empty text file, then i have to generate a dummy XML send message for it for my BPM.......
So i made a J2EE adapter module to generate dummy xml for empty file.....when i give a file with data in it, then my adapter module is called..... but when i give an empty file, then my adapter module is not called.........
Can anybody suggest why the module processor is not invoking my customer-adapter module when an empty file is given.............but the module processor is invoking my customer-adapter module when a file with data is given.
Thanks,
Rajeev Gupta

Hi Amit,
Below is the code of process method which i used:
public ModuleData process(ModuleContext moduleContext,
ModuleData inputModuleData)
 throws ModuleException
Object obj;
Message msg_audit;
AuditMessageKey amk;
try
File f = new File("/components/XITEMP/sample/PWC/check.txt");
PrintStream ps;
if (f.canWrite())
FileOutputStream fos =new FileOutputStream(f);
ps = new PrintStream(fos);
ps.println("Testing");
ps.close();
fos.close();
else
f = new File("/components/XITEMP/sample/PWC/check4.txt");
if (f.exists() ==false)
f.createNewFile();
obj = inputModuleData.getPrincipalData();
if (obj!=null)
msg_audit = (Message)obj;
amk = new AuditMessageKey(msg_audit.getMessageId(),AuditDirection.OUTBOUND);
Audit.addAuditLogEntry(amk,AuditLogStatus.SUCCESS,"FileCheck: Module called");
else
String str = new String();
String str1 = new String();
 str1="<?xml version=\"1.0\" encoding=\"utf-8\" ?>";
 str1+="<ns:MT_PWC_RECORD xmlns:ns=\"urn://PWC_SR3_01/PWC/Customer\">";
 str1+="<RECORD_SET>";
 str1+="<RECORD>";
 str1+="<RECORD_DATA>BLANK_FILE</RECORD_DATA>";
 str1+="</RECORD>";
 str1+="</RECORD_SET>";
 str1+="</ns:MT_PWC_RECORD>";
str=str1;
inputModuleData.setPrincipalData(str);
catch(Exception e)
try
File f = new File("/components/XITEMP/sample/PWC/check.txt");
PrintStream ps;
if (f.canWrite())
FileOutputStream fos =new FileOutputStream(f);
ps = new PrintStream(fos);
ps.println(e.toString());
ps.close();
fos.close();
catch(Exception ex)
return inputModuleData;
 }
in the above methood, i used file operations at start just to see whether the module is getting invoked or not...so when i give a data file, then the file operations are performed and messages are written in audit log.........but when i give an empty file, then the file operations are not performed - meaning the module is not getting invoked........
Thanks,
Rajeev Gupta

Getting Hashcode of a file in an Adapter Module

Hi all
Does anybody know to dynamically get the hashcode of a file in an Adapter Module.

Hi Michal
contentToString()
          String serialization for logging purposes.
getPrincipalData()
          Retrieves the current principle data, usually the message
getSupplementalData(String name)
          Gets one supplement data value.
getSupplementalDataNames()
          Returns the names of all currently existing supplement data.
setPrincipalData(Object principalData)
          Sets the principle data that represents usually the message to be processed.
setSupplementalData(String name, Object supplementalData)
          Sets a supplement data for the rest of the module chain
this all methods I can see. Can u please elaborate a bit

Regarding Adapter Module for Duplicate file handling at Sender side

Hi All
my requirement is to develop a adapter module . Source is FTP target is R/3 .source Commuincation cahnnel is File sender .
want to handle duplicate files
can any one provide me what are all the steps being used for the same .
any step-via -step doc will be helpful for me ......and adapetr module will be written in NetWeaver Developer studio or which s/W ????
also where to put the ejb or import the ejb s...
Pls help
Regards
Priya

Hi Priya,
YES YOU NEED SAP NetWeaver Developer studio to develop Adapter Modules,or you can use any othe IDEs like Eclipse ....many documents available in sdn ,how to develop AM,refer below link it explaines clearly
let me know which version of XI u working on,if it PI7.1 jar files different.
http://wiki.sdn.sap.com/wiki/display/stage/AdapterModuleToReadExcelFilewithMultipleRowsandMultiple+Columns
Regards,
Raj

Duplicate File issue using FTP adapter - BizTalk 2010

Hi We encountered an Issue of picking Duplicate files in BizTalk 2010 from the FTP Location. Need your assistance on this.
Mainframe sends multiple files a 0 KB file to FTP and BizTalk picks during a particular service window using FTP Adapter.
Receive Location has a pipeline component which decodes the MF file(EBCDICRow format).
Send Port transmits the file which is decoded in the Receive location. There is no mapping or orchestration involved.
When 2 files are placed in FTP Location. BizTalk Transmits the files successfully
No suspended messages in BizTalk but we see below error in event log. we tried reproduce the issue but no luck.
Host Instance running FTP Location is Clustered
There was a failure executing the receive pipeline: "XXXX.XX.Pipelines.Receive_XXX_TransactionsMC_passthru, XXX.XX.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=c2d1f476d5c2f97d"
Source: "EbcdicRowDeCode" Receive Port: "XXX.XX.MCSTransactionsMC" URI: "ftp://XXXXXXXX:21/'XXXX'/MCSDT.DEFKOP.R001.D*.T*" Reason: Unable to cast object of type 'Microsoft.BizTalk.Streaming.BasicStreamWrapper' to type 'XXX.BizTalk.Pipeline.Components.Streams.V3.VirtualStream
MF Puts below Files at FTP Location
PSNOX.MCSDT.DEFKOP.R001.D150406.T009000
PSNOX.MCSDT.DEFKOP.R001.D150406.T002100
BizTalk Picks ( at specified Service window) and Transmits as
PSNOX.MCSDT.DEFKOP.R001.D150406.T009000
PSNOX.MCSDT.DEFKOP.R001.D150406.T009000
PSNOX.MCSDT.DEFKOP.R001.D150406.T002100
Regards
-Sri

Hi Sri,
There could be two reason for such a behavior:
1) Using Non-Clustered Hosts: It is always recommended to use the clustered host for FTP adapter. Because FTP don't allow
any locking mechanism on the files so in case of non-clustered with multiple host instances you might receive same file multiple times through different host instances.
2) If the original document is still being written to the FTP server by the host application, the FTP adapter cannot
delete the document and will retrieve another copy of the document at the next polling interval that is configured for the receive location. This behavior causes document duplication to occur.
Workaround could be:
Configure the host application to write to a temporary folder on the same hard disk as the public FTP folder and to periodically move the contents of the temporary folder
to the FTP folder. The temporary folder should be on the same hard disk as the public FTP folder to make sure that the move operation is atomic. An atomic operation is an operation that is functionally indivisible. If you write data to the public FTP folder
by using the BizTalk Server FTP adapter, you can do this by specifying a Temporary Folder property in the FTP Transport Properties dialog box when you configure a send port. If you specify a Temporary Folder property, make sure that this folder is on the same
physical disk as the public FTP folder.
Configure the FTP receive location to operate within a service window when the host application is not writing data to the FTP server. You can specify the service window
when you configure the receive location properties.
Refer: Known Issues with the FTP
Adapter
Rachit
Please mark as answer or vote as helpful if my reply does

Effective error handling in Adapter Module

eX(i)perts..
Can anybody share the effective error handling strategies inside adapter modules?
Specifically looking for
1. Module parameters configuration related error handling
2. Business logic failure owing to incorrect data.
-- Amol

Hi,
Please see the following PDF documents to develop the Module processor programs.
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/02706f11-0d01-0010-e5ae-ac25e74c4c81
http://help.sap.com/saphelp_nw04/helpdata/en/0d/00453c91f37151e10000000a11402f/frameset.htm
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/02706f11-0d01-0010-e5ae-ac25e74c4c81
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/e21106cc-0c01-0010-db95-dbfc0ffd83b3
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/9913a954-0d01-0010-8391-8a3076440b6e
help : http://ifr.sap.com
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/6f83d790-0201-0010-629d-ab5bf10c94e4
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/f02d12a7-0201-0010-5780-8bfc7d12f891
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/7bf6e190-0201-0010-83b7-cc557eb76abe
Some weblogs for JAVA mapping for IDoc mapping
/people/prasad.ulagappan2/blog/2005/06/29/java-mapping-part-i
/people/prasad.ulagappan2/blog/2005/06/29/java-mapping-part-ii
Difference in using java,xslt,message mapping
Regards
Chilla..

Converting INVOIC02 idoc to EDIFACT INVOIC message (Using Adapter Module

Hi Friends,
We are sending customer idoc (INVOIC02) from R/3 to external system through PI. We want to generate EDIFACT output file (message type : INVOIC).
Is it possible to convert IDoc message to EDI file format through java adapter module ? Will adapter module support this without using Seeburger adapter ?
kindly clarify, friends.
Kind regards,
Jegathees P.

There is an alternative to Seeburger.
DataDirect also has complete EDI-XML conversion by way of deploying a module which is called from an adapter. The range of parameters with the conversion is also quite extensive, allowing tolerance for partners who may have included non-standard EDI qualifiers and such. The XML tags make for easier mapping. StylusStudio XML Suite includes the EDI<->XML adapter so you can try the conversions each way directly in that package. If you are happy with it, you need to then purchase the package from DataDirect. We did this and implemented EDI for X12 and EDIfact, including Orders and Invoices and some other messages. The communications part was not included. We wrote the EDI files and used a separate process to actually send and receive the files. The IDoc <-> EDI mapping was done using the normal graphical mapping of SAP PI. The Stylus Studio package includes the functionality to generate the XSD, which we imported to SAP PI.

HELP!!! ... need to get Window Handle using PDA module

I am trying to find a way to get the current window handle using the PDA module. I need this handle to call other function via Win dll's. Has anyone done this before? Any help would be greatly appreciated (using windows mobile 6.1 and Labview PDA 8.5) Thanks
Greycat

Thanks Mike ...
I understand that this function does exist, but somehow I am not writing my wrapper properly or calling the function properly because everytime I build my labview app I get a "GetForegroundWindow is a missing VI or C file." - I cannot for the life of me figure this out ... what I was really asking is if anyone has succeeded calling this function and getting the Window Handle. What I really want to do is run my Labview program on the PDA in full screen mode with no title bar and no SIP button visible or accessible, but to my knowledge, that will take a function call to the aygshell.dll (namely the SHFullScreen function) and that function requires a window handle to work properly ... any more help would be appreciated ... Thanks again
Greycat

Issue with overwriting msg using adapter module when WSS enabled in SOAP CC

Hi guys,
I have a SOAP->RFC scenario where on the sender side I'm using an adapter module. This adapter module should modify the message content based on some conditions. This works fine unless the WSS is enabled in the sender SOAP CC. When enabled, the message doesn't get modified. If disabled, it works fine. Do you have any explanation for this? Is there any workaround for this?
Thanks, Olian

Hi Olian,
For 2004s (PI71) too sender adapter doesn't support modules (in normal SOAP mode and not in AXIS)
http://help.sap.com/saphelp_nwpi71/helpdata/EN/a4/f13341771b4c0de10000000a1550b0/frameset.htm
>>if I use the module w/o WSS enabled on the CC,
Can you tell me how you did this??
Regards
Suraj

How to get file name in adapter module

Hi all
Is it possible to get the file name in a adapte module? If so how can I achieve this?

Hi,
Check below thread. it shows code for file name.
[Re: Query regarding Adapter Module development]
-Gouri

Naming a file in the adapter module

Hi All,
I have written an adapter module that is used in a receiver mail adapter. In my module I am renaming the attachment name to "abc.cpi". The problem is I am getting the output from the adapter as abc.cpi.xml. I tried changing the content type in the adapter module to text/plain that gave me an output as abc.cpi.txt. How can I get only abc.cpi as the name of the attachment using this module?
Thanks
Abinash

Hi,
This might be help you
http://help.sap.com/saphelp_nw04/helpdata/en/cd/5af7c0c994e24fb0d0088443513de2/frameset.htm
http://help.sap.com/saphelp_nw04/helpdata/en/cd/5af7c0c994e24fb0d0088443513de2/frameset.htm
http://help.sap.com/saphelp_nw04/helpdata/en/14/80243b4a66ae0ce10000000a11402f/frameset.htm
Regards
Agasthuri Doss

Maximum file handled using dom4j

Hi,
Wanna know whats the maximum xml file size that can be handled using dom4j.
To be precise, I have a 1gb xml file that I wanna parse, I have implemented it using SAX, but wanna know if dom4j can be used?
Regards,
R

Just try, you'll probably only be limited by available memory. I assume that building any kind of DOM tree of the document (be it org.w3c.dom or dom4j) will use more memory than the original file. So you'll need > 1 GB of memory for your file. If you have to handle files that big and don't absolutely need it all at the same time then SAX is probably the better solution as it doesn't require the whole file to be in memory at a time.

Error in Duplicate file Handling

Hi All,
We want to avoid duplicate files in XI for that we used below code in UDF in mapping
try{
String processedFileDatabase = processedFile[0];
String sourceFileName;
DynamicConfiguration attrib = (DynamicConfiguration)container.getTransformationParameters().get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
DynamicConfigurationKey fileKey = DynamicConfigurationKey.create("http:/"+"/sap.com/xi/XI/System/File","FileName");
attrib.put(fileKey,attrib.get(fileKey));
sourceFileName = attrib.get(fileKey);
File fileDB=new File(processedFileDatabase);
if (!(fileDB.exists() && fileDB.canWrite() && fileDB.canRead())){
fileDB.createNewFile();
Vector fileNameList = new Vector();
BufferedReader br = null;
br = new BufferedReader(new FileReader(processedFileDatabase));
String name = new String();
//loop and read a line from the file as long as we dont get null
while ((name = br.readLine()) != null)
//add the read word to the wordList
fileNameList.add(name);
br.close();
boolean fileAlreadyProcessed = fileNameList.contains(sourceFileName);
if (!fileAlreadyProcessed) {
Writer output = new BufferedWriter(new FileWriter(new File(processedFileDatabase),true));
output.write(sourceFileName + "\r\n");
output.flush();
output.close();
result.addValue("" + !fileAlreadyProcessed);
}catch(java.io.IOException e){
e.printStackTrace();
But this is not working ..its unable to Map....
Plz help us in this regard,

The file name is not going to fileDB and hence it is unable to create the target element.
I followed the below link solution 2
http://wiki.sdn.sap.com/wiki/display/XI/DifferentwaystokeepyourInterfacefromprocessingduplicate+files
I seen that fileDB file is empty ,

Duplicate File Handling using Adapter Module

Similar Messages

Maybe you are looking for