Counting number of files read by File adapter

I have configured a file adapter in a BPEL prcs to read files from a location. The adapter is polling files with a specific file ext(say txt). Is it possible to know the number of files read by the BPEL prcs at the end of the day??? i,e.
is it possible that a BPEL instance after reading a file updates the count in a variable and this variable value is seen by other BPEL instances and the other instances can also update the value....
Any pointers will be helpful...
Thanks

You could make use of some logtable and log it inhere.
In the bpel after a file is pickup up, read this value from the table, add 1 to it, and write it back.
Or try this approach from Lucas :
http://technology.amis.nl/blog/?p=2781

Similar Messages

How to count number of files on zfs filesystem

Hi all,
Is there a way to count the number of files on a zfs filesystem similar to how "df -o i /ufs_filesystm" works? I am looking for a way to do this without using find as I suspect there are millions of files on a zfs filesystem that is causing slow performance sometimes on a particular zfs file system
Thanks.

So I have finished 90% of my testing and I have accepted _df -t /filesystem | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'_ as acceptable in the absence of a known built in zfs method. My main conern was with the reduction of available files from the df -t output as more files were added. I used a one liner for loop to just create empty files to conserve on space used up so I would have a better chance of seeing what happens if the available files reached 0.
root@fj-sol11:/zfstest/dir4# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_5133680_
root@fj-sol11:/zfstest/dir4# df -t /zfstest
/zfstest (pool1 ): 7237508 blocks *7237508* files
total: 10257408 blocks 12372310 files
root@fj-sol11:/zfstest/dir4#
root@fj-sol11:/zfstest/dir7# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_6742772_
root@fj-sol11:/zfstest/dir7# df -t /zfstest
/zfstest (pool1 ): 6619533 blocks *6619533* files
total: 10257408 blocks 13362305 files
root@fj-sol11:/zfstest/dir7# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_7271716_
root@fj-sol11:/zfstest/dir7# df -t /zfstest
/zfstest (pool1 ): 6445809 blocks *6445809* files
total: 10257408 blocks 13717010 files
root@fj-sol11:/zfstest# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_12359601_
root@fj-sol11:/zfstest# df -t /zfstest
/zfstest (pool1 ): 4494264 blocks *4494264* files
total: 10257408 blocks 16853865 files
I noticed the total files kept increasing and the creation of 4 millions files (4494264) after the above example was taking up more time than I had after already creating 12 million plus ( _12359601_ ) which took 2 days on a slow machine on and off (mostly on). If anyone has any idea of creating them quicker than "touch filename$loop" in a for loop let me know :)
In the end I decided to use a really small file system 100mb on a virtual machine to test what happens as the free files approached 0. Turns out if never does ... it somehow increased
bash-3.00# df -t /smalltest/
/smalltest (smalltest ): 31451 blocks *31451* files
total: 112640 blocks 278542 files
bash-3.00# pwd
/smalltest
bash-3.00# mkdir dir4
bash-3.00# cd dir4
bash-3.00# for arg in {1..47084}; do touch file$arg; done <--- I created 47084 files here, more that the free listed above ( *31451* )
bash-3.00# zfs list smalltest
NAME USED AVAIL REFER MOUNTPOINT
smalltest 47.3M 7.67M 46.9M /smalltest
bash-3.00# df -t /smalltest/
/smalltest (smalltest ): 15710 blocks *15710* files
total: 112640 blocks 309887 files
bash-3.00#
The other 10% of my testing will be to see what happens when I try to a find on 12 million plus files and try to pipe it to wc -l :)

Counting number of files in ID

Hello Experts,
                   I have a scenario where a job runs weekly and creates 6 files on one of the prod folders in a legacy system. The files are FTPd to PI. Before picking the files we need to ensure that the count for the files on that folder is 6. If it is 6 or greater then we can pick all those 6 files and send it across to PI (also with archiving it on another folder structure in the legacy system) where PI will send it to another system. If the count is less than 6, then we do not need to do any thing. If count 6 is reached and the files are picked, reset the counter to 0.
    So for this scenario do we need to use a BPM or we can manage with the "Run OS Command...." option??. If any of you have come across such a scenario, then pls let me know the solution. Current version of PI is 7.1 .
     Points to best answer/solution will be given.
Thanks.

Hi, This is just sample script you can reference:
Script_1:
Dim objFileScripting, objFolder
Dim filename, filecollection, currentFolder,counter
currentFolder = ActiveDocument.Path
On Error Resume Next
Set objFileScripting = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFileScripting.GetFolder(currentFolder)
Set filecollection = objFolder.Files
        For Each filename In filecollection
          counter = counter+1
        Next
if counter = 6 then
        For Each filename In filecollection
          objFileScripting.CopyFile filename,"..\Folder_1\" & filename
        Next
else
      call sendmail("mailaddress", "Title Here","Content Here")
endif
set objFileScripting = nothing
     set objFolder = nothing
     set filecollection = nothing
'This function is used to send out email notification, the usage are like:
'call sendmail("mailaddress", "Title Here","Content Here")
Function sendmail (recipient, title, message)
Dim oLapp
Dim oItem
Set oLapp = CreateObject("Outlook.application" )
Set oItem = oLapp.CreateItem(0)
With oItem
.To = recipient
.Subject = title
.Body = message
End With
oItem.Send
Set oLapp = Nothing
Set oItem = Nothing
end function
Script_2: scheduler
Dim ws
On Error Resume Next
set ws = WScript.CreateObject("WScript.Shell")
ws.Run ("script_1.vbs")
while 1 = 1
   WScript.Sleep 1000*60*60       'sleep 1 hour, run your first script every 1 hour
   ws.Run ("script_1.vbs")
Wend
set ws = nothing
Make sure in your source system, you have outlook mail setup there.
Put the two scripts in folder_2, to check the number of files
Regards.
Liang
Edited by: Liang Ji on Apr 4, 2009 4:14 AM

How to count number of files in a folder?

The folder "_Current Assets" has 16 .wav files and nothing else. I'd like Applescript to get this number.
Below, the code that changed between trials is in bold
*Code tried:*
_Trial 1:_
set theFolder to "Macintosh HD:Users:dannykalman:Desktop:_Current Assets"
set numberOfAudioFiles to count of files in theFolder
display dialog numberOfAudioFiles
dialog says: 0
_Trial 2:_
set theFolder to "Macintosh HD:Users:dannykalman:Desktop:_Current Assets"
set numberOfAudioFiles to count of items in theFolder
display dialog numberOfAudioFiles
dialog says: 56
_Trial 3:_
set theFolder to "Macintosh HD:Users:mr:Desktop:_Current Assets"
set numberOfAudioFiles to count of *(files whose kind is "WAVE Audio File")* in theFolder
display dialog numberOfAudioFiles
dialog says: Can’t get every file of "Macintosh HD:Users:dannykalman:Desktop:_Current Assets".
I'm out of ideas : (. What would you suggest?
Thank you so much!

It worked because the path is in the correct syntax. BTW, the count is correct without the -1. Every folder that's been opened has a hidden .DS_Store file (which gets counted) that contains all of its properties, such as view, size, and location. Use the -1 if you want to suppress that file; however, it will count any other hidden ones. For AppleScript queries, use this forum. I had the Hosts move this post to it.

Records count in Receiver File Adapter

Hi All,
I have a scenario IODC to File. The requirement is to get the total record count in the Trailer record of the Target File .Please help.
Regards,
Kevin

Hello,
The requirement is to get the total record count in the Trailer record of the Target File
In your XML, decide which IDOC field you get your records from and then do a similar mapping
IDOC field -> removeContext -> count -> TrailerRecordfield
Hope this helps,
Mark

Performance of File Adapter

Hi Experts,
Many of the Messages are failed in File Adapter that is in the Module. We found it is due to heavy load.
How to improve performance of File Adapter.
How to check the Thread Count for the File Adapter.
What are the parameters i need to change to improve performance.
Regards
Sankar

Hi
Look this blog to inhance performace of File Adapter.
XI/PI File and JDBC Receiver Adapter performance and availability improvements
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/c0ed3b8c-698c-2a10-fbaf-b1df0c82fc8a
Re: Performance Problem with File Adapter using FTP Conection

How to count the number of files read through sender file adapter

Hi ,
I have a scenario where I am reading files from a third party system and trying to send those files to R/3 system .The frequency is 1 file per day .But there can be situation where the source system can put 2 to 3 files in source folder and so XI sender adapter will pick up all the 3 files at time . And I want to send only 1 file at a time to r/3 .
Do you know any way to handle this requirement ?
Thanks ,
Suvarna

Hi,
Like Bhavesh pointed out, you could EOIO as the Quality of Servie in the sender file adapter.
But I guess this would solve your issue only to a certain extent.
Marking this QOS in the sender file adapter would only mean sequential processing of the files. That is, the second file will be sent only when the first file is successfully received by the end system.
It would not ensure that the files are sent only after 15 mins of the first file is sent.
If the 15 mins time period is important, then this needs to be achieved using a BPM which ensures that the messages are sent to the target only every 15 mins.
In case, the frequency of the incoming files is not known (in which case, the having a BPM would be a bad idea) you could use a ztable, which stores the filename and time the file was sent to the R/3.
You could query this table each time, before sending the file data to the R/3
Regards,
Smitha.
Message was edited by:
Smitha Rao

File Adapter - how to get the file count from a folder

Hi All,
I have a requirement that have to poll a directory when the file count is reached to number N (ex:number of files avilable in folder is 5) otherwise it should wait and not pick any of the files. Is it possible to get the file count from a folder using file adapter ?? otherwise please suggest me an approach to achieve this requirement.
Thanks,
JJ

Hi Sarath,
Thank you for your reply.
Go with the list files operation of file adapter it will gives you the number of files in the specified folder as you given. . - this step is already done.
When the number of files reaches your count startup your webservice that which can polls the files. . . - how can i acheive this?? Have to poll the directory and process the number files - please let me know, what could be added to the webservice which is being invoked after cheking file count from parent process.
The reason for the above question is - we cannot use ReadFile operation in second webservice because it will be automatically triggered when the file is avilable. Also SyncRead operation supports reading one file in b/w bpel process. Kindly explain me the implementation steps.
Thanks,
JJ

File adapter: file_name with counter, stable after restart J2EE

Hello guys,
we are on XI 7.0.
in a receiver file channel we have to create file names with a stable counter (7digits).
The payload is IDOC flat file after ABAP mapping (not XML format).
The bad thing with the counter is that it is not stable. After restart of J2EE engine the counter starts again more or less
from the beginning (also described in a FAQ OSS note).
Does anybody have an idea how we could realize a stable counter for file names that also increases the counter after restart of J2EE. Is individual programming necessary? module ?
We thought about SAP number ranges (DB) but how to integrate this in the file channel. Also generating the the number in ERP and send it in the IDOC to XI, but after the ABAP mapping to IDOC flat file we cannot use the payload for variable replacement.
Every help is appreciated
best regards
Hans

Hello Georg,
You can achieve this by developing a Customized adapter module,
Try to maintain the counter in one of the temporary file in the XI system, read the counter number from the temporary file and increment the counter by 1.
rename the receiver filename with the sufix as counter.
replace the counter number in the temporary file with the latest counter number i.e increment by '1'.
regards,
Chandra Shekhar

Reading Multiple files of same kind wth File adapter urgnt plz solve this.

Hi evry one
i am new to BPEL and i am in need of file adapter functionality. I have a folder which has files of naming convention
NSD 123-222 090714T01:23:23 if i suppose i have some hundred files in that folder and i am pointing the file adapter to read files from that folder.The pois all the files have same naming convention with different time stamp and at the same time if all the files have same start name all of the files get picked up at the same time my concern here is if i want to process each file in a while loop as soon as the file get picked up according to the target schema.what can be the condition inthe while loop so that each file gets processed in the loop.
Is there any process where i can count the number of files in the folder so that i can make use of that count in the while process.The files in the folder are in xml format these multiple files of same target schema.
please help me in figuring out this issue.
Regards,
P

There is no direct solution in my opinion. What you can do is - write some java code (either embedded activity or through WSIF). In this code you access the directory and count the files, return that count to your bpel process...then use the while loop with sync read

OracleSOA - File Adapter - Correct mechanism to get the line number from the file

We are using Oracle SOA process to read data from a file, process the data and write it to the database. SOA process is created using Oracle File Adapter to read the data, Oracle BPEL to transform & process the data and Oracle DB Adapter to write the data into the database.
Oracle File Adapter uses native schema to process the fixed length data in the file. The number of rows in the file may exceed more than 500,000, to support processing of large files we use chunk size in the file adapter as 4000 rows. In each read operation, file adapter reads the data that matches to the schema, there might be few rejected rows as well.
Oracle File Adapter property - jca.file.LineNumber gives me only the line number till what line the data is read. Is there any way to get the line number in the xml message, which exactly matches to the line number in the file.?
Flat File Format: (Every file has one header, trailer and multiple data lines),
HEADER02 -
L01
L01
L01
TRAILER02
Line number is one of the crucial requirement in the development of the SOA service, this is used to back trace the information from which source and line the data has come.

Hi Anshul,
Thanks for sharing your thoughts.
I am using DB adapter to insert the data into db.
While doing the xsl transformation I can get the index of the message using position(), but that is not the correct way as there can be rejected lines by the file adapter.
Even if I maintain a local counter variable, it will not consider rejected lines and increment that, which is not correct.
Thanks,
Ravindra

How to Count number of words in a file....

Hi Experts,
I have uploaded the text file, from the application server, like this:
call function 'GUI_UPLOAD'
exporting
    filename = LV_ip_FILENAME
tables
    data_tab = LT_FILETABLE.
The text file contains some number character words.... like "sap labs india..... "
Now, I wanted to count number of words in an internal table LT_FILETABLE.... can anybody help me?

Hi,
Special Characters in Regular Expressions
The following tables summarize the special characters in regular expressions:
Escape character
Special character Meaning
Escape character for special characters
Special character for single character strings
Special character Meaning
. Placeholder for any single character
C Placeholder for any single character
d Placeholder for any single digit
D Placeholder for any character other than a digit
l Placeholder for any lower-case letter
L Placeholder for any character other than a lower-case letter
s Placeholder for a blank character
S Placeholder for any character other than a blank character
u Placeholder for any upper-case letter
U Placeholder for any character other than an upper-case letter
w Placeholder for any alphanumeric character including _
W Placeholder for any non-alphanumeric character except for _
[ ] Definition of a value set for single characters
[^ ] Negation of a value set for single characters
[ - ] Definition of a range in a value set for single characters
[ [:alnum:] ] Description of all alphanumeric characters in a value set
[ [:alpha:] ] Description of all letters in a value set
[ [:blank:] ] Description for blank characters and horizontal tabulators in a value set
[ [:cntrl:] ] Description of all control characters in a value set
[ [:digit:] ] Description of all digits in a value set
[ [:graph:] ] Description of all graphic special characters in a value set
[ [:lower:] ] Description of all lower-case letters in a value set
[ [:print:] ] Description of all displayable characters in a value set
[ [:punct:] ] Description of all punctuation characters in a value set
[ [:space:] ] Description of all blank characters, tabulators, and carriage feeds in a value set
[ [:unicode:] ] Description of all Unicode characters in a value set with a code larger than 255
[ [:upper:] ] Description of all upper-case letters in a value set
[ [:word:] ] Description of all alphanumeric characters in a value set, including _
[ [:xdigit:] ] Description of all hexadecimal digits in a value set
a f
      v Diverse platform-specific control characters
[..] Reserved for later enhancements
[==] Reserved for later enhancements
u2192 More
Special characters for character string patterns
Special character Meaning
Concatenation of n single characters
{n,m} Concatenation of at least n and a maximum of m single characters
{n,m}? Reserved for later enhancements
? One or no single characters
Concatenation of any number of single characters including 'no characters'
*? Reserved for later enhancements
+ Concatenation of any number of single characters excluding 'no characters'
+? Reserved for later enhancements
| Linking of two alternative expressions
( ) Definition of subgroups with registration
(?: ) Definition of subgroups without registration
1, 2, 3 ... Placeholder for the register of subgroups
Q ... E Definition of a string of literal characters
(? ... ) Reserved for later enhancements
for more details please refer the following,
[http://help.sap.com/abapdocu_70/en/ABENREGEX_SYNTAX_SIGNS.htm]
[http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/902ce392-dfce-2d10-4ba9-b4f777843182?QuickLink=index&overridelayout=true]
Thanks,
Renuka S.

Counter in file name in receiver file adapter

Hi,
I am using FTP, my requirement is to add counter in the file name i.e XYZ001.
And added to this i can increment the counter till XYZ020, after this i am supposed to start again from XYZ001.It will be some how handled that when i place XYZ001 for the second time , it will not overwrite the earlier one as it will be proccess by that time.
I could have used addcounter but it doesnt work for FTP:(...also have seen /people/shabarish.vijayakumar/blog/2009/10/05/pixi-maintaining-sequence-numbers-in-file-name-scheme
Do we have any new feature added in PI 7.1 for handling this.
Regards,
Abhi

thanks shabz..
Request you to please let me know how to proceed with the first option which you have mentioned i.e,how to create tables and will i be able to start the counter again from xyz001 once i reach xyz020? is there any abap code to be written (wiil be asking SAP counterpart ) or we need to do it in the dynamic configuration udf? Please let me know how to do the same.
1. Maintain a table in SAP. Use/Code a RFC call to look up the sequence number from this table, incrementing the value after it is read. The result is then used in a dynamic configuration code, and the file name can be generated at that point.
Will this impact performance ?..mine is a credit card authentication interface and response time is imp...
Regards,
Abhi

Programatically count the number of files in a folder

Hi Folks
I need to programatically count the number of files in a given folder and return the value to a Program Unit to be included in further processing.
Can anyone point me in the right direction. I have looked at UTL_FILE but this does not offer what I want.
Does anyone know of an obscure package in Oracle that I could use?
I'm using 9i in a W2K environment.
Many thanks
Simon Gadd

Hi,
following a java based solution i found somewhere on Tom Kyte's AskTom forum.
GRANT JAVAUSERPRIV to TESTUSER;
dbms_java.grant_permission( 'TESTUSER', 'SYS:java.io.FilePermission', 'c:\', 'read' );
create global temporary table DIR_LIST
( filename varchar2(255) )
on commit delete rows
create or replace and compile java source named "DirList"
as
import java.io.*;
import java.sql.*;
public class DirList
    public static void getList(String directory)
                       throws SQLException
      File path = new File( directory );
      String[] list = path.list();
      String element;
      for(int i = 0; i < list.length; i++)
          element = list;
#sql { INSERT INTO DIR_LIST (FILENAME)
VALUES (:element) };
create or replace procedure get_dir_list( p_directory in varchar2 )
as language java
name 'DirList.getList( java.lang.String )';
exec get_dir_list( 'c:\temp' );
select count(*) from dir_list;

Sender File Adapter- Incrementing the counter in file name

Hello,
We have a requirement to read a different file name with every read. Every subsequent file would have a numeric counter updated when we receive it, we need to read that file as the directory would hold the previous files as well.
File structure :- EP<counter>.txt ; so first file would be EP001.txt, next one would be EP002.txt.
How could i do this in Sender file adapter, as there is no variable substitution available like receiver file adapter.
I cannot use wild card characters as the directory would contain previous files as well and we ned to pick the latest file.
Regards

Hi,
>>How could i do this in Sender file adapter, as there is no variable substitution available like receiver file adapter.
Yes true this is not available in sender channel
>>I cannot use wild card characters as the directory would contain previous files as well and we ned to pick the latest file.
You can use EP* as the filename and use processing mode as Set to Read-Only
Regards
Suraj

Counting number of files read by File adapter

Similar Messages

Maybe you are looking for