Split large file : Recordset per message

Hi experts,
I have a large file that I want to split: flat file -XI. This is an exemple of file :
HEADER;XYZ;123;123456
DETAIL;XYZ;123;123456
DETAIL;XYZ;123;123456
DETAIL;XYZ;123;123456
DETAIL;XYZ;123;123456
TRAILER;XYZ;123;456
Every splited file shoud contain 1500 of 'DETAIL' and also should contain the 'HEADER' structure.
Any idea ?
Cordialy,

Hi,
Yes you can do this by using Multi mapping. Just search a bit online and you will find a lot of information.
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/90dcc6f4-0829-2d10-b0b2-c892473f1571?overridelayout=t…
XI/PI &#8211; 1:n Multi-Mapping using BPM
Split XSLT Output into Multiple Files
http://benxbrain.com/en/E1LTCAH-IDoc-to-Multiple-Files-Mapping-Based-Split-thread-1-2046571.htm
SAP PI 7.3 AEX Multi mapping: Mapping failed with exception
Regards,
Jannus Botha

Similar Messages

File SENDER - "Recordsets per message" being ignored..

Hi
I have a file to file scenario
Sender file:
A1,A2,A3
B1,B2,B3
I want to split the SENDER/SOURCE file in the adapter (NO BPM) so that I am creating n number of RECEIVER files - ONE for each line.
I am trying to use content conversions and setting "Recordsets per message" to 1 but it is completely ignoring this. and ONLY creating 1 file.
I am NOT using any mapping and we are running PI 7.0
Please help
Cheers

Hi
As per my knwoledge you can split files at mapping level in integration engine but at adapter engine you are trying that is FCC which will just convert the data into XML.In FCC it will never split files, you have to do it at mapping level.
Kindly check this link:
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/600b455f-01cd-2b10-0cab-a50e47a8f800?quicklink=index&overridelayout=true
this is a step by process for your query.
Regards,
Nutan

Recordsets per Message in File adapter

Hello All,
I am using Recordsets per Message option in Sender file adapter. And my source file contains more than the specified in Recordsets per Message.
Can you please tell me why it is not working
Regards,
Moorthy

Hi,
Please try to make small addition in your Sender FCC.
Recordset Structure: Row,5000
Recordset per Message: 1
It will divide your file into 5000 records and generate separate messages.
Hope this helps to you.
Regards,
P.Rajesh

Detecting message part after Recordsets per Message

Hi !
Scenario: File -> XI -> RFC
Because of some big text files (about 30 or 40 mb) as input files, we need to use partition our files using Recordsets Per Message in the content conversion parameters of our file adapter (sender).
We need to send to the RFC, the total number of messages that we are sending or at least, a flag to indicate that we are sending the LAST message.
Example:
one file has 21.000 records. We set the "recordsets per message" to 3000. Then XI will receive 7 messages with 3000 records each. When we call the RFC (receiver) after processing the 7th package, we need to send also a flag to indicate that that was the last message.
Any ideas?
Thanks !

Hi Matias,
There's a BPM pattern that deals with something like this. it's called BpmPatternCollectMessage and collects all messages of the same type until it gets a terminating message. so in your case you could define the terminating message as being the last / summary line of your input file (creating a new recordset type for that) and once you receive this message, which should be after the split up parts of your file, you could send out the RFC calls.
Question is, how that will perform with large files.... probably not too well, although there have been some performance improvements to BPMs lately.
Maybe you should try that as well and run a performance test against your solution with the Z Table. The problem with the Z table solution is, that you have to deal with the fact, that mappings can be executed more than once, in the case errors did occurr. so this can mean, that your function and hence the call to the Z table could occurr more than once for the same message. (remember mappings are stateless....)
Regards
Christine

Recordsets per Message not working

I found that Recordsets per Message will split the source message into multiple messages and process them separately. This will save my alot of time and is exactly what I need - but it does not work.
I have in my sender file adapter the following parameters in File Content conversion:
Document Name = MT_SND_ZADIG
Document Namespace = urn://federalmogul.com/ADP/FINFRADP001/00
Recordset Name = RECORDSET
Recordset Structure = RECORDSET,*
Recordset Sequence = Ascending
Recordsets per Message = 10
Key Field Type = String (case-sensitive)
RECORDSET.fieldNames RECORDCODE,ACCOUNTID,CONST_E,CONST_Z2M,MONTHYEAR,JOURNALCODE,FOLIOCODE,RECORDTYPE,DAY,GLACCOUNT,CODINGBLOCK,DEBITAMOUNT,CREDITAMOUNT,TEXT,FILLER1,FILLER2,ACCREF1,ACCREF3,EMPLOYEENO,REPORTTYPE,ZADIGCODE,ACC_TYPE,RESERVED,CENTURY
RECORDSET.fieldFixedLengths 2,6,1,3,4,3,2,1,2,12,18,13,13,20,8,8,12,12,6,4,6,1,1,2
RECORDSET.lastFieldsOptional yes
When I feed in a file with 77 records I would expect it to split it into 7 * 10 recordset and then 1 * 7 - why is it not doing this?

Hi Lynn,
And... what is the result you are getting?
Try to change Recordset Structure with something different that Recorset name.
Something like:
Recordset Name = RECORDSET
Recordset Structure = row,*
row.fieldNames RECORDCODE,ACCOUNTID,CONST_E,CONST_Z2M,MONTHYEAR,JOURNALCODE,FOLIOCODE,RECORDTYPE,DAY,GLACCOUNT,CODINGBLOCK,DEBITAMOUNT,CREDITAMOUNT,TEXT,FILLER1,FILLER2,ACCREF1,ACCREF3,EMPLOYEENO,REPORTTYPE,ZADIGCODE,ACC_TYPE,RESERVED,CENTURY
row.fieldFixedLengths 2,6,1,3,4,3,2,1,2,12,18,13,13,20,8,8,12,12,6,4,6,1,1,2
row.lastFieldsOptional yes
Follow the example is described in:
http://help.sap.com/saphelp_nw04/helpdata/en/2c/181077dd7d6b4ea6a8029b20bf7e55/content.htm
Regards,
Carlos

RecordSets per message

Hi
Im doing a file to file scenario with content conversion. I have 10000 records in the file and i need to split the file using Recordsets per message option. I gave the value as 1000. In SXMB_MONI im getting 10 messages with proper payload. but in the target directory i can see 10 different files with only one (first) record in it.
Im using this configuration
Recordset Name = (left this empty)
Recordset Structure = RECORDSET,1
specified parameter ignoreRecordsetName to true
Did i miss anything

My source and target structure are same
   <Record> -- 0 to unbunded
      <Row>   -- 0 ro unbounded
            <Type/>
            <Org/>
            <Datel/>
       </Row>
       <Row>
            <Type/>
            <Org/>
            <Datel/>
       </Row>
         10000 nodes
   </Record>
Im using the following in the CC
Recordset Name:Record
Recordset structure:Row,1000
Recordset per message:1
This is working fine. I mean ini SXMB moni im getting correct structure as expected. But the all the files in the target directory have only first 1000 records.. what i see in moni is different from what the files have

Recordsets per message - only works with RECORDSET

I have previously got recordsets per message working when I use RECORDSET. I now have a scenario that uses "row" instead of "RECORDSET".
This is working for RECORDSET - it splits into a new message every 100 records........
Recordset Name   RECORDSET
Recordset Structure   RECORDSET,100
Recordsets per Message   100
ignoreRecordsetName   true
RECORDSET.fieldNames   RECORDCODE,ACCOUNTID,CO..............
RECORDSET.fieldFixedLengths   2,6,1,3,4,3,2,1,.............
RECORDSET.lastFieldsOptional   yes
When I change RECORDSET to be row - it only produces 1 message, even though 11,000 records
Recordset Name   row
Recordset Structure   row,950
Recordsets per Message   950
ignoreRecordsetName   true
row.fieldNames   RECORDCODE,ACCOUNTID,CO..............
row.fieldFixedLengths   2,6,1,3,4,3,2,1,.............
row.lastFieldsOptional   yes
Can anyone tell me what I am doing wrong?

I can't believe there are no examples of these kinds of things because it is all about syntax most of the time in File Content Conversion and this can be so time consuming. I got it to work eventually - hopefully this will save someone a couple of hours. This worked:
Recordset Name row
Recordset Structure row,1
Recordsets per message 950
ignoreRecordsetName true

How to split Large file into Multiple small files..

Hi,
My source file is simple XML structure...and target side also i need to send in XML files .. but in the source file I'm getting all the data at a time.. so source file is more than 1000 records.. but i want to process 50 records at a time...
bu using FCC (RecordSet per Message) we can solve this.. but I dont want to do any File conversion.. i want to send in XML file only...
for this.. how we can handle....
Regards
Jain

Jain,
Please see the below screenshots.
http://www.flickr.com/photos/23855877@N07/2991137391/sizes/o/
http://www.flickr.com/photos/23855877@N07/2991137441/sizes/o/
http://www.flickr.com/photos/23855877@N07/2991988028/sizes/o/
http://www.flickr.com/photos/23855877@N07/2991988084/sizes/o/
For No, Name, City Map Directly.
In the example, I've given 5 records to split. If you want 50 records to split, then instead of 5 give 50 in the constant.
raj.

Recordset per Message and then group again

Hi All,
    Advise would be aprreciated for the following issue we are having on a productive environment.
    The interface gets a 3.4MB file from a FTP server, performa a mapping and then saves a new file to the receiver ftp server.
    Sometimes, we get an error because the xml message that the AE sends to the IE does not contains all the information as the flat file. It seems that it runs out of resources and does not finish the xml transformation process.
     I have read several post in the SDN and even sap notes, about using the Recordset per Message option. However, this way I get a lot of IE messages but, also a lot of files in the FTP server and I only need 1. Is there any way I can split them to avoid this issue and then put them all together again? I don't like the "Append option".
     Thanks and kind regards,
         David Encinas.

Hi David,
   Using split you can achive your requirement.
   First create the Structure as per your Source Format.
If your structure has different Sub structure , create for each substructure at the target end.
do the massage mapping in message tab source one message type , target multiple messages...do the same thing in Interface mapping .(Here select source and targets occurence 1 only)
create the Integration Process (BPM)..once receive the message you can validate the message using Switch statement , once condtion has satisfied then use the Transformation step (1:n) and then message use many Send steps.
You can create multiple files with same locations.
I had done same thing for my req....
Reagads,
Venu.

Recordset per message not working

Hi All,
I got a scenario where i need to read a file of more than 1mb which has got around 6000 records.i want to read 100 records at a time and have mentioned
"Recordset per message-100"
in the file configuration.But still i am seeing a xml message of 6000 records in the XI.
This is the configuration part for the file"
Recordset Name:Record
Recordset structure:Row1,*
Recordset per message:100
Row1.fieldNames:FirstName,LastName,MiddleName,City,Place,State,Country
Row1.fieldSeparator:£
Row1.processConfiguration:FromConfiguration
Row1.endSeparator:'nl'
the xml message i get now
i need to process only 100 records at a time...
Any Help greatly appreciated...
Thanks,
Aravind

Hi,
Can you please try this...
Recordset Name:Record
Recordset structure:Row1,*
Recordset Sequence: Ascending
Recordset per message:100
But i assume you might get an xml like this...
<MT>
<Record>
<Row1>
</Row1>
</Record>
<Record>
<Row1>
</Row1>
</Record>
<Record>
<Row1>
</Row1>
</Record>
100 records...
</MT>
Thanks,
Renjith

JMS Content Conversion - Need to control RecordSet per Message

Hi All,
I have done a simple JMS to JMS scenario with content conversion.
I need to control RecordSet per Message ( similarlly we do for the FILE . that is , Recordset Structure = Records, 500 and Recordset per message = 1)
How to achive this in JMS adapter.
Regards,
B.Jude

There doesnt seem to be an option for that in the messagetransforbean
http://wiki.sdn.sap.com/wiki/display/XI/HowTo...ContentconversionmodulewithJ2EEJMS+adapter

Recordset per message for Header-Data-Trailer

Hi Experts,
I need to process a text file with the following structure:
Header
Record1
Record2
Record N
Trailer
How do i use Recordset per message to break the files into multiple files with EACH havign a Header and Trailer
Thanks,
Shobhit

Hi Shobhit,
I think it cannot be possible from File Content Conversion using Recordset per message.
You should do that by means of multimapping 1:N
Regards,
Carlos

"Recordsets per Message" for jdbc adapter

Hi all,
is "Recordsets per Message" kind of option is available for JDBC adapter.
Any help will be appriciated
Thanks
-Kulwant

Hi Kulwant,
its not available.
You can a stored procedure whick picks only certain number of records per each call.
Regards
Inder

FCC : use of recordsets per message

Hi experts,
what is the use of Recordsets per message in FCC,
where we will use this option,
in what type of situations we will use...
Thx in Advance
John

Hi John,
what is the use of Recordsets per message in FCC,
This Parameter Defines How many Records you wish to send per Message,
where we will use this option,
This option is very useful where you have huge volume data to be processed.
This option will process the messages in small bunches there by improving the resource consumption.
Thanks
Sunil Singh

Application to split large file sze?

Dear guys....
I am looking for an application to split large file size before moving out from my Mac into the external HD ... any good suggestion? I have googled it but it is hard to find a reliable one ...I hope you guys could offer your expertise here ... million thanks

Splitting a document may corrupt it and make it no longer workable. Get a larger storage space for your document. If you must do it, do it on a copy, not the original:
http://www.soft32.com/download_192750.html

Split large file : Recordset per message

Similar Messages

Maybe you are looking for