XML to Flat File design issue

Hi,
A newbie to SSIS but was able to create an SSIS package which extracts XML data from one of the SQL server columns using "Execute SQL Task" and passes that to a for each loop container which contains an XML task for apply transform to each input
xml and append it to a flat file(additonally using a script task within for each container).
All good so far but now I want to apply conditional splitting to "Execute SQL Task" in the above and additionally pipe it to the exact similar additional process(For each container doing xml task as described above) for a different kind of
flat file.
If I alter the design to use the data flow approach and apply OOTB conditional split, I run into not knowing how to connect and execute more than one  foreach container and embedded XML and script task (dataflow to control flow connection)
It is easy to put everything in a sequence container and repeat the Execute SQL Task . But to me that is very inefficient.
Any creative ideas or pointers to some internet content which tells me how can this be done most efficiently.
Hope my question makes sense and let me know if you need more clarification.
Thanks in advance.
SM

As I understand what you're asking for is a way to create conditional branches to do different typeof processing for each subset of data. For this you can do like below 
1. Add set of tasks to additional processing for each type of flat file and link them to Execute sql task.
2. Make the precedence constraint option as Expression And Constraint for each of them. Make constraint as OnSuccess and expression based on your condition. You may need to create SSIS variables to capture value of fields to be used in expression manipulation.
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Similar Messages

  • Urgent, Help on conversion: xml to flat file

    Dear experts,
    Here is the scenario, XML FILE ->XI-> JMS
    In JMS recevier side, we use localejbs/AF_Modules/MessageTransformBean to convert xml to flat file.
    Transform.Class: com.sap.aii.messaging.adapter.Conversion
    Transform.ContentType: text/plain;charset=utf-8
    xml.conversionType: SimpleXML2Plain
    xml.fieldFixedLengths: 5,20,5,8
    When input source xml file contents are non-unicode, it works fine.
    But when the contents contain unicode characters such as Chinese word,  it works but with wrong length in the output flat file. It seems xi treat one Chinese word length is 1. Actually, one chinese word should take 2 bytes.
    My question is how to make XI handle unicode characters  with correct lengths while conversing to flat file.
    Thanks in advance.

    Jai Shankar
    Thanks for your response.
    I am already using the standard adapter modules provided by SAP:
    localejbs/AF_Modules/MessageTransformBean
    Transform.Class: com.sap.aii.messaging.adapter.Conversion
    My input file's encoding is UTF-8
    "You need to change the encoding scheme according to your input file" , do you mean change the output file's encoding or else?
    I can't find the article you mentioned: "Encoding schemes in XI", Would you please give me the link?
    Thanks a lot.

  • Call Pipeline in orchestration for converting XML to flat file

    Hi ,
    Can we call Pipeline in orchestration for converting XML to flat file ?
    Or we have some another option for this task ?
    Prakash

    Hi,
    You can refer this article from Abdul Rafay on how to work with Flat files and how to call receive and send pipelines within your orchestration. He
    has provided detailed step by step explanation of the process.
    Calling Send and Receive Pipelines
    from the orchestration expression shapes
    The code shared by Abdul at http://abdulrafaysbiztalk.wordpress.com/files/2009/06/processingflatfiles.doc (Change the extention to rar) and try it.
    One common mistake I have seen people doing will working with Flat File Assembler is that they miss to set the Document schema property in the Properties
    window to the flat file schema.
    If the Document schema property is not specified, runtime schema discovery will be attempted. During schema discovery, BizTalk Server attempts
    to determine the correct flat file schema to assemble the message with, based on the namespace and root node of the message, its added performance hit.
    Rachit

  • OSB XML to Flat file(write)- MFL?

    Hi,
    OSB 11G
    I will be invoking the db to get XML data and I have to write to a text file(pipe delimiter).
    I saw below forum, but I am not sure how to create MFL .Can you anyone pls mention the steps. (MFL ->XML to flat file )
    Do I have import xsd in Native Builder and convert to a flat file with pipe delimiter ?
    How to write a CSV file in OSB
    Thanks
    Edited by: soauser on Jul 7, 2011 8:07 PM

    truth must be said, MFL is very sensitive to input data, any deviation from the contract is punished with a NullPointerException, especially when the input is in binary format. One would expect a better error message, but what can we do.
    Make sure your input data complies with the Schema, for instance do a xsd validation...

  • Xml to flat file scenario

    hi guys,
    I am new to XI.am tryin to configure xml to flat file scenario.i have configured the scenario but the problem is am getting the flat file into my receiver system with no content..what might be the problem.pls do help me out

    Hi,
    Please find here with the required parameters for receiver end file to get it as flat file
    Please refer below scenarios, this could be used as reference for your scenario
    IDOC-XML to flat file
    https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/46759682-0401-0010-1791-bd1972bc0b8a
    XML to Flat file conversion
    XML to flat file conversion
    File Receiver with Content Conversion
    /people/arpit.seth/blog/2005/06/02/file-receiver-with-content-conversion
    Configuring the Receiver File/FTP Adapter
    http://help.sap.com/saphelp_nw04/helpdata/en/95/bb623c6369f454e10000000a114084/frameset.htm
    Please see the below links for file content conversion..
    /people/michal.krawczyk2/blog/2004/12/15/how-to-send-a-flat-file-with-fixed-lengths-to-xi-30-using-a-central-file-adapter - FCC
    /people/michal.krawczyk2/blog/2004/12/15/how-to-send-a-flat-file-with-fixed-lengths-to-xi-30-using-a-central-file-adapter - FCC
    File Content Conversion for Unequal Number of Columns
    /people/jeyakumar.muthu2/blog/2005/11/29/file-content-conversion-for-unequal-number-of-columns - FCC
    Content Conversion (Pattern/Random content in input file)
    /people/anish.abraham2/blog/2005/06/08/content-conversion-patternrandom-content-in-input-file - FCC
    Thanks
    Swarup

  • JCA flat file read issue

    Hi,
    We needs to read a flat file and transform it to destination xml format. Then send it to destination file location.
    Steps we have done:
    1. Create JCA adapter and configure the flat file schema
    2. Create proxy based on the jca
    3. create transformation for the source to target schema (this has no namespace)
    4. Create BS for sending the output
    Everything workins as expected when testing from OSB test console. But then the file is placed in the source folder, the output xml has the namespace xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/" to the root node.
    e.g,
    <Root xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/" >
    <Child>
    <Child1/>
    <Child2/>
    <Child3/>
    </Child>
    </Root>
    But expected output is
    <Root>
    <Child>
    <Child1/>
    <Child2/>
    <Child3/>
    </Child>
    </Root>
    We tried converting the xml to string then repalcing the xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/" value with balnk.
    Also we tried with hadcorded xml using assign action instead of transformation. Even the harded xml also having the namespace to the root node.
    But the end system is failing due to this namespace value.
    Help me to resolve the issue.
    Thanks,
    Vinoth
    Edited by: Vinoth on Jun 8, 2011 10:12 PM

    Ideally your endsystem should not fail if you specify any number of namespace identifiers in the XML unless you are using them in elements or attributes within the XML. They should try to resolve this on their side.
    But to see whats going on in OSB, can you please paste log the $body variable in the before the publish action and paste the content here. Or send the sbconfig of the Proxy and business to me on my email mentioned in the profile if possible.

  • XML to Flat File DB Table So I and Others Can Use SSRS for Reporting

    I'm using a program called Orbeon Forms to quickly create and distribute forms on the local network.
    The program just recently got updated to Version 4.6 so as to use SQL Server DB as a backend. (I was using MySQL in a lower versions)
    The data created when you submit a form into the database is held in  xml fomat in one of the columns. (see below).
    I am trying to create a separate table, to pull that data from the xml column (for that particular form) and place it in to a seperate table using the xml tags as columns.
    I have no idea where to start or how to do this. Please keep in mind I'm a novice when it comes to sql server (but I do know how to do select statements :-)....
    Is this very difficult to do? If it is (for me at least) where can I go to get someone to do it for me along with the documentation to do it?  I was also thinking of a way to create a trigger on the Orbeon table (this is how it is setup currently in
    the MySQL database) to automagically insert any saved form data to the new form data table. Is this also difficult to do? And can I have a trigger for each form I create on the table?
    If anyone can help, I'd appreciate it. Thanks.
    I would eventually like to add the form data from the old MySQL database table into this new Orbeon table but it appears the upgrade also added two extra columns. I figuring once in the Orbeon table then I will have to run whatever statements to place into
    the separate table so I can use the data.
    Randy Marrero

    Erland thanks for the reply. Even after reviewing the link you sent I'm still having problems. Guess that is why I'm not a programmer. Anyway I've create a test data form xml I've created and it is copy and pasted below.
    Again I'm only really interested seeing if I can get the information out of the xml and create it in its own flat file table. I keep reading that creating a trigger (when the form gets submitted into this database in the xml format, an on insert trigger
    would take all the data submitted and inserted into another table with just the data, into separate columns)  for this is the best way but I'm not sure how to do this either. I have a copy of what the MySQL trigger would look like but keep running
    into errors when I try to run it in SQL Server.  Also the issue with how I would create another trigger if / when I create another form. I read somewhere I can only create one on insert trigger per table, still don't know this to be true. Anyway below
    is the xml as it is in the xml table in the database.
    XML STARTS HERE------------------
    <
    formxmlns:oxf="http://www.orbeon.com/oxf/processors"xmlns:xxi="http://orbeon.org/oxf/xml/xinclude"xmlns:xh="http://www.w3.org/1999/xhtml"xmlns:xi="http://www.w3.org/2001/XInclude"xmlns:ev="http://www.w3.org/2001/xml-events"xmlns:saxon="http://saxon.sf.net/"xmlns:xs="http://www.w3.org/2001/XMLSchema"xmlns:odt="http://orbeon.org/oxf/xml/datatypes"xmlns:fb="http://orbeon.org/oxf/xml/form-builder"xmlns:xxf="http://orbeon.org/oxf/xml/xforms"xmlns:xbl="http://www.w3.org/ns/xbl"xmlns:xpl="java:org.orbeon.oxf.pipeline.api.FunctionLibrary"xmlns:sql="http://orbeon.org/oxf/xml/sql"xmlns:p="http://www.orbeon.com/oxf/pipeline"xmlns:fr="http://orbeon.org/oxf/xml/form-runner"xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"xmlns:xf="http://www.w3.org/2002/xforms"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xmlns:exf="http://www.exforms.org/exf/1-0">
      <
    Header>
        <
    HHIFR_Patch>/fr/service/persistence/crud/bfp/permits/form/738433584d6c0aff25e0ab8ca1bd99bcc7e10b49.bin</HHIFR_Patch>
      </
    Header>
      <
    Permit_Number_Hdr>
        <
    Permit_Number>Test123</Permit_Number>
      </
    Permit_Number_Hdr>
      <
    Permit_Info_Hdr>
        <
    Permit_Date>2014-07-15-05:00</Permit_Date>
        <
    Permit_Station>HQ</Permit_Station>
        <
    Permit_Employee_Number>0676</Permit_Employee_Number>
        <
    Permit_Status>Active</Permit_Status>
        <
    Employee_Name>Randy
    Marrero</Employee_Name>
      </
    Permit_Info_Hdr>
      <
    Applicant_Info_Hdr>
        <
    Applicant_Name>Randy
    Marrero</Applicant_Name>
        <
    Applicant_Phone>8434741099</Applicant_Phone>
        <
    Applicant_Email>[email protected]</Applicant_Email>
      </
    Applicant_Info_Hdr>
      <
    Mailing-Address-Section>
        <
    Mailing_Address>40
    Summit Dr</Mailing_Address>
        <
    Mailing_Suite/>
        <
    Mailing_City>Hilton
    Head Island</Mailing_City>
        <
    Mailing_State>SC</Mailing_State>
        <
    Mailing_Zip>29926</Mailing_Zip>
      </
    Mailing-Address-Section>
      <
    Physical_Burn_Hdr>
        <
    Physical_Address>35
    Summit Dr</Physical_Address>
        <
    Physical_Suite/>
        <
    Physical_City>Hilton
    Head Island</Physical_City>
        <
    Physical_State>South
    Carolina</Physical_State>
        <
    Physical_Zip>29926</Physical_Zip>
      </
    form>
    XML ENDS HERE ------------
    Randy Marrero

  • XML to flat file conversion

    Hi,
    Iam working on Idoc to file scenatio.Using java mapping I have mapped the Idoc to an XML format. Now I have a problem in converting the XML format to flat file format.
    My XML is as follows:--
    <Record_Idoc>
      <Record>
        <Header>
            <Order_no>10000398</Order_no>
            <Order_Type>ZSTQ</Order_Type>
            <Material_Head_no>1800406</Material_Head_no>
            <Material_Head_Desc>MARC MOL MEZ R16 FY02.</Material_Head_Desc>
            <Production_version>0003</Production_version>
            <Test_Order/>
            <Quantity_planned>801.00</Quantity_planned>
            <unit>PCE</unit>
         </Header>
        <operations>
             <operation_number>0010</operation_number>
             <Work_center>P_SIG</Work_center>
             <setup_start_date>20020619</setup_start_date>
             <Setup_start_time>060000</Setup_start_time>
             <Processing_start_date>20020619</Processing_start_date>
             <Processing_Start_Time>060000</Processing_Start_Time>
             <End_date>20020619</End_date>
            <End_Time>061600</End_Time>
         </operations>
        <operations>
             <operation_number>0020</operation_number>
             <Work_center>P_BFB</Work_center>
             <setup_start_date>20020619</setup_start_date>
             <Setup_start_time>060000</Setup_start_time>
                <Processing_start_date>20020619</Processing_start_date>
             <Processing_Start_Time>060000</Processing_Start_Time>
             <End_date>20020619</End_date>
             <End_Time>074926</End_Time>
          </operations>
      </Record>
    </Record_Idoc>
    My output flat file should be as follows:--
    Each record contains the following lines
    Header.........
    Operations......
    Operations......
    I have used File content Conversion in message Protocol
    These are my setting:--
    Header.addHeaderLine  nl
    Header.fieldSeparator  ,
    Header.endSeparator   nl
    operations.addHeaderLine  nl
    operations.fieldSeparator  ,
    operations.endSeparator   nl
    But seems that these settings does not work.
    Could some one help in solving this issue.
    regards
    Praveen

    try,
    Recordset Structure - Header,operations
    Header.fieldSeparator ,
    Header.endSeparator nl
    Header.addHeaderLine X  *****
    operations.fieldSeparator ,
    operations.endSeparator nl
    and in case it doesnt work try removing the Record_Idoc node from the XML in mapping and then try the Cont. Conv.
    X can be
    0 – No header line
    1 – Header line with column names from the XML document
    2 – As for 1, followed by a blank line
    3 – Header line is stored as NameA.headerLine in the configuration and is applied
    4 – As for 3, followed by a blank line

  • Storing Persistent Data In A Flat File -- Design Ideas?

    I have an application that needs to store a small amount of persistent data. I want to store it in a flat config file, with categories and key-value pairs. The flat file might look something like this:
    John:
    hair=green
    weight=170
    Sally:
    eyes=blue
    weight=110
    and so on. My application will initialize a custom class with the data stored in the file, and then work with that class. When updates are made to the data as the application runs, the file will need to be changed too (so that changes will be reflected even if the program crashes, eg).
    What is the best way to implement this? Does Java have any built in classes that allow for something like this? I was thinking about Serializable (which I've never used), but I want the file to be human readable and editable. How about using RandomAccessFile? I'm guessing there is a better way....
    Thanks for any advice,
    John

    I'd use a XML structure; classes for XML storing/parsing are part of the API, the structure of XML is flexible enough and human-readable.

  • Flat file truncation issue

    I am attempting to perform a fairly standard operation, extract a table to a flat file.
    I have set all the schemas, models and interfaces, and the file is produced how I want it, apart from one thing. In the source, one field is 100 characters long, and in the output, it needs to be 25.
    I have set the destination model to have a column physical and logical length of 25.
    Looking at the documentation presented at http://docs.oracle.com/cd/E25054_01/integrate.1111/e12644/files.htm - this suggests that setting the file driver up to truncate fields should solve the issue.
    However, building a new file driver using the string 'jdbc:snps:dbfile?TRUNC_FIXED_STRINGS=TRUE&TRUNC_DEL_STRINGS=TRUE' does not appear to truncate the output.
    I noticed a discrepancy in the documentation - the page above notes 'Truncates strings to the field size for fixed files'. The help tooltip in ODI notes 'Truncates the strings from the fixed files to the field size'. Which might explain the observed lack of truncation.
    My question is - what is the way to enforce field sizes in a flat file output?
    I could truncate the fields separately in each of the mapping statements using substr, but that seems counter-intuitive, and losing the benefits of the tool.
    Using ODI Version:
    Standalone Edition Version 11.1.1 - Build ODI_11.1.1.5.0_GENERIC_110422.1001

    bump
    If this is an elementary issue, please let me know what I've missed in the manual.

  • J2SE XML to Flat File Content Conversion

    Hi
    I've currently got a scenario which sends a flat file to the integration server, it gets mapped and sent to a receiver adapter on a deployed j2se engine.
    I'm now trying to convert the xi-xml structured file to a flat file again on the j2se side (the same flat file format it originally had).
    My original flat file looks like this -
    477
    477=AA1
    My xml file looks like this -
    <?xml version="1.0" encoding="UTF-8"?>
    <ns0:ResultMessage xmlns:ns0="urn:xxxx-com:a_test_j2se_filetofile">
    <Item>
              <field1>477</field1>
    </Item>
    <Item>
              <field1>477</field1>
              <field2>AA1</field2>
    </Item>
    </ns0:ResultMessage>
    I am using these content conversion parameters:
    xml.addHeaderLine=0
    xml.fieldSeparator==
    xml.endSeparator='nl'
    I get this error on the integration engine (sxmb_moni):
    Error while sending by HTTP (error code: 500, error text: Internal Server Error:java.lang.NullPointerException) (See attachment HTMLError for details)
    and the j2se adpater log says this:
    17:16:32 (4120): Message "13b9d644-54c9-4ffb-0c40-db4c14458d77" of type "application/xml", kind "B" received
    17:16:32 (4124): Parsing XML message
    17:16:32 (4131): ERROR: Message processing failed with "java.lang.NullPointerException"
    What am i missing?

    So, now... I did a test configuration in XI and sent your test-payload...it worked.
    The J2SE adapter configuration:
    File adapter java class
    classname=com.sap.aii.messaging.adapter.ModuleXMB2File
    version=30
    mode=XMB2FILEWITHCONVERSION
    #Adress for XMB endpoint -
    XI.httpPort=8111
    XI.httpService=/file/receiver
    #File Adapter specific parameters -
    file.createDir=1
    file.targetDir=c:/transfer/inbound
    file.targetFilename=xmboutput.txt
    #file.writeMode=append
    #file.writeMode=overwrite
    file.writeMode=addCounter
    file.counterMode=immediately
    #file.counterMode=afterFirst
    file.counterSeparator=_
    file.counterFormat=00000
    file.counterStep=1
    #File Content Conversion specific parameters -
    xml.addHeaderLine=0
    xml.fieldSeparator==
    xml.endSeparator='nl'
    And here the configuration of the receiver communication channel in the integration directory:
    Adapter Type: XI
    Receiver
    Transport-Protocol: HTTP 1.0
    Message-Protocol: XI 3.0
    Adapter-Engine: Integration Server
    Adressing-Type: URL Address
    Target Host: <yourJ2SEip>
    Service Number: 8111
    Path Prefix:http://<yourJ2SEip>:8111/file/receiver
    Authentication Data
    Logon data for non-SAP systems
    User
    Password
    That's it... sent your payload and got the wished result:
    477
    477=AA1
    Regards,
    Heinrich

  • Convert XML to flat file with File adapter

    Hi all.
    Trying to configure a file adapter according to the following link.
    http://help.sap.com/erp2005_ehp_04/helpdata/EN/d2/bab440c97f3716e10000000a155106/frameset.htm
    What i want it to do is the following.
    I have the following incoming message:
    <header>
         <h_field1></h_field1>
         <h_field2></h_field2>
    </header>
    <data>
         <d_field1></d_field1>
         <d_field2></d_field2>
         <infotext>
              <i_field1></i_field1>
              <i_field2></i_field2>
         </infotext>
    </data>
    I want this to become a flat file that looks as follow.
    h_field1¤h_field2
    d_filed1¤d_field2
    i_field1¤i_filed2
    Every field in each segment should be concatenated with the ¤ char as separator.
    I've done the following:
    Content Conversions Parameters
    Record Structure: header,data,infotext
    header.fieldSeparator   ¤
    data.fieldSeparator    ¤
    infotext.fieldSeparator   ¤
    The adapter just gets into wait mode.
    Does anybody know why?
    BR
    Kalle

    Hello kalle,
    As per your source structure, if you perform FCC on it you should have 2 level hierarchie, Your RecordSet  will be your root node.
    and the RecordSetStructure is : header,1,data,1,
    Since the node infotext is in the data, you can not refer this node and is not appropriate.
    <header>
        <h_field1/>
    <h_field2/>
                 </header>
                <data>
      <d_field1/>
       <d_field2/>
        <infotext>
    <i_field1/>
    <i_field2/>
    <  /infotext>
              </data>
    Change The above strucutre to like this
    <header>
        <h_field1/>
         <h_field2/>
      </header>
       <data>
          <d_field1/>
          <d_field2/>
        </data>
          <infotext>
                <i_field1/>
               <i_field2/>
           < /infotext>
    Your RecordSet  will be your root node.
    and the RecordSetStructure is : header,1,data,1,infotext,1
    This resolves your issue..
    Regards,
    Prasanna

  • XML to flat File:ABAP Class

    Hi ,
    Please can someone answer my question asap.
    I have to write an ABAP Class which receives an XML Document which has multiple IDOCS.The IDOC has several Segments which can repeat and also the segments have Sub-Segments.
    Please can someone provide me a sample code which can convert the fields of each segment to a line of the output flat file.
    I need a flat file where fields under each segment should come in one line.I mean the control record should come in one line.The fields of 1st segment ex:E1EDK01 should come in 2nd line.....
    Please revert.
    Thanks in Advance,
    Harish.

    You will need to use ABAP ixml technology to parse the content from the xml file:
    You can reference following ABAP mapping.
    https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/15ecdf90-0201-0010-d792-941a3c3c30a4
    or SAP help doc:
    http://help.sap.com/erp2005_ehp_04/helpdata/DE/86/8280d212d511d5991b00508b6b8b11/frameset.htm
    Regards.
    Liang

  • XML to flat file

    Hi all,
    Scenario:
    R/3 -> Proxy -> XI -> JMS -> MQ
    Is it possible to deliver a FLAT FILE to MQ via the JMS adaptor?
    I need to get reed of all the XML tags, since the receiving system is expecting a flat structure.
    Thanks - /Thomas

    hi,
    you can use the following module to remove xml tags and namespaces
    the module is XMLAnonymizerBean
    the url:
    http://help.sap.com/saphelp_nw04/helpdata/en/45/d169186a29570ae10000000a114a6b/frameset.htm
    regards,
    ujjwal

  • SAP IDOC XML to Flat File Conversion

    Hello,
    I have downloaded SAP schema through the "Consume Adapter Service" with properties GenerateFlatFileCompatibleIdocSchema = "true" and FlatFileSegmentIndicator = "SegmentType". I am trying to convert IDOC XML into a IDOC Flat File.
    Though the flat file gets created it has a "/" at the end of each blank field which I do not want in the flat file. I see the schema has <recordInfo structure="delimited"...> which I believe should be "positional". I tried
    changing preserve_delimiter_for_empty_data="True" and suppress_trailing_delimiters="False" properties also but did not help.
    I would appreciate any help on this.
    Thanks,
    Tarun

    Hi Tarun....
    please check this property set this property GenerateFlatFileCompatibleIDoc is set to false

Maybe you are looking for