Blank spaces(hex 00) inserted when extracting data

I'm trying to extract data from a file with select-string and a regular expression.
The data are SWIFT messages and I'm reading all files in a directory and then extracting the part I want with a regular expression which works fine, with the exception that a blank(hex 00) is inserted between all characters in the resulting file,
which causes problems later.
As far as I can see does this happen in the statement
$tempvar = (Get-Content $basePath$inputPathElement | Out-String)
but I haven't so far been able to figure out how to correct the script.
One of the rows do e.g. look like this
:28C:0/1
which in hex will be:3a 32 38 43 3a 30 2f 31 0d 0a
but in the resulting file will look
3a 00 32
00 38 00 43
00 3a 00 30
00 2f 00 31
00 0d 00 0a
00
$date = Get-Date -Format "yyyyMMdd"
$accountList =@(
                    ("123","File01.$date.DAT"),  
                    ("456","File02.$date.DAT"),
$basePath = '\\Server\Folder01\'
$archive = 'archive\'
$extracted = 'extracted\'
$inputPath = get-childitem *.DAT -path $basePath
foreach($inputPathElement in $inputPath)
    $tempvar = (Get-Content $basePath$inputPathElement | Out-String)
    foreach($element in $accountList)
        $account = $element[0]
        $output_file = $element[1]
        $regex ="(?<header>{1:.*\s)(?<Tag2>:2[0-4]:.*\s){1,}(?<Tag25>:25:$account\s\s)((?<Tag28>:28.:.*\s)|(?<Tag6x>:6\d\w*:[\d\w,]*\s\s)|(?<Tag61>:61\w*:[\d\w,\\/\s]*){1,}|(?<Tag86>:86:[\w/\s]*))*(?<Trailer>.*(?<=adm}}))"
        if($tempvar -match  $regex)
            select-string -InputObject $tempvar -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } > $basePath$extracted$output_file
    Move-Item  $basePath$inputPathElement -Destination $basePath$archive -Force
Any help would be appreciated.
/Frank

Hi,
PS defaults to using UTF-16, so that's why you get the extra hex 00.
Try changing the line
 select-string -InputObject $tempvar -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } > $basePath$extracted$output_file
to
 select-string -InputObject $tempvar -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } | Out-File $basePath$extracted$output_file -Encoding UTF8 -append
Regards,
René

Similar Messages

  • Error when extracting data with extractor 2lis_04_matnr - NEED HELP ASAP !!

    Hi experts!
    Got an error when extracting data with extractor 2lis_04_matnr.
    System says (short dump):
    DUMP TEXT START----
    Runtime error:    CONNE_IMPORT_WRONG_COMP_TYPE
    Exception:   CX_SY_IMPORT_MISMATCH_ERROR
    Error when attempting to import object "MC04P_0MAT_TAB".
    The current ABAP program "SAPLMCEX" had to be terminated because one of the statements could not be executed. This is probably due to an error in the ABAP program. When attempting to import data, it was discovered that the data type of the stored data was not the same as that specified in the program.
    An exception occurred. This exception is dealt with in more detail below. The exception, which is assigned to the class 'CX_SY_IMPORT_MISMATCH_ERROR', was neither caught nor passed along using a RAISING clause, in the procedure  "MCEX_BW_LO_API" "(FUNCTION)".                                                                             
    Since the caller of the procedure could not have expected this exception      
    to occur, the running program was terminated.                                
    The reason for the exception is:  When importing the object "MC04P_0MAT_TAB", the component no. 5 in the dataset has a different type from the corresponding component of the target object in the program "SAPLMCEX". <b>The data type is "D" in the dataset, but "C" in the program.</b>
    DUMP TEXT END----
    Please, can someone explain me how to solve it? 
    Really need help ASAP!
    Thanks in advance,
    Jaume
    Message was edited by:
            Jaume Saumell
    Message was edited by:
            Jaume Saumell

    Hi,
    Check this note: 328181
    So you need to delete entries in SM13/LBWQ for application and also detup table content.
    And then refill teh set up table.
    If you are in production clear the entries by running collective run no of times for this application 04.
    With rgds,
    Anil Kumar Sharma .P

  • Doing setBytes() but when extracting data get BLOB error?

    I am inserting a video into an oracle 10g db, and I insert it using the setBytes method but when trying to pull that video from the db, it gives me this error
    SQL Exception in getting video ORA-00932: inconsistent datatypes: expected - got BLOB
    The field value is a blob type, but don't have any issues when doing this with a jpg vs a wmv file. here is my code for insertion and extraction of the data.
                            String query=("INSERT INTO VIDEO (P_ID, P_VIDEONAME, P_VIDEO) Values (?,?,?)");
                            prepStmt = connection.prepareStatement(query);
                            prepStmt.setInt(1,patient.getP_ID());
                            prepStmt.setString(2,patient.getVideoName());
                            prepStmt.setBytes(3,patient.getVideo());
                            prepStmt.executeUpdate();
    rs = stmnt.executeQuery("SELECT P_VIDEONAME, P_VIDEO FROM VIDEO WHERE P_VIDEO = " + patient.getVideoName() + " and P_ID = " + patient.getP_ID());
                            ResultSetMetaData metaData = rs.getMetaData();
                            int columns=metaData.getColumnCount();
                            while(rs.next()){
                                for(int i=1;i<=columns;++i) {
                                    patient.setAction("getVideo");
                                    patient.setVideoName(rs.getString(i));
                                    patient.setVideo(rs.getBytes(++i));
                                    sendPatient();
                        }catch(SQLException e) {
                            //this is similar to the definition i gave you above.
                            System.out.println("SQL Exception in getting video " + e.getMessage());

    That inner for loop makes no sense. You don't need to iterate through the columns; you already know what's supposed to be in each one. Get rid of the loop and just refer to them by their absolute indices, like you did when you inserted the record.
    Also, I would advise always using the postfix operators i++ and i-- instead of the prefix forms, ++1 and --i.  It's just too easy to lose track of what's going on when you mix the the two styles, and postfix operators are much more commonly used.
    I don't know if any of this will solve your problem, but it should at least make the probelm easier to see.

  • Is it possible to change the dimension order when extracting data from HFM?

    Hello,
    When accessing Consolidation/Extract/Data is it possible to change the dimension order of the extract rather than extracting the standard order?
    I can see you can change the dimension order visually by going to "Reorder dimensions" but this does not carry through to the actual extract.
    If its not possible using this method is there another method? Trying to avoid using smartview for this purpose.
    Many thanks and kind regards,
    Rich

    Hi,
    No, it is not possible to change the order type.
    -Paul

  • Problem when extracting data from R/3 to BI

    Hi Experts,
    I am facing a strange problem. While i am extracting data from R/3 yo BI. One Infopackage is running successfully and i can see the data at Bi side. But while I am running another infopackage for another data source it is taking long time and at last loading results into failure.
    I have gone through SDN also but not able to find any inputs.
    I think it might be the reason with the Idocs.  The error it is giving is
    Request still running
    Diagnosis
    No errors found. The current process has probably not finished yet.
    System Response
    The ALE inbox of BI is identical to the ALE outbox of the source system
    or
    the maximum wait time for this request has not yet been exceeded
    or
    the background job has not yet finished in the source system.
    Current status
    All of the packets from the source system arrived
    I am not able to see list of Idocs in sm58.
    What might be the reoson?
    Regards,
    sridhar

    Hi,
    See if its a full schedule it again by deleting any old requests....
    And then when going thru job overview>in the sourcesystem>u can execute there itself and chk the status of the job...
    If its no there then the job might have missed up due to some unfair reason....it happens sometimes also..some times it will be in scheduled only....once select all active/cancled/finished and so on...
    And also chk under details tab...under extraction....whether data selection scheduled and requested and finished....????
    If not there try to run it again and at the same moment monitor in source with jobname as BIREQU_* and select all input entries as canceled/active/finished......
    rgds,

  • Error when extracting data from ETL - Missing messages and missing InfoIdoc

    Hi All,
    We are using BW 3.0 and extracting data from other source systems through Informatica Powercenter (ETL). The system is working fine but when we try to extract data on 31st Dec , we get the following error. Only this load gives the error and all the other load to other data targets are going fine. All the data are one-to-one mapped from ETL to BW.
    Error messages from Monitor -Status tab:-
       "InfoIdocs missing; external system
       Diagnosis :- InfoIDocs are missing. The IDocs are created in BW with non-SAP systems as source    
       systems that transfer the data and metadata to BW using BAPIs. Since the IDocs are missing, a   
       short dump in BW probably occurred.
       System response:  No selection information arrived from the source system"
    Error messages from Monitor -Details tab:-
        Missing message: Number of sent records,   Missing message: Selection completed
    Highly appretiate your suggestions.
    Vinod.CH

    Hi Rathy Moorthy,
    Thank you very much for your reply. The source system connections are OK and we are able to load data to other Data targets from ETL, we have issue only with this this particular load. The load extracts data and I have loaded the data from ETL to PSA and have checked the data content and it is correct. But when I update the same data to the target I get this error. I have also tried to update from PSA to target and also directly from ETL to target.
    Appretiate your suggestions.

  • Error when extracting data in rsa3

    Hi,
    we are using BI7.0. While when i am extracting data to PSA  i am getting an error that error in source system. when i am extracting data in RSA3 source system side there also it was showing the same error. Data souce is active.
    So please let me know if it as anything to do.
    regards,
    mahesh.

    Hi,
    Which datasource are you using to extract data ? Go rsa5 and try to activate it then retry.
    Regards,
    ®

  • When extracting data some recors are missing

    I am extracting data for inventory.
    it is delta .When data is extracting from r/3 no process chain fails.
    But some of the records are missing. it is only happening for inventory datasource
    Please
    guide me on this

    Hi,
    when we say records missing, where is this happening?
    Is this at the Ds level or while update into the target.
    Is the initialization done with any selection conditions, if yes then delta also will be fetched only for those selections.
    Naveen.A

  • Error when extracting data from R/3 into BW

    Hi,
      There is an infoobject Payscale Level(Datsource 0paysclaelv_attr) in BW module HR that is extracting data from R/3. While extracting data, the following error occurs "Assignment to feature PFREQ did not take place". I debugged the standard function module HR_BW_EXTRACT_IO_PAYSCALELV and I believe there is definitely an error in the data. But Im not able to find out that record. Has anyone come across this error before? Please let me know if theres any solution to this error. The help for this error provides the following details.  But I havent understood it. Any help would be greatly appreciated.
    You can assign a specific operation (assignment) to each decision option
    within a feature. You can also define decision option ** whose
    assignment is effected for all unlisted decision options (this is known
    as a default value).
    Example:
    o                D PERSK
    o     K1           &ABRKS=D1,    <- D1 for employee subgroup K1
    o     K2           &ABRKS=D1,    <- D1 for employee subgroup K2
    o     K3           &ABRKS=D2,    <- D2 for employee subgroup K3
    o     **           &ABRKS=D0,    <- D0 for all other employee subgroups
    If you now create employee subgroup K4 and the default entry ** is
    missing, the system is unable to find a decision option for payroll area
    ABKRS.
    Procedure
        Define a decision option for K4 or for default value **.

    Hi ,
    The error is  due to the existing customizing of feature PFREQ for MOLGA = 05, there the value to be returned depends on the WERKS, but apparently it was forgotten to add a new line for the "Otherwise" case.
    Like it was already done for the case of MOLGA = 03, for example. Please add then a line for MOLGA = 05 and "Otherwise" and the issue should be solved.
    If this doesnot solves your issue , then kindly check these OSS notes :
    1033423 -> 0PAYSCALELV_ATTR: Short dump feature_error
    842212   -> 0PAYSCALELV_ATTR: DataSources deliver wrong results
    92055     ->  RP_FROM_PERIOD_TO_PERIOD: FEATURE_ERROR dump
    Regards,
    Lokesh

  • Error when extracting data from Hyperion enterprise 6.3.1

    I extracted data and at the end it's returning me an error for 2 entities. The problem it's extarcting data from an entity that doesn't exist. Any idea why?
    There is no journal posted to it.
    This is from the error log.
    !\\tdwm-aawhy-cp01\entdata\FSS\IFRS_YE_Rollover_R1\Extract Data AL_IFRS ACTMAVG.txt
    ACTMAVG
    1
    12
    D0786.ADJML,N250099.CDN.RCDN,1806525.000000,1755614.000000,1768445.000000,1780035.000000,,,,,,,,
    D0786.ADJML,N257099.CDN.RCDN,-16904.000000,-15093.000000,-15093.000000,-15093.000000,,,,,,,,
    D0786.ADJML,N607599.CDN.RCDN,17694.000000,17191.000000,17335.000000,17392.000000,,,,,,,,
    D0786.ADJML,P200099.CDN.RCDN,2644.000000,2500.000000,2514.000000,2529.000000,,,,,,,,
    D0786.ADJML,P200099.CDN.ROTH,4694.000000,4694.000000,4694.000000,4694.000000,,,,,,,,
    D0786.ADJML,P300599.CDN.RCDN,49977.000000,403.000000,12897.000000,24182.000000,,,,,,,,
    D0786.ADJML,P301199.CDN.RCDN,1750000.000000,1750000.000000,1750000.000000,,,,,,,,,
    D0786.ADJML,P304099.CDN.RCDN,,115.000000,582.000000,929.000000,,,,,,,,
    D0786.ADJML,P301189.CDN.RCDN,,,,1750000.000000,,,,,,,,
    D1124.ADJML,N250099.CDN.RCDN,360104.000000,352066.000000,354059.000000,356051.000000,,,,,,,,
    D1124.ADJML,P200099.CDN.RCDN,2075.000000,2000.000000,2008.000000,2015.000000,,,,,,,,
    D1124.ADJML,P200099.CDN.ROTH,39.000000,-7.000000,-7.000000,-7.000000,,,,,,,,
    D1124.ADJML,P203099.CDN.RCDN,0.000000,-1.000000,0.000000,0.000000,,,,,,,,
    D1124.ADJML,P300599.CDN.RCDN,7990.000000,66.000000,2047.000000,4028.000000,,,,,,,,
    D1124.ADJML,P301199.CDN.RCDN,350000.000000,350000.000000,350000.000000,,,,,,,,,
    D1124.ADJML,P304099.CDN.RCDN,,8.000000,11.000000,15.000000,,,,,,,,
    D1124.ADJML,P301189.CDN.RCDN,,,,350000.000000,,,,,,,,
    !\\tdwm-aawhy-cp01\entdata\FSS\IFRS_YE_Rollover_R1\Extract Data AL_IFRS ACTMENT.txt
    ACTMEND
    1
    12
    D0786.ADJML,N250099.CDN.RCDN,1806525.000000,1755614.000000,1768445.000000,1780035.000000,,,,,,,,
    D0786.ADJML,N257099.CDN.RCDN,-16904.000000,-15093.000000,-15093.000000,-15093.000000,,,,,,,,
    D0786.ADJML,N607599.CDN.RCDN,17694.000000,17191.000000,17335.000000,17392.000000,,,,,,,,
    D0786.ADJML,P200099.CDN.RCDN,2644.000000,2500.000000,2514.000000,2529.000000,,,,,,,,
    D0786.ADJML,P200099.CDN.ROTH,4694.000000,4694.000000,4694.000000,4694.000000,,,,,,,,
    D0786.ADJML,P300599.CDN.RCDN,49977.000000,403.000000,12897.000000,24182.000000,,,,,,,,
    D0786.ADJML,P301199.CDN.RCDN,1750000.000000,1750000.000000,1750000.000000,,,,,,,,,
    D0786.ADJML,P304099.CDN.RCDN,,115.000000,582.000000,929.000000,,,,,,,,
    D0786.ADJML,P301189.CDN.RCDN,,,,1750000.000000,,,,,,,,
    D1124.ADJML,N250099.CDN.RCDN,360104.000000,352066.000000,354059.000000,356051.000000,,,,,,,,
    D1124.ADJML,P200099.CDN.RCDN,2075.000000,2000.000000,2008.000000,2015.000000,,,,,,,,
    D1124.ADJML,P200099.CDN.ROTH,39.000000,-7.000000,-7.000000,-7.000000,,,,,,,,
    D1124.ADJML,P203099.CDN.RCDN,0.000000,-1.000000,0.000000,0.000000,,,,,,,,
    D1124.ADJML,P300599.CDN.RCDN,7990.000000,66.000000,2047.000000,4028.000000,,,,,,,,
    D1124.ADJML,P301199.CDN.RCDN,350000.000000,350000.000000,350000.000000,,,,,,,,,
    D1124.ADJML,P304099.CDN.RCDN,,8.000000,11.000000,15.000000,,,,,,,,
    D1124.ADJML,P301189.CDN.RCDN,,,,350000.000000,,,,,,,,

    If you are sure the entity doesn't exist, trying using the purge entities feature. Go to the Entities module, from its main menu select Task->Unowned Entities and run the Unknown entities report, if the entity that is causing the problem is there, run the purge unowned entities task. Make sure you dont purge other entities that you may need.

  • No data comes when extracting data for 2LIS_06_INV

    Hi,
    I am trying to extract data from 2LIS_06_INV into my BI system.I see the invoice numbers in RSEG table as created from SRM 5.0 into my R/3 system but unfortunately no data gets extracted.Please provide some pointers if some setting needs to be done for the same.
    Joy

    Hi,
    It is working fine I tried with out activating in LBWE.
    Thanks!
    Surendra.

  • Error when extracting data from Data Source

    Hi All,
    I have a Generic datasource in BW which is based on a Function Module.When I am extracting the data from this source in RSA3 it is ending up in Run time error.
    the error message is
    The current ABAP program "SAPLYBWPU" had to be terminated because one of the
    statements could not be executed.
    and the error analysis is
    You wanted to add an entry to table "\FUNCTION-POOL=YBWPU\DATA=GT_HIERTAB2", which you declared with a UNIQUE KEY. However, there was already an entry with the same key.
    This may have been in an INSERT or MOVE statement, or within a
    SELECT ... INTO statement.
    In particular, you cannot insert more than one initial line into a
    table with a unique key using the INSERT INITIAL LINE... statement.
    Can anybody explain me how can I resolve this
    Thanks
    Sreeja

    Hello
    You can refer following link.
    http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/a0f46157-e1c4-2910-27aa-e3f4a9c8df33&overridelayout=true

  • Run time error COMPUTE_BCD_OVERFLOW when extracting data

    Dear all,
    We are facing an issue when trying to extract the data for CRM Service contract item Extractor. It is giving runtime error when trying to extract particular range of transaction number say 1 to 50. but if i split these into 2 packages i.e 1 to 25 & 26 to 50, the data is getting extracted without any problems. If anyone has come across the similar issue, please share your views.
    details of the errro:
    Runtime Errors         COMPUTE_BCD_OVERFLOW
    Except.                CX_SY_ARITHMETIC_OVERFLOW
    Short text
        Overflow during the arithmetical operation (type P) in program
    Thanks in Advance!
    Regards,
    Pavan

    Hi,
    I can see that there are several SAP notes regarding this issue, you just need to check their symptons and if they are valid for your SAP_APPL release and support package.
    Here are the SAP notes:
    - 1295055: Product cost collector: Field AFPO-WEMNG set incorrectly
    - 964308: COMPUTE_BCD_OVERFLOW in CKMO_GET_ORD_HIST_NETWR_MMIM
    I hope they help you.
    Regards,
    Marcelo

  • Error when extracting data using DB Connect from an Oracle database

    Hi All,
    We are on BI 3.5 and loading data from a non-SAP (Oracle 10g) database using DB Connect.
    I received the following error messages when trying to perform "Check DataSource" using the DB Connect setup in RSA1.
    Event with errors RSDL_META_UPLOAD in DataSource
    Message no. R8287
    Warning Messages;
    Message no. RSDL053
    Field name ReceiptID contains non-alpha-numeric characters
    Field name ActivityDate contains non-alpha-numeric characters
    Field name ReceiptDate contains non-alpha-numeric characters
    Field name ReceiptLineNumber must have between 000001 and 000016 digits
    etc...
    Views are created under the owner's user name. I have checked with DB personnel and they confirm that all the authorization and security access privileges have been correctly granted to the views.
    We are also able to extract the data by running a sql query statement from the views using SQL Plus
    Currently we also have a similar extraction process set up with an Oracle 9 database, which extracts the information without any issues.
    Thanks,
    Anthony

    Hi Mr Loh.
    I know that this thread has over a year ago but i was looking for something similar for over 2 weeks ago and i ialways found this thread...I want to share my experience even that maybe you already found the same solution or something similar (i saw that you check your own post as answered):
    I made a similar connection from BI 7 SP 18, but in my case was to a Microsoft SQL database 2005. When i "check the datasource" i receive like almost 150 warnings and 1 error and all the warnings and errors are the same ones as you describe above. What i did is create a custom view in the MSSQL database and convert all the column names to uppercase and the column names i limited to 16 characters at most. after doing that all the warnings and the error described disappear and i can view the database content when you made click in the "view content table" button.
    I hope that this will help somebody and best regards
    Martin Olmos

  • Failure when extracting data from R/3

    We have many InfoPacjkages running business content extractors from R/3 into PSA  Occaisonally these extracts fail with the following message:
    Non-updated Idocs found in Source System
    Diagnosis
    IDocs were found in the ALE inbox for Source System that are not updated.
    Processing is overdue.
    Error correction:
    Attempt to process the IDocs manually. You can process the IDocs manually using the Wizard or by selecting the IDocs with incorrect status and processing them manually.
    The failure appears to be caused by the extract job in R/3 not starting, no job BIREQU* in SM37.
    When this situation occurs the BW job waits a lenth of time, in our case approximately 8 hours, before going red.
    Does anyone else suffer this kind of issue and if so is there any way in reducing the time elapsed before BW errors?

    Hi Terry,
    you can reduce BW job waits a lenth of time at Infopackage level.
    RSA1-> select you infopackage -> Schedule (menu) -> Timeout time -> Maximum wait time for this infopackage (here you can se 1 or 2 hrs).
    Best Regards.

Maybe you are looking for