Blank spaces(hex 00) inserted when extracting data
I'm trying to extract data from a file with select-string and a regular expression.
The data are SWIFT messages and I'm reading all files in a directory and then extracting the part I want with a regular expression which works fine, with the exception that a blank(hex 00) is inserted between all characters in the resulting file,
which causes problems later.
As far as I can see does this happen in the statement
$tempvar = (Get-Content $basePath$inputPathElement | Out-String)
but I haven't so far been able to figure out how to correct the script.
One of the rows do e.g. look like this
:28C:0/1
which in hex will be:3a 32 38 43 3a 30 2f 31 0d 0a
but in the resulting file will look
3a 00 32
00 38 00 43
00 3a 00 30
00 2f 00 31
00 0d 00 0a
00
$date = Get-Date -Format "yyyyMMdd"
$accountList =@(
("123","File01.$date.DAT"),
("456","File02.$date.DAT"),
$basePath = '\\Server\Folder01\'
$archive = 'archive\'
$extracted = 'extracted\'
$inputPath = get-childitem *.DAT -path $basePath
foreach($inputPathElement in $inputPath)
$tempvar = (Get-Content $basePath$inputPathElement | Out-String)
foreach($element in $accountList)
$account = $element[0]
$output_file = $element[1]
$regex ="(?<header>{1:.*\s)(?<Tag2>:2[0-4]:.*\s){1,}(?<Tag25>:25:$account\s\s)((?<Tag28>:28.:.*\s)|(?<Tag6x>:6\d\w*:[\d\w,]*\s\s)|(?<Tag61>:61\w*:[\d\w,\\/\s]*){1,}|(?<Tag86>:86:[\w/\s]*))*(?<Trailer>.*(?<=adm}}))"
if($tempvar -match $regex)
select-string -InputObject $tempvar -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } > $basePath$extracted$output_file
Move-Item $basePath$inputPathElement -Destination $basePath$archive -Force
Any help would be appreciated.
/Frank
Hi,
PS defaults to using UTF-16, so that's why you get the extra hex 00.
Try changing the line
select-string -InputObject $tempvar -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } > $basePath$extracted$output_file
to
select-string -InputObject $tempvar -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } | Out-File $basePath$extracted$output_file -Encoding UTF8 -append
Regards,
René
Similar Messages
-
Error when extracting data with extractor 2lis_04_matnr - NEED HELP ASAP !!
Hi experts!
Got an error when extracting data with extractor 2lis_04_matnr.
System says (short dump):
DUMP TEXT START----
Runtime error: CONNE_IMPORT_WRONG_COMP_TYPE
Exception: CX_SY_IMPORT_MISMATCH_ERROR
Error when attempting to import object "MC04P_0MAT_TAB".
The current ABAP program "SAPLMCEX" had to be terminated because one of the statements could not be executed. This is probably due to an error in the ABAP program. When attempting to import data, it was discovered that the data type of the stored data was not the same as that specified in the program.
An exception occurred. This exception is dealt with in more detail below. The exception, which is assigned to the class 'CX_SY_IMPORT_MISMATCH_ERROR', was neither caught nor passed along using a RAISING clause, in the procedure "MCEX_BW_LO_API" "(FUNCTION)".
Since the caller of the procedure could not have expected this exception
to occur, the running program was terminated.
The reason for the exception is: When importing the object "MC04P_0MAT_TAB", the component no. 5 in the dataset has a different type from the corresponding component of the target object in the program "SAPLMCEX". <b>The data type is "D" in the dataset, but "C" in the program.</b>
DUMP TEXT END----
Please, can someone explain me how to solve it?
Really need help ASAP!
Thanks in advance,
Jaume
Message was edited by:
Jaume Saumell
Message was edited by:
Jaume SaumellHi,
Check this note: 328181
So you need to delete entries in SM13/LBWQ for application and also detup table content.
And then refill teh set up table.
If you are in production clear the entries by running collective run no of times for this application 04.
With rgds,
Anil Kumar Sharma .P -
Doing setBytes() but when extracting data get BLOB error?
I am inserting a video into an oracle 10g db, and I insert it using the setBytes method but when trying to pull that video from the db, it gives me this error
SQL Exception in getting video ORA-00932: inconsistent datatypes: expected - got BLOB
The field value is a blob type, but don't have any issues when doing this with a jpg vs a wmv file. here is my code for insertion and extraction of the data.
String query=("INSERT INTO VIDEO (P_ID, P_VIDEONAME, P_VIDEO) Values (?,?,?)");
prepStmt = connection.prepareStatement(query);
prepStmt.setInt(1,patient.getP_ID());
prepStmt.setString(2,patient.getVideoName());
prepStmt.setBytes(3,patient.getVideo());
prepStmt.executeUpdate();
rs = stmnt.executeQuery("SELECT P_VIDEONAME, P_VIDEO FROM VIDEO WHERE P_VIDEO = " + patient.getVideoName() + " and P_ID = " + patient.getP_ID());
ResultSetMetaData metaData = rs.getMetaData();
int columns=metaData.getColumnCount();
while(rs.next()){
for(int i=1;i<=columns;++i) {
patient.setAction("getVideo");
patient.setVideoName(rs.getString(i));
patient.setVideo(rs.getBytes(++i));
sendPatient();
}catch(SQLException e) {
//this is similar to the definition i gave you above.
System.out.println("SQL Exception in getting video " + e.getMessage());That inner for loop makes no sense. You don't need to iterate through the columns; you already know what's supposed to be in each one. Get rid of the loop and just refer to them by their absolute indices, like you did when you inserted the record.
Also, I would advise always using the postfix operators i++ and i-- instead of the prefix forms, ++1 and --i. It's just too easy to lose track of what's going on when you mix the the two styles, and postfix operators are much more commonly used.
I don't know if any of this will solve your problem, but it should at least make the probelm easier to see. -
Is it possible to change the dimension order when extracting data from HFM?
Hello,
When accessing Consolidation/Extract/Data is it possible to change the dimension order of the extract rather than extracting the standard order?
I can see you can change the dimension order visually by going to "Reorder dimensions" but this does not carry through to the actual extract.
If its not possible using this method is there another method? Trying to avoid using smartview for this purpose.
Many thanks and kind regards,
RichHi,
No, it is not possible to change the order type.
-Paul -
Problem when extracting data from R/3 to BI
Hi Experts,
I am facing a strange problem. While i am extracting data from R/3 yo BI. One Infopackage is running successfully and i can see the data at Bi side. But while I am running another infopackage for another data source it is taking long time and at last loading results into failure.
I have gone through SDN also but not able to find any inputs.
I think it might be the reason with the Idocs. The error it is giving is
Request still running
Diagnosis
No errors found. The current process has probably not finished yet.
System Response
The ALE inbox of BI is identical to the ALE outbox of the source system
or
the maximum wait time for this request has not yet been exceeded
or
the background job has not yet finished in the source system.
Current status
All of the packets from the source system arrived
I am not able to see list of Idocs in sm58.
What might be the reoson?
Regards,
sridharHi,
See if its a full schedule it again by deleting any old requests....
And then when going thru job overview>in the sourcesystem>u can execute there itself and chk the status of the job...
If its no there then the job might have missed up due to some unfair reason....it happens sometimes also..some times it will be in scheduled only....once select all active/cancled/finished and so on...
And also chk under details tab...under extraction....whether data selection scheduled and requested and finished....????
If not there try to run it again and at the same moment monitor in source with jobname as BIREQU_* and select all input entries as canceled/active/finished......
rgds, -
Error when extracting data from ETL - Missing messages and missing InfoIdoc
Hi All,
We are using BW 3.0 and extracting data from other source systems through Informatica Powercenter (ETL). The system is working fine but when we try to extract data on 31st Dec , we get the following error. Only this load gives the error and all the other load to other data targets are going fine. All the data are one-to-one mapped from ETL to BW.
Error messages from Monitor -Status tab:-
"InfoIdocs missing; external system
Diagnosis :- InfoIDocs are missing. The IDocs are created in BW with non-SAP systems as source
systems that transfer the data and metadata to BW using BAPIs. Since the IDocs are missing, a
short dump in BW probably occurred.
System response: No selection information arrived from the source system"
Error messages from Monitor -Details tab:-
Missing message: Number of sent records, Missing message: Selection completed
Highly appretiate your suggestions.
Vinod.CHHi Rathy Moorthy,
Thank you very much for your reply. The source system connections are OK and we are able to load data to other Data targets from ETL, we have issue only with this this particular load. The load extracts data and I have loaded the data from ETL to PSA and have checked the data content and it is correct. But when I update the same data to the target I get this error. I have also tried to update from PSA to target and also directly from ETL to target.
Appretiate your suggestions. -
Error when extracting data in rsa3
Hi,
we are using BI7.0. While when i am extracting data to PSA i am getting an error that error in source system. when i am extracting data in RSA3 source system side there also it was showing the same error. Data souce is active.
So please let me know if it as anything to do.
regards,
mahesh.Hi,
Which datasource are you using to extract data ? Go rsa5 and try to activate it then retry.
Regards,
® -
When extracting data some recors are missing
I am extracting data for inventory.
it is delta .When data is extracting from r/3 no process chain fails.
But some of the records are missing. it is only happening for inventory datasource
Please
guide me on thisHi,
when we say records missing, where is this happening?
Is this at the Ds level or while update into the target.
Is the initialization done with any selection conditions, if yes then delta also will be fetched only for those selections.
Naveen.A -
Error when extracting data from R/3 into BW
Hi,
There is an infoobject Payscale Level(Datsource 0paysclaelv_attr) in BW module HR that is extracting data from R/3. While extracting data, the following error occurs "Assignment to feature PFREQ did not take place". I debugged the standard function module HR_BW_EXTRACT_IO_PAYSCALELV and I believe there is definitely an error in the data. But Im not able to find out that record. Has anyone come across this error before? Please let me know if theres any solution to this error. The help for this error provides the following details. But I havent understood it. Any help would be greatly appreciated.
You can assign a specific operation (assignment) to each decision option
within a feature. You can also define decision option ** whose
assignment is effected for all unlisted decision options (this is known
as a default value).
Example:
o D PERSK
o K1 &ABRKS=D1, <- D1 for employee subgroup K1
o K2 &ABRKS=D1, <- D1 for employee subgroup K2
o K3 &ABRKS=D2, <- D2 for employee subgroup K3
o ** &ABRKS=D0, <- D0 for all other employee subgroups
If you now create employee subgroup K4 and the default entry ** is
missing, the system is unable to find a decision option for payroll area
ABKRS.
Procedure
Define a decision option for K4 or for default value **.Hi ,
The error is due to the existing customizing of feature PFREQ for MOLGA = 05, there the value to be returned depends on the WERKS, but apparently it was forgotten to add a new line for the "Otherwise" case.
Like it was already done for the case of MOLGA = 03, for example. Please add then a line for MOLGA = 05 and "Otherwise" and the issue should be solved.
If this doesnot solves your issue , then kindly check these OSS notes :
1033423 -> 0PAYSCALELV_ATTR: Short dump feature_error
842212 -> 0PAYSCALELV_ATTR: DataSources deliver wrong results
92055 -> RP_FROM_PERIOD_TO_PERIOD: FEATURE_ERROR dump
Regards,
Lokesh -
Error when extracting data from Hyperion enterprise 6.3.1
I extracted data and at the end it's returning me an error for 2 entities. The problem it's extarcting data from an entity that doesn't exist. Any idea why?
There is no journal posted to it.
This is from the error log.
!\\tdwm-aawhy-cp01\entdata\FSS\IFRS_YE_Rollover_R1\Extract Data AL_IFRS ACTMAVG.txt
ACTMAVG
1
12
D0786.ADJML,N250099.CDN.RCDN,1806525.000000,1755614.000000,1768445.000000,1780035.000000,,,,,,,,
D0786.ADJML,N257099.CDN.RCDN,-16904.000000,-15093.000000,-15093.000000,-15093.000000,,,,,,,,
D0786.ADJML,N607599.CDN.RCDN,17694.000000,17191.000000,17335.000000,17392.000000,,,,,,,,
D0786.ADJML,P200099.CDN.RCDN,2644.000000,2500.000000,2514.000000,2529.000000,,,,,,,,
D0786.ADJML,P200099.CDN.ROTH,4694.000000,4694.000000,4694.000000,4694.000000,,,,,,,,
D0786.ADJML,P300599.CDN.RCDN,49977.000000,403.000000,12897.000000,24182.000000,,,,,,,,
D0786.ADJML,P301199.CDN.RCDN,1750000.000000,1750000.000000,1750000.000000,,,,,,,,,
D0786.ADJML,P304099.CDN.RCDN,,115.000000,582.000000,929.000000,,,,,,,,
D0786.ADJML,P301189.CDN.RCDN,,,,1750000.000000,,,,,,,,
D1124.ADJML,N250099.CDN.RCDN,360104.000000,352066.000000,354059.000000,356051.000000,,,,,,,,
D1124.ADJML,P200099.CDN.RCDN,2075.000000,2000.000000,2008.000000,2015.000000,,,,,,,,
D1124.ADJML,P200099.CDN.ROTH,39.000000,-7.000000,-7.000000,-7.000000,,,,,,,,
D1124.ADJML,P203099.CDN.RCDN,0.000000,-1.000000,0.000000,0.000000,,,,,,,,
D1124.ADJML,P300599.CDN.RCDN,7990.000000,66.000000,2047.000000,4028.000000,,,,,,,,
D1124.ADJML,P301199.CDN.RCDN,350000.000000,350000.000000,350000.000000,,,,,,,,,
D1124.ADJML,P304099.CDN.RCDN,,8.000000,11.000000,15.000000,,,,,,,,
D1124.ADJML,P301189.CDN.RCDN,,,,350000.000000,,,,,,,,
!\\tdwm-aawhy-cp01\entdata\FSS\IFRS_YE_Rollover_R1\Extract Data AL_IFRS ACTMENT.txt
ACTMEND
1
12
D0786.ADJML,N250099.CDN.RCDN,1806525.000000,1755614.000000,1768445.000000,1780035.000000,,,,,,,,
D0786.ADJML,N257099.CDN.RCDN,-16904.000000,-15093.000000,-15093.000000,-15093.000000,,,,,,,,
D0786.ADJML,N607599.CDN.RCDN,17694.000000,17191.000000,17335.000000,17392.000000,,,,,,,,
D0786.ADJML,P200099.CDN.RCDN,2644.000000,2500.000000,2514.000000,2529.000000,,,,,,,,
D0786.ADJML,P200099.CDN.ROTH,4694.000000,4694.000000,4694.000000,4694.000000,,,,,,,,
D0786.ADJML,P300599.CDN.RCDN,49977.000000,403.000000,12897.000000,24182.000000,,,,,,,,
D0786.ADJML,P301199.CDN.RCDN,1750000.000000,1750000.000000,1750000.000000,,,,,,,,,
D0786.ADJML,P304099.CDN.RCDN,,115.000000,582.000000,929.000000,,,,,,,,
D0786.ADJML,P301189.CDN.RCDN,,,,1750000.000000,,,,,,,,
D1124.ADJML,N250099.CDN.RCDN,360104.000000,352066.000000,354059.000000,356051.000000,,,,,,,,
D1124.ADJML,P200099.CDN.RCDN,2075.000000,2000.000000,2008.000000,2015.000000,,,,,,,,
D1124.ADJML,P200099.CDN.ROTH,39.000000,-7.000000,-7.000000,-7.000000,,,,,,,,
D1124.ADJML,P203099.CDN.RCDN,0.000000,-1.000000,0.000000,0.000000,,,,,,,,
D1124.ADJML,P300599.CDN.RCDN,7990.000000,66.000000,2047.000000,4028.000000,,,,,,,,
D1124.ADJML,P301199.CDN.RCDN,350000.000000,350000.000000,350000.000000,,,,,,,,,
D1124.ADJML,P304099.CDN.RCDN,,8.000000,11.000000,15.000000,,,,,,,,
D1124.ADJML,P301189.CDN.RCDN,,,,350000.000000,,,,,,,,If you are sure the entity doesn't exist, trying using the purge entities feature. Go to the Entities module, from its main menu select Task->Unowned Entities and run the Unknown entities report, if the entity that is causing the problem is there, run the purge unowned entities task. Make sure you dont purge other entities that you may need.
-
No data comes when extracting data for 2LIS_06_INV
Hi,
I am trying to extract data from 2LIS_06_INV into my BI system.I see the invoice numbers in RSEG table as created from SRM 5.0 into my R/3 system but unfortunately no data gets extracted.Please provide some pointers if some setting needs to be done for the same.
JoyHi,
It is working fine I tried with out activating in LBWE.
Thanks!
Surendra. -
Error when extracting data from Data Source
Hi All,
I have a Generic datasource in BW which is based on a Function Module.When I am extracting the data from this source in RSA3 it is ending up in Run time error.
the error message is
The current ABAP program "SAPLYBWPU" had to be terminated because one of the
statements could not be executed.
and the error analysis is
You wanted to add an entry to table "\FUNCTION-POOL=YBWPU\DATA=GT_HIERTAB2", which you declared with a UNIQUE KEY. However, there was already an entry with the same key.
This may have been in an INSERT or MOVE statement, or within a
SELECT ... INTO statement.
In particular, you cannot insert more than one initial line into a
table with a unique key using the INSERT INITIAL LINE... statement.
Can anybody explain me how can I resolve this
Thanks
SreejaHello
You can refer following link.
http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/a0f46157-e1c4-2910-27aa-e3f4a9c8df33&overridelayout=true -
Run time error COMPUTE_BCD_OVERFLOW when extracting data
Dear all,
We are facing an issue when trying to extract the data for CRM Service contract item Extractor. It is giving runtime error when trying to extract particular range of transaction number say 1 to 50. but if i split these into 2 packages i.e 1 to 25 & 26 to 50, the data is getting extracted without any problems. If anyone has come across the similar issue, please share your views.
details of the errro:
Runtime Errors COMPUTE_BCD_OVERFLOW
Except. CX_SY_ARITHMETIC_OVERFLOW
Short text
Overflow during the arithmetical operation (type P) in program
Thanks in Advance!
Regards,
PavanHi,
I can see that there are several SAP notes regarding this issue, you just need to check their symptons and if they are valid for your SAP_APPL release and support package.
Here are the SAP notes:
- 1295055: Product cost collector: Field AFPO-WEMNG set incorrectly
- 964308: COMPUTE_BCD_OVERFLOW in CKMO_GET_ORD_HIST_NETWR_MMIM
I hope they help you.
Regards,
Marcelo -
Error when extracting data using DB Connect from an Oracle database
Hi All,
We are on BI 3.5 and loading data from a non-SAP (Oracle 10g) database using DB Connect.
I received the following error messages when trying to perform "Check DataSource" using the DB Connect setup in RSA1.
Event with errors RSDL_META_UPLOAD in DataSource
Message no. R8287
Warning Messages;
Message no. RSDL053
Field name ReceiptID contains non-alpha-numeric characters
Field name ActivityDate contains non-alpha-numeric characters
Field name ReceiptDate contains non-alpha-numeric characters
Field name ReceiptLineNumber must have between 000001 and 000016 digits
etc...
Views are created under the owner's user name. I have checked with DB personnel and they confirm that all the authorization and security access privileges have been correctly granted to the views.
We are also able to extract the data by running a sql query statement from the views using SQL Plus
Currently we also have a similar extraction process set up with an Oracle 9 database, which extracts the information without any issues.
Thanks,
AnthonyHi Mr Loh.
I know that this thread has over a year ago but i was looking for something similar for over 2 weeks ago and i ialways found this thread...I want to share my experience even that maybe you already found the same solution or something similar (i saw that you check your own post as answered):
I made a similar connection from BI 7 SP 18, but in my case was to a Microsoft SQL database 2005. When i "check the datasource" i receive like almost 150 warnings and 1 error and all the warnings and errors are the same ones as you describe above. What i did is create a custom view in the MSSQL database and convert all the column names to uppercase and the column names i limited to 16 characters at most. after doing that all the warnings and the error described disappear and i can view the database content when you made click in the "view content table" button.
I hope that this will help somebody and best regards
Martin Olmos -
Failure when extracting data from R/3
We have many InfoPacjkages running business content extractors from R/3 into PSA Occaisonally these extracts fail with the following message:
Non-updated Idocs found in Source System
Diagnosis
IDocs were found in the ALE inbox for Source System that are not updated.
Processing is overdue.
Error correction:
Attempt to process the IDocs manually. You can process the IDocs manually using the Wizard or by selecting the IDocs with incorrect status and processing them manually.
The failure appears to be caused by the extract job in R/3 not starting, no job BIREQU* in SM37.
When this situation occurs the BW job waits a lenth of time, in our case approximately 8 hours, before going red.
Does anyone else suffer this kind of issue and if so is there any way in reducing the time elapsed before BW errors?Hi Terry,
you can reduce BW job waits a lenth of time at Infopackage level.
RSA1-> select you infopackage -> Schedule (menu) -> Timeout time -> Maximum wait time for this infopackage (here you can se 1 or 2 hrs).
Best Regards.
Maybe you are looking for
-
HT201079 Am i able to use a different payment method if i am in family sharing?
I am in a family sharing group through iTunes store and app store. I do not want to charge the shared payment method, but instead pay with my own payment method. Am I able to do so? If so, How?
-
Download error on movie rented from Itunes Store
Hello, I have tried to download a movie i just rented from the itunes store and an error message stating that: There was a problem downloading "file name". The disk could not be read from or written to. Please check the connection to the network is a
-
How do I import photos to my canvas in Photoshop cc 2014
I tried going to file > import but it doesnt come up with images or pdf or jpeg so i dunno how to do it
-
Where do i find the server name or URL?
where do i find the server name or URL for my icloud account
-
Handling Error caught by B1 Test Composer Tool
Hi All I got an error message : " Unable to cast COM objects of type System._ComObject' to interface type 'SAPbouiCOM.Matrix'. This opearation failed because the Query Interface call the COM Component for the interface with IID {DD7804DC-811B-4829-8E