Data Services X-File

Dear all,
Let's see if anyone can help me with this, as I am stuck.
We have two Data Services installations: DEV and PROD. Both of them DS 3.2 (no SP although I am currently downloading the last one just in case).
DEV is working perfectly: running the jobs and inserting in the Data Warehouse the desired information.
PROD, on the other hand is not storing the desired information. Jobs are the same 100%. I imported several times the .atl files.
Sources from both environments are also the same, so data transforms should finish with the same output.
The archetectural differences between both servers are the following:
DEV is 2003 server (same machine for BOE, DS and Database). Database is 11g 32-bit.
PROD is 2008 server only with DS and Oracle Client. Client Oracle client is 32-bit. Database is in a separate server with Windows 2008 server 64-bit.
If I test same job in DEV with different targets (DEV Database and PROD Database) results are different using both DS instances.
Has anyone found same problem or can help me solving it???
Many thanks!!!
Bea

The trace log is the normal log you see in the designer when you execute a job. It is also visible via the web admin console. By the way, it might be worth comparing the monitor log too (the one with the row counts in it). If your source data volume is the same, then the monitor logs will show this. If your monitor logs have more or less entries(look very different), it means that the optimiser is working differently between environments. This can happen if the datastore configuration doesn't allow optimised SQL to be generated.
Losing rows sounds very odd. The only way this would normally happen is when using "overflow files" (see target table options), but in that case you'd have job errors.
Transcoders are automatically created to translate data between code pages, and in certain circumstances can cause characters to be displayed not as intended, but only when the codepages have been defined incorrectly or the default settings used in the datastores.
Check the monitor logs and compare the row counts between environments, that'll give us more info.
Michael

Similar Messages

.bat file result not giving correctly in data services

Hi ,
I am using .bat file to count the number of files in the folder. I want to capture the value of the number of files.
I have written small .bat file as shown below
@echo off
setlocal enableextensions
CD %1
set cnt=0
for %%A in (*) do set /a cnt+=1
echo File count = %cnt%
endlocal
I have used this batch file in the script as shown below.
$COUNT =exec('D:\\test.bat', 'D:\\docs',8);
print($COUNT);
In the above $COUNT is varchar(255).
actually i have only 11 file sin the docs folder . But it is giving the 442 files.
If I run the same batch file in the windows cmd command I am getting the correct result.
Please let me know what will be issues?
I am unable figure out the issues for weird result when I am using DS.
Please help me in this issue.
Thanks & Regards,
Ramana.

Hi,
While executing the batch file it is taking the Data services installation directory path as shown below.
14504 6420 PRINTFN 3/30/2015 9:40:10 AM 0: C:\Program Files (x86)\SAP BusinessObjects\Data Services\bin File count = 442
I have changed the the batch file with two parameters. one for changing directory and another one for changing the folder.
Now my batch file looks like this.
@echo off
setlocal enableextensions
%1:
CD %2
echo %CD%
set cnt=0
for %%A in (*) do set /a cnt+=1
echo File count = %cnt%
endlocal
Then my script is like this .
$COUNT =exec('D:\\test.bat', ' D D:\\docs',8);
print($COUNT);
Now it is giving the expected result as shown below.
14700 13912 PRINTFN 3/30/2015 9:52:18 AM 0: D:\Docs File count = 11
Thanks & Regards,
Ramana.

Data Services 4.0 Designer. Job Execution but empty log file no matter what

Hi all,
am running DS 4.0. When i execute my batch_job via designer, log window pops up but is blank. i.e. cannot see any trace messages.
doesn't matter if i select "Print all trace messages" in execution properties.
Jobserver is running on a seperate server. The only thing i have locally is just my designer.
if i log into the Data Services management console and select the job server, i can see trace and error logs from the job. So i guess what i need is for this stuff to show up in my designer?
Did i miss a step somewhere?
can't find anything in docs about this.
thanks
Edited by: Andrew Wangsanata on May 10, 2011 11:35 AM
Added additional detail

awesome. Thanks Manoj
I found the log file. in it relevant lines for last job i ran are
(14.0) 05-11-11 16:52:27 (2272:2472) JobServer: Starting job with command line -PLocaleUTF8 -Utip_coo_ds_admin
                                                -P+04000000001A030100100000328DE1B2EE700DEF1C33B1277BEAF1FCECF6A9E9B1DA41488E99DA88A384001AA3A9A82E94D2D9BCD2E48FE2068E59414B12E
                                                48A70A91BCB -ek******** -G"70dd304a_4918_4d50_bf06_f372fdbd9bb3" -r1000 -T1073745950 -ncollect_cache_stats
                                                -nCollectCacheSize -ClusterLevelJOB -Cmxxx -CaDesigner -Cjxxx -Cp3500 -CtBatch -LocaleGV
                                                -BOESxxx.xxx.xxx.xxx -BOEAsecLDAP -BOEUi804716
                                                -BOEP+04000000001A0301001000003F488EB2F5A1CAB2F098F72D7ED1B05E6B7C81A482A469790953383DD1CDA2C151790E451EF8DBC5241633C1CE01864D93
                                                72DDA4D16B46E4C6AD -Sxxx.xxx.xxx -NMicrosoft_SQL_Server -Qlocal_repo coo ds local
                                                repo_azdzgq4dnuxbm4xeriey1_e" -l"C:\Program Files (x86)\SAP BusinessObjects\Data Services/log/js01/tip coo ds local
                                                repo_azdzgq4dnuxbm4xeriey1_e/trace_05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3.txt" -z"C:\Program Files
                                                (x86)\SAP BusinessObjects\Data Services/log/js01/tip coo ds local
                                                repo_azdzgq4dnuxbm4xeriey1_e/error_05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3.txt" -w"C:\Program Files
                                                (x86)\SAP BusinessObjects\Data Services/log/js01/tip coo ds local
                                                repo_azdzgq4dnuxbm4xeriey1_e/monitor_05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3.txt" -Dt05_11_2011_16_52_27_9
                                                (BODI-850052)
(14.0) 05-11-11 16:52:27 (2272:2472) JobServer: StartJob : Job '05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3' with pid '148' is kicked off
                                                (BODI-850048)
(14.0) 05-11-11 16:52:28 (2272:2072) JobServer: Sending notification to <inet:10.165.218.xxx:56511> with message type <4> (BODI-850170)
(14.0) 05-11-11 16:52:28 (2272:2472) JobServer: AddChangeInterest: log change interests for <05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3> from client
                                                <inet:10.165.218.xxx:56511>. (BODI-850003)
(14.0) 05-11-11 17:02:32 (2272:2472) JobServer: RemoveChangeInterest: log change interests for <05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3> from client
                                                <inet:10.165.218.xxx:56511>. (BODI-850003)
(14.0) 05-11-11 19:57:45 (2272:2468) JobServer: GetRunningJobs() success. (BODI-850058)
(14.0) 05-11-11 19:57:45 (2272:2468) JobServer: PutLastJobs Success. (BODI-850001)
(14.0) 05-11-11 19:57:45 (2272:2072) JobServer: Sending notification to <inet:10.165.218.xxx:56511> with message type <5> (BODI-850170)
(14.0) 05-11-11 19:57:45 (2272:2472) JobServer: GetHistoricalLogStatus() Success. 05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3 (BODI-850001)
(14.0) 05-11-11 19:57:45 (2272:2472) JobServer: GetHistoricalLogStatus() Success. 05_11_2011_16_52_27_9__70dd304a_4918_4d50_bf06_f372fdbd9bb3 (BODI-850001)
it does not look like i have any errors with respect to connectivity? ( or any errors at all....)
Please advise on what, if anything you notice from log file and/or next steps i can take.
thanks.

Is there a way to import large XML files into HANA efficiently are their any data services provided to do this?

1. Is there a way to import large XML files into HANA efficiently?
2. Will it process it node by node or the entire file at a time?
3. Are there any data services provided to do this?
This for a project use case i also have an requirement to process bulk XML files, suggest me to accomplish this task

Hi Patrick,
I am addressing the similar issue. "Getting data from huge XMLs into Hana."
Using Odata services can we handle huge data (i.e create schema/load into Hana) On-the-fly ?
In my scenario,
I get a folder of different complex XML files which are to be loaded into Hana database.
Then I gotta transform & cleanse the data.
Can I use oData services to transform and cleanse the data ?
If so, how can I create oData services dynamically ?
Any help is highly appreciated.
Thank you.
Regards,
Alekhya

Data Services XI3.1 function module files for SAP R/3 is not working

Hi guys,
Thank you for the quick response so far. I am very grateful to you all.
I got an issue and will try to explain as detail as I could and
hope ur guys won't mind. :P
I pass the functions module to SAP tech guy to install to SAP using CTS
method, and using 900086.R63 file type because my chinese client is
running SAP on unicode environment.
My DS installed on my laptop client; job server plus db2 is on HP
unix. SAP is on another HP Unix server. The function module I used is
supplied from Data Services XI3.1.
Anyway, the installation is successfully done with GUI wizard windows of
SAP Workbench instead of tp command line method.
1)However, I checked the ZAW0 function group, I opened and can't see any functions
listed under it, although the table structures ZACTA, ZTAB2048 and other items
were created.
2)So I tried to create SAP Datastore, Transport Target, etc and tried to
extract file. It failed and log said "can't open file ---
/db2/temp/curcode.txt". I checked and SAP working directory /db2/temp didn't have
curcode.txt, because the result file supposed to be extracted and stored there.
The SAP tech guy checked SAP and cant find generated ABAP program of
ZCURCD is running anyway, although my local CURCD ABAP program is
created.
3)Is it the 900086.R63 not complete? Is my function installation correct? I could view data
on datastore, but just cant run job to extract. Another non-unidcode 900200.S08 group
has larger file sizes. Should I try to install also this 900200.S08 files? Will it corrupt SAP
and cause system errors?
4)After that, I had tried to install the functions manually into ZAW0
one by one using cut and paste, however after that, I cant view the data view anymore with
error. Besides, I tried to run Check on the functions and they all returned
syntax errors, so I cant Activate any of them with SAP. Maybe dependency is missing??
Then, I deleted the manually installed functions and can view data again, but just cant extract
data by job.
5)Another side issue is all these SAP datastore creation and viewing is
through DI 11.5 old installation. The new Data Services X3.1 unable to
create datastore with database error, although the function module
installed is came from Data Services. Does anyone know the issue?
Thank you very much to read it patiently. :P

Hi,
Thank you for the informative link. It did help a lot in solving the problem.
(1) to (4) --- The Sap tech guy didnt install the function programs correctly, and he redo the transport again using CTS. Now I can see all the functions inside the function group ZAW0. While for the problem of cannot generate extracted file to SAP working directory for downloading, it is due to unix directory access permission setting. And also have to include the ftp username into SAPSYS user group as the help link suggested.
(5) For this problem, I reinstalled again the DS with all required components like Server Manager. The network technical guy helped me with the first installation, and the installation was not complete. So now the issue is solved, I can create R/3 Datastore with DS.

ODSI csv file creation from a data service that read data from many tables.

Hi,
We have few dataservices running in our enviroment. Now we have a requirement to save the data as CSV files on demand. My question, can we do this using ODSI 10gR3?
I know we can read the csv file and create a physical data service. We have a compleded a use case that data comes in CSV file and saved to Oracle table when required.
Now how to do the reverse is my question. I am not findind any help in documents for this. I have opend a case with Oracle support for this, looks like they are not understading and taking it as a issue in ODSI 10gR3.
Please help.
Thanks in advance.
Rahul

we have a requirement to save the data as CSV files on demand. My question, can we do this using ODSI 10gR3? Yes. But you have to write the java code to do it. Create a class with a static method that takes two arguments - an array of XmlObject (or of the specific type that extends XmlObject that is created by compiling the schema) and a filename - and writes the values of the children of the array elements into your CSV file.
Put that class in a jar and put the jar in DSP-INF/lib, then create a physical datasource -> Java from that method, make sure you specify that resulting datasource procedure is a libraryProcedure.
So you'll have to do a little homework on ODSI Java Functions with XmlObject arguments.
- Mike

Ld namespace prefix in data service files

Mike,
Strange question. As you know typically when importing a schema the import will look something like this:
import schema namespace ns="http://www.test.com/test/1/" at "ld:schemas/SomeSchemaFile_1_0.xsd";
The namespace prefix in the URI "ld", I know you need that because it tells the data service that the file is local to the data service project. My question is it possible to import schema files in an external project outside the data space project? Also where is the ld prefix defined?
The only way I was able to import a schema file was using an absolute path from my hard drive...this would not be a good thing to do.
Mike

It would be embarrassing for me to tell you it is possible, and then three minutes later you post saying that you tried it and it isn't. So try it first and then let us all know.Yes, it would be embarrassing! Of coarse I tried, but according to the xquery spec when defining schema import they must be absolute paths, so I wasn't sure if there was an ODSI implementation specific mechanism for declaring other types of paths...like the ld namespace prefix, but a custom one.
>
import schema namespace ns="http://www.test.com/test/1/" at "ld:schemas/SomeSchemaFile_1_0.xsd";
Also where is the ld prefix defined?I believe the string following the "at" in an import statement in xquery is "implementation dependent" (you can check the spec). The 'protocol' ld is just something ODSI uses to indicate that schema will be self-contained in the dataspace. I'm guessing that the protocols http and file will also work - you can try that to answer the first part of your question. I believe if you have an update function, you'll find you that datagraph.xsd is accessed via http.
- MikeI haven't tried but I am willing to bet the http path will work because that is an acceptable path based on the xquery spec. I am thinking to get what I want to happen, I could write a small ant task to copy the schemas from the external project to the data space when building. As always thanks for the quick reply!
Mike

BO Data Services - Reading from excel file and writing to SQL Server Table

Hi,
I would like to read data from an excel file and write it to a SQL Server Data base table without making any transformations using Data Services. I have created an excel file format as source and created target table in SQL Server. The data flow will just have source and target. I am not sure how to map the columns between source and target. Appreciate your quick help in providing a detailed steps of mapping.
Regards,
Ramesh

Ramesh,
were you able to get this to work? if not, let me know and I can help you out with it.
Lynne

How do I create rules files in Sap Data Services

Hello Guys,
I'm new with Bo Sap Development Data Services, and am migrating SSIS packages 2008 for the bods.
I am studying the process of cleanup and word processing via bods.
But I noticed that my installation does not have Rules Files.
My question is:
How do I create rules files in Sap Data Services library with support for Portuguese?

Hello Ramana,
So with version 4.2 of Sap Data Services do not currently have the rules files?
How to mount files for text comparison Cleanse facing Addresses, Names, Clients, Companies? If you can give me some help link?
Thanks again ...

Data Services result as a .txt file

I am trying to output the results of a JOB in data services to a .txt tab delimited file. Is there a way to wrie out at a .txt file. The job is currently outputting as a template table and would like for it to be txt instead
Please help,
Travis

Hi Travis.
Yes, it is possible. Create a flat file format and use it as a target.
Designer Guide: http://help.sap.com/businessobject/product_guides/boexir4/en/sbo401_ds_designer_en.pdf (6.2 File format editor).
Could you have any questions please let me know.
Thank you,
Viacheslav.

Data Service+File Technology

Hi,
Im trying to transform data from Source CSV to target CSV with staging area different from target(the staging area is a RDBMS)
LKM - File to SQL
IKM - SQL to File Append...
Im unable to generate the Data Service on the target datstore(File datastore). While generating i get an error message during the Generating code phase.
Could any1 share the KM's to be used and general process to follow for File to File transformations.
Error:
com.sunopsis.tools.core.exception.SnpsRuntimeException: java.sql.SQLException: Invalid format description
     at com.sunopsis.dwg.dataservices.AbstractWSGenerator.k(AbstractWSGenerator.java)
     at com.sunopsis.dwg.dataservices.AbstractWSGenerator.a(AbstractWSGenerator.java)
     at com.sunopsis.dwg.dataservices.WSGenerationTask.a(WSGenerationTask.java)
     at com.sunopsis.graphical.i.c.b(c.java)
     at com.sunopsis.graphical.tools.utils.swingworker.v.call(v.java)
     at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
     at com.sunopsis.graphical.tools.utils.swingworker.l.run(l.java)
     at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
     at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
     at java.lang.Thread.run(Thread.java:595)

The file driver (ODI JDBC Driver for files) requires more than just a SQL command to access the data, and that is what will be generated by default. It would not be straightforward to implement data services on flat file technology.

How to read .db file for Data services in Linux Environment

Hi all,
Can anyone tell me how to read a .db file for SAP BODS 4.0 which got installed in Linux?
This is Urgent...

If we are talking about MS Access, AFAIK there is no native support of that format in SAP Data Services. You may try to find ODBC driver fot that.
Thank you,
Viacheslav.

Can Data Services take .gz (zipped) file

Hello, I am trying to figure to how to setup a Data Services job to process an .xml file that is stored in a .gz (zipped) file. Can Data Services process .gz files? I don't think it can. I need to figure out how to unzip it in the job and process the file..
Thanks.

From reading this thread, i'm guessing you re fairly new to unix ?
Before getting too involved in the data services part of this, make sure you aren't getting problems because of other issues.
first, write your script to call gzip taking a filename as a parameter.
I suggest you create a standard directory you keep all the scripts you will call from Data Services in.
When you are login in to the account Data services runs under
type:
cd ~
(This should change to your home directory if your not already there) then press return.
next type:
pwd
then press return
This should display the home directory for data services. Lets assume it is '/home/dataservices'
Next create a directory for your script:
mkdir /home/dataservices/scripts
change into that directory:
cd /home/dataservices/scripts
write your script:
vi gunzip.ksh
(Replace vi with the editor or your choice)
Your script should at its most basic look something like this:
#!/bin/ksh
gzip - d $1
NB: If you don't understand shell scripting, google is your friend. A link I found quickly:
http://www.dartmouth.edu/~rc/classes/ksh/
next set the script as executable:
chmod u+x gunzip.ksh
finally verify it works from the command line:
/home/dataservices/scripts/gunzip.ksh myfile.gz
If it uncompresses the file, you should be able to run this from data services with no trouble.
If you don't understand these unix commands, you really need to forget about data services for a while and go and learn unix There is plenty of good stuff online and millions of books for beginners like dummies guides, etc.
Good luck !

Any one know how to use "custom" option present under the data access tab in XLS file format of Data Services

Hi Experts,
Any one know how to use or what is the purpose of "custom" option present under the data access tab in Excel workbook file format of Data Services
Thanks in Advance,
Rajesh.

Rajesh, what is the Custom Protocol you are trying to use? It should be something like PSFTP, etc.,
Cheers
Ganesh Sampath

Data Services Designer 14 - Large Log files

Hello,
we're running several jobs with the Data Services Designer 14, all works fine.
But today a problem occur:
The Data Designer on a client produces after finishing a big job a very large log file in the Data Services Designer folder with 8 GB.
Is it possible to delete these log files automatically or restrict the maximum size of the created log files in the designer?
What's the best way?
Thanks!

You can set to automatically delete the log files based on number of days.
I have done this in XI 3.2, but as per the document, this is how it can be done in DS 14.0.
In DS 14.0, this is handled in CMC.
1. Log into the Central Management Console (CMC) as a user with administrative rights to the Data Services application.
2. Go to the u201CApplicationsu201D management area of the CMC. The u201CApplicationsu201D dialog box appears.
3. Right-click the Data Services application and select Settings. The u201CSettingsu201D dialog box appears.
4. In the Job Server Log Retention Period box, enter the number of days that you want to retain the following:
u2022 Historical batch job error, trace, and monitor logs
u2022 Current service provider trace and error logs
u2022 Current and historical Access Server logs
The software deletes all log files beyond this period. For example:
u2022 If you enter 1, then the software displays the logs for today only. After 12:00 AM, these logs clear and the software begins saving logs for the next day.
u2022 If you enter 0, then no logs are maintained.
u2022 If you enter -1, then no logs are deleted.
Regards,
Suneer.

Data Services X-File

Similar Messages

Maybe you are looking for