Regarding TXT File data truncation due to large amount of data

Hi Guys,
I am downloading data to txt.file in background.I am getting truncation of the records due to large amount of data. If it is less data it works good.
I have checked the Internal table SIZE for this and anywhy i have declared in OCCURS 0 only.
So please help me to find out what may this reason.I am confuced is there any limitation for TXT file??
Please help me guys..
Thanks in advance..
Prabhu.R

Hi Rakesh,
two ways.
1. Ask ur BASIS team to increase the memory level.
2. Check the PACKAGE SIZE option of select statement
Here u won't select all the data once but in packets of specified size. So get the packets of data and process.
Just press F1 on package size. That explanation will be enough to proceed further.
Thanks,
Vinod.

Similar Messages

Getting Short dump due to large amount of data

Hi experts,
When we are running RALM_ME_MEASP_FULL_DOWNLOAD_SD program, every time we are getting Short dump due to large amount of data.
please suggest how to run this program without short dump.
Thanks & Regards
Prashant Gupta

Hi,
you do run the wrong APP I guess. The service you are running is for MAM20. If you are interested in MAM30 and MAM25, please use the correct service.
If you have a look for
*FULL_DOWNLOAD_SD
You find them all. Use the ones without the RALM in front - then the timeout should be solved as well. Bside that there is a great guid available that helps as well to solve some issues around server driven replication:
[MAM SERVER DRIVEN SETUP GUIDE|http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/818ac119-0b01-0010-ba8b-b6e3f3490a63?quicklink=index&overridelayout=true]
Hope that helps to solve yourt issue.
Regards,
Oliver

Creation of data packages due to large amount of datasets leads to problems

Hi Experts,
We have build our own generic extractor.
When data packages (due to large amount of datasets) are created, different problems occur.
For example:
Datasets are now doubled and appear twice, one time in package one and a second time in package two. Since those datsets are not identical, information are lost while uploading those datasets to an ODS or Cube.
What can I do? SAP will not help due to generic datasource.
Any suggestion?
BR,
Thorsten

Hi All,
Thanks a million for your help.
My conclusion from your answers are the following.
a) Since the ODS is Standard - within transformation no datasets are deleted but aggregated.
b) Uploading a huge amount of datasets is possible in two ways:
b1) with selction criteria in InfoPackage and several uploads
b2) without selction criteria in InfoPackage and therefore an automatic split of datasets in data packages
c) both ways should have the same result within the ODS
Ok. Thanks for that.
So far I have only checked the data within PSA. In PSA number of datasets are not equal for variant b1 and b2.
Guess this is normal technical behaviour of BI.
I am fine when results in ODS are the same for b1 and b2.
Have a nice day.
BR,
Thorsten

Data Transfer Prozess (several data packages due two huge amount of data)

Hi,
a)
I`ve been uploading data from ERP via PSA, ODS and InfoCube.
Due to a huge amount of data in ERP - BI splits those data in two data packages.
When prozessing those data to ODS the system delete a few dataset.
This is not done in step "Filter" but in "Transformation".
General Question: How can this be?
b)
As described in a) data is split by BI into two data packages due to amount of data.
To avoid this behaviour I enterd a few more selection criteria within InfoPackage.
As a result I upload data a several time, each time with different selction criteria in InfoPackage.
Finally I have the same data in ODS as in a), but this time without having data deleted in step "Transformation".
Question: How is the general behaviour of BI when splitting data in several data packages?
BR,
Thorsten

Hi All,
Thanks a million for your help.
My conclusion from your answers are the following.
a) Since the ODS is Standard - within transformation no datasets are deleted but aggregated.
b) Uploading a huge amount of datasets is possible in two ways:
b1) with selction criteria in InfoPackage and several uploads
b2) without selction criteria in InfoPackage and therefore an automatic split of datasets in data packages
c) both ways should have the same result within the ODS
Ok. Thanks for that.
So far I have only checked the data within PSA. In PSA number of datasets are not equal for variant b1 and b2.
Guess this is normal technical behaviour of BI.
I am fine when results in ODS are the same for b1 and b2.
Have a nice day.
BR,
Thorsten

Azure Cloud service fails when sent large amount of data

This is the error;
Exception in AZURE Call: An error occurred while receiving the HTTP response to http://xxxx.cloudapp.net/Service1.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being
aborted by the server (possibly due to the service shutting down). See server logs for more details.
Calls with smaller amounts of data work fine. Large amounts of data cause this error.
How can I fix this??

Go to the web.config file, look for the <binding> that is being used for your service, and adjust the various parameters that limit the maximum length of the messages, such as
maxReceivedMessageSize.
http://msdn.microsoft.com/en-us/library/system.servicemodel.basichttpbinding.maxreceivedmessagesize(v=vs.100).aspx
Make sure that you specify a size that is large enough to accomodate the amount of data that you are sending (the default is 64Kb).
Note that even if you set a very large value here, you won't be able to go beyond the maximum request length that is configured in IIS. If I recall correctly, the default limit in IIS is 8 megabytes.

Open Large amount of data

Hi
I have a file on application server in .dat format, it contains large amount of data may be 2 million of records or more, I need to open the file to check the record count, is there any software or any option to open the file, I have tried opening with Notepad, excel .... it gives error..
please let me know
Thanks

Hi,
Try this..
Go to AL11..
Go to the file directory..Then in the file there will be field called length..which is the total length of the file in characters..
If you know the length of a single line..
Divide the length of the file by the length of single line..I believe you will get the number of records..
Thanks,
Naren

With journaling, I have found that my computer is saving a large amount of data, logs of all the changes I make to files; how can I clean up these logs?

With journaling, I have found that my computer is saving a large amount of data, logs of all the changes I make to files; how can I clean up these logs?
For example, in Notes, I have written three notes; however if I click on 'All On My Mac' on the side bar, I see about 10 different versions of each note I make, it saves a version every time I add or delete a sentence.
I also noticed, that when I write an email, Mail saves about 10 or more draft versions before the final is sent.
I understand that all this journaling provides a level of security, and prevents data lost; but I was wondering, is there a function to clean up journal logs once in a while?
Thanks
Roz

Are you using Microsoft word? Microsoft thinks the users are idiots. They put up a lot of pointless messages that annoy & worry users. I have seen this message from Microsoft word. It's annoying.
As BDaqua points out...
When you copy information via edit > copy, command + c, edit > cut, or command +x, you place the information on the clipboard. When you paste information, edit > paste or command + v, you copy information from the clipboard to your data file.
If you edit > cut or command + x and you do not paste the information and you quite Word, you could be loosing information. Microsoft is very worried about this. When you quite Word, Microsoft checks if there is information on the clipboard & if so, Microsoft puts out this message.
You should be saving your work more than once a day. I'd save every 5 minutes. command + s does a save.
Robert

Is there any way to connect time capsule to a MacBook Pro directly via USB. I have a large amount of data that I want to back up and it is taking a very long time (35GB is taking 3 hrs, I have 2TB if files in total)...)?

Perhaps via USB. I have a large amount of data that I want to back up and it is taking a very long time (35GB is taking 3 hrs, I have 2TB if files in total)...? I want to use TimeCapsule as back-up for an archive which is curently stored on a 2 TB WESC HD.

No, you cannot backup via direct usb connection..
But gigabit ethernet is much faster anyway.. are you connected directly by ethernet?
Is the drive you are backing up from plugged into the TC? That will slow it down something chronic.. plug that drive in by its fastest connection method.. WESC sorry I have no idea. If ethernet use that.. otherwise USB direct to the computer.. always think what way the files come and go.. but since you are copying from the computer everything has to go that way.. it makes things slower if they go over the same cable.. if you catch the drift.

Best practice for storing/loading medium to large amounts of data

I just have a quick question regarding the best medium to store a certain amount of data. Currently in my application I have a Dictionary<char,int> that I've created, that I subsequently populate with hard-coded static values.
There are about 30 items in this Dictionary, so this isn't presented as much of a problem, even though it does make the code slightly more difficult to read, although I will be adding more data structures in the future with a similar number of items.
I'm not sure whether it's best practice to hard-code these values in, so my question is, is there a better way to store this information, retrieve and load it at run-time?

You could use one of the following methods:
Use the application.config file. Upside is that it is easy to maintain. Downside is a user could edit it manually as its just an xml file.
You could use a settings file. You can specify where the setting file is persisted including under the user's profile or the application. You could serialize/deserialize your settings to a section in the settings. See
this MSDN help section
on details abut the settings.
Create a .txt, .json, or .xml file (depending on the format you will be deserializing your data) in your project and have it be copied to the output path with each build. The upside is that you could push out new versions in the future of the file without
having to re-compile your application. Downside is that it could be altered if the user has O/S permissions to that directory.
If you really do not want anyone to access it and are thinking of pushing out a new application version every time something changes you could create a .txt, .json, .xml file (depending on the format you will be deserializing your data) just like the previous
step but this time mark it as an embedded resource in your project (you can do this in the properties of the file in visual studio). It will essentially get compiled in your application. Content retrieval is outlined in
this how to from Microsoft and then you just deserialize the retrieved content the same as the previous step.
As far as formats of your data. I recommend you use either XML or JSON or a text file if its just a flat list of items (ie. list of strings). Personally I find JSON much easier to read compared to XML and change and there are plenty of supported serializers
out there. XML is great too if you need to be strict as to what the schema is.
Mark as answer or vote as helpful if you find it useful | Igor

General practices - managing large amounts of data

I have a basic question regarding data management in Java. Right now, I'm working on a program that deals with hundreds of "pages" of content, each with images, HTML, and some state data. I maintain info on each page in a ContentItem object. Similarly, there can be many categories of content, each of which I maintain in a ContentCategory object.
At this point, I am controlling access to data using manager classes. For example, I have a ContentManager class that contains a global, static list of content and has the methods you'd expect such a class to have (getContent, addContent, deleteContent, etc.). Similarly, I have a CategoryManager class.
Basically, any time my program has to deal with large amounts of data or objects, I sort of fall back on these manager classes. I'm wondering if this is a reasonable practice or not. If not, perhaps some of the more experienced developers here could recommend a design pattern that fits these situations.

Thanks for the reply. I do have a sort of ad-hoc database that I'm using. I have a class called ContentReference which holds just the basic state information about a ContentItem. The actual HTML and image data are stored on disk and retrieved as needed. The files for each content item are just serialized copies of ContentItem objects. Each content item has a unique ID value which is passed to the ContentManager's getContent method. The getContent method retrieves the object from disk and returns the ContentItem object. The filenames for all the content items are based on the ID value, so the getContent method doesn't have to search through all the files until it finds the right one.
In doing this, I don't have to keep all the HTML and image data in memory. Only the ContentReference objects are kept in memory. ContentItems are loaded as needed. It seems a little messy to have two objects that refer to the same thing, but I didn't see any other way of doing it.

Interpolating large amounts of data

Does anyone have any recommendations regarding a method for interpolating large portions of data. From what I've seen through my testing, is that I'm able to sample a lot of points (incoming messages from a gps device). My problem is when I go to interpolate these sets of data.
I've tried capturing a (lat, long, timestamp), then interpolating both lat/long togetherand one at a time. I've also tried storing the readings to different .xls files, reading the data back into LabVIEW and then interpolating together, and separately. But niether of these methods have been able to avoid the lack of system resources issue, when the program has captured near 4000 signals.
I'm not sure where else to go from here.
All these methods have

"systems_eng" <[email protected]> wrote in message news:[email protected]..
Does anyone have any recommendations regarding a method for interpolating large portions of data.  From what I've seen through my testing, is that I'm able to sample a lot of points (incoming messages from a gps device).  My problem is when I go to interpolate these sets of data.
 
I've tried capturing a (lat, long, timestamp), then interpolating both lat/long togetherand one at a time.  I've also tried storing the readings to different .xls files, reading the data back into LabVIEW and then interpolating together, and separately.  But niether of these methods have been able to avoid the lack of system resources issue, when the program has captured near 4000 signals.
 
I'm not sure where else to go from here.
 
All these methods have
4000 points is not that much. Interpolation (linear?) should take about 1 sec. max. What lack of system resources do you get? Working memory, memory leaks or proccesor load?
I'd be hard to recommend a solution without seeing some code.
If you'd be able to post 1 vi that has the input data as default values, and a working solution with leaks, we can help you much better. (In LabVIEW 7.1 for me, but someone will help you if you post in LV8, or 8.2)
Are you using real excel files, or are the .xls files tab dilimited asci files?
Regards,
Wiebe.

Bex Report Designer - Large amount of data issue

Hi Experts,
I am trying to execute (on Portal) report made in BEx Report Designer, with about 30 000 pages, and the only thing I am getting is a blank page. Everything works fine at about 3000 pages. Do I need to set something to allow processing such large amount of data?
Regards
Vladimir

Hi Sauro,
I have not seen this behavior, but it has been a while since I tried to send an input schedule that large. I think the last time was on a BPC NW 7.0 SP06 system and it worked OK. If you are on a recent support package, then you should search for relevant notes (none come to mind for me, but searching yourself is always a good idea) and if you don't find one then you should open a support message with SAP, with very specific instructions for recreating the problem from a clean input-schedule.
Good luck,
Ethan

Advice needed on how to keep large amounts of data

Hi guys,
Im not sure whats the best way is to make large amounts of data available to my android app on the local device.
For example records of food ingredients, in the 100's?
I have read and successfully created .db's using this tutorial.
http://help.adobe.com/en_US/AIR/1.5/devappsflex/WS5b3ccc516d4fbf351e63e3d118666ade46-7d49. html
However to populate the database I use flash? So this kind of defeats the purpose of it. No point in me shifting a massive array of data from flash to a sql database, when I could access the data direct from the as3 array?
So maybe I could create the .db with an external program? but then how would I include that .db in the apk file and then deploy it to users android device.
Or maybe I create a as3 class with an xml object init and use that as a means of data storage?
Any advice would be appreciated

You can use any means you like to populate your SQLite database, including using external programs, (temporarily) embedding a text file with SQL statements, executing some SQL from AS3 code etc etc.
Once you have populated your db, deploy it with your project:
http://chrisgriffith.wordpress.com/2011/01/11/understanding-bundled-sqlite-databases-in-ai r-for-mobile/
Cheers, - Jon -

Send large amounts of data

Hello everyone,
I made an applet that receives data, signs and returns the signed data. When the amount of data is too big, I break it into blocks of 255 bytes and use the method Signature.update.
OK, this is working fine, but perfomarnce is poor due to large amount of blocks. Is it possible to increase the size of the blocks?
Thanks.

Hi,
You cannot change the block size but you can change what you send in.
You may get better performance by sending multiples of your hash function block size as the card will not have to do internal buffering.
You could also do as much of the work in your host application as possible and then just send in the data that you need to operate on with the private key. Generating the hash of the message does not require a private key so can be done in your host. You then send the result of the hash to the card to be encrypted with the private key. This will be the fastest method.
Cheers,
Shane

DSS problems when publishing large amount of data fast

Has anyone experienced problems when sending large amounts of data using the DSS. I have approximately 130 to 150 items that I send through the DSS to communicate between different parts of my application.
There are several loops publishing data. One publishes approximately 50 items in a rate of 50ms, another about 40 items with 100ms publishing rate.
I send a command to a subprogram (125ms) that reads and publishes the answer on a DSS URL (app 125 ms). So that is one item on DSS for about 250ms. But this data is not seen on my man GUI window that reads the DSS URL.
My questions are
1. Is there any limit in speed (frequency) for data publishing in DSS?
2. Can DSS be unstable if loaded to much?
3. Can I lose/miss data in any situation?
4. In the DSS Manager I have doubled the MaxItems and MaxConnections. How will this affect my system?
5. When I run my full application I have experienced the following error Fatal Internal Error : ”memory.ccp” , line 638. Can this be a result of my large application and the heavy load on DSS? (se attached picture)
Regards
Idriz Zogaj
Idriz "Minnet" Zogaj, M.Sc. Engineering Physics
Memory Profesional
direct: +46 (0) - 734 32 00 10
http://www.zogaj.se

LuI wrote:
>
> Hi all,
>
> I am frustrated on VISA serial comm. It looks so neat and its
> fantastic what it supposes to do for a develloper, but sometimes one
> runs into trouble very deep.
> I have an app where I have to read large amounts of data streamed by
> 13 µCs at 230kBaud. (They do not necessarily need to stream all at the
> same time.)
> I use either a Moxa multiport adapter C320 with 16 serial ports or -
> for test purposes - a Keyspan serial-2-USB adapter with 4 serial
> ports.
Does it work better if you use the serial port(s) on your motherboard?
If so, then get a better serial adapter. If not, look more closely at
VISA.
Some programs have some issues on serial adapters but run fine on a
regular serial port. We've had that problem recent
ly.
Best, Mark

Regarding TXT File data truncation due to large amount of data

Similar Messages

Maybe you are looking for