Data structure to hold frequencies or counts

i need to know if there is a data structure with a supporting algorithm that can query counts. for example, if i have two variables, eye color and hair color, where the values of eye color are brown, blue, green, hazel, and the values of hair color are black and blonde, i'd like to set the frequencies for the combination of these values and also be able to query very quickly the frequencies for the combination i.e. how many people had brown eyes and black hair?
what i need could be accomplished easily using a database i.e. select count(*) as total from tbl where eye_color='brown' and hair_color='black'. however, i have tried using various databases (i.e. oracle, ms sql server, mysql), and the performance is poor when the number of variables or rows grow. also, a database approach introduces various complexities (i.e. trying to code database vendor agnostic coding, learning how to index the fields to optimize reads, figuring out field types or efficient storage, etc...).
any help is appreciated. thanks.

You should check apache-commons' MultiKey and MultiKeyMap:
http://commons.apache.org/collections/api-release/org/apache/commons/collections/map/MultiKeyMap.html
Also adding an index to a database column is quite straightforward, there are many optimization tips available, e.g.:
http://www.databasejournal.com/features/mssql/article.php/1443581
http://www.databasejournal.com/features/mysql/article.php/10897_1382791_1

Similar Messages

Implement a program by using data structure

i've learn link-list,b-tree,red black tree,hash table. now it's time for me to implement a program using any one of these. i've no idea about how to start, and what kind of program should i write? what kind of program use data structure? i want some simple example program to make me understand. can anyone help me?
thank 4 u r time,
makio

what sort of program do use data structure? let's say
i want to use binary search on data , how can i?As nasch said: pretty much any program uses data structures.
Okay, let's say you want to try binary search. You could do that on an array, on a singly linked list, on a doubly linked list, probably on others that I can't think of right now.
If you just want practice using the data structures you've learned, then why not write a binary search over several different ones? That will give you a feel for how they differ in usage and performance.
Or, you can just pick any kind of program that interests you. Unless it's trivially small and simple, there will probably be a use for one or more of the data strux you've learned.
A chat program needs data structures to hold curently connected users, ongoing conversations, maybe a queue for messages waiting to be sent.
A card game needs to be able to search to see if a given card is in a given hand. Needs to hold all the cards in a deck that has to be shuffled. Dealing is like popping off a stack.
A database of students and their courses might want to keep something in an ordered tree--by last name for instance.
What applications interest you?

Need help on efficient searches using hashes on large #s of data structures

I have a massive amount of data and I need a way to access it using generated indexes instead of traditional searching such as binary chop as I require a fast response time.
I have 100,000 Array data structures each containing roughly 10 to 60 elements. The elements of the Arrays do not store any raw data they just hold references to objects. I have about 10,000 unique objects which are referenced to by the arrays.
What I need to do is supply a number of objects as the query and the system should return all the arrays which contain two or more different objects from the query.
So for example>
Given query objects> “DER” , “ERE” , “YPS”
and arrays> [DER, SFR, PPR]
     [PER, ERE, SWE, YPS]
     [ERE, PPD, DER, YPS, SWE]
     [PRD, LDF, WSA, MMD]
It should return arrays 2 and 3.
I had a look at hashMaps but I’m not entirely sure if that’s what I need.
Would I need to hash the each Array and then hash the query objects and look for overlaps in the two hashes or something similar?
Thanks In Advance

For a single query, it's more efficient to do less work, so don't create intermediate objects, don't create a index of objects to sets of possibles, and stop counting matches when you find enough:.
                final HashSet<String> querySet = new HashSet<String>(Arrays.asList(query));
                final Set<String> results = new LinkedHashSet<String>();
                // find matches using scan of all arrays
                for (String[] array : data) {
                    int matchCount = 0;
                    for (String elt : array) {
                        if (querySet.contains(elt)) {
                            ++matchCount;
                            if (matchCount >= 2) {
                                results.add(Arrays.asList(array).toString());
                                break;
                }Using random data, 100,000 arrays of selections from a list of 10,000 trigrams, scanning all the arrays takes about as long as the set code, but doesn't require generation of the map, which can take up a lot of memory if you pre-create the sets (I didn't complete a run doing that, although it should be faster to query, it used too much memory and my computer started swapping so I gave up after a few minutes).
Running queries of size 2 to 25, times in ms:
linear scan of all arrays in data set:
elapsed: 2196
creating index of trigram to arrays containing trigram:
elapsed: 4635
query using set union and index lookup:
elapsed: 2195
query using direct scan and index lookup:
elapsed: 164
So if I was doing one query per dataset, just use a simple scan as above. If you're doing lots on the same dataset, then create a map<String, List<String[]>> to index the arrays, and the lookup will save quite a bit of time.
Creating the index:.
            dataSets = new HashMap<String, List<String[]>>();
            for (String trigram : trigrams)
                dataSets.put(trigram, new ArrayList<String[]>());
            for (String[] array : data)
                for (String trigram : array)
                    dataSets.get(trigram).add(array);Finding matches using scan of arrays with at least one elt in query:.
                final HashSet<String> querySet = new HashSet<String>(Arrays.asList(query));
                final Set<String> results = new LinkedHashSet<String>();
                for (String trigram : query) {
                    for (String[] array : dataSets.get(trigram)) {
                        for (String elt : array) {
                            if (elt != trigram && querySet.contains(elt)) {
                                results.add(Arrays.asList(array).toString());
                                break;
                }

Can I automate the creation of a cluster in LabView using the data structure created in an autogenerated .CSV, C header, or XML file?

Can I automate the creation of a cluster in LabView using the data structure created in an auto generated .CSV, C header, or XML file? I'm trying to take the data structure defined in one or more of those files listed and have LabView automatically create a cluster with identical structure and data types. (Ideally, I would like to do this with a C header file only.) Basically, I'm trying to avoid having to create the cluster by hand, as the number of cluster elements could be very large. I've looked into EasyXML and contacted the rep for the add-on. Unfortunately, this capability has not been created yet. Has anyone done something like this before? Thanks in advance for the help.
Message Edited by PhilipJoeP on 04-29-2009 04:54 PM
Solved!
Go to Solution.

smercurio_fc wrote:
Is this something you're trying to do at runtime? Clusters are fixed data structures so you can't change them programmatically. Or, are you just trying to create some typedef cluster controls so that you can use them for coding? What would your clusters basically look like? Perhaps another way of holding the information like an array of variants?
You can try LabVIEW scripting, though be aware that this is not supported by NI.
Wow! Thanks for the quick response! We would use this cluster as a fixed data structure. No need to change the structure during runtime. The cluster would be a cluster of clusters with multiple levels. There would be not pattern as to how deep these levels would go, or how many elements would be in each. Here is the application. I would like to be able to autocode a Simulink model file into a DLL. The model DLL would accept a Simulink bus object of a certain data structure (bus of buses), pick out which elements of the bus is needed for the model calculation, and then pass the bus object. I then will take the DLL file and use the DLL VI block to pass a cluster into the DLL block (with identical structure as the bus in Simulink). To save time, I would like to auto generate the C header file using Simulink to define the bus structure and then have LabView read that header file and create the cluster automatically. Right now I can do everything but the auto creation of the cluster. I can manually build the cluster to match the Simulink model bus structure and it runs fine. But this is only for an example model with a small structure. Need to make the cluster creation automated so it can handle large structures with minimal brute force. Thanks!

Urgent! Need help in deciding data structure to use

Hi all,
I need to implement a restaurant system by which a customer can make a reservation.
I was wondering whether vector would be able to be store such an object, The thing is I don't want a complex data structure. just sumthin simple cos i hardly have anytime left to my submission. sighz...
The thing is I need to able to search based on 2 properties of an object. Eg. I need to search for a reservation based on the customer name and the date he reserved a table.
But I am totally clueless how to search thru a vector based on 2 properties of the object... Would really appreciate some help. Like an example how to so so based on my program. Feelin so lost...This is all I have so far:
class AddReservation
     static BufferedReader stdin = new BufferedReader (new InputStreamReader(System.in));
     //Main Method
     public static void main (String[]args)throws IOException
          String custName, comments;
          int covers, date, startTime, endTime;
          int count = 0;
          //User can only add one reservation at a time
          do
               //Create a new reservation
               Reservation oneReservation=new Reservation();
               System.out.println("Please enter customer's name:");
               System.out.flush();
               custName = stdin.readLine();
               oneReservation.setcustName(custName);
               System.out.println("Please enter number of covers:");
               System.out.flush();
               covers = Integer.parseInt(stdin.readLine());
               oneReservation.setCovers(covers);
               System.out.println("Please enter date:");
               System.out.flush();
               date = Integer.parseInt(stdin.readLine());
               oneReservation.setDate(date);
               System.out.println("Please enter start time:");
               System.out.flush();
               startTime = Integer.parseInt(stdin.readLine());
               oneReservation.setstartTime(startTime);
               System.out.println("Please enter end time:");
               System.out.flush();
               endTime = Integer.parseInt(stdin.readLine());
               oneReservation.setendTime(endTime);
               System.out.println("Please enter comments, if any:");
               System.out.flush();
               comments = stdin.readLine();
               oneReservation.setComments(comments);
               count++;
          while (count<1);
          class Reservation
          private Reservation oneReservation;
          private String custName, comments;
          private int covers, startTime, endTime, date;
          //Default constructor
          public Reservation()
          public Reservation(String custName, int covers, int date, int startTime, int endTime, String comments)
               this.custName=custName;
               this.covers=covers;
               this.date=date;
               this.startTime=startTime;
               this.endTime=endTime;
               this.comments=comments;
          //Setter methods
          public void setcustName(String custName)
               this.custName=custName;
          public void setCovers(int covers)
               this.covers=covers;;
          public void setDate(int date)
               this.date=date;
          public void setstartTime(int startTime)
               this.startTime=startTime;
          public void setendTime(int endTime)
               this.endTime=endTime;
          public void setComments(String comments)
               this.comments=comments;
          //Getter methods
          public String getcustName()
               return custName;
          public int getCovers()
               return covers;
          public int getDate()
               return date;
          public int getstartTime()
               return startTime;
          public int getendTime()
               return endTime;
          public String getComments()
               return comments;
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
class searchBooking
     static BufferedReader stdin = new BufferedReader (new InputStreamReader(System.in));
     public static void main (String[]args)throws IOException
          int choice, date, startTime;
          String custName;
               //Search Menu
               System.out.println("Search By: ");
               System.out.println("1. Date");
               System.out.println("2. Name of Customer");
               System.out.println("3. Date & Name of Customer");
               System.out.println("4. Date & Start time of reservation");
               System.out.println("5. Date, Name of customer & Start time of reservation");
               System.out.println("Please make a selection: ");
               //User keys in choice
               System.out.flush();
               choice = Integer.parseInt(stdin.readLine());
               if (choice==1)
                    System.out.println("Please key in Date (DDMMYY):");
                    System.out.flush();
                    date = Integer.parseInt(stdin.readLine());
               else if (choice==2)
                         System.out.println("Please key in Name of Customer:");
                         System.out.flush();
                         custName = stdin.readLine();
               else if (choice==3)
                         System.out.println("Please key in Date (DDMMYY):");
                         System.out.flush();
                         date = Integer.parseInt(stdin.readLine());
                         System.out.println("Please key in Name of Customer:");
                         System.out.flush();
                         custName = stdin.readLine();
               else if (choice==4)
                         System.out.println("Please key in Date (DDMMYY):");
                         System.out.flush();
                         date = Integer.parseInt(stdin.readLine());
                         System.out.println("Please key in Start time:");
                         System.out.flush();
                         startTime = Integer.parseInt(stdin.readLine());
               else if (choice==5)
                         System.out.println("Please key in Date (DDMMYY):");
                         System.out.flush();
                         date = Integer.parseInt(stdin.readLine());
                         System.out.println("Please key in Name of Customer:");
                         System.out.flush();
                         custName = stdin.readLine();
                         System.out.println("Please key in Start time:");
                         System.out.flush();
                         startTime = Integer.parseInt(stdin.readLine());
                    }

Please stop calling your questions urgent. Everybody's question is urgent to them. Nobody's are urgent to the people who are going to answer them. Calling your questions urgent suggests that you think they are more important than others' (They're not.) and will only serve to irritate those who would help you. It won't get your questions answered any sooner.

A robust data structure for a histogram of events over time.

Hi all,
I've been thinking for days, I am still lost on how to solve the following:
Let's say that we have a list of events that occur over a certain time interval. Each event has a start timestamp and a finish timestamp. The events occur in no particular order and expand over various length of periods.
With respect to the histogram, we mark the time period of an event occuring with a frequency of one. When events overlap over a time period, we mark that time period with a frequency of number of overlapping events.
Given this raw data, we want to examine the data's frequency under various granularity such as second, minutes, hours, days, weeks, month and years for 60sec, 60min, 24 hrs, 7 days, 4 wks ranges respectively.
So, what is a good intermediate (general) data structure that can store all the various events such that they can be easily transformed to a specific granularity. My goal here is to read the event list once, and histogram of events data with respect to various timeframe can be calculated.
Does a timeline like data strucuture that stores aggregated frequencies solve my problem? With a fine granuarlity say seconds and a long time range, the data structure will bound to overflow.
Any help is greatly appreciated!
Brian

You data structure would be in terms of Classes. You need a class Anevent{
String eventName;
Date startTime;
Date endTime;
In your main program have a Vector object which can store unllimited objects and which would not overflow. Store your event objects as they happen in this Vector. After your time interval of events is completed and you have the final Vector, you can pass the vector object to various methods in your main program to be simple or pass it to specailized graph drawing classes you would have to write to draw various Histograms.

Data structure for web-based discussion forum

Hello all. I have begun work on a discussion board forum but have not yet decided upon a data structure for the hierarchy of messages to be left on a board. Any one have any thoughts or experience in this matter? My first thought was to implement a basic linked list in which every reply to a message would be a new child of a node. This would require I use recursion in some areas for counting and finding messages but I was also thinking that this way I can store each thread object in a simple fashion using serialization.
SJ

THE STANDARS DIRECTORY STRUCTURE IS :-
PROJECT FOLDER
|
|---------->WEB-INF
|------> web.xml file
|-------> classes (folder to keep java files)
|-------> lib (folder to keep jar files)

How to access data structures in C dll from java thru JNI?

We have been given API's( collection of C Functions) from some vendor.
SDK from vendor consist of:
Libpga.DLL, Libpga.h,Libpga.lib, Along with that sample program Receiver.h (i don't know its written in C or C++), I guess .C stnads for C files?
Considering that I don't know C or C++ (Except that I can understand what that program is doing) & i have experience in VB6 and Java, In order to build interface based on this API, I have two option left, Use these dll either from VB or Java.
As far as I know, calling this DLL in VB requires all the data structures & methods to be declared in VB, I guess which is not the case with Java (? I'm not sure)
I experiemnted calling these function from Java through JNI, and I successfully did by writting wrapper dll. My question is whether I have to declare all the constants & data structures defined in libpga.h file in java, in order to use them in my java program??
Any suggesstion would be greatly appreciated,
Vini

1. There are generators around that claim to generate suitable wrappers, given some dll input. I suggest you search google. Try JACE, jni, wrapper, generator, .... Also, serach back through this forum, where there have been suggestions made.
2. In general, you will need to supply wrappers, and if you want to use data from the "C side" in java, then you will need java objects that hold the data.

Looking for an efficient data structur & search algorithm

Hi all
i have a list of digits (international phone network prefixes) with some hundreds to some thousends entries. An entry may be in the form
^00[1-9]{1}[0-9]{0,7}$
I.e. this might be 001, 0041, 00317545, 00317548, 00317549 and so on. Regarding the last three examples, it might even be that there is an additonal 0031754.
What i need a a data structur that allows to match these prefixes against a phonenumber.
I.e. if i have the phonennumber 001123456789 it would match the prefix 001. If i had 00317549111 it would match 00317549.
The easiest way would be to but all prefixes into an Vector or similar and loop over all entries, trying to match the phonenumber with startsWith(). But this wouldn't always result in a absolutely perfect match, since, i.e. for the phonenumber 00317549111 the check against the prefix 0031754 would return a match even if there was a more specific match with the prefix 00317549. But more than that, this simple algorithm is not very efficient.
So i am looking for a more efficient way/pattern to do this. I thought about a kind of tree structure, starting with 00 in the top level, than provding [1-9] in the second level, and [0-9] from third level on. Then on every node it would either store if there is a matching prefix on that level, or if there is a prefix starting with that digits on a lower level or if there is no prefix on that level or any lower.
I.e. when i have the phonenumber 00317549111 it would start at the top level with 00. That would be ok. On the next level it would check if there is a node for digit 3. If there is, it would go one level deeper and check if there is a node for digit 1. If yes, again it would go one level deeper to check if there is a node for digit 7. If that algorithm comes to a level where, for the request digit, it get's a prefix indicator rather than a node indicator, the algortihm would know, that a matching prefix was found and that there is no more specifig match on deeper levels.
One thing i forgot to mention - the prefixes might be read once during startup/init and there it might take some time for building up the datastructur - i don't care about that. But, when running, then the maching process should be as efficient as possible, that's the most important point for me.
What do you think about a pattern like this? Could this be efficient? Do you see other patterns, that might be easier to implement and that might be faster/need less memory?
Thanks a lot for your help.
Cheers, Frank

I would really have gone for your first approach. With mperemsky5's approach you have the loop with (potential) n iterations (Let n be the length of the number) and in each iteration to compute the hash-code for the string which again takes time proportional to the strings length.
The tree approach takes time equal to the length of the prefix and is imho not more complicated.
Perhaps this way:
public class DigitTree
private class Node {
     private Object content;
     private Node[] children = new Node[10];
private Node root = new Node();
public DigitTree() {
public void addPrefix(String prefix, Object value) {
     char[] numberChars = prefix.toCharArray();
     Node node = root;
     for (int i=0; i<numberChars.length; i++) {
          int number = numberChars[i] - '0';
          if (node.children[number] == null) node.children[number] = new Node();
          node = node.children[number];
     node.content = value;
public Object match(String phonenumber) {
     char[] numberChars = prefix.toCharArray();
     Node node = root;
     for (int i=0; i<numberChars.length; i++) {
          int number = numberChars[i] - '0';
          if (node.children[number] == null) return code.content;
          node = node.children[number];
     return node.content;
}The method addPrefix lets you add a prefix to the tree. The content-Object can hold a String or whatever to identify the prefix. If your data is not complete (i.e. if there are numbers for which no prefix exists) you might want to initialize the content-object of a node with a default value (e.g. "not found").
The method match lets you look up a prefix for a given number and returns the Object associated with the prefix..
The code was not tested.
Greetings
Thomas

MRS for Data structure table

I need to various data structure table for MRS

A possible solution is create a materialized view and then a check constraint on the materialized view: this should work in a multiuser environment.
Example:
SQL> drop table t;
Table supprimee.
SQL>
SQL> create table t (
2 x integer,
3 y varchar2(10)
4 );
Table creee.
SQL>
SQL> CREATE MATERIALIZED VIEW LOG on t
2 WITH ROWID (x, y)
3 including new values;
Journal de vue materialisee cree.
SQL>
SQL> CREATE MATERIALIZED VIEW t_mv
2 REFRESH FAST ON COMMIT AS
3 SELECT count(*) cnt
4 FROM t;
Vue materialisee creee.
SQL>
SQL> ALTER TABLE t_mv
2 ADD CONSTRAINT chk check(cnt<=1);
Table modifiee.
SQL>
SQL> insert into t values(1,'Ok');
1 ligne creee.
SQL> commit;
Validation effectuee.
SQL>
SQL> insert into t values(2,'KO');
1 ligne creee.
SQL> commit;
commit
ERREUR a la ligne 1 :
ORA-12008: erreur dans le chemin de regeneration de la vue materialisee
ORA-02290: violation de contraintes (TEST.CHK) de verification

N97 packet data connection on hold

Good day,I have a N97 with an unlimited data plan. But I´m having a problem, when I receive or make call or send text message my phone show this message “packet data connection on hold”, and when I finish the call or sent the text message the phone shows “Packet data connection active”. There is any way that I can change this? I want to have my data connection always active.Thanks in advance for your help
Solved!
Go to Solution.

you would have to go into the settings - connectivity- the first tab is network - select this and it tells you network mode, select this and you can set it to dual mode - umts or gms only mode. if your phone was made for your region and it supports the frequency that the network provides then it should work in 3g if it does not which is possible then it will only work on edge
Message Edited by radical24 on 14-Jul-2009 01:26 PM
You know what I love about you the most, the fact that you are not me ! In love with technology and all that it can offer. Join me in discovery....

What data structure is better?

hello all!
i would like to ask you a suggestion.. i am implementing a project in JSP+java in mvc and i have still not understood what data structure would be better to do the following:
i have "Catalogs" of entities (e.g. payments) where each one is composed by several double data. the structure of the site is that the jsp talks to its Bean which in turn talks to the controller (normal java class- no servlet) which finally ask the data to the entitiy class. How could i pass these vectors of datas to the jsp page without let it talk directly to the entity? I think that bean+jsp are MVC's boundary classes and i don't want them to talk with entities (and then they cannot directly use entity methods as bill.getAmount())..
How could i do?
Thanks a lot! Bye

I use PNG as it is smaller than TIF and holds transparencym unlike JPG.

Simple Transformation to deserialize an XML file into ABAP data structures?

I'm attempting to write my first simple transformation to deserialize
an XML file into ABAP data structures and I have a few questions.
My simple transformation contains code like the following
<tt:transform xmlns:tt="http://www.sap.com/transformation-templates"
              xmlns:pp="http://www.sap.com/abapxml/types/defined" >
<tt:type name="REPORT" line-type="?">
<tt:node name="COMPANY_ID" type="C" length="10" />
<tt:node name="JOB_ID" type="C" length="20" />
<tt:node name="TYPE_CSV" type="C" length="1" />
<tt:node name="TYPE_XLS" type="C" length="1" />
<tt:node name="TYPE_PDF" type="C" length="1" />
<tt:node name="IS_NEW" type="C" length="1" />
</tt:type>
<tt:root name="ROOT2" type="pp:REPORT" />
    <QueryResponse>
    <tt:loop ref="ROOT2" name="line">
      <QueryResponseRow>
        <CompanyID>
          <tt:value ref="$line.COMPANY_ID" />
        </CompanyID>
        <JobID>
          <tt:value ref="$line.JOB_ID" />
        </JobID>
        <ExportTypes>
          <tt:loop>
            <ExportType>
               I don't know what to do here (see item 3, below)
            </ExportType>
          </tt:loop>
        </ExportTypes>
        <IsNew>
          <tt:value ref="$line.IS_NEW"
          map="val(' ') = xml('false'), val('X') = xml('true')" />
        </IsNew>
      </QueryResponseRow>
      </tt:loop>
    </QueryResponse>
    </tt:loop>
1. In a DTD, an element can be designated as occurring zero or one
time, zero or more times, or one or more times. How do I write the
simple transformation to accommodate these possibilities?
2. In trying to accommodate the "zero or more times" case, I am trying
to use the <tt:loop> instruction. It occurs several layers deep in the
XML hierarchy, but at the top level of the ABAP table. The internal
table has a structure defined in the ABAP program, not in the data
dictionary. In the simple transformation, I used <tt:type> and
<tt:node> to define the structure of the internal table and then
tried to use <tt:loop ref="ROOT2" name="line"> around the subtree that
can occur zero or more times. But every variation I try seems to get
different errors. Can anyone supply a working example of this?
3. Among the fields in the internal table, I've defined three
one-character fields named TYPE_CSV, TYPE_XLS, and TYPE_PDF. In the
XML file, I expect zero to three elements of the form
<ExportType exporttype='csv' />
<ExportType exporttype='xls' />
<ExportType exporttype='pdf' />
I want to set field TYPE_CSV = 'X' if I find an ExportType element
with its exporttype attribute set to 'csv'. I want to set field
TYPE_XLS = 'X' if I find an ExportType element with its exporttype
attribute set to 'xls'. I want to set field TYPE_PDF = 'X' if I find
an ExportType element with its exporttype attribute set to 'pdf'. How
can I do that?
4. For an element that has a value like
<ErrorCode>123</ErrorCode>
in the simple transformation, the sequence
<ErrorCode> <tt:value ref="ROOT1.CODE" /> </ErrorCode>
seems to work just fine.
I have other situations where the XML reads
<IsNew value='true' />
I wanted to write
<IsNew>
        <tt:value ref="$line.IS_NEW"
        map="val(' ') = xml('false'), val('X') = xml('true')" />
       </IsNew>
but I'm afraid that the <tt:value> fails to deal with the fact that in
the XML file the value is being passed as the value of an attribute
(named "value"), rather than the value of the element itself. How do
you handle this?

Try this code below:
data l_xml_table2 type table of xml_line with header line.
W_filename - This is a Path.
if w_filename(02) = '
    open dataset w_filename for output in binary mode.
    if sy-subrc = 0.
      l_xml_table2[] = l_xml_table[].
      loop at l_xml_table2.
        transfer l_xml_table2 to w_filename.
      endloop.
    endif.
    close dataset w_filename.
else.
    call method cl_gui_frontend_services=>gui_download
      exporting
        bin_filesize = l_xml_size
        filename     = w_filename
        filetype     = 'BIN'
      changing
        data_tab     = l_xml_table
      exceptions
        others       = 24.
    if sy-subrc <> 0.
      message id sy-msgid type sy-msgty number sy-msgno
                 with sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
    endif.

OC4J: marshalling does not recreate the same data structure onthe client

Hi guys,
I am trying to use OC4J as an EJB container and have come across the following problem, which looks like a bug.
I have a value object method that returns an instance of ArrayList with references to other value objects of the same class. The value objects have references to other value objects. When this structure is marshalled across the network, we expect it to be recreated as is but that does not happen and instead objects get duplicated.
Suppose we have 2 value objects: ValueObject1 and ValueObject2. ValueObject1 references ValueObject2 via its private field and the ValueObject2 references ValueObject1. Both value objects are returned by our method in an ArrayList structure. Here is how it will look like (number after @ represents an address in memory):
Object[0] = com.cramer.test.SomeVO@1
Object[0].getValueObject[0] = com.cramer.test.SomeVO@2
Object[1] = com.cramer.test.SomeVO@2
Object[1].getValueObject[0] = com.cramer.test.SomeVO@1
We would expect to see the same (except exact addresses) after marshalling. Here is what we get instead:
Object[0] = com.cramer.test.SomeVO@1
Object[0].getValueObject[0] = com.cramer.test.SomeVO@2
Object[1] = com.cramer.test.SomeVO@3
Object[1].getValueObject[0] = com.cramer.test.SomeVO@4
It can be seen that objects get unnecessarily duplicated â“ the instance of the ValueObject1 referenced by the ValueObject2 is not the same now as the instance that is referenced by the ArrayList instance.
This does not only break referential integrity, structure and consistency of the data but dramatically increases the amount of information sent across the network. The problem was discovered when we found that a relatively small but complicated structure that gets serialized into a 142kb file requires about 20Mb of network communication. All this extra info is duplicated object instances.
I have created a small test case to demonstrate the problem and let you reproduce it.
Here is RMITestBean.java:
package com.cramer.test;
import javax.ejb.EJBObject;
import java.util.*;
public interface RMITestBean extends EJBObject
public ArrayList getSomeData(int testSize) throws java.rmi.RemoteException;
public byte[] getSomeDataInBytes(int testSize) throws java.rmi.RemoteException;
Here is RMITestBeanBean.java:
package com.cramer.test;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import java.util.*;
public class RMITestBeanBean implements SessionBean
private SessionContext context;
SomeVO someVO;
public void ejbCreate()
someVO = new SomeVO(0);
public void ejbActivate()
public void ejbPassivate()
public void ejbRemove()
public void setSessionContext(SessionContext ctx)
this.context = ctx;
public byte[] getSomeDataInBytes(int testSize)
ArrayList someData = getSomeData(testSize);
try {
java.io.ByteArrayOutputStream byteOutputStream = new java.io.ByteArrayOutputStream();
java.io.ObjectOutputStream objectOutputStream = new java.io.ObjectOutputStream(byteOutputStream);
objectOutputStream.writeObject(someData);
objectOutputStream.flush();
System.out.println(" serialised output size: "+byteOutputStream.size());
byte[] bytes = byteOutputStream.toByteArray();
objectOutputStream.close();
byteOutputStream.close();
return bytes;
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
return null;
public ArrayList getSomeData(int testSize)
// Create array of objects
ArrayList someData = new ArrayList();
for (int i=0; i<testSize; i++)
someData.add(new SomeVO(i));
// Interlink all the objects
for (int i=0; i<someData.size()-1; i++)
for (int j=i+1; j<someData.size(); j++)
((SomeVO)someData.get(i)).addValueObject((SomeVO)someData.get(j));
((SomeVO)someData.get(j)).addValueObject((SomeVO)someData.get(i));
// print out the data structure
System.out.println("Data:");
for (int i = 0; i<someData.size(); i++)
SomeVO tmp = (SomeVO)someData.get(i);
System.out.println("Object["+Integer.toString(i)+"] = "+tmp);
System.out.println("Object["+Integer.toString(i)+"]'s some number = "+tmp.getSomeNumber());
for (int j = 0; j<tmp.getValueObjectCount(); j++)
SomeVO tmp2 = tmp.getValueObject(j);
System.out.println(" getValueObject["+Integer.toString(j)+"] = "+tmp2);
System.out.println(" getValueObject["+Integer.toString(j)+"]'s some number = "+tmp2.getSomeNumber());
// Check the serialised size of the structure
try {
java.io.ByteArrayOutputStream byteOutputStream = new java.io.ByteArrayOutputStream();
java.io.ObjectOutputStream objectOutputStream = new java.io.ObjectOutputStream(byteOutputStream);
objectOutputStream.writeObject(someData);
objectOutputStream.flush();
System.out.println("Serialised output size: "+byteOutputStream.size());
objectOutputStream.close();
byteOutputStream.close();
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
return someData;
Here is RMITestBeanHome:
package com.cramer.test;
import javax.ejb.EJBHome;
import java.rmi.RemoteException;
import javax.ejb.CreateException;
public interface RMITestBeanHome extends EJBHome
RMITestBean create() throws RemoteException, CreateException;
Here is ejb-jar.xml:
<?xml version = '1.0' encoding = 'windows-1252'?>
<!DOCTYPE ejb-jar PUBLIC "-//Sun Microsystems, Inc.//DTD Enterprise JavaBeans 2.0//EN" "http://java.sun.com/dtd/ejb-jar_2_0.dtd">
<ejb-jar>
<enterprise-beans>
<session>
<description>Session Bean ( Stateful )</description>
<display-name>RMITestBean</display-name>
<ejb-name>RMITestBean</ejb-name>
<home>com.cramer.test.RMITestBeanHome</home>
<remote>com.cramer.test.RMITestBean</remote>
<ejb-class>com.cramer.test.RMITestBeanBean</ejb-class>
<session-type>Stateful</session-type>
<transaction-type>Container</transaction-type>
</session>
</enterprise-beans>
</ejb-jar>
And finally the application that tests the bean:
package com.cramer.test;
import java.util.*;
import javax.rmi.*;
import javax.naming.*;
public class RMITestApplication
final static boolean HARDCODE_SERIALISATION = false;
final static int TEST_SIZE = 2;
public static void main(String[] args)
Hashtable props = new Hashtable();
props.put(Context.INITIAL_CONTEXT_FACTORY, "com.evermind.server.rmi.RMIInitialContextFactory");
props.put(Context.PROVIDER_URL, "ormi://lil8m:23792/alexei");
props.put(Context.SECURITY_PRINCIPAL, "admin");
props.put(Context.SECURITY_CREDENTIALS, "admin");
try {
// Get the JNDI initial context
InitialContext ctx = new InitialContext(props);
NamingEnumeration list = ctx.list("comp/env/ejb");
// Get a reference to the Home Object which we use to create the EJB Object
Object objJNDI = ctx.lookup("comp/env/ejb/RMITestBean");
// Now cast it to an InventoryHome object
RMITestBeanHome testBeanHome = (RMITestBeanHome)PortableRemoteObject.narrow(objJNDI,RMITestBeanHome.class);
// Create the Inventory remote interface
RMITestBean testBean = testBeanHome.create();
ArrayList someData = null;
if (!HARDCODE_SERIALISATION)
// ############################### Alternative 1 ##############################
// ## This relies on marshalling serialisation ##
someData = testBean.getSomeData(TEST_SIZE);
// ############################ End of Alternative 1 ##########################
} else
// ############################### Alternative 2 ##############################
// ## This gets a serialised byte stream and de-serialises it ##
byte[] bytes = testBean.getSomeDataInBytes(TEST_SIZE);
try {
java.io.ByteArrayInputStream byteInputStream = new java.io.ByteArrayInputStream(bytes);
java.io.ObjectInputStream objectInputStream = new java.io.ObjectInputStream(byteInputStream);
someData = (ArrayList)objectInputStream.readObject();
objectInputStream.close();
byteInputStream.close();
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
// ############################ End of Alternative 2 ##########################
// Print out the data structure
System.out.println("Data:");
for (int i = 0; i<someData.size(); i++)
SomeVO tmp = (SomeVO)someData.get(i);
System.out.println("Object["+Integer.toString(i)+"] = "+tmp);
System.out.println("Object["+Integer.toString(i)+"]'s some number = "+tmp.getSomeNumber());
for (int j = 0; j<tmp.getValueObjectCount(); j++)
SomeVO tmp2 = tmp.getValueObject(j);
System.out.println(" getValueObject["+Integer.toString(j)+"] = "+tmp2);
System.out.println(" getValueObject["+Integer.toString(j)+"]'s some number = "+tmp2.getSomeNumber());
// Print out the size of the serialised structure
try {
java.io.ByteArrayOutputStream byteOutputStream = new java.io.ByteArrayOutputStream();
java.io.ObjectOutputStream objectOutputStream = new java.io.ObjectOutputStream(byteOutputStream);
objectOutputStream.writeObject(someData);
objectOutputStream.flush();
System.out.println("Serialised output size: "+byteOutputStream.size());
objectOutputStream.close();
byteOutputStream.close();
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
catch(Exception ex){
ex.printStackTrace(System.out);
The parameters you might be interested in playing with are HARDCODE_SERIALISATION and TEST_SIZE defined at the beginning of RMITestApplication.java. The HARDCODE_SERIALISATION is a flag that specifies whether Java serialisation should be used to pass the data across or we should rely on OC4J marshalling. TEST_SIZE defines the size of the object graph and the ArrayList structure. The bigger this size is the more dramatic effect you get from data duplication.
The test case outputs the structure both on the server and on the client and prints out the size of the serialised structure. That gives us sufficient comparison, as both structure and its size should be the same on the client and on the server.
The test case also demonstrates that the problem is specific to OC4J. The standard Java serialisation does not suffer the same flaw. However using the standard serialisation the way I did in the test case code is generally unacceptable as it breaks the transparency benefit and complicates interfaces.
To run the test case:
1) Modify provider URL parameter value on line 15 of the RMITestApplication.java for your environment.
2) Deploy the bean to the server.
4) Run RMITestApplication on a client PC.
5) Compare the outputs on the server and on the client.
I hope someone can reproduce the problem and give their opinion, and possibly point to the solution if there is one at the moment.
Cheers,
Alexei

Hi,
Eugene, wrong end user recovery. Alexey is referring to client desktop end user recovery which is entirely different.
Alexy - As noted in the previous post:
http://social.technet.microsoft.com/Forums/en-US/bc67c597-4379-4a8d-a5e0-cd4b26c85d91/dpm-2012-still-requires-put-end-users-into-local-admin-groups-for-the-purpose-of-end-user-data?forum=dataprotectionmanager
Each recovery point has users permisions tied to it, so it's not possible to retroacively give the users permissions. Implement the below and going forward all users can restore their own files.
This is a hands off solution to allow all users that use a machine to be able to restore their own files.
1) Make these two cmd files and save them in c:\temp
2) Using windows scheduler – schedule addperms.cmd to run daily – any new users that log onto the machine will automatically be able to restore their own files.
<addperms.cmd>
Cmd.exe /v /c c:\temp\addreg.cmd
<addreg.cmd>
set users=
echo Windows Registry Editor Version 5.00>c:\temp\perms.reg
echo [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Data Protection Manager\Agent\ClientProtection]>>c:\temp\perms.reg
FOR /F "Tokens=*" %%n IN ('dir c:\users\*. /b') do set users=!users!%Userdomain%\\%%n,
echo "ClientOwners"=^"%users%%Userdomain%\\bogususer^">>c:\temp\perms.reg
REG IMPORT c:\temp\perms.reg
Del c:\temp\perms.reg
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT]
This posting is provided "AS IS" with no warranties, and confers no rights.

Error saving data structure CE11000 (please read log) message number KX 655

while activating the data structure in the operating concern of CO PA sap gives the following errors.
1.Error saving data structure CE11000 (please read log)
Message no. KX655
2.Error saving table CE01000
Message no. KX593
3.in Log Reference field CE31000-REC_WAERS for CE31000-VVQ10001 has incorrect type.
Pls suggest

Hey,
Below tables are related to application logs
BAL_AMODAL :                   Application Log: INDX table for amodal communication
BALC        :                   Application Log: Log or message context
BALDAT      :                   Application Log: Log data
BALHANDLE   :                   Application Log: Lock object dummy table
BALHDR      :                   Application log: log header
BALHDRP     :                   Application log: log parameter
BAL_INDX    :                   Application Log: INDX tables
BALM        :                   Application log: log message
BALMP       :                   Application log: message parameter
BALOBJ      :                   Application log: objects
BALOBJT     :                   Application log: object texts
BALSUB      :                   Application log: sub-objects
BALSUBT     :                   Application log: Sub-object texts
-Kiran
*Please mark useful answers

Data structure to hold frequencies or counts

Similar Messages

Maybe you are looking for