Which data structure?

Hi there...
Could you let me know what is the most efficient data structure to use (in terms of storage and access) if I have to store around 1 million <key,value> pairs?
Thanx :)
Tariq

Yes, concurrency, when two or more threads would act upon the same data or data structure. Concurrent access, etc. Your question has nothing to do with concurrency, but rather Collections. But yes, HashMap or Hashtable depending on if one thread is going to be accessing the same data or multiple threads. Also a database might come in handy as well, such as MySQL or JavaDB (Apache Derby).

Similar Messages

Which data structure to use in this scenario? queue, stack, list, ...

Hi,
I'm hesitant about what data structure to use (best efficient one) in the following scenario
At a given moment I have an ordered set of data such as:
(1,3,6,9,10)
and a current value (by instance, 11)
Then I need to access the LAST one (10) and
If my value is greater I will add it to the list. In this example I will add 11 so the list will be (1,3,6,9,10,11)
On the other hand, as a second step, I will iterate on the list as follows: I will get the first element, do something, then remove it, and go to next one until the list is empty. Therefore, basically I will access always the first element on the list.
I don't really care about the elements in the middle.
Which data structure do you suggest me to use?
I was thinking on Queue (easy to access the head, and extract one by one in order), but I don't know how to get the tail element (10 in this case), as it is the one that tells me if I have to add another element...
Thank you.

>
At a given moment I have an ordered set of data such as:
(1,3,6,9,10)
and a current value (by instance, 11)
Then I need to access the LAST one (10) and
If my value is greater I will add it to the list. In this example I will add 11 so the list will be (1,3,6,9,10,11)Any data structure will do, I would suggest a LinkedList.
On the other hand, as a second step, I will iterate on the list as follows: I will get the first element, do something, then remove it, and go to next one until the list is empty. Therefore, basically I will access always the first element on the list.Use an iterator and the corresponding remove methods.
Mel

Data structure for simulation of message queue

Hello,
I have undertaken a project of simulating the point to point and publish/subscribe protocols of message queueing. This whole project would be done just in Java. There won't be any system level programming. Which data structure in Java would be the most efficient one for the message queue?

Hello,
I have undertaken a project of simulating the point to point and publish/subscribe protocols of message queueing. This whole project would be done just in Java. There won't be any system level programming. Which data structure in Java would be the most efficient one for the message queue?

How to store three values in a suitable data structure?

Hi Friends!
Usually, if we want to store a single value we choose vector/set data structure.
And for two values we prefer map<int, vector<int>> or multimap<int, vector<int>>.
If I want to store three values like;
1=>3=>2 4 5
1=>5=>7
2=>3=>10 12
Then, which data structure is suitable for it?
And how to insert entry into it?
Could anyone help me?

You could try
     typedef vector<int> INTVEC;
     typedef map<int, INTVEC> INTMAP;
     typedef multimap<int, INTMAP> INTMULT;
You should convince yourself that the above is equivalent to (but hopefully easier to understand)
     typedef multimap<int, map<int, vector<int>>> INTMULT;
When you try to store values in an object of this type, remember to build it from the inside out:
     Build an INTVEC with the values you want (such as 2 4 5 per your first example).
     Assign that INVEC to an INTMAP with a suitable key (3 in your example).
     Insert that INTMAP into an INTMULT with its appropriate key (1 per your example).
     Repeat these three steps using 7, 5, 1, respectively.
     Repeat again using 10 12, 3, 2, respectively.
Alternately, you could define a structure like
     struct INTSTRUCT{
          int key;
          vector<int> vec;};
and then define your multimap as <int, INSTSTRUC>.   This allows you to assign values to both members of a structure object directly and then insert the structure into the multimap with the appropriate key.
On the other hand, if you change your concept by removing the second => and always put the "second" int as the first element of the vector, as in
     1 => 3 2 4 5
     1 => 5 7
     2 => 3 10 12
you could simplify things to a multimap<int, vector<int>> you are already familiar with.
A slightly different concept change to
     1 => 3 => 2 4 5
              5 => 7
     2 => 3 => 10
would allow you to use map<int, map<int, vector<int>>> where element [1] of this object contains a map with two elements and element [2] contains a map with a single element.
Or you could tell us how you intend to use the data and maybe somebody can suggest a more practical approach.
Yes, I could understand the way you explained as
typedef multimap<int, map<int, vector<int>>> INTMULT;
and I used typedef
multimap<int, map<int, set<int>>> INTMULT;
but, when I used it, I get,
1=>3=>2
1=>3=>4
1=>3=>5
1=>5=>7
2=>3=>10
2=>3=>12

Fastest Iterating data structure

Forgetting about the semantics about list and set, purely from iteration point of view, which data structure class would give fastest iteration?

That question is meaningless.
The only way to get an answer is to test it for yourself. And then your answer will only apply to the particular version, JVM implementer, and OS an which you run your test. And the difference between the best and the worst will be completely negligible in all but the most bizarre corner cases.
So if you're asking just for curiosity's sake, your question has been completely answered, and you are now satisfied.
If you have an actual use case where this matters, you'll need to provide details about your use case and clarify exactly what you mean by "fastest iterating".
Also note that, even if List were to iterate 1000 times faster than Set (whatever that means), if you need what Set offers and List doesn't--no duplicates--then you'll want to use Set anyway. That is, your "forget about semantics" comment is a bogus requirement.

Spreadsheet data structure

Hi,anybody knows which data structure(s) is/are/should be used while making spreadsheet type application?Array seems to be impossible.For example lets assume we want to make simple spreadsheet application(say 1000x1000 size).User enters some integer values in cells A1,B1,in some cell I write A1+B1,and output in cell A1+B1 should be addition of integers in A1 and B1.How to represent and work with this 1000x1000 matrix?
Thanks
pathreading

How about a HashMap with the position as the key and data as value? The rules (C1 = A1 + B1) can be represented in a different form.

Best data structure for tablemodel

Hi everybody
I'm building an application that has a JTable which gets its data from a resultSet. I'd like to know which data structure is the best for this situation. I've seen some examples that passes the data from the resultset to a Vector or an ArrayList, but if you have a scroollable resultset, why will I instantiate another data structure?
Thanks very much
Mauricio

Hi everybody
I'm building an application that has a JTable which gets its data
from a resultSet. I'd like to know which data structure is the best
for this situation. I've seen some examples that passes the data from
the resultset to a Vector or an ArrayList, but if you have a
scroollable resultset, why will I instantiate another data structure?
Thanks very much
Mauricio I've used TableModels that get their data directly form the ResultSet and had some success. But you are at the mercy of the implementation of the ResultSet. If it is slow your table model will be slow too. I would suggest that you use an array of Object arrays, i.e. Object[][], as your data stucture and copy your result set into it.

Which is a efficient Data Structure to store data in a file

I want tostire a data in file in sorted order. I am thinking about Bainary Search tree & heap sort Which is efficient technique. Please reply..

which is the best data structure to be used in Thread
pool to store the thread instances? and why ?
I have seen lots of people using Queue....Why do you ask? You do know that there are thread pool classes in Java 5?
Kaj

OC4J: marshalling does not recreate the same data structure onthe client

Hi guys,
I am trying to use OC4J as an EJB container and have come across the following problem, which looks like a bug.
I have a value object method that returns an instance of ArrayList with references to other value objects of the same class. The value objects have references to other value objects. When this structure is marshalled across the network, we expect it to be recreated as is but that does not happen and instead objects get duplicated.
Suppose we have 2 value objects: ValueObject1 and ValueObject2. ValueObject1 references ValueObject2 via its private field and the ValueObject2 references ValueObject1. Both value objects are returned by our method in an ArrayList structure. Here is how it will look like (number after @ represents an address in memory):
Object[0] = com.cramer.test.SomeVO@1
Object[0].getValueObject[0] = com.cramer.test.SomeVO@2
Object[1] = com.cramer.test.SomeVO@2
Object[1].getValueObject[0] = com.cramer.test.SomeVO@1
We would expect to see the same (except exact addresses) after marshalling. Here is what we get instead:
Object[0] = com.cramer.test.SomeVO@1
Object[0].getValueObject[0] = com.cramer.test.SomeVO@2
Object[1] = com.cramer.test.SomeVO@3
Object[1].getValueObject[0] = com.cramer.test.SomeVO@4
It can be seen that objects get unnecessarily duplicated â“ the instance of the ValueObject1 referenced by the ValueObject2 is not the same now as the instance that is referenced by the ArrayList instance.
This does not only break referential integrity, structure and consistency of the data but dramatically increases the amount of information sent across the network. The problem was discovered when we found that a relatively small but complicated structure that gets serialized into a 142kb file requires about 20Mb of network communication. All this extra info is duplicated object instances.
I have created a small test case to demonstrate the problem and let you reproduce it.
Here is RMITestBean.java:
package com.cramer.test;
import javax.ejb.EJBObject;
import java.util.*;
public interface RMITestBean extends EJBObject
public ArrayList getSomeData(int testSize) throws java.rmi.RemoteException;
public byte[] getSomeDataInBytes(int testSize) throws java.rmi.RemoteException;
Here is RMITestBeanBean.java:
package com.cramer.test;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import java.util.*;
public class RMITestBeanBean implements SessionBean
private SessionContext context;
SomeVO someVO;
public void ejbCreate()
someVO = new SomeVO(0);
public void ejbActivate()
public void ejbPassivate()
public void ejbRemove()
public void setSessionContext(SessionContext ctx)
this.context = ctx;
public byte[] getSomeDataInBytes(int testSize)
ArrayList someData = getSomeData(testSize);
try {
java.io.ByteArrayOutputStream byteOutputStream = new java.io.ByteArrayOutputStream();
java.io.ObjectOutputStream objectOutputStream = new java.io.ObjectOutputStream(byteOutputStream);
objectOutputStream.writeObject(someData);
objectOutputStream.flush();
System.out.println(" serialised output size: "+byteOutputStream.size());
byte[] bytes = byteOutputStream.toByteArray();
objectOutputStream.close();
byteOutputStream.close();
return bytes;
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
return null;
public ArrayList getSomeData(int testSize)
// Create array of objects
ArrayList someData = new ArrayList();
for (int i=0; i<testSize; i++)
someData.add(new SomeVO(i));
// Interlink all the objects
for (int i=0; i<someData.size()-1; i++)
for (int j=i+1; j<someData.size(); j++)
((SomeVO)someData.get(i)).addValueObject((SomeVO)someData.get(j));
((SomeVO)someData.get(j)).addValueObject((SomeVO)someData.get(i));
// print out the data structure
System.out.println("Data:");
for (int i = 0; i<someData.size(); i++)
SomeVO tmp = (SomeVO)someData.get(i);
System.out.println("Object["+Integer.toString(i)+"] = "+tmp);
System.out.println("Object["+Integer.toString(i)+"]'s some number = "+tmp.getSomeNumber());
for (int j = 0; j<tmp.getValueObjectCount(); j++)
SomeVO tmp2 = tmp.getValueObject(j);
System.out.println(" getValueObject["+Integer.toString(j)+"] = "+tmp2);
System.out.println(" getValueObject["+Integer.toString(j)+"]'s some number = "+tmp2.getSomeNumber());
// Check the serialised size of the structure
try {
java.io.ByteArrayOutputStream byteOutputStream = new java.io.ByteArrayOutputStream();
java.io.ObjectOutputStream objectOutputStream = new java.io.ObjectOutputStream(byteOutputStream);
objectOutputStream.writeObject(someData);
objectOutputStream.flush();
System.out.println("Serialised output size: "+byteOutputStream.size());
objectOutputStream.close();
byteOutputStream.close();
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
return someData;
Here is RMITestBeanHome:
package com.cramer.test;
import javax.ejb.EJBHome;
import java.rmi.RemoteException;
import javax.ejb.CreateException;
public interface RMITestBeanHome extends EJBHome
RMITestBean create() throws RemoteException, CreateException;
Here is ejb-jar.xml:
<?xml version = '1.0' encoding = 'windows-1252'?>
<!DOCTYPE ejb-jar PUBLIC "-//Sun Microsystems, Inc.//DTD Enterprise JavaBeans 2.0//EN" "http://java.sun.com/dtd/ejb-jar_2_0.dtd">
<ejb-jar>
<enterprise-beans>
<session>
<description>Session Bean ( Stateful )</description>
<display-name>RMITestBean</display-name>
<ejb-name>RMITestBean</ejb-name>
<home>com.cramer.test.RMITestBeanHome</home>
<remote>com.cramer.test.RMITestBean</remote>
<ejb-class>com.cramer.test.RMITestBeanBean</ejb-class>
<session-type>Stateful</session-type>
<transaction-type>Container</transaction-type>
</session>
</enterprise-beans>
</ejb-jar>
And finally the application that tests the bean:
package com.cramer.test;
import java.util.*;
import javax.rmi.*;
import javax.naming.*;
public class RMITestApplication
final static boolean HARDCODE_SERIALISATION = false;
final static int TEST_SIZE = 2;
public static void main(String[] args)
Hashtable props = new Hashtable();
props.put(Context.INITIAL_CONTEXT_FACTORY, "com.evermind.server.rmi.RMIInitialContextFactory");
props.put(Context.PROVIDER_URL, "ormi://lil8m:23792/alexei");
props.put(Context.SECURITY_PRINCIPAL, "admin");
props.put(Context.SECURITY_CREDENTIALS, "admin");
try {
// Get the JNDI initial context
InitialContext ctx = new InitialContext(props);
NamingEnumeration list = ctx.list("comp/env/ejb");
// Get a reference to the Home Object which we use to create the EJB Object
Object objJNDI = ctx.lookup("comp/env/ejb/RMITestBean");
// Now cast it to an InventoryHome object
RMITestBeanHome testBeanHome = (RMITestBeanHome)PortableRemoteObject.narrow(objJNDI,RMITestBeanHome.class);
// Create the Inventory remote interface
RMITestBean testBean = testBeanHome.create();
ArrayList someData = null;
if (!HARDCODE_SERIALISATION)
// ############################### Alternative 1 ##############################
// ## This relies on marshalling serialisation ##
someData = testBean.getSomeData(TEST_SIZE);
// ############################ End of Alternative 1 ##########################
} else
// ############################### Alternative 2 ##############################
// ## This gets a serialised byte stream and de-serialises it ##
byte[] bytes = testBean.getSomeDataInBytes(TEST_SIZE);
try {
java.io.ByteArrayInputStream byteInputStream = new java.io.ByteArrayInputStream(bytes);
java.io.ObjectInputStream objectInputStream = new java.io.ObjectInputStream(byteInputStream);
someData = (ArrayList)objectInputStream.readObject();
objectInputStream.close();
byteInputStream.close();
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
// ############################ End of Alternative 2 ##########################
// Print out the data structure
System.out.println("Data:");
for (int i = 0; i<someData.size(); i++)
SomeVO tmp = (SomeVO)someData.get(i);
System.out.println("Object["+Integer.toString(i)+"] = "+tmp);
System.out.println("Object["+Integer.toString(i)+"]'s some number = "+tmp.getSomeNumber());
for (int j = 0; j<tmp.getValueObjectCount(); j++)
SomeVO tmp2 = tmp.getValueObject(j);
System.out.println(" getValueObject["+Integer.toString(j)+"] = "+tmp2);
System.out.println(" getValueObject["+Integer.toString(j)+"]'s some number = "+tmp2.getSomeNumber());
// Print out the size of the serialised structure
try {
java.io.ByteArrayOutputStream byteOutputStream = new java.io.ByteArrayOutputStream();
java.io.ObjectOutputStream objectOutputStream = new java.io.ObjectOutputStream(byteOutputStream);
objectOutputStream.writeObject(someData);
objectOutputStream.flush();
System.out.println("Serialised output size: "+byteOutputStream.size());
objectOutputStream.close();
byteOutputStream.close();
} catch (Exception e) {
System.out.println("Serialisation failed: "+e.getMessage());
catch(Exception ex){
ex.printStackTrace(System.out);
The parameters you might be interested in playing with are HARDCODE_SERIALISATION and TEST_SIZE defined at the beginning of RMITestApplication.java. The HARDCODE_SERIALISATION is a flag that specifies whether Java serialisation should be used to pass the data across or we should rely on OC4J marshalling. TEST_SIZE defines the size of the object graph and the ArrayList structure. The bigger this size is the more dramatic effect you get from data duplication.
The test case outputs the structure both on the server and on the client and prints out the size of the serialised structure. That gives us sufficient comparison, as both structure and its size should be the same on the client and on the server.
The test case also demonstrates that the problem is specific to OC4J. The standard Java serialisation does not suffer the same flaw. However using the standard serialisation the way I did in the test case code is generally unacceptable as it breaks the transparency benefit and complicates interfaces.
To run the test case:
1) Modify provider URL parameter value on line 15 of the RMITestApplication.java for your environment.
2) Deploy the bean to the server.
4) Run RMITestApplication on a client PC.
5) Compare the outputs on the server and on the client.
I hope someone can reproduce the problem and give their opinion, and possibly point to the solution if there is one at the moment.
Cheers,
Alexei

Hi,
Eugene, wrong end user recovery. Alexey is referring to client desktop end user recovery which is entirely different.
Alexy - As noted in the previous post:
http://social.technet.microsoft.com/Forums/en-US/bc67c597-4379-4a8d-a5e0-cd4b26c85d91/dpm-2012-still-requires-put-end-users-into-local-admin-groups-for-the-purpose-of-end-user-data?forum=dataprotectionmanager
Each recovery point has users permisions tied to it, so it's not possible to retroacively give the users permissions. Implement the below and going forward all users can restore their own files.
This is a hands off solution to allow all users that use a machine to be able to restore their own files.
1) Make these two cmd files and save them in c:\temp
2) Using windows scheduler – schedule addperms.cmd to run daily – any new users that log onto the machine will automatically be able to restore their own files.
<addperms.cmd>
Cmd.exe /v /c c:\temp\addreg.cmd
<addreg.cmd>
set users=
echo Windows Registry Editor Version 5.00>c:\temp\perms.reg
echo [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Data Protection Manager\Agent\ClientProtection]>>c:\temp\perms.reg
FOR /F "Tokens=*" %%n IN ('dir c:\users\*. /b') do set users=!users!%Userdomain%\\%%n,
echo "ClientOwners"=^"%users%%Userdomain%\\bogususer^">>c:\temp\perms.reg
REG IMPORT c:\temp\perms.reg
Del c:\temp\perms.reg
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT]
This posting is provided "AS IS" with no warranties, and confers no rights.

Xcode, how do you take an app from an idea to a data model, data structure, or class structure?

Hey everyone!
I'm a beginner xcoder and computer engineering sophomore and have literally spent every hour of the past few weeks reading as much as I possibly can about becoming a developer.
I've found plenty of documentation, and guides on how to do very specific things in xcode and objective-C but am still looking for more information on one topic; how do you decide which data models/structures/controllers etc to use?
I mean, you personally. I've been looking for a resource on this and have not been able to find one. I just finished Apple's iOS Developers Guide and they mention "choosing a data model" but do not describe them in much detail.
The following is what is provided in the guide:
● Choose a basic approach for your data model:
● Existing data model code—If you already have data model code written in a C-based language, you
can integrate that code directly into your iOS apps. Because iOS apps are written in Objective-C, they
work just fine with code written in other C-based languages. Of course, there is also benefit to writing
an Objective-C wrapper for any non Objective-C code.
● Custom objects data model—A custom object typically combines some simple data (strings, numbers,
dates, URLs, and so on) with the business logic needed to manage that data and ensure its consistency.
Custom objects can store a combination of scalar values and pointers to other objects. For example,
the Foundation framework defines classes for many simple data types and for storing collections of
other objects. These classes make it much easier to define your own custom objects.
● Structured data model—If your data is highly structured—that is, it lends itself to storage in a
database—use Core Data (or SQLite) to store the data. Core Data provides a simple object-oriented
model for managing your structured data. It also provides built-in support for some advanced features
like undo and iCloud. (SQLite files cannot be used in conjunction with iCloud.)
● Decide whether you need support for documents:
The job of a document is to manage your app’s in-memory data model objects and coordinate the storage
of that data in a corresponding file (or set of files) on disk. Documents normally connote files that the user
created but apps can use documents to manage non user facing files too. One big advantage of using
documents is that the UIDocument class makes interacting with iCloud and the local file system much
simpler. For apps that use Core Data to store their content, the UIManagedDocument class provides similar
support.
I suppose my question boils down to, how do you decide which structures to use? If you can provide an example of an app idea and how its implemented that would be very helpful and much appreciated!
For example, to implement the idea of an app which allows users to progress through levels of knowledge of a certain subject and rewarding them with badges and such (this is not an actual app just a whim) how would one model that?
Thanks in advance for all your help!!!

SgtChevelle wrote:
how do you decide which structures to use?
Trial and error.
I wish I had a better answer for you, but that pretty much encapsulates it. There is some, but not much, good wisdom out there, but it takes a significant amount of experience to be able to recognize it. The software development community if currently afflicted with a case of copy-and-paste-itis. And the prognosis is poor.
The solution is to be brutal to yourself and others. Focus on what you need and ignore everything else. Remember that other people have their own needs and methods and they might not be applicable to you. Apple, for example, can hire thousands of programmers, set them to coding for six months, pick the best results, and have the end-users spend their own time and monety to test it. If you don't have Apple's resources and power, think twice about adopting Apple's approach. And I am talking from a macro to a micro perspective. Apple's sample and boilerplate code is just junk. Don't assume you can't do better. You can.
Unfortunately, all this takes time and practice. You can read popular books, but never assume that anyone knows more than you do. Maybe they do and maybe they don't. It takes time to figure that out. Just do your best, ignore the naysayers, and doubt other people even more than you doubt yourself.

Can I automate the creation of a cluster in LabView using the data structure created in an autogenerated .CSV, C header, or XML file?

Can I automate the creation of a cluster in LabView using the data structure created in an auto generated .CSV, C header, or XML file? I'm trying to take the data structure defined in one or more of those files listed and have LabView automatically create a cluster with identical structure and data types. (Ideally, I would like to do this with a C header file only.) Basically, I'm trying to avoid having to create the cluster by hand, as the number of cluster elements could be very large. I've looked into EasyXML and contacted the rep for the add-on. Unfortunately, this capability has not been created yet. Has anyone done something like this before? Thanks in advance for the help.
Message Edited by PhilipJoeP on 04-29-2009 04:54 PM
Solved!
Go to Solution.

smercurio_fc wrote:
Is this something you're trying to do at runtime? Clusters are fixed data structures so you can't change them programmatically. Or, are you just trying to create some typedef cluster controls so that you can use them for coding? What would your clusters basically look like? Perhaps another way of holding the information like an array of variants?
You can try LabVIEW scripting, though be aware that this is not supported by NI.
Wow! Thanks for the quick response! We would use this cluster as a fixed data structure. No need to change the structure during runtime. The cluster would be a cluster of clusters with multiple levels. There would be not pattern as to how deep these levels would go, or how many elements would be in each. Here is the application. I would like to be able to autocode a Simulink model file into a DLL. The model DLL would accept a Simulink bus object of a certain data structure (bus of buses), pick out which elements of the bus is needed for the model calculation, and then pass the bus object. I then will take the DLL file and use the DLL VI block to pass a cluster into the DLL block (with identical structure as the bus in Simulink). To save time, I would like to auto generate the C header file using Simulink to define the bus structure and then have LabView read that header file and create the cluster automatically. Right now I can do everything but the auto creation of the cluster. I can manually build the cluster to match the Simulink model bus structure and it runs fine. But this is only for an example model with a small structure. Need to make the cluster creation automated so it can handle large structures with minimal brute force. Thanks!

What is the best data structure for loading an enterprise Power BI site?

Hi folks, I'd sure appreciate some help here!
I'm a kinda old-fashioned gal and a bit of a traditionalist, building enterprise data warehouses out of Analysis Service hypercubes with a whole raft of MDX for analytics. Those puppies would sit up and beg when you asked them to deliver up goodies
to SSRS or PowerView.
But Power BI is a whole new game for me.
Should I be exposing each dimension and fact table in the relational data warehouse as a single Odata feed?
Should I be running Data Management Gateway and exposing each table in my RDW individually?
Should I be flattening my stars and snowflakes and creating a very wide First Normal Form dataset with everything relating to each fact?
I guess my real question, folks, is what's the optimum way of exposing data to the Power BI cloud?
And my subsidiary question is this: am I right in saying that all the data management, validation, cleansing, and regular ETTL processes are still required
before the data is suitable to expose to Power BI?
Or, to put it another way, is it not the case that you need to have a clean and properly structured data warehouse
before the data is ready to be massaged and presented by Power BI?
I'd sure value your thoughts and opinions,
Cheers, Donna
Donna Kelly

Dear All,
My original question was:
what's the optimum way of exposing data to the Power BI cloud?
Having spent the last month faffing about with Power BI – and reading about many people’s experiences using it – I think I can offer a few preliminary conclusions.
Before I do that, though, let me summarise a few points:
Melissa said “My initial thoughts: I would expose each dim & fact as a separate OData feed” and went on to say “one of the hardest things . . . is
the data modeling piece . . . I think we should try to expose the data in a way that'll help usability . . . which wouldn't be a wide, flat table ”.
Greg said “data modeling is not a good thing to expose end users to . . . we've had better luck with is building out the data model, and teaching the users
how to combine pre-built elements”
I had commented “. . . end users and data modelling don't mix . . . self-service so
far has been mostly a bust”.
Here at Redwing, we give out a short White Paper on Business Intelligence Reporting. It goes to clients and anyone else who wants one. The heart
of the Paper is the Reporting Pyramid, which states: Business intelligence is all about the creation and delivery of actionable intelligence to the right audience at the right time
For most of the audience, that means Corporate BI: pre-built reports delivered on a schedule.
For most of the remaining audience, that means parameterised, drillable, and sliceable reporting available via the web, running the gamut from the dashboard to the details, available on
demand.
For the relatively few business analysts, that means the ability for business users to create their own semi-customised visual reports when required, to serve
their audiences.
For the very few high-power users, that means the ability to interrogate the data warehouse directly, extract the required data, and construct data mining models, spreadsheets and other
intricate analyses as needed.
On the subject of self-service, the Redwing view says: Although many vendors want tot sell self-service reporting tools to the enterprise, the facts of the matter are these:
v
80%+ of all enterprise reporting requirement is satisfied by corporate BI . . . if it’s done right.
v Very few staff members have the time, skills, or inclination to learn and employ self-service business intelligence in the course of their activities.
I cannot just expose raw data and tell everyone to get on with it. That way lies madness!
I think that clean and well-structured data is a prerequisite for delivering business intelligence.
Assuming that data is properly integrated, historically accurate and non-volatile as well, then I've just described
a data warehouse, which is the physical expression of the dimensional model.
Therefore, exposing the presentation layer of the data warehouse is – in my opinion – the appropriate interface for self-service business intelligence.
Of course, we can choose to expose perspectives as well, which is functionally identical to building and exposing subject data marts.
That way, all calculations, KPIs, definitions, and even field names, and all consistent because they all come from the single source of the truth, and not from spreadmart hell.
So my conclusion is that exposing the presentation layer of the properly modelled data warehouse is – in general - the way to expose data for self-service.
That’s fine for the general case, but what about Power BI? Well, it’s important to distinguish between new capabilities in Excel, and the ones in Office 365.
I think that to all intents and purposes, we’re talking about exposing data through the Data Management Gateway and reading it via Power Query.
The question boils down to what data structures should go down that pipe.
According to
Create a Data Source and Enable OData Feed in Power BI Admin Center, the possibilities are tables and views. I guess I could have repeating data in there, so it could be a flattened structure of the kind Melissa doesn’t like (and neither do I).
I could expose all the dims and all the facts . . . but that would mean essentially re-building the DW in the PowerPivot DM, and that would be just plain stoopid. I mean, not a toy system, but a real one with scores of facts and maybe hundreds of dimensions?
Fact is, I cannot for the life of me see what advantages DMG/PQ
has over just telling corporate users to go directly to the Cube Perspective they want, that has already all the right calcs, KPIs, security, analytics, field names . . . and most importantly, is already modelled correctly!
If I’m a real Power User, then I can use PQ on my desktop to pull mashup data from the world, along with all my on-prem data through my exposed Cube presentation layer, and PowerPivot the
heck out of that to produce all the reporting I’d ever want. It'd be a zillion times faster reading the data directly from the Cube instead of via the DMG, as well (I think Power BI performance sucks, actually).
Of course, your enterprise might not
have a DW, just a heterogeneous mass of dirty unstructured data. If that’s the case,
choosing Power BI data structures is the least of your problems! :-)
Cheers, Donna
Donna Kelly

Urgent! Need help in deciding data structure to use

Hi all,
I need to implement a restaurant system by which a customer can make a reservation.
I was wondering whether vector would be able to be store such an object, The thing is I don't want a complex data structure. just sumthin simple cos i hardly have anytime left to my submission. sighz...
The thing is I need to able to search based on 2 properties of an object. Eg. I need to search for a reservation based on the customer name and the date he reserved a table.
But I am totally clueless how to search thru a vector based on 2 properties of the object... Would really appreciate some help. Like an example how to so so based on my program. Feelin so lost...This is all I have so far:
class AddReservation
     static BufferedReader stdin = new BufferedReader (new InputStreamReader(System.in));
     //Main Method
     public static void main (String[]args)throws IOException
          String custName, comments;
          int covers, date, startTime, endTime;
          int count = 0;
          //User can only add one reservation at a time
          do
               //Create a new reservation
               Reservation oneReservation=new Reservation();
               System.out.println("Please enter customer's name:");
               System.out.flush();
               custName = stdin.readLine();
               oneReservation.setcustName(custName);
               System.out.println("Please enter number of covers:");
               System.out.flush();
               covers = Integer.parseInt(stdin.readLine());
               oneReservation.setCovers(covers);
               System.out.println("Please enter date:");
               System.out.flush();
               date = Integer.parseInt(stdin.readLine());
               oneReservation.setDate(date);
               System.out.println("Please enter start time:");
               System.out.flush();
               startTime = Integer.parseInt(stdin.readLine());
               oneReservation.setstartTime(startTime);
               System.out.println("Please enter end time:");
               System.out.flush();
               endTime = Integer.parseInt(stdin.readLine());
               oneReservation.setendTime(endTime);
               System.out.println("Please enter comments, if any:");
               System.out.flush();
               comments = stdin.readLine();
               oneReservation.setComments(comments);
               count++;
          while (count<1);
          class Reservation
          private Reservation oneReservation;
          private String custName, comments;
          private int covers, startTime, endTime, date;
          //Default constructor
          public Reservation()
          public Reservation(String custName, int covers, int date, int startTime, int endTime, String comments)
               this.custName=custName;
               this.covers=covers;
               this.date=date;
               this.startTime=startTime;
               this.endTime=endTime;
               this.comments=comments;
          //Setter methods
          public void setcustName(String custName)
               this.custName=custName;
          public void setCovers(int covers)
               this.covers=covers;;
          public void setDate(int date)
               this.date=date;
          public void setstartTime(int startTime)
               this.startTime=startTime;
          public void setendTime(int endTime)
               this.endTime=endTime;
          public void setComments(String comments)
               this.comments=comments;
          //Getter methods
          public String getcustName()
               return custName;
          public int getCovers()
               return covers;
          public int getDate()
               return date;
          public int getstartTime()
               return startTime;
          public int getendTime()
               return endTime;
          public String getComments()
               return comments;
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
class searchBooking
     static BufferedReader stdin = new BufferedReader (new InputStreamReader(System.in));
     public static void main (String[]args)throws IOException
          int choice, date, startTime;
          String custName;
               //Search Menu
               System.out.println("Search By: ");
               System.out.println("1. Date");
               System.out.println("2. Name of Customer");
               System.out.println("3. Date & Name of Customer");
               System.out.println("4. Date & Start time of reservation");
               System.out.println("5. Date, Name of customer & Start time of reservation");
               System.out.println("Please make a selection: ");
               //User keys in choice
               System.out.flush();
               choice = Integer.parseInt(stdin.readLine());
               if (choice==1)
                    System.out.println("Please key in Date (DDMMYY):");
                    System.out.flush();
                    date = Integer.parseInt(stdin.readLine());
               else if (choice==2)
                         System.out.println("Please key in Name of Customer:");
                         System.out.flush();
                         custName = stdin.readLine();
               else if (choice==3)
                         System.out.println("Please key in Date (DDMMYY):");
                         System.out.flush();
                         date = Integer.parseInt(stdin.readLine());
                         System.out.println("Please key in Name of Customer:");
                         System.out.flush();
                         custName = stdin.readLine();
               else if (choice==4)
                         System.out.println("Please key in Date (DDMMYY):");
                         System.out.flush();
                         date = Integer.parseInt(stdin.readLine());
                         System.out.println("Please key in Start time:");
                         System.out.flush();
                         startTime = Integer.parseInt(stdin.readLine());
               else if (choice==5)
                         System.out.println("Please key in Date (DDMMYY):");
                         System.out.flush();
                         date = Integer.parseInt(stdin.readLine());
                         System.out.println("Please key in Name of Customer:");
                         System.out.flush();
                         custName = stdin.readLine();
                         System.out.println("Please key in Start time:");
                         System.out.flush();
                         startTime = Integer.parseInt(stdin.readLine());
                    }

Please stop calling your questions urgent. Everybody's question is urgent to them. Nobody's are urgent to the people who are going to answer them. Calling your questions urgent suggests that you think they are more important than others' (They're not.) and will only serve to irritate those who would help you. It won't get your questions answered any sooner.

ABAP proxy class - data structure

I generated a ABAP Proxy Class and the data structure I want to use is put automatically under item structure which has 0...unbounded type.
1. How can I get rid of this item structure as it will create another unnecessary level for my mapping
2. If my source structure has only 3 level, and the target structure has more than 3 (including item), how to map it?
e.g.
Source structure: Level 1(occurrence 1) > Level 2(1)> Level 3(0..1)
Target structure: Level 1(1])--> Level 2(0...1) --> item (0..unbounded) ---> Level 4(0..1)
I need to map level 3 from my source to level 4 in target, but it didn't seem to work.
Thanks.

--->1. How can I get rid of this item structure as it will create another unnecessary level for my mapping
You can delete the proxy at Application Server.....make necessary changes at XI Message Interface and again generate the proxy...
-->Source structure: Level 1(occurrence 1) > Level 2(1)> Level 3(0..1)
Target structure: Level 1(1])--> Level 2(0...1) --> item (0..unbounded) ---> Level 4(0..1)
For this you need to make use of context change features of XI Mapping.
Regards,

A robust data structure for a histogram of events over time.

Hi all,
I've been thinking for days, I am still lost on how to solve the following:
Let's say that we have a list of events that occur over a certain time interval. Each event has a start timestamp and a finish timestamp. The events occur in no particular order and expand over various length of periods.
With respect to the histogram, we mark the time period of an event occuring with a frequency of one. When events overlap over a time period, we mark that time period with a frequency of number of overlapping events.
Given this raw data, we want to examine the data's frequency under various granularity such as second, minutes, hours, days, weeks, month and years for 60sec, 60min, 24 hrs, 7 days, 4 wks ranges respectively.
So, what is a good intermediate (general) data structure that can store all the various events such that they can be easily transformed to a specific granularity. My goal here is to read the event list once, and histogram of events data with respect to various timeframe can be calculated.
Does a timeline like data strucuture that stores aggregated frequencies solve my problem? With a fine granuarlity say seconds and a long time range, the data structure will bound to overflow.
Any help is greatly appreciated!
Brian

You data structure would be in terms of Classes. You need a class Anevent{
String eventName;
Date startTime;
Date endTime;
In your main program have a Vector object which can store unllimited objects and which would not overflow. Store your event objects as they happen in this Vector. After your time interval of events is completed and you have the final Vector, you can pass the vector object to various methods in your main program to be simple or pass it to specailized graph drawing classes you would have to write to draw various Histograms.

Which data structure?

Similar Messages

Maybe you are looking for