Horizontal Fragmentation in NoSql Data Stores

Hello every body
My question is :
can we apply the optimization technique of " Horizontal fragmentation (partitioning)" in this new type of data stores (NoSql) just like in relational databases?
thanks at advance

No, not really. I suggest reading the documentation here:
http://www.oracle.com/technetwork/products/nosqldb/overview/index.html
You may want to start with the link "Intelligent Driver" and also read the Getting Started Guide.
--mark

Similar Messages

Cannot attach data store shared-memory segment using JDBC (TT0837) 11.2.1.5

Hi,
I found the thread Cannot attach data store shared-memory segment using JDBC (TT0837) but it can't help me out.
I encounter this issue in Windows XP, and application gets connection from jboss data source.
url=jdbc:timesten:direct:dsn=test;uid=test;pwd=test;OraclePWD=test
username=test
password=test
Error information:
java.sql.SQLException: [TimesTen][TimesTen 11.2.1.5.0 ODBC Driver][TimesTen]TT0837: Cannot attach data store
shared-memory segment, error 8 -- file "db.c", lineno 9818, procedure "sbDbConnect"
at com.timesten.jdbc.JdbcOdbc.createSQLException(JdbcOdbc.java:3295)
at com.timesten.jdbc.JdbcOdbc.standardError(JdbcOdbc.java:3444)
at com.timesten.jdbc.JdbcOdbc.standardError(JdbcOdbc.java:3409)
at com.timesten.jdbc.JdbcOdbc.SQLDriverConnect(JdbcOdbc.java:813)
at com.timesten.jdbc.JdbcOdbcConnection.connect(JdbcOdbcConnection.java:1807)
at com.timesten.jdbc.TimesTenDriver.connect(TimesTenDriver.java:303)
at com.timesten.jdbc.TimesTenDriver.connect(TimesTenDriver.java:159)
I am confused that if I use jdbc, there is no such error.
Connection conn = DriverManager.getConnection("url", "username", "password");
Regards,
Nesta

I think error 8 is
net helpmsg 8
Not enough storage is available to process this command.
If I'm wrong I'm happy to be corrected. If you reduce the PermSize and TempSize of the datastore (just as a test) does this allow JBOSS to load it?
You don't say whether this is 32bit or 64bit Windows. If it's the former, the following information may be helpful.
"Windows manages virtual memory differently than all other OSes. The way Windows sets up memory for DLLs guarantees that the virtual address space of each process is badly fragmented. Other OSes avoid this by densely packing shared libraries.
A TimesTen database is represented as a single contiguous shared segment. So for an application to connect to a database of size n, there must be n bytes of unused contiguous virtual memory in the application's process. Because of the way Windows manages DLLs this is sometimes challenging. You can easily get into a situation where simple applications that use few DLLs (such as ttIsql) can access a database fine, but complicated apps that use many DLLs can not.
As a practical matter this means that TimesTen direct-mode in Windows 32-bit is challenging to use for those with complex applications. For large C/C++ applications one can usually "rebase" DLLs to reduce fragmentation. But for Java based applications this is more challenging.
You can use tools like the free "Process Explorer" to see the used address ranges in your process.
Naturally, 64-bit Windows basically resolves these issues by providing a dramatically larger set of addresses."

Getting Error : Cannot attach data store shared-memory segment,

HI Team,
I am trying to integrate Timesten IMDB in my application.
Machine details
Windows 2003, 32 bit, 4GB RAM.
IMDB DB details
Permanent size 500MB, temp size 40MB.
If I try to connect to database using ttisql it get connected. But If I try to connect in my Java application I get following exception.
java.sql.SQLException: [TimesTen][TimesTen 11.2.1.3.0 ODBC Driver][TimesTen]TT0837: Cannot attach data store shared-memory segment, error 8 -- file "db.c", lineno 7966, procedure "sbDbCreate"
at com.timesten.jdbc.JdbcOdbc.createSQLException(JdbcOdbc.java:3269)
at com.timesten.jdbc.JdbcOdbc.standardError(JdbcOdbc.java:3418)
at com.timesten.jdbc.JdbcOdbc.standardError(JdbcOdbc.java:3383)
at com.timesten.jdbc.JdbcOdbc.SQLDriverConnect(JdbcOdbc.java:787)
at com.timesten.jdbc.JdbcOdbcConnection.connect(JdbcOdbcConnection.java:1800)
at com.timesten.jdbc.TimesTenDriver.connect(TimesTenDriver.java:303)
at com.timesten.jdbc.TimesTenDriver.connect(TimesTenDriver.java:159)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:207)
Maximum permanent size that works with Java application is 100MB. But it would not be enough for our use.
Could anybody let me know the way to resolve/reason for getting this error? Any response would be appreciated.
Thanks in Advance,
Regards,
atul

This is a very common problem on 32-bit Windows. A TimesTen datastore is a single region of 'shared memory' allocated as a shared mapping from the paging file. In 'direct mode', when the application process(in your case either ttIsql or the JVM) 'connects' to the datastore the datastore memory region is mapped into the process address space. In order for this to happen it is necessary for there to be a free region in the process adddress space that is at least the size of the datastore. This region must be contiguous (i.e. a single region). Unfortunately, the process memory map in 32-bit Windows is typically highly fragmented and the more DLLs that a process uses the worse this is. Also, JVMs typically use a lot of memory, depending on configuration.
Your options to solve this are really limited to:
1. Significantly reduce the memory used by the JVM (may not be possible).
2. Use a local client/server connection from Java instead of a direct mode connection. To minismise the performance overhead make sure you use the optimised ShmIpc connectivity rather than TCP/IP. Even with this there is likely to be a >50% reduction in performance compared to direct mode.
3. Switch to 64-bit Windows, 64-bit TimesTen and 64-bit Java. Even without adding any extra memory to your machine thsi will very likely fix the problem.
Option (3) is by far the best one.
Regards,
Chris

Error while activating Data Store Object

Hi Guru's,
When I try to activate a data store object i get the error message :
     The creation of the export DataSource failed
     No authorization to logon as trusted sys tem (Trusted RC=2).
     No authorization to logon as trusted sys tem (Trusted RC=2).
     Error when creating the export DataSource and dependent      Program ID 4SYPYCOPQ94IXEGA3739L803Z retrieved for DataStore object ZODS_PRA

Hi,
you are facing a issue with your source system 'myself', check and repair it. Also check if the communication user (normally ALEREMOTE) has all permissions needed.
kind regards
Siggi

Want the data store values to be displayed in input field of form

Hi,
I wanted to know wether is there possibility of displaying the data in a input field instead of expression field.
In our model i have used a form which has material type,plant and Vendor connected to data service1 in turn gives the out put in chart and i also have the second input form which has To(0CALMONTH) combo box which is connected to Data service2 and gives the data in table.
But what i wanted is to i also want to use the input fields of first form to second Data service2 and get the data based on on inputs of form1 and form2i.e also To field.
We can connect the port from form1 to DS2 but problem is we need to click the submit of form1 and form2 and it doesnt gives the output according the input of form1 and form2 as it gives the output of that form when we click that submit button.
I have followed help.sap for data tore procdure.
So i have used a datstore where i will store the values of form1 and call it in expression field which i will add and hide it in form2(hide because user should not see that input fields).
Formula used in data store for expression field.
IF(CONTAINS(STORE@matltype,@Material_Type),STORE@matltype,STORE@matltype &(IF(LEN(STORE@matltype)>0,'; ',''))&@Material_Type)
But whats happening is the value is getting concatenated when i goon change the values in input field, so i wanted to get the values to be replaced as soon as i change input field of form1(if use replace function its not working) and also it would be more preferrable if we use input field instead of expression field.
I would also like to know is there any alternate solution for the above requirement instead of datastore.
Thank You
K.Srinivas

Hi,
I have Form1 connected to Data service1 displays the data in a Chart and i have another Form2 connected to Dataservice2 displays the data in table.
In form1 there are Material Type,plant, and Vendor and form2 i have To (0CALMONTH)SO now i want also Form1 inputs for Table which gets teh data from Dataservice2.
What i have said earlier is connecting Form1 to Dataservice doesnt fetch the data correectly,its because if i click submit in form1 i get the data of those 3 inputs and i need to click the submit button in form2 after giving input which shows the data accordingly where it doesnt fulfill the requirement.
So i wanted some solution for that.For that reason i have used the data store and the procedure i have followed the help.sap as i said in my above mesage.
If Data store is also suggestable than i want to display the data in Text input Field instead of Expression field which should replace the previous values as soon as values change in data store.
Hope i have tried to be clear if still it is not i am ready to explain again.
Thank You
K.Srinivas

JDODataStoreException: The instance null does not exist in the data store

I'm unable to figure out how this exception occurs.
I have a class IDCounter which has a number of fields such as
'm_Name' (String)
'm_AccountName' (String)
'm_UserName' (String)
'm_Description' (String)
'm_CreationDate' (Date)
'm_LastModifiedDate' (Date)
'm_DeletedDate' (Date)
'm_Count' (long)
The filter I'm using is "m_AccountName == \"test\" && m_UserName
==\"test\" && m_DeletedDate == null"
The generated SQL statement is "SELECT t0.M_IDX, t0.JDOCLASSX,
t0.JDOLOCKX, t0.M_ACCOUNTNAMEX, t0.M_CREATIONDATEX, t0.M_DELETEDDATEX,
t0.M_DESCRIPTIONX, t0.M_LASTMODIFIEDDATEX, t0.M_NAMEX, t0.M_USERNAMEX,
t0.M_COUNTX FROM ABSTRACTENTITYX t0 WHERE ((t0.M_DELETEDDATEX IS NULL) AND
t0.JDOCLASSX = 'com.ewarna.pdm.entities.IDCounter')"
Exception Trace:
javax.jdo.JDODataStoreException: The instance null does not exist in the
data store.
 at
com.solarmetric.kodo.impl.jdbc.runtime.LazyResultList.instantiateRow(LazyResultList.java:165)
 at
com.solarmetric.kodo.impl.jdbc.runtime.LazyResultList.get(LazyResultList.java:96)
 at java.util.AbstractList$Itr.next(AbstractList.java:416)
 at
com.solarmetric.kodo.runtime.ResultListIterator.next(ResultListIterator.java:49)
 at
com.solarmetric.kodo.impl.jdbc.runtime.ResultListFactory.createResultList(ResultListFactory.java:85)
 at
com.solarmetric.kodo.impl.jdbc.runtime.JDBCStoreManager.executeQuery(JDBCStoreManager.java:646)
 at
com.solarmetric.kodo.impl.jdbc.runtime.JDBCQuery.executeQuery(JDBCQuery.java:150)
 at com.solarmetric.kodo.query.QueryImpl.executeWithMap(QueryImpl.java:580)
 at com.solarmetric.kodo.query.QueryImpl.execute(QueryImpl.java:428)
 at
com.solarmetric.kodo.query.QueryImpl$SynchronizedQuery.execute(QueryImpl.java:1331)
 at
com.ewarna.pdm.sessions.persistence.BasicQuery.getByAdvancedFormula(BasicQuery.java:78)
 at
com.ewarna.pdm.sessions.persistence.BasicQuery.getByFormula(BasicQuery.java:119)
 at
com.ewarna.pdm.sessions.persistence.BasicQuery.getByFormula(BasicQuery.java:95)
 at
com.ewarna.pdm.sessions.persistence.BasicQuery.getAll(BasicQuery.java:131)
 at
com.ewarna.pdm.sessions.persistence.GenericEntityManager$7.execute(GenericEntityManager.java:305)
 at
com.ewarna.pdm.sessions.persistence.GenericEntityManager.execute(GenericEntityManager.java:251)
 ... 18 more

Youcan no longer display a workbook. You receive an error message when opening: <Internal error>: 1201 document storage
Cause and prerequisites
In very rare cases, when you store a workbook, you might not be able to open it again.
Solution
Function module BDS_PHIOS_GET_RIGHT has to be changed so that the last available version of the Workbooks can be displayed.

Unable to load data store in a new model

Hi,
I have Win XP Pro SP3. MS SQL 2008 which has adventure works DW2008R2 database. I have oracle data integrator.
I have created a physical and logical schema and have established a connection with my MS SQL 2008 and the test connection is successful. I create a new model folder and a new model within. In the new model, I gave technology as microsoft sql server , logical schema name selected, and in reverse tad: standard, context:global and when i click on "reverse" no tables with are seens under the new model.
no error message either. I have been trying to figure this out for the last 12 hrs...can someone please suggest/guide me here please. this is my first time using ODI.
Goo

Hi,
Thanks for the reply. I thread you provided gives details about how to get connected. But as i mentioned earlier, i can connect. I am just not able to bring in data store...
Any help will be appreciated. I can send the screen shot if you would like.
Goo

How can I transfer data form one data store to another iview /dats store

Hello,
I have two iview and want to transfer the data from my data store (main iview) to the nested iview (or the data store in nested iview)
Can you give me instructions, how I can implement this with the signal in and outports.
Thanks
Bst regards
thomas
I

Hi Thomas
Pass the output data of one iview to another(by connecting the output of the 1st to the input of the 2nd) and then Signal Out the combined data of the two iviews from the 2nd iView using a event(say, a submit button) from the 2nd iView.
Connect the signal out to the nested iView(main iView) by composing a Signal In on the nested iView.
Next, inside the nested iView design layout push the values from Signal In to a data store.
Now you could access all the data @datastore from any no. of forms inside the nested iViews.
Regards
Navneet

What do you recommend to use as an offline data store, since SQL CE support is not in VS 2013?

A few years back I was architecting an occasionally connected .Net desktop application.
VS 2010 was offering full support for Microsoft Sync Framework and SQL CE with Entity Framework.
This seemed like the perfect marriage, so I ran with it, and the resulting software solution is still successfully running in production, years later.
Jump forward to today, and I am architecting a new occasionally connected .Net desktop application.
I was really looking forward to taking advantage of the advances made by Microsoft in using the tools built into VS 2013.
However, what I discovered has dumbfounded me. VS 2013 has no designer support for Sync Framework, and worse, built in support for SQL CE has been completely removed, including the ability to generate Entity Framework models from a
CE database using the designer.
My question to the community is, what tools should I be using to solve the problem of offline storage in my brand new .Net application?
I am aware of ErikEJ’s SQL Server Compact Toolbox, which brings back some support for these features in VS 2013, but it is not as fully featured as the VS 2010 native support was, plus it does not have the institutional “Microsoft” stamp on it.
I am building a multimillion dollar corporate solution that I will have to support for many years.
I would like to have some comfort that the technologies I select, today, will still be supported 5 years from now, unlike the way Microsoft has discontinued supporting Sync Framework and CE in the most recent VS.
I can accept open source technologies, because there is a community behind them, or off the shelf corporate solutions, since they will be driven by financial gain, but I have trouble committing to a solution that is solely supported by an individual,
even if that person is a very talented Microsoft MVP.
Some of the features of SQL CE that I would like to keep are
Built in encryption of the file on disk
Easy querying with an ORM, like Entity Framework
Tools to easily sync up the offline data store with values from SQL Server (even better if this can be done remotely, such as over WCF)
Does not require installation of additional software on the client machine, as SQL Express would
Please, provide your feedback to let me know how you have achieved this, without resorting to simply using an older version of VS or Management Studio.
Thank you.

Hello,
Based on your description, you can try to use SQL Server 2012 Express LocalDB.
LocalDB is created specifically for developers. It is very easy to install and requires no management, but it offers the same T-SQL language, programming surface and client-side providers as the regular SQL Server Express.
SQL Server LocalDB can work with Entity Framework and ADO.NET Syc Framework. However, there is no built-in encryption feature in LocalDB which
can let you encrypt database. You should decrypt/encrypt data on your own, for example, using
Cryptographic Functions
Reference:SQL Express v LocalDB v SQL Compact Edition
Regards,
Fanny Liu
If you have any feedback on our support, please click here.
Fanny Liu
TechNet Community Support

How check what's stored in the data store at different steps in the model?

I would like to place a table or form temporarily within the model to display the values in the data store as they change at different points at runtime.
I tried simply adding a form, creating the same field types as are in the store, and then mapping the form fields' default values to the data store's fields. This does not work as it doesn't update automatically.
Tips?
Henning

Hi Jarod,
I have the similar problem of using the data form datastore as input for getting results.
I wanted to know wether is it possible to use input field instead of expression box so that i can click submit button based on input displayed from data store.
Also if i use expression box the values are getting concatenating instead i want to overwrite(dont know wether expressionbox value can be used as input) i couldnot overwrite the values.Can you let me know what is the function i need to use in the formula of expression box.
I have posted the same in a thread
want the data store values to be displayed in input field of form
Thank You
K.Srinivas

Error while Data Store Object activation

Dear Guru's
    I'm new to BI 7.......
    I'm trying to load data to Data Store Object from PSA through DTP and it was successful upto newdata table. But while activating the request to active table the total status of the request turned as red and in DETAILS all the messages were in green.
    Kindly let me know what could be the problem.
Surely points will be awarded
thanks in advance.....
with regards,
Viswa

Dear KK,
Here I printed the log pls go theough it..
Job started
Step 001 started (program RSDELPART1, variant &0000000000000, user ID 160624)
Delete is running: Data target ZIC_EX1, from 57 to 57
FB RSM1_CHECK_DM_GOT_REQUEST called from PRG RSDELPART1; row 000875
Request '57'; DTA 'ZIC_EX1'; action 'D'; with dialog 'X'
Leave RSM1_CHECK_DM_GOT_REQUEST in row 70; Req_State ''
Overall status 'Red' (user 160624)
Incorrect data could be visible in Reporting (see long text)
Status transition 2 / 3 to 9 / 9 completed successfully
SQL: 15.05.2007 13:26:26 160624
ALTER TABLE "/BIC/B0000161000" TRUNCATE PARTITION
"/BIC/B00001610000000000002"
SQL-END: 15.05.2007 13:26:26 00:00:00
Request DTPR_6K1ZWLCJLY9ZF4NLHC91N1J10 deleted from PSA;REQICODS entry also deleted
Request DTPR_6K1ZWLCJLY9ZF4NLHC91N1J10 not found in IC ZIC_EX1;CODS also deleted
Request DTPR_6K1ZWLCJLY9ZF4NLHC91N1J10 deleted from PSA;EQICODS entry also deleted
Overall status 'Deleted' (user 160624)
Delete is finished: Data target ZIC_EX1, from 57 to 57
Job finished
with regards
Viswa

Use of time series functions with horizontally fragmented fact tables

Hi Guys,
in OBIEE 10g it wasn't possible to use time series functions [AGO, TO_DATE] on horizontally fragmented fact tables. This was due to be fixed in 11g.
Has this been fixed? Has somebody used this new functionality? What the the limitations?
Tkx
Emil

Hello,
Can you give us some examples for "horizontally fragmented fact tables", we can tell you whether we can do that or not?
Thanks,

How to select from multiple tables which reside on different data stores ?

Suppose I have two data stores in one TimesTen instance:
1) Datastore A:
table1
2) Datastore B:
table2
I want to make a query like this:
select ... from table1, table2 where table1.colA = table2.colB
Can I ? If not, is there a workaround ?
BTW, because of business, we have to use two or more different datastores, so we can not put table1/table2 in the same datastore.
Thanks very much.

You can query multiple TimesTen databases, but your original question was about joining tables from two databases, which is not supported.
Using Cache Connect to Oracle to query an Oracle database is not distributed. It's still one single Oracle database you are querying. You cannot join a table in the TimesTen database with a table in the Oracle database, this is not allowed.
If you are willing to share your business requiremens, we can take a look and see what solution might work for you. Would you like to discuss this offline?
Susan

How to Set a Variable with data from Srouce Data Store

Hello ODI Experts,
I have created a Physical & Logical Schema and a Source Data store to pickup data from a database table.
On the other hand, I have a few variable that I will pass in a web service call (ODIInvokeWebService tool).
Would yo please guide how I can set variables from my source data store.
Thanks & Regards,
Ahsan

Hello Bos/Damodhar/ODI Experts,
Doesn't it gives me a less optimized approach picking one column per query (per variable)?
Lets say, I have to pick 35 columns from a table and put those in 35 variables...It would mean running 35 queries for fetching one record from the database table.
Doesn't it seem less performance effective (less optimized)..a little scary..any thing better that I can do to make it more optimized?
Another question, what if multiple new values have come in the DB table, since I am using Refresh Variable, would this variable have multiple values in it?
Thanks for all your help,
Ahsan
Edited by: Ahsan Asghar on 21-Jun-2011 07:46

Persistence data store...how to

Greetings!
I have a JMS application that I am developing using durable subscribers connecting to my Topic. Everything works like a charm, so now I want to extend capability ( aka, break my application =P ). I've read a bunch of documentation that says JMS has the capability to store these messages to a flat file or a database, but that's the extent of the documentation I can find.
I am wondering if, specifically, someone can tell me HOW do I configure where the messages get stored. Do I have to write code to store these messages to a database or is this functionality implicit within my application server (WebSphere 5)?
If someone could explain this voodoo magic to me or point me to a doc that explains it, I would be greatly appreciative!
Thanks in advance!
Matt,
JMS n00b

Matt
If I understand correctly .. what you need is a functionaltity to LOG all the messages being transferred on the JMS Server to a data store (RDBMS or File)
If yes, what all JMS Servers provide in case of Durable subscribers is the temp storage of messages in the data store till the message is received by the receiver to ensure gauranteed message delievery.
These messages are deleted from the data store once they are delievered to the subscriber.
In order to achieve Logging like functionality you will need to built a custom application ( a message consumer) which reads messages from the destination (topic/queue) and either writes to the file or using JDBC writes into the database.

Horizontal Fragmentation in NoSql Data Stores

Similar Messages

Maybe you are looking for