Caching Query..

Hi Experts,
We are planning to implement non dynamic Caching for data storage.
Could anyone please suggest the best Java Collection to use in terms of speed and size?
Thanks in advance
Shagil

Your question is a bit vague, but generally a HashMap is the most efficient cache. Consider using a HashMap whose values are a SoftReference to the cached data, that way you won't fill your heap. If you do that you should set up a ReferenceQueue and a background thread to remove cleared entries from the Map. Be careful you don't inadvertently create a hard reference to the data.

Similar Messages

Caching Query Issues?

I have a test(on the server) and a development(localhost) sites so is the oracle database schemas. Both Oracle database schemas reside on the same server. I have the same datasource name for both schemas in the CF Administrators (local and server). For some reason, it seems to me that the data being pulled only from the test environment database eventhough I run the code on the development(localhost). I cleared the cache on the CF administrator. Any advise would be great. Thanks.

Soumen Mondal,
If you are facing this problem from the same client PC and in the same session, it's very strange.. but if there are two sessions running on the same PC this kind of issue may come..
Something about caching:
To decide whether to cache a query or not, consider the number of times you intend to run the query in a minute. For data changing in seconds, and queried in minutes, I would recommend not to cache query at all.
I may give a typical example for query caching in a query on material master cached for 24 hours, where we know that after creating material master, we are not creating a PO on the same day.
BR,
SB

Cannot query using both conforming and cached query result

TopLink doesn't allow me to both use conforming and cached query result at the same time.
Conforming is certainly not a superset of the [cached query result] features.
Can you confirm that it's a limitation of TopLink?
Any know workaround to end-up with the same features as using both conforming and cached query result?
Conforming is about seeing modifications you do in the same transaction. As a bonus, if you query for one object and specify at least the id as criteria because TopLink will have to check in memory anyway it can avoid going to the database.
But if I do a query like "give me employees hired before now and after 30 days ago" it's about more than one objects and about finding existance so cached query result is needed to get acceptable performance in a complex application trying to avoid the same SQL generated over and over again.

Thats where the trace just ends? It doesnt look like there's any LIKE or filtering going on (with respect to the Oracle pieces anyway), apparently MSAccess simply requested the whole table.
What do you mean by 'hang' exactly? Are you sure it's just not taking a long time to complete? How long have you waited? How fast does it complete on the other environment?
ODBC tracing isnt likely to help much for that. SQLNet tracing would be better to see what is going on at a lower level. Specifically, what is going on at the network level? Is the client waiting for a packet to be returned from the database?
Is the database having a hard time processing the query, perhaps due to index/tuning issues?
Assuming that is indeed the query that is "hung", how much data does that return?
Are you able to reproduce the same behavior with that query and vbscript for example?
Greg

cache-query-results question

I have another post for general descriptor tag information but I do have a specific question. In a project I am looking at I see:
<cache-usage> check cache by primary key </cache-usage>
<cache-query-results>false</cache-query-results>
<maintain-cache>true</maintain-cache>
I'm not sure how to interpret this. Does this mean that a cache is in place or not? cache-query-rests is set to false which implies no caching, yet the other parameters imply a cache is in place. What overrides here?
Thanks

The XML maps directly to the API so the JavaDocs and related documentation are the best tools:
cache-usage: query.setCacheUsage(int)
This option indicates how the object cache should be used when processing the query. This is how in-memory query is configured as well as support for cache-hits on ReadObjectQuery.
cache-query-result: query.setShouldCacheQueryResults(boolean)
This option allows you to indicate that the results returned from the query execution should be held. When the query is executed again these results will be returned without going to the database or searching the object cache. This is just caching the results locally within the query.
maintain-cache: query.maintainCache() or query.dontMaintainCache()
This setting determines if the results returned from the query should be cached in the shared object cache. It is on by default and turning this off is very rare. Occasionally done to compare the cache version with the database verision when handling an optimistic locking failure.
Doug

Is it possible to stop APEX using a cached query?

I was hoping that the issue where report regions sometimes show the error: "ORA-06502: PL/SQL: numeric or value error: NULL index table key value" would be fixed in 3.1.1 however it seems not to have been (We have had it several times now). The workaround seems to be to change the query slightly such as selecting NULL as an additional column and then hiding that column.
This obviously only works AFTER the issue has occured, ie, after we have received a complaint from someone!
Is it possible to stop APEX using the cached version of some queries altogether? The ones we are calling are quite simple and so will not give that much overhead.
Thanks.

I've actually found several threads on this from the past:
ORA-06502: PL/SQL: numeric or value error: NULL index table key value
http://kr.forums.oracle.com/forums/thread.jspa?threadID=644172&tstart=120 (This thread states the problem has been fixed however it would appear either it hasn't or there is a similar bug elsewhere.)

Caching Query in Portal - Variable set to System Current Date

Hi,
I am facing issue on caching for query in portal in Production. I have query called ABC in production with variable Master valid on for user to chooce the date. This variable is hardcoded value in General Precalculation in variable assigment session. Eg. If i run OLAP Cache today then it will hardcoded to 18.9.2008. I cannot change the production query to set to System current date unless I use new technical name for any changes eg XYZ.
Can i use different technical name for query in portal eg. XYZ in order to capture master valid default to today date ? I will do Broadcaster - OLA Cache for XYZ and does it also cache for ABC (query in portal)
The purpose of caching is to happen to query ABC in portal. Please advise.
PY

name_in gets the value of the item. So you should use copy.
copy('',:system.current_block||'.'||:system.current_item)

Coherence cache query - distinct

Hi,
I am trying to find out how to do a distinct query in coherence cache. For example, if my key object has LastName, FirstName. How do I get the collection of keys of distinct last names? In another words, I am trying to find out what are the distinct Last names in the cache. Is there any filter can be used to do that?
Thanks.

Hi,
You can use DistinctValues aggregator.
http://download.oracle.com/docs/cd/E14526_01/coh.350/e14977/toc.htm
Wei
Edited by: welin on Jun 10, 2010 8:38 AM

Cache query in Portal

Hi,
I am facing issue on caching for query in portal in Production. I have query called ABC in production with variable Master valid on for user to chooce the date. This varible is hardcoded value in General Precalculation in variable assigment session. Eg. If i run OLAP Cache today then it will hardcoded to 18.9.2008. I cannot change the production query to set to System current date unless I use new technical name for any changes eg XYZ.
Can i use different technical name for query in portal eg. XYZ in order to capture master valid default to today date ? I will do Broadcaster - OLA Cache for xyz and does it also cache for ABC (query in portal)
The purpose of caching is to happen to query in portal. Please advise.
PY

Hi,
Then looks like a problem with their user roles. Try to do the Root Cause. Meantime copy thier roles to new ones or change them slightly so they work.
Cheers,
Kedar

Cache query results in too much garbage collection activity

Oracle Coherence Version 3.6.1.3 Enterprise Edition: Production mode
JRE 6 Update 21
Linux OS 64 bit
The application is using a object Customer having following structure
Customer(CustID, FirstName, LastName, CCNumber, OCNumber)
Each property of Customer is a inner classes having getValue as one of the methods retuning a value. The getValue method of CCNumber and OCNumber return a Long value. There are 150m instances of Customer in cache. To hold this much data in cache we are running several nodes on 2 machines.
The following code is used to create indexes on CCNumber and OCNumber:
     ValueExtractor[] valExt = new ValueExtractor[]{
          new ReflectionExtractor("getCCNumber"), new ReflectionExtractor("getValue")};
     ChainedExtractor chExt = new ChainedExtractor(valExt);
     Long value = new Long(0);
     Filter f = new NotEqualsFilter(chExt, value);
     ValueExtractor condExtractor = new ConditionalExtractor(f, chExt, true);
     cache.addIndex(condExtractor, false, null);The client code queries the cache with following code:
     ValueExtractor[] valExt1 = new ValueExtractor[]{
          new ReflectionExtractor("getCCNumber"), new ReflectionExtractor("getValue")};
     ChainedExtractor chExt1 = new ChainedExtractor(valExt1);
     EqualsFilter filter1 = new EqualsFilter(chExt1, ccnumber);
     ValueExtractor[] valExt2 = new ValueExtractor[]{
          new ReflectionExtractor("getOCNumber"), new ReflectionExtractor("getValue")};
        ChainedExtractor chExt2 = new ChainedExtractor(valExt2);
     EqualsFilter filter2 = new EqualsFilter(chExt2, ocnumber);
     AnyFilter anyFilter = new AnyFilter(new Filter[]{filter1, filter2})
     cache.entrySet(anyFilter);The observation is that for 20 client threads the application performs well(avg response time = 200ms) but as the number of client threads increases the application performance goes down disproportionately(query returns anywhere between 1000ms to 8000ms for 60 threads). I think this is because of the eden space filling up very fast when the number of client thread goes up. The number of collections per second goes up with the number of client threads. There are almost 2-3 ParNew collections every second when there are 60 client threads where as only 1 collection per second for 20 client threads. Even 100-200ms pause degrades the overall query performance.
My question is why coherence is creating so many objects that fills up eden so fast? Is there anything I need to do in my code?

Hi Coh,
The reason for so much garbage is that you are using ReflectionExtractors in you filters, I assume you do not have any indexes on your caches either. This means that each time you execute a query Coherence has to scan the cache for matches to the filter - like a full table scan in a DB. For each entry in the cache Coherence has to deserialize that entry into a real object then using reflection call the methods in the filters. Once the query is finished all these deserialized objects are garbage that needs to be collected. For a big cache this can be a lot of garbage.
You can change to POF extractors to save the deserialization step which should reduce the garbage quite a bit, although not eliminate it. You could also use indexes, which should eliminate pretty much more of the garbage you are seeing during queries.
JK

Caching query results?

Hi guys,
I have this page which calls the same query (across a database link!) four times! First when an initial LOV is generated (and the page is loaded), again for another LOV based off the initial one and finally for the report generated based off the 2 selections. This is crazily slow (30-40 seconds each load~) and also: When you try to sort with an Interactive Report by clicking the column names it then queries the database again (I assumed this part was cached but apparently not). I therefore need to cache the results from an initial query and then just limit the results locally. How can I do this? Guides? Examples?
Thanks for help.
Mike

I tried this example based on a view:
CREATE MATERIALIZED VIEW MV_TEST2
     REFRESH COMPLETE
     START WITH SYSDATE
     NEXT SYSDATE + 1/48
     WITH ROWID
     AS SELECT * FROM test1;REFRESH COMPLETE -- The complete refresh re-creates the entire materialized view.
START WITH SYSDATE -- run now
NEXT SYSDATE + 1/48 -- run again in half an hour
WITH ROWID -- I think this option is important if you use partial refresh of the view.
AS SELECT * FROM test1; -- test1 is a view:
CREATE OR REPLACE VIEW TEST1 AS
SELECT st_id, st_name
    FROM aaw_solution_tree;Are column indexes still possible? I'm not sure:
Indexing: !with respect to MV's on 10gR2 Jonathan Lewis wrote! ... you are allowed to create indexes on the tables that sit under materialized views - just don't make them unique indexes
How much freedom is there in setting the refresh rate?
What type of refreshing do you need?
Another useful link: [http://asktom.oracle.com/pls/ask/search?p_string=materialized+view|http://asktom.oracle.com/pls/ask/search?p_string=materialized+view]
Hope it helps.
Tobias

Disabling cached query responses?

Here's the problem I have...
Whenever I go to our SharePoint site to upload a document, CSA will flag whatever file I click on in the "Open file" window (before I go to upload) and query me if I want to give access to iexplore.exe to read the file. If a normal user is in this situation, they'd probably click no. Then, when you go to upload that file, the access is automatically blocked by the remembered query answer.
The query I've made (I made a specific one for this) DISABLES the "Don't Ask Me Again" option. My question is how can I disable all cached responses so that users can deny access in a certain situation, then grant access in another? Or is there a way to force the Agent to clear it's cached responses every couple minutes?

It is the Don't ask again checkbox available from the Query Settings page in the MC that controls whether the end user has the ability to have a query response remembered permanently. As the administrator, you decide whether or not users have the option to choose Don't ask again. If user queries do not include a Don't ask again option and responses are only cached temporarily (for approximately an hour) users can click the Clear button in the agent Query User Responses window to delete all temporarily cached responses.
Look at the URL for more information
http://www.cisco.com/en/US/products/sw/secursw/ps5057/products_configuration_guide_chapter09186a0080424780.html#wp962337

Query cache,query monitor

Hi
wt is the purpose of Query monitor and query cache.. can u plz.. explain
ponts assured*
Regards
Rekha

As the verbeage indicates, query monitor is to monitor the runtime performance of BW queries. Query monitor is one of the tools in BW to monitor the query performance. The transaction to run query monitor is RSRT.
In RSRT, you can exceute queries in various modes and you can to some extent enforece a query to be executed in a certain path; For example, you can simulate the execution of q query without using a aggregate, without using cache, etc.
In monitor you can also view how the query is getting executed and diagnose the possible causes as to why a query is running slow.
Caching is to store the query results in the memory of the BW system's application server. If you cache a query, the run time performance will improve considerably, because the result ste is stored in the meonry and every time when the query is run, the OLAP engine will not have to read the data base to fetch the records.
The query caching has some limitations; if the query result changes, the cache will not help, because the new result set has to be again read from database and presented.
You can get more on this in help.sap.com
Ravi Thothadri

Store Query Output or Cache Query

My search query returns almost about 5000 records or more at
a time out of 7million records. and I am outputing about 35 records
at a time on a page. The problem is when the user pages to the next
page, I hit the DB again and again through the paging process of
the user.
The Cache attribute of <cfquery> does not work when you
are using <cfqueryparam> methods in the Components which we
are using.
How can I Store all the outputs of the Query returned into a
var and cycle through it instead of hitting the DB every time or is
there another method of caching this returned query output?
Please give example.

Thanks, that did the trick, another problem that came out of
this is Internet Explorer. When Users page to lets say page #6 and
then use the back button to come back, when it reaches the page
one,the Query executes again. This only happens with internet
explorer. I have tried caching the page but still same problem
persists. I have changed the form method to "get" and it is still
problematic.

Performance problem...is there a way to cache query results?

Greetings team,
I've been deploying DS5.2 for a while now, and am on the cusp of pushing it into our production environment, however I've been noticing lately that some hosts are taking an exorbitantly long time to log in (actually, a user noted it, and I'm now investigating).
Logins to hosts in this environment can take anywhere from 10-50 seconds. One thing that I've noticed is that any time you run a command that requires any amount of awareness of uid->username translation (i.e. if you ls -l /opt/home), queries are made to the configured directory server for this information. Is this normal? Since uid's and usernames don't often change (in most environments, anyway), is there a way this could be cached?
I see also in my access log for my primary server (configured as a hub, btw) that there is near constant traffic to that host for LDAP info. I'm not sure why it's so chatty, but it does appear to be slowing things down a bit. The load on my LDAP host (a SunFire V210 w/ 1GHz processor, 1024MB RAM) seems to float between 1 and 12, with sar reporting an average idle time of about 44%.
Any ideas? I'm really at a loss to explain why there's so much traffic to this host when much of it seems to come from hosts with nobody logged into them.
Patrick

It is great that you have found the root cause of
your issue.
nscd is by default started at boottime by a usual OS
install. There is a /etc/nscd.conf but I doubt that
anyone will change anything there as the default
settings are good for most cases.
I think LDAP search performance is affected by the
existence of Search Indexes also.
I have observed that if the user home directory is
NFS mounted especially over a WAN, be it via
/etc/fstab or automount maps, the login process will
be very slow, it will take a while to obtain a
command prompt at the home directory level.
GaryGary et al,
In my environment nscd has been explicitly disabled for some historical reasons, none of which are still a problem. So, I'm going to enable it for only passwd and group caching, with the default values for those caches.
I'm in the process of working out my performance tuning plan for my LDAP servers, but I'm definitely going to have an eye on indices and caches. Those will probably have the least impact on search times and such for the moment since my directory is so tiny (261 entries!), but preventing that traffic from hitting the server at all will be a huge savings.
I can definitely see why WAN mounted homedirs would cause things to lag. That's not the case here since NFS is a big no-no.
Patrick

How to force Work Item Query Policy to refresh its cached query results?

I have enabled the Work Item Query Policy on my TFS project. It works as expected, except when using Visual Studio 2013 with the following sequence of steps:
User selects Check in Pending Changes from the Source Control Explorer
User enters in the ID of the work item to be associated to the changeset
User clicks the Check In button
Work Item Query Policy displays message ' Work item ##### was not found in the results of stored query...'
User realizes his mistake, and modifies (and saves) the work item so that it is returned in in the query result set
User clicks the Check In button again expecting the TFS policy to accept the association
Work Item Query Policy still displays message ' Work item ##### was not found in the results of stored query...'
Removing the Work Item and re-associating it does not make a difference. The only workaround that I have found is to close Visual Studio and reopen it. Does any one have a better solution than this?

Our setup is different from the one you are using:
- User is using VS 2013 Update 4.
- We are running TFS 2010 SP1
The test case that you described is different from the one that is causing my problem (that scenario works fine for me as well). I am trying to associate the check in to the same work item both times; whereas, you are associating it to a different
work item the second time. I can consistently reproduce the error using the following steps:
1) Create a query that returns All Bugs in Active state, and set it as the query for the Work Item Query Policy
2) Create and save a new Bug
3) Run the query to confirm that the new bug does not appear in the result set
4) Checkout a file, modify it, save it
5) Check in the file and associate it to the bug from step 2)
- the Work Item Query Policy will issue an error message saying that the work item cannot be found in the associated query
6) Change the state of the bug to Active, and save
7) Refresh the query to confirm that the bug now appears in the result set
8) Check in the file again
- error message from step 5) will not go away

Caching Query Image Stream

Currently I am working on a CMS of sorts and the higher-ups
wanted to have some digital asset management capibilities. All
images are now being stored in SQL2k5 in binary form. On a page
that contains the image, I have a the image
src="myImageLoader.cfm?imageId=x" where x is the id of the image to
load.
This all works fine, however if the user returns to a page
that they have previously visited, the image is not cached and must
restream out. Any ideas on how to cache the image after it has been
streamed?

The only smart way to do this is to write the image out as a
regular file and link to that.
Caching, handshaking, and compression are then handled
beautifully by the web server and client browser.
The only thing you would (possibly) need to do is to run a
job every few days to clean out stale (or access limited) files.
Otherwise, you will need to set the "last modified" and
"expires" headers and to monitor for and process the
"If-Modified-Since" header.

Caching Query..

Similar Messages

Maybe you are looking for