Problem setting up data collection for a performance problem

We have a performance problem on our production environment: there is a query that is long to display specially on specific records involving a lot of détails. I had the idea to set up data collection to investigate the problem.
I used the wizard to set up data collection with default settings.
I had created a database PERF_DW on our test server for that purpose.
I didn't have any problem with the wizard: it created data collection sets and SQL Server agents tasks and probably many objects. But I don't know where I missed something but I didnt, have the chance to specify the database I wanted to tune
and even when I started data collection, I couldn't figure out where were the reports.
Then I tought that I did it all wrong so I stopped data collection and dropped the database and tried to deleted the jobs created and I had the following message:
TITRE : Microsoft SQL Server Management Studio
Échec de Supprimer pour Travail « collection_set_1_noncached_collect_and_upload ». (Microsoft.SqlServer.Smo)
Pour obtenir de l'aide, cliquez sur :
http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=11.0.2100.60+((SQL11_RTM).120210-1917+)&EvtSrc=Microsoft.SqlServer.Management.Smo.ExceptionTemplates.FailedOperationExceptionText&EvtID=Supprimer+Job&LinkId=20476
INFORMATIONS SUPPLÉMENTAIRES :
Une exception s'est produite lors de l'exécution d'une instruction ou d'un lot Transact-SQL. (Microsoft.SqlServer.ConnectionInfo)
The DELETE statement conflicted with the REFERENCE constraint "FK_syscollector_collection_sets_collection_sysjobs". The conflict occurred in database "msdb", table "dbo.syscollector_collection_sets_internal", column 'collection_job_id'.
The statement has been terminated. (Microsoft SQL Server, Erreur : 547)
Pour obtenir de l'aide, cliquez sur :
http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&ProdVer=11.00.3128&EvtSrc=MSSQLServer&EvtID=547&LinkId=20476
BOUTONS :
OK
Now I understand that the collectors prevent the jobs from being deleted but I can't delete the collectors.
What should I do?
Any help is appreciated.
Thank you for your advice.
Sylvie P

Hello,
Please refer to the following article.
http://blogs.msdn.com/b/sqlagent/archive/2011/07/22/remove-associated-data-collector-jobs.aspx
You can run the dcclenaup.sql script mentioned in the article.
Another option is using sp_syscollector_cleanup_collector.
http://blogs.msdn.com/b/sqlagent/archive/2012/04/05/remove-associated-data-collector-jobs-in-sql-2012.aspx
Hope this helps.
Regards,
Alberto Morillo
SQLCoffee.com

Similar Messages

Recommendation for setting up Data Guard for EBS R12 12.1.3

Hello Experts,
I would like to get some experts opinion and recommendation on setting up Data GUARD for EBS R12.1.3 running on Linux based systems.
Firstly I would wanna let you all know that I am just geek in this subject matter so do excuse me for any mistakes.
While we are planning on setting up Data Gurad fro R12 what are the best practices to consider ?.
We have 2 node rac enabled database and would wanna have non-rac based physical standby, based on this I believe that having db tier on remote or standby site is enough ?
if db node is enough for standby location what are things to be done when a fail over / switch over / role transitioned to standby for the app node from primary site to point db on the
standby location or do we have to have both apps and db tier @ stand by location
--However having a apps tier/node at standby will be of no use as we are planing to run db on mounstate meaning not a active data guard setup.
Please do reply with your recommendation/suggestions/pointers
Thanks in Advance.

Service Provider Access resulted in exception 'oracle.apps.fnd.soa.util.SOAException: SystemError: Error while sending message to server. http://ec2-107-22-78-224.compute-1.amazonaws.com:8000/webservices/SOAProvider/EbizAuth?Generate=1132&soa_ticket=Tu2z7GYoWPwq-VdgimKRFg..' when attempting to perform 'GENERATE'. Please view Service Provider logs for more details
Can you find any details about the error in the log?
I have looked at the following Note as well:
Error using the Generate WSDL Button in Oracle E-Business Suite Integrated Soa Gateway Release 12.1.1 [ID 1090946.1]Did the doc help?
Any one know what may be causing this issue? Do we have to do additional setup for SOA gateway after the standard install of the OVM? I tried to follow the steps in the note above but I do not see any entry for "<jdbc_url oa_var="s_apps_jdbc_connect_descriptor"/>" in the file data-sources.xml.
Anyone have any ideas?Have you reviewed these docs?
Oracle E-Business Suite Integrated SOA Gateway Troubleshooting Guide, Release 12 [ID 726414.1]
Oracle E-Business Suite Integrated SOA Gateway 12.1.1 Consolidated One-Off [ID 815196.1]
Thanks,
Hussein

Can you set up a template for live performance in GarageBand for iPad Air?

II'm trying to set up a template for live performance using GarageBand on my iPad Air. Can anyone advise? Thanks.

QuickTime is AppleScript aware and has its own "dictionary". Many scripts and droplets have already been compiled and you can download them from this link:
http://www.apple.com/applescript/quicktime/
Some need further editing to update them to version 7 features but it's not difficult using the Script Editor app.
Tiger's new Automator app could also be used but its feature set is rather basic. Newer actions are available for download from the Mac OSX Downloads page.

How to set a date range for date field ?

Dear Experts,
Scenario:
I have a query in validating the date field in my BSP application. My application is for maintain infotype 0023 Other/Previous Employers online by employees in the company.
As per our design we are maintaining the all employment details of the employee both ( with in the current company / previous employment outside the company) in the same infotype.
Every employee will have a hiring date within the SAP HR system. We consider this date as the cutoff date between current and previous employment in our application. When the employee updating the details wia BSP page I need to check the following.
Record inside current company: Validation that, the user should only able to enter BEGIN DATE (BEGDA) greater than or equal HIRING DATE and END DATE(ENDDA) should be greater than FROM DATE (BEGDA).
Record outside current Company: Validation that, the user should only able to enter BEGIN DATE (BEGDA) less than or equal HIRING DATE and END DATE (ENDDA) should be greater than FROM DATE (BEGDA) and less than HIRING DATE.
Technical Requirement:
How to set a date range for date field, i.e. how we can limit the date range in a HTMLB date field? Can this it be achieved via standard functionality of HTMLB?
Following is the code to describe date field in my application.
    <htmlb:inputField id= "ENDDA_NEW_IN"
                      type= "date"
                doValidate= "TRUE"
                  showHelp= "TRUE"
                  disabled= "FALSE"
                     width= "183"
                     style= "cssTextAreadate"
                     value= "<%='99991231'%>"/>
Thanks a lot in advance for your assistance and help.
Cibinu2026
Edited by: cibin kuruvilla on Nov 12, 2008 11:13 AM

Hi,
This functionality is known to be very important and is a key part of the next major release of the JRC planned for the first half of 2008.
Regards,
<p>Blair Wheadon</p>
<p>Product Manager, Crystal Reports</p>

Firefox users cannot presently exercise choice, to opt-in or out of "data collected for improving services." As a result, Firefox is constantly dialing home, sometimes four times a day or more up to 48 times a day. There is some over-stepping and redunda

I understand the charter on this is to "check-in" once every time the program is turned on, and then once every 6 hours after that, or, once every 24 hours presumably. But the phone-home-effect is over-stepping these basic parameters. If a user turns their browser off and then on, Firefox is still obligated to "check-in" even if it just checked in 3 minutes prior. If the coding is not pre-designed to overstep, or act excessively in a redundant focus, the instance of once every six hours or once every 24 hours, is still overmuch if a user has been doing this (non-voluntarily-participating) for 9 months or longer.
== This happened ==
Every time Firefox opened
== This started when Firefox took up the initiative of "improving service. to end users" or similar idea, making the web safer for novice users, etcetera.

Opening question was truncated. Should read: "Firefox users cannot presently exercise choice, to opt-in or out of "data collected for improving services." As a result, Firefox is constantly dialing home, sometimes four times a day or more up to 48 times a day. There is some over-stepping and redundancy here. It would seem the practice of "improving service" has been accomplished with as much information as Mozilla has gathered in the last 6-18 months about its users habits. '''Isnt it about time to give users the option to opt out of that now that most the heavy liftiing has been accomplished?'''"

Problem setting data collection for SLD

Good day
I have the following situation.
A Solution Manager 71 installation, just recently patched to SP11. Previously I had the SLD on this system. And then a colleague informed that this could be a problem in the future as my ECC systems are Basis 731 which is higher level than the 702 on the Solution Manager system. So I installed a separate Java stack system on the same server as the Solution Manager.
The problem that I am faced with now is that I cant get the data collection (RZ70) to connect with the new Java system that I have installed. When I go into SLD and Administration, I see under Details>Data Suppliers that the server is specified as the Gateway Host, with a Gateway port of 3303. On the Solution Manager system it specified the gateway host as "localhost" and the gateway port as sapgw00.
So, from RZ70 I specify the gateway service as sapgw03 and then I get the result:
0: DSIMSID001_SID_00
: Collection of SLD data finished
0: DSIMSID001_SID_00
: Data collected successfully
0: DSIMSID001_SID_00
: RFC data prepared
0: DSIMSID001_SID_00
: Used RFC destination: SLD_NUC
0: DSIMSID001_SID_00
: RFC call failed: Error when opening an RFC connection (CPIC-CALL: 'ThSAPOCMINIT' : cmRc=17 thRc=2
0: DSIMSID001_SID_00
: Batch job not scheduled
And from the SLD_NUC entry in SM59 I get
Logon    Connection Error
Error Details    Error when opening an RFC connection (CPIC-CALL: 'ThSAPOCMINIT' : cmRc=17 thRc=2
Error Details    ERROR: SAP gateway communication error (Is SAP gateway closed?)
Error Details    LOCATION: SAP-Server DSIMSID001_SID_00 on host DSIMSID001 (wp 1)
Error Details    COMPONENT: CPIC
Error Details    COUNTER: 47
Error Details    MODULE:
Error Details    LINE:
Error Details    RETURN CODE: 239
Error Details    SUBRC: 0
Error Details    RELEASE: 721
Error Details    TIME: Mon Mar 16 14:04:05 2015
Error Details    VERSION:
Can anyone give me a few ideas what may be wrong here please?
Regards
Ray Phillips

Hi Ray,
Please check if the below SLD configuration is done.You can use any one of the two below link depending on how you want to configure your SLD.
Configure SLD for JCo and Creation of JCo Destinations - Portal - SCN Wiki
or
How to Create & Configure Local SLD on SAP NetWeaver 7.3 AS Java after Installation
If your SLD server is not a local SLD then make sure that in both the SLD system and the remote system host fine entry is there in /etc/hosts and etc/services.

Problems setting up username & password for SQL

Due to the outstanding advice I recieved from this excellent forum, I have managed to overcome my first problem with declaring a new Class.
This leads me to request help with my next biggest problem:
Setting up a user GUI that takes a "username" & "password" that will be used to access a password protected database.
I am a simple bloke, with simple thought processes, so please, go easy on me...
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.sql.*;
public class DBQuery1 {
     String username = "" , password = "";
     public static void main(String[] arguments) {
          PassDB UPass = new PassDB();
          String data = "jdbc:odbc:JavaTestDataBase";
          try {
               Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
               Connection conn = DriverManager.getConnection(data, "" + username, "" + password);
               Statement st = conn.createStatement();
               ResultSet rec = st.executeQuery(
                    "SELECT Title, ContactID, First, Last, Dear FROM Contacts "
                    + "WHERE (Title='Mr') ORDER BY ContactID");
          /*     ResultSet rec2 = st.executeQuery(
                    "SELECT Subject, ContactID FROM Calls "
                    + "WHERE (Subject Is Not Null) ORDER BY ContactID");
          System.out.println("\nFirst Name\tSurname\t\tNick Name\t\tSubject\n");
          while(rec.next()) {
               System.out.println(rec.getString(3) + "\t\t" + rec.getString(4) + "\t\t" + rec.getString(5) /* + rec2.getString(1) */ );
          st.close();
          catch (SQLException s) {
               System.out.println("SQL Error: " + s.toString() + " " + s.getErrorCode() + " " + s.getSQLState());
          catch (Exception e) {
               System.out.println("Error: " + e.toString() + e.getMessage());
class PassDB extends javax.swing.JFrame implements ActionListener {
     String username = "", password = "";
     JTextField uname = new JTextField(10);
     JTextField pword = new JTextField(10);
     // JPasswordField pword = new JTextField(10);
     PassDB() {
          super("duBe's database logon");
          setSize(220, 160);
          setDefaultCloseOperation(EXIT_ON_CLOSE);
          JPanel pane = new JPanel();
          JLabel unameLabel = new JLabel ("Username: ");
          JLabel pwordLabel = new JLabel ("Password: ");
          JButton submit = new JButton("OK");
          submit.addActionListener(this);
          pane.add(unameLabel);
          pane.add(uname);
          pane.add(pwordLabel);
          pane.add(pword);
          pane.add(submit);
          setContentPane(pane);
          setVisible(true);
     public void actionPerformed(ActionEvent evt) {
          PassDB clicked = (PassDB)evt.getSource();
          username = uname.getText();
          password = pword.getText();
This code generates two errors, stating:
C:\Java_progs>javac DBQuery1.java
DBQuery1.java:14: non-static variable username cannot be referenced from a static context
Connection conn = DriverManager.getConnection(data, "" +
username, "" + password);
^
DBQuery1.java:14: non-static variable password cannot be referenced from a static context
Connection conn = DriverManager.getConnection(data, "" +
username, "" + password);
                ^
2 errors*****************************
The code works when I remove the reference to the variables "username" & "password" in Connection "conn" call & replace them with the actual username & password, but this is not exactly what I was after. I was hoping to make the program responsive to each individual user, not set in code.
I also would like to make the program pause after the call in "main" to "PassDB" to wait for "PassDB" to exit before continuing.
I would also like to make "PassDB" destroy itself after the "OK" button is pressed & the "username" & "password" set.
If that isn't enough for you, I would really like the program to search 2 different database tables, return their values & compare them to be sure that they are the same.
When I try & search 2 different tables, as in:
ResultSet rec = st.executeQuery(
                    "SELECT Title, ContactID, First, Last, Dear FROM Contacts "
                    + "WHERE (Title='Mr') ORDER BY ContactID");
               ResultSet rec2 = st.executeQuery(
                    "SELECT Subject, ContactID FROM Calls "
                    + "WHERE (Subject Is Not Null) ORDER BY ContactID")javac tells me that "ResultSet" is set to null 0
As always, I am extremely appreciative of any assistance you are able to offer.
Kind regards
duBedat
[email protected]

This is where I'm at now:
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.sql.*;
public class DBQuery {
     static String username = "" ;
     static String password = "" ;
     public static void main(String[] arguments) {
     PassDB UPass = new PassDB();
     String data = "jdbc:odbc:JavaTestDataBase";
     try {
          Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
          Connection conn = DriverManager.getConnection(data, "" + DBQuery.username, "" + DBQuery.password);
          Statement st = conn.createStatement();
          ResultSet rec = st.executeQuery(
          "SELECT Title, ContactID, First, Last, Dear FROM Contacts "
          + "WHERE (Title='Mr') ORDER BY ContactID");
          /*     ResultSet rec2 = st.executeQuery(
               "SELECT Subject, ContactID FROM Calls "
               + "WHERE (Subject Is Not Null) ORDER BY ContactID");
          System.out.println("\nFirst Name\tSurname\t\tNick Name\t\tSubject\n");
          while(rec.next()) {
               System.out.println(rec.getString(3) + "\t\t" + rec.getString(4) + "\t\t" + rec.getString(5) /* + rec2.getString(1) */ );
          st.close();
     catch (SQLException s) {
          System.out.println("SQL Error: " + s.toString() + " " + s.getErrorCode() + " " + s.getSQLState());
     catch (Exception e) {
          System.out.println("Error: " + e.toString() + e.getMessage());
class PassDB extends javax.swing.JFrame implements ActionListener {
     static boolean getOut = false;
     JTextField uname = new JTextField(10);
     JTextField pword = new JTextField(10);
     // JPasswordField pword = new JTextField(10);
     public PassDB() {
          super("duBe's database logon");
          setSize(220, 160);
          setDefaultCloseOperation(EXIT_ON_CLOSE);
          JPanel pane = new JPanel();
          JLabel unameLabel = new JLabel ("Username: ");
          JLabel pwordLabel = new JLabel ("Password: ");
          JButton submit = new JButton("OK");
          submit.addActionListener(this);
          pane.add(unameLabel);
          pane.add(uname);
          pane.add(pwordLabel);
          pane.add(pword);
          pane.add(submit);
          setContentPane(pane);
          while(getOut == false)
               setVisible(true);
     public void actionPerformed(ActionEvent evt) {
          PassDB clicked = (PassDB)evt.getSource();
          DBQuery.username = uname.getText();
          DBQuery.password = pword.getText();
          getOut = true;
}          Any advice is greatly appreciated
duBe

Trying to programmatically set the data-source for a Crystal reports report.

I've got the following existing procedure that I need to add to in order to programmatically set the data-source (server, database, username, and password) for a Crystal reports report.
I added the connectionInfo parts, but can’t figure out how to attach this to the existing
this._report object.
This is currently getting the connection data from the report file, but I now need to populate this connection data from a 'config.xml' text file.
Am I trying to do this all wrong?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using CrystalDecisions.CrystalReports.Engine;
using WGS.Reports.Reports;
using CrystalDecisions.Shared;
using WGS.Reports.Forms;
namespace WGS.Reports
public class ReportService
ReportClass _report;
ParameterFields paramFields;
ConnectionInfo connectionInfo; // <- I added this
public ReportService()
public void DisplayReport(string reportName, int allocationNo)
if (reportName.ToLower() == "allocationexceptions")
this._report = new AllocationExceptions();
PrepareConnection(); // <- I added this
PrepareAllocationExceptionReport(allocationNo);
this.DisplayReport();
private void PrepareConnection() // <- I added this
//test - these will come from the config.xml file
this.connectionInfo = new ConnectionInfo();
this.connectionInfo.ServerName = "testserv\\test";
this.connectionInfo.DatabaseName = "testdb";
this.connectionInfo.UserID = "testuser";
this.connectionInfo.Password = "test";
this.connectionInfo.Type = ConnectionInfoType.SQL;
private void PrepareAllocationExceptionReport(int allocationNo)
this.paramFields = new ParameterFields();
this.paramFields.Clear();
ParameterField paramField = new ParameterField { ParameterFieldName = "@AllocationNo" };
ParameterDiscreteValue discreteVal = new ParameterDiscreteValue { Value = allocationNo };
paramField.CurrentValues.Add(discreteVal);
paramFields.Add(paramField);
private void DisplayReport()
frmReportViewer showReport = new frmReportViewer();
showReport.ReportViewer.ReportSource = this._report;
showReport.ReportViewer.ParameterFieldInfo = paramFields;
showReport.ShowDialog();
showReport.Dispose();
Any help would be much appreciated.

Hi Garry,
Please post SAP Crystal Reports questions in their own forums here:
SAP Crystal Reports, version for Visual Studio
We don't provide support for this control now. Thanks for your understanding.
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

PROBLEM TRANSFERRING   MULTIPLE   DATA ENTRIES    FOR ONE KEY-FIELD.

DEAR   EXPERTS ,
   I HAVE TRANSFERRED DATA FROM THE FINAL INTERNAL TABLE OF MY ABAP REPORT (NOT ALV) TO CUSTOM Z-TABLE CREATED IN SE11.
BUT MY PROBLEM IS : I   COULD NOT   TRANSFER MULTIPLE   DATA ENTRIES   FOR A PARTICULAR FIELD.
FOR EXAMPLE : IN TABLE EKKO THERE ARE   FOUR EBELN-4900006375 AND FOR THAT DIFFERENT EBELP S ARE
PRESENT. I COULD TRANSFER ONLY THE FIRST ENTRY , THAT IS : EBELN - 4900006375 AND   EBELP - 0010,
AFTER THAT THE ZTABLE IS NOT GETTING UPDATED TO EBELN-4900006375 FOR EBELP - 0020 AND SO ON.
I HAVE TRIED ALL THE ' MODIFY, INSERT, UPDATE ' STATEMENTS. I HAVE USED AT - USERCOMMAND - HIDE AND CHECKBOXES.
   PLEASE   SUGGEST   A   SAMPLE   CODE   FOR   THIS.
Moderator message: please post again, but not in all upper case.
[Rules of engagement|http://wiki.sdn.sap.com/wiki/display/HOME/RulesofEngagement]
Edited by: Thomas Zloch on Jun 19, 2011 10:05 PM

There are actually 5 queries in this report now. From what I understand about a union query, I don't think it will work here because the data being returned in each of the queries is so different. I basically need to know how to make all the criteria for each individual to be displayed before proceeding to the next data set, which will include the same data as the first, but for the next employee, and so on. I need to basically create a repeating frame with each individual's respective data I guess, but every time I do, it tells me that it's referencing an invalid group.

HT1595 Problem setting network date/time?

I've connected appleTV2 to a public available wi-fi network (Openzone). It is unable to set the date and time whcih means the device is unable to connect to my apple ID or be used at all. network connection is fine from other devices (iphone, ipad, PC). anyone got any suggestions? Thanks.

Welcome to the Apple Community.
Assuming this is not the first time you have used your Apple TV
You might try restarting the Apple TV by removing ALL the cables for 30 seconds.
Also try restarting the router.
If the problem persists, try a restore, you may want to try the previous procedures several times before doing this.
If this is a new Apple TV, it may also be that your network router is not allowing access to the timeserver, check that your router allows access over port 123.

How to do filter data after Data Collection for Oracle PS

Hi Experts,
I have a critical issue on the "Data Volume" after data collection program.
If I give this big chunk of data as a input to PS, then integration program is not taking this High volume of data into PS.
Hence now I have a critical task to reduce number of Data.
In this front, first I need to delete some items which are not required for PS in the base table(In MSC_SYSTEM_ITEMS) after collection program, but I do not know the dependent entities which also has to be taken care in MSC side after collection, If I delete the itrems from MSC_SYSTEM_ITEMS..
Hence I require help on this data filter issue.
Appreciating anyone's help in advance...
Thanks & Regards,
Bharathram N

Hello,
Have you looked at the request Purge ODS Data of Collections? Using this you can delete all the collected data for a specific instance id.
Once that is done you can run a complete refresh for all entities for all enabled organizations. This will ensure that old data pertaining to disabled orgs is removed.
Additionally in 11.5.10 (collections rup 32 and later) there is a feature implemented which does not pull data from disabled organizations into the source snapshots. This in itself does not make a difference for the ODS data, but does give better performance in the refresh snapshots.
One thing to take into account if your purge the ODS data that in the event when you still have ascp plans that are based on this data, then you may end up with no data being displayed in the workbench (because the lid tables have been purged). Here you would have to rerun the plans.
best regards
Geert

Asking for some guidance for a performance problem

Hello,
We found some very slow sql queries, and they all share one pattern, large query number in trace file.
For example:
call count cpu elapsed disk query current rows
Parse 1 0.00 0.00 0 0 0 0
Execute 150 0.05 0.09 0 0 0 0
Fetch 300 442.38 464.74 170 86428872 0 150
total 460 442.43 464.83 170 86428872 0 150
a. 300 Fetch to get only 150 rows back
b. 170 disk visits, 86428872 consistent reads
Since it is a complicate application (indexes/stats are all up-to-date), I just want to ask for some general guidance for investigating the problem.
1. What/where should I look for?
2. Any possible explanation for the huge query number?

query = consistent gets = making the result set consistent
It could be because the query runs a long time and thus has to read from the undo alot.
Can you give us a peek at the sql and plan? I'd imagine it's doing some full table scans. If it's OLTP sql then you might be able to get a better plan, but if it's a report that's scanning every row in a table you might be stuck with it.

How can I set a date range for a Date/Time Field in LiveCycle ES2?

I need to set a specific date range for a form set up in LiveCycle ES2. Cannot see where i allows me to do that without a code entered.

Hi,
You can't set a start and end date for the date/time field, the best you can do is validate the range after the user has selected it.
An alternate is to make your own date / time field, like this sample http://adobelivecycledesignercookbookbybr001.blogspot.com.au/2013/05/an-alternative-date-p icker-for.html
Regards
Bruce

Looking for the right collection for a specific problem

Hi guys,
Just wanted to say great work you guys are doing here. the forum has definitely helped me a lot in the past.
I am currently having a problem selecting a collection for this specific application that i have to do for my final year project. I am looking for a way to store a bunch of strings a collection that uses the actual string as its key and stores the number of times that the string have been added to the collection as its value.
so for e.g
String a = "a";
String b = "b";
String a2 = "a";
String c = "c";
/**Here is where i create my collection */
// Collection mycollection = new collection
// mycollection.add(a);
// mycollection.add(b);
// mycollection.add(a2);
// mycollection.add(c);
/**if you request for the string a or a2 the collection should return the number of times that string appead e.g */
// System.out.println(mycollection.get(a));
//this should print 2
Does such a collection exist? I remember using something similar when i was learning c++ but i havent been able to find what i am looking for in the API documentation. Please help.
Frank

I think what the OP is looking for is probably a Bag. Unfortunately the default collections framework does not provide a Bag interface or implementation. You could look into using the Jakarta Commons Collection, they do provide those (look at the API[url] as well.

Using a hashmap for caching -- performance problems

Hello!
1) DESCRIPTION OF MY PROBLEM:
I was planing to speed up computations in my algorithm by using a caching-mechanism based on a Java HashMap. But it does not work. In fact the performance decreased instead.
My task is to compute conditional probabilities P(result | event), given an event (e.g. "rainy day") and a result of a measurement (e.g. "degree of humidity").
2) SITUATION AS AN EXCERPT OF MY CODE:
Here is an abstraction of my code:
====================================================================================================================================
int numOfevents = 343;
// initialize cache for precomputed probabilities
HashMap<String,Map<String,Double>> precomputedProbabilities = new HashMap<String,Map<String,Double>>(numOfEvents);
// Given a combination of an event and a result, test if the conditional probability has already been computed
if (this.precomputedProbability.containsKey(eventID)) {
if (this.precomputedProbability.get(eventID).containsKey(result)) {
return this.precomputedProbability.get(eventID).get(result);
} else {
// initialize a new hashmap to maintain the mappings
Map<String,Double> resultProbs4event = new HashMap<String,Double>();
precomputedProbability.put(eventID,resultProbs4event);
// unless we could use the above short-cut via the cache, we have to really compute the conditional probability for the specific combination of the event and result
* make the necessary computations to compute the variable "condProb"
// store in map
precomputedProbabilities.get(eventID).put(result, condProb);
====================================================================================================================================
3) FINAL COMMENTS
After introducing this cache-mechanism I encountered a severe decrease in performance of my algorithm. In total there are over 94 millions of combinations for which the conditional probabilities have to be computed. But there is a lot of redundancy in this set of feasible combinations. Basically it can be brought down to just about 260.000 different combinations that have to be captured in the caching structure. Therefore I expected a significant increase of the performance.
What do I do wrong? Or is the overhead of a nested HashMap so severe? The computation of the conditional probabilities only contains basic operations.
Only for those who are interested in more details
4) DEEPER CONSIDERATION OF THE PROCEDURE
Each defined event stores a list of associated results. These results lists include 7 items on average. To actually compute the conditional probability for a combination of an event and a result, I have to run through the results list of this event and perform an Java "equals"-operation for each list item to compute the relative frequency of the result item at hand. So, without using the caching, I would estimate to perform on average:
7 "equal"-operations (--> to compute the number of occurences of this result item in a list of 7 items on average)
plus
1 double fractions (--> to compute a relative frequency)
for 94 million combinations.
Considering the computation for one combination (event, result) this would mean to compare the overhead of the look-up operations in the nested HashMap with the computational cost of performing 7 "equal' operations + one double fration operation.
I would have expected that it should be less expensive to perform the lookups.
Best regards!
Edited by: Coding_But_Still_Alive on Sep 10, 2008 7:01 AM

Hello!
Thank you very much! I have performed several optimization steps. But still caching is slower than without caching. This may be due to the fact, that the eventID and results all share long common prefixes. I am not sure how this affects the computation of the values of the hash method.
* Attention: result and eventID are given as input of the method
Map<String,Map<String,Double>> precomputedProbs = new HashMap<String,Map<String,Double>>(1200);
HashMap<String,Double> results2probs = (HashMap<String,Double>)this.precomputedProbs.get(eventID);
if (results2Probs != null) {
       Double prob = results2Probs.get(result);
       if (prob != null) {
             return prob;
} else {
       // so far there are no conditional probs for the annotated results of this event
       // initialize a new hashmap to maintain the mappings
       results2Probs = new HashMap<String,Double>(2000);
       precomputedProbs.put(eventID,results2Probs);
* Later, in case of the computation of the conditional probability... use the initialized map to save one "get"-operation on "precomputedProbs"
// the variable results2probs still holds a reference to the inner HashMap<String,Double> entry of the HashMap "precomputedProbs"
results2probs.put(result, condProb);And... because it was asked for, here is the computation of the conditional probability in detail:
* Attention: result and eventID are given as input of the method
// the computed conditional probabaility
double condProb = -1.0;
ArrayList resultsList = (ArrayList<String>)this.eventID2resultsList.get(eventID);
if (resultsList != null) {
            // listSize is expected to be about 7 on average
            int listSize = resultsList.size();
            // sanity check
            if (listSize > 0) {
                // check if given result is defined in the list of defined results
                if (this.definedResults.containsKey(result)) {
                    // store conditional prob. for the specific event/result combination
                    // Analyze the list for matching results
                    for (int i = 0; i < listSize; i++) {
                        if (result.equals(resultsList.get(i))) {
                            occurrence_count++;
                    if (occurrence_count == 0) {
                        condProb = 0.0;
                    } else {
                        condProb = ((double)occurrence_count) / ((double)listSize);
                    // the variable results2probs still holds a reference to the inner HashMap<String,Double> entry of the HashMap "precomputedProbs"
                    results2probs.put(result, condProb);
                    return condProb;
                } else {
                    // mark that result is not part of the list of defined results
                    return -1.0;
            } else {
                throw new NullPointerException("Unexpected happening. Event " + eventID + " contains no result definitions.");
        } else {
            throw new IllegalArgumentException("unknown event ID:"+ eventID);
        }I have performed tests on a decreased data input set. I processed only 100.000 result instances, instead of about 250K. This means there are 343 * 100K = 34.300K = 34M calls of my method per each iteration of the algorithm. I performed 20 iterations. With caching it took 8 min 5 sec, without only 7 min 5 sec.
I also tried to profile the lookup-operations for the HashMaps, but they took less than a ms. The same as with the operations for computing the conditional probability. So regarding this, there was no additional insight from comparing the times on ms level.
Edited by: Coding_But_Still_Alive on Sep 11, 2008 9:22 AM
Edited by: Coding_But_Still_Alive on Sep 11, 2008 9:24 AM

Problem setting up data collection for a performance problem

Similar Messages

Maybe you are looking for