Environment getStats() in multiple processes
Here's my question. Can I open a read-only Environment to report statistics from a one process and see the same stats that I would see if I called getStats from the Environment running on another process that has the database open for writing?
Here's why I am asking. I am trying to troubleshoot a performance problem with our application that depends on a third-party project that uses Java Berkeley DB for storage. We suspect a BDB tuning problem, as we are seeing explosive BDB log file growth at the same time as response times for gets and puts degrade.
I have read in the documentation and on this forum that periodically calling Environment.getStats() and analyzing the output is the first step in any tuning effort. The trouble is that the third-party project that we use does not implement this call. We could crack open the source code and add it, but that's a last resort for us.
Instead, I figured I could write a little script that runs in a separate process and periodically creates a read-only Environment and calls getStats(). This seems to work. I am collecting data, but some of the numbers don't seem accurate. For instance, all the cleaner stats always report 0. This makes me wonder if my approach is sound.
EnvironmentConfig environmentConfig = new EnvironmentConfig()
environmentConfig.setTxnNoSync(true)
environmentConfig.setAllowCreate(false)
environmentConfig.setTransactional(true)
environmentConfig.setReadOnly(true)
environment = new Environment(new File(bdbDir), environmentConfig)
StatsConfig statsConfig = new StatsConfig()
statsConfig.setClear(true)
statsConfig.setFast(fast)
println "----- ${new Date()} -----"
println environment.getStats(statsConfig).toString()
Hi Linda,
I modified the Voldemort source code to register the JEMonitor MBean, which allows me to get statistics. I also included a configurable way to periodically call Environment.getStats() from within the application. Both approaches seem to be working well and give me the data I need. I am going to contribute it back to the Voldemort project when things settle down. For now I am going to be busy interpreting the results we have collected.
--Jon
Similar Messages
-
Read-only snapshot of writable environment in the same process
According to the documentation, if you open a JE environment from multiple processes, then all processes except the writer process get a read-only, point-in-time snapshot of the database that won't reflect any writes processed in the database. Is it possible to achieve this same behavior within the same process that already has the environment open for writing?
I have an application for which I would like to have two backup mechanisms: one which performs a simple file copy of the database files as is documented elsewhere, and a second one which cursors through the data and serializes it to a file that could potentially be processed for other purposes and may be accessed even if the database itself becomes damaged or corrupt. It would be very nice if this second backup mechanism of exporting the data could get the point-in-time snapshot (so that I don't have to worry about getting inconsistent data if the database environment is updated during the export) but didn't require a separate process.You've asked an interesting question! I'm afraid that the answer is that it's not possible in BDB JE today.
What you'd really like for your second case is part of the type of functionality provided by MVCC (Multi-Version Concurrency Control). In case you're not familiar with that term, MVCC is the database feature which guarantees that all reads made in a transaction will see a consistent snapshot of the database and that the transaction will successfully commit only if no updates it has made conflict with any concurrent updates made since that snapshot. Unfortunately, while our sister product, BDB (C) has that feature, we do not yet support it.
I think your alternatives are to do achieve your point in time snapshot from the file copy based backup, from the second process, or from an application level mechanism that locks out other updates.
Regards,
Linda -
Multiple processes accessing a replicated database
Hi
I am after some help with multiple processes and replicated databases.
I have a primary and secondary database replicated across a pair of servers and this seems to be working well. I'm trying to run another process on one of the machines that opens the environment and databases to view and/or modify the data.
The problem is that when I run this process it causes some sort of corruption such that the server process on the same box gets a DB_EVENT_PANIC the next time it accesses the database. I would like to understand what I am doing wrong.
The servers and standalone process all use the same code to open and close the environment and databases (see below). Just calling
open_env();
open_databases();
close_databases();
close_env();
in the utility process causes DB_EVENT_PANIC in the server process.
Can anybody spot what I am doing wrong? I am using DB Version 4.7
Thanks
Ashley
open_env() {
db_env_create(&dbenv, 0);
dbenv->app_private = &my_app_data;
dbenv->set_event_notify(dbenv, event_callback);
dbenv->rep_set_limit(dbenv, 0, REPLIMIT);
dbenv->set_flags(dbenv, DB_AUTO_COMMIT | DB_TXN_NOSYNC, 1);
dbenv->set_lk_detect(dbenv, DB_LOCK_DEFAULT)
int flags = DB_CREATE | DB_INIT_LOCK |
DB_INIT_LOG | DB_INIT_MPOOL |
DB_INIT_TXN | DB_RECOVER | DB_THREAD;
flags |= DB_INIT_REP;
dbenv->repmgr_set_local_site(dbenv, listen_host, port, 0);
dbenv->rep_set_priority(dbenv, 100);
dbenv->repmgr_set_ack_policy(dbenv, DB_REPMGR_ACKS_ONE);
for (x = 0; x < num_peers; x++) {
dbenv->repmgr_add_remote_site(dbenv, peers[x].name, peers[x].port, &peers[x].eid, 0);
dbenv->rep_set_nsites(dbenv, num_peers + 1);
dbenv->open(dbenv, ".", flags, S_IRUSR | S_IWUSR);
dbenv->repmgr_start(dbenv, 3, DB_REP_ELECTION);
sleep(SLEEPTIME);
close_env() {
dbenv_p->txn_checkpoint(dbenv_p, 0, 0, 0);
dbenv_p->close(dbenv_p, 0);
open_databases() {
db_create(&dbp, dbenv_p, 0)
flags = 0;
if (app_data->is_master)
flags |= DB_CREATE;
dbp->open(dbp, NULL, "primary", NULL, DB_HASH, flags, 0);
... Wait for db if slave and ENOENT ...
primary = dbp;
dbp->open(dbp, NULL, "secondary", NULL, DB_BTREE, flags, 0);
... Wait for db if slave and ENOENT
secondary = dbp;
while (app_data->client_sync) {
sleep(SLEEPTIME);
close_databases() {
secondary->close(secondary, 0);
primary->close(primary, 0);
dbenv_p->txn_checkpoint(dbenv_p, 0, 0, 0);
}Running recovery (DB_RECOVER flag to env->open()) must be done only in the first process to open the environment.
This is a general rule of Berkeley DB, not specific to replication. You can read more about it in the Reference Guide, on the page entitled "Architecting Transactional Data Store applications". -
Multiple processes for a host instance on one server
Hi,
One of the prod environment server memory utilization is getting very high every few days. I searched a bit and found that there are many processes for "BTSNTSvc64.exe" in the task manager. Whenever we restart the host instance a new process is
getting created and the previous process is not getting killed.
This caused multiple processes in task manager and every process is occupying some memory. The process is actually not active but it is creating some bad sectors which is causing the Memory utlization to become high.
I am not able to kill the zombie processes either from Task manager. Server reboot is the option but wanted to know a permanent fix.
Can you please tell me the reason as to why this is happening and also the solution?How many BizTalk Hosts do you have defined on the system? If you have multiple Hosts defined then you're likely to see one host instance associated with each of the hosts. Each host instance is a service and in the "Task Manager" - Processes you will
see multiple BTSNTSvc64.exe OR BTSNTSvc.exe * 32. This is OK is you have multiple host instances defined on your system (Services MMC will show you BizTalk Service BizTalk Group : BizTalk Host) and BizTalk Administration Console ->BizTalk Group ->Platform
Settings -> Host Instances will also show you the different host instances. Each instance is a multi threaded process running different aspects of the BizTalk Framework/Components. So each process will occupy memory if it is "started". All this is normal
behavior. If you're facing performance issues, you might want to ask the solution designer on their rationale for the number of hosts instances.
So when you stop the service, it does not stop? goes into a non-responsive state? and creates another instance?
Regards. -
BOM Explosion for Multiple Process Orders
Hi,
I hope someone can assist me with the following: Is there a report I can run or transaction I can do to have BOM explosion for multiple Process Orders? I would like to input process orders for a week and see the BOM details per PO.
ThanksDear Sukendar,
1.For this requirement you have to go for a Z report and you can give the Functional Spec's to prepare
this report to your ABAP consultant.Prepare the input format,logic of the program and the output format.
In the logic part you can make use of this Functional Modules's CSAP_MAT_BOM_READ or
CSEP_MAT_BOM_READ or CS_BOM_EXPLOSION or CS_BOM_EXPL_MAT_V2 .
2.Using CEWB helps you to identify all the material that's having the BOM,but here the report does not
shows you the level by level by BOM for a FERT.
Check and revert back.
Regards
Mangalraj.S -
Target Vs Acual Cost Comparison for multiple process orders
Dear SAPians,
I want to take a report for target vs actual cost for all process orders and product cost collectors created in a particular period. I have checked the standard report KKBC_ORD that serve my purpose but for a single order. I want the same report for multiple process orders and product cost collectors.
Is there any standard report or should I create a report painter report. I have tried to create a report painter report based on library 7KO, but target cost was not showing for selected order.
Kindly suggest.
ShiraziTarget Costs are available in library 601 for internal orders (and can be used for process/production orders).
Regards,
Adrian -
How to delete multiple process at a time in a Process chain
Hi,
Is there any option to delete multiple processes at once in a Process chain in BI 7.0 ?Hi Swetha,
If u want to stop a perticular proces go to sm37 give user ID (If process chain is schduled then give user ID as ALEREMOTE and if u have triggered it manually then give ur ID) select time and other options like type of process means whether it is active, canceled, ready etc. U will get ur running job.
In sm37 at top of the screen u will get options to cancel or stop active job. But before deleting the active job just
check the job log and also check the business impact.
Also u can not use one process variant in multiple
Regards,
NR -
Create a CR for multiple processing with me as the current processor
Hi
How to create a change request for multiple processing with myself as the current processor.
I have created a change request for multiple processing and the ystem throws me the error that "There are no suitable change requests"
Display help shows the below
For the upload you need a change request that has the status Considered
and Approved or To Be Revised and ideally has a type that does
not have the property Object List Is Binding. If the
change request does have this property, all the entities that are uploaded must
already be assigned to this object list.
Please can someoen guide me on how to fix this.
I found few threads related to this but cudint understand on how to assign a status to CR
Thanks,
SharmaHi Prateek,
Please can anyone give any pointers on how to set the status of change request (for multiple processing):-
For assigning the statuses to the change request:-
Go to t code:- MDGIMG->General settings>Process Modeling>Change Request>Edit status of change request.
the path you mentioned is to create and edit the existing Change request statuses, what I am looking for is when I create a CR for Multi record processing and try to upload a file I get the error "There are no suitable change requests". and when I check the Status of the change request I created I see that the status is Changes to be executed (Set automatically).
how can i set myself as the processor.
tcode: MDGIMG->General settings>Process Modeling>Change Request>Create Change request type. Ucheck the single object checkbox and assign your userid for that CR step.
Single object is already Unchecked and I am using BRF+ so when I am the processor First approver is say X, I am not sure if we have any place where we can set ourselves as the user. Please let me know if I missed sumting
Thanks
Hi Shephalika,
Yes I am trying to upload a file and Single object is Unchecked in the Change request type.
Thanks,
Sharma -
Launching Multiple Processes With JWS
Is it possible to launch multiple processes with JWS? I'd like to specify in the jnlp 2 applications to run. Each application must run in its own java.exe process. Why you ask?
I've created a mission critical application in JWS that requires the application to be active at all times. If the application crashes, the second application detects the crash and restarts the jnlp.
How can this be accomplished? I'd like to stay away from created another JVM via JNI if possible.billmanhillman wrote:
Invoking javaws from the application URL is not scalable. LOL! When people talk about 'scalable' they are usually referring to numbers of the order of '2000', not '2'.
To cover something mentioned in another reply..
Can you get the path to the jar by getResource()?Nope. Sun considers it not to be the business of apps., to know the location of the classes on disk. They have gone to considerable trouble to make it unavailable to applications. -
Information Text for multiple process.
Hi All,
I have a page with multiple process.
Process 1. Upload multiple files to table using Process Row
Process 2. The Files are rearranged to another table, ready for analysis.
Process 3. Uploaded file is analysed for errors 1,
Process 4. Analyse for Error 2.
The problem is when the upload file size growth with more number of uploads. It take quite a longer time to upload and then analyse.
Is there a way i can pass back a value to the Page and display what process has been done and what process is running currently.
or Any ideas ?
Thanks in Advance
BalaHi,
You can create background process
See if this help you
http://download.oracle.com/docs/cd/E14373_01/appdev.32/e11838/advnc.htm#BABGHIFB
Br, Jari -
Implement Process order if there are multiple processes in production of FP
Hi Experts,
I have another scenario in which for a production of final product i need to go through
multiple processes.
For Eg.
I have a raw product suppose as rubber and i want to make an Tyre
for which i have to do 3 procedures followed one by the other in sequence
So can u please explain me how can i achieve the same.
Thanking you
Regards,
Ashish P.
Edited by: Ashish@123 on Nov 14, 2011 2:06 PM") expected" means exactly what it says. You must be missing brackets there. Correct those and you will prpbably find that the "Illegal start of expressions" will go away too.
Rachel -
Sharing in-memory database among multiple processes
From the bdb document, due to disk-less operation, it is not possible to share in-memory among multiple processes.
I am wondering, if it is possible to use DB_ENV but without DB->sync or any other sync() function call, such that the database, cache, etc in the shared region not flushed to the disk, to achieve disk-less operation. Please share your thought.See "Using the Resource Capping Daemon on a System With Zones Installed" in SystemAdministration Guide: Solaris Containers-Resource
Management and Solaris Zones. You can get the manual from docs.sun.com.
Chapter 10 discuss physical memory control.
Regards -
Newbie: why are there multiple process started?
Hi I have a simple question to ask...
Q1. Why are there multiple process for webserver instance and http-admserv for a default installed web server? Please refer below:
$ ps -ef | grep web
root 931 930 0 Jan 21 ? 0:00 webservd -r /opt/SUNWwbsvr -d /opt/SUNWwbsvr/https-admserv/config -n https-adms
root 930 1 0 Jan 21 ? 0:00 ./webservd-wdog -r /opt/SUNWwbsvr -d /opt/SUNWwbsvr/https-admserv/config -n htt
root 948 1 0 Jan 21 ? 0:00 ./webservd-wdog -r /opt/SUNWwbsvr -d /opt/SUNWwbsvr/https-mta01.mydomain.com
root 932 931 0 Jan 21 ? 0:03 webservd -r /opt/SUNWwbsvr -d /opt/SUNWwbsvr/https-admserv/config -n https-adms
root 949 948 0 Jan 21 ? 0:00 webservd -r /opt/SUNWwbsvr -d /opt/SUNWwbsvr/https-mta01.mydomain.com/config
root 950 949 0 Jan 21 ? 42:09 webservd -r /opt/SUNWwbsvr -d /opt/SUNWwbsvr/https-mta01.mydomain.com/config
jes 6203 5994 0 10:52:36 pts/2 0:00 grep web
I understand we need 1 watch dog for web server and 1 for admin server but why 2 process for "https-mta01.mydomain.com" and "https-admserv"?
Q2. This is in relation to Q1. If I need to turn on CA Agent for monitoring, which process should I monitor for web server instance and also for http-admserv?
Newbie.Ans1) That is by design and default behavior. There is one watch-dog process and 2 webservd processes that get started by default. The first webservd is called primordial and the second child webservd is the one that actually serves all the HTTP requests.
Example:
# ptree 6898
6898 ./webservd-wdog -r /space/iws61sp2 ...
6899 webservd -r /space/iws61sp2 -d ..
6900 webservd -r /space/iws61sp2 -d ...
In this example, the pid 6898 is called the watchdog process,
6899 webservd process is called the primordial process
and the pid 6900 is called the child webservd process that actually serves all the HTTP requests.
Ans2) You would monitor the child webservd process.
Thanks
Manish -
Multiple processes in the workflow
Hello Everyone.
Why do we require multiple Processes in a single Workflow? I see that in the seeded workflows, if i open a process, in the flow there will be another process which again might contain several different process attached to the same flow.
In realtime when do we require to use this functionality?
ThanksIt is not a requirement - it is simply a way of modularizing - just like how any piece of code can call other pieces of nested codes.
Srini -
Same JVM multiple processes?
Summary: Does any one know how to get a JVM to be shared across different Windows processes, as follows:
Context: I have created a Java ActiveX component. This is instantiated and used as an object embedded in HTML. The HTML will always be executed in Internet Explorer. When I have multiple windows open in the same Internet Explorer process with the same HTML code the static variables are shared by all the Java ActiveX components. If I have multiple process of Internet Explorer open each with the same HTML code the static variables are not shared because different instances of the JVM are running in each Internet Explorer process.
Question: Is it possible to do any of the following:
- Get multiple instances of the JVM to run as multiple Threads i.e. not as separate processed with separate memory segments?
- Get multiple instances of the JVM to share variables?
- Prevent multiple instances of the JVM being created so that all Java objects run in the same JVM regardless of the process that instantiated them.
This is possible in C++ ActiveX components using shared segments across different Internet Explorer processes, as follows:
// in DLL .cpp file
#pragma data_seg (�.mydata�)
int sharedVariable = 1;
#pragma data_seg ( )
#pragma comment(linker, �/SECTION:.mydata, RWS�)Therefore I would expect that if I could achieve a similar approach with Java this would not be prevented by the OS (i.e. Windows) due to memory faults, etc. I know that Java has different (less) memory handling capabilites so this may not be possible but any suggestions would be very much appreciated.
Thanks,
JSummary: Does any one know how to get a JVM to
be shared across different Windows processes, as
follows:
Context: I have created a Java ActiveX
component. This is instantiated and used as an
object embedded in HTML. The HTML will always be
executed in Internet Explorer. When I have multiple
windows open in the same Internet Explorer process
with the same HTML code the static variables are
shared by all the Java ActiveX components. If I have
multiple process of Internet Explorer open each with
the same HTML code the static variables are not
shared because different instances of the JVM are
running in each Internet Explorer process.
Question: Is it possible to do any of the
following:
- Get multiple instances of the JVM to run as
s multiple Threads i.e. not as separate processed
with separate memory segments? No, each instance of the VM is a seperate process. The closest you can come without hacking the VM and/or IE, is running multiple Java programs in the same VM, each in its own thread.
- Get multiple instances of the JVM to share
e variables?
- Prevent multiple instances of the JVM being
g created so that all Java objects run in the same
JVM regardless of the process that instantiated them.
RMI provides some means of inter-VM communication, but this is probably not suitable for your application. The combination of JNI + NIO also provide some capability for the VM to share memory with native processes, but its not immediately obvious to me how to extend this capability to multiple VM's.
>
This is possible in C++ ActiveX components using
shared segments across different Internet Explorer
processes, as follows:
// in DLL .cpp file
#pragma data_seg (�.mydata�)
int sharedVariable = 1;
#pragma data_seg ( )
#pragma comment(linker, �/SECTION:.mydata,
RWS�)Therefore I would expect that if I could achieve a
similar approach with Java this would not be
prevented by the OS (i.e. Windows) due to memory
faults, etc. I know that Java has different (less)
memory handling capabilites so this may not be
possible but any suggestions would be very much
appreciated.
Thanks,
JAlso, you may want to keep in mind that VM's that run in a browser have a SecurityManager installed that restricts them from performing certain restricted operations. For this reason it may be difficult to do anything really ambitious with an applet or, presumably, a Java ActiveX control, though I have no idea what that is.
Good luck.
Maybe you are looking for
-
Please help me. I have already login the store with my apple ID which already filled with billing address but there is still purchase error. What can I do? The billing information that I filled in is correct but when I click purchase it appear error
-
Itunes doesnt detect ipod but has with different ipod
I just bought a used apple ipod after my old one was dropped in my drink and shorted out. I downloaded the ipod updator like it told me after saying that i needed for itunes to detect my ipod and restarted my computer after installing. Even this new
-
I am working on an application. Every time a user (User A) submits an article or a message, the Greenwich time of the submission is stored in the database no matter which the time zone the user (User A) is. Another user (User B who may be in another
-
Why do my firewalls keep turning off?
when i start up my computer windows tells me that firewalls are turned off, and each time i turn them back on, why do they keep turning themselves off? and how can i stop this? this is getting annoying!
-
Hi All I have bought a new lenovo v570 IdeaPad,It is just 3 months since I had bought that and now it is not powering on. I bought it a he US and have now moved o INDIA for good and this model is not supported in INDIA. Any suggestions how I can get