CPU Usage per thread
I have been looking for an accurate way to obtain the cpu usage time per thread in a multi-threaded C,C++ application and I have not been able to find a way to do so. Is there a library which gives you access to this information? Standard time functions like getrusage, time, times, and clock are all per process.
Thanks for your help in advance,
Tom
Shark also does this pretty well, albeit indirectly (% of app CPU time per thread) ...
Regards,
John
Falling You - exploring the beauty of voice and sound
http://www.fallingyou.com
Similar Messages
-
Save location for CPU usage per hour
Hi Gurus:
Can you please tell me where CPU usage per hour (tcode: ST06>Detail Analysis Menu>Previous 24 hours CPU) is permanently stored? I have attached the ST06 output that I wish to see for, say 30 days.
Regards,
Md. Soadrul AminRun an SQL trace for the user execution the transaction ST06 until the end of the selection using the Tx ST05. This will provide you the tables queried by the Tx ST06 to provide you the result.
Regards
RB -
Hi Everyone,
I am currently working with ASA 5585 with several contexts and I would like to have an idea of what is the percentage of the CPU used per context.
I already have the opportunity to do it for the whole ASA (context admin) using the SNMP mib CISCO-PROCES but, unfortunalty, this mib doesn't allow us to know the percentage of used CPU per context.
I was able to know the number of core used per context but not the percentage of the CPU used.
Do you know if it is possible ?
Thank you for your help.
François.Dear Francois,
Here's the answer for the snmp part:
CPU usage on ASA
The CPU usage is displayed per context (prior to 8.1) when queried by SNMP.
For example:
show cpu output:
shweta-asa-5520/coke(config)# show cpu
CPU utilization for 5 seconds = 33.3%; 1 minute: 25.8%; 5 minutes: 8.7%
snmpwalk output:
root@sw8-ilinux root]# snmpwalk -OS -c public -v 2c 10.7.14.61 cpmCPUTotalTable
CISCO-PROCESS-MIB::cpmCPUTotalPhysicalIndex.1 = 1
CISCO-PROCESS-MIB::cpmCPUTotal5sec.1 = Gauge32: 33
CISCO-PROCESS-MIB::cpmCPUTotal1min.1 = Gauge32: 26
CISCO-PROCESS-MIB::cpmCPUTotal5min.1 = Gauge32: 9
The output displayed here is information specific to the coke context.
Prior to 8.1 release, it is not possible to get the total CPU usage of the ASA using SNMP. This is because there is no SNMP agent in the system context. You can get CPU usage for each context present on the ASA.
To get the entire CPU usage you can run the command: show cpu context all on the CLI.
shweta-asa-5520(config)# show cpu context all
5 sec 1 min 5 min Context Name
3.2% 3.2% 3.1% system
0.0% 0.0% 0.0% admin
33.0% 33.1% 29.4% coke
0.0% 0.0% 0.0% pepsi
The sum of all the user context and system context gives the total CPU usage.
However, we did add change this behavior in 8.1 release and will also be present in the 8.2 release. With this change, you will be able to obtain aggregate device CPU load via the admin context. i.e. by polling the admin context, the customer will get the total cpu usage. -
How to find the cpu usage per query for a time period
Hi All,
Is there a way to find the cpu used per query for a given time period?
DB:10.2.0.5
OS:AIX
Thanksuser13364377 wrote:
if there are multiple queries starting at the same time, then what to do?Handle: user13364377
Status Level: Newbie (10)
Registered: Jul 5, 2010
Total Posts: 264
Total Questions: 113 (84 unresolved)
why so many unanswered questions?
clarify your question
same SQL multiple times from different sessions?
or
different SQLs from different sessions? -
Sol. 10 T5120 T2 SPARC CPU utilization per thread...
Hello all:
We have an issue with a t5120 server UltraSPARC T2 (32 cores) . We installed oracle 10g on the global zone. When we run oracle the process cannot use more than 3.2% CPU (It shows in prstat). Also when whe run a cpuhog (while :; do :; done ) we cannot get also more than 3.2% CPU.
When we can get more CPU is connect with the same user two or more times and run the cpu hog script
this way when can get 3.2 CPU for each connection.
We tried using FSS, TS. But no luck We read a lot o zones, cpu-shares, etc.
We are DBA's , so the question is..is there a default cap on CPU use the solaris has?
We found no capping no the global zone !
Your help is appreciated !Single-threaded applications can use no more than 1 CPU at a time. If you have 32 CPUs online, then each CPU is about 3.2% of the system CPU total.
I'm sure Oracle can make use of multiple threads depending on exactly what you're doing.
Darren -
Making a thread so it doesn't consume so much CPU Usage
I have a simple application consisting of merely a timer that counts down from 2 minutes to 0. Running the timer is a simple thread that uses a run() method which has the following form:
public void run() {
while(threadContinue) {
updateTimer();
}And then I start the read like this:
new Thread(this).start();But from viewing my processes in the task manager, it is apparent that this java application is using up a huge amount of my computer's CPU usage. How can I make a thread that is not so heavily run on my computer, per se.
Thank you!Well you're updating the time awfully a lot especially by using a while loop (considering the computer is trying to jam a lot of update executions for just a second), how about inserting a Thread.sleep(<inc. of time>) then have your updateTimer() method update in increments of the inc. you supplied?
-
High number of threads and high CPU usage on a single instance
Without any apparent reason, in some moments of day the website reports an increment of the HTTP Queue Length from an average of 10/20 queued requests to 100/200 and more.
In that period of time the Average Response Time increases but the number of the requests on the website don't. Even increasing the instances it doesn't help to resolve the problem (maybe it just reduced it).
It seems that the problem is not related to the traffic on the site. This issue happens either when the traffic is high or low.
The autoscaling works well, we don't have peak of traffic but always a slow increasing of requests and the CPU is always below the 50% (memory too). The issue is resolved by the swap of the site and sometimes it's resolve by itself after a while.
What I discovered when the issue happens, is that there's an instance (just an instance only), that has an high number of threads and CPU usage on the W3WP.exe process and these values are above the mean than the other instances. Eg, instances are around
50/60 threads and 10/20% of CPU, the instance with the problem has 200 threads and 50/60% of CPU. If I kill the W3WP process on that instance the issue is resolved.
I monitored the HTTP requests, I tried the website extension "Diagnostics as service" but I can't discover anything that can help me to understand the problem. Of course, there are many requests that go in timeout but it is a consequences of a
unresponsive instance.
On the web hosting plan there are only two sites: the production site and its staging which is used for update of the production only and it's always sleeping. Plus, no webjobs are running on this site.
What can I do to gather useful information that can help me to understand the reason of this problem?
Thank you!"Does the instance with the high counts receive more traffic than the other instances? Is it possible that the load balancer not
working the way it should be?"
How can I get that information? I can't see metrics for a specific instance
"Does it always happen to the first instance?"
I will check it better, but in the order given by the Processes panel it's the second instance which has the issue. The scaling is at 2 instances for the most part of the time (it's the minimum).
Maybe one time it was the 3th instance but I'm not sure, I'll give it more attention.
" How long do these moment last?"
The time can be 10 to 30 minutes, but I fix it as soon as I see the problem. It's not the down-scaling to resolve it because in these situations the CPU as well is high so the scaling holds or increases the number of instances.
"- How often do these moments occurs?"
It occurs quite often, 2/4 times a week. -
100% thread pool/CPU usage prevent MII Applications from working
Hello to all,
our MII Applications were running on MII 11.5 (windowsserver 2003, with IIS) without any problems.
We migrated these MII Applications to SAP xMII Version 12.0 SP8, SAP NW 7.00 SP20, Java HotSpot(TM) 64-Bit Server VM (build 1.4.2_22-rev-b03, mixed mode).
Since then we faced several problems that prevent the MII Applications from working, e.g.
- 100% application thread pool usage rate
- 100% CPU usage
- awful lot of http connections
Several sap support calls didn't find the root cause.
Workaround: restart sap system weekly
Does anyone has any idea how to go on?
Thanks
SimoneHi Mike,
sap support told us to install this Java version.
Furthermore SAP Note 716604 says "do not use J2SE 5.0" and "Currently, we recommend 1.4.2_24 b06".
Any more ideas?
Regards,
Simone -
Abysmal After Effects CC 2014 threaded CPU usage
It's pretty saddening (see screenshot below) how ill equipped After Effects is with maximizing CPU usage while rendering. I've run so many tests and configurations with the multiproccessing options in the preferences and have come to the realization it's not even worth using... the amount of time AE takes to set up all the processes, then to start and still have it only use 20%, etc... it's just not worth it. Often (because of third party plugins used in the comp) it will drop the MP rendering anyway because the plugins are not compatible.
At what point, Adobe, will you fix this?Eden, please see Todd and my responses here:
https://forums.adobe.com/thread/1603800
Although we're not sharing specific details right now, the performance-related work we are doing will impact how After Effects utilizes multiple processor cores. -
CPU usage in never ending loop in thread
hello,
In my application I have a thread that establishes tcp/ip connection with
a remote server and receives data. This connection is established and the server is infinitely checked for any data to be received
something like
do{
//establish connection
//get data
//process it
}while(true);problem is the CPU usage hits 100%. Whats the right way to perform such
infinite operations?You might want to consider using my InfoFetcher. It will work on any kind of InputStream (like the one from URLConnection.openStream()) and will passively wait until there is data to be read. I will also post the TimeOut class which you need to get it to compile.
You are welcome to use and modify this class, but please don't change the package or take credit for it as your own work
InfoFetcher.java
=============
package tjacobs.io;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Iterator;
import tjacobs.io.TimeOut.TimeOutCmd;
* InfoFetcher is a generic way to read data from an input stream (file, socket, etc)
* InfoFetcher can be set up with a thread so that it reads from an input stream
* and report to registered listeners as it gets
* more information. This vastly simplifies the process of always re-writing
* the same code for reading from an input stream.
* <p>
* I use this all over
public class InfoFetcher implements Runnable, TimeOutCmd {
public byte[] buf;
public InputStream in;
public int waitTime;
private ArrayList<InputStreamListener> mListeners;
private ArrayList<FetcherListener> mFetcherListeners;
public int got = 0;
protected boolean mClearBufferFlag = false;
public InfoFetcher(InputStream in, byte[] buf, int waitTime) {
this.buf = buf;
this.in = in;
this.waitTime = waitTime;
* @deprecated use fetcherlistener
* @param fll
public void addInputStreamListener(InputStreamListener listener) {
if (mListeners == null) {
mListeners = new ArrayList<InputStreamListener>(2);
if (!mListeners.contains(listener)) {
mListeners.add(listener);
public void addFetcherListener(FetcherListener listener) {
if (mFetcherListeners == null) {
mFetcherListeners = new ArrayList<FetcherListener>(2);
if (!mFetcherListeners.contains(listener)) {
mFetcherListeners.add(listener);
* @deprecated use fetcherlistener
* @param fll
public void removeInputStreamListener(InputStreamListener fll) {
if (mListeners == null) {
return;
mListeners.remove(fll);
public void removeFetcherListener(FetcherListener fll) {
if (mFetcherListeners == null) {
return;
mFetcherListeners.remove(fll);
public byte[] readCompletely() {
run();
return buf;
public int got() {
return got;
/** Override this to implement other implementations
public void timeOut() {
try {
in.close();
catch (IOException iox) {
iox.printStackTrace();
public void run() {
TimeOut to = null;
if (waitTime > 0) {
to = new TimeOut(this, waitTime);
Thread t = new Thread(to);
t.start();
int b;
try {
while ((b = in.read()) != -1) {
if (to != null) to.tick();
if (got + 1 > buf.length) {
buf = IOUtils.expandBuf(buf);
int start = got;
buf[got++] = (byte) b;
int available = in.available();
//System.out.println("got = " + got + " available = " + available + " buf.length = " + buf.length);
if (got + available > buf.length) {
buf = IOUtils.expandBuf(buf, Math.max(got + available, buf.length * 2));
got += in.read(buf, got, available);
signalListeners(false, start);
if (mClearBufferFlag) {
mClearBufferFlag = false;
got = 0;
} catch (IOException iox) {
throw new PartialReadException(got, buf.length);
} finally {
if (to != null) to.stop();
buf = IOUtils.trimBuf(buf, got);
signalListeners(true);
private void setClearBufferFlag(boolean status) {
mClearBufferFlag = status;
public void clearBuffer() {
setClearBufferFlag(true);
private void signalListeners(boolean over) {
signalListeners (over, 0);
private void signalListeners(boolean over, int start) {
if (mFetcherListeners != null) {
Iterator<FetcherListener> i = mFetcherListeners.iterator();
while (i.hasNext()) {
FetcherListener fll = i.next();
if (over) {
fll.fetchedAll(buf);
} else {
fll.fetchedMore(buf, start, got);
if (mListeners != null) {
Iterator i = mListeners.iterator();
InputStreamEvent ev = new InputStreamEvent(got, buf, start);
//System.out.println("got: " + got + " buf = " + new String(buf, 0, 20));
while (i.hasNext()) {
InputStreamListener fll = (InputStreamListener) i.next();
if (over) {
fll.gotAll(ev);
} else {
fll.gotMore(ev);
public static interface FetcherListener {
public void fetchedMore(byte[] buf, int start, int end);
public void fetchedAll(byte[] buf);
==========
Timeout.java
==========
package tjacobs.io;
import java.io.IOException;
import java.io.InputStream;
public class TimeOut implements Runnable {
private long mWaitTime;
private boolean mRunning = true;
private Thread mMyThread;
private TimeOutCmd mTimeOutCmd;
public static final int DEFAULT_WAIT_TIME = 30 * 1000; // 30 Seconds
public static final int NO_TIMEOUT = -1;
public static interface TimeOutCmd {
public void timeOut();
public TimeOut(TimeOutCmd cmd) {
this(cmd, DEFAULT_WAIT_TIME);
public TimeOut(TimeOutCmd cmd, int timeToWait) {
mWaitTime = timeToWait;
mTimeOutCmd = cmd;
public void stop() {
mRunning = false;
mTimeOutCmd.timeOut();
mMyThread.interrupt();
* reset the TimeOut
public void tick() {
if (mMyThread != null)
mMyThread.interrupt();
public void run () {
mMyThread = Thread.currentThread();
while (true) {
try {
Thread.sleep(mWaitTime);
stop();
catch (InterruptedException ex) {
if (!mRunning) {
return;
} -
How to decrease CPU usage when using multiple threads?
I am using vb.net , .NetFramework 2.0 .
My application gets live stock prices from google and updates stocks in database each 10 seconds.
I get this code for multithreading. When application starts updating stocks in database(about 200 stocks) , the update takes up to 3 seconds but it increase CPU usage from 10 % to 70 % or 80 %.
What is the best way to to update database without getting CPU increase to high level?
I observed that all threads works at the same time. how to make each thread wait until the second ends?
This is my code. The problem is in function updateThreaded2().
Please I need quick help. Thanks
Public Function Update2(ByVal l As SortableBindingList(Of NewStockList)) As Long
res = 0
UThread = New System.Threading.Thread(AddressOf UpdateThreaded2)
If Me.URunning = True Then
Else
Try
Me.URunning = True
Interlocked.Exchange(Me.UCount, 0) 'threadsafe method of assigning static value
Interlocked.Exchange(Me.UDone, 0) 'threadsafe method of assigning static value
UThread.Priority = Threading.ThreadPriority.BelowNormal
UThread.IsBackground = True
UThread.Start(l)
Return 0
Catch ex As Exception
End Try
End If
End Function
Private Sub UpdateThreaded2(ByVal l As SortableBindingList(Of NewStockList))
Dim i As Integer = 0
Dim threadcount As Integer = Math.Min(Me.MaxThreads, Me.Stocks.Count)
Dim threads(threadcount - 1) As SUTC
Try
While i < Me.Stocks.Count
For j As Integer = 0 To threadcount - 1
If threads(j) Is Nothing Then
If i < Me.Stocks.Count Then
threads(j) = New SUTC(Me.Stocks(i), Me.DefaultService, AdjustSplits, Use20Minutes, l)
threads(j).Thread.Priority = Threading.ThreadPriority.BelowNormal
threads(j).Thread.IsBackground = True
threads(j).Thread.Start()
i += 1
End If
ElseIf threads(j).UpdateState = 0 Then
If i < Me.Stocks.Count Then
SecUpd(j) = Me.Stocks(i).symbol
threads(j) = New SUTC(Me.Stocks(i), Me.DefaultService, AdjustSplits, Use20Minutes, l)
threads(j).Thread.Priority = Threading.ThreadPriority.BelowNormal
threads(j).Thread.IsBackground = True
threads(j).Thread.Start()
i += 1
End If
End If
Next
Dim running As Boolean = True
While running
For j As Integer = 0 To threadcount - 1
If threads(j).UpdateState = 0 Then
Thread.Sleep(10)
running = False
SecUpd(j) = ""
Interlocked.Increment(UDone) 'threadsafe method of incrementing a variable by 1
Interlocked.Exchange(UCount, UCount + threads(j).UpdateCount) 'Threadsafe method for assigning a value
End If
Next
End While
End While
Dim pending As Integer = threadcount
Dim tempcount As Integer = 0
Dim oldcount As Integer = UCount
While pending > 0
pending = threadcount
tempcount = 0
For i = 0 To threadcount - 1
If threads(i).UpdateState = 0 Then
SecUpd(i) = ""
pending -= 1
tempcount += threads(i).UpdateCount
Thread.Sleep(10)
End If
Next
Interlocked.Exchange(UDone, Me.Stocks.Count - pending) 'Threadsafe method for assigning a value
Interlocked.Exchange(UCount, oldcount + tempcount) 'Threadsafe method for assigning a value
End While
Me.URunning = False
Catch ex As System.Threading.ThreadAbortException 'handle abort correctly
Dim pending As Integer = threadcount
Dim tempcount As Integer = 0
Dim oldcount As Integer = UCount
While pending > 0
pending = threadcount
tempcount = 0
For i = 0 To threadcount - 1
If threads(i).UpdateState = 0 Then
SecUpd(i) = ""
pending -= 1
tempcount += threads(i).UpdateCount
End If
Next
Interlocked.Exchange(UDone, Me.Stocks.Count - pending) 'Threadsafe method for assigning a value
Interlocked.Exchange(UCount, oldcount + tempcount) 'Threadsafe method for assigning a value
End While
End Try
End SubWhen the market is opened, stock prices changed every 500 ms . You can see this change on google finance.
I figured as much, but my thoughts are that if you can somehow find a way to make your program more event driven, then it may be possible to reduce unnecessary looping...
“If you want something you've never had, you need to do something you've never done.”
Don't forget to mark
helpful posts and answers
! Answer an interesting question? Write a
new article
about it! My Articles
*This post does not reflect the opinion of Microsoft, or its employees. -
Need to limit ram per user +cpu usage % and internet speed
I go into Windows System Resource Management but I only see limit by process. I need to be able to say limit "greg" to only be able to use a total of 5GB of ram and say 10% of the total available cpu usage. I saw some people suggesting to set up
proxies for speed limiting but have no idea on how to go about doing that. Or if any1 has any software ideas like I'v tried Net Limiter 3 b4 but it was buggy and stopped other things from working properly on my pc plus don't know if I can use it on server.
so just to clarify if they are running 1 program they can only uses say the 5GB of ram or if there using 5 differant programs there still limited to only using 5GB of ram.
Thanks all in advance,
BlueHi,
You can use the System Resource Manager if you are using the Server 2008R2 or previous edition server, You can set an upper limit on the working set of a matched process,
but about the RAM limits there have some additional considerations:
• Do not use memory limits in Windows System Resource Manager to manage applications or processes that modify their own memory limits dynamically. This can interfere with
the correct operation of Windows System Resource Manager and the managed application.
•As a best practice, use CPU targets to manage resources. Apply memory limits selectively to applications that exhibit memory consumption issues. Excessively limiting the
memory that is available to an application can increase the time it takes the application to complete a task, and it can increase disk usage.
More detail please refer the following related KB:
Understanding Memory Management in Windows System Resource Manager
http://technet.microsoft.com/en-us/library/cc753446.aspx
More information:
Can a process be limited on how much physical memory it uses?
http://blogs.technet.com/b/clinth/archive/2012/10/11/can-a-process-be-limited-on-how-much-physical-memory-it-uses.aspx
Install Windows System Resource Manager
http://technet.microsoft.com/en-us/library/cc753939.aspx
Hope this helps.
We
are trying to better understand customer views on social support experience, so your participation in this
interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place. -
100% CPU Usage Overhead running EM DBConsole 11g on OEL-5.2
After upgrading to OEL-5.2 and relinking all Oracle binaries, my old Oracle 11g installation, installed several months before on OEL-5.1, has been working well, including Enterprise Manager Database Console working nicely as always with respectful performance. Unfortunatelly, it lasted just several days.
Yesterday I decided to uninstall the 11g completely and perform new clean installation (software and database) with the same configuration options and settings as before, including EM dbconsole, all configured using dbca. After completing the installation (EM was started automatically by dbca), oracle continued to suck 80-85% CPU time. In further few minutes CPU utilization raised up to 99% due to only one (always the same PID) client process - "oracleorcl (LOCAL=NO)". For first ten minutes I didn't care too much since I always enable Automatic Management in dbca. But after two hours, I started to worry. The process was still running, consuming sustained 99% of CPU power. No other system activity, no database activity, no disks activity at all!
I was really puzzled since I installed and reinstalled the 11g at least 20 times on OEL-5.0 and 5.1, experimenting with ASM, raw devices, loopback devices and various combinations of installation options, but never experienced such a behaviour. It took me 3 minutes to log in to EM dbconsole as it was almost unusable performing too slow. After three hours CPU temperature was nearly 60 degrees celsius. I decided to shutdown EM and after that everything became quiet. Oracle was running normally. Started EM again, the problem was back again. Tracing enabled, it filled a 350 MB trace file in just 20 minutes. Reinstalling the software and database once again didn't help. Whenever EM is up, the CPU usage overhead of 99% persists.
Here is a cca 23 minutes session summary report taken from EM dbconsole's Performance page. The trace file is too big to list it here, but it shows the same.
Host CPU: 100%
Active Sessions: 100%The details for the Selected 5 Minute Interval (the last 5 min interval) are shown as follow:
TOP SESSIONS: SYSMAN, Program: OMS
Activity: 100%
TOP MODULES: OEM.CacheModeWaitPool, Service: orcl
Activity: 100%
TOP CLIENT: Unnamed
Activity: 99.1%
TOP ACTIONS: Unnamed (OEM.CacheModeWaitPool) (orcl)
Activity: 100%
TOP OBJECTS: SYSMAN.MGMT_JOB_EXEC_SUMMARY (Table)
Activity: 100%
TOP PL/SQL: SYSMAN.MGMT_JOB_ENGINE.INSERT_EXECUTION
PL/SQL Source: SYSMAN.MGMT_JOB_ENGINE
Line Number: 7135
Activity: 100%
TOP SQL: SELECT EXECUTION_ID, STATUS, STATUS_DETAIL FROM MGMT_JOB_EXEC_SUMMARY
WHERE JOB_ID = :B3 AND TARGET_LIST_INDEX = :B2 AND EXPECTED_START_TIME = :B1;
Activity: 100%
STATISTICS SUMMARY
cca 23 minutes session
with no other system activity
Per
Total Execution Per Row
Executions 105,103 1 10,510.30
Elapsed Time (sec) 1,358.95 0.01 135.90
CPU Time (sec) 1,070.42 0.01 107.04
Buffer Gets 85,585,518 814.30 8,558,551.80
Disk Reads 2 <0.01 0.20
Direct Writes 0 0.00 0.00
Rows 10 <0.01 1
Fetches 105,103 1.00 10,510.30
----------------------------------------Wow!!! Note: no disk, no database activity !
Has anyone experienced this or similar behaviour after clean 11g installation on OEL-5.2? If not, anyone has a clue what the hell is going on?
Thanks in advance.Hi Tommy,
I didn't want to experiment further with already working OEL-5.2, oracle and dbconsole on this machine, specially not after googling the problem and finding out that I am not alone in this world. There are another two threads on OTN forums (Database General) showing the same problem even on 2GB machines:
DBConsole easting a CPU
11g stuck. 50-100% CPU after fresh install
So, I took another, a smaller free machine I've got at home (1GB RAM, 2.2MHz Pentium4, three 80GB disks), on which I used to experiment with new releases of software (this is the machine on which I installed 11g for the first time when it was released on OEL-5.0, and I can recall that everything was OK with EM). This is what I did:
1. I installed OEL-5.0 on the machine, adjusted linux and kernel parameters, and performed full 11g installation. Database and EM dbconsole worked nice with acceptable performance. Without activity in the database, %CPU = zero !!! The whole system was perfectly quiet.
2. Since everything was OK, I shutdown EM and oracle, and performed the full upgrade to OEL-5.2. When the upgrade finished, restarted the system, relinked all oracle binaries, and started oracle and EM dbconsole. Both worked perfectly again, just as before the upgrade. I repeated restarting the database and dbconsole several times, always with the same result - it really rocks. Without database activity, %CPU = zero%.
3. Using dbca, I dropped the database and created the new one with the same configuration options. Wow! I'm again in trouble. A half an hour after the creation of the database, %CPU raised up to 99%. That's it.
The crucial question here is: what is that in OEL-5.2, not existing in the 5.0, that causes dbca/em scripts to be embarrassed at the time of EM agent configuration?
Here are the outputs you required picked 30 minutes after starting the database and EM dbconsole (sustained 99% CPU utilization). Note that this is just a 1GB machine.
Kernel command line: ro root=LABEL=/ elevator=deadline rhgb quiet
[root@localhost ~]# cat /proc/meminfo
MemTotal: 1034576 kB
MemFree: 27356 kB
Buffers: 8388 kB
Cached: 609660 kB
SwapCached: 18628 kB
Active: 675376 kB
Inactive: 287072 kB
HighTotal: 130304 kB
HighFree: 260 kB
LowTotal: 904272 kB
LowFree: 27096 kB
SwapTotal: 3148700 kB
SwapFree: 2940636 kB
Dirty: 72 kB
Writeback: 0 kB
AnonPages: 328700 kB
Mapped: 271316 kB
Slab: 21136 kB
PageTables: 14196 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 3665988 kB
Committed_AS: 1187464 kB
VmallocTotal: 114680 kB
VmallocUsed: 5860 kB
VmallocChunk: 108476 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 4096 kB
[root@localhost ~]# cat /proc/slabinfo
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
rpc_buffers 8 8 2048 2 1 : tunables 24 12 8 : slabdata 4 4 0
rpc_tasks 8 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
rpc_inode_cache 6 7 512 7 1 : tunables 54 27 8 : slabdata 1 1 0
ip_conntrack_expect 0 0 96 40 1 : tunables 120 60 8 : slabdata 0 0 0
ip_conntrack 68 68 228 17 1 : tunables 120 60 8 : slabdata 4 4 0
ip_fib_alias 7 113 32 113 1 : tunables 120 60 8 : slabdata 1 1 0
ip_fib_hash 7 113 32 113 1 : tunables 120 60 8 : slabdata 1 1 0
fib6_nodes 22 113 32 113 1 : tunables 120 60 8 : slabdata 1 1 0
ip6_dst_cache 13 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
ndisc_cache 1 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
RAWv6 4 5 768 5 1 : tunables 54 27 8 : slabdata 1 1 0
UDPv6 9 12 640 6 1 : tunables 54 27 8 : slabdata 2 2 0
tw_sock_TCPv6 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
request_sock_TCPv6 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
TCPv6 1 3 1280 3 1 : tunables 24 12 8 : slabdata 1 1 0
jbd_1k 0 0 1024 4 1 : tunables 54 27 8 : slabdata 0 0 0
dm_mpath 0 0 28 127 1 : tunables 120 60 8 : slabdata 0 0 0
dm_uevent 0 0 2460 3 2 : tunables 24 12 8 : slabdata 0 0 0
dm_tio 0 0 16 203 1 : tunables 120 60 8 : slabdata 0 0 0
dm_io 0 0 20 169 1 : tunables 120 60 8 : slabdata 0 0 0
jbd_4k 1 1 4096 1 1 : tunables 24 12 8 : slabdata 1 1 0
scsi_cmd_cache 10 10 384 10 1 : tunables 54 27 8 : slabdata 1 1 0
sgpool-128 36 36 2048 2 1 : tunables 24 12 8 : slabdata 18 18 0
sgpool-64 33 36 1024 4 1 : tunables 54 27 8 : slabdata 9 9 0
sgpool-32 34 40 512 8 1 : tunables 54 27 8 : slabdata 5 5 0
sgpool-16 35 45 256 15 1 : tunables 120 60 8 : slabdata 3 3 0
sgpool-8 60 60 128 30 1 : tunables 120 60 8 : slabdata 2 2 0
scsi_io_context 0 0 104 37 1 : tunables 120 60 8 : slabdata 0 0 0
ext3_inode_cache 4376 8216 492 8 1 : tunables 54 27 8 : slabdata 1027 1027 0
ext3_xattr 165 234 48 78 1 : tunables 120 60 8 : slabdata 3 3 0
journal_handle 8 169 20 169 1 : tunables 120 60 8 : slabdata 1 1 0
journal_head 684 1008 52 72 1 : tunables 120 60 8 : slabdata 14 14 0
revoke_table 18 254 12 254 1 : tunables 120 60 8 : slabdata 1 1 0
revoke_record 0 0 16 203 1 : tunables 120 60 8 : slabdata 0 0 0
uhci_urb_priv 0 0 28 127 1 : tunables 120 60 8 : slabdata 0 0 0
UNIX 56 112 512 7 1 : tunables 54 27 8 : slabdata 16 16 0
flow_cache 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
cfq_ioc_pool 0 0 92 42 1 : tunables 120 60 8 : slabdata 0 0 0
cfq_pool 0 0 96 40 1 : tunables 120 60 8 : slabdata 0 0 0
crq_pool 0 0 44 84 1 : tunables 120 60 8 : slabdata 0 0 0
deadline_drq 140 252 44 84 1 : tunables 120 60 8 : slabdata 3 3 0
as_arq 0 0 56 67 1 : tunables 120 60 8 : slabdata 0 0 0
mqueue_inode_cache 1 6 640 6 1 : tunables 54 27 8 : slabdata 1 1 0
isofs_inode_cache 0 0 368 10 1 : tunables 54 27 8 : slabdata 0 0 0
hugetlbfs_inode_cache 1 11 340 11 1 : tunables 54 27 8 : slabdata 1 1 0
ext2_inode_cache 0 0 476 8 1 : tunables 54 27 8 : slabdata 0 0 0
ext2_xattr 0 0 48 78 1 : tunables 120 60 8 : slabdata 0 0 0
dnotify_cache 2 169 20 169 1 : tunables 120 60 8 : slabdata 1 1 0
dquot 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
eventpoll_pwq 1 101 36 101 1 : tunables 120 60 8 : slabdata 1 1 0
eventpoll_epi 1 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0
inotify_event_cache 1 127 28 127 1 : tunables 120 60 8 : slabdata 1 1 0
inotify_watch_cache 23 92 40 92 1 : tunables 120 60 8 : slabdata 1 1 0
kioctx 135 135 256 15 1 : tunables 120 60 8 : slabdata 9 9 0
kiocb 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
fasync_cache 0 0 16 203 1 : tunables 120 60 8 : slabdata 0 0 0
shmem_inode_cache 553 585 436 9 1 : tunables 54 27 8 : slabdata 65 65 0
posix_timers_cache 0 0 88 44 1 : tunables 120 60 8 : slabdata 0 0 0
uid_cache 5 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
ip_mrt_cache 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
tcp_bind_bucket 32 203 16 203 1 : tunables 120 60 8 : slabdata 1 1 0
inet_peer_cache 1 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
secpath_cache 0 0 32 113 1 : tunables 120 60 8 : slabdata 0 0 0
xfrm_dst_cache 0 0 384 10 1 : tunables 54 27 8 : slabdata 0 0 0
ip_dst_cache 6 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
arp_cache 2 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
RAW 2 7 512 7 1 : tunables 54 27 8 : slabdata 1 1 0
UDP 3 7 512 7 1 : tunables 54 27 8 : slabdata 1 1 0
tw_sock_TCP 3 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0
request_sock_TCP 4 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0
TCP 43 49 1152 7 2 : tunables 24 12 8 : slabdata 7 7 0
blkdev_ioc 3 127 28 127 1 : tunables 120 60 8 : slabdata 1 1 0
blkdev_queue 23 24 956 4 1 : tunables 54 27 8 : slabdata 6 6 0
blkdev_requests 137 161 172 23 1 : tunables 120 60 8 : slabdata 7 7 0
biovec-256 7 8 3072 2 2 : tunables 24 12 8 : slabdata 4 4 0
biovec-128 7 10 1536 5 2 : tunables 24 12 8 : slabdata 2 2 0
biovec-64 7 10 768 5 1 : tunables 54 27 8 : slabdata 2 2 0
biovec-16 7 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
biovec-4 8 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
biovec-1 406 406 16 203 1 : tunables 120 60 8 : slabdata 2 2 300
bio 564 660 128 30 1 : tunables 120 60 8 : slabdata 21 22 204
utrace_engine_cache 0 0 32 113 1 : tunables 120 60 8 : slabdata 0 0 0
utrace_cache 0 0 32 113 1 : tunables 120 60 8 : slabdata 0 0 0
sock_inode_cache 149 230 384 10 1 : tunables 54 27 8 : slabdata 23 23 0
skbuff_fclone_cache 20 20 384 10 1 : tunables 54 27 8 : slabdata 2 2 0
skbuff_head_cache 86 210 256 15 1 : tunables 120 60 8 : slabdata 14 14 0
file_lock_cache 22 40 96 40 1 : tunables 120 60 8 : slabdata 1 1 0
Acpi-Operand 1147 1196 40 92 1 : tunables 120 60 8 : slabdata 13 13 0
Acpi-ParseExt 0 0 44 84 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-Parse 0 0 28 127 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-State 0 0 44 84 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-Namespace 615 676 20 169 1 : tunables 120 60 8 : slabdata 4 4 0
delayacct_cache 233 312 48 78 1 : tunables 120 60 8 : slabdata 4 4 0
taskstats_cache 12 53 72 53 1 : tunables 120 60 8 : slabdata 1 1 0
proc_inode_cache 622 693 356 11 1 : tunables 54 27 8 : slabdata 63 63 0
sigqueue 8 27 144 27 1 : tunables 120 60 8 : slabdata 1 1 0
radix_tree_node 6220 8134 276 14 1 : tunables 54 27 8 : slabdata 581 581 0
bdev_cache 37 42 512 7 1 : tunables 54 27 8 : slabdata 6 6 0
sysfs_dir_cache 4980 4992 48 78 1 : tunables 120 60 8 : slabdata 64 64 0
mnt_cache 36 60 128 30 1 : tunables 120 60 8 : slabdata 2 2 0
inode_cache 1113 1254 340 11 1 : tunables 54 27 8 : slabdata 114 114 81
dentry_cache 11442 18560 136 29 1 : tunables 120 60 8 : slabdata 640 640 180
filp 7607 10000 192 20 1 : tunables 120 60 8 : slabdata 500 500 120
names_cache 19 19 4096 1 1 : tunables 24 12 8 : slabdata 19 19 0
avc_node 14 72 52 72 1 : tunables 120 60 8 : slabdata 1 1 0
selinux_inode_security 814 1170 48 78 1 : tunables 120 60 8 : slabdata 15 15 0
key_jar 14 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0
idr_layer_cache 170 203 136 29 1 : tunables 120 60 8 : slabdata 7 7 0
buffer_head 38892 39024 52 72 1 : tunables 120 60 8 : slabdata 542 542 0
mm_struct 108 135 448 9 1 : tunables 54 27 8 : slabdata 15 15 0
vm_area_struct 11169 14904 84 46 1 : tunables 120 60 8 : slabdata 324 324 144
fs_cache 82 177 64 59 1 : tunables 120 60 8 : slabdata 3 3 0
files_cache 108 140 384 10 1 : tunables 54 27 8 : slabdata 14 14 0
signal_cache 142 171 448 9 1 : tunables 54 27 8 : slabdata 19 19 0
sighand_cache 127 135 1344 3 1 : tunables 24 12 8 : slabdata 45 45 0
task_struct 184 246 1360 3 1 : tunables 24 12 8 : slabdata 82 82 0
anon_vma 3313 5842 12 254 1 : tunables 120 60 8 : slabdata 23 23 0
pgd 84 84 4096 1 1 : tunables 24 12 8 : slabdata 84 84 0
pid 237 303 36 101 1 : tunables 120 60 8 : slabdata 3 3 0
size-131072(DMA) 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-131072 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-65536(DMA) 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-65536 2 2 65536 1 16 : tunables 8 4 0 : slabdata 2 2 0
size-32768(DMA) 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-32768 9 9 32768 1 8 : tunables 8 4 0 : slabdata 9 9 0
size-16384(DMA) 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-16384 6 6 16384 1 4 : tunables 8 4 0 : slabdata 6 6 0
size-8192(DMA) 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0
size-8192 5 5 8192 1 2 : tunables 8 4 0 : slabdata 5 5 0
size-4096(DMA) 0 0 4096 1 1 : tunables 24 12 8 : slabdata 0 0 0
size-4096 205 205 4096 1 1 : tunables 24 12 8 : slabdata 205 205 0
size-2048(DMA) 0 0 2048 2 1 : tunables 24 12 8 : slabdata 0 0 0
size-2048 260 270 2048 2 1 : tunables 24 12 8 : slabdata 135 135 0
size-1024(DMA) 0 0 1024 4 1 : tunables 54 27 8 : slabdata 0 0 0
size-1024 204 204 1024 4 1 : tunables 54 27 8 : slabdata 51 51 0
size-512(DMA) 0 0 512 8 1 : tunables 54 27 8 : slabdata 0 0 0
size-512 367 464 512 8 1 : tunables 54 27 8 : slabdata 58 58 0
size-256(DMA) 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0
size-256 487 495 256 15 1 : tunables 120 60 8 : slabdata 33 33 0
size-128(DMA) 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
size-128 2242 2490 128 30 1 : tunables 120 60 8 : slabdata 83 83 0
size-64(DMA) 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
size-32(DMA) 0 0 32 113 1 : tunables 120 60 8 : slabdata 0 0 0
size-64 1409 2950 64 59 1 : tunables 120 60 8 : slabdata 50 50 0
size-32 3596 3842 32 113 1 : tunables 120 60 8 : slabdata 34 34 0
kmem_cache 145 150 256 15 1 : tunables 120 60 8 : slabdata 10 10 0
[root@localhost ~]# slabtop -d 5
Active / Total Objects (% used) : 97257 / 113249 (85.9%)
Active / Total Slabs (% used) : 4488 / 4488 (100.0%)
Active / Total Caches (% used) : 101 / 146 (69.2%)
Active / Total Size (% used) : 15076.34K / 17587.55K (85.7%)
Minimum / Average / Maximum Object : 0.01K / 0.16K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
25776 25764 99% 0.05K 358 72 1432K buffer_head
16146 15351 95% 0.08K 351 46 1404K vm_area_struct
15138 7779 51% 0.13K 522 29 2088K dentry_cache
9720 9106 93% 0.19K 486 20 1944K filp
7714 7032 91% 0.27K 551 14 2204K radix_tree_node
5070 5018 98% 0.05K 65 78 260K sysfs_dir_cache
4826 4766 98% 0.01K 19 254 76K anon_vma
4824 3406 70% 0.48K 603 8 2412K ext3_inode_cache
3842 3691 96% 0.03K 34 113 136K size-32
2190 2174 99% 0.12K 73 30 292K size-128
1711 1364 79% 0.06K 29 59 116K size-64
1210 1053 87% 0.33K 110 11 440K inode_cache
1196 1147 95% 0.04K 13 92 52K Acpi-Operand
1170 814 69% 0.05K 15 78 60K selinux_inode_security
936 414 44% 0.05K 13 72 52K journal_head
747 738 98% 0.43K 83 9 332K shmem_inode_cache
693 617 89% 0.35K 63 11 252K proc_inode_cache
676 615 90% 0.02K 4 169 16K Acpi-Namespace
609 136 22% 0.02K 3 203 12K biovec-1
495 493 99% 0.25K 33 15 132K size-256
480 384 80% 0.12K 16 30 64K bio
440 399 90% 0.50K 55 8 220K size-512
312 206 66% 0.05K 4 78 16K delayacct_cache
303 209 68% 0.04K 3 101 12K pid
290 290 100% 0.38K 29 10 116K sock_inode_cache
[root@localhost ~]# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
# Controls IP packet forwarding
net.ipv4.ip_forward=0
# Controls source route verification
net.ipv4.conf.default.rp_filter=1
# Do not accept source routing
net.ipv4.conf.default.accept_source_route=0
# Oracle
net.ipv4.ip_local_port_range=1024 65000
net.core.rmem_default=4194304
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=262144
net.ipv4.tcp_rmem=4096 65536 4194304
net.ipv4.tcp_wmem=4096 65536 4194304
# Keepalive Oracle
net.ipv4.tcp_keepalive_time=3000
net.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_keepalive_probes=15
net.ipv4.tcp_retries2=3
net.ipv4.tcp_syn_retries=2
net.ipv4.tcp_sack=0
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_window_scaling=0
# Oracle
fs.file-max = 6553600
fs.aio-max-nr=3145728
kernel.shmmni=4096
kernel.sem=250 32000 100 142
kernel.shmmax=2147483648
kernel.shmall=3279547
kernel.msgmnb=65536
kernel.msgmni=2878
kernel.msgmax=8192
kernel.exec-shield=0
# Controls the System Request debugging functionality of the kernel
kernel.sysrq=1
kernel.panic=60
kernel.core_uses_pid=1
[root@localhost ~]# free | grep Swap
Swap: 3148700 319916 2828784
[root@localhost ~]# cat /etc/fstab | grep "/dev/shm"
tmpfs /dev/shm tmpfs size=1024M 0 0
[root@localhost ~]# df | grep "/dev/shm"
tmpfs 1048576 452128 596448 44% /dev/shm
NON-DEFAULT DB PARAMETERS:
db_block_size 8192
memory_target 633339904 /* automatic memory management */
open_cursors 300
processes 256
disk_async_io TRUE
filesystemio_options SETALL -
Lightroom Mobile Sync - Extremely High CPU Usage/Sync Process Causes LR To Lag
Since my other thread doesn't seem to be getting any responses, I'm pasting what I've found here. Please keep in mind I am not a beginner with Lightroom and consider myself very familiar with Lightroom's features excluding the new mobile sync.
1st message:
I'm on Lr 5.5 and using the 30 day trial of Adobe CC to try syncing one collection of slight more than 1000 images. Despite already having generated the Smart Previews, I can see my CPU crunching through image after image (the rolling hills pattern in the task manager) while doing the sync. I was assuming, since I already created the Smart Previews, that the sync of this collection would begin immediately and be done by simply uploading all of the existing Smart Previews. The Smart Previews folder of the catalog is 871MB and has stayed the same despite the CPU obviously doing *something*. As it is now, the sync progress is incredibly slow, almost at a pace like it's actually exporting full-res JPGs from the RAW images (as a comparison only, I know this should not be what it's actually doing).
Another side effect of this is that I'm basically unable to use my computer for other tasks due to the high CPU utilization.
Win 7 x64 / Lightroom 5.5
Intel i5 2500k OC'd 4.5GHz
16GB RAM
SSD for OS, separate SSD for working catalog and files
2nd message:
As a follow up, now Lightroom thinks all 1026 photos are synced (as shown in "All Sync Photographs" portion of the Catalog though all images after the 832nd image show the sync icon per image stuck at "Building Previews for Lightroom Mobile" and the status at the top left corner has been stuck at "Syncing 194 photos" for over 12 hours. Is there no option to force another sync via Lightroom Desktop and also force the iOS app to manually refresh (perhaps by pulling down on the collections view, like refreshing via the Mail app)?
3rd message:
One more update, I went into Preferences and deleted all mobile data, which automatically signed me out of Adobe CC and then I signed back in. Please keep in mind the Smart Previews were long generated before even starting the trial, and I also manually generated them again (it ran through quickly since it found they were already generated) many times. Now that I'm re-syncing my collection of 1026 images, I can clearly see Lightroom using the CPU to regenerate the Smart Previews which already exist. I have no idea why it's doing this except that it's making the process of uploading the Smart Previews extremely slow. I hope this time around it will at least sync all 1026 images to the cloud.
4th message:
All 1026 images synced just fine and I could run through my culling workflow on the iPad/iPhone perfectly. Now I'm on a new catalog (my current workflow unfortunately uses one catalog per event) and I see the same problem: Smart Previews already generated but when syncing, Lightroom seems to re-generate them again anyway (or take up a lot of CPU simply to upload the existing Smart Previews). Can anyone else chime in on what their CPU utilization is like during the sync process when Smart Previews are already created?
New information:
Now I'm editing a catalog of images that is synced to Lightroom Mobile and notice that my workflow has gotten even slower between photos (relative to what it was before, this is not a discussion about how fast/slow LR should perform). Obviously Lightroom is syncing the edited settings to the cloud, but I can see my CPU running intensively (all 4 cores) on every image I edit and the CPU utilization graph looks different than before I started using LR mobile sync. It still feels like every change isn't simply syncing an SQLite database change but re-generating a Smart Preview to go with it (I'm not saying this is definitely what's happening, but something is intensively using the CPU that wasn't prior to using LR Mobile).
For example: I only update the tint +5 on an image. I see the CPU spike up to around 30-40%, then falls back down, then back up to 100%, then back down to another smaller spike while Lightroom says "Syncing 1 photo". I've attached a screenshot of my CPU graph when doing this edit on just one image. During this entire time, if I try to move onto edit another image, the program is noticeably slower to respond than it was prior to using LR mobile, due to the fact that there appear to be much more CPU intensive tasks running to sync the previous edit. This is proven by un-syncing the collection and immediately the lag goes away.
I'd be happy to test/try anything you have in mind, because it's my understanding that re-syncing photos that are edited that are already in the cloud should be simply updating the database file rather than require regenerating any Smart Previews or other image data. If indeed that's what it should be doing, then some other portion of LR is causing massive CPU usage. If this continues, I will probably not choose to proceed with a subscription despite the fact that i think LR mobile adds a lot of value and boosts my workflow significantly if it wasn't causing the program to lag so badly in the process.
I know this message was incredibly long and probably tedious to read through so thanks in advance to anyone who gets through it
-JeffThanks for reporting. Just passed along your info to some of our devs. One of the things that needs to be created (beside smart previews) during an initial sync are thumbnails + previews for the LrM app - Guido
Hi Guido,
Thanks for pointing this out. I realized the same thing when I tried syncing a collection for offline mode and found out the required space sounded more like Previews + Smart Previews rather than just the Smart Previews.
greule wrote:
Hi Jeff, are your images particularly large or do you make a lot of changes which you save to the original file as part of your workflow?
The CPU usage is almost certainly from us uploading JPEG previews not the Smart Previews - particularly during develop edits as these force new JPEG previews to be sent from Lightroom desktop, but would not force new Smart Previews (unless the develop edits are modifying the original file making us think the Smart Preview is out of date) to be sent.
Guido
My images are full-resolution ~22mp Canon 5D Mark III RAW files so they're fairly large. Even if I only make one basic change such as exposure changes, I saw the issue. By "save to the original file" I'm assuming metadata changes such as timestamps, otherwise edits to the images aren't actually written to the original file. I'm only doing develop module edits so I shouldn't be touching the original file at all at this point in my workflow.
I think it makes sense now that you mention that new JPEG previews need to be generated and sent to the cloud due to updated develop edits. My concern is that this seems to be done in real-time as opposed to how Lightroom Desktop works (which is to render a new Standard Preview or 1:1 Preview on demand, which means only one is being rendered at any given time while viewing it in Loupe View or possibly 2 in Compare View). If I edit, for example, 10 images quickly in a row, once the sync kicks in a few seconds later, editing the 11th image is severely hindered due to the previous 10 images' JPEG previews being rendered and sync'd to the cloud (I'm assuming the upload portion doesn't take much CPU, but the JPEG render will utilize CPU resources to the fullest if it can). Rendering Standard/1:1 Previews locally and being able to walk away while the process finishes works because it is at the start of my workflow, but having to deal with on-the-fly preview rendering while I'm editing greatly impacts my ability to edit. Perhaps there can be a way to limit max CPU utilization for background sync tasks?
It may help to know that I'm running a dual-monitor setup, with Lightroom on a 27" 2560x1440 display maximized to fit the display (2nd display not running LR's 2nd monitor). Since I'm using a retina iPad, the optimal Standard Previews resolution should be the same at 2880 pixels.
Thanks again for the help! -
SQL query against ConfigMgr DB causing 100% cpu usage on SQL server throughout day
Hi, I am hoping someone here may have an idea of what I can do to resolve this as I am not much of a SQL guy. Here is what I know. Throughout the day, monday-friday, my ConfigMgr SQL server sits at 98-100% processor usage.
Using the following query I have narrowed down the culprit of the high cpu usage throughout the day:
SELECT p.spid, p.status, p.hostname, p.loginame, p.cpu, r.start_time, r.command,
p.program_name, text
FROM sys.dm_exec_requests AS r,
master.dbo.sysprocesses AS p
CROSS APPLY sys.dm_exec_sql_text(p.sql_handle)
WHERE p.status NOT IN ('sleeping', 'background')
AND r.session_id = p.spid
Even though my knowledge of SQL is light, I believe the following output clearly defines the thread responsible for the high cpu usage. This "program_name" is present throughout the day and as long as its there, cpu usage remains at or near 100%:
The text output of the above culprit is as follows (sorry this is long):
-- Name : spDrsSummarizeSendHistory
-- Definition : SqlObjs
-- Scope : CAS_OR_PRIMARY_OR_SECONDARY
-- Object : P
-- Dependencies : <Detect>
-- Description : Summarize DrsSendHistory information into DrsSendHistorySummary
CREATE PROCEDURE spDrsSummarizeSendHistory
AS
BEGIN
SET NOCOUNT ON;
DECLARE @CurrentTime DateTime;
DECLARE @SiteCode nvarchar(3);
DECLARE @LastSummarizationTime DateTime;
DECLARE @NextSummarizationTime DateTime;
DECLARE @Interval INT; -- summarize interval in minutes
DECLARE @ReplicationID INT
DECLARE @iMaxID INT
DECLARE @iCurrentID INT
DECLARE @iBatchSize INT = 50
DECLARE @ReplicationPattern NVARCHAR(255)
DECLARE @TargetSite NVARCHAR(3);
DECLARE @TimeTable TABLE (TargetSite NVARCHAR(3), StartTime DATETIME, EndTime DATETIME, PRIMARY KEY (TargetSite, StartTime, EndTime))
DECLARE @TargetSites TABLE (SiteCode NVARCHAR(3) PRIMARY KEY)
DECLARE @ChildTargetSites TABLE (TargetSite NVARCHAR(3), LastSummarizationTime DATETIME, [Interval] INT, PRIMARY KEY (TargetSite))
DECLARE @ReplicationGroup TABLE (ID INT, ReplicationPattern NVARCHAR(255))
IF OBJECT_ID(N'TempDB..#DrsSendHistorySummary') IS NOT NULL
DROP TABLE #DrsSendHistorySummary
CREATE TABLE #DrsSendHistorySummary (ID BIGINT IDENTITY(1, 1) NOT NULL, SourceSite NVARCHAR(3), TargetSite NVARCHAR(3), ReplicationGroupID INT, SyncDataSize BIGINT, CompressedSize BIGINT, UnCompressedSize BIGINT, MessageCount INT, ChangeCount INT, SummarizationTime DATETIME, PRIMARY KEY (ID))
SET @CurrentTime = GETUTCDATE();
SET @SiteCode = dbo.fnGetSiteCode();
INSERT INTO @ReplicationGroup (ID, ReplicationPattern)
SELECT RD.ID, RD.ReplicationPattern
FROM ReplicationData RD
WHILE EXISTS (SELECT * FROM @ReplicationGroup)
BEGIN
SELECT TOP (1) @ReplicationID = ID, @ReplicationPattern = ReplicationPattern FROM @ReplicationGroup
TRUNCATE TABLE #DrsSendHistorySummary
IF (((@ReplicationPattern = N'global' OR @ReplicationPattern=N'site') AND dbo.fnIsPrimary() = 1)
OR (@ReplicationPattern = N'global_proxy' AND dbo.fnIsSecondary() = 1))
BEGIN -- when current site is child
SET @LastSummarizationTime = ISNULL(( SELECT MAX(SummarizationTime) FROM DrsSendHistorySummary WHERE SourceSite = @SiteCode AND ReplicationGroupID = @ReplicationID),
(SELECT MIN(EndTime) FROM DrsSendHistory WHERE ReplicationGroupID = @ReplicationID) );
SET @Interval = ISNULL((SELECT VALUE FROM RCMSQLCONTROLPROPERTY SCP, RCMSQLCONTROL SC
WHERE SCP.ControlID = SC.ID AND SCP.Name = N'Send History Summarize Interval' AND SC.TypeID = 3 and SC.SiteNumber = dbo.fnGetSiteNumber()), 15); -- default 15 minutes
DELETE @TargetSites
INSERT INTO @TargetSites (SiteCode)
SELECT DISTINCT SiteCode FROM DRS_MessageActivity_Send WHERE ReplicationID = @ReplicationID
IF @LastSummarizationTime IS NOT NULL AND EXISTS (SELECT * FROM @TargetSites)
BEGIN
SET @NextSummarizationTime = DATEADD( minute, DATEDIFF(minute, 0, @LastSummarizationTime)/@Interval *@Interval + @Interval, 0);
-- Summarized according to interval
DELETE @TimeTable
WHILE (@CurrentTime >= @NextSummarizationTime )
BEGIN
INSERT INTO @TimeTable (TargetSite, StartTime, EndTime)
SELECT SiteCode, @LastSummarizationTime, @NextSummarizationTime FROM @TargetSites
SET @LastSummarizationTime = @NextSummarizationTime;
SET @NextSummarizationTime = DATEADD(mi, @Interval, @NextSummarizationTime );
END
INSERT INTO #DrsSendHistorySummary (SourceSite, TargetSite, ReplicationGroupID, SyncDataSize, CompressedSize, UnCompressedSize, MessageCount, ChangeCount, SummarizationTime)
SELECT @SiteCode, T.TargetSite, @ReplicationID, SUM(ISNULL(D.SyncDataSize, 0)), SUM(ISNULL(D.CompressedSize, 0)), SUM(ISNULL(D.UnCompressedSize, 0)), SUM(ISNULL(D.MessageCount, 0)), SUM(ISNULL(ChangeCount, 0)), T.EndTime
FROM DrsSendHistory D RIGHT JOIN @TimeTable T ON D.EndTime >= T.StartTime AND D.EndTime < T.EndTime AND ReplicationGroupID = @ReplicationID AND T.TargetSite = D.TargetSite
GROUP BY T.TargetSite, T.StartTime, T.EndTime;
END
END
ELSE IF ((@ReplicationPattern = N'global' AND dbo.fnIsCAS() = 1)
OR (@ReplicationPattern = N'global_proxy' AND dbo.fnIsPrimary() = 1))
BEGIN -- when current site is parent
DELETE @ChildTargetSites
INSERT INTO @ChildTargetSites(TargetSite, LastSummarizationTime, [Interval])
SELECT DISTINCT(DSH.SiteCode),
ISNULL(( SELECT MAX(SummarizationTime) FROM DrsSendHistorySummary WHERE SourceSite = @SiteCode AND ReplicationGroupID = @ReplicationID AND TargetSite=DSH.SiteCode),
(SELECT MIN(EndTime) FROM DrsSendHistory WHERE ReplicationGroupID = @ReplicationID) ) AS LastSummarizeTime,
ISNULL((SELECT VALUE FROM RCMSQLCONTROLPROPERTY SCP, RCMSQLCONTROL SC WHERE SCP.ControlID = SC.ID AND SCP.Name = N'Send History Summarize Interval' AND SC.TypeID = 3 and SC.SiteNumber = dbo.fnGetSiteNumberBySiteCode(DSH.SiteCode)), 15) AS Interval
FROM DRS_MessageActivity_Send DSH WHERE DSH.ReplicationID = @ReplicationID
GROUP BY DSH.SiteCode;
DELETE @TimeTable
WHILE EXISTS (SELECT * FROM @ChildTargetSites)
BEGIN
SELECT TOP (1) @TargetSite = TargetSite, @LastSummarizationTime = LastSummarizationTime, @Interval = Interval FROM @ChildTargetSites
SET @NextSummarizationTime = DATEADD( minute, DATEDIFF(minute, 0, @LastSummarizationTime)/@Interval *@Interval + @Interval, 0);
-- Summarized according to interval
WHILE (@CurrentTime >= @NextSummarizationTime )
BEGIN
INSERT INTO @TimeTable (TargetSite, StartTime, EndTime)
SELECT @TargetSite, @LastSummarizationTime, @NextSummarizationTime
SET @LastSummarizationTime = @NextSummarizationTime;
SET @NextSummarizationTime = DATEADD(mi, @Interval, @NextSummarizationTime );
END
DELETE @ChildTargetSites WHERE TargetSite = @TargetSite
END;
INSERT INTO #DrsSendHistorySummary (SourceSite, TargetSite, ReplicationGroupID, SyncDataSize, CompressedSize, UnCompressedSize, MessageCount, ChangeCount, SummarizationTime)
SELECT @SiteCode, T.TargetSite, @ReplicationID, SUM(ISNULL(D.SyncDataSize, 0)), SUM(ISNULL(D.CompressedSize, 0)), SUM(ISNULL(D.UnCompressedSize, 0)), SUM(ISNULL(D.MessageCount, 0)), SUM(ISNULL(D.ChangeCount, 0)), @NextSummarizationTime
FROM DrsSendHistory D RIGHT JOIN @TimeTable T ON D.EndTime >= T.StartTime AND D.EndTime < T.EndTime AND ReplicationGroupID = @ReplicationID AND T.TargetSite = D.TargetSite
GROUP BY T.TargetSite, T.StartTime, T.EndTime;
END;
SELECT @iMaxID = MAX(ID), @iCurrentID = 0 FROM #DrsSendHistorySummary
-- BATCH INSERT TO REAL TABLE
WHILE (@iCurrentID < @iMaxID)
BEGIN
INSERT INTO DrsSendHistorySummary (SourceSite, TargetSite, ReplicationGroupID, SyncDataSize, MessageCount, ChangeCount, SummarizationTime)
SELECT SourceSite, TargetSite, ReplicationGroupID, SyncDataSize, MessageCount, ChangeCount, SummarizationTime
FROM #DrsSendHistorySummary WHERE ID > @iCurrentID AND ID <= (@iCurrentID + @iBatchSize)
SET @iCurrentID = @iCurrentID + @iBatchSize
END
TRUNCATE TABLE #DrsSendHistorySummary
DELETE @ReplicationGroup WHERE ID = @ReplicationID
END
END
If anyone has any experience with this issue or anything similar and might be able to point me in the right direction, any responses would be greatly appreciated. Thanks for reading!Normally, this summary job every few mins which only summarize last few mins sending history. If the job was not running for long time, the first time, this sproc will summarize a lot more data than normal. So it could fail which will make things worse and
worse over time.
If sending summary data is not a big concern here, you can manually insert a dummy row with most recent time as Summarizationtime, so that, when summary job kicks in next time, it will only summary short interval data.
DrsSendHistory is that the table records every sending on every group. This table is one of the key of replication logic. For your environment, on primary site, it could have up to 24(hrs) * 60 (mins) / 2 (sync interval) * 36(secondary sites) records per
day for Secondary_Site_Replication_Configuration group and 24 * 60 / 5 * 22 records for Secondary Site Data.
Umair Khan | http://blogs.technet.com/umairkhan
Hi Umair,
Thank you for the reply. My plan was to go ahead with your suggestions but what I tried first was to get my VM guy to give the SQL server access to more CPU resources. We had already configured it with what would have seemed to be plenty, as per Microsoft's
sizing requirements, but I figured it couldn't hurt to rule out - he gave the SQL server access to a huge portion of the host processing power and since then CPU usage on the server has NOT been an issue.
It would appear that the large amounts of secondary sites we have just simply put a lot of stress on the SQL server with out of the box replication settings. Since the entire VM host is for System Center only, and our other System Center servers are using
very little resources, this solution has ended up being valid.
Thanks again for everyone's help with this.
Maybe you are looking for
-
DV9500, when I press the on button, all the normal indicator light briefly blink then go out.
-
Unique circumstance: SC create This takes 3 minutes and 30 seconds on one domain when using Windows 8.1 & 2012R2. We have two domains: D1.com and D2.D1.com. If we place a Windows 8.1 or 2012R2 VM on the D2.D1.com domain and run the sc command to c
-
2110 all-in-one not printing all text in documents
My printer has stopped printing all parts of documents. It just seems to skip certain parts of word documents, excel spreadsheets, pdfs, etc. And, if I print the same document more than once, different parts of the document will print, but never the
-
Set sales employee as inactive
Hello, in SAP BO you can add sales employee, but can not be erased. Number of buyers is growing - when you create a document (invoice, ...) list of sales employees becomes confusing. We would welcome the possibility of setting sales employee as INACT
-
I am interested in a 200gb bundle . Is it possible to pay extra mobile phones included.
No. Mobiles are not included as part of bundles. You can have them all billed together on the one bill, but they will be separate contracts.