Lock collisions/rejections, processes in ENQ, SAPLSENA, enqueue issues

Question

Hi all
Tricky one for ya...
We have had a number of performance complaints recently and everything in the system looks okay, apart from a re-occurrance of a problem we have seen previously, with processes going into ENQ status.
Around three or four pages (out of about ten pages in total) in SM66 show processes with a status/reason of stopped/ENQ. The process shown for a wide range of the users is SAPLSENA, or sometimes SAPLTHFB (then the user always seem to be SAPSYS). No table name is shown.
This can happen for a few seconds or a few minutes, then will clear up, only to come back 30 seconds or 5 minutes later. These symptons can happen on and off for 10 minutes, or up to an hour. Then everything is fine again...
We can't seem to isolate any one particular program, user or background job causing this (looked at the processes holding UPD in ENQ mode for example at this time but can be anything...).
We've already looked at the following notes:
Note 142054 - Processes in status stopped ENQ
Note 1384191 - Transaction SM50- Many processes with report SAPLSENA
With the first note, it's not cause 1 or 4, and 2 and 3 reasons/solutions are too vague.
The second note doesn't really help either; creating an enqueue lock doesn't reveal anything particularly useful.
Looking in SM12 we can see that the number of enqueues rejected/collisions is quite high - 29% currently. In our relatively busy QA system it's only 5% and another production system only 6%... should it be this high?
We have a lot of locks in SM12... but always have done. I did notice that about one sixth of the 5000 locks in SM12 are on ATPENQ and one fifth are on EM07M.
The last time we had this it was actually a major incident, and SAP got involved. We noticed that there had been 4 times as many updates in the system in the last couple of days. We then noticed a handful of users from one site, never logged in for long, just a few seconds here and there - turned out they were connecting through some kind of interface and hammering the system with updates. There was some bespoke code that was faulty...
Not the case this time, no suspect users, number of updates looks the same for the last two weeks...
Previously SAP advised us to reduce the number of ENQ processes from 4 to 3 on the CI, which we did. We don't use a standalone enqueue server currently. We regularly see all 3 ENQ processes in use and in semaphore 26 (i.e. enqueue lock).
Any ideas how to resolve this or at least isolate what's causing this? Is the number of rejects too high?
Thanks
Ross

Answer 1

Hello Ross,
I think first you should check the Enqueue is not hangind due to the issues described in SAP Note 1304197.
By running program Z_ENQUEUE_PERF you check the enqueue configuration and some test are performed to detect known problems.
At the time of the issue you could as well get "enqueue logging" and see if there is any massive application overloading the enqueue.
hope it helps,
Mercedes