Need better understanding of password cache

Question

This issue has also been submitted to iPlanet support.
In NSMS 4.15 p6, Solaris 2.6, two IMAP daemons, two mshttpd daemons:
I am trying to get a better understanding of password caching. See service.authcachettl and service.authcachesize in the NSMS tuning guide.
The guide says "The authentication cache ... is not shared between different processes". It also says "Its value should be greater than or equal to the number of users who have their mailbox on the machine to avoid pushing entries out of the cache before the TTLs expire."
Our user population is about 60% webmail, 40% IMAP. I turned logging for each of these to Debug level to see the password caching numbers. Here is a log entry for each of our four processes:
HTTP:
httpd[3568]: General Debug: Password cache: 7344 entries, 20635 hits, 57565 misses, 22651 collisions.
httpd[3564]: General Debug: Password cache: 7255 entries, 20573 hits, 57327 misses, 21798 collisions.
IMAP:
imapd[3515]: General Debug: Password cache: 4109 entries, 250843 hits, 73057 misses, 16132 collisions.
imapd[3522]: General Debug: Password cache: 4120 entries, 252521 hits, 93979 misses, 16582 collisions.
We have approximately 20,000 mailboxes so I set the cache size somewhat higher than that, but it appears that each daemon only needs around 5 to 10 thousand. Is that a setting for each process as is implied by the tuning guide? If so, why so they suggest setting it at the number of mailboxes? Does each process have to allocate memory for the auth cache at startup? Am I wasting memory?
Other questions I have:
What are collisions?
What are some possible explanations for such a poor hit to miss ratio in http?
From what time are these numbers? That is, is there any time at which they reset without restarting services?
Thanks.
Mark Evangelista
Georgetown University
Washington, DC, US

Answer 1

It took awhile, but I got a response from iPlanet support. Look for responses interspersed with my original question below:
Mark Evangelista wrote:
The guide says "The authentication cache ... is not shared between different processes". It also says "Its value should be greater than or equal to the number of users who have their mailbox on the machine to avoid pushing entries out of the cache before the TTLs expire."
Our user population is about 60% webmail, 40% IMAP. I turned logging for each of these to Debug level to see the password caching numbers. Here is a log entry for each of our four processes:
HTTP:
httpd[3568]: General Debug: Password cache: 7344 entries, 20635 hits, 57565 misses, 22651 collisions.
httpd[3564]: General Debug: Password cache: 7255 entries, 20573 hits, 57327 misses, 21798 collisions.
IMAP:
imapd[3515]: General Debug: Password cache: 4109 entries, 250843 hits, 73057 misses, 16132 collisions.
imapd[3522]: General Debug: Password cache: 4120 entries, 252521 hits, 93979 misses, 16582 collisions.
We have approximately 20,000 mailboxes so I set the cache size somewhat higher than that, but it appears that each daemon only needs around 5 to 10 thousand. Is that a setting for each process as is implied by the tuning guide?
A: It has been confirmed that the cache size setting is per daemon. In our case with two each of mshttpd and imapd, we have allocated memory of four times the cache
size setting.
If so, why so they suggest setting it at the number of mailboxes?
A: The cache size will have some impact on collisions. See the question on collisions below.
Does each process have to allocate memory for the auth cache at startup?
A: Yes
Am I wasting memory?
A: Possibly not. See collisions below.
What are collisions?
A: iPlanet says "Collisions are when multiple uids have the same hash value in the cache. They only becomes a problem when the number of entries becomes close to the
max cache size. Since you have 7344 entries, you might want to increase service.authcachesize from the 10000 default to, say, 20000."
Comment: I infer from this that there is some hashing scheme that is dependent on the size of the cache and that with a larger cache we should have fewer collisions.
What are some possible explanations for such a poor hit to miss ratio in http?
A: Not answered by iPlanet. This is my speculation only, but it is likely that the pattern of use of webmail vs imap. The bulk of our webmail users are students who are more likely to sporadically wander over to a computer lab and check their email via the web rather than head back to thier dorm room and their own workstation. The vast majority of imap users are staff and faculty who spend more time at their own workstations, sign on in the morning and have their client check for new mail every 15 minutes or so.
From what time are these numbers?
A: Numbers are from the start of the services.
That is, is there any time at which they reset without restarting services?
A: No.
Additionally iPlanet provided:
"Miss - Is when one of the following happens:
1. no match for a given entry,
2. there is a match but this cache entry is expired,
3. entry is too long to be cached"
There was no further explanation of reason #3, which I think just begs for another question