How to calculate the in use percentage of cache ?

Question

Hi, All,
I have two questions here:
1. Can we calculate the proper cache size based on the average key/data pair size and the number of key/data pairs? Is there any formula or something? ( I doubt it :), so next is the second question )
2. If we set a fixed cache size, and ensure that it has almost 100% hit rate for all the entries accesses. Does this mean that the cache is enough? And if yes, can we calculate how much percentage of the cache is actually in use?
Thanks in advance.

Answer 1

Hi,
user647934 wrote:
1. Can we calculate the proper cache size based on the average key/data pair size and the number of key/data pairs? Is there any formula or something? ( I doubt it :), so next is the second question ) You can estimate that, just that you should also consider: the page size and how many records will fit on a page.
Depending on your type of access method, you can check here for formulas that are estimating the total database size (you can find them here: http://www.oracle.com/technology/documentation/berkeley-db/db/ref/am_misc/diskspace.html ), and just translate that to the cache size. Be aware that any cache size less than 500MB is automatically increased by 25% to account for buffer pool overhead.
user647934 wrote:
2. If we set a fixed cache size, and ensure that it has almost 100% hit rate for all the entries accesses. Does this mean that the cache is enough? And if yes, can we calculate how much percentage of the cache is actually in use?I think that you can find everything you need by calling "db_stat -m" or DB_ENV->memp_stat for detailed cache statistics.
db_stat: http://www.oracle.com/technology/documentation/berkeley-db/db/utility/db_stat.html
DB_ENV->memp_stat: http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/memp_stat.html
The fields returned by this are:
size_t st_gbytes;
* Gigabytes of cache (total cache size is st_gbytes + st_bytes).
size_t st_bytes;
* Bytes of cache (total cache size is st_gbytes + st_bytes).
u_int32_t st_ncache;
* Number of caches.
u_int32_t st_max_ncache;
* Maximum number of caches, as configured with the DB_ENV->set_cache_max method.
roff_t st_regsize;
* Individual cache size, in bytes.
size_t st_mmapsize;
* Maximum memory-mapped file size.
int st_maxopenfd;
* Maximum open file descriptors.
int st_maxwrite;
* Maximum sequential buffer writes.
db_timeout_t st_maxwrite_sleep;
* Microseconds to pause after writing maximum sequential buffers.
u_int32_t st_map;
* Requested pages mapped into the process' address space (there is no available information about whether or not this request caused disk I/O, although examining the application page fault rate may be helpful).
u_int32_t st_cache_hit;
* Requested pages found in the cache.
u_int32_t st_cache_miss;
* Requested pages not found in the cache.
u_int32_t st_page_create;
* Pages created in the cache.
u_int32_t st_page_in;
* Pages read into the cache.
u_int32_t st_page_out;
* Pages written from the cache to the backing file.
u_int32_t st_ro_evict;
* Clean pages forced from the cache.
u_int32_t st_rw_evict;
* Dirty pages forced from the cache.
u_int32_t st_page_trickle;
* Dirty pages written using the DB_ENV->memp_trickle method.
u_int32_t st_pages;
* Pages in the cache.
u_int32_t st_page_clean;
* Clean pages currently in the cache.
u_int32_t st_page_dirty;
* Dirty pages currently in the cache.
u_int32_t st_hash_buckets;
* Number of hash buckets in buffer hash table.
u_int32_t st_hash_searches;
* Total number of buffer hash table lookups.
u_int32_t st_hash_longest;
* Longest chain ever encountered in buffer hash table lookups.
u_int32_t st_hash_examined;
* Total number of hash elements traversed during hash table lookups.
u_int32_t st_hash_nowait;
* Number of times that a thread of control was able to obtain a hash bucket lock without waiting.
u_int32_t st_hash_wait;
* Number of times that a thread of control was forced to wait before obtaining a hash bucket lock.
u_int32_t st_hash_max_nowait;
* The number of times a thread of control was able to obtain the hash bucket lock without waiting on the bucket which had the maximum number of times that a thread of control needed to wait.
u_int32_t st_hash_max_wait;
* Maximum number of times any hash bucket lock was waited for by a thread of control.
u_int32_t st_region_wait;
* Number of times that a thread of control was forced to wait before obtaining a cache region mutex.
u_int32_t st_region_nowait;
* Number of times that a thread of control was able to obtain a cache region mutex without waiting.
u_int32_t st_mvcc_frozen;
* Number of buffers frozen.
u_int32_t st_mvcc_thawed;
* Number of buffers thawed.
u_int32_t st_mvcc_freed;
* Number of frozen buffers freed.
u_int32_t st_alloc;
* Number of page allocations.
u_int32_t st_alloc_buckets;
* Number of hash buckets checked during allocation.
u_int32_t st_alloc_max_buckets;
* Maximum number of hash buckets checked during an allocation.
u_int32_t st_alloc_pages;
* Number of pages checked during allocation.
u_int32_t st_alloc_max_pages;
* Maximum number of pages checked during an allocation.
u_int32_t st_io_wait;
* Number of operations blocked waiting for I/O to complete.
Please let me know if this helps.
Thanks,
Bogdan