[SOLVED] SGA_MAX_SIZE pre-allocated with Solaris 10?

Question

Hi all,
I'm about to build a new production database to migrate an existing 8.1.7 database to 10.2.0.3. I'm in the enviable position of having a good chunk of memory to play with on the new system (compared with the existing one) so was looking at a suitable size for the SGA... when something pinged in my memory about SGA_MAX_SIZE and memory allocation in the OS where some platforms will allocate the entire amount of SGA_MAX_SIZE rather than just SGA_TARGET.
So I did a little test. Using Solaris 10 and Oracle 10.2.0.3 I've created a basic database with SGA_MAX_SIZE set to 400MB and SGA_TARGET 280MB
$ sqlplus
SQL*Plus: Release 10.2.0.3.0 - Production on Wed Jan 30 18:31:21 2008
Copyright (c) 1982, 2006, Oracle.  All Rights Reserved.
Enter user-name: / as sysdba
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options
SQL> show parameter sga
NAME                                 TYPE        VALUE
lock_sga                             boolean     FALSE
pre_page_sga                         boolean     FALSE
sga_max_size                         big integer 400M
sga_target                           big integer 280MSo I was expecting to see the OS pre-allocate 280MB of memory but when I checked the segment is actually the 400MB (i.e. SGA_MAX_SIZE) (my database owner is 'ora10g'):
$ ipcs -a
IPC status from <running system> as of Wed Jan 30 18:31:36 GMT 2008
T         ID      KEY        MODE        OWNER    GROUP  CREATOR  
CGROUP CBYTES  QNUM QBYTES LSPID LRPID   STIME    RTIME    CTIME
Message Queues:
T         ID      KEY        MODE        OWNER    GROUP  CREATOR  
CGROUP NATTCH      SEGSZ  CPID  LPID   ATIME    DTIME    CTIME
Shared Memory:
m         22   0x2394e4   rw-r---   ora10g   10gdba   ora10g  
10gdba     20  419438592  2386  2542 18:31:22 18:31:28 18:28:18
T         ID      KEY        MODE        OWNER    GROUP  CREATOR  
CGROUP NSEMS   OTIME    CTIME
Semaphores:
s         23   0x89a070e8 ra-r---   ora10g   10gdba   ora10g  
10gdba   154 18:31:31 18:28:18
$ I wasn't sure whether Solaris 10 was one of the OSs with truly dynamic memory for the SGA but had hoped it was... this seems to say different. Really I'm just after some confirmation that I'm reading this correctly.
Thanks.
Joseph
Message was edited by:
Joseph Crofts
Edited for clarity

Answer 1

I don't want to get bogged down in too many details, as the links provided in previous posts have many details of SGA tests and the results of what happened. I just want to add a bit of explanation about the Oracle SGA and shared memory on UNIX and Solaris in particular.
As you know Oracle's SGA is generally a single segment of shared memory. Historically this was 'normal' memory and could be paged out to the swap device. So a 500 MB SGA on a 1 GB physical memory system, would allocate 500 MB from the swap device for paging purposes, but might not use 500 MB of physical memory i.e. free memory might not decrease by 500 MB. How much physical memory depended on what pages in the SGA were accessed, and how frequently.
At some point some people realised that this paging of the SGA was actually slowing performance of Oracle, as now some 'memory' accesses by Oracle could actually cause 'disk' accesses by paging in saved pages from the swap device. So some operating systems introduced a 'lock' option when creating a shared memory segment (shmat system call if memory serves me). And this was often enabled by a corresponding Oracle initialisation parameter, such as lock_sga.
Now a 'locked' SGA did use up the full physical memory, and was guaranteed not to be paged out to disk. So Oracle SGA access was now always at memory speed, and consistent.
Some operating systems took advantage of this 'lock' flag to shared memory segment creation to implement some other performance optimisations. One is not to allocate paging storage from swap space anyway, as it cannot be used by this shared memory segment. Another is to share the secondary page tables within the virtual memory sub-system for this segment over all processes attached to it i.e. one shared page table for the segment, not one page table per process. This can lead to massive memory savings on large SGAs with many attached shadow server processes. Another optimisation on this non-paged, contiguous memory segment is to use large memory pages instead of standard small ones. On Solaris instead of one page entry covering 8 KB of physical memory, it covers 8 MB of physical memory. This reduces the size of the virtual memory page table by a factor of 1,000 - another major memory saving.
These were some of the optimisations that the original Red Hat Enterprise Linux had to introduce, to play catch up with Solaris, and to not waste memory on large page tables.
Due to these extra optimisations, Solaris chose to call this 'locking' of shared memory segments 'initimate shared memory' or ISM for short. And I think there was a corresponding Oracle parameter 'use_ism'. This is now the default setting in Oracle ports to Solaris.
As a result, this is why when Oracle grabs its shared memory segment up front (SGA_MAX_SIZE), it results in that amount of real physical memory being allocated and used.
With Oracle 9i and 10g when Oracle introduced the SGA_TARGET and other settings and could dynamically resize the SGA, this messed things up for Solaris. Because the shared memory segment was 'Intimate' by default, and was not backed up by paging space on the swap device, it could never shrink in size, or release memory as it could not be paged out.
Eventually Sun wrote a work around for this problem, and called it Dynamic Intimate Shared Memory (DISM). This is not on by default in Oracle, hence you are seeing all your shared memory segments using the same amount of physical memory. DISM allows the 'lock' flag to be turned on and off on a shared memory segment, and to be done over various memory sizes.
I am not sure of the details, and so am beginning to get vague here. But I remember that this was a workaround on Sun's part to still get the benefits of ISM and the memory savings from large virtual memory pages and shared secondary page tables, while allowing Oracle to manage the SGA size dynamically and be able to release memory back for use by other things. I'm not sure if DISM allows Oracle to mark memory areas as pageable or locked, or whether it allows Oracle to really grow and shrink the size of a single shared memory segment. I presumed it added yet more flags to the various shared memory system calls.
Although DISM should work on normal, single Solaris systems, as you know it is not enabled by default, and requires a special initialisation parameter. Also be aware that there are issues with DISM on high end Solaris systems that support Domains (F15K, F25K, etc.) and in Solaris Zones or Containers. Domains have problems when you want to dynamically remove a CPU/Memory board from the system, and the allocations of memory on that board must be reallocated to other memory boards. This can break the rule that a locked shared memory segment must occupy contiguous physical memory. It took Sun another couple of releases of Solaris (or patches or quarterly releases) before they got DISM to work properly in a system with domains.
I hope I am not trying to teach my granny to suck eggs, if you know what I mean. I just thought I'd provide a bit more background details.
John