Is properly sized sga?

hello all,
my database size is around 60gb, frequently iam getting performance problems, that job executing not in a time some times 4 hours some times 40 mins like that , when i look for sga and pga i thought parameters not tuned properly iam mentioning all sga & pga parameter here, can anyone tell what could be the best size we can put for every parameter?
to improve performance.
thanks in advance! its urgent.
SQL> show sga
Total System Global Area 629145600 bytes
Fixed Size 2067848 bytes
Variable Size 381682296 bytes
Database Buffers 243269632 bytes
Redo Buffers 2125824 bytes
SQL> show parameter pg
NAME TYPE VALUE
pga_aggregate_target big integer 115762790
SQL> show parameter sort
NAME TYPE VALUE
nls_sort string
sort_area_retained_size integer 0
sort_area_size integer 65536
SQL> show parameter hash
NAME TYPE VALUE
hash_area_size integer 131072
SQL>

SGA_SIZE SGA_SIZE_FACTOR ESTD_DB_TIME ESTD_DB_TIME_FACTOR ESTD_PHYSICAL_READS
552 1 618141 1 48146030
276 .5 1875872 3.0347 130687584
414 .75 697139 1.1278 52267330
1104 2 504218 .8157 39888986
828 1.5 504218 .8157 39888986
966 1.75 504218 .8157 39888986
690 1.25 511079 .8268 39888986

Similar Messages

Problem in sizing sga

Dear All,
We are using Oracle 10g R2 on windows 2003.Our sga_max_size=2000m and sga_target=800m.We have enabled ASM.
I got an alert message in enterprize manger which says virtual memory paging is happening and its impact is 100%.
Now,Is this problem related to SGA?Do i have to resize my sga_max_size ?
We have 4gb RAM 460 gb hard disk what would be the ideal size for sga?
Regards,
Kris

For guidelines over SGA & memory, refer discussion here...
Re: windows 32bit sga and pga limits
But actual sizing of SGA depends on your application needs. If it needs more SGA, it does. You might need to add more RAM to your server to cope up with existing workload for a given number of users.

Difficulty Importing a Properly Sized TIFF image (I think)

Hello,
I am having difficulty importing an appropriately sized TIFF image that is a screen shot from my film. It seems to be a resizing issue of some sort.
The TIFF image is a straight frame grab from my film, which is sized 1920 x 1080.
here is the issue:
The screen shot from the film has the reclining face buddha image overlay upon "the walk in the forest" perfectly centered in the frame of the film, but when I import that screen shot from the film, the mouth of the Buddha and chin is cut off on the left hand side.
Any thoughts?
Is this because I am making this into a smaller DVD frame in Standard Definition?
Will it do this cutting off of the image for my entire film as well?
Matt

Hey Matt. If you are using this tiff in a menu in your SD project, there is going to be some issues going from an HD size of 1920x1080 down to 720x480. Is your project 4:3 aspect ratio or 16:9? Generally you want your menus to match what the video elements are.
If your encoded video elements are 4:3 (hard) letterboxed, then the menu should be 4:3 as well, although it could be 4:3 full frame. No sense having black bars on your menu pages. If this is the case, that 16:9 tiff image will be edge-cropped.
Likely since your video was shot in HD, you want to have a widescreen dvd. The source video will have to be encoded as 16:9 SD. I usually make all my elements before bringing them into Encore. For any menus, I start in Photoshop using the appropriate "Film and Video" preset. Probably the 720x480 widescreen preset (1.21 pixel aspect ratio) will work for your project if it is indeed a 16:9 disc you want to make. Being a 16:9 frame size, the tiff file will scale down almost perfectly into a 720x480 widescreen frame.
I would suggest making a new file in Photoshop using the appropriate film and video preset. Then open your tiff file in Photoshop and drag it into that one. Scale it down and save. That way when you bring it into Encore, the scaling has been done and you know what it is going to look like.
Note that when you are working with files in Photoshop that are not square pixels, they will appear jaggy or squeezed depending on if you have pixel-aspect correction turned On or Off in the View menu. The files are fine and will look good on a disc. Also, I almost always change the viewer in Encore to 100% or some other whole number rather than "Fit". Fit just seems to make things look terrible, especially text and the outer frame edges. There is also the option to toggle pixel aspect correction On or Off in that viewer as well.
Hope this helps.
Cheers.

Help on 10g 64 bit --Sizing SGA and sysctl.conf

Dear Sir,
I have an oracle10g installed on a 64 bit OS and want to allocate 8GB to SGA and want to confirm if i am using the right settings
My sysctl.conf settings are :
cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.
# Controls IP packet forwarding
net.ipv4.ip_forward = 0
# Controls source route verification
net.ipv4.conf.default.rp_filter = 1
# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0
# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0
# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1
# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1
# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536
# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536
kernel.shmmax = 8589934592
kernel.shmmni = 4096
kernel.shmall = 2097152
kernel.sem = 3000 32000 100 128
net.core.rmem_default=262144
net.core.rmem_max=262144
net.core.wmem_default=262144
net.core.wmem_max=262144
My init Settings are :
*._DB_BLOCK_LRU_LATCHES=16
*._spin_count=7000
*.archive_lag_target=1800
*.audit_sys_operations=TRUE
*.BACKGROUND_DUMP_DEST='/redf/DB/dbf/dump'
*.circuits=10000
*.COMPATIBLE='10.2.0.4'
*.CONTROL_FILES='/redf/DB/dbf/cntl1/DB_cntl1_01.ctl','/redf/DB/dbf/cntl2/DB_cntl2_02.ctl','/redf/DB/dbf/cntl3/DB_cntl3_03.ctl'
*.CORE_DUMP_DEST='/redf/DB/dbf/dump'
*.cursor_sharing='FORCE'
*.DB_BLOCK_SIZE=8192
*.DB_CACHE_SIZE=1024M
*.db_file_multiblock_read_count=64
*.db_file_name_convert='/redf/DB/dbf','/redf/DB/dbf'
*.DB_NAME='DB'
*.db_writer_processes=8
*.dispatchers='(PROTOCOL=TCP)(DISPATCHERS=10)'
*.FAST_START_MTTR_TARGET=300
*.INSTANCE_NAME='DB'
*.JAVA_POOL_SIZE=64M
*.JOB_QUEUE_PROCESSES=1
*.large_pool_size=150M
*.LOCAL_LISTENER='(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=XX.XX.XX.XXX)(PORT=1521)))'
*.LOG_ARCHIVE_DEST_1='LOCATION=/redf/DB/arch'
*.LOG_ARCHIVE_DEST_2='LOCATION=/orabackup/redf/DB/arch_new'
*.log_archive_dest_state_2='ENABLE'
*.log_archive_format='DB%t_%r_%s.arc'
*.log_archive_max_processes=2
*.log_archive_min_succeed_dest=1
*.LOG_ARCHIVE_START=TRUE
*.log_buffer=50000000
*.MAX_DISPATCHERS=40
*.max_shared_servers=40
*.O7_DICTIONARY_ACCESSIBILITY=FALSE
*.open_cursors=10000
*.optimizer_index_cost_adj=20
*.parallel_adaptive_multi_user=TRUE
*.parallel_automatic_tuning=TRUE
*.parallel_max_servers=20
*.parallel_min_servers=2
*.processes=4000
*.QUERY_REWRITE_ENABLED='TRUE'
*.QUERY_REWRITE_INTEGRITY='ENFORCED'
*.recovery_parallelism=5
*.recyclebin='OFF'
*.remote_archive_enable='TRUE'
*.REMOTE_LOGIN_PASSWORDFILE='EXCLUSIVE'
*.remote_os_authent=FALSE
*.resource_limit=TRUE
*.session_cached_cursors=200
*.sessions=4000
*.SGA_MAX_SIZE=8000M
*.SGA_TARGET=6500M
*.shared_pool_size=1024M
*.shared_servers=10
*.sort_area_size=10485760
*.sql92_security=TRUE
*.standby_archive_dest='/redf/DB/arch'
*.standby_file_management='AUTO'
*.statistics_level='TYPICAL'
*.timed_statistics=TRUE
*.UNDO_MANAGEMENT='AUTO'
*.undo_retention=1800
*.undo_tablespace='UNDOTBS'
*.USER_DUMP_DEST='/redf/DB/dbf/dump'
Want to confirm is this a proper tuend settings as per memory provided ?
I am aware that it also depends on my appication behaviour , but i just want to check whether the above settings would suffice for a 16GB RAM machine and 64 bit oracle10g
Also let me know if any other setttings apart from this are required , i choose to use SGA Target .Further my application is read intensive . Your suggestions will be highly appreciated.
Thanx

Hi,
I suggest you to increase the value kernel.shmmax = 8589934592 of this parameter in sysctl.conf to kernel.shmmax = 1288490188
Later if you want to increase the size of SGA then you need to modify the kernel.shmmax = 8589934592 value. In general practice we keep kernel.shmmax value little greater then the value of SGA.
Refer MOS tech note:
*ORA-27102 Database Will Not Start With SHMMAX Set To 8589934592 (8GB) [ID 461519.1]*
*Linux Big SGA, Large Memory, VLM - White Paper [ID 260152.1]*
thanks,
X A H E E R
Edited by: ora_tech on Dec 26, 2010 10:14 AM

Sizing SGA

Hi All,
what is the standard method to resize sga, and what are all the things to be considered. Is there any formulat to do that.
Regards
Venu

There are couple of good notes in metalink. However, there is no any thumb rule for siging SGA. I would recommend you to start with a reasonable value and increase accordingly, if required. It alos depends on whether your db is OLTP or DSS.
Jaffar
null

Thump rule for SGA sizing depends on TOTAL DB SIZE

Is there any thump rule for sizing SGA depends on TOTAL database size for OLTP system?

Please dont do that. There was a thumb rule that if you are using oracle >=9i, you can go upto 80% of your physical RAM. But, I will not advice you so.
Start your SGA with an optimal value, keep on eye on your performance, take statspack report and if further increase required in the SGA, go ahed.
Because, setting over size SGA cause problems as well setting below par sga also provide poor performance. It should be sized optimially.
There are few good notes available in metalink.
Jaffar

CS6, Fluid Grid Layouts, and Spry Menu - sizing issues

Hi. I am a full-fledged nubie to Dreamweaver, so please forgive my ignorance. But I really need to build a new site, and I really like DW so far (much better than what I was using).
I am building a site with fluid grid layouts so it is properly sized across all decices. I have inserted a spry horizontal menu as my primary navigation. I got the menu to look like I want it as far as formatting goes.
But when I view the page live and change the size of my browser, the menu wraps around to the next row, even with fairly large browser windows.
Can anyone tell me how to scale the spry menu bar the same as an image does in a fluid grid layout?
Thanks,
Steven

Don't use Spry Menus. #1 they're not Responsive. #2 They don't work all that well on Touch Screen devices. #3 Adobe abandoned the Spry framework last year. Is that enough to convince you? See links below for alternatives.
Pop-Menu Magic2 by PVII (commercial DW Extension)
http://www.projectseven.com/products/menusystems/pmm2/index.htm
jQuery Superfish
http://users.tpg.com.au/j_birch/plugins/superfish/
jQuery Mega Menus http://www.javascriptkit.com/script/script2/jkmegamenu.shtml
CSS3 Dropdown Menus
http://www.red-team-design.com/css3-dropdown-menu
10 Responsive Menus
http://speckyboy.com/2012/08/29/10-responsive-navigation-solutions-and-tutorials/
Nancy O.

Query running long time

hi
I'm having a query running for long time, Im new to dba can any one suggest me methods to make it faster it's running now and i have to make it execute it faster
parallel servers=4, and there are no inactive sessions.
thanks in advance

Make a habit of putting the database version in the post
As i told u before i depends on lot of things not only merge(cartisian ) joins,
1)It depends on the load the database is having,Was this query running fastly before?if it was running fastly then was the workload same as today?
2)Any changes done to database recently or the server?
3)only this query is slow all the queris are slow?
4)When was database last restarted?
5)Are u using bind variable in the query?
6)Is you library cache properly sized?If the query is doing lots of sorts then is your PGA properly sized?
7)Database buffer cache is properly sized?
8)How much memory your database is having?
9)Is your SGA properly fits in your memory or its getting swaped?
Etc...Etc
Check all these things
Regards
Kaunain

Two database in same machine

Hai
I want to create two database in same machine. I searched in google but no use.
Is it possible in oracle 9i r2?
If so,Anybody please help on that.

jey84 wrote:
I want to create two database in same machine. You can have a 100 Oracle databases on the same machine. The issue though is why?
Why do you want to duplicate database overheads? Why two sets of system and process monitoring processes? Log writers? Database writers? Etc.
Why two SGAs? Why two system tablespaces? Two temp tablespaces, undo tablespaces? Two sets of redo logs? Etc.
You now have two database footprints. Instead of a single large SGA with properly sized buffer cache, shared pool, large pool and so on - you know have two smaller SGAs and smaller caches and pools that are now less capable and less scalable.
Why would you want to do this?
There are no sound technical reasons for running multiple database instances on a server - unless it is something like a 32 CPU/multicore server with 256 GBs of memory. But then why not use this as a cloud-type server on which you can run multiple server VMs? One VM for each database, as that provides more flexibility than multiple databases on a single server(physical or VM)?
In today's world of cluster and cloud computing, running 2 databases instances on the same server does not make much sense. And needs to be backed up by reason and logic that justify such an approach.

SGA_TARGET Doubt

Hi,
We have Oracle 10g database, which have the following parameters.
sga_max_size=1600m
shared_pool_size=704m
So planned to use SGA_TARGET and set the size to 2G.
(ie) sga_target=2G
sga_max_size=2G
Do I have to comment the shared_pool_size=704m ??
Regards,
Mahesh

Chinar wrote:
Aman.... wrote:
So I won't suggest that you remove the shared pool parameter completely. You may reduce the value given to it and start of from a smaller value rather than from the default one granule.
If you want to manage shared memory automatically(by enabling ASMM) there you should be give this to oracle fully automate it.So in general do not need for minimum values for specific automatically sized components.This is also important when investigating performance related problems in future(how application designed?).Hi Chinar
Sometimes your application needs a minimum amount of specific pool as it's written in the documentation:
You can exercise some control over the size of the automatically sized SGA components by specifying minimum values for the parameters corresponding to these components. Doing so can be useful if you know that an application cannot function properly without a minimum amount of memory in specific components. You specify the minimum amount of SGA space for a component by setting a value for its corresponding initialization parameter

How to refresh/redraw visuals in a mobile view

I have an issue with a list properly sizing itself. Here's my basic layout:
<s:Scroller width="100%" height="100%" >
        
        <s:Group width="100%" >
            <s:layout >
                <s:VerticalLayout gap="0" />
            </s:layout>
               
               <s:List width="100%" id="textBlocks"
                                    dataProvider="{calendar.taskDetails}"
                                    itemRenderer="components.BBWListMultilineRenderer">
                <s:layout>
                    <s:VerticalLayout variableRowHeight="true" gap="0" horizontalAlign="justify"/>
                </s:layout>
            </s:List >
            <components:BBWHeaderSubComp label="Team Members"/>
            <s:List width="100%"
                         dataProvider="{calendar.taskLogins}" />
The issue is with how the list handles the test block area. My custom renderer allows for multiple lines of text. Let's assume I've got two "blocks" of text (two rows within the list each with multiple lines of text) as shown in the screen shot below. If the first text block is larger than the second then the second block is drawn reserving the same height as the first. As you can see below, the second block "ends" and there is still more space to make up for the difference in size.
HOWEVER, if I make an orientation change (emulator or device) then the list instantly snaps everything to the right size.
Here is how it looks after the orientation change:
So, how do I get the visuals to update via code as I can't just tell users to switch their orientation? Thanks!

take a look at this:
Diagnosis
    In the view, not all of the mandatory fields were included as key
    fields. The key of the view therefore is not unique.
System Response
    All the fields are regarded as key fields in order to guarantee that th
    key lengths are unique.
Procedure
    For views with a large number of fields, this could have a negative
    effect on the performance for the language construction described below
    In this case, all the mandatory key fields should be included in the
    view.
regards

Slideshow in QT 7.1.3 in Windows: Frame Rate Fixed at 29.97

I have my sequentially numbered JPGS, Open Slide Show in File menu, highlight the first JPG, but then the Frame Rate is fixed at 29.97, no opportunity to set to 3 seconds per frame or 10 seconds a frame or anything except 29.97 per second.
I have uninstalled and reinstalled the program: no change. The program was downloaded recently from Apple and says that it is validating the install when the installer is run. Antivirus is switched off when the program is installed.

I thought this Windows version bug had been "squashed" in 7.0.4. Maybe not.
You should get two dialog windows when choosing "Image Sequence". If your files are properly sized and named sequentially you highlight just the first image file in the list and click OK. A second dialog window should then open that offers a "frame rate" for your import.
You say that it refuses to allow you to edit the frame rate at import? Bummer. A bug.
A simple workaround would be to accept any frame rate it offers and then open this new file. Select all (Control-A) and Copy (Control-C). This places your image sequence in your Clipboard.
Open any audio file and trim, or add to it, to fit the duration you need for your image sequence:
30 images at a duration of 3 seconds each would be 90 seconds of audio.
Select all of this newly trimmed audio and choose "Add to Selection & Scale" from the Edit menu. This will add your video track to the duration of the audio track.
Open the Movie Properties window and either "Extract" the video track or "Delete" the audio track and then Save As self-contained. mov file.
Maybe not what you wanted to hear.
Does the same issue occur under different user accounts? It may be something we could isolate.

How do I stop Firefox from claiming to have sessionrestore errors or opening the wrong session items and/or remember the wrong window size and/or position.

Since automatically updating to version 25 Firefox is randomly opening recent tabs and windows from the browsing history, and/or randomly positioning the browser window from history.
Currently using Firefox 25.0.1 on Windows Vista and Windows 7. Problems started immediately after automatic updates to 25.0.0. Problems are most noticeable on the Vista box because it has so much screen geography (the Windows 7 box is a laptop).
Problems include:
Firefox may open the correct tabs into a properly sized and positioned window, without problem. (Not a problem, included for completeness since it does, occasionally, work without error).
Firefox open a single empty tab (about:blank, my Home page). (Listed as a problem because Firefox simply forgot the tab which was open.)
Firefox may claim it cannot open the correct tabs and pop up about:sessionrestore. Proceeding will open all tabs (usually only one or two for me). This gives the impression Firefox did not actually have a problem. But, then, often, the tab Firefox is having problems restoring is about:blank so I should hope there was no problem!
Firefox may open randomly selected recent browsing items, without other problems. This gives the impression that Firefox does not properly record recent history for about:sessionrestore. Checking History appears correct; Firefox simply reached back hours, or days, or even weeks, and restored a random session from long ago.
Any of the above may occur, but the window position and/or size will be randomly selected from another recent history item. This is most noticeable when the position is on the wrong monitor and can be annoying since Firefox simply "disappears" if (as it often is for me) that other monitor is disconnected or powered off.

If you use cleanup software like CCleaner then check the settings for the Firefox application.<br />
Also make sure that your anti-virus software doesn't mess with files in the Firefox profile folder.
*http://kb.mozillazine.org/Profile_folder_-_Firefox
You can check for problems with the sessionstore.js and sessionstore.bak files in the Firefox profile folder that store session data.
Delete the sessionstore.js file and possible sessionstore-##.js files with a number and sessionstore.bak in the Firefox profile folder.
*Help > Troubleshooting Information > Profile Directory: Show Folder (Linux: Open Directory; Mac: Show in Finder)
*http://kb.mozillazine.org/Profile_folder_-_Firefox
*http://kb.mozillazine.org/Multiple_profile_files_created
Deleting sessionstore.js will cause App Tabs and Tab Groups and open and closed (undo) tabs to get lost and you will have to recreate them (make a note or bookmark them if possible).

LabVIEW 8.6.1 crashes

Good Day All,
   I've got an application where I'm getting intermittent crashes, sometime when in edit mode, sometimes when my program is actually running. I have gotten a couple of "MemoryManager.cpp line 437" crashes, but more often, but still intermittently, I just get the window that says something has happened to LabVIEW and it needs to be terminated. When you click on the more information link it pops up a window that says:
"AppName:labview.exe   AppVer: 8.6.0.4001 ModName: Unknown
ModVer.: 0.0.0.0        Offset 6f636974"
This happens when running the application or when editing. It may happen twice in an hour, or after several hours of running. I've watched the memory and thread usage in "Task Manager" and they seem to be stable.
The program has two main parts, a UI that at the beginning of the test run uses a .NET call to read back data over the network from the Unit Under Test, parse the returned XML string (using string functions, no "XML parsing" functions from LabVIEW or Windows) and a user interface to start the program. The UI sends a message over a queue to the "Test Engine" which then, depending on the "test recipe" installs and launches the appropriate test program using a "Call by Reference Node". The test program makes measurements of the UUT and saves the results to an Excel file using NI Report vi's from the NI Report Generation Toolkit. The measurements are visa calls to standard "rack and stack" instruments like spectrum analyzers and signal generators.
I've done a moderately deep search on the NI site, haven't found anything that looks like the issue, need ideas how to trap this beast and eliminate it.
I hope that many of the major players being at NIWeek won't impact the response time. It seems like those years I haven't been able to attend are the ones where I hit a snag like this one. I guess the moral is that I should always attend
Thanks in advance,
Putnam
Putnam
Certified LabVIEW Developer
Senior Test Engineer
Currently using LV 6.1-LabVIEW 2012, RT8.5
LabVIEW Champion
Solved!
Go to Solution.

Looking at this specific error, LabVIEW's memory manager has decided that a memory handle is invalid. One case in which this can happen is when different versions of LabVIEW are in use simultaneously and a data handle is passed from one version of LV to another. One such scenario would be:
LabVIEW 2009 calls the function 'myFunc' from 'foo.dll'
'foo.dll' was built using LabVIEW 8.6
'myFunc' expects an array of data, passed as a LabVIEW array handle
The problem arises in this case because the array handle allocated by the LabVIEW 2009 instance of the LV memory manager is sent to a VI running in the LabVIEW 8.6 Run-Time Engine, which has the LabVIEW 8.6 version of the memory manager. Because the data is an array handle, and LabVIEW may need to resize the handle, the code in the LV 8.6 Run-Time inspects the data handle to see if its memory manager considers it a 'good' memory handle (i.e. one that it allocated itself). It discovers that the LV 8.6 Run-Time didn't allocate the handle, and, being mistrustful of data from other memory managers ( ;-) ) it has no choice but to abort the application. It's either that or corrupt memory.
In such cases, a workaround is to pass array pointers (or strings) that cannot be resized -- an unsavory situation if the underlying implementation must resize the array (or string). The workaround in that case is more complex - e.g.. having two functions - a 'preflight' that tells you how big the output will be, and the 'actual' function that requires a properly sized output buffer.
For more complex data structures (e.g.. clusters) the workaround is to break apart the cluster into individual, simple types that can be passed as pointers to data or simple scalar values.
Whether this is at all similar to your case is unclear - it sounds like there are several component layers involved with different technologies. But the crash itself is still that LabVIEW 8.6 has gotten a memory handle from somewhere that's *not* been allocated by the LabVIEW 8.6 memory manager.
One thing you could do with Process Explorer and similar tools is determine if there are multiple versions of LabVIEW in the main process. For example, if both LabVIEW.exe and one or more lvrt.dll instances from different locations on disk are in memory, you've found a red flag. The quest then is to find out which components in your software stack are responsible for bringing in those different versions of LabVIEW, and how they're interacting via code.
Best regards,
intvsteve
LabVIEW R&D

Extremely Poor Read Performance

Hey guys,
For a work project, I have been instructed to use a Berkeley DB as our data storage mechanism. Admittedly, I know little about BDB, but I've been learning more in the past day as I am reading up on it. I'm hoping, though, that even if no one can help me with the problem I am having, they can at least tell me if what I am seeing is typical/expected, or definitely wrong.
Here's what I got:
- Parent table A - Has 0 or 1 key for table B, and 0 or 1 key for table C
- Table B
- Table C
For purpose of discussion, let's ignore table C as it is logically the same as Table B.
Table B has 25 million rows, keyed by a 34-36 digit string, and a payload of 500-1000 bytes.
Table A has 26 million rows, 25 million of which reference the 25 million rows in Table B.
My question is not on the merits of why the data is structured the way it is, but rather about the performance I am seeing, so please refrain from questions such as "why is your data structured that way - can you structure it another way?" I know I can do that - again I just want to know what other people are experiencing for performance.
Anyway, what's happening is this - my program runs a cursor on Table A to get all records. As it gets each record in Table A, it retrieves the referenced records in Table B. So, the cursor on table A represents sequential key access. The subsequent retrievals from Table B represent "random" retrievals - i.e. the key may be anywhere in the index, and is not at all related to the previous retrieval.
Cruising the cursor on Table A, I am seeing performance of about 100,000 records per 2 seconds. However, when I add in the retrievals from Table B, performance stoops all the way down to 100,000 records per 1000 seconds, or better put 100 per second. At this rate, it will take nearly 70 hours to traverse my entire data set.
My question is, am I simply running into a fundamental hardware issue in that I am doing random retrievals from Table B, and I cannot expect to see better performance than 100 retrievals per second due to all of the disk reads? Being that the DB is 20 GB in size, I cannot cache the entire table in memory, so does that mean that reading the data in this fashion is simply not feasible?
If it isn't feasible, does anyone have a suggestion on a different way to read the data, without changing the table relationship as it currently stands? Considering Table B has a reverse reference to table A, I've considered putting a secondary index on table B so that instead of doing random seeks into table B, I can run a cursor on the secondary index of table B at the same time I run the cursor on table A. Then, for each record in table A that has a reference to table B, the first record in the cursor for table B should be the one I need. However, reading about secondary indexes, it looks like all a secondary index does is give a reference to the key to the table. Thus, my concern is that running a cursor on the secondary index of table B will really be no different than randomly selecting the records from table B, as it will be doing just that in the background anyway. Am I understanding this correctly, or would a secondary index really help in my case?
Thank you for your time.
-Brett

Hi Brett,
Is the sorting order the same between the two databases, A and B, that is, are the keys ordered in the same way? For example, to key N in database A, key N in database B is referred.
I would guess not, because you mention the "randomness" in retrieving from B when doing the cursor sequential traversal of A, and the 34-36 digit keys in B are probably randomly generated.
With B as a secondary database, associated to A as the primary database, it would make sense having a cursor on secondary database B to iterate, if you expect that the same ordering of keys in A (as mentioned in the beginning of this post). For example, you would use DBcursor->get to iterate in the secondary database B, or DBcursor->pget if you also want to retrieve the key from the primary database A: DBcursor->get(), DBcursor->pget()
Basically secondary indexes allow for accessing records in a database (primary db) based on a different piece of information other than the primary key:
Secondary indexes
So, when you iterate with a cursor in B you would retrieve the data from A (and in addition the key from A) in the order given by the keys (secondary keys) in B.
However, a secondary database does not seem to me feasible in your case. You seem to have about 1 mil records in primary db A for which you would not have to generate a secondary key, so you would have to return DB_DONOTINDEX from the secondary callback: DB->associate()
(it may be difficult to account exactly for the records in A for which you do not want to generate secondary keys)
Also, the secondary key, the 34-36 digit string, would have to somehow be derived from the primary key and data in A.
If the ordering is not similar (in the sense explained at the beginning of the post) between A and B, then having the secondary index does not too much value, other than simplifying retrieval from A in queries where the query criteria involves the 34-36 digit string.
Back to your current way of structuring data, there are some suggestion that could improve retrieval times:
- try using the latest Berkeley DB release, 5.1.19: Berkeley DB Release History
- try configuring a page size for the databases A and B equal to that of the filesystem's block size: Selecting a page size
- try to avoid the creation of overflow items and pages by properly sizing the page size -- you can inspect database statistics using db_stat -d: db_stat
- try increasing the cache size to a larger value: Selecting a cache size
- if there's a single process accessing the environment, try to back the environment shared region files in per-process private memory (using DB_PRIVATE flag in the DB_ENV->open() call);
- try performing read operations outside of transactions, that is, do not use transactional cursors.
For reference, review these sections in the Berkeley DB Reference Guide:
Access method tuning
Transaction tuning
Regards,
Andrei

Is properly sized sga?

Similar Messages

Maybe you are looking for