Why Berkeley DB caches entire ".db" file to Linux memory?

Hi,
We use the sample codes from the Berkeley documents to write a large amount of data to Berkeley DB in Linux. We found following problems:
1. System free memory drops quickly as more (key,data) pairs are written into db.
2. When we delete ".db" file, the memory is reclaimed. The system free memory increases to normal level.
3. When we run the same program in Windows, there is no such problem.
It looks that Linux cache the whole ".db" file to the system memory.
It is basically the same problem from this post
Berkeley DB is taking up too much memory
We have tried to use different flags such as DB_PRIVATE or DB_SYSTEM_MEM. It doesn't seems to work.
Many thanks for your help.

Hi there,
841393 wrote:
We use the sample codes from the Berkeley documents to write a large amount of data to Berkeley DB in Linux. We found following problems:What sample codes are you talking about?
What flags are you using when opening the database and the environment?
Do you want for the database to be stored in memory or on disk?
Bogdan Coman

Similar Messages

Reading entire txt file into memory?

When you are using BufferedReader to read info into a buffer, that means you are reading the file into memory, correct? (Is that what buffer means?)
I want to look for pattern matches in text files (about 1000 of them) using the regex utils. But I don't want to read and examine the text files line by line. I want to read in the entire text file into memory first and then look for the pattern matches. The text files generally don't exceed about 15K in size. I'm only going one file at a time, too, so this won't give me any out of memory errors, will it?
And more importantly, how do I do it? I mean the "reading in the file" part only. I have my RegEx, I have my array of files to examine already. I just can't figure out the right code to use to read each file into memory before I look for pattern matches.
Could someone help, please?

When you are using BufferedReader to read info into a
buffer, that means you are reading the file into
memory, correct? (Is that what buffer means?)Yes.
I want to look for pattern matches in text files
(about 1000 of them) using the regex utils. But I
don't want to read and examine the text files line by
line.Why not?
I want to read in the entire text file into
memory first and then look for the pattern matches.Why?
The text files generally don't exceed about 15K in
size. I'm only going one file at a time, too, so
this won't give me any out of memory errors, will
it?Depends on how much memory you've given the VM and how much of that it's using already at the time you read the files, but in general, probably not a problem.
And more importantly, how do I do it? I mean the
"reading in the file" part only.Use BufferedReader to read line by line and then append each line (plus a newline, since BR.readLine() strips those off) to a StringBuilder.
Or use a BufferedInputStream and and array that's as big as the file, and in a loop, try to read as much as is left into that array at an offset equal to how much has been read so far.
I still think this is probably not a good approach though.

Cannot Read Entire TXT Files in "Notes" folder

Ihave a 4G iPod nano (red, if relevant... I think it's a 4g?) I cannot read entire text files in the notes folder. I can read part of the file, usually up to about a 2 KB size in standard Unicode format, but I am unable to view the entire text file. For example, I have a copy of Beowulf on my iPod that is about 267KB in size; however, I can only read through 1.87KB of it. The rest shows up in Windows Explorer and is fully functional there, but I Cannot view it from my iPod.
Anyone else have this problem, and know what I can do to fix this (besides breaking it up into about 50 chapters?)
Thanks Much!
intel.ectual
programmer, code-guru, and graphic designer
Compaq Presario Athlon 1.7GHz 64-bit 1GB RAM and a very comepetent user Windows XP Pro
Compaq Presario Athlon 1.7GHz 64-bit 1GB RAM and a very comepetent user Windows XP Pro

The max size for any note that can be read in the notes on ipod is 4kb. That said there is no alternative way of getting around this size. here is a good article about working with ipod notes.
http://www.macworld.com/2004/09/secrets/septgeekfactor/index.php
and on another note from another of your posts.
jsut to make you aware of not doing something stupid adn cursing about it.
DO NOT TRY AND USE IPODLINUX ON YOUR IPOD NANO.
unless you really want to VOID your warranty and also fry your nano to become a expensive PAPERWEIGHT!!
many have stupidly done this and have come here cursing as to why their ipod nano is fried adn not working at all! Just be smart and make note that that hack software doenst even work for the 2g ipod nano!
GFF:)

Shifting entire site file structure! (and other issues!!)

Hi Folks
I've been working on a totally new look for my site and have pretty much finished it off. in my site file structure within my httpdocs folder I have my current index page and associated files, also within here I have a folder called Beta2 which is home to my new site (within this is the entire new file structure).
Now I'm ready to launch the 'new' site, but how do I get all the files from the Beta2 file to the main httpdocs folder without buggering all the links up!
Every time I've tried so far all my includes fail and all the links become defunct.
I should point out I'm not running a test server all my testing is online so I have a local copy of the file structure and an online copy.
On a slightly different point (an html one) can anyone work out why including this - in my form causes it to stop submitting! (it calls up a box displaying if the entered membershipno already exists or not, the function works fine, and the form submits fine, but when you put the two together only the function works, the forms doesn't submit!)
Membership Number :
 <input name="membershipno" type="text" id="membershipno" maxlength="7" />
 
 

This is partly a shameless bump, but also to say ignore the code issue I just really need to change file structure pretty quickly (by the 11th March!!)
If anyone can offer an explaination I'd really really appreciate it
Thanks a lot

Why the size of the tif file is different in photoshop workspace compare to the Preview in Finder?

1. Why the size of the tif file is different in photoshop's workspace compare to the Preview in Finder?
Please look at the screen captures.
Bottom left panel of Photoshop's workspace.
Finder's Preview.
2. I have an imac with 8gb ram, and assigned 6gb ram to photoshop use. But when I worked on a 24mb tif file, photoshop was lagging. And at the bottom left panel of Photoshop's workspace, the doc size is something/1gb. The tif file is only 24mb, why is it slow in photoshop? I have a lot of ram.
Let me know, please.

Thank you MIchelPM for giving me a very detailed explanation.
So I understood why the photoshop doc size is bigger when working in photoshop.
Regarding to your questions:
1. How full is your Mac's hard drive?
    Not even 30% full, I even have a 1TB backup hard drive connected.
2. How many applications do you run simultaneously in the background while working in another application?
    I usually only have photoshop or illustrator opened. Sometimes I ran 3-4 tif files at the same time, but when a big file (ex. 24mb tif) gets very slow, I closed all other files and programs, and "purged all" in photoshop, yet that 24mb tif file was still slow.
   I checked photoshop's ram and cpu usage in the activity monitor. Rarely photoshop's cpu usage would hit 100. But when I worked with the 24mb tif file, it would used all the ram that I've allocated to photoshop.
But the myth is, my coworker who worked with the same file in CS4 with only 4gb ram in their imac don't have photoshop slowed down, when I have double the ram and CS6.
3. What year, screen size, CPU speed of your Mac?
   I have 27" imac,
   • 2.9GHz 2.9GHz quad-core Intel Core i5 processor (Turbo Boost up to 3.6GHz) with 6MB L3 cache,
   • 1TB (7200-rpm) hard drive
   • NVIDIA GeForce GTX 660M graphics processor with 512MB of GDDR5 memory
4. What version of OS X is your Mac using?
    Mountain Lion with the latest updates.
5. What version of Photoshop are you using?
   • Photoshop CS6

Why can't I open DWG files in Adobe Illustrator CS6?

Why can't I open DWG files in Adobe Illustrator CS6? I recently upgraded to Mac OS X Mountain Lion & got a new Mac Pro. Then I signed up for the Creative Cloud. Now I cannot open any DWG files in Illustrator. Why?

No, not really. I do get a few error messages when I start up now, but I had transferred everything from my old computer, so figured it was a glitch. The error message was a dialogue box like the others, only it said this:
"Insecure start-up item disabled. Library/Startup Items/.DS_Store/ has not been started because it does not have the proper security settings."
"Insecure start-up item disabled. Library/Startup Items/ES1000Service/ has not been started because it does not have the proper security settings."
"Insecure start-up item disabled. Library/Startup Items/Parallels/ has not been started because it does not have the proper security settings."
I know for sure we have not used Parallels in the past 2 years that I have been here.
As for the other two star-tup items, I am not entirely sure what thier purpose is.
Would any of these have anything to do with it?

Is it possible to personalize the size or the Path to your cache, history, temporary files and others?

Is it possible to personalize the size or the Path to your cache, history, temporary files and others? If it is, how can I do it?
I explain why. I am willing to create a RamDisk partition where I would place that kind of files with support to a bigger size, that way increasing my speed in Internet, and avoiding permanent writes in the SSD drive.
I am in i7 920 CPU, with system and programs working in a SSD drive and have 24 GB of RAM memory.

Hi cor-el
In my Desktop I have the 9.0.1 version of Firefox.
I don’t know if it is or not the portable one, but I presume not.
I couldn’t follow your first link, because page is not found.
The second link takes me to somewhere where I have already been and that gives no solution to my question.
I don’t want to disable cache in Firefox. By the contrary, I want it enabled and I’m trying to make it more efficient, making it bigger and placed in a very fast driver (A RamDisk driver, which is a driver made from RAM memory).
'''''What I want is to configure by myself the place where the cache and other temporary files are placed in my computer, and their size.'''''
So, I repeat my first question. Is this possible?
Although I am Portuguese you don´t need to redirect me to Portuguese pages, cause I’m perfectly familiar with English.

Why can't i send multiple files through airdrop ? , it accepts but shows only one photo

why can't i send multiple files through airdrop ? , it accepts but shows only one photo

Hello there Laurengrayce,
I message all of the time and I want you to do the same, to anyone you would like! Demmo provided a good suggestion of having this person check their services and ensuring they're setup as a 10-digit number, but I would like to provide some of my own suggestions.
If they are in fact already setup as a 10-digit number, I would recommend deleting this persons contact from your phone as well as the entire messaging thread (long press and hold on thread to delete). Readd them to your contact list and try again.
Please advise if this continues and they've already checked with their provider.
Thanks,
MelissaM_VZW
Follow us on Twitter @vzwsupport

Why does it show 1,435 files to be printed?

Why does the printing icon in the side bar show that there are 1,435 files to be printed? When I click on it, iPhoto freezes and I need to Force Quit so I cannot identify the files that are in this location.

If clearing the cache and preference file doesn't as a test launch iPhoto with the Option key held down and try to create a new, test library. If yhou can import some photos and test to see if the same problem persists. Does it?
If it doesn't then your library is damage and needs to be repaired. apply the two fixes below in order as needed:
Fix #1
1 - launch iPhoto with the Command+Option keys held down and rebuild the library.
2 - run Option #4 to rebuild the database.
Fix #2
Using iPhoto Library Manager to Rebuild Your iPhoto Library
1 - download iPhoto Library Manager and launch.
2 - click on the Add Library button and select the library you want to add in the selection window..
3 - Now that the library is listed in the left hand pane of iPLM, click on your library and go to the Library ➙ Rebuild Library menu option.
4 - In the next window name the new library and select the location you want it to be placed.
5 - Click on the Create button.
Note: This creates a new library based on the LIbraryData.xml file in the library and will recover Events, Albums, keywords, titles and comments. However, books, calendars, cards and slideshows will be lost. The original library will be left untouched for further attempts at fixing the problem or in case the rebuilt library is not satisfactory.
OT

Why is my cache results not highligted anymore?

why is my cache results not highligted anymore?

When I do a word/s search.
Normally the words I typed and searched on Google will be highlighted in different colors each word at the top of the webpage.
The same color will be highligted for each word/s whenever they appears all throught out the page. when I open it as cached.
Now the words don't highlight any more and no color even.
Here below info from About.com about what I am asking about:
http://google.about.com/od/searchingtheweb/qt/cache_syntax.htm
''''''Highlight Keywords With Google Cache Search''''''
Find Specific Information Faster With Google's Cache
By Marziah Karch, About.com Guide
Is it hard to find a specific piece of information on a large Web page? You can simply this by using Google's cached page to highlight your search term.
As Google indexes Web pages, it retains a snapshot of the page contents, known as a cached page. When a cached page is available, you'll see a Cached link at the bottom of the search result.
Clicking on the Cached link will show you the page as it was last indexed on Google, but with your search keywords highlighted. This is extremely useful if you want to find a specific piece of information without having to scan the entire page.
Keep in mind that this shows the last time the page was indexed, so sometimes images will not show up and the information will be out of date. For most quick searches, that doesn't matter. You can always go back to the current version of the page and double check to see if the information has changed.

A error message from running the jpa-cache-server.cmd file

When I try to run jpa-cache-server.cmd file, it show the following error message:
"failed to load main-class manifest attribute from %coherence_home%\lib\coherence-jpa.jar"
What is the problem?

Forgive me but I do not see that file (jpa-cache-server.cmd) in the Coherence 3.5 distribution. With that said, I don't believe that coherence-jpa.jar is a executable jar. What exactly are you trying to do?
If you are trying to start a cache server with the jpa classes available to it I would suggest just adding the coherence-jpa.jar to the class path.
Rob
:Coherence Team:

Java API for running entire ".sql" files on a remote DB ( mySQL or Oracle)?

Hi,
Would anyone happen to know if there's a java API for executing entire ".sql" files (containing several different SQL commands), on a remote database server ?
It's enough if the API works with MySQL and/or Oracle.
Just to demonstrate what i'm looking for:
Suppose you've created sql file "c:/test.sql" with several script lines:
-- test.sql:
insert into TABLE1 values(3,3);
insert into TABLE1 values(5,5);
create table TABLE2 (name VARCHER) ENGINE innoDB; -- MYSQL specific
Then the java API should look something like:
// Dummy java code:
String driver="com.mysql.jdbc.Driver";
String url= "jdbc:mysql://localhost:3306/myDb";
SomeAPI.executeScriptFile( "c:/test.sql", driver, url);
Thanks.

No such a API, but it's easy to parse all sqls in a file, then run those command:
For instance:
import java.sql.*;
import java.util.Properties;
/* A demo show how to load some sql statements. */
public class testSQL {
private final static Object[] getSQLStatements(java.util.Vector v) {
Object[] statements = new Object[v.size()];
Object temp;
for (int i = 0; i < v.size(); i++) {
temp = v.elementAt(i);
if (temp instanceof java.util.Vector)
statements[i] = getSQLStatements( (java.util.Vector) temp);
else
statements[i] = temp;
return statements;
public final static Object[] getSQLStatements(String sqlFile) throws java.
io.IOException {
java.util.Vector v = new java.util.Vector(1000);
try {
java.io.BufferedReader br = new java.io.BufferedReader(new java.io.
FileReader(sqlFile));
java.util.Vector batchs = new java.util.Vector(10);
String temp;
while ( (temp = br.readLine()) != null) {
temp = temp.trim();
if (temp.length() == 0)
continue;
switch (temp.charAt(0)) {
case '*':
case '"':
case '\'':
// System.out.println(temp);
break; //Ignore any line which begin with the above character
case '#': //Used to begin a new sql statement
if (batchs.size() > 0) {
v.addElement(getSQLStatements(batchs));
batchs.removeAllElements();
break;
case 'S':
case 's':
case '?':
if (batchs.size() > 0) {
v.addElement(getSQLStatements(batchs));
batchs.removeAllElements();
v.addElement(temp);
break;
case '!': //Use it to get a large number of simple update statements
if (batchs.size() > 0) {
v.addElement(getSQLStatements(batchs));
batchs.removeAllElements();
String part1 = temp.substring(1);
String part2 = br.readLine();
for (int i = -2890; i < 1388; i += 39)
batchs.addElement(part1 + i + part2);
for (int i = 1890; i < 2388; i += 53) {
batchs.addElement(part1 + i + part2);
batchs.addElement(part1 + i + part2);
for (int i = 4320; i > 4268; i--) {
batchs.addElement(part1 + i + part2);
batchs.addElement(part1 + i + part2);
for (int i = 9389; i > 7388; i -= 83)
batchs.addElement(part1 + i + part2);
v.addElement(getSQLStatements(batchs));
batchs.removeAllElements();
break;
default:
batchs.addElement(temp);
break;
if (batchs.size() > 0) {
v.addElement(getSQLStatements(batchs));
batchs.removeAllElements();
br.close();
br = null;
catch (java.io.FileNotFoundException fnfe) {
v.addElement(sqlFile); //sqlFile is a sql command, not a file Name
Object[] statements = new Object[v.size()];
for (int i = 0; i < v.size(); i++)
statements[i] = v.elementAt(i);
return statements;
public static void main(String argv[]) {
try {
String url;
Object[] statements;
switch (argv.length) {
case 0: //Use it for the simplest test
case 1:
url = "jdbc:dbf:/.";
if (argv.length == 0) {
statements = new String[1];
statements[0] = "select * from test";
else
statements = argv;
break;
case 2:
url = argv[0];
statements = getSQLStatements(argv[1]);
break;
default:
throw new Exception(
"Syntax Error: java testSQL url sqlfile");
Class.forName("com.hxtt.sql.dbf.DBFDriver").newInstance();
//Please see Connecting to the Database section of Chapter 2. Installation in Development Document
Properties properties = new Properties();
Connection con = DriverManager.getConnection(url, properties);
Statement stmt = con.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_READ_ONLY);
//Statement stmt = con.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE,ResultSet.CONCUR_UPDATABLE);
// stmt.setMaxRows(0);
stmt.setFetchSize(10);
final boolean serializeFlag = false;//A test switch to serialize/deserialize the resultSet
ResultSet rs;
for (int i = 0; i < statements.length; i++) {
if (statements[i] instanceof java.lang.String) {
String temp = (java.lang.String) statements;
switch (temp.charAt(0)) {
case 'S':
case 's':
case '?':
System.out.println(temp);
rs = stmt.executeQuery(temp);
if (serializeFlag) {
// serialize the resultSet
try {
java.io.FileOutputStream fileOutputStream = new
java.io.FileOutputStream("testrs.tmp");
java.io.ObjectOutputStream
objectOutputStream = new java.io.
ObjectOutputStream(fileOutputStream);
objectOutputStream.writeObject(rs);
objectOutputStream.flush();
objectOutputStream.close();
fileOutputStream.close();
catch (Exception e) {
System.out.println(e);
e.printStackTrace();
System.exit(1);
rs.close(); //Let the CONCUR_UPDATABLE resultSet release its open files at once.
rs = null;
// deserialize the resultSet
try {
java.io.FileInputStream fileInputStream = new
java.io.FileInputStream("testrs.tmp");
java.io.ObjectInputStream objectInputStream = new
java.io.ObjectInputStream(
fileInputStream);
rs = (ResultSet) objectInputStream.
readObject();
objectInputStream.close();
fileInputStream.close();
catch (Exception e) {
System.out.println(e);
e.printStackTrace();
System.exit(1);
ResultSetMetaData resultSetMetaData = rs.
getMetaData();
int iNumCols = resultSetMetaData.getColumnCount();
for (int j = 1; j <= iNumCols; j++) {
// System.out.println(resultSetMetaData.getColumnName(j));
/* System.out.println(resultSetMetaData.getColumnType(j));
System.out.println(resultSetMetaData.getColumnDisplaySize(j));
System.out.println(resultSetMetaData.getPrecision(j));
System.out.println(resultSetMetaData.getScale(j));
System.out.println(resultSetMetaData.
getColumnLabel(j)
+ " " +
resultSetMetaData.getColumnTypeName(j));
Object colval;
rs.beforeFirst();
long ncount = 0;
while (rs.next()) {
// System.out.print(rs.rowDeleted()+" ");
ncount++;
for (int j = 1; j <= iNumCols; j++) {
colval = rs.getObject(j);
System.out.print(colval + " ");
System.out.println();
rs.close(); //Let the resultSet release its open tables at once.
rs = null;
System.out.println(
"The total row number of resultset: " + ncount);
System.out.println();
break;
default:
int updateCount = stmt.executeUpdate(temp);
System.out.println(temp + " : " + updateCount);
System.out.println();
else if (statements[i] instanceof java.lang.Object[]) {
int[] updateCounts;
Object[] temp = (java.lang.Object[]) statements[i];
try {
for (int j = 0; j < temp.length; j++){
System.out.println( temp[j]);
stmt.addBatch( (java.lang.String) temp[j]);
updateCounts = stmt.executeBatch();
for (int j = 0; j < temp.length; j++)
System.out.println((j+1)+":"+temp[j]);
for (int j = 0; j < updateCounts.length; j++)
System.out.println((j+1)+":" +updateCounts[j]);
catch (java.sql.BatchUpdateException e) {
updateCounts = e.getUpdateCounts();
for (int j = 0; j < updateCounts.length; j++)
System.out.println((j+1)+":"+updateCounts[j]);
java.sql.SQLException sqle = e;
do {
System.out.println(sqle.getMessage());
System.out.println("Error Code:" +
sqle.getErrorCode());
System.out.println("SQL State:" + sqle.getSQLState());
sqle.printStackTrace();
while ( (sqle = sqle.getNextException()) != null);
catch (java.sql.SQLException sqle) {
do {
System.out.println(sqle.getMessage());
System.out.println("Error Code:" +
sqle.getErrorCode());
System.out.println("SQL State:" + sqle.getSQLState());
sqle.printStackTrace();
while ( (sqle = sqle.getNextException()) != null);
stmt.clearBatch();
System.out.println();
stmt.close();
con.close();
catch (SQLException sqle) {
do {
System.out.println(sqle.getMessage());
System.out.println("Error Code:" + sqle.getErrorCode());
System.out.println("SQL State:" + sqle.getSQLState());
sqle.printStackTrace();
while ( (sqle = sqle.getNextException()) != null);
catch (Exception e) {
System.out.println(e.getMessage());
e.printStackTrace();

XI 7.0 Advnaced User Defined Function how to check Cache Entire Queue ?

Hi Friends ,
                   We are using XI 7.0 (PI)   . There is no Selection For Advanecd UserDefinedFunction . I think in this version Both will perfom in the User DefinedFunction Itself .
           There is option for Context and Queue While we are creating Function . If we select Queue is it eqivalent to Check box of Cache Entire Queue ? Is My assuption is correct ?
          I want to cache the Entire Queue ? Can you please tell me about this.
          My environment is like
Session Information
Application:
Design: Integration Builder
User:
Test
Logon Language:
English
Server:
xidev_XID
Server Node:
server0
Runtime Environment
Java version:
1.4.2_13
Java vendor:
Sun Microsystems Inc.
Version
Service pack:
10
Release:
645_VAL_REL
Best Regards .,
V.Rangarajan

you advanced user defined functions are now known as
Enhanced Functions (Cache = Context or Cache = Queue)
From Help -
+Features
Depending on which values are available in the cache for a user-defined function, different functions are available:
·        Simple Functions (Cache = Value)
This function type can process individual input values of a field for each function call. Therefore, simple functions expect strings as input values and return a string.
·        Enhanced Functions (Cache = Context or Cache = Queue)
This function type can process multiple input values of a field for each function call. Before you call the function, you can either import all the field values of a context or the whole queue for the field in an array.+
Have this link as your reference - http://help.sap.com/saphelp_nw04s/helpdata/en/22/e127f28b572243b4324879c6bf05a0/content.htm

Why can I not open PDF-files in InDesign CS4?

Hi,
Well then, as I wrote in the headline... Why can I not open PDF-files?
I get a notification that says that either the file is already open (which it's not) or it is missing a plug-in thingy for this kind of file.
So, how do I know which plug-in thing I should chose from Adobe which will solve my problem?
Could it be enough just to update the program? I tried with 6.0.6. upgrade but it wouldn't even open up. I'm not so good at this type of computerproblems (choosing upgrades, plug-in etc.). The program is not upgraded for a long time, unless it do it automatically, but I think we've chosen it not to....
Please help!
/S

See: Zanelli Multipage Importer
http://indesignsecrets.com/zanelli-releases-multipageimporter-for-importing-both-pdf-and-i ndd-files.php/comment-page-1

Why I cannot open camera raw files from camera Nikon 3200 through Adobe Bridge and/or Photoshop?

Why I cannot open raw files (NEF) from camera Nikon D3200 through Adobe Bridge and/or Photoshop ?

Thanks for the tip. It worked out perfectly.
De: ssprengel [email protected]
Enviada em: terça-feira, 14 de janeiro de 2014 19:14
Para: Aecio Paes Leme
Assunto: Why I cannot open camera raw files from camera Nikon 3200 through Adobe Bridge and/or Photoshop?
Re: Why I cannot open camera raw files from camera Nikon 3200 through Adobe Bridge and/or Photoshop?
created by ssprengel <http://forums.adobe.com/people/ssprengel> in Adobe Camera Raw - View the full discussion <http://forums.adobe.com/message/6012775#6012775

Why Berkeley DB caches entire ".db" file to Linux memory?

Similar Messages

Maybe you are looking for