Too many open cursors exception caused by LRS Iterator
Using Kodo4.1.4 with Oracle10, and Large Result Set Proxies, I encountered
the error "maximum number of open cursors exceeded".
It seems to have been caused because of incomplete LRSProxy iterators within
the context of a single PersistenceManager. These iterators were over
collections obtained by reachability, not directly from Queries or Extents.
The Iterator is always closed, but the max-cursors exception still occurs.
Following is a pseudocode example of the case... Note that if the code is
refactored to remove the break; statement, then the program works fine, with
no max-cursors exception.
Any suggestions?
// This code pattern is called hundreds of times
// within the context of a PersistenceManager
Collection c = persistentObject.getSomeCollection(); // LRS Collection
Iterator i = c.iterator()
try
while(i.hasNext())
Object o = i.next();
if (someCondition)
break; // if this break is removed, everything is fine
finally
KodoJDOHelper.close(i);
}
XSQL Servlet v. 0.9.9.1
Netscape Enterprise / JRUN 2.3.3 / Windows NT
I modified the document demo (insert request).
The XSQL document:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="newdocinsform.xsl"?>
<page connection="demo" xmlns:xsql="urn:oracle-xsql">
<xsql:insert-request table="xmlclob" transform="newdocins.xsl"/>
<data>
<xsql:query null-indicator="yes" max-rows="4">
select id, doc
from xmlclob
order by id desc
</xsql:query>
</data>
</page>
The difference between this and your demo is the table: the table xmlclob has
ID NUMBER and DOC CLOB. No constraints were enforced, so I was inserting the ID and the DOC. Upon page reload, several rows with the same values were inserted.
I had a similar problem in the previous release.
As a general question, how can I configure the XSQLConfig file for optimal performance?
Although you provided default values, I'm not sure how much is necessary for connection pooling.
Similar Messages
-
Report causes too many open cursors
Hello there!
I've got the following situation:
I've a very heavy report used for generating our Users Manual. In Reports 6i this Report works fine, generating the Manual works.
In 10g the Report starts, and formats about 240 pages (in 6i I can generate over 1000 pages and more with this report), and cancels with the message "too many open cursors".
So I took a look at the open cursors:
In 6i there are about 100 open cursors caused by this report; in 10g there are...uhm...in all cases to much for the max_open_cursors parameter of the database (standard value which is used by our application is 1000; increasing this to e.g. 5000 resulted in the same behaviour => too many open cursors).
Checked the open cursors while running the report which showed the following behaviour:
The report formats about 230 pages, and opens about 20 cursors (~30 sec.). for the next 10 sites the report opens the pending 980 cursors (~5 sec.), and stops formatting...
So it seems the report server causes some bad recursion: When restarting the reports server and re-running the report, I get sometimes the following error:
Mit Fehler beendet: REP-536870981: Interner Fehler REP-62204: Interner Fehler beim Schreiben des Bildes BandCombine: a row of the matrix does not have the correct number of entries, should be OpImage.getExpandedNumBands(source0.getSampleModel(), source0.getColorModel()) + 1.. REP-0069: Interner Fehler REP-50125: Exception abgefangen: java.lang.NullPointerException REP-0002: Unable to retrieve a string from the Report Builder message file. REP-536870981:
or maybe the report server tries to paralellize some querys (as this report consists of about 5 querys)?
As said - this is a very complex report (my colleague spent about 3 months of his life with creating it and that's not why he is a lamer in reports ;-)) so it's very hard to give you a repcase, but if anyone knows some advice like "edit the <repservername>.conf; append 'DO NEVER EVER PARALLELYZE QUERYS' to the config" or something this would be very useful ;-).
many thanks
best regards
ChristianI've now located the problem:
The report consists of several querys based on a ref cursor; and this cursors are opend and not closed in 10g...
I'll open a SR on metalink....
best regards
Christian -
"Too many open files" Exception on "tapestry-framework-4.1.1.jar"
When a browser attempts accessing to my webwork, the server opens a certain number of file descriptors to "tapestry-framework-4.1.1.jar" file and don't release them for a while.
Below is the output from "lsof | grep tapestry":
java 26735 root mem REG 253,0 62415 2425040 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-portlet-4.1.1.jar
java 26735 root mem REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
java 26735 root mem REG 253,0 320546 2425036 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-contrib-4.1.1.jar
java 26735 root mem REG 253,0 49564 2424979 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-annotations-4.1.1.jar
java 26735 root 28r REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
java 26735 root 29r REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
java 26735 root 30r REG 253,0 2280602 2425039 /usr/local/apache-tomcat-5.5.20/my_webwork/WEB-INF/lib/tapestry-framework-4.1.1.jar
These unknown references are sometimes released automatically, but sometimes not.
And I get "Too many open files" exception after using my application for a few hours.
The number of the unknown references increases as I access to my webwork or just hit on "F5" key on my browser to reload it.
I tried different types of browsers to see if I could see any differences in consequence, and in fact it differed by the browser I used.
When viewed by Internet Explorer it increased by 3 for every access.
On the other hand it increased by 7 for each attempt when accessed by FireFox.
I have already tried optimizing the max number of file discriptors, and it solved the "Too many open files" exception.
But stil I'm wondering who actually is opening "tapestry-framework-4.1.1.jar" this many.
Could anyone figure out what is going on?
Thanks in advance.
The following is my environmental version info:
- Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
- Java: 1.5.0_11
- Tomcat: 5.5.20
- Tapestry: 4.1.1Hi,
Cause might The server got an exception while trying to accept client connections. It will try to backoff to aid recovery.
The OS limit for the number of open file descriptor (FD limit) needs to be increased. Tune OS parameters that might help the server to accept more client connections (e.g. TCP accept back log).
http://e-docs.bea.com/wls/docs90/messages/Server.html#BEA-002616
Regards,
Prasanna Yalam -
TOO many OPEN CURSORS during loop of INSERT's
Running ODP.NET beta2 (can't move up yet but will do that soon)
I don't think it is related with ODP itself but probably on how .Net works with cursors. We have a for/next loop that executes INSERT INTO xxx VALUES (:a,:b,:c)
statements. Apparently, when monitoring v$sysstat (current open cursors) we see these raising with 1 INSERT = 1 cursor. If subsequently we try to perform another action, we get max cursors exceeded. We allready set open_cursor = 1000, but the number of inserts can be very high. Is there a way to release these cursors (already wrote oDataAdaptor.dispose, oCmd.dispose but this does not help.
Is it normal that each INSERT has it's own cursor ? they all have the same hashvalue in v$open_cursor. They seem to be released after a while, especially when moving to another asp.net page, but it's not clear when that happens and if it is possible to force the release of the (implicit?) cursors faster.
Below is a snippet of the code, I unrolled a couple of function-calls into the code so this is just an example, not sure it will run without errors like this, but the idea should be clear (the code looks rather complex for what it does but the unrolled functions make the code more generic and we have a database-independend datalayer):
Try
' Set the Base Delete statement
lBaseSql = _
"INSERT INTO atable(col1,col2,col3) " & _
"VALUES(:col1,:col2,:col3)"
' Initialize a transaction
lTransaction = oConnection.BeginTransaction()
' Create the parameter collection, containing for each
' row in the list the arguments
For Each lDataRow In aList.Rows
lOracleParamters = New OracleParameterCollection()
lOracleParameter = New OracleParameter("luserid", OracleDbType.Varchar2,
_ CType(aCol1, Object))
lOracleParamters.Add(lOracleParameter)
lOracleParameter = New OracleParameter("part_no", OracleDbType.Varchar2, _
CType(lDataRow.Item("col2"), Object))
lOracleParamters.Add(lOracleParameter)
lOracleParameter = New OracleParameter("revision", OracleDbType.Int32, _
CType(lDataRow.Item("col3"), Object))
lOracleParamters.Add(lOracleParameter)
' Execute the Statement;
' If the execution fails because the row already exists,
' then the insert should be considered as succesfull.
Try
Dim aCommand As New OracleCommand()
Dim retval As Integer
'associate the aConnection with the aCommand
aCommand.Connection = oConnection
'set the aCommand text (stored procedure name or SQL statement)
aCommand.CommandText = lBaseSQL
'set the aCommand type
aCommand.CommandType = CommandType.Text
'attach the aCommand parameters if they are provided
If Not (lOracleParameters Is Nothing) Then
Dim lParameter As OracleParameter
For Each lParameter In lOracleParameters
'check for derived output value with no value assigned
If lParameter.Direction = ParameterDirection.InputOutput _
And lParameter.Value Is Nothing Then
lParameter.Value = Nothing
End If
aCommand.Parameters.Add(lParameter)
Next lParameter
End If
Return
' finally, execute the aCommand.
retval = cmd.ExecuteNonQuery()
' detach the OracleParameters from the aCommand object,
' so they can be used again
cmd.Parameters.Clear()
Catch ex As Exception
Dim lErrorMsg As String
lErrorMsg = ex.ToString
If Not lTransaction Is Nothing Then
lTransaction.Rollback()
End If
End Try
Next
lTransaction.Commit()
Catch ex As Exception
lTransaction.Rollback()
Throw New DLDataException(aConnection, ex)
End TryI have run into this problem as well. To my mind
Phillip's solution will work but seems completey unnecessary. This is work the provider itself should be managing.
I've done extensive testing with both ODP and OracleClient. Here is one of the scenarios: In a tight loop of 10,000 records, each of which is either going to be inserted or updated via a stored procedure call, the ODP provider throws the "too many cursor errors at around the 800th iteration. With over 300 cursors being open. The exact same code with OracleClient as the provider never throws an error and opens up 40+ cursors during execution.
The applicaation I have updates a Oracle8i database from a DB2 database. There are over 30 tables being updated in near real time. Reusing the command object is not an option and adding all the code Phillip did for each call seems highly unnecessary. I say Oracle needs to fix this problem. As much as I hate to say it the microsoft provider seems superior at this point. -
Runtime Error - DBIF_RSQL_INVALID_RSQL - Too many OPEN CURSOR
When I try to train a Decision Tree Model via an APD process in RSANWB, I get a runtime error when my model is configured with too many parameter fields or too many leaves (with 2 leaves it works, with more it fails).
By searching SAP Notes I see that there are many references to this kind of runtime errors. But no note on occurences of it in RSANWB / RSDMWB .
Any information on this anyone?
Claudio Ciardelli
Runtime Errors DBIF_RSQL_INVALID_RSQL
Date and Time 29.07.2005 16:19:21
|ShrtText |
| Error in RSQL module of database interface. |
|What happened? |
| Error in ABAP application program. |
| |
| The current ABAP program "SAPLRS_DME_DECISION_TREE_PRED" had to be terminated |
| because one of the |
| statements could not be executed. |
| |
| This is probably due to an error in the ABAP program. |
| |
|Error analysis |
| The system attempted to open a cursor for a SELECT or OPEN CURSOR |
| statement but all 16 cursors were already in use. |
| The statement that failed accesses table "/BIC/0CDT000030 ". |
| The erroneous statement accesses table "/BIC/0CDT000030 ". |
|Trigger Location of Runtime Error |
| Program SAPLRS_DME_DECISION_TREE_PRED |
| Include LRS_DME_DECISION_TREE_PREDU06 |
| Row 103 |
| Module type (FUNCTION) |
| Module Name RS_DME_DTP_EVALUATE |
|Source Code Extract |
|Line |SourceCde |
| 73|* Prepare for Data evaluation |
| 74| CATCH SYSTEM-EXCEPTIONS OTHERS = 15. |
| 75| CREATE DATA ref TYPE (i_enum_dbtab). |
| 76| ASSIGN ref->* TO <fs_wkarea>. |
| 77| ASSIGN COMPONENT gv_class_dbposit OF STRUCTURE |
| 78| <fs_wkarea> TO <fs_class>. |
| 79| CREATE DATA ref TYPE TABLE OF (i_enum_dbtab). |
| 80| ASSIGN ref->* TO <ft_data>. |
| 81| |
| 82| ENDCATCH. |
| 83| IF sy-subrc = 15. |
| 84|* Error on Assignment. |
| 85| CALL FUNCTION 'RS_DME_COM_ADDMSG_NOLOG' |
| 86| EXPORTING |
| 87| i_type = 'E' |
| 88| i_msgno = 301 |
| 89| i_msgv1 = 'EVALUATION_PHASE' |
| 90| IMPORTING |
| 91| es_return = ls_return. |
| 92| APPEND ls_return TO e_t_return. |
| 93| EXIT. |
| 94| ENDIF. |
| 95| |
| 96|* For the un-trained Rec-Ids, evaluate..... |
| 97| REFRESH lt_recinp. |
| 98| APPEND LINES OF i_t_records TO lt_recinp. |
| 99| SORT lt_recinp . |
| 100|* Open Cursor.. |
| 101| DATA: l_curs TYPE cursor. |
| 102| DATA: l_psize TYPE i VALUE 10000. |
|>>>>>| OPEN CURSOR WITH HOLD l_curs FOR |
| 104| SELECT * FROM (i_enum_dbtab) |
| 105| WHERE rsdmdt_recid NOT IN |
| 106| ( SELECT rsdmdt_recid FROM |
| 107| (i_learn_tab) ). |
| 108| |
| 109|* Start Fetch... |
| 110| DO. |
| 111| FETCH NEXT CURSOR l_curs |
| 112| INTO CORRESPONDING FIELDS OF TABLE <ft_data> |
| 113| PACKAGE SIZE l_psize. |
| 114| IF sy-subrc NE space. |
| 115| EXIT. |
| 116| ENDIF. |
| 117| |
| 118|* Process records... |
| 119| LOOP AT <ft_data> ASSIGNING <fs_wkarea>. |
| 120| |
| 121|* Call Prediction Function. |
| 122| CALL FUNCTION 'RS_DME_DTP_PREDICT_STRUCTURE' |Hi Claudio,
well the message is very clear and I think in your case you need to split your model into a few somehow equal models, each not having more than 2 leaves.
Another option might be to do more things serially instead of parallel.
Hope it helps
regards
Siggi -
Too many open socket connections causing ColdFusion to crash?
I’m currently working on an e-commerce site which sends and receives information to/from the client’s order management system via XML over a TCP/IP socket. It uses a very old java-based custom tag called CFX_JSOCKET (which appears to have been written in 2002) to open the socket, send the data, and get the response. The code that calls the custom tag and sends/receives data from the OMS pre-dates my working on the site, but its always worked, so I haven’t paid it much attention.
Back in the summer of 2009 we started experiencing issues with ColdFusion (v.7 on Window 2003 at the time) locking up on a more and more frequent basis, until it ultimately became a daily issue. After extensive research we narrowed the issue down to the communication between the web server and our client’s order management server. It seemed the issue with ColdFusion hanging was either related to there being too many connections open, or to these connections hanging and resulting in dead threads. This an educated guess based on a blog post I’d seen online, not actual monitoring of either CF or the TCP/IP connections. As soon as we dialed back the timeout on the CFX_JSOCKET tag from 20 seconds to 10, the issue disappeared, so we left it at that and moved on.
Fast forward to this January. The site is hosted at a new location, on a 64-bit Windows 2008 box running ColdFusion 9. Over the years traffic on the site has continued to grow. The nature of the clients business means that August and January are their business times of the year (back to school for college kids) and in January ColdFusion once again started locking up on an almost-daily basis.
One significant difference is that the address cleansing software that previously ran on the box and was used to verify shipping addresses is not available for 64-bit, so when we moved to the new server last summer, that task was moved to the client’s order management software and handled via XML like all other interaction with that system. However, while most XML calls to that server (order input, inventory check, etc) take under a second to complete, the address cleansing call regularly takes over 5 seconds to return data, and frequently times out.
Once we eliminated the address cleansing call from the checkout process, ColdFusion once again stopped locking up regularly. So it appears that once again it’s the communication between the web server and the order management server that’s causing problems. We currently have that address cleansing call disabled on the web site in order to keep ColdFusion from crashing, but that’s not a long term solution.
We don’t have, nor can I find online, the source code for the CFX_JSOCKET custom tag, so I decided I’d write some CF code utilizing the java methods to open the socket, send the data, get the response, and close the connection. My test code is working fine (under no load). However, in trying to troubleshoot an issue I had with it, I started monitoring the TCP/IP connections using TCPView. And I noticed that all the connections to the order management server, whether opened via the custom tag or my new code, remain open in either a TIME_WAIT or FIN_WAIT2 status for well over 2 minutes, even though I know for a fact that my new code is definitely closing the connection from the web server side.
They do all close eventually, but I’m wondering 1. Why they’re remaining open that long; 2. Is that normal; and 3. If all these connections remaining open could be what’s causing ColdFusion to choke.
Does this sound plausible? If so, does anyone have any suggestions/recommendations about how to fix it? My research seems to indicate this might be a matter of the order management system not closing the connection on its end, but I’m in way over my head, and before I go to client and tell them it’s their OMS causing the issue, I need to feel a little more confident that I’m on the right track.
Any help or advice would be very greatly appreciated. And thanks for taking the time to read through my long-winded explanation of the problem.
Set-up details:
ColdFusion Version: 9,0,0,251028 Standard
Operating System: Windows Server 2008
Java Version: 1.6.0_14
Java VM Name: Java HotSpot(TM) 64-Bit Server VM
Java VM Version: 14.0-b16
Thanks,
LaurieHi Laurie,
Not aware of custom tag called CFX_JSOCKET. I guess the process you described very well is consuming a resource then you are getting a problem. Trick is what parameter to adjust. Perhaps you are running out of one the threads in CFadmin > Server Settings > Request Tuning.
I expect if you enable CF Metrics logging where you can log the threads and other resources then you can find out which parameter needs adjusting. Let me know if you want some details on enabling CF Metrics. Perhaps others will have much better idea than me and help without the overhead of logging.
The other interesting thing is you are using CF9.0.0. Do you have some reasons for not being on updater1 CF9.0.1?
HTH, Carl.
PS I posted before however seems to have gone, just hope does not come back and then I have posted twice. -
Could someone help me understand this problem, and how to remedy it? We're getting warnings as the number of open cursors nears 1200. I've located the V$OPEN_CURSOR view, and after investigating it, this is what I think:
Currently:
SQL> select count(*)
2 from v$open_cursor;
COUNT(*)
535
1) I have one session open in the database, and 40 records in this view. Does that mean my cursors are still in the cursor cache?
2) Many of these cursors are associated with our analysts, and it looks like they are likely queries TOAD runs in order to gather meta-data for the interface. Can I overcome this?
3) I thought that the optimizer only opened a new cursor when a query that didn't match one in the cache was executed. When I run the following, I get 105 SQL statements with the same hash_value and sql_id, of which, they total 314 of the 535 open cursors (60% of the open cursors):
SQL> ed
Wrote file afiedt.buf
1 SELECT COUNT(*), SUM(cnt)
2 FROM (SELECT hash_value,
3 sql_id,
4 COUNT(*) as cnt
5 FROM v$open_cursor
6 GROUP BY hash_value, sql_id
7* HAVING COUNT(*) > 1)
SQL> /
COUNT(*) SUM(CNT)
104 314
4) Most of our connections in production will use Oracle Forms. Is there something we need to do in order to get Forms to use bind variables, or will it do so by default?
Thanks for helping me out with this.
-ChuckCURSOR_SHARING=EXACT
OPEN_CURSORS=500
CURSOR_SHARING
From what I've read, cursor sharing is always in effect, although we have the most conservative method set. So I'm not sure how this affects things. Several identical queries are being submitted in several separate cursors.
OPEN_CURSORS
This value corresponds with the maximum number of cursors allowed for a single session. We're using shared servers, so I'm exactly sure if this is still 'per session' or 'per shared server', but 500 should be more than enough.
It sounds like you're suggesting that a warning is being triggered based upon our init params. If that's the case, then what are people seeing as a limit for cursors on a 2-CPU Linux box with 2G of memory?
-Chuck -
ORA-01000: Too many open cursors -- Need Help
Hi All,
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
PL/SQL Release 11.1.0.7.0 - Production
I am getting error ora-01000 for the following procedures gather stats
Could you please guide how to get-rid-off this error.
thanks in advance;
CREATE OR REPLACE PROCEDURE SHEMA_NAME ANALYZE_TABLES IS
rec_table_name VARCHAR2 (30);
CURSOR c1
IS
SELECT table_name
FROM USER_tables; ------ 18000 table for this cursor
BEGIN
OPEN c1;
LOOP
FETCH c1 INTO rec_table_name;
EXIT WHEN c1%NOTFOUND;
-- block was hereÿÿÿ
BEGIN
DBMS_STATS.
GATHER_TABLE_STATS (
OWNNAME => 'SHEMA_NAME',
TABNAME => rec_table_name,
PARTNAME => NULL,
ESTIMATE_PERCENT => 30,
METHOD_OPT => 'FOR ALL COLUMNS SIZE AUTO',
DEGREE => 5,
CASCADE => TRUE);
END;
END LOOP;
CLOSE c1;
EXCEPTION
WHEN OTHERS
THEN
raise_application_error (
-20001,
'An error was encountered - ' || SQLCODE || ' -ERROR- ' || SQLERRM);
END;Look at the following:
SQL> begin
2 raise no_data_found;
3 end;
4 /
begin
ERROR at line 1:
ORA-01403: no data found
ORA-06512: at line 2
The error code the caller executing this code receive is -01403. A unique error number that has a known and specific meaning.
In addition, the error stack tells the caller that this unique error occurred on line 2 in the source code.
The caller knows EXACTLY what the error is and where it occurred.
SQL> begin
2 raise no_data_found;
3 exception when OTHERS then
4 raise_application_error(
5 -20000,
6 'oh damn some error happened. the error is '||SQLERRM
7 );
8 end;
9 /
begin
ERROR at line 1:
ORA-20000: oh damn some error happened. the error is ORA-01403: no data found
ORA-06512: at line 4
In this case the caller gets the error code -20000. It is meaningless as the same error code will be use for ALL errors (when OTHERS). So the caller will never know what the actual real error is.
For the caller to try and figure that out, it will need to parse and process the error message text to look for the real error code. A very silly thing to do.
In addition, the error stack says that the error was caused by line 4 in the code called.. except that this is the line that raised the meaningless generic error and not the actual line causing the error.
There are 3 basic reasons for writing an exception handler:
- the exception is not an error
- the exception is a system exception (e.g. no data found) and needs to be turned into meaningful application exceptions (e.g. invoice not found, customer not found, zip code not found, etc)
- the exception handler is used as a try..finally resource protection block (which means it re-raises the exception)
If your exception handler cannot tick one of these three reasons for existing, you need to ask yourself why you are writing that handler. -
TOO MANY OPEN CURSORS PROBLEM ... PLEASE HELP
Hi,
my problem is the following :
I got data from a system in flat file format. ( ascii, semicolon separated )
I wrote mapping classes to different tables and insert via Oracle thin driver.
The data I got isn't 100% consistent. It may happen that there are double
records for tables whith unique indexes.
I catched the Exception like in the segment below
Statement insertStmnt = null;
try{
insertStmnt = connection.createStatement();
insertStmnt.execute(insertString);
connection.commit(); // autocommit is diabled
} catch ( Exception sql ) {
System.out.println(sql.toString());
connection.rollback();
insertStmnt.close();
The Problem : when receiving the SQLException ( UNIQUE CONSTRAINT VIOLATED )
the cursor remains open.
After exeeding the open_cursors system property ( Oracle )
no more data is loaded.
( the input files contains sometimes more than one million rows )
Any suggestion to my Mail
[email protected]
ThanksHi!
Now you only close your statement when you catch an error. You will have to close it if things works out fine as well:
Statement insertStmnt = null;
try{
insertStmnt = connection.createStatement();
insertStmnt.execute(insertString);
connection.commit(); // autocommit is diabled
insertStmnt.close();
} catch ( Exception sql ) {
System.out.println(sql.toString());
connection.rollback();
insertStmnt.close();
Good luck!
/Tale -
Runtime.exec - Too Many Open Files
System version : Red Hat Enterprise Linux 2.4.21-47.ELsmp AS release 3 (Taroon Update 8)
JRE version : 1.6.0-b105
Important : the commands described below are launched from a Web application : Apache Tomcat 6.0.10
Hello,
I'm facing a problem already known, but appearantly never really solved ??!! ;)
When I invoke many system commands with the 'Runtime.exec(...)' method, there are open files that are not released (I can see them with the "lsof" system command) .
At the end, the unavoidable "too many open files" Exception.
The lauched commands are "ssh ... " commands.
In the topics relating to this problem, the solution is always to close all Streams / threads and to explicitely invoke the method "Process.destroy()".
My problem is that this is what I do ! And I can't do more...
Here is the code :
Runtime rt = Runtime.getRuntime();
Process process = rt.exec("ssh ...");
// ProcessStreamHolder extends Thread and reads from the InputStream given in constructor...
ProcessStreamHolder errorStream = new ProcessStreamHolder(process.getErrorStream());
ProcessStreamHolder outputStream = new ProcessStreamHolder(process.getInputStream());
errorStream.start();
outputStream.start();
exitValue = process.waitFor();
try {
errorStream.interrupt();
} catch (RuntimeException e) {
logger.warn("...");
try {
outputStream.interrupt();
} catch (RuntimeException e) {
logger.warn("...");
try {
process.getInputStream().close();
} catch (RuntimeException e) {
logger.warn("...");
try {
process.getOutputStream().close();
} catch (RuntimeException e) {
logger.warn("...");
try {
process.getErrorStream().close();
} catch (RuntimeException e) {
logger.warn("...");
process.destroy();Does someone know if my code is wrong or if there's a workaround for me ?
Thanks by advance !
Richard.Don't interrupt those threads. Close the output stream first, then wait for the process to exit, then both threads reading the stdout and stderr of the process should get EOFs, so they should exit naturally, and incidentally close the streams themselves.
-
Getting java.io.IOException: Too many open files+ClassNotFoundException
Dear All
We have a web application deployed on Rational Application Developer 6.0 (Operating System is Windows 2000 professional) our users are randomly getting java.io.IOException: Too many open files+ClassNotFoundException when they click on some servlet or jsp link, but they are getting this error randomly for example when they click some link they may get these exceptions but refreshing page or clicking once again on same link executes servlet successfully.If anyone could help on this topic we will be grateful
ThanksI think this these two exceptions are occuring in differrent environment
java.io.IOException is occuring under heavy load to web server its stack trace is as follows:
JSPG0225E: An error occurred at line: 2 in the statically included file: /SessionCheck.jsp
JSPG0093E: Generated servlet error from file: /Admin/AdminInsuranceCertificates.jsp
E:\WebSphere_6\AppServer\profiles\AUSECert\temp\centraNode04\server1\AUSECert\Vero.war\Admin\_AdminInsuranceCertificates.java:259: cannot access com.bplus.natmar.LoginDetails
bad class file: E:\WebSphere_6\AppServer\java\jre\lib\core.jar(java/io/Writer.class)
unable to access file: E:\WebSphere_6\AppServer\profiles\AUSECert\installedApps\centraNode04Cell\AUSECert.ear\Vero.war\WEB-INF\classes\com\bplus\natmar\LoginDetails.class (Too many open files)
Please remove or make sure it appears in the correct subdirectory of the classpath.
(source unavailable)
1 error
]: com.ibm.ws.jsp.JspCoreException: JSPG0049E: /Admin/AdminInsuranceCertificates.jsp failed to compile :
this error always occurs in reference to logindetails class this clas is used to make a session check on different roles in our project for e.g., user having end user role should not be able to log in as a user having admin role
we have included a sessioncheck.jsp in our every jsp page in this jsp we have simply used logindetails class as useBean and called its getresource() method
above stacktrace is from our live application server
while testing same project on our local system we are not getting too many open files exception but we are getting following ClassNotFoundException
[11/30/05 17:11:42:797 EST] 0000004a SystemOut O SELECT count(*) as NoofRecs FROM resourcerolebindings WHERE ResourceName = 'mainEdit.jsp' and IsEndUser=1
[11/30/05 17:12:50:891 EST] 000001eb SystemOut O SELECT count(*) as NoofRecs FROM resourcerolebindings WHERE ResourceName = 'InsuranceCertificates.jsp' and IsEndUser=1
[11/30/05 17:17:40:828 EST] 0000008d SystemOut O AppURL is: http://www.VeroECert.com/Vero/indexU.jsp
[11/30/05 17:17:58:141 EST] 0000008b SystemOut O SELECT count(*) as NoofRecs FROM resourcerolebindings WHERE ResourceName = 'InsuranceCertificates.jsp' and IsEndUser=1
[11/30/05 17:20:41:703 EST] 00000034 ServletWrappe E SRVE0026E: [Servlet Error]-[com.servlet.UserHelpServlet]: java.lang.ClassNotFoundException: com.servlet.UserHelpServlet
at com.ibm.ws.classloader.CompoundClassLoader.findClass(CompoundClassLoader.java(Compiled Code))
at com.ibm.ws.classloader.CompoundClassLoader.loadClass(CompoundClassLoader.java(Compiled Code))
at java.lang.ClassLoader.loadClass(ClassLoader.java(Compiled Code))
at java.beans.Beans.instantiate(Beans.java:202)
at java.beans.Beans.instantiate(Beans.java:63)
at com.ibm.ws.webcontainer.servlet.ServletWrapper$3.run(ServletWrapper.java:1384)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.ServletWrapper.loadServlet(ServletWrapper.java(Compiled Code))
at com.ibm.ws.webcontainer.servlet.ServletWrapper.initialize(ServletWrapper.java:1312)
at com.ibm.wsspi.webcontainer.extension.WebExtensionProcessor.createServletWrapper(WebExtensionProcessor.java:84)
at com.ibm.ws.webcontainer.extension.InvokerExtensionProcessor.handleRequest(InvokerExtensionProcessor.java:238)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:2841)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:220)
at com.ibm.ws.webcontainer.VirtualHost.handleRequest(VirtualHost.java:204)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java(Compiled Code))
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java(Compiled Code))
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java(Compiled Code))
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java(Compiled Code))
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java(Compiled Code))
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.requestComplete(WorkQueueManager.java(Compiled Code))
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.attemptIO(WorkQueueManager.java(Compiled Code))
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.workerRun(WorkQueueManager.java(Compiled Code))
at com.ibm.ws.tcp.channel.impl.WorkQueueManager$Worker.run(WorkQueueManager.java(Compiled Code))
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java(Compiled Code))
this error is occuring on concurrent clicks suppose three users click on same button then it is possible that one user gets correct output while other users experience this exception
Also one more point we are not using web.xml for calling servlets we are directly calling it giving its full path.
Thanks -
Too many open files in system cause database goes down
Hello experts I am very worry because of the following problems. I really hope you can help me.
some server features
OS: Suse Linux Enterprise 10
RAM: 32 GB
CPU: intel QUAD-CORE
DB: There is 3 instances RAC databases (version 11.1.0.7) in the same host.
Problem: The database instances begin to report Error message: Linux-x86_64 Error: 23: Too many open files in system
and here you are other error messages:
ORA-27505: IPC error destroying a port
ORA-27300: OS system dependent operation:close failed with status: 9
ORA-27301: OS failure message: Bad file descriptor
ORA-27302: failure occurred at: skgxpdelpt1
ORA-01115: IO error reading block from file 105 (block # 18845)
ORA-01110: data file 105: '+DATOS/dac/datafile/auditoria.519.738586803'
ORA-15081: failed to submit an I/O operation to a disk
At the same time I search into the /var/log/messages as root user and I the error notice me the same problem:
Feb 7 11:03:58 bls3-1-1 syslog-ng[3346]: Cannot open file /var/log/mail.err for
writing (Too many open files in system)
Feb 7 11:04:56 bls3-1-1 kernel: VFS: file-max limit 131072 reached
Feb 7 11:05:05 bls3-1-1 kernel: oracle[12766]: segfault at fffffffffffffff0 rip
0000000007c76323 rsp 00007fff466dc780 error 4
I think I get clear about the cause, maybe I need to increase the fs.file-max kernel parameter but I do not know how to set a good value. Here you are my sysctl.conf file and the limits.conf file:
sysctl.conf
kernel.shmall = 2097152
kernel.shmmax = 17179869184
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6553600
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 4194304
limits.conf
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536process limit
bcm@bcm-laptop:~$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited -
UTL_MAIL --- ORA-30678: too many open connections
Hello,
I have a pl/sql package that sends out emails using UTL_MAIL pkg pointing to an Exchange server, an APEX app calls this pkg.. This package used to work fine for months but I recently noticed that some emails are not being sent as expected. The package loops through a set of action items satisfying some conditions and send emails based on that ( this number is expected to grow every day ). I checked the errors log and I found this error:
ORA-30678: too many open connections
I think this means that I have to close the connection everytime I send en email, but UTL_MAIL does NOT have a function or a proc to close connections, right ?
I don't know what causes this error to happen, but I suspect that this started happening right after we re-pointed the UTL_MAIL pkg from a Lotus Notes server to an Exchange server.
I am also seeing this error:
ORA-29279: SMTP permanent error: 501 5.1.3 Invalid address
I know where this error comes from (usually a null email id in the FROM or TO field ), but can this be causing the first error to happen ?
Please advise if you got this error before, is it a bug in oracle 10g as I read in some blog ? or is the second error happening make the Exchange server refuse SMTP connections ???
Thanks,
SamHi Sam,
seems to be a bug in UTL_MAIL if you ask me, as you are right - there is only /send/, no option to close, so I'd expect this to be done automatically.
Anyway, though UTL_MAIL is usable for basic mailing, I prefer using a custom mail implementation based on UTL_SMTP. The most important reason is that most mail servers don't work without authentication. And if you have done this once, you can reuse the function/procedure/package as simple as UTL_MAIL. The good news is, that there are several examples published that provide you with the functionality you have in UTL_MAIL at once - with the difference, that you definetly get your connection closed when you expect it to be closed.
You'll also be able to handle empty addresses. Perhaps this error actually causes UTL_MAIL to "forget" to close the connection, if this exception isn't caught before in order to close an open connection before raising it to the outside.
One example implementation for using UTL_SMTP can be found [url http://www.morganslibrary.com/reference/pkgs/utl_smtp.html]here
-Udo -
WLS 92MP1: Application Poller issue Too many open files
Hi,
We have a wls92mp1 domain on linux AS4(64bit) with Sun jdk 1.5.0_14. It contains only Admin server where we have deployed the application. Over a period of time the server start showing up below message in the logs. We have not deployed the application from autodeploy directory. And the file "/home/userid/wls92/etg/servers/userid_a/cache/.app_poller_lastrun " is available in the location, still it throws FileNotFoundException.
<Error> <Application Poller> <BEA-149411> <I/O exception encountered java.io.FileNotFoundException: /home/userid/wls92/etg/servers/userid_a/cache/.a
pp_poller_lastrun (Too many open files).
java.io.FileNotFoundException: /home/userid/wls92/etg/servers/userid_a/cache/.app_poller_lastrun (Too many open files)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
at java.io.FileWriter.<init>(FileWriter.java:73)
at weblogic.management.deploy.GenericAppPoller.setLastRunFileMap(GenericAppPoller.java:423)
Any help regarding this would be highly appreciated.
Thanks.Hi,
By above seeing in error, this error code (BEA-149411) describe the following
149411: I/O exception encountered {0}.
L10n Package: weblogic.management.deploy.internal
I18n Package: weblogic.management.deploy.internal
Subsystem: Application Poller
Severity: Error
Stack Trace: true
Message Detail: An I/O exception denotes a failure to perform a read/write operation on the application files.
Cause: An I/O exception can occur during file read/write while deploying an application.
Action: Take corrective action based on the exception message details.
i think it helps u.
-abhi -
ORA-29270: too many open HTTP requests
Hi,
We have a requirement like, we will get bulk data (2 GB) data file with different languages(Russian, chinese, Japanese, German) etc. and we need to load these data into our databse also we have to translate the same.
We are using Oracle 11g.
We had used Oracle httpuritype - And the below mentioned procedure PROCEDURE TEST.PRC_TRANS_GERMAN will call the function - trans_jef
Input parameter for PRC_TRANS_GERMAN - > To_language and from Language. and this procedure will store the translated word in the seperate table.
Input parameter for the function trans_jef - > Word to be translated , from languae, to_language. And this function will return the translated word.
The issue I am facing is when I am passing the table of around 10000 records, around 4000 rows are not getting translated. So when i try to capture the error in exception- the error is
ORA-29273: HTTP request failed
ORA-06512: at "SYS.UTL_HTTP", line 1819
ORA-29270: too many open HTTP requests
When I try to re execute the missed words, they were translated. But they are not getting translated in one go because of the too many http open request.
So could you please let me know how to resolve this ORA-29270 errror. I know we have to end the request and start again. But not sure how to do. Can some one help with this~!!!!
Translator
CREATE OR REPLACE FUNCTION trans_jef (v_RawText IN VARCHAR2,
v_SrcLang IN VARCHAR2,
v_DestLang IN VARCHAR2)
RETURN CLOB
IS
v_EscRawText VARCHAR2 (3000);
v_XlateUrl VARCHAR (3000);
--v_SrcLang Varchar(10) ;-- := 'de';
--v_DestLang Varchar(10) ; -- := 'en';
v_XlateResult CLOB;
l_result CLOB;
BEGIN
DBMS_OUTPUT.ENABLE (10000000);
v_EscRawText := UTL_URL.escape (v_RawText, TRUE, 'UTF-8');
v_XlateUrl :=
'http://translate.google.com/translate_a/t?client=t&text='
|| v_EscRawText
|| '&hl='
|| v_DestLang
|| '&sl='
|| v_SrcLang
|| '&tl='
|| v_DestLang
|| '&ie=UTF-8&oe=UTF-8&multires=1&oc=1&prev=btn&ssel=0&tsel=0&sc=1';
v_XlateResult := httpuritype (v_XlateUrl).getclob ();
-- dbms_output.put_line (v_XlateResult);
l_result := TRIM (BOTH '"' FROM REGEXP_SUBSTR (v_XlateResult, '"[^"]+"'));
RETURN l_result;
END;
/* Formatted on 07-09-2012 09:21:05 (QP5 v5.215.12089.38647) */
CREATE OR REPLACE PROCEDURE TEST.PRC_TRANS_GERMAN (TO_LANG1 IN VARCHAR2,
FROM_LANG1 IN VARCHAR2)
IS
P_CHINESE VARCHAR2 (3000);
FROM_LANG VARCHAR2 (3000);
FROM_LANG_CODE VARCHAR2 (3000);
TO_LANG_CODE VARCHAR2 (3000);
FROM_LANG_CODE1 VARCHAR2 (3000);
TO_LANG_CODE1 VARCHAR2 (3000);
cnt NUMBER := 1;
counter NUMBER;
CURSOR C1
IS
SELECT RAW_VAL_TXT FROM T_TRANS_VAL;
VC1 C1%ROWTYPE;
BEGIN
OPEN C1;
LOOP
FETCH C1 INTO VC1;
EXIT WHEN C1%NOTFOUND;
BEGIN
TO_LANG_CODE :=
CASE TO_LANG1
WHEN 'English' THEN 'en'
WHEN 'Chinese Simplified' THEN 'zh-CN'
WHEN 'Chinese Tradational' THEN 'zh-TW'
WHEN 'German' THEN 'de'
WHEN 'Japanese' THEN 'ja'
--other languages
END;
FROM_LANG_CODE :=
CASE FROM_LANG1
WHEN 'English' THEN 'en'
WHEN 'Chinese Simplified' THEN 'zh-CN'
WHEN 'Chinese Tradational' THEN 'zh-TW'
WHEN 'German' THEN 'de'
WHEN 'Japanese' THEN 'ja'
other languages
-- else 'auto'
END;
SELECT trans_jef (VC1.RAW_VAL_TXT, FROM_LANG_CODE, TO_LANG_CODE)
INTO P_CHINESE
FROM DUAL;
-- same translator function is used. Language used is simplied chinese.
-- We can use traditional Chinese as well ('zh-TW'), which works perfectly.
FROM_LANG_CODE1 := --decode(from_lang,'en','English','ar','Arabic', from_lang);
CASE FROM_LANG_CODE
WHEN 'en' THEN 'English'
WHEN 'zh-CN' THEN 'Chinese Simplified'
WHEN 'zh-TW' THEN 'Chinese Tradational'
WHEN 'de' THEN 'German'
WHEN 'ja' THEN 'Japanese'
--other languages
END;
TO_LANG_CODE1 := --decode(from_lang,'en','English','ar','Arabic', from_lang);
CASE TO_LANG_CODE
WHEN 'en' THEN 'English'
WHEN 'zh-CN' THEN 'Chinese Simplified'
WHEN 'zh-TW' THEN 'Chinese Tradational'
WHEN 'de' THEN 'German'
WHEN 'ja' THEN 'Japanese'
-- other languages
END;
INSERT INTO t_trans
VALUES (FROM_LANG_CODE1,
VC1.RAW_VAL_TXT,
TO_LANG_CODE1,
P_CHINESE);
COMMIT;
EXCEPTION
WHEN OTHERS
THEN
INSERT INTO excep_trans (P_WORD_EXP, UPDATE_TIME, COUNTER)
VALUES (VC1.RAW_VAL_TXT, SYSDATE, cnt);
INSERT INTO error_message
VALUES (DBMS_UTILITY.FORMAT_ERROR_STACK);
COMMIT;
END;
END LOOP;
CLOSE C1;
END;
/Edited by: 950814 on 6 Sep, 2012 11:16 PMSimply add the call to your exception hander. E.g.
EXCEPTION
WHEN OTHERS
THEN
UTL_TCP.Close_All_Connections;
INSERT INTO excep_trans (P_WORD_EXP, UPDATE_TIME, COUNTER)
VALUES (VC1.RAW_VAL_TXT, SYSDATE, cnt);
..etc..
END;Also consider using proper coding standards. There is NO modern day programming language standard that says to write (selective) reserved words in uppercase. Write in lowercase. Use camelcase and pascalcase for variable names and procedures/functions/methods. Restrict the use of uppercase to defining identifiers such as compiler macros or constants.
Do yourself a favour and read, understand and apply the basic naming standards as detailed in .Net Guidelines for Names and Code Conventions for the Java Programming Language.
Maybe you are looking for
-
why does pointer become a pinwheel when I open one particular folder only and prevent me from opening the apps in it and then keep me from closing the folder unless I shut down the computer completely. I can open all other folders on the desk top wi
-
Can't open spreadsheet 3.2 in numbers '09
Made spreadsheets on my Macbook Pro with Numbers 3.2 (Mavericks) which is in reparation at the moment. Trying to open these spreadsheets at our IMac with Numbers '09 but: you need a newer version of Numbers to open this spreadsheet etc... Don't want
-
Simply Copying All Fields/Info On A Contact
Is there a way to simply copy all contents of a contact from the address book so that one can simply paste it into another document? I do not seek to import, export, merge, or anything else but simply COPY AND PASTE. For some reason I cannot seem to
-
When I try and open a jpg or png file with Camera Raw, I keep getting and error message saying that a bridge parent application has not been opened. I am then told to open a qualifying app to continue. Does anybody have any ideas what this means. Th
-
Hello all, I was just wondering about the efficiency of some programming practices. I have a program I am working on with about 20 different files. One of the files contains all of my constants ("Constants.java")... I am wondering if it is a bad prac