Out of memory error in migrating from mysql to oracle

Hi,
I'm trying migrate from mysql to oracle9i. The hardware and software requirements are matching or even much higher than mentioned in the documentation. I have installed OMWB is properly(hope so). When I try to migrate the desired database from mysql to oracle, the repository, the Oracle model are created successfully. But while migrating data, say for around 35000 rows insertion everything works pretty fine. But after sometime, the migration stops and no error or anything appears in the screen.And in the Errorlog I get "OutOfMemoryError".
I tried to migrate single table also. But I still get this problem after sometime. The tables may contain from 10 - 130000 rows...
I even tried to change the %JRE..% memory value to 64M or 128M in the omwb.bat file and the O/s is Microsoft Windows XP, RAM 256MB.
Any help is much appreciated.
Thanks in advance.

Hi Viji,
did the message in the error.log provide any extra information? For example, was the OutOfMemory error prefixed with a string
like "cannot create new native thread" ? My suggestion to workaround the problem would have been to increase the JVM heap size to 128
but this appears to have been unsuccessful for you. Can you send a mail to the Migration Workbench support services
([email protected]) explaining the problem, and attach the full error.log file please ? (please also copy me on the mail).
In the meantime, you can workaround this by migrating the schema only (there is a switch on step 3 of the migration wizard asking:
"Do you want to migrate the table data to Oracle?" - just select the "No" option). When the schema has migrated, you can then use the
offline data loading facility to migrate the data. This uses data extraction scripts and SDQL*Loader to migrate the data to the target
Oracle database.You can learn more about the offline data loading facility from the plugins referenceguide (from the Help menu in
the Migration Workbench).
I hope this helps,
Tom.

Similar Messages

Out of Memory Error when writing from mySQL to JPanel

Hi,
I have a JPanel with BoxLayout to which I am adding JCheckBoxes, which are the result of an Select query to mySQL.
My select query is giving me 15000 lines of result, in which case my program is screwed up, and I am getting OutofMemory Error.
Does somebody know how to fix this problem?
This is my code for the JPanel :
srchRP = new JPanel();
     BoxLayout bl = new BoxLayout(srchRP, BoxLayout.Y_AXIS);
     srchRP.setLayout(bl);
     srchSP = new JScrollPane(srchRP);
     srchSP.setPreferredSize(new Dimension(300,470));

If I understand correctly, your JVM simply cannot deal with 15,000 objects (the result set generated by your query). You can either make more memory available to the JVM as described above, or you can consider redesigning your application.
It seems what you are trying to do is get every record in the database table, and display those records in a JPanel, allowing the end user to select which records he wants to perform an action on (such as delete).
Instead of using a "select * from table" query, you could do a "select id, title from table" type of query, in which case, your data object will be a lot smaller and still give the user enough information to decide whether to delete the record or not.
Alternatively, you could write a utility program to get the information you need from a text file generated as a result of a database dump and load that information upon startup, i.e., cache the master list of records.
But, it seems to me, if I were your customer, I would not want to scroll through a list of 15,000 entries and deciding which ones I wanted to check off. If you figure out an easier way to manage 15,000+ records, such as allowing the user to narrow down his criteria, then you go a long way towards optimizing your code to reduce memory requirements.

DB Migration from MYSQL to ORACLE Using Offline Capture

Hi
Am doing a database migration from MySQL to Oracle using SQL Developer (version 2.1.1.64). So far, I've successfully captured the MySQL database and converted it to the Oracle Model. However, when generating offline scripts to create the converted model schema into Oracle DDL scripts it managed to generate SQL to create: 1) User 2) Sequences 3) Tables 4) Triggers and 5) constraints.
It has created the SQL to add the primary key constraints and index constraints. Although it did the foreign key constraints in the SQL, the foreign key constraints seems to have missed the cascading options for the foreign key constraint. I.e. theres no reference of whether the foreign key constraint will restrict on delete or cascade etc.
We have a foreign keys in the MySql database that have different cascading options and these have not being ported over into the migration SQL. Therefore, all the foreign keys generated in the SQL by default are cascade to restrict on delete.
Does 'Generate Oracle DDL' not take into account a foreign key's on delete cascading option?
Any help or information would be greatly appreciated.
Thanks

Hello,
that reminded me for the following thread:
Migration Microsoft SQL Sever 2005 to Oracle 11g cascade on delete problem
That is a similar issue, isn't it?
I opened a bug for that, and it will be fixed in SQL Developer 3.1 (not in any 3.0 Early Adopter version). If you hit the same issue, there is no other way then using the workaround as used in the mentioned thread.
Regards
Wolfgang

Migrating from MySQL to Oracle 11g

can u tell me the steps to migrating database from MySQL to Oracle using SQL Developer?
I have installed Oracle 11g(11.1.0.6.0) in server and SQl Developer in my local system.
I connected to MySQL and Oracle in SQL Developer and tried to MIgrate everything in MySQL to Oracle using Quick Migrate option.
it took one and half day but didn't work at last...
is there any pre tasks that i need to do before migrating to oracle from MySQL?
and also when i export the MySQL data to Oracle using SQL Developer,it is giving data type mismatch errors like (date, boolean etc...).
please give a reply what i need to do and any refferences also very much appreciated.

hi Turloch ,
I followed the steps mentioned above.
When I right click my MySQL server db Connection and click "Capture MySQL server", it just shows a dialog with "Close" enabled.
after clicking the close button, it gives following error messages.
oracle.dbtools.metadata.persistence.PersistableObject.doInsert(PersistableObject.java:238)
the same error is coming no of times.
Kindly help me in this regard.

How to handle DATE type problems in migrating from mysql to oracle.?

Hi,
I'm migrating only the data from mysql to oracle with the help of sql loader.But with this type i cannot able to insert the date values from mysql to oracle.In mysql i have defined date as "DATETIME" type and in oracle it is in TimeStamp.Whenever i'm inserting the values thru CTL file ,i'm getting an error invalid date format entered.How to solve this problem?
Thanks in Advance
JAI

you need to supply a mask to the timestamp entry. see http://download-west.oracle.com/docs/cd/B19306_01/server.102/b14215/ldr_field_list.htm#i1006714 for details on datatypes within the ctl file
B

Migrating from MySql to Oracle

Hi,
I have installed latest version of SQL Developer with new Extensions (added mysql-connector-java-3.1.14-bin.jar under third-party database extension tab) , also installed latest updates from Help Menu. I have created a database in Mysql and now migrating it to Oracle, while I click on Capture MySQL / Quick MySql migrate... i get error "Error ocurred during capture: ORA-01400: cannot insert NULL into ("ASHISH"."MD_PROJECTS"."ID") ".
Thanks,
Ashish Sohane.
OracleDirect.

I am having the same problem. I have deleted and recreated the repository many times. I'm getting:
"SQL Error on Script Execution. Try deleting repository before Creating Repository."
then
"The Repository installtaion has failed. Please check the user you are registering and make sure that objects of the same name do not already exist"
I have closed the connection and reopened, closed SQL Developer and reopened. When I expand Tables, there are NO md_ tables. So it seems the repository deletion is working. But when I try to recreate it, oracle somehow thinks the repository I deleted is still there.... even tho it doesn't show it in the ui.
Although this thread began w/ someone converting from MySQL i think that is probably irrelevant. We are getting the same errors with the oracle repository. I am converting from MSAccess2003 using SqlDeveloper 1.2.1. I have successfully created the XML and SQL exports from MSAccess using SqlDeveloper\Migration\MicrosoftAccess Exporter\2003

"Default " problem while migrating from mysql to oracle.

I'm migrating from mysql3.23 to oracle9i via the OMWB tool.While migrating i'm getting an error message "Failed to create default for the table <table>:ora00907 missing right parenthesis".This error is due to the difference in the syntax used for the keyword "default" in mysql and oracle.In mysql we have to give after the Not Null option whereas in oracle we have to give before Not Null.
The problem is i'm migrating via the OWMB tool.Is is possible to handle this error with the help of tool itself.
Thanks
Jai.M

Jai, I'm having Frank have a look at this for you to see if we can replicate it ans help solve this issue for you. are there any issues logged on the mysql site regarding saving blobs as file?
thanks
Barry

Migrating from MySQL to Oracle 11gR2 containing accented characters

Hello Gurus,
I am going to migrate a MySQL database that contains accented characters. The collation of the tables holding these data is utf8_unicode_ci.
I used SQLDeveloper to generate data migration scripts. After a test migration run, the web pages displays "?" instead of the accented character.
I am currently tracing where the problem is.
From what I have read, mysqldump uses utf8 as the default character set, so I assume the accented characters in <tablename>.txt files are in tact.
Could the problem be on sqlldr? Has anyone come across this problem?
I really would appreciate any help.
Thanks

Hi Srini, thanks for the reply.
here is the result of select * from nls_database_parameters;
PARAMETER VALUE
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
NLS_CHARACTERSET AL32UTF8
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-MON-RR
NLS_DATE_LANGUAGE AMERICAN
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CONV_EXCP FALSE
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_RDBMS_VERSION 11.2.0.1.0

Migration from MySQL - connection problem

I've just installed OMWB 1.3.1 for WINDOWS and created repository in ORACLE 8.0.5 DB.
When I enter Source Database Details in Capture Wizard and push Next button nothing is happened. In OMWB log-file I've found the following message among many others:
Exception occurred during event dispatching:
java.lang.NoClassDefFoundError: org/gjt/mm/mysql/Driver
at java.util.Vector.<init>(Vector.java:128)
The directory C:\Oracle\oracle81\Omwb\drivers\
is empty while in CLASSPATH env variable I've found the following item:
C:\Oracle\oracle81\Omwb\drivers\mm.mysql.jdbc-1.2a
If anybody could help me to solve this problem, please let me know.
Thank you in advance.
null

Hi Vladimir,
I send you a message from the Migration Workbench team that solved my problem. I hope this will resolve yours too.
Bye,
Pierfrancesco
Hi Pierfrancesco
MySQL 3.22, 3.23 Requirements
In order to migrate from MySQL to Oracle, you must install the MM MySQL JDBC
driver release 1.2a on the same system as the Migration Workbench. You can
obtain this by downloading the MM MySQL JDBC driver from the WorldServer Web
site at:
http://www.worldserver.com/mm.mysql
Copy and unzip the mm.mysql.jdbc-1.2a.zip file to your
%ORACLE_HOME%\Omwb\drivers directory. This automatically creates a new
directory called
mm.mysql.jdbc-1.2a within the drivers directory.
Can you please verify that you have this driver extracted to the above
specified directory. Another check you can do is see whether all the files in
the zip
have extracted.
Please refer to the Release notes which you have access to by selection the Help
drop down menu in the Workbench.
I hope this helps
Best regards
Oracle Migration Workbench Support
null

Out of memory error in adobe migration tool

Hello,
While migrating data from version cue server to DAM server adobe migration tool is showing out of memory error.
Is there any way to increase memory size for migration tool.
Thanks,

Hi,
I had this issue as well. Does anyone know why this happens

Uploading large files from applet to servlet throws out of memory error

I have a java applet that needs to upload files from a client machine
to a web server using a servlet. the problem i am having is that in
the current scheme, files larger than 17-20MB throw an out of memory
error. is there any way we can get around this problem? i will post
the client and server side code for reference.
Client Side Code:
import java.io.*;
import java.net.*;
// this class is a client that enables transfer of files from client
// to server. This client connects to a servlet running on the server
// and transmits the file.
public class fileTransferClient
private static final String FILENAME_HEADER = "fileName";
private static final String FILELASTMOD_HEADER = "fileLastMod";
// this method transfers the prescribed file to the server.
// if the destination directory is "", it transfers the file to
"d:\\".
//11-21-02 Changes : This method now has a new parameter that
references the item
//that is being transferred in the import list.
public static String transferFile(String srcFileName, String
destFileName,
String destDir, int itemID)
if (destDir.equals(""))
destDir = "E:\\FTP\\incoming\\";
// get the fully qualified filename and the mere filename.
String fqfn = srcFileName;
String fname =
fqfn.substring(fqfn.lastIndexOf(File.separator)+1);
try
//importTable importer = jbInit.getImportTable();
// create the file to be uploaded and a connection to
servlet.
File fileToUpload = new File(fqfn);
long fileSize = fileToUpload.length();
// get last mod of this file.
// The last mod is sent to the servlet as a header.
long lastMod = fileToUpload.lastModified();
String strLastMod = String.valueOf(lastMod);
URL serverURL = new URL(webadminApplet.strServletURL);
URLConnection serverCon = serverURL.openConnection();
// a bunch of connection setup related things.
serverCon.setDoInput(true);
serverCon.setDoOutput(true);
// Don't use a cached version of URL connection.
serverCon.setUseCaches (false);
serverCon.setDefaultUseCaches (false);
// set headers and their values.
serverCon.setRequestProperty("Content-Type",
"application/octet-stream");
serverCon.setRequestProperty("Content-Length",
Long.toString(fileToUpload.length()));
serverCon.setRequestProperty(FILENAME_HEADER, destDir +
destFileName);
serverCon.setRequestProperty(FILELASTMOD_HEADER, strLastMod);
if (webadminApplet.DEBUG) System.out.println("Connection with
FTP server established");
// create file stream and write stream to write file data.
FileInputStream fis = new FileInputStream(fileToUpload);
OutputStream os = serverCon.getOutputStream();
try
// transfer the file in 4K chunks.
byte[] buffer = new byte[4096];
long byteCnt = 0;
//long percent = 0;
int newPercent = 0;
int oldPercent = 0;
while (true)
int bytes = fis.read(buffer);
byteCnt += bytes;
//11-21-02 :
//If itemID is greater than -1 this is an import file
transfer
//otherwise this is a header graphic file transfer.
if (itemID > -1)
newPercent = (int) ((double) byteCnt/ (double)
fileSize * 100.0);
int diff = newPercent - oldPercent;
if (newPercent == 0 || diff >= 20)
oldPercent = newPercent;
jbInit.getImportTable().displayFileTransferStatus
(itemID,
newPercent);
if (bytes < 0) break;
os.write(buffer, 0, bytes);
os.flush();
if (webadminApplet.DEBUG) System.out.println("No of bytes
sent: " + byteCnt);
finally
// close related streams.
os.close();
fis.close();
if (webadminApplet.DEBUG) System.out.println("File
Transmission complete");
// find out what the servlet has got to say in response.
BufferedReader reader = new BufferedReader(
new
InputStreamReader(serverCon.getInputStream()));
try
String line;
while ((line = reader.readLine()) != null)
if (webadminApplet.DEBUG) System.out.println(line);
finally
// close the reader stream from servlet.
reader.close();
} // end of the big try block.
catch (Exception e)
System.out.println("Exception during file transfer:\n" + e);
e.printStackTrace();
return("FTP failed. See Java Console for Errors.");
} // end of catch block.
return("File: " + fname + " successfully transferred.");
} // end of method transferFile().
} // end of class fileTransferClient
Server side code:
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
import java.util.*;
import java.net.*;
// This servlet class acts as an FTP server to enable transfer of
files
// from client side.
public class FtpServerServlet extends HttpServlet
String ftpDir = "D:\\pub\\FTP\\";
private static final String FILENAME_HEADER = "fileName";
private static final String FILELASTMOD_HEADER = "fileLastMod";
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException,
IOException
doPost(req, resp);
public void doPost(HttpServletRequest req, HttpServletResponse
resp)
throws ServletException,
IOException
// ### for now enable overwrite by default.
boolean overwrite = true;
// get the fileName for this transmission.
String fileName = req.getHeader(FILENAME_HEADER);
// also get the last mod of this file.
String strLastMod = req.getHeader(FILELASTMOD_HEADER);
String message = "Filename: " + fileName + " saved
successfully.";
int status = HttpServletResponse.SC_OK;
System.out.println("fileName from client: " + fileName);
// if filename is not specified, complain.
if (fileName == null)
message = "Filename not specified";
status = HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
else
// open the file stream for the file about to be transferred.
File uploadedFile = new File(fileName);
// check if file already exists - and overwrite if necessary.
if (uploadedFile.exists())
if (overwrite)
// delete the file.
uploadedFile.delete();
// ensure the directory is writable - and a new file may be
created.
if (!uploadedFile.createNewFile())
message = "Unable to create file on server. FTP failed.";
status = HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
else
// get the necessary streams for file creation.
FileOutputStream fos = new FileOutputStream(uploadedFile);
InputStream is = req.getInputStream();
try
// create a buffer. 4K!
byte[] buffer = new byte[4096];
// read from input stream and write to file stream.
int byteCnt = 0;
while (true)
int bytes = is.read(buffer);
if (bytes < 0) break;
byteCnt += bytes;
// System.out.println(buffer);
fos.write(buffer, 0, bytes);
// flush the stream.
fos.flush();
} // end of try block.
finally
is.close();
fos.close();
// set last mod date for this file.
uploadedFile.setLastModified((new
Long(strLastMod)).longValue());
} // end of finally block.
} // end - the new file may be created on server.
} // end - we have a valid filename.
// set response headers.
resp.setContentType("text/plain");
resp.setStatus(status);
if (status != HttpServletResponse.SC_OK)
getServletContext().log("ERROR: " + message);
// get output stream.
PrintWriter out = resp.getWriter();
out.println(message);
} // end of doPost().
} // end of class FtpServerServlet

OK - the problem you describe is definitely what's giving you grief.
The workaround is to use a socket connection and send your own request headers, with the content length filled in. You may have to multi-part mime encode the stream on its way out as well (I'm not about that...).
You can use the following:
http://porsche.cis.udel.edu:8080/cis479/lectures/slides-04/slide-02.html
on your server to get a feel for the format that the request headers need to take.
- Kevin
I get the out of Memory Error on the client side. I
was told that this might be a bug in the URLConnection
class implementation that basically it wont know the
content length until all the data has been written to
the output stream, so it uses an in memory buffer to
store the data which basically causes memory issues..
do you think there might be a workaround of any kind..
or maybe a way that the buffer might be flushed after
a certain size of file has been uploaded.. ?? do you
have any ideas?

Out of memory error - from parsing a "fixed width file"

This may be fairly simple for someone out there but I am trying to write a simple program that can go through a "fixed width" flat txt file and parse it to be comma dilmeted.
I use a xml file with data dictionary specifications to do the work. I do this because there are over 430 fields that need to be parsed from a fixed width with close to 250,000 lines I can read the xml file fine to get the width dimensions but when I try to apply the parsing instructions, I get an out of memory error.
I am hoping it is an error with code and not the large files. If it is the latter, does anyone out there know some techniques for getting at this data?
Here is the code
   import java.io.*;
   import org.w3c.dom.Document;
   import org.w3c.dom.*;
   import javax.xml.parsers.DocumentBuilderFactory;
   import javax.xml.parsers.DocumentBuilder;
   import org.xml.sax.SAXException;
   import org.xml.sax.SAXParseException;
    public class FixedWidthConverter{
      String[] fieldNameArray;
      String[] fieldTypeArray;
      String[] fieldSizeArray;
       public static void main(String args []){
         FixedWidthConverter fwc = new FixedWidthConverter();
         fwc.go();
         fwc.loadFixedWidthFile();
        //System.exit (0);
      }//end of main
       public void go(){
         try {
            DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance();
            DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
            Document doc = docBuilder.parse (new File("files/dic.xml"));
            // normalize text representation            doc.getDocumentElement ().normalize ();
            System.out.println ("Root element of the doc is " +
                 doc.getDocumentElement().getNodeName());
            NodeList listOfFields = doc.getElementsByTagName("FIELD");
            int totalFields = listOfFields.getLength();
            System.out.println("Total no of fields : " + totalFields);
            String[] fldNameArray = new String[totalFields];
            String[] fldTypeArray = new String[totalFields];
            String[] fldSizeArray = new String[totalFields];
            for(int s=0; s<listOfFields.getLength() ; s++){
               Node firstFieldNode = listOfFields.item(s);
               if(firstFieldNode.getNodeType() == Node.ELEMENT_NODE){
                  Element firstFieldElement = (Element)firstFieldNode;
                  NodeList firstFieldNMList = firstFieldElement.getElementsByTagName("FIELD_NM");
                  Element firstFieldNMElement = (Element)firstFieldNMList.item(0);
                  NodeList textFNList = firstFieldNMElement.getChildNodes();
                  //System.out.println("Field Name : " +
                           //((Node)textFNList.item(0)).getNodeValue().trim());
                  //loads values into an array
                  //fldNameArray[s] = ((Node)textFNList.item(0)).getNodeValue().trim();
                  NodeList typeList = firstFieldElement.getElementsByTagName("TYPE");
                  Element typeElement = (Element)typeList.item(0);
                  NodeList textTypList = typeElement.getChildNodes();
                  //System.out.println("Field Type : " +
                           //((Node)textTypList.item(0)).getNodeValue().trim());
                  //loads values into an array
                  //fldTypeArray[s] = ((Node)textTypList.item(0)).getNodeValue().trim();
                  NodeList sizeList = firstFieldElement.getElementsByTagName("SIZE");
                  Element sizeElement = (Element)sizeList.item(0);
                  NodeList textSizeList = sizeElement.getChildNodes();
                  //System.out.println("Field Size : " +
                           //((Node)textSizeList.item(0)).getNodeValue().trim());
                  //loads values into an array
                  fldSizeArray[s] = ((Node)textSizeList.item(0)).getNodeValue().trim();
               }//end of if clause
            }//end of for loop with s var
            //setFldNameArray(fldNameArray);
            //setFldTypeArray(fldTypeArray);
            setFldSizeArray(fldSizeArray);
             catch (SAXParseException err) {
               System.out.println ("** Parsing error" + ", line "
                  + err.getLineNumber () + ", uri " + err.getSystemId ());
               System.out.println(" " + err.getMessage ());
             catch (SAXException e) {
               Exception x = e.getException ();
               ((x == null) ? e : x).printStackTrace ();
             catch (Throwable t) {
               t.printStackTrace ();
      }//end go();
       public void setFldNameArray(String[] s){
         fieldNameArray = s;
      }//end setFldNameArray
       public void setFldTypeArray(String[] s){
         fieldTypeArray = s;
      }//end setFldTypeArray
       public void setFldSizeArray(String[] s){
         fieldSizeArray = s;
      }//end setFldSizeArray
       public String[] getFldNameArray(){
         return fieldNameArray;
      }//end setFldNameArray
       public String[] getFldTypeArray(){
         return fieldTypeArray;
      }//end setFldTypeArray
       public String[] getFldSizeArray(){
         return fieldSizeArray;
      }//end setFldSizeArray
       public int getNumLines(){
         int countLines = 0;
         try {
                //File must be in same director and be the name of the string below
            BufferedReader in = new BufferedReader(new FileReader("files/FLAT.txt"));
            String str;
            while ((str = in.readLine()) != null) {
               countLines++;
            in.close();
             catch (IOException e) {}
         return countLines;
      }//end of getNumLines
       public void loadFixedWidthFile(){
         int c = getNumLines();
         int i = 0;
         String[] lineProcessed = new String[c];
         String chars;
         try {
                //File must be in same director and be the name of the string below
            BufferedReader in = new BufferedReader(new FileReader("files/FLAT.txt"));
            String str;
            while ((str = in.readLine()) != null) {
               //System.out.println(str.length());
               lineProcessed[i] = parseThatLine(str);
               i++;
            in.close();
             catch (IOException e) {}
            //write out the lineProcess[] array to another file
         writeThatFile(lineProcessed);
      }//end loadFixedWidthFile()
       public void writeThatFile(String[] s){
         try {
            BufferedWriter out = new BufferedWriter(new FileWriter("files/outfilename.txt"));
            for(int i = 0; i < s.length -1; i++){
               out.write(s);
}//end for loop
out.close();
catch (IOException e) {}
}//end writeThatFile
public String parseThatLine(String s){
int start = 0;
int end = 0;
String parsedLine = "";
int numChars = getFldSizeArray().length;
//Print number of lines for testing
//System.out.println(numChars);
String[] oArray = getFldSizeArray();
//String chars = oArray[0];
//System.out.println(chars.length());
//oArray
for(int i = 0; i < numChars -1; i++ ){
if(i == 0){
start = 0;
end = end + Integer.parseInt(oArray[i])-1;
else
start = end;
end = end + Integer.parseInt(oArray[i]);
parsedLine = parsedLine + s.substring(start, end) + "~";
}//end for loop
return parsedLine;
}//End of parseThatLine
I have tried to illeminate as many arrays as I can thinking that was chewing up the memory but to no avail.
Any thoughts or ideas?
Message was edited by:
SaipanMan2005

You should not keep a String array of all the lines of the file read.
Instead for each line read, parse it, then write the parsed line in the other file:      public void loadFixedWidthFile() {
         BufferedReader in = null;
         BufferedWriter out = null;
         try {
            //File must be in same director and be the name of the string below
            in = new BufferedReader(new FileReader("files/FLAT.txt"));
            out = new BufferedWriter(new FileWriter("files/outfilename.txt"));
            String str;
            while ((str = in.readLine()) != null) {
               //System.out.println(str.length());
               str = parseThatLine(str);
               //write out the parsed str to another file
               out.write(str);
         catch (IOException e) {
            e.printStackTrace(); // At least print the exception - never swallow an exception
         finally { // Use a finally block to be sure of closing the files even when exception occurs
            try { in.close(); }
            catch (Exception e) {}
            try { out.close(); }
            catch (Exception e) {}
      }//end loadFixedWidthFile()Regards

Out of Memory error while builng HTML String from a Large HashMap.

Hi,
I am building an HTML string from a large map oject that consits of about 32000 objects using the Transformer class in java. As this HTML string needs to be displayed in the JSP page, the reponse time was too high and also some times it is throwing out of memory error.
Please let me know how i can implement the concept of building the library tree(folder structure) HTML string for the first set of say 1000 entries and then display in the web page and then detect an onScroll event and handle it in java Script functions and come back and build the tree for the next set of entries in the map and append this string to the previous one and accordingly display it.
please let me know whether
1. the suggested solution was the advisable one.
2. how to build tree(HTML String) for a set of entries in the map while iterating over the map.
3. How to detect a onScroll event and handle it.
Note : Handling the events in the JavaScript functions and displaying the tree is now being done using AJAX.
Thanks for help in Advance,
Kartheek

Hi
Sorry,
I haven't seen any error in the browser as this may be Out of memory error which was not handled. I got the the following error from the web logic console
org.apache.struts.actions.DispatchAction">Dispatch[serviceCenterHome] to method 'getUserLibraryTree' returned an exceptionjava.lang.reflect.InvocationTargetException
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     at java.lang.reflect.Method.invoke(Method.java:324)
     at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:276)
     at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:196)
     at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:421)
     at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:226)
     at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
     at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:415)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
     at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:996)
     at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419)
     at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:315)
     at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6452)
     at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
     at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
     at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3661)
     at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2630)
     at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
     at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
Caused by: java.lang.OutOfMemoryError
</L_MSG>
<L_MSG MN="ILHD-1109" PID="adminserver" TID="ExecuteThread: '14' for queue: 'weblogic.kernel.Default'" DT="2012/04/18 7:56:17:146" PT="WARN" AP="" DN="" SN="" SR="org.apache.struts.action.RequestProcessor">Unhandled Exception thrown: class javax.servlet.ServletException</L_MSG>
<Apr 18, 2012 7:56:17 AM CDT> <Error> <HTTP> <BEA-101017> <[ServletContext(id=26367546,name=fcsi,context-path=/fcsi)] Root cause of ServletException.
*java.lang.OutOfMemoryError*
Please Advise.
Thanks for your help in advance,
Kartheek

Migration from MySql to Derby

I am currently migrating from MySql to Derby.
I have been developing a MRPII package for a while now and whilst my classes worked with MySql I am finding difficulty with implementing them using Derby.
I have developed a GUI interface for MRPII functions and use a Database Interface class to connect to the database.
The Database Interface class gives me the capability of reading and writing from a database.
I extend this capability to sub classes ie part, customers, suppliers, work orders.
The problem I have is with the ' "+standardCost+" ' type statements. In MySql, the database engine recognises that the standard cost variable is a double though using Derby I get the following Errors.
SQL Exception: Columns of type 'DOUBLE' cannot hold values of type 'CHAR'. In Save Data error code30000
SQL Exception: Columns of type 'DOUBLE' cannot hold values of type 'CHAR'. In Save Data error code30000
Executed statement INSERT INTO stock VALUES('BUDGET500','A1', '100.0','50.0','1.0','100.0')
Executed statement INSERT INTO part VALUES('BUDGET500', 'PENTIUM','76.0', '86.0','EACH','MP','7','33')
The only solution I see is giving each subclass of Database Interface the individual capability within that class to connect to the database and use prepared statements.
Does anyone know if Derby has the capability of processing the updates using a similar format as shown in my code.
package delta.databaseInterface;
import delta.databaseInterface.DatabaseInterface;
public class Part extends DatabaseInterface{
     private String partNumber;
     private String partType;
     private String description;
     private double standardCost;
     private double sellingPrice;
     private String stockLocation;
     private double quantityOnHand;
     private String unitOfMeasure;
     private double minimumStock;
     private double minimumOrderQuantity;
     private int leadTime;
     private double avaliableStock;
                     private int commodityCode;
     public void setPartNumber(String pn){
          this.partNumber = pn.toUpperCase();
     public void setDescription(String ds){
          this.description = ds.toUpperCase();
     public void setStandardCost(double sc){
          this.standardCost = sc;
     public void setSellingPrice(double sp){
          this.sellingPrice = sp;
     public void setQuantityOnHand(double qt){
          this.quantityOnHand = qt;
          this.avaliableStock = qt;
     public void setStockLocation(String sl){
          this.stockLocation = sl.toUpperCase();
     public void setUnitOfMeasure(String uom){
          this.unitOfMeasure = uom.toUpperCase();
     public void setTypeOfPart(String top){
          this.partType = top.toUpperCase();
     public void setMinimumStock(double ms){
          this.minimumStock= ms;
     public void setMinimumOrderQuantity(double moq){
          this.minimumOrderQuantity = moq;
     public void setLeadTime(int lt){
          this.leadTime = lt;
                     public void setCommodityCode(int cc){
          this.commodityCode = cc;
     public void insertIntoPartDatabase(){
       super.setDatabase("mrpii");
                       super.setUpdate( "INSERT INTO part VALUES('"+partNumber+"', '"+description+"','"+standardCost+"'," +
       " '"+sellingPrice+"','"+unitOfMeasure+"','"+partType+"','"+leadTime+"','"+commodityCode+"')");
                         super.saveData();
     public void insertIntoStockDatabase(){
        super.setDatabase("mrpii");
        super.setUpdate("INSERT INTO stock VALUES('"+partNumber+"','"+stockLocation+"',"+
" '"+quantityOnHand+"','"+minimumStock+"','"+minimumOrderQuantity+"','"+avaliableStock+"')");
                super.saveData();
     public void changePartDetails(){
        super.setDatabase("mrpii");
        super.setUpdate( "UPDATE part SET description = '"+description+"',unitOfMeasure = '"+unitOfMeasure+"',"+
          "partType = '"+partType+"' WHERE partNumber = '"+partNumber+"' ");
        super.saveData();
     public void adjustStock(){
          super.setDatabase("mrpii");
          super.setUpdate( "UPDATE stock SET stockLocation = '"+stockLocation+"',"+
          "avaliableStock = (('"+quantityOnHand+"' - qtyOnHand ) + avaliableStock), "+
          " qtyOnHand = '"+quantityOnHand+"' WHERE partNumber = '"+partNumber+"' ");
          super.saveData();
     public void insertAllocation(String allocationID, double quantity){
          super.setDatabase("mrpii");
          super.setQuery("INSERT INTO allocations VALUES('"+partNumber+"','"+quantity+"',"+
             " '"+allocationID+"')");
          super.saveData();
     public void reduceAvaliable(double allocated){
          super.setDatabase("mrpii");
          super.setUpdate( "UPDATE stock SET avaliableStock = (avaliableStock - '"+allocated+"') WHERE " +
          "partNumber = '"+partNumber+"' ");
          super.saveData();
     public void increaseAvaliable(double allocated){
          super.setDatabase("mrpii");
          super.setUpdate( "UPDATE stock SET avaliableStock = (avaliableStock - '"+allocated+"') WHERE " +
          "partNumber = '"+partNumber+"' ");
          super.saveData();
}Any help would be deeply appreciated
Thanks
Jim

I am currently migrating from MySql to Derby. Dear God, why?I am using the embedded functionality.
>
I have been developing a MRPII package for a whilenow
MRP as in "manufacturing resource planning"?Yes
>
and whilst my classes worked with MySql I am
finding difficulty with implementing them using
Derby.That suggests that your Java code relies too heavily
on MySQL features. You didn't make your code
portable enough.
I have developed a GUI interface for MRPIIfunctions
and use a Database Interface class to connect tothe
database.
The Database Interface class gives me thecapability
of reading and writing from a database.
I extend this capability to sub classes ie part,
customers, suppliers, work orders.
The problem I have is with the ' "+standardCost+"'
type statements. In MySql, the database engine
recognises that the standard cost variable is a
double though using Derby I get the following
Errors.Right - you're counting on something that might not
be true for all databases. MySQL appears to be doing
an implicit conversion from string to double. Bad
idea.
SQL Exception: Columns of type 'DOUBLE' cannothold
values of type 'CHAR'. In Save Data errorcode30000
SQL Exception: Columns of type 'DOUBLE' cannothold
values of type 'CHAR'. In Save Data errorcode30000
Executed statement INSERT INTO stock
VALUES('BUDGET500','A1',
'100.0','50.0','1.0','100.0')
Executed statement INSERT INTO part
VALUES('BUDGET500', 'PENTIUM','76.0',
'86.0','EACH','MP','7','33')Do you use PreparedStatements to escape strings and
dates for you? If not, you're in for a bumpy ride.
The only solution I see is giving each subclass of
Database Interface the individual capabilitywithin
that class to connect to the database and use
prepared statements.I would not have a single database interface. Each
class has its own requirements. How can a single
class know about all of them? Better to break your
persistence layer into several interaces, one per
persistent class.That is what I thought
>
Does anyone know if Derby has the capability of
processing the updates using a similar format as
shown in my code.I think your code is in trouble, Jim.Probably so, thought a redesign is not so much of a problem now as I am a bit more experienced than when I started out.
>
You don't use prepared statements, which would help
with your portability issues.
You have SQL embedded in the objects. I'd move it
out into a separate persistence layer.
You might want to read about Hibernate, an
object/relational mapping layer. You've got objects
and tables. Hibernate tools or Middlegen can
generate the XML mapping files for you. Once you
have those, you'll find that Hibernate will make
porting to another database a lot easier. It can be
as easy as changing configuration files.
I'd also look into Spring. It has some great
plumbing to help you deal with databases and lots of
other stuff.
%My main class is as follows;
package delta.databaseInterface;
* @author James Charles
import java.sql.SQLException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Statement;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.util.Vector;
import javax.swing.JTable;
public class DatabaseInterface{
     private String database;
     private String query;
                     private String update;
            private String[][] results = new String[0][0];
     private int columnCount = 0;
     private int rowCount = 0;
     private boolean resultsExist = false;
     Object data[][];
     private JTable table;
     public void setDatabase(String database){
          this.database = database;
     public void setQuery(String query){
          this.query = query;
            public void setUpdate(String update){
          this.update= update;
        public void saveData(){
          try{
          Class.forName("org.apache.derby.jdbc.EmbeddedDriver").newInstance();
          String sourceURL = new String("jdbc:derby:" + this.database);
          Connection databaseConnection = DriverManager.getConnection(sourceURL);
          Statement statement = databaseConnection.createStatement();
          statement. executeUpdate(this.update);
                databaseConnection.close();
          } // end of try
          catch(ClassNotFoundException cnfe){
               System.err.println(cnfe);
          catch(SQLException sqle){
               System.err.println(sqle + " In Save Data error code" + sqle.getErrorCode());
                         System.out.println("Executed statement " + this.query);
                catch(InstantiationException ie){
                    System.err.println(ie);
         catch(IllegalAccessException iae){
                    System.err.println(iae);
     public String getDatabase(){
          return this.database;
     public String getQuery(){
          return this.query;
     public void setResults(){
     try{
          Class.forName("org.apache.derby.jdbc.EmbeddedDriver").newInstance();
          String sourceURL = new String("jdbc:derby:" + this.database);
          Connection databaseConnection = DriverManager.getConnection(sourceURL);
          Statement statement = databaseConnection.createStatement();
          ResultSet thisResult = statement.executeQuery(this.query);
                databaseConnection.commit();
          System.out.println("Executed statement");
          convertToArray(thisResult);
          databaseConnection.close();
     } // end of try
     catch(ClassNotFoundException cnfe){
          System.err.println(cnfe);
     catch(SQLException sqle){
          System.err.println(sqle);
     catch(InstantiationException ie){
                    System.err.println(ie);
         catch(IllegalAccessException iae){
                    System.err.println(iae);
     public void convertToArray(ResultSet resultsIn) throws SQLException{
          this.resultsExist = false;
          String[] columnNames = new String[0];
          Vector dataRows = new Vector();
          ResultSetMetaData metadata = resultsIn.getMetaData();
          this.columnCount = metadata.getColumnCount();
          columnNames = new String[this.columnCount];
          for(int i = 0; i < this.columnCount; i++)
               columnNames[i] = metadata.getColumnLabel(i+1);
          String[] rowData = new String[this.columnCount]; // Stores one row
          while(resultsIn.next()){ // For each row...
               rowData = new String[this.columnCount]; // create array to hold thedata
                    for(int i = 0; i < this.columnCount; i++)// For each column
                    rowData[i] = resultsIn.getString(i+1); // retrieve the data item
                    dataRows.addElement(rowData); // Store the row in the vector
          this.results= new String[dataRows.size()][this.columnCount];
          this.data= new Object[dataRows.size()][this.columnCount];
          for(int column = 0; column < this.columnCount; column++)
               for(int row = 0; row < dataRows.size(); row++){
          this.results[row][column] = ((String[])(dataRows.elementAt(row)))[column];
          this.data[row][column] = ((String[])(dataRows.elementAt(row)))[column];
          this.rowCount = dataRows.size();
          try{
               if (results[0][0].equals(null)){ this.resultsExist = false;} //ckecks results
                    else {this.resultsExist = true;}
               catch (ArrayIndexOutOfBoundsException e){
               System.out.println(e + "in catch");
               this.resultsExist = false;
               System.out.println("Value of exists in catch = " + this.resultsExist);
     }// end of set results
     public JTable getTable(Object[] columnNames) {
          table = new JTable(data, columnNames);
          return table;
     public int getColumnCount(){
          return this.columnCount;
     public int getRowCount(){
          return this.rowCount;
     public String getData(int row, int column){
          return this.results[row][column];
     public boolean doResultsExist(){
          return this.resultsExist;
} // end of database InterfaceI developed this code about a year ago and maybe it is time to simplify it.
With MySql I could create an instance of Database Interface or use one of its subclasses and perform any function on the database by calling its methods.
maybe it is time for a redesign!
thanks
jim

Migration from Mysql : "No suitable driver"

Hi,
I'm trying a migration from Mysql 4, but always have the following error message : "No suitable driver".
I have already downloaded and tried many mysql J/Connectors. And I ziped the content of the \com directory and named it "mysql-connector-java.zip" as described in the user manual ('Before you migrate' section)... But always have the same message
Please help!

Hi,
If you carry out the following steps, you should hopefully be able to successfully connect to MySQL4:
Follow the steps outlined in Chapter 3 "Before Migrating From MySQL" of the User's Guide.
1. From the MySQL website,www.mysql.com, under the "Products" -> "Connectors" link, download the MySQL Connector/J 3.0 driver. The downloaded file will be named "mysql-connector-java-3.0.16-ga.zip".
2. Extract the contents of this zipfile.
3. Zip up the /com subdirectory to a new zipfile named "mysql-connector-java.zip". Check the zip to ensure that the path information of the contained files begins with the /com path name.
4. Copy the new "mysql-connector-java.zip" to the /drivers directory under your OMWB install.
5. Re-start the workbench.
6. On step 1 of the Capture Wizard, enter your MySQL login information, then click Next.
You should now be able to successfully connect, and move on to Step 2 of the wizard.
It sounds like you followed the steps already, but you may not have restarted the workbench after installing the new driver.
I hope this helps.
Regards,
Hilary

Out of memory error in migrating from mysql to oracle

Similar Messages

Maybe you are looking for