Parsing File

Which is more efficient: parsing a file then store the results in an ArrayList/Vector/etc or parsing a file then puting the results in a file?
Note:
1. the file to be parsed is a huge log file.
2. results will be used (stored in DB maybe).

this is a very cool advise...i appreciate it coz i
think am one of those guys who worries about memories
issues too much :)
people sometimes worry about memory too much. Youshould probably just do the easiest thing first
(e.g., use the ArrayList, if you find it easier) and
only worry about memory if you find that you're
actually getting memory issues
i'll go for saving individual results to DB during
parsing.
thanks again :)As an example here is where you can take full advantage of the Jav™
Let's say you decide well gee I really do want to store all the data before I add it. And right now the sets are small but one day they could get really large.
Store the data in a List. Always make sure that except for the one line you instantiate the List (ArrayList let's say) you reference the List as a List.
Then one day when you run into the memory size issue you could implement a simple List backed by a file of some sort (perhaps RandomAccessFile) and it will work with the rest of your code as is.
It would obviously be slower to go on disk then from memory but it would work and be scalable. And best of all you didn't have to bust apart all your code to get there.

Similar Messages

Wrong result then parsing file in applet

My applet is parsing a 3ds-file and it's working good in both applet viewer and as a stand alone application but then i try it as an applet in a browser it sometimes parses wrong.
This is the printout i get then parsing is correct:
Found data/model/chest.3ds in jar file.
main chunk
unused chunks : 2 with length: 10
3d editor chunk
unused chunks : 3d3e with length: 10
unused chunks : ffffafff with length: 170
unused chunks : 100 with length: 10
object chunk
Namnet p� objektet �r inl�st: Chest01
triangular mesh chunk
vertices list chunk
Antalet vertices 424
Antalet uv-koordinater 424
unused chunks : 4160 with length: 54
Antalet faces 168
unused chunks : 4130 with length: 349
unused chunks : 4150 with length: 678
unused chunks : ffffb000 with length: 221
This is the printout then run in an applet:
Found data/model/chest.3ds in jar file.
main chunk
unused chunks : 2 with length: 10
3d editor chunk
unused chunks : 3d3e with length: 10
unused chunks : ffffafff with length: 170
unused chunks : 100 with length: 10
object chunk
Namnet p� objektet �r inl�st: Chest01
triangular mesh chunk
vertices list chunk
Antalet vertices 424
Antalet uv-koordinater 424
unused chunks : ffffed3e with length: -1572921687
unused chunks : 7856 with length: 1381233982
This problem only occurs in some files.
I'm parsing it by reading in the correct number of bytes and converting them to the correct type (int, char ..ect).
Is there any difference in parsing files in a browser than in an application? The only difference I had so far is the browser beeing case sensitive in the filenames then the appletviewer isn't.
I can post the code if you want to.
Please help me, thanks.
/ Robert

Ifanyone else having the same trouble I'm publishing the wrapper I used successfully.
private static void read(InputStream in, byte[] buffer) throws IOException
        int bytesRead = 0;
        int bytesToRead = buffer.length;
        while (bytesToRead > 0)
            int read = in.read(buffer, bytesRead, bytesToRead);
            bytesRead += read;
            bytesToRead -= read;
    }

Does parse file change on v3.0 ?

Hi everybody
I tried to create parse file on 3.0 version of Apex with the same rules as old version. It didn't work.
Before 3.0 we had
In the page process "Parse file", it has...
htmldb_tools.parse_file(:P1_FILENAME,'P1_COLLECTION','P1_HEADINGS','P1_COLUMNS','P1_DDL');
And on 3.0 ? I tried
In the page process "Parse file", it has...
apex_tools.parse_file(:P1_FILENAME,'P1_COLLECTION','P1_HEADINGS','P1_COLUMNS','P1_DDL');
But It didn't work
Please follow me the right way ^^
Regards
Message was edited by:
user450801

Hi user450801,
you have to be a little bit more patient. It's weekend!
The htmldb_tools package is a package written by Vikas (http://htmldb.oracle.com/pls/otn/f?p=38131:1) which doesn't have to do anything with the upgrade of APEX to 3.0.
As long as you haven't renamed your package to APEX_TOOLS it will still be accessible as HTMLDB_TOOLS.
Patrick
My APEX Blog: http://inside-apex.blogspot.com
The ApexLib Framework: http://apexlib.sourceforge.net
The APEX Builder Plugin: http://sourceforge.net/projects/apexplugin/

Error parsing file

Hi! My program compile fine and runs fine, however on the main file it has the red exclamation mark that says error parsing file. What does that mean??
THX
Edited by: jjcpp on Mar 3, 2010 9:55 PM

That probably means that the JavaFX plugin isn't perfect...
I suppose you mean you see that in NetBeans, right?
These IDEs are powerful but sometime confused; for example from time to time Eclipse tells me a Java class (in a big Java project) doesn't exist just because it is stale (changed by VCS) and its index isn't up to date.

Error message when using DOMParser.parse(file)

When I try to use the DOMParser.parse(file) to parse an invalid XML file against a XML schema, with the schema_validation feature set on, error message saying the XML file is invalid because it violates some constraints set in the schema. But I cannot find out how the error message is print. It is not caught as exception. May I know how can I detect the error in the code instead of viewing it as output?

The error message I got is like the followings:
[Error] abc3.xml:2:310: cvc-pattern-valid: Value '' is not facet-valid with respect to pattern '($\d\d\d$-)?[\d]{8}' for type 'phoneNoType'.
[Error] abc3.xml:2:310: cvc-type.3.1.3: The value '' of element 'mobile' is not valid.
[Error] abc3.xml:2:318: cvc-pattern-valid: Value '' is not facet-valid with respect to pattern '[^@]+@[^\.]+\..+' for type 'emailAddressType'.
[Error] abc3.xml:2:318: cvc-type.3.1.3: The value '' of element 'email' is not v
alid.
May I know how can I catch these errors?

Drawing graphic component using parsed file

hi,
I have a problem in drawing the graphic components like rectangle,oval etc. I need to draw them on a panel after selecting a xml file from menu,parse it first, based on the contents of the parsed file the dimentions of the component should change.
however i am able to parse the file and save the parsed contents in a separate file, but unable to draw components using its contents

this is the module given to me where i am supposed to
draw the graphic components using the contents of a
xml file. I was told to parse the xml file to get
some values that would represent parameters like
packet size, length etc.. and using these values
draw a data packet representation. But prob here is
the i am able to parse file but not pass its values
simultaneously to draw image on the panel
immediately after parsing. i am storin the contents
in the text format.
ya i have good knowlege of swings,IO and awt...Parse the XML file filling a model that allows listeners (Observer/Observable is one approach). When the parsing is complete fire an event to indicate that the model has changed. Have your GUI listen to the model and when it receive the 'changed' event it updates it's display according to the content of the model.

DNG error parsing file (Fuji S100fs)

Windows Vista Home Premium, DNG Converter 3.7, Fuji S100fs
I tried to convert Raw-files into DNG-files but got an error message (literally translated from German): "error parsing file".
The Fuji S100fs has been included in the latest RAW Update so I assume the DNG-Converter should recognize the format.
What else could be the reason?
Thanks.
Hans

The problem is DNG Converter 3.7 is NOT the most recent version of the DNG Converter. You need to use the DNG Converter that is included in the latest raw update.

CS4 cannot parse file????

I am runing Vista Ultimate 64 bit with SP1 and whenever I use CS4 to read a .gif file Photoshop reports "Could not complete your request because the file-format module cannot parse the file". I have un-installed CS4 and then re-installed again but I still get the same message. Can anyone please tell me what this means and perhaps help me to resolve the problem?

Middleleaze wrote:
Both Paint and Opera will open my .gif files but CS4 and IE8 will not!
Then you should be able to take a screen shot with Alt Print Screen, and then open the image in PS, and then crop to size. Use Save for Web to get the most compatible GIF format.

Parse files and store to "timeseries" for ouput to jpg/png?

Hi.
I am trying to parse through some dirs for known files and store the content + name of dir in a "timeseries" (the contents are csv and name of dir is string - hence "timeseries"), for later output to PNG.
The file structure will be:
dir1/filename.csv (together with other files)
dir2/filename.csv (together with other files)
dir3/filename.csv (together with other files)
My idea so far is:
public main {
   JFreeChart ... createChart(createDataset);
private static createDataset {
    traverse("my dirs and files");
    TimeSeries series1 = new TimeSeries(name of dir1, data from file1);
    TimeSeries series2 = new TimeSeries(name of dir2, data from file2);
     etc....
private static createChart {
private static traverse(File dir) {
     this method scans the dirs;
     if file = "filename.csv" readFile();
private static readFile(file, path) {
     read the data;
}I have come to a stop, because my method/idea would only return timeseries for the first dir/file over and over again.
Maybe I'm doing this awkwardly, but I'm very fresh at java.
Any inspiration would be appreciated.
Cheers,
Chotor

Thanks for answering.
Maybe I should clearify a bit.
I am already able to read csv files (thanks to opencsv). I can also travese directories looking for the correct file. I can even output timeseries to png thanks to JFreeChart.
The problem is stiching it all togheter, and more precisely passing data between methods.
When creating timeseries within my ceateDataset method, I am calling traverse, which then calls readData. I want these two (dir-name from traverse() and actual data from readData()) to be read into a new timeseries.
Maybe this could be solved differently...
We haven't covered this topic in my course yet. I'm fresh but eager. :)

Parse file and evaluate expressions

Hi, I'm fairly new to Java and would be grateful for advice on the following problem:
I have a text file containing a set of equations in the form
-7 < a0 < 10
9 < a1
a0+a1 = 0...
I need a method that can check quickly if a given array of numbers a0, a1... fulfills ALL of the conditions specified in the text file. I would like the method to parse the text file only once, and then create some temporary representation of it. I do NOT want to hard-code the set of equations in my source code.
A time-efficient implementation is crucial for my purpose. The idea is that the text file represents a high-dimensional (10-20 dim.) space, and I would generate a random point a0, a1 ... and check if it lies in that space (i.e. fulfills ALL the conditions).
Any suggestions would be much appreciated.

jstmills1 wrote:
It's actually systems biology research. I am investigating the constraints on metabolic systems (in this case, the yeast Saccharomyces cerevisiae). This yeast is capable of performing about 750 reactions between say, 500 metabolites. I'd like to toast you and your research.
Actually, in a past life I did research that indirectly involved S. cerevisiae: I studied human polymorphisms in the alcohol and aldehyde dehydrogenase genes and how this related to alcohol-induced liver disease. Now a-days, my only experience with this bug, other than enjoying it's effects on my drinks, is to check antibodies against it in patients suspected of having Crohns disease.
I am looking at flux distributions through that system that do not change the concentrations of the metabolites, i.e. if a metabolite is formed by reaction A and consumed by reaction B, then I am interested in the solution where the rates of A and B are equal. Also, there are lower and upper limits on the reaction rates, giving rise to inequalities a1_min < a1 < a1_max.
I currently want to generate a set of points that lie in that solution space and correspond to functional states of that metabolic system. A representative set of points gives me an idea of how robust a particular metabolic function of that organism is. I have implemented the rest of this project in Java (visualization of fluxes through metabolic networks, generation of the constraints as given above) without worrying about processor time. Now the generation of this set of points, and checking if they lie in that space, is where processor time becomes important for me.So do you already have a solution, but are looking for a faster implementation? Or are you working from scratch on this issue?

Parse file text file

Hi,
I like read Apache error log using external table.
For this I did create directory APACHE_LOG and grant read permission to it for my schema.
Then external table
CREATE TABLE EXT_APACHE_ERROR_LOG
    "TEXT" VARCHAR2(2000 BYTE)
ORGANIZATION EXTERNAL
    TYPE ORACLE_LOADER DEFAULT DIRECTORY "APACHE_LOG" ACCESS PARAMETERS ( records delimited BY newline nobadfile nodiscardfile nologfile ) LOCATION ( "APACHE_LOG":'error.log' )
REJECT LIMIT UNLIMITED;Log format is like below
[Wed Oct 11 14:32:52 2000] [error] [client 127.0.0.1] client denied by server configuration: /export/home/live/ap/htdocs/testI have try read file using query
SELECT REGEXP_SUBSTR(text, '[^\[]+(.*?)')      AS mesg_date,
REGEXP_SUBSTR(text, '\[([a-z]*?)\]')         AS mesg_type,
REGEXP_SUBSTR(text, '\[client ([0-9\.]*)\]') AS client,
REGEXP_SUBSTR(text, '\] (.*)$')              AS mesg
FROM ext_apache_error_log;Output it is not quite what I like have as I'm bad with regexp.
I like have result in 4 columns like
MESG_DATE               MESG_TYPE     CLIENT          MESG
Wed Oct 11 14:32:52 2000     error          127.0.01     client denied by server configuration: /export/home/live/ap/htdocs/testSo I need bit help
What is best method parse that log format?
Can I have those 4 columns some way already to external table?
Thanks advance
Regards,
Jari

Hi,
I just noticed that log have lines where is no data for third column (client).
This query demonstrate situation
WITH test_data AS
(SELECT '[Wed Oct 11 14:32:52 2000] [error] [client 127.0.0.1] client denied by server configuration: /export/home/live/ap/htdocs/test' text
FROM dual
UNION ALL
SELECT '[Wed Oct 11 14:32:52 2000] [notice] SIGUSR1 received. Doing graceful restart' text
FROM dual
SELECT REGEXP_replace(text, '^\[(.*?)\].*$', '\1')   AS mesg_date,
REGEXP_SUBSTR(text, '\[(.*?)\]',1,2)               AS mesg_type,
REGEXP_SUBSTR(text, '\[(.*?)\]',1,3)               AS client,
REGEXP_replace(text, '^(\[.*?\]\s){3}(.*)$', '\2') AS mesg
FROM test_data ;In case there is no client info whole line from file is in fourth column (mesg).
What could be best way handle this case so that client column is null but only the message should shown in mesg column ?
Thanks advance
Regards,
Jari
Edited by: jarola on Aug 31, 2011 8:24 PM
I did come to this query
WITH test_data AS
(SELECT '[Wed Oct 11 14:32:52 2000] [error] [client 127.0.0.1] client denied by server configuration: /export/home/live/ap/htdocs/test' text
FROM dual
UNION ALL
SELECT '[Wed Oct 11 14:32:52 2000] [notice] SIGUSR1 received. Doing graceful restart' text
FROM dual
SELECT TO_DATE(REGEXP_REPLACE(text, '^\[(.*?)\].*$', '\1'),'Dy Mon DD HH24:MI:SS YYYY', 'NLS_DATE_LANGUAGE = AMERICAN') AS mesg_date,
REGEXP_SUBSTR(text, '\[(.*?)\]',1,2)                                                                                  AS mesg_type,
REGEXP_SUBSTR(text, '\[(.*?)\]',1,3)                                                                                  AS client,
REGEXP_REPLACE(REGEXP_REPLACE(text, '^(\[.*?\]\s){3}(.*)$','\2'), '^(\[.*?\]\s){2}(.*)$','\2')                        AS mesg
FROM test_data ;It seems work. If somebody have better query please post.
And also if there is in good way combine expression that remove opening bracket and closing bracket from MESG_TYPE and CLIENT columns.
Thanks

Problem in Parsing File

Hi,
I have one parse.PROPERTIES file.. In that all data is in key-value pair. There are some keys has multiple value, and some keys has multiline value. I want to parse that file and store result in Map depending on key....
Content of File is as FOLLOWS:
   value=server
   multilinevalue = {
            "width" : "100px",
            "values" : [ { "25GB" : "25 GB" },
                                { "50GB" : "50 GB" },
                                { "75GB" : "75 GB" },
                                { "100GB" : "100 GB" },
                                { "150GB" : "150 GB" },
                                { "200GB" : "200 GB" },
                                { "500GB" : "500 GB" },
                                { "1 TB" : "1 TB" } ] }
    multiplevalue= { "status.x" : 0, "status.y" : 30, "status.width" : 270,     "status.height" : 30 }
                              Can anyone give me code how I can parse it...
Thank u,

Can anyone give me code how I can parse it...This is not a difficult problem and I'm sure someone can help you if you show some effort.

Parse files in directory newer than MM/DD/YYYY

In the attached VI I am trying to figure out how to pull in and parse all .txt log files in Folder path newer than a particular date instead of just the *.txt (all text files) as it is set to now. The date I will be entering from a sub VI is a string in the format MM/DD/YYYY. The date & time are included in the file names. The comma delimited .txt file names are broken down into "TesterID UUTSerial# Date TimeTested.txt". I have attached one of the comma delimited .txt files that the path contains. Anyone have any suggestions? I am running LV2009 SP1.
Thanks!
Solved!
Go to Solution.
Attachments:
Concatenate Files.vi ‏12 KB
2 123456789 Jun 24 2010 124729.txt ‏1 KB

Extracting the date from the filename and comparing it to an input date seemed like a fun thing to write. So I wrote a VI to do that. Here it is. Just wire in the filenames array and the desired date to compare. The output is a list of files that are greater than or equal to the specified date. You can change the compare function if you want something different. The filename must have a 3-letter month followed by a space, followed by a date, another space, the year, a space then whatever else. The compare date must be in the form mm/dd/yyyy.
- tbob
Inventor of the WORM Global
Attachments:
File_GTEQ_Date.vi ‏22 KB

Parse file without xml specification or document element

I have a large (600mb) log file that is in xml format but it does not have an xml specification and has no document element.
file looks like this:
<message>...</message>
<message>...</message>

<message>...</message>
<message>...</message>
I have written a class that overrides the SAX DefaultHandler but now want to be able to parse the document without having to add the xml spec and document element manually.
I've thought about writing a subclass of FileReader that adds the xml specification and document element before reading physical file but would also need to add closing document element at end of file.
Is there a simpler way?

Hi,
There is another way around the problem of adding a missing root node. This involves adding an extra DTD file and a xml file, like this one:
<?xml version='1.0' encoding='UTF-8' standalone="no"?>
<!DOCTYPE messageSet SYSTEM "logfile.dtd"
[<!ENTITY data SYSTEM "logfile.xml">]
>
<messageSet>
&data;
</messageSet>
This file "includes" the logfie.xml, as an external entity, with your messages as child nodes of element messageSet.
In your program you refer to this xml file when parsing the messages.

Parse file.

hi, i'm using:
NodeList internalList = (<a>).getElementsByTagName("");
//and this method returns number of nodes: 2. but i want to have only one counting node without the deepest one.
// internalList = 1 <---- how can i do that??<a>

<c>

</c>

</a>

I think I need a little more on the possible contents of you XML file (which I assume it is?) Besides my XML parsing is a little rusty.
If you knew there would always be 2 bs inside each other, you could just throw away the inner one?
If you know there's always an a around the outer b, you can iterate through the contents of a and you'll meet the b you want.
If, if, ... As I said, I don't know enough.

Parsing file filled with integers fast (?)

Hi,
I am trying to parse a file that has a bunch of integers seperated by spaces. I am using a BufferedReader, then looking at each char to see if it's a digit, then getting the digit and computing the integer. This seems to be 10 times slower than the same parser written in C++.
How can I boost the performance?
Thanks!
M.

So I use a LineNumberReader to read one line at the time. I pass that string to getIntegers (see below). This returns a list of integers.
Dealing with bytes directly sounds great. How do I do that?
Thanks!
M.
public ArrayList<Integer> getIntegers(String line){
 line += " ";
 ArrayList<Integer> res = new ArrayList<Integer>();
 boolean start = false;
 boolean neg = false;
 int l = 0;
 char c;
 for(int i = 0; i< line.length(); i++){
 c = line.charAt(i);
 if (c == '-'){
 neg = true;
 } else if (! Character.isWhitespace(c)) {
 if (start == false) start = true;
 l = l * 10 + Character.digit(c, 10);
 } else if (start == true) {
 if (neg) l = -l;
 res.add(new Integer(l));
 l = 0;
 start = false;
 neg = false;
 return res;
}

Parsing File

Similar Messages

Maybe you are looking for