How to make this faster?? read millions of record from txt file

Hi there,
I got an issue. There is a txt file contains 2 million records, I also got another file contains over 10000 numbers. Now, I need to compare this 10000 numbers with that 2 million records if any records contains a number which belongs to 10000 number set, i retrieve this record and keep it. later on, when i finish the comparison i'll write all the result records into a txt file.
What kind of data structure shall i use to keep the records and numbers? how to make the comparison quicker? Any idea will do!
Thanks!

if i were to do it, i will insert bout the records into the db. then do an sql statement on the two tables to get the results. Then get the rs and output it to another text file.
just my opinion. not sure if this is faster.
Message was edited by:
clarenceloh

Similar Messages

How to get string (specified by line and column) from txt file with labview

Hi everyone
How to get string (specified by line and column) from txt file with labview
thx
Solved!
Go to Solution.

As far as I know, a text file has no columns. Please be more specific. Do you mean something like the 5th word on line 4, where words are separated by a space, and lines are separated by a newline character? You could the Read from Spreadsheet String function and set the delimiter to a space. This will produce a 2D array of strings. Then use index array and give the line number and column number.
- tbob
Inventor of the WORM Global

How to specify path to read E$ tab records into .xls file using odisqlunlod

Hi
Can any one help me how to specify specific path in odisqlunload tools which is useful for both windows and linux.
I am developing and testing in windows and moving generated scenario into linux box to test for testing people
If in case any error out records are populated in E$ table then how that records will populated on xls file, later I am sending that attachement for email notification to concern people
Below code is present in odisqlunload tool
OdiSqlUnload "-FILE=d:\ODI_Error_Out_Files\Notification_Error_Records.xls" "-DRIVER=oracle.jdbc.OracleDriver" "-URL=jdbc:oracle:thin:@10.75.114.146:1521:POCWCDS" "-USER=wcds" "-PASS=h2yXeih4hFlXXV,QaMeRR2Fy" "-FILE_FORMAT=VARIABLE" "-ROW_SEP=\r\n" "-DATE_FORMAT=yyyy/MM/dd HH:mm:ss" "-CHARSET_ENCODING=ISO8859_1" "-XML_CHARSET_ENCODING=ISO-8859-1"
select * from E$_notification
Please help how to make a single code which is useful and work on both windows and linux
Any sugession willl help me
Thanks in advance
Regards,
Phanikanth

Hi Bhabani,
I have written below code in KM itself and select technology as Java BeanShall
Code:
<@
String OS = System.getProperty("os.name").toLowerCase();
String v_path="";
if((OS.indexOf("win") >= 0))
v_path="D:\Unload_Dir\<%=snpRef.getSession("SESS_NO")%>.xlsx";
else if (OS.indexOf("mac") >= 0)
v_path="path details";
else if (OS.indexOf("nix") >= 0 || OS.indexOf("nux") >= 0 || OS.indexOf("aix") > 0 )
v_path="/odi_a/Middleware/logs/wcds/odi_logs/<%=snpRef.getSession("SESS_NO")%>.xlsx";
else if (OS.indexOf("sunos") >= 0)
v_path="soliaris path";
@>
OdiSqlUnload "-FILE=<@=v_path@>" "-DRIVER=<%=odiRef.getInfo("DEST_JAVA_DRIVER")%>" "-URL=<%=odiRef.getInfo("DEST_JAVA_URL")%>" "-USER=<%=odiRef.getInfo("DEST_USER_NAME")%>" "-PASS=<%=odiRef.getInfo("DEST_ENCODED_PASS")%>" "-FILE_FORMAT=VARIABLE" "-ROW_SEP=\r\n" "-DATE_FORMAT=yyyy/MM/dd HH:mm:ss" "-CHARSET_ENCODING=ISO8859_1" "-XML_CHARSET_ENCODING=ISO-8859-1"
select * from <%=odiRef.getTable("L","ERR_NAME", "W")%>
It was executed well and below is the Execution code of the above code
Execution Code:
<@
String OS = System.getProperty("os.name").toLowerCase();
String v_path="";
if((OS.indexOf("win") >= 0))
v_path="D:\Unload_Dir\1341360.xlsx";
else if (OS.indexOf("mac") >= 0)
v_path="path details";
else if (OS.indexOf("nix") >= 0 || OS.indexOf("nux") >= 0 || OS.indexOf("aix") > 0 )
v_path="/odi_a/Middleware/logs/wcds/odi_logs/1341360.xlsx";
else if (OS.indexOf("sunos") >= 0)
v_path="soliaris path";
@>
OdiSqlUnload "-FILE=<@=v_path@>" "-DRIVER=oracle.jdbc.OracleDriver" "-URL=jdbc:oracle:thin:@10.75.114.146:1521:POCWCDS" "-USER=wcds" "-PASS=<@=snpRef.getInfo("DEST_ENCODED_PASS") @>" "-FILE_FORMAT=VARIABLE" "-ROW_SEP=\r\n" "-DATE_FORMAT=yyyy/MM/dd HH:mm:ss" "-CHARSET_ENCODING=ISO8859_1" "-XML_CHARSET_ENCODING=ISO-8859-1"
select * from WCDS.E$_CDS_COMPANY
Please confirm me if the above Code is correct or not, if Not, please correct it and DESC_ENCODE_PASS is not encoding the password
Regards
Phanikanth
Edited by: Phanikanth on Feb 18, 2013 1:09 AM

Read and write binary from txt file

Well
i need to know how to convert strings to binary data
("Chapter One"--->100111000011) to compress them using Lempel Ziv compression algorithm, and print a new file with binary codes.
I know that i have to use OutputStream and ImputStream classes, but i don know how it works...
Please help me with this problem...

String.getBytes(String charsetName)

How to make this code go faster?

Hi,
I have this code and its really slow. My question is how to make it faster since its taking about 10 hours.
Code:
loop at ti_equi.
    select matnr sernr lief_nr kunde datum uzeit bwart posnr
appending corresponding fields of table ti_ser01
      from z_objk_ser01
     where sernr eq ti_equi-sernr
       and matnr eq ti_equi-matnr.
endloop.
delete ti_ser01 where not ( kunde in s_kunnr and
                              bwart = '631' ).
sort ti_ser01 by matnr sernr datum descending uzeit descending.
delete adjacent duplicates from ti_ser01 comparing matnr sernr.
What this code does is fetch all these fields and then just keeps the one that's been modified at last.
I have created an index by sernr and matnr in table objk which makes it just a little bit faster.
Any helpful answer will be appreciated.
Regards,
Roberto
Edited by: Julius Bussche on Jan 27, 2009 4:29 PM
Code tags added. Please use more meaningfull subject titles

Hi roberto
please avoid select statment inside loop.
use code
data: index type sy-tabix.
data : ti-ser011 type standard table of zobjk-ser01 with header line.
select matnr sernr lief_nr kunde datum uzeit bwart posnr
      from z_objk_ser01 into table ti_ser01.
sort ti_ser01 by sernr matnr.
sort ti_equi by sernr matnr.
index = 1.
loop at ti_equi.
    loop at ti_ser01 from index
     if ti_ser01-sernr = ti_equi-sernr
          and ti_ser01-matnr = ti_equi-matnr.
        move-corresponding ti_ser01 to ti_ser011.
        append ti_ser011.
        index = sy-tabix.
        exit.
     endif.
    endloop.
endloop.
here i have used one more internal table ti_ser011.
you can delete specific record from ti_ser01 internal table on condition. at that time u don't need to create any other internal table like ti-ser011.
in place of loop select statement---endloop.
you can use this code. I think it will help you.
please find if it is udeful.

How to DELETE millions of records. How to make it fast.

Hi
I need to delete near abt 134 millions of records from tables.
How to make it faster? any trick , any settings.
I am using Oracle 9i on Linux box.
If suppose i use TRUNCATE . does it deletes all objects defined over tables like constraint, indexes etc.
Thanks,
Kuldeep

hi
SQL> create table te as select * from all_objects;
Table created.
SQL> create index te_ind on te ( owner);
Index created.
SQL> truncate table te;
Table truncated.
SQL> select index_name , status from user_indexes where table_name = 'TE';
INDEX_NAME STATUS
TE_IND VALID
SQL> create table ti as select * from all_objects;
Table created.
SQL> create index ti_ind on ti ( owner);
Index created.
SQL> drop table ti;
Table dropped.
SQL> select index_name , status from user_indexes where table_name = 'TI';
no rows selected
SQL>regards
Taj

How to make this work with Firefox, HELP!

Downloading for Real-player, after watching the full movie, I click download and it has to reread the movie from the internet. When using explorer, after downloading the movie, it reads it from memory, which makes it a fast download. How to make this work with Firefox, I like not to use Microsoft products, and I really like Firefox 7.0.1!!!! HELP!

-> click '''Firefox''' button and click '''Options''' (OR File Menu -> Options)
* Advanced panel -> Network tab
* place Checkmark on '''Override Automatic Cache Management''' -> under '''Limit Cache''' specify a large size of space
* Remove Checkmark from '''Tell me when websites asks to store data for offline use'''
* click OK on Options window
* Restart Firefox
Check and tell if ts working.

Something keeps trying to download on my mac and I don't know what it is. It is not in the apple store and just comes out nowhere and request for my password to download something and I don't know what it is. How to make this stop?

Something keeps trying to download on my mac and I don't know what it is. It is not in the apple store and just comes out nowhere and request for my password to download something and I don't know what it is. How to make this stop? It pops up every single day.

Erica,
     I can, with 99.99% certainty, tell you that you are absolutely right in not wanting to download or install this "Helper," whatever it is (but we can be equally certain it would not "help" anything).
     I cannot comment as to Oglethorpe's recommendation of 'adwaremedic'; I am unfamiliar with it. His links to the Apple discussion and support pages warrant your time and attention.
     It might be really simple -- Trying looking in your Downloads folder, trash anything that you don't know with certainty is something you want to keep, and then Secure Empty your Trash. Then remove the AdBlock extension, LastPass, and Web of Trust extensions to Safari and re-boot. If the issue goes away, still be extraordinarily careful in the future.
     Unfortunately, it's probably not going to be that simple to get rid of, in which case I'd then try the line by line editing in HT203987.
     I have no further suggestions (other than a complete wipe and re-install...but that's a pain because trying to restore from Time Machine would simply ... restore the Mal).
   For the rest of us, please post when you find a solution.
     Best.
     BPW
(Also, try to edit your second post -- black out your last name on the screenshot and re-post it for others)

TS2634 I bought a composite AV cable with 30 pin connector at a proper apple store for my ipad 2 which no longer works now i have updated to ios7 - please advise how to make this work ?

I bought a composite AV cable with 30 pin connector at a proper apple store for my ipad 2 which no longer works now i have updated to ios7 - please advise how to make this work ?

I have the same problem.
Two similar discussions:
https://discussions.apple.com/message/23081658#23081658
https://discussions.apple.com/message/23281391#23281391
I have not yet seen any official response to the question: "Is the Apple AV Composite cable fully supported with 30pin connector devices upgraded with iOS7 - specifically ? - eg. iPad 2, iPhone 4, iPhone 4s"
If it is not currently supported is that then due to a bug / oversight and in that case is it something that will be fixed in the near future?
Please let us know what feedback you got from asking Apple support.

Slow Phone 4s with ios8!! How to make it faster?

Slow Phone 4s with ios8!! How to make it faster? After upgraded ios8.1.1, my iPhone 4s Facebook become very slow response and always hand. Whastapp having same problem! Very lagging with safari too! It is really regret to upgrade since many peoples say upgrade will have this issue always

Slow Phone 4s with ios8!! How to make it faster? After upgraded ios8.1.1, my iPhone 4s Facebook become very slow response and always hand. Whastapp having same problem! Very lagging with safari too! It is really regret to upgrade since many peoples say upgrade will have this issue always

Anyone knows how to make this code to netbeans??

anyone knows how to make this code to netbeans?? i just want to convert it into netbeans... im not really advance with this software... anyway..just reply if you have any idea...or steps how to build it... etc.... thanks guys...
   import javax.swing.*;
   import javax.swing.table.*;
   import java.awt.*;
   import java.awt.event.*;
   import java.util.regex.*;
   public class FilterTable {
     public static void main(String args[]) {
       Runnable runner = new Runnable() {
         public void run() {
           JFrame frame = new JFrame("Sorting JTable");
           frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
           Object rows[][] = {
             {"AMZN", "Amazon", 41.28},
             {"EBAY", "eBay", 41.57},
             {"GOOG", "Google", 388.33},
             {"MSFT", "Microsoft", 26.56},
             {"NOK", "Nokia Corp", 17.13},
             {"ORCL", "Oracle Corp.", 12.52},
             {"SUNW", "Sun Microsystems", 3.86},
             {"TWX", "Time Warner", 17.66},
             {"VOD", "Vodafone Group", 26.02},
             {"YHOO", "Yahoo!", 37.69}
           Object columns[] = {"Symbol", "Name", "Price"};
           TableModel model =
              new DefaultTableModel(rows, columns) {
             public Class getColumnClass(int column) {
               Class returnValue;
               if ((column >= 0) && (column < getColumnCount())) {
                 returnValue = getValueAt(0, column).getClass();
               } else {
                 returnValue = Object.class;
               return returnValue;
           JTable table = new JTable(model);
           final TableRowSorter<TableModel> sorter =
                   new TableRowSorter<TableModel>(model);
           table.setRowSorter(sorter);
           JScrollPane pane = new JScrollPane(table);
           frame.add(pane, BorderLayout.CENTER);
           JPanel panel = new JPanel(new BorderLayout());
           JLabel label = new JLabel("Filter");
           panel.add(label, BorderLayout.WEST);
           final JTextField filterText =
               new JTextField("SUN");
           panel.add(filterText, BorderLayout.CENTER);
           frame.add(panel, BorderLayout.NORTH);
           JButton button = new JButton("Filter");
           button.addActionListener(new ActionListener() {
             public void actionPerformed(ActionEvent e) {
               String text = filterText.getText();
               if (text.length() == 0) {
                 sorter.setRowFilter(null);
               } else {
                 try {
                   sorter.setRowFilter(
                       RowFilter.regexFilter(text));
                 } catch (PatternSyntaxException pse) {
                   System.err.println("Bad regex pattern");
           frame.add(button, BorderLayout.SOUTH);
           frame.setSize(300, 250);
           frame.setVisible(true);
       EventQueue.invokeLater(runner);
   }

its okay onmosh.....what we need to
this...forum....is to have a good......relationship
of programmers......and to start with .....we need to
have a good attitude........right.....???.....no
matter how good you are in programming but if you did
not posses the right kind of attitude....everything
is useless.....in the first place....all we
want....is just to ask...some....help....but
conflicts......but unluckily......we did not expect
that there are members in here which......not
good...to follow.....just as suggestion for those
people not having the right kind of attidude...why
can't you do in a very nice message sharing in a very
nice....way....why we need to put some
stupid....stuff...words.....can't you live with out
******* ****....its not.....lastly especially you
have your children right now and people around...that
somehow......idiolize...you even me....is one of
them......but showing but attitude....is not
good......tnx....hope you'll take it this....in the
positive side.....be optimistic...guys....the
world..is not yours....all of us will just past
away....always..remember that one.....treasure..our
stay in this....temporary home.....which...is
world....Whoa. That post seems to be killing my brain.
URK
Join........us..........do not be..........afraid.......

How to make this pattern?

Hello guys Please I am looking for advice how to make this kind of pattenr in pohtoshop- you can see it on background- the rounded one- en the end whole piece makes nice surface.
http://vladstudio.deviantart.com/art/Typographic-World-Map-106395788
thank a lot

Sometimes a patten is defined for some pixel size and may even need to have the document rotated to have the proper orientation for the pattern. For example a frame pattern for a one inch wide strip. An Action can be created to add the pattern to a mitered 1" wide strip at 300 dpi on the left side then rotate the document for the other side rotate back and dupe and flip the frame side added.
If you search the web there are a countless number of patterns that can be downloaded for free textures like woods ,paper, textiles etc.

How to make this logo effect? Help please!

How to make this effect? I'm new in illustrator.

It looks like 3 discs with a 2d image applied to them. I would try the 3d revolve tool to create one, then duplicate it for the other two.

How to make script faster

Hi,
I have a table with huge volume of data. I am doing following activity using SQL script
1). alter table my_tab add new_col NUMBER(10);
2). update my_tab set new_col= 50;
3). alter table my_TAB modify new_col NOT NULL;
4). Create index on my_tab(new_col);
Above scrip is taking around 10 hours to excecute.
Could any one please suggest how to make it fast.
Thanks.

amit_hum wrote:
Hi,
I have a table with huge volume of data. I am doing following activity using SQL script
1). alter table my_tab add new_col NUMBER(10);
2). update my_tab set new_col= 50;
3). alter table my_TAB modify new_col NOT NULL;
4). Create index on my_tab(new_col);
Above scrip is taking around 10 hours to excecute.
Could any one please suggest how to make it fast.
Thanks.You haven't told us your database version.
You haven't shown us the details of your table structure or an example of the data, or even how many rows is a "huge volume of data".
Without proper information we would just be guessing and that doesn't help anybody.

How do make this query ?

CREATE TABLE CLIENT (
     CLIENTID NUMBER PRIMARY KEY,
     NAME VARCHAR2(40)
CREATE TABLE SALESMAN (
     SALESMANID NUMBER PRIMARY KEY,
     NAME VARCHAR2(40),
     ALL_CLIENTS NUMBER -- ACCESS ALL CLIENTS IF VALUE IS 1
CREATE TABLE CLIENTSALESMAN (
     SALESMANID NUMBER,
     CLIENTID NUMBER
INSERT INTO CLIENT VALUES(1,'Bob');
INSERT INTO CLIENT VALUES(2,'Jhon');
INSERT INTO CLIENT VALUES(3,'Robert');
INSERT INTO CLIENT VALUES(4,'Clarck');
INSERT INTO SALESMAN VALUES (1,'Christina',0);
INSERT INTO SALESMAN VALUES (2,'Doug',1); -- access all clients
INSERT INTO CLIENTSALESMAN VALUES (1,1);
INSERT INTO CLIENTSALESMAN VALUES (1,2);
How do you return all the clients of a salesman ? same that SALESMAN.ALL = 1 OR SALESMAN.ALL = 0.
the salesman.all is 1 then he access all clients no need insert in CLIENTSALESMAN. If SALESMAN.ALL = 0 he
access clientsalesman.
Does somebody some suggestion how i make this ?
thanks !

1 select *
2 from (select a.name SALESMAN, b.name CLIENT
3 from   salesman a, client b, clientsalesman
4 where a.SALESMANID = c.SALESMANID
5 and    b.CLIENTID = c.CLIENTID
6 union
7 select a.name SALESMAN, b.name CLIENT
8 from   salesman a, client b
9 where   a.ALL_CLIENTS = 1)
10* where salesman = '&1'
SQL> /
Enter value for 1: Christina
old 10: where salesman = '&1'
new 10: where salesman = 'Christina'
Christina                                Bob
Christina                                Jhon
SQL> /
Enter value for 1: Doug
old 10: where salesman = '&1'
new 10: where salesman = 'Doug'
Doug                                     Bob
Doug                                     Clarck
Doug                                     Jhon
Doug                                     RobertNicolas.

How to make this faster?? read millions of record from txt file

Similar Messages

Maybe you are looking for