Find Record count in a file.

Hi,
Is it possible to find the total no. of records (lines) in a file?
With External Table: and using select count(*) on that external table.
But here, we will only get the rows which are valid. Non Valid records will go to the bad file.
Another ways is: Reading the entire file using utl_file.get_line and then incrementing a counter.
But, we need to read through the whole file just to get the count.
Is there any better way of doing this?
Any suggestions?
( I accept sql/plsql is for db related things and not to handle OS related tasks. But I don't like writing a seperate shell script for this small thing because, I just moved every thing like data load, emailing etc in to plsql from unix and proc :) )
Version:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi
PL/SQL Release 10.2.0.1.0 - Production
Unfortunately No Java is Installed
Thanks in advance,
Pradeep.

pradeep.vupala wrote:
Hi,
Is it possible to find the total no. of records (lines) in a file?
With External Table: and using select count(*) on that external table.
But here, we will only get the rows which are valid. Non Valid records will go to the bad file.
Another ways is: Reading the entire file using utl_file.get_line and then incrementing a counter.
But, we need to read through the whole file just to get the count.
Is there any better way of doing this?Not really, that's just about your two ways.
1 alternative...
You could set up a dummy external table pointing to the file that just has a single column of varchar2 with no delimiters so it reads a line at a time, and won't get any bad rows, then do a count(*) on that table.

Similar Messages

Records count in Receiver File Adapter

Hi All,
I have a scenario IODC to File. The requirement is to get the total record count in the Trailer record of the Target File .Please help.
Regards,
Kevin

Hello,
The requirement is to get the total record count in the Trailer record of the Target File
In your XML, decide which IDOC field you get your records from and then do a similar mapping
IDOC field -> removeContext -> count -> TrailerRecordfield
Hope this helps,
Mark

Record count in txt file

Hi Experts,
Actually I have an output file with 3 records as like this
sl.no,EmpNo,EmpName
1,1001,abc
2,1002,xyz
3,1003,lkj
but the required output was i dont need the header record instead in that place i require current date, company name and recordcound such as
06/23/2009,CompanyName,4 (Record count includes number of records + Header Record i.e 3+1=4)
so i just want to know how to populate the record count, i have 3 differient nodes with same fields where in based on some condition any one of the nodes will execute how to count the records..
regards,
sai

hi,
u r understanding was correct and just want to know how the record count is handled
rgrds,
sai

Find record count between two time stamps

Hi All,
Problem: My custom table having a datetime column ;
I want to find records processed between two time stamps.
say: records between 2010-04-28 10:15:00 and 2010-04-28 12:30:00
Could you please help me in this.
Thanks in advance
Karri

use To_date function in where column on that Dat_col
WHERE date_col BETWEEN TO_DATE ('2010-04-28 10:15:00',
'mm-dd-rrrr hh24:mi:ss')
AND TO_DATE ('2010-04-28 12:30:00',
'mm-dd-rrrr hh24:mi:ss')

Generate target/out file with header record as Record Count ?

Hi Kareem, Please try the below approach. Pipeline 1: Load actual data(without header with record count) from source to target. Let say your file name is intermediate1.dat Pipeline 2: Take the target from pipeline 1 as source and create the header with count of source file using an aggregator. The filename of target for pipeline 2 will be your final file(header and detail data). Pipeline 3: Take the target of pipeline 1 again and do 1-to-1 load to the target file of second pipeline. In session properties, dont forget to tick the check box append if exists for the third pipeline target. There may be other simple approaches also. If you have no time in hand try the above approach. Let me know if you find any issues. Thanks,Deeshan.

Generate target/out file with header record as Record Count ? Out file:---------------------------Record Count :2000 Coulmn1, Column2...Data, data........

How to find Maximum Record Count in a particular section

Hi ALL
My report has 3 detail sections. I am showing different set of data in each section.
Example:
Details a
Details b
Details c
In the Details c section I have 5 columns.
I want to find out MAXIMUM RECORD COUNT in only Details c.
Actually I have done a running total on column1 field which gives me the records number in that section only. But I am not able to find the MAXIMUM record count.
Rtotal   Colum1   Column 2 Column3
1          Data1     Data2      Data3
2          Data1     Data2      Data3
3          Data1     Data2      Data3
4          Data1     Data2      Data3
Now I need the Maximum record count=4
Thanks.

If I have understood correctly you want to count the amount of records in each section.
In this case, go to "Insert > Summary", choose your ID field, and choose "Count" in "Calculate this summary:". You can put the summary in your group footer.
If you want the maximum (the highest number value in that column), you can choose "Maximum" instead of "Count".

Need a relative sequence (count of record) in an input file in ESB input fi

I am using ESB to pick up text files and pass them to a stored procedure. I have multiple variable length records in the file. I pass each record to the
output file adapter, along with the input filename. I would also like to get a relative record number for each record within the file. I use an xsd file
to convert the input to XML. The entire file is at the root level, each record is an element. Is there a way to get a relative count of each element?
Here is the xsd file:
<?xml version="1.0" encoding="UTF-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd"
targetNamespace="http://TargetNamespace.com/Filein"
xmlns:tns="http://TargetNamespace.com/Filein"
elementFormDefault="qualified"
attributeFormDefault="unqualified" nxsd:encoding="ASCII" nxsd:stream="chars" nxsd:version="NXSD">
<xsd:element name="Root-Element">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="C1" minOccurs="1" maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="C1" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="${eol}" nxsd:quotedBy=""">
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>


I need to map the relative record number to a field in the output.
Is there a function similar to the get filename that would return this value to me?
<xsl:value-of select='ehdr:getRequestHeader("/fhdr:InboundFileHeaderType/fhdr:fileName","fhdr=http://xmlns.oracle.com/pcbpel/adapter/file/;")'/>
I use this in the xsl file to get the file name .

does each record start with a new line? if it does a
simple wc -l can help you. If not, you can pre
process this XML file by first removing all the new
line characters using sed and then adding new line
characters after the end tag of the xml using sed and
finally doing a wc -l . this would be a simple shell
script. Not sure if you want to use Unix shell
scripting to achieve this.The sheer nature of XML means that it does not conform to any visual formatting. i.e. An xml file containing...
<employee><empno>3</empno></employee>
is identical in XML terms to...
<employee>
<empno>3</empno>
</employee>
which is also identical in XML terms to...
<employee>
<empno>
3
</empno>
</employee>
which is also identical in XML terms to...
<employee><empno>
3
</empno></employee>
etc.
You cannot rely on the number of CR's or CR/LF's in the file to count the records. Proper XML processing should be used for such a task.

Record Count of Lines in the Reciever Txt file

Hi,
I am Working with File to Idoc Scenarion. Where I have field "Record Count in the reciever. Where the total number of lines in the reciever file has to mapped with the field and the file to be recieved to the destination.
eg:Each Idoc that we receive, 2 lines are written out, one starts with BATHDR, the other is SECPTY.
There is also then 1 extra line, the header line. I looked at a bunch of iFiles, and it seems to be if we knew the total # of idocs for a given payment run, the total # of lines we would output is:
( ( #ofIDOCS * 2) + 1).
With this observation can you advise on the action to be taken for the out put.
Thanks ,
Manoj

Hi,
I am Working with Idoc to File Scenarion. Where I have field "Record Count in the reciever. Where the total number of lines in the reciever file has to mapped with the field and the file to be recieved to the destination.
eg:Each Idoc that we receive, 2 lines are written out, one starts with BATHDR, the other is SECPTY.
There is also then 1 extra line, the header line. I looked at a bunch of iFiles, and it seems to be if we knew the total # of idocs for a given payment run, the total # of lines we would output is:
( ( #ofIDOCS * 2) + 1).
With this observation can you advise on the action to be taken for the out put.
Thanks ,
Manoj

Getting OS File Record Count

Hi,
I've been looking for ways to get the record count(number of lines) of a txt file from a OS using a Oracle procedure without having to loop into the file line by line as on UTL_FILE.GET_LINE proc. If anyone has a one way method to get the number of lines of a OS file thru PLSQL without using loops, please let me know.
I appreciate your inputs on this matter.
Thiago Santana

Hi,
Simple example using external tables (I have an external directory called EXT_FILES where HR user can read and write; the test.txt file is also in this directory):
File: test.txt
1 hi
2
3 how
4
5 are
6
7
8
9
10you.External table creation:
CREATE TABLE test_ext (TEXT VARCHAR2(4000))
ORGANIZATION EXTERNAL (
TYPE oracle_loader
DEFAULT DIRECTORY EXT_FILES
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
NOBADFILE NODISCARDFILE NOLOGFILE
FIELDS TERMINATED BY '0x0A'
MISSING FIELD VALUES ARE NULL)
LOCATION ('test.txt'))
REJECT LIMIT unlimited;Test:
Connected to Oracle Database 10g Express Edition Release 10.2.0.1.0
Connected as hr
SQL> select count(*) from test_ext t;
COUNT(*)
10
SQL>Regards,
Edited by: Walter Fernández on May 11, 2009 11:35 PM - Adding line numbers to test file...

Need to calculate the number of records in an XML file

I need to find out how many records I have in an XML file. The file resides on a unix server and before doing anything with, i need to insert the record count into a separate table. Is there any function or procedure that can achieve this?

does each record start with a new line? if it does a
simple wc -l can help you. If not, you can pre
process this XML file by first removing all the new
line characters using sed and then adding new line
characters after the end tag of the xml using sed and
finally doing a wc -l . this would be a simple shell
script. Not sure if you want to use Unix shell
scripting to achieve this.The sheer nature of XML means that it does not conform to any visual formatting. i.e. An xml file containing...
<employee><empno>3</empno></employee>
is identical in XML terms to...
<employee>
<empno>3</empno>
</employee>
which is also identical in XML terms to...
<employee>
<empno>
3
</empno>
</employee>
which is also identical in XML terms to...
<employee><empno>
3
</empno></employee>
etc.
You cannot rely on the number of CR's or CR/LF's in the file to count the records. Proper XML processing should be used for such a task.

Restrictions in Oracle Server (table size, record count ...)

Hello,
can somebody tell me if there are any restrictions of table size, record count, file size (apart from operation system restrictions) in Oracle 8.1.7 and 7.3.4?
Or where can i find information? I couldn4t find anything in the generic documentation.
Thank you in advance,
Hubert Gilch
SEP Logistik AG
Ziegelstra_e 2, D-83629 Weyarn
Tel. +49 8020 905-214, Fax +49 8020 905-100
EMail: [email protected]

Hello,
if you are executing a DBMS_AQ.DEQUEUE and then perform a rollback in your code the counter RETRY_COUNT will not go up by 1.
You are only reversing your own AQ action. This counter will be used only internally to log unsuccessful dequeue actions.
Kind regards,
WoG

How to read some records from a text file into java(not all records)

hello,
how to read text files into java. i need only few records from the text file not all records at a time.
If any one knows plz reply me
my id is [email protected]

this snipet reads a text file line by line from line 1 to 3
try {
              FileReader fr = new FileReader(directory);
              BufferedReader br = new BufferedReader(fr);
              int counter = 0;
              while ((dbconn = br.readLine()) != null) {
                  switch(counter){
                      case 0:
                        status = dbconn;
                      break;
                      case 1:
                        userName = dbconn;
                      break;
                      case 2:
                        apword = dbconn;
                      break;
                  counter++;
              br.close();
    }catch(IOException e){
    }

How to count number of files on zfs filesystem

Hi all,
Is there a way to count the number of files on a zfs filesystem similar to how "df -o i /ufs_filesystm" works? I am looking for a way to do this without using find as I suspect there are millions of files on a zfs filesystem that is causing slow performance sometimes on a particular zfs file system
Thanks.

So I have finished 90% of my testing and I have accepted _df -t /filesystem | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'_ as acceptable in the absence of a known built in zfs method. My main conern was with the reduction of available files from the df -t output as more files were added. I used a one liner for loop to just create empty files to conserve on space used up so I would have a better chance of seeing what happens if the available files reached 0.
root@fj-sol11:/zfstest/dir4# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_5133680_
root@fj-sol11:/zfstest/dir4# df -t /zfstest
/zfstest (pool1 ): 7237508 blocks *7237508* files
total: 10257408 blocks 12372310 files
root@fj-sol11:/zfstest/dir4#
root@fj-sol11:/zfstest/dir7# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_6742772_
root@fj-sol11:/zfstest/dir7# df -t /zfstest
/zfstest (pool1 ): 6619533 blocks *6619533* files
total: 10257408 blocks 13362305 files
root@fj-sol11:/zfstest/dir7# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_7271716_
root@fj-sol11:/zfstest/dir7# df -t /zfstest
/zfstest (pool1 ): 6445809 blocks *6445809* files
total: 10257408 blocks 13717010 files
root@fj-sol11:/zfstest# df -t /zfstest | awk ' { if ( NR==1) F=$(NF-1) ; if ( NR==2) print $(NF-1) - F }'
_12359601_
root@fj-sol11:/zfstest# df -t /zfstest
/zfstest (pool1 ): 4494264 blocks *4494264* files
total: 10257408 blocks 16853865 files
I noticed the total files kept increasing and the creation of 4 millions files (4494264) after the above example was taking up more time than I had after already creating 12 million plus ( _12359601_ ) which took 2 days on a slow machine on and off (mostly on). If anyone has any idea of creating them quicker than "touch filename$loop" in a for loop let me know :)
In the end I decided to use a really small file system 100mb on a virtual machine to test what happens as the free files approached 0. Turns out if never does ... it somehow increased
bash-3.00# df -t /smalltest/
/smalltest (smalltest ): 31451 blocks *31451* files
total: 112640 blocks 278542 files
bash-3.00# pwd
/smalltest
bash-3.00# mkdir dir4
bash-3.00# cd dir4
bash-3.00# for arg in {1..47084}; do touch file$arg; done <--- I created 47084 files here, more that the free listed above ( *31451* )
bash-3.00# zfs list smalltest
NAME USED AVAIL REFER MOUNTPOINT
smalltest 47.3M 7.67M 46.9M /smalltest
bash-3.00# df -t /smalltest/
/smalltest (smalltest ): 15710 blocks *15710* files
total: 112640 blocks 309887 files
bash-3.00#
The other 10% of my testing will be to see what happens when I try to a find on 12 million plus files and try to pipe it to wc -l :)

Split records into Multiple csv files using a Threshold percentage

Hi Gurus,
I have a requirement to split the data into two csv file from a table using a threshold value(in Percentage) .
Assume that If my source select query of interface fetches 2000 records , I will provide a threshold value like 20%.
I need to generate a csv1 with 400 records(20% of 2000) and the rest of the records into another csv2.
For implementing this I am trying to use the following process.
1) Create a procedure with the select query to get the count of records.
Total Records count: select count(1) from source_table <Joins> <Lookups> <Conditions>;
2) Calculate the Record count to first CSV using the threshold_value.
CSV1_Count=Total records count /threshold_value
3) Create a view that fetches the CSV1_Count(400) records for CSV1 as follows.
Create view CSV1_view as select Col1,Col2,Col3 from source_table <Joins> <Lookups> <Conditions>
Where rownum<=CSV1_Count;
4) Generate CSV1 file using View 'CSV1_View'
5) Generate CSV2 File using the Interface with same select statement (with columns ) to generate a CSV.
select Col1,Col2,Col3 from source_table ST <Joins> <Lookups> <Conditions>
Left outer join (Select Col1 from CSV1_View ) CS on CS.Col1=ST.Col1 where CS.Col1 is null;
Which gives the Total records minus the CS1_View records.
The above process seems a bit complex and very simple . If any changes in my Interface I also need to change the procedure (counts the no:of records).
Please provide your comments and feedback about this and looking for your inputs for any new simple approach or fine tune the above approach.
Thanks,
Arjun

Arjun,
This are my thoughts and Lets do it in 3 Steps
Step 1. ODI Procedure
Drop table Temp_20 ;
Create table Temp_20 as select * from table where rownum < ( SELECT TRUNC( COUNT(1) /5) FROM TABLE ) .
[ ** This way iam fetching approx 20% of the table data and loading into Temp table . 1/5 th is 20% so i am dividing count by 5
I don't believe View will help you especially with RowNum as if you run the same query with rownum < N the rows order might differ . so Temp table is great ]
Step 2 . Use OdiSqlUnload with select columns from temp_20
Step 3 . Use again OdiSqlUnload with select columns from table where ( uk keys ) not in ( selecy uk_keys from temp_20)
[** this way you can pick the remaining 80% ** and the data will be not repeat itself across 20% and 80% , as might happen with view ]
what do you think ?

Record Count Issue in Search page ?

Hi Friends
I am trying to display record count on Search page and I have the following code in
The table has total of 62 Rows
AM : for testing purpose, i am printing in log
public void getISAGRecordCount()
OAViewObject vo = (OAViewObject)getXXG2_ELIM_ISAGSView1(); //getXXG2ISAGSearchVO1();
int fetchedRowCount = 0;
vo.setMaxFetchSize(-1);
vo.executeQuery();
fetchedRowCount = vo.getRowCount();
System.out.println("RowCount: "+ fetchedRowCount);
IN PFR of CO: I am calling by this way
am.invokeMethod("getISAGRecordCount");
the Issue is
for the first time, when i search Based on search criteria, I know it should dispaly 10 records but in the Log file i see the total Record Count of the Table Ex; 62
When i do the serach for the second time, based on search criteria, I know it should display 20 records, but the log file has 10 records ( previous search results)
If I do the search for 3rd time, I see 2nd search results in log
Any help in this issue, how to fix this?
Thanks
Ravi

Hi,
Please donot use getRowCount(), use getFetchedRowCount() and remove setMaxFetchSize(-1).
Hope it will resolve your issue.
Just for understanding of the concept:
getFetchedRowCount() returns the number of rows cached in the view object -- including programmatically inserted rows.
The getRowCount() implicitly execute a query against the database (if a query has not been executed yet)
Regards,
Reetesh Sharma

Find Record count in a file.

Similar Messages

Maybe you are looking for