Count Records from JDBC Sender

Hi Experts,
I have a requirement to process 80,000 records once a month from MS SQL Server to BW.
After tuning of server and assigning the XBTM queue for the scenario didnt help much and brings down the J2EE engine everytime.
Now i am thinking of processing 5000 records at a time in a BPM (in a while loop). But i somehow need to trap the overall number of records which is not always 80k.
set rowcount 5000 SELECT * table_name where XIProcessed is NULL (SELECT COUNT(*) from table_name)
and then sending an UPDATE of XIProcessed flag to get the next 5000 rows in the JDBC sender.
I somehow need to get the total number of records and the COUNT function will not help as i am getting 5000 rows at a time and not all.
Thanks

Hi Siva,
why do you attempt counting in the BPM and adding all this overhead? Can't you just do that on the Database directly? In Oracle this is fairly easy as you can use the pseudo column rownum to limit your results. A search for rownum and SQL Server shows, that it is not as easy on SQL Server but there are still fairly simple ways to do this.
Some sources:
http://blogs.x2line.com/al/articles/173.aspx
http://www.databasejournal.com/features/mssql/article.php/10894_2244821_2
http://www.mcse.ms/archive94-2005-3-1502710.html
From a performance standpoint i'm sure, it will work a lot better than any BPM into which you put 80000 records.
Regards
Christine

Similar Messages

How to count records from 2 tables and show in RDLC Report

hi all,
its being a one day searching for the solution but No Luck.
I have two SQL tables tblstudetail and tblfeereceiptdetail.
i just want to count records from both tables and show in RDLC report.
I tried SQl Query Like This:
select a.session, a.course,
Count(CASE a.ADstatus WHEN 'OK' THEN 1 ELSE 0 END ) AS Admission,
Count(CASE s .I_receiptstatus WHEN 'OK' THEN 1 ELSE 0 END) AS Feeprint
from
tblstudetail a
FULL join
tblfeereceiptdetail s on s.studentID = a.studentID
where a.session = '2015' AND s.Fsession = '2015' AND a.adcat = 'Regular'
GROUP BY a.session,a.course
ORDER by a.course
The result Show the Same Value in Both columns
Session    Course      Admission       FeeDetail
2015          B.A. I               275              275
2015          B.A. II              307             307
2015         B.A. III             255            255
2015          B.Sc. I             110             110
2015   B.Sc. II           105            105
2015          B.Sc. III            64               64
Actully I want to Count How many ADMISSION have been Taken(FROM tblstudetail) and How many FEE RECEIPT have been Print (From tblfeereceiptdetail).
please guide me for this as soon as possible.
thanks in advance...

I am counting 'OK' in both the table columns I.e 'ADstatus' in tblstudetail and 'feereceiptstatus' in tblfeereceiptdetail
please suggest me

MessageSplit from JDBC-Sender to JMS-Receiver

Dear all,
we are getting data in a resultset from JDBC-Sender and have to process each row as a own message. When we use message-splitting we are not able to generate plainText for the receiving JMS-channel. So we thought that we first have to generate xml and afterwards in JMS-Receiver-Adapter we have to use MessageTransformBean Struct2XML, but it does not work.
Howcan we achieve this?
Here is our mapping:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
     <xsl:output method="text" encoding="utf-8" />
     <xsl:template match="//resultset">
          <xsl:text disable-output-escaping="yes"><ns0:Messages xmlns:ns0="http://sap.com/xi/XI/SplitAndMerge"></xsl:text>
          <xsl:text disable-output-escaping="yes"><ns0:Message1></xsl:text>
          <xsl:for-each select="row">
                    <xsl:value-of select="NODENAME" />
          </xsl:for-each>
          <xsl:text disable-output-escaping="yes"></ns0:Message1></xsl:text>
          <xsl:text disable-output-escaping="yes"></ns0:Messages></xsl:text>
     </xsl:template>
</xsl:stylesheet>

Hi Abid,
we've tried it with MessageTransformBean, but unfortunately we were'nt able to generate the correct messages for JMS.
If we adapt our original mapping to that one below than we are able to send 2 messages to JMS having the xml-structure. Now, we just have to remove the xml structure. How do we have to configure the Module-Tab? Which one's are the exact parameters we have to use?
new MAPPING:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
     <xsl:output method="xml" encoding="utf-8" />
     <xsl:template match="//resultset">
          <xsl:text disable-output-escaping="yes"><ns0:Messages xmlns:ns0="http://sap.com/xi/XI/SplitAndMerge"></xsl:text>
          <xsl:text disable-output-escaping="yes"><ns0:Message1></xsl:text>
          <xsl:for-each select="row">
               <MSG>
                    <xsl:value-of select="NODENAME" />
               </MSG>
          </xsl:for-each>
          <xsl:text disable-output-escaping="yes"></ns0:Message1></xsl:text>
          <xsl:text disable-output-escaping="yes"></ns0:Messages></xsl:text>
     </xsl:template>
</xsl:stylesheet>
resulting messages:
<?xml version="1.0" encoding="utf-8" ?><MSG>VALUE_1</MSG>
<?xml version="1.0" encoding="utf-8" ?><MSG>VALUE_2</MSG>
Kind regards, Stefan

Stored Procedure call from JDBC sender for Oracle Database

Hi,
I have a requirement to call stored procedure using JDBC sender adapter for retriving data from an Oracle database.
I need to execute first stored procedure call from Querry SQL statement and then trigger second stored procedure call for confirming the succesful retrival of those records from the update SQL statement.
Querries:
1. Can we trigger stored procedure from Update statement of JDBC sender.
2. Can we call stored procedure from Querry statement, since I have read on other sdn threads that stored procedure calls on Oracle database are not allowed. If not possible to have "Execute stored procedure" would the function call (select * from table(function name)) work same as stored procedure.
3. In a JDBC sender the Querry statement and Update statement are executed in same session call and same database connection. What happens if the querry statement was not succesful, would the update be still triggered.
Please note PI does not have direct access to main table and hence the need to have separate stored procedure calls.
The PI version is PI 7.11 sp4.
Appreciate your inputs.
Thanks.
Siddhesh S.Tawate

>1. Can we trigger stored procedure from Update statement of JDBC sender.
I think not possible using update statement.
> Can we call stored procedure from Querry statement, since I have read on other sdn threads that stored procedure calls on Oracle database are not allowed. If not possible to have "Execute stored procedure" would the function call (select * from table(function name)) work same as stored procedure.
Yes using select query you can call stored procedure.
>. In a JDBC sender the Querry statement and Update statement are executed in same session call and same database connection. What happens if the querry statement was not succesful, would the update be still triggered.
No select and update handles in the same session using the same thread so if one transaction fails update should not proceed.

Processing of large number of records using JDBC Sender Channel

Hi experts,
We have a JDBC-File scenario where in the tables contain about 500K records on an average.
I used multimapping to generate a flat file for every 10K. The problem is ..when I start the JDBC Sender CC, the memory goes up and the J2EE engine restarts. In the Sender CC, I gave the Disconnect from the database option too. The query is SELECT * from TABLE and the Update statement is <TEST>. Please help me out how to solve this.
Regards.

Hi
Use the below query,
// Oracle
SELECT Statement : select colname from tblname where rownum<=1000 ;
// MSSQL
select top 1000 colname from tblname
Regards
Ramg

Counting Record from OPEN FOR statement

Hi,
Is there a way to count the number of record produce by OPEN ... FOR statement...?
I tried using %ROWCOUNT. It doesn't seems to be working. It returned me 0 though when i printed the cursor, i had 7 records.
OPEN p_cursor FOR sqlstatement;
pnum_record := p_cursor%ROWCOUNT;Thank you everyone!

Dave,
My procedure is supposed to return back the records to the caller (application)
Below is the spec
PROCEDURE query_licensetype (
      p_cursor     IN OUT   cursortype,
      p_num_page   IN OUT   PLS_INTEGER,
      p_string              VARCHAR2 DEFAULT NULL,
      p_pageno              PLS_INTEGER DEFAULT 1,
      p_pagerow             PLS_INTEGER DEFAULT 20,
      p_asc                 PLS_INTEGER DEFAULT 1
   )And a piece of the code that open it
OPEN p_cursor FOR vsql USING p_string, p_string, vstart_rec, vend_rec;I am not sure how to do what you have suggested.
Can you show me how to FETCH all the record from p_cursor and still returning back the record to the caller?
Thanks Dave!

Oracle DB 10g - Count records from a subquery is to slow. (after tuning)

Hi everybody:
I have the following problem.
1.- I have a transactional table with 11 millions of records.
2.- The user asked for a few reports, to know some sumarized data for business decisions.
3.- Initialy the query was developed in a development environment, with a subset smaller than the actual data mentioned in label 1.
4.- When the report was delivered to the end user, the report never return data.
5.- at this point, we performed tuning, adding indexes, re-writing the query, using hints, etc. and the following scenario is ocurring:
a) the query without the count, before the tuning was taking about 3 to 5 minutes, and returned aproximatelly 332,000 records-.
b) the numer of records was counted and this query takes aproximatelly 15 to 23 minutes.
c) after the tuning, the raw data, returns in 1 second, we used some b-tree indexes, some FBI (because the report needs to filter by to_char functions).
that time is ok for us, 1 second is a great time in comparison with the 3 to five minutes.
d) the funny thing, is that when we add a simple count(1), count(x) or wathever flavor of count, the count takes about 3 minutes to return, with the 332,00
records. is better than de 15 minutes of course, but we dont need count(1), we need to use group by, order by, etc, and will increase the time of query.
6.- Another thing is happening, if I use count(1) with a transactional table with 600,000 records, without filtering, the count returns in les than 2 seconds. Even if the data is more than the result of my query defined in label 5.c, and that query returns in 1 second.
Please help me with this, I know that maybe is something that i'm not considering on tuning. Or if there is a way to run this count query faster, please let me know.
This is the query:
WITH historial_depurado_v AS (
select serie, identificador,
       cc_form_cc_tifo_tipo_forma
      ,cc_form_serie
      ,cc_form_folio
      ,estatus_nuevo
      ,to_char(fecha,'MM') Mes
      ,to_char(fecha,'YYYY') Anio -- the table has a FBI for this.
      ,get_recaudacion_x_forma (hifo.CC_FORM_CC_TIFO_TIPO_FORMA
                               ,hifo.CC_FORM_SERIE
                               ,hifo.CC_FORM_FOLIO) Recaudacion -- function for description.
from cc_historial_formas hifo
where not exists (select 1
                     from ve_tipos_placas tipl
                    where tipl.cc_tifo_tipo_forma = hifo.cc_form_cc_tifo_tipo_forma)
SELECT serie
FROM historial_depurado_v hide
WHERE Anio = '2009' -- using the function based index.
   AND Mes = '01'
   AND Estatus_nuevo = 'UT'
-- returns in 1 second, but if I use count(serie) takes 3 to 5 minutes, and still is not complete, I need to use group by some fields returned by the firs subset.
-- if I count a table with more records, returns in 2 seconds.

alopez wrote:
WHERE Anio = '2009' -- using the function based index.
AND Mes = '01'
AND Estatus_nuevo = 'UT'
Also, i'm not sure if your internal comment is correct, but if you have a function based index on any of those columns, it will ONLY work when you apply the exact function as defined in the FBI (you aren't using any functions on ANY of your columns).
An example.
create table use_the_fbi
   column1 date
insert into use_the_fbi
select
   trunc(sysdate, 'dd') - level
from dual connect by level <= 10000;
create index use_the_fbi_FBI on use_the_fbi (to_char(column1, 'YYYYMM') );
exec dbms_stats.gather_table_stats( user, 'USE_THE_FBI', cascade=> true, method_opt=> 'for all hidden columns');Let's try without querying in the manner in which we defined the index.
explain plan for
select *
from use_the_fbi
4 where column1 = '201006';
Explained.
Elapsed: 00:00:00.01
TUBBY_TUBBZ?@xplain
PLAN_TABLE_OUTPUT
Plan hash value: 1722805582
| Id | Operation         | Name        | Rows | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT |             |   100 |   900 |     7   (0)| 00:00:01 |
|* 1 | TABLE ACCESS FULL| USE_THE_FBI |   100 |   900 |     7   (0)| 00:00:01 |
Query Block Name / Object Alias (identified by operation id):
   1 - SEL$1 / USE_THE_FBI@SEL$1
Predicate Information (identified by operation id):
   1 - filter("COLUMN1"='201006')
Column Projection Information (identified by operation id):
   1 - "COLUMN1"[DATE,7]
23 rows selected.
Elapsed: 00:00:00.02Sad Christmas .. no index love.
Now, query as we defined the index
explain plan for
select *
from use_the_fbi
where to_char(column1, 'YYYYMM') = '201006';
Explained.
Elapsed: 00:00:00.01
TUBBY_TUBBZ?TUBBY_TUBBZ?@xplain
PLAN_TABLE_OUTPUT
Plan hash value: 268331390
| Id | Operation                   | Name            | Rows | Bytes | Cost (%CPU)| Time     |
|   0 | SELECT STATEMENT            |                 |    30 |   450 |     2   (0)| 00:00:01 |
|   1 | TABLE ACCESS BY INDEX ROWID| USE_THE_FBI     |    30 |   450 |     2   (0)| 00:00:01 |
|* 2 |   INDEX RANGE SCAN          | USE_THE_FBI_FBI |    30 |       |     1   (0)| 00:00:01 |
Query Block Name / Object Alias (identified by operation id):
   1 - SEL$1 / USE_THE_FBI@SEL$1
   2 - SEL$1 / USE_THE_FBI@SEL$1
Predicate Information (identified by operation id):
   2 - access(TO_CHAR(INTERNAL_FUNCTION("COLUMN1"),'YYYYMM')='201006')
Column Projection Information (identified by operation id):
   1 - "USE_THE_FBI"."COLUMN1"[DATE,7]
   2 - "USE_THE_FBI".ROWID[ROWID,10], TO_CHAR(INTERNAL_FUNCTION("COLUMN1"),'YYYYMM')[VA
       RCHAR2,6]
27 rows selected.
Elapsed: 00:00:00.03
TUBBY_TUBBZ?Happy times.

Getting the un updated records from receiver JDBC

Hi Friends,
I am doing the Proxy - jdbc integration. Consider my proxy sends 10 records to JDBC to update. Consider bcoz of some reason the jdbc updates only 8 records i need to get the 2 records back to Proxy which is not updated in the DB.
Should i need to use the Proxy and JDBC channel as syn or seperate integration should be built. I have seen some documents, that we can get the count of the updated records from jdbc receiver, but i dont need the count i need the record which is not been updated.
How can i do it. please give ur inputs.
thanks
Prem

Unless you are using SP on JDBC side and explicitly build a record set to send back the data it might not be possible. I am not sure if a custom record set could be returned in JDBC sync receive adapter. You can realize this by updating some other table on JDBC side and then polling that table to update ecc back.
VJ

Significance of Document Name in JDBC Sender

hi folks,
tell me what exactly mean by the Document Name parameter in the JDBC Sender adapter? where does that plays?
thanks
nikhil bos

> tell me what exactly mean by the Document Name
> parameter in the JDBC Sender adapter? where does
> that plays?
If you created MT_JDBC_Sender (for Example)
Go to IR and Test your mapping..then take the input message from there and paste it to the Notepad...you will find the Root Element will be
<<b>MT_Sender_JDBC</b> >
<row>
<>
</MT_Sender_JDBC >
now when you configured the JDBC adapter then by defalut the Document Name is <b>resultset</b>
so XML from From JDBC sender adapter will come in this format.:
<<b>resultset</b> >
<row>
<>
</<b>resultset</b> >
So coming XML does not match with the structure that u define..so finally need to overide the name of the defult document namespace so that it will not fail at runtime:-)
Thanks
Farooq.

To create 5 different files in target for each record in the sender file

Hi SapAll,
i have got a a requirement where pi need to create 5 different files with data of each record from the sender side.
Its an file to file scenario
suppose there are 5 rows in a file , then in target side i need to create 5 diff with each one record in it.
Can we do it without using 3 comm channels
Thanks&regards,
Sai

Hey,
As a more generic solution, can you try using MultiMapping. It is supposed to generate multiple instances of messages on receiver side.
/people/jin.shin/blog/2006/02/07/multi-mapping-without-bpm--yes-it146s-possible
You just need to put a simple logic in Message Mapping to generate new Node for every occurence of source field.
Thanks
Aamir

Sender JDBC Adapter : Fetch records from multiple tables

Hi Friends,
I am using sender JDBC adapter to select few records from DB2 database table.
This scenario is working fine.
Now my requirement is to fetch records from 3 tables. These table are independent to each other. There is no primary key or foreign key.
Please let me know how to write the sql in sender JDBC adapter to fetch records from these 3 tables.
Thanks,
Sandeep Maurya

hi sandeep...
if the tables are completely independent and do not share any primary / foreign key relation ship...
why dont u think towards creating a seperate interface for each of them..
or if u still want to select from multiple table at once..the best way would be to write a stored procedure on the sender side which do all the fetching n processing and pass the final resultset to PI
or u can think towards fetching the data from 1 table and then in UDF do lookup from other tables..which again is tricky and performace intensive

JDBC Sender Adapter - Records sent one at a time

Hello everyone,
Here's a summary of my scenario JDBC > XI > RFC.
My select statement (JDBC) looks like this:
SELECT * FROM BATCH WHERE STATUS='Y'
The problem is, whenever it returns multiple rows, the RFC can only accept one record at a time (1...1), so I need to pick up the records from the DB and send them one at a time.
Currently, this is how the output XML looks like:
<?xml version="1.0" encoding="utf-8" ?>
<ns:SQL_RESP_MT xmlns:ns="http://www.pharmaindustries.com">
<row>
<number>200000000472</number>
<status>Y</status>
</row>
<row>
<number>200000000473</number>
<status>Y</status>
</row>
<row>
<number>200000000474</number>
<status>Y</status>
</row>
</ns:SQL_RESP_MT>
Instead, I need each row to be picked up and sent one at a time.
<?xml version="1.0" encoding="utf-8" ?>
<ns:SQL_RESP_MT xmlns:ns="http://www.pharmaindustries.com">
<row>
<number>200000000472</number>
<status>Y</status>
</ns:SQL_RESP_MT>
<?xml version="1.0" encoding="utf-8" ?>
<ns:SQL_RESP_MT xmlns:ns="http://www.pharmaindustries.com">
<number>200000000473</number>
<status>Y</status>
</row>
</ns:SQL_RESP_MT>
<?xml version="1.0" encoding="utf-8" ?>
<ns:SQL_RESP_MT xmlns:ns="http://www.pharmaindustries.com">
<number>200000000474</number>
<status>Y</status>
</row>
</ns:SQL_RESP_MT>
Does anyone know the changes I need to make in my current SELECT statement or the JDBC Sender Adapter to make this possible.
Thanks in advance!
Glenn

Thank you all for your replies. I'll check out your link in a bit, Phani. Thanks!
A SpliByValue won't work since the BAPI Header is (1...1).
Luckily, during testing, I found out that the BAPI (BAPI_PRODORDCONF_CREATE_TT) handles multiple lines in one of the subelements of the header so that was where I mapped the Rows.
I might need to trigger multiple BAPIs in the future, so I'll take a good look at your suggestions, but for now, I'm marking this question answered.
Warm regards,
Glenn

JDBC Sender -Not selecting records

Hi all
I have set up a JDBC sender. The communication channel monitor shows that "Polling interval started" but It is not retrieving data from the JDBC table using a select statement.
This is a sql select statement.
SELECT * FROM table
there is also a corresponding update statement.
clearly ,XI is unable to pick up data from the table.
if i change the name of the table , it does throw an SQL exception that the object doesnt exist. I have also checked with other tables and it is fetching data correctly. The table under problem does contain a row of data in ORacle side.
Why doesnt XI select the records of the table?
Any help will be highly appreciated
thanks
krishna

Hi
yes, the table does contain data
and the sql stmnt is actually
select * from table where flag= 'true' ( i also tried it without the where condition)
here are the other parameters
QOS : Exactly Once
Poll interval: 6
Document name: Resultset
Update sql stmnt : UPDATE TBL SET FLAG = 'FALSE' WHERE FLAG = 'TRUE'
hope this helps
regards
krishna

JDBC sender: after read insert records into another table

Hi all,
I send records from a SQL view (not a table) to RFC via proxy method.
After reading a record from the view I would like to move this record to a database table called CHECKED. Can anyone provide me a solution for this or is this impossible. (there are 6 fields that have to be inserted in DB CHECKED after reading: admin, bukrs, dagbknr, bkstnr, datum, regel)
Can I use an INSERT statement in my Update SQL Statement with variables or do I have to create an SP?
Hope someone can help me out of here...
Thanks in advance!
Wouter.

ur strucutre would of:
<root>
<Satement1>
<TableName action=INSERT>
<table>checked</table>
<access>
<admin>val</admin>
<bukrs>val</bukrs>
<dagbknr>val</dagbknr>
<bkstnr>val</bkstnr>
<datum>val</datum>
<regel>val</regel>
</access>
</dbTableName>
</StatementName1>
</root>
or u can use a SQL DML which wud pose a strucutre similar to
<root>
<stmt>
    <Customers action="SQL_DML">
      <access> INSERT CHECKED SET admin=$admin$, bukrs=$bukrs$', dagbknr=$dagbknr$, bkstnr=$bkstnr$', datum=$datum$, regel.=$regel.$'
      </access>
    </Customers>
</stmt>
</root>
for more inputs refer to <a href="/people/sap.user72/blog/2005/06/01/file-to-jdbc-adapter-using-sap-xi-30 link on jdbc</a>

JDBC Sender - Different number of records selected and updated.

Hi people,
We have a JDBC -> Abap proxy scenario. The JDBC sender is pooling an Oracle database to retrieve data from a table X, each 30 minutes. The select and update statements in jdbc sender are below
SELECT FIELD1, FIELD2, FIELD3 FROM MY_TABLE WHERE STATUS = 1
UPDATE MY_TABLE SET STATUS = 2 WHERE STATUS = 1
Sometimes the message sent to Abap proxy has, for example, 400 records. Looking at runtime workbench, message monitoring, for the same message there is a log like this
Channel SENDER_JDBC_CHANNEL: Query executed successfully. Start update
Channel SENDER_JDBC_CHANNEL: 510 row(s) updated successfully
Someone has already experienced something like this? How can I handle this to guarantee to update only those read records?
regards.
roberti

Hi All,
Even we are facing the same problem.
In our scenario, receiver is SAPR3. (IDOC)
Will this parameter serialization work in our case?
1. SELECT XBLNR, WERKS, MATNR, MDV01, BACKFLQUANT, STATUS, SAPTIMESTAMP, PITSTIMESTAMP, PMTIMESTAMP, BATCH FROM PMBPITS.PITS_UNITY WHERE STATUS = '01' and rownum<200 .
2. UPDATE PMBPITS.PITS_UNITY SET STATUS = '02' , SAPTIMESTAMP = sysdate WHERE STATUS = '01' and rownum<200 ( currently the value is rownum < 5 )
Thanks!!
Regards
Gouri

Count Records from JDBC Sender

Similar Messages

Maybe you are looking for