Sending Large amount of data (250 K +) from Oracle to flex client

I have an oracle database with more than 250K rows of data that needs to be sent to the AdvancedDataGrid and Flex charts for runtime analysis. What would be and ideal solution to implement this?

I would say paging would be the way to go, downloading 250K rows to a client is not sensible. Any analysis that may be required should be performed server-side. Dumping 250K data-points into a chart is also unlikely to be performant or necessary.
You get paging with LCDS, and the new FlashBuilder 4 code generation features also support paging.

Similar Messages

Sending large amounts of data spontaneously

In my normal experience with the internet connection, the bits of data sent is about 50 to 80% of that received, but occasionally Firefox starts transmitting large amounts of data spontaneously; what it is, I don't know and where it's going to, I don't know. For example, today the AT&T status screen showed about 19 MB received and about 10 MB sent after about an hour of on-line time. A few minutes later, I looked down at the status screen and it showed 19.5 MB received and 133.9 MB sent, and the number was steadily increasing. Just before I went on line today, I ran a complete scan of the computer with McAfee and it reported nothing needing attention. I ran the scan because a similar effusion of sending data spontaneously had happened yesterday. When I noticed the data pouring out today, I closed Firefox and it stopped. When I opened Firefox right afterward, the transmission of data from did not recommence. My first thought was that my computer had been captured by the bad guys and now I was a robot, but McAfee says not to worry. But should I worry anyway? What's going on, or that not having a good answer now, how can I find out what's going on? And how can I make it stop, unless I'm seeing some kind of maintenance operation that Mozilla or Microsoft is subjecting me to?

Instead of using URLConnection open a Socket to the server port (80 probably) send a POST http request followed by the data, you may then (optional) recieve data from the server to check that the servlet is ok, this is the same protocol as URLConnection, but you have control over when the data is actually sent...
Socket sock=new Socket(getHost(),80);
DataOutputStream dos=new DataOutputStream(sock.getOutputStream());
dos.writeBytes("POST servletname\r\n");
dos.writeBytes("Content-type: text/plain\r\n"); //optional, but good if you know
dos.writeBytes("Content-length: "+lengthOfData+"\r\n") //again, optional, but good if you can know it without caching the data first
dos.writeBytes("\r\n"); // gotta have a blank line before the data
// send data now
DataInputStream=new DataInputStream(sock.getInputStream()); //optional if you want to recieve
// recieve any feedback from servlet if you want
dis.close();
dos.close();
sock.close();im guessing that URLConnection caches the data so it can fill in "Content-length"

Send large amounts of data

Hello everyone,
I made an applet that receives data, signs and returns the signed data. When the amount of data is too big, I break it into blocks of 255 bytes and use the method Signature.update.
OK, this is working fine, but perfomarnce is poor due to large amount of blocks. Is it possible to increase the size of the blocks?
Thanks.

Hi,
You cannot change the block size but you can change what you send in.
You may get better performance by sending multiples of your hash function block size as the card will not have to do internal buffering.
You could also do as much of the work in your host application as possible and then just send in the data that you need to operate on with the private key. Generating the hash of the message does not require a private key so can be done in your host. You then send the result of the hash to the card to be encrypted with the private key. This will be the fastest method.
Cheers,
Shane

Error when exporting large amount of data to Excel from Apex4

Hi,
I'm trying to export over 30,000 lines of data from a report in Apex 4 to an Excel spreadsheet, this is not using a csv file.
It appears to be working and then I get 'No Response from Application Web Server'. The report works fine when exporting smaller amounts of data.
We have just upgraded the application to Apex 4 from Apex 3, where it worked without any problem.
Has anyone else had this problem? We were wondering if it was a parameter in Apex4 that needs to be set.
We are using Application Express 4.1.1.00.23 on Oracle 11g.
Any help would be appreciated.
Thanks
Sue

Hi,
>
I'm trying to export over 30,000 lines of data from a report in Apex 4 to an Excel spreadsheet, this is not using a csv file.
>
How? Application Builder > Data Workshop? Apex Page Process? (Packaged) procedure?
>
It appears to be working and then I get 'No Response from Application Web Server'. The report works fine when exporting smaller amounts of data.
We have just upgraded the application to Apex 4 from Apex 3, where it worked without any problem.
>
Have you changed your webserver in the process? Say moved from OHS to ApexListener?
>
Has anyone else had this problem? We were wondering if it was a parameter in Apex4 that needs to be set.
We are using Application Express 4.1.1.00.23 on Oracle 11g.
Any help would be appreciated.

How to Send and Receive Large Amounts of Data to and from a Web Service

Hi All,
My requirement is: .Net Web service should receive a file and services do some modification in that file and return the file. I need to write client (using Java) to invoke that web service. Please help in writing java client code for accessing that service. Suggest if any changes required in Web service code also.
My .Net web Service web method code:
[WebMethod]
public byte [] ByteEcho(byte[] data) {
- Some modification code -
return data;
}

that will work fine for small files. it will potentially cause OutOfMemoryErrors on client and/or server for large files. if you want to send/recieve large files, you need to stream them.
also, be aware that you cannot send raw byte[] via xml, you will need to encode the data using some sort of binary -> text encoding like Base64 encoding.

DSS problems when publishing large amount of data fast

Has anyone experienced problems when sending large amounts of data using the DSS. I have approximately 130 to 150 items that I send through the DSS to communicate between different parts of my application.
There are several loops publishing data. One publishes approximately 50 items in a rate of 50ms, another about 40 items with 100ms publishing rate.
I send a command to a subprogram (125ms) that reads and publishes the answer on a DSS URL (app 125 ms). So that is one item on DSS for about 250ms. But this data is not seen on my man GUI window that reads the DSS URL.
My questions are
1. Is there any limit in speed (frequency) for data publishing in DSS?
2. Can DSS be unstable if loaded to much?
3. Can I lose/miss data in any situation?
4. In the DSS Manager I have doubled the MaxItems and MaxConnections. How will this affect my system?
5. When I run my full application I have experienced the following error Fatal Internal Error : ”memory.ccp” , line 638. Can this be a result of my large application and the heavy load on DSS? (se attached picture)
Regards
Idriz Zogaj
Idriz "Minnet" Zogaj, M.Sc. Engineering Physics
Memory Profesional
direct: +46 (0) - 734 32 00 10
http://www.zogaj.se

LuI wrote:
>
> Hi all,
>
> I am frustrated on VISA serial comm. It looks so neat and its
> fantastic what it supposes to do for a develloper, but sometimes one
> runs into trouble very deep.
> I have an app where I have to read large amounts of data streamed by
> 13 µCs at 230kBaud. (They do not necessarily need to stream all at the
> same time.)
> I use either a Moxa multiport adapter C320 with 16 serial ports or -
> for test purposes - a Keyspan serial-2-USB adapter with 4 serial
> ports.
Does it work better if you use the serial port(s) on your motherboard?
If so, then get a better serial adapter. If not, look more closely at
VISA.
Some programs have some issues on serial adapters but run fine on a
regular serial port. We've had that problem recent
ly.
Best, Mark

Pull large amounts of data using odata, client API and so takes a long time in project server 2013

We are trying to pull large amounts of data in project server 2013 using both client API and odata calls, but it seem to take a long time. How is this done
In project server 2010 we did this creating SQL views in both the reporting database and for list creating a view in the content database. Our IT dept is saying we can't do this anymore. How does a view in Project database or content database create issues?
As long as we don't add a field in the table. So how's one to do this with creating a view?

Hello,
If you are using Project Server 2013 on premise I would recommend using T-SQL against the dbo. schema in the Project Web Database for your reports, this will be far quicker that the APIs. You can create custom objects in the dbo. schema, see the link below:
https://msdn.microsoft.com/en-us/library/office/ee767687.aspx#pj15_Architecture_DAL
It is not supported to query the SharePoint content database directly with T-SQL or add any custom objects to the content database.
Paul
Paul Mather | Twitter |
http://pwmather.wordpress.com | CPS |
MVP | Downloads

How to send large amount of XML data in one CLOB variable

Hi,
I am sending large amount of XML data to TCP/IP port in one CLOB variable.
My requirement is to send the whole data in one go in one CLOB variable.
But that CLOB variable is not sufficient to hold all the data.
Please suggest some solution.
Thanks in advance

Hi Here is my code:
CREATE OR REPLACE PACKAGE BODY APPS.XXMB_WIP_PROD_TAG_DOOR_PKG
AS
PROCEDURE xxmb_get_xml_data_1270 (
-- errbuf OUT VARCHAR2,
-- retcode OUT NUMBER,
p_org IN VARCHAR2,
p_limit_to_global IN VARCHAR2,
p_label IN VARCHAR2,
p_printer IN VARCHAR2,
p_quantity IN VARCHAR2,
p_print_method IN VARCHAR2,
p_enable_release IN VARCHAR2,
p_enable_serial_no IN VARCHAR2,
p_release IN VARCHAR2,
p_rep_group IN VARCHAR2,
p_cart_type IN VARCHAR2,
p_cart_no_from IN VARCHAR2,
p_cart_no_to IN VARCHAR2,
p_serial_no IN VARCHAR2
AS
CURSOR c_xml_data_door (
p_org IN VARCHAR2,
p_label IN VARCHAR2,
p_printer IN VARCHAR2,
p_quantity IN VARCHAR2,
p_print_method IN VARCHAR2,
p_rep_group IN VARCHAR2,
p_release IN VARCHAR2,
p_cart_type IN VARCHAR2,
p_cart_no_from IN VARCHAR2,
p_cart_no_to IN VARCHAR2,
p_serial_no IN VARCHAR2
IS
SELECT xxasa.item_id AS item_id, xcs.serial_number AS serial_number,xxcpf.cart_type,xcs.destination_cart_num cart,xcs.destination_slot_num slot
CURSOR c_product_detail (
l_product IN NUMBER,
l_serial_num IN VARCHAR2,
p_limit_to_global IN VARCHAR2
IS
SELECT xcra_specie.reference_id AS reference_id,
xcra_ege.attribute_value AS ege, xcs.item_id AS item_id,
AND msib.inventory_item_id = l_product
and xcs.organization_id = nvl(p_org, xcs.organization_id)
AND xcs.serial_number = NVL (l_serial_num, xcs.serial_number);
/*-------------------------------------------------------+
| Cursor to fetch the data for special Message Label |
+-------------------------------------------------------*/
CURSOR c_count (p_item_id IN NUMBER)
IS
SELECT xcrav.attribute_value, xcs.serial_number, xcs.cabinet_number
FROM xxmb_czmfg_ref_attributes xcrav,
cz_config_attributes cca,
AND msib.organization_id = xcs.organization_id
AND msib.inventory_item_id = xcs.item_id;
/*--------------------------+
| Common variables |
+--------------------------*/
v_limit_to_global VARCHAR2 (100);
l_label_count NUMBER := 1;
total_rec NUMBER;
l_rewrite VARCHAR2 (1) := 'N';
l_file_count NUMBER := 1;
l_separate_line VARCHAR2 (10);
BEGIN
fnd_profile.get ('WMS_LABEL_OUTPUT_DIRECTORY', l_output_dir);
fnd_profile.get ('WMS_LABEL_FILE_PREFIX', l_output_file_prefix);
l_request_id := apps.fnd_global.conc_request_id;
l_output_file_name :=
l_output_file_prefix || l_request_id || l_file_end;
l_dir_seperator := '/';
IF (INSTR (l_output_dir, l_dir_seperator) = 0)
THEN
l_dir_seperator := '\';
END IF;
v_label := p_label;
v_printer := p_printer;
v_quantity := p_quantity;
V_LIMIT_TO_GLOBAL := P_LIMIT_TO_GLOBAL;
L_XML_CONTENT := '<?xml version="1.0" encoding="UTF-8" ?>';
L_XML_CONTENT := L_XML_CONTENT || '<!DOCTYPE labels SYSTEM "label.dtd">';
L_XML_CONTENT := L_XML_CONTENT || '<labels>';
FOR r_xml_data_door IN c_xml_data_door (p_org,
p_label,
p_printer,
p_quantity,
p_print_method,
p_rep_group,
p_release,
p_cart_type,
p_cart_no_from,
p_cart_no_to,
p_serial_no
LOOP
-- dbms_output.put_line ( 1 );
FOR r_product_detail IN
c_product_detail (r_xml_data_door.item_id,
r_xml_data_door.serial_number,
v_limit_to_global
LOOP
-- dbms_output.put_line ( 2 );
-- l_xml_content := '<?xml version="1.0" encoding="UTF-8" ?>';
-- l_xml_content := l_xml_content || '<!DOCTYPE labels SYSTEM "label.dtd">';
-- l_xml_content := l_xml_content || '<labels>';
fnd_file.put_line (fnd_file.LOG, 'label cnt: ' || l_label_count);
dbms_output.put_line (l_label_count);
L_XML_CONTENT := L_XML_CONTENT || '<label _FORMAT='
|| '"'
|| 'lib://FRD/'
|| v_label
|| '"'
|| ' _PRINTERNAME='
|| '"'
|| v_printer
|| '"'
|| ' _QUANTITY='
|| '"'
|| v_quantity
|| '"'
|| '>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Color">'
|| R_PRODUCT_DETAIL.COLOR
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT ||'<variable name= "Model">'
|| R_PRODUCT_DETAIL.model
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Build_Date">'
|| R_PRODUCT_DETAIL.BUILD_DATE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Assy_Cart">'
|| R_PRODUCT_DETAIL.ASSY_CART
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Assy_Slot">'
|| R_PRODUCT_DETAIL.ASSY_SLOT
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Assy_Line">'
|| R_PRODUCT_DETAIL.ASSY_LINE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Finish_Cart">'
|| R_PRODUCT_DETAIL.FINISH_CART
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Finish_Slot">'
|| R_PRODUCT_DETAIL.FINISH_SLOT
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Serial_Number">'
|| R_PRODUCT_DETAIL.SERIAL_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Serial_Number_Barcode">'
|| R_PRODUCT_DETAIL.SERIAL_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Specie">'
|| R_PRODUCT_DETAIL.SPECIE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT ||'<variable name= "Truck_Group">'
|| R_PRODUCT_DETAIL.TRUCK_GROUP
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Label_Sequence_No">'
|| L_LABEL_COUNT
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT ||'<variable name= "WIP_Cart">'
|| R_PRODUCT_DETAIL.WIP_CART
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "WIP_Slot">'
|| R_PRODUCT_DETAIL.WIP_SLOT
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Cabinet_Sequence_No">'
|| R_PRODUCT_DETAIL.CAB_SEQ_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "RAW_PART_NO">'
|| R_PRODUCT_DETAIL.RAW_PART_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "JC">'
|| R_PRODUCT_DETAIL.JC
|| '</variable>' ;
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "QC">'
|| R_PRODUCT_DETAIL.QC
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Thickness">'
|| R_PRODUCT_DETAIL.THICKNESS
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Width">'
|| R_PRODUCT_DETAIL.width
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Length">'
|| R_PRODUCT_DETAIL.length
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Overlay">'
|| R_PRODUCT_DETAIL.OVERLAY
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Options">'
|| R_PRODUCT_DETAIL.OPTIONS
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Stop">'
|| R_PRODUCT_DETAIL.stop
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Profile_No">'
|| R_PRODUCT_DETAIL.PROFILE_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Door_Style">'
|| R_PRODUCT_DETAIL.DOOR_STYLE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Glaze">'
|| R_PRODUCT_DETAIL.GLAZE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Shape">'
|| R_PRODUCT_DETAIL.SHAPE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Glass">'
|| R_PRODUCT_DETAIL.GLASS
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Hinge_Side">'
|| R_PRODUCT_DETAIL.HINGE_SIDE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Hinge_Type">'
|| R_PRODUCT_DETAIL.HINGE_TYPE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "EGE">'
|| R_PRODUCT_DETAIL.EGE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Door_Style_Code">'
|| R_PRODUCT_DETAIL.DOOR_STYLE_CODE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Finish_Technique">'
|| R_PRODUCT_DETAIL.FINISH_TECHNIQUE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Hinge_Location">'
|| R_PRODUCT_DETAIL.HINGE_LOCATION
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Construction_Type">'
|| R_PRODUCT_DETAIL.CONSTRUCTION_TYPE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Panel_Type">'
|| R_PRODUCT_DETAIL.PANEL_TYPE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Panel_Profile_No">'
|| R_PRODUCT_DETAIL.PANEL_PROFILE_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Rail_Profile_No">'
|| R_PRODUCT_DETAIL.RAIL_PROFILE_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Rail_1_Length">'
|| R_PRODUCT_DETAIL.RAIL_1_LENGTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Stile_Profile_No">'
|| R_PRODUCT_DETAIL.STILE_PROFILE_NO
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Rail_2_Length">'
|| R_PRODUCT_DETAIL.RAIL_2_LENGTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Stile_1_Length">'
|| R_PRODUCT_DETAIL.STILE_1_LENGTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Stile_2_Length">'
|| R_PRODUCT_DETAIL.STILE_2_LENGTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Panel_1_Width">'
|| R_PRODUCT_DETAIL.PANEL_1_WIDTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Panel_1_Length">'
|| R_PRODUCT_DETAIL.PANEL_1_LENGTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Panel_2_Width">'
|| R_PRODUCT_DETAIL.PANEL_2_WIDTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Panel_2_Length">'
|| R_PRODUCT_DETAIL.PANEL_2_LENGTH
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT ||'</label>';
/*-----------------------------------------+
| Handling XML data for special message |
+-----------------------------------------*/
FOR rec_count IN c_count (r_product_detail.item_id)
LOOP
L_XML_CONTENT := L_XML_CONTENT || '<label _FORMAT='
|| '"'
|| 'lib://FRD/SpecMessage_Door.btw'
|| '"'
|| ' _PRINTERNAME='
|| '"'
|| v_printer
|| '"'
|| ' _QUANTITY='
|| '"'
|| v_quantity
|| '"'
|| '>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Serial_Number">'
|| REC_COUNT.SERIAL_NUMBER
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Special_Message">'
|| REC_COUNT.ATTRIBUTE_VALUE
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT || '<variable name= "Cabinet_Sequence_No">'
|| REC_COUNT.CABINET_NUMBER
|| '</variable>';
L_XML_CONTENT := L_XML_CONTENT ||'</label>';
EXIT WHEN c_count%NOTFOUND;
end LOOP;
-- L_XML_CONTENT := L_XML_CONTENT || '</labels>';
fnd_file.put_line (fnd_file.LOG, l_xml_content);
dbms_output.put_line ( l_xml_content );
L_LABEL_COUNT := L_LABEL_COUNT + 1;
-- apps.inv_print_request.sync_print_tcpip (l_xml_content,
-- l_job_status,
-- l_printer_status,
-- l_status_type,
-- l_return_status,
-- l_return_msg
END LOOP;
END LOOP;
l_xml_content := l_xml_content || '</labels>';
fnd_file.put_line (fnd_file.LOG, l_xml_content);
apps.inv_print_request.sync_print_tcpip (l_xml_content,
l_job_status,
l_printer_status,
l_status_type,
l_return_status,
l_return_msg
L_XML_CONTENT := null;
/*--------------------------------------------------------------------------------------+
| APPS.INV_PRINT_REQUEST.SYNC_PRINT_TCPIP will send the XML data to TCP/IP Port |
+--------------------------------------------------------------------------------------*/
fnd_file.put_line (fnd_file.LOG,
'Printer Status:' || ' ' || l_printer_status
fnd_file.put_line (fnd_file.LOG,
'Return Status:' || ' ' || l_return_status
fnd_file.put_line (fnd_file.LOG,
'Return Message:' || ' ' || L_RETURN_MSG
COMMIT;
EXCEPTION
WHEN OTHERS
THEN
fnd_file.put_line
(fnd_file.LOG,
'Unexpected error in the xxmb_get_xml_data_1270 procedure, error is : '
|| SQLERRM
|| ', '
|| SQLCODE
END xxmb_get_xml_data_1270;
END xxmb_wip_prod_tag_door_pkg;
/

Couldn't copy large amount of data from enterprise DB to Oracle 10g

Hi,
I am using i-batis to copy data from enterprise DB to oracle and viceversa.
The datatype of a field on EDB is 'text' and the datatype on oracle is 'SYS.XMLTYPE'
i am binding these to a java string property in a POJO to bind values.
I could successfully copy limited amount of data from EDB to oracle but if there is more data, i am getting the following exceptions with different oracle drivers ( but i could read large amount of data from EDB):
--- Cause: java.sql.SQLException: ORA-01461: can bind a LONG value only for insert into a LONG column
at com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeUpdate(MappedStatement.java:107)
at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.update(SqlMapExecutorDelegate.java:457)
at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.update(SqlMapSessionImpl.java:90)
at com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.update(SqlMapClientImpl.java:66)
at com.aqa.pojos.OstBtlData.updateOracleFromEdbBtlWebservice(OstBtlData.java:282)
at com.aqa.pojos.OstBtlData.searchEdbAndUpdateOracleBtlWebservice(OstBtlData.java:258)
com.ibatis.common.jdbc.exception.NestedSQLException:
--- The error occurred in com/aqa/sqlmaps/SQLMaps_OSTBTL_Oracle.xml.
--- The error occurred while applying a parameter map.
--- Check the updateOracleFromEDB-InlineParameterMap.
--- Check the parameter mapping for the 'btlxml' property.
--- Cause: java.sql.SQLException: setString can only process strings of less than 32766 chararacters
at com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeUpdate(MappedStatement.java:107)
at com.iba
I have latest oracle 10g jdbc drivers.
remember, i could copy any amount of data from oracle to EDB but not otherway around.
PLease let me know if you have come across this issue, any recommendation is very much appreciated.
Thanks,
CK.

Hi,
I finally remembered how I solved this issue previously.
The jdbc driver isn't able to directly call the insert with a column xml_type. The solution I was using was to build a wrapper function in plSQL.
Here it is (for insert but I suppose tha update will be the same)
create or replace procedure insertXML(file_no_in in number, program_no_in in varchar2, ost_XML_in in clob, btl_XML_in in clob) is
begin
insert into AQAOST_FILES (file_no,program_no,ost_xml,btl_xml) values(file_no_in, program_no_in, xmltype(ost_XML_in), xmltype(btl_XML_in));
end insertXML;
here is the sqlmap file I used
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE sqlMap
PUBLIC "-//ibatis.apache.org//DTD SQL Map 2.0//EN"
"http://ibatis.apache.org/dtd/sql-map-2.dtd">
<sqlMap>
     <typeAlias alias="AqAost" type="com.sg2net.jdbc.AqAost" />
     <insert id="insert" parameterClass="AqAost">
          begin
               insertxml(#fileNo#,#programNo#,#ostXML:CLOB#,#bltXML:CLOB#);
          end;
     </insert>
</sqlMap>
an here is a simple program
package com.sg2net.jdbc;
import java.io.IOException;
import java.io.Reader;
import java.io.StringWriter;
import java.sql.Connection;
import oracle.jdbc.pool.OracleDataSource;
import com.ibatis.common.resources.Resources;
import com.ibatis.sqlmap.client.SqlMapClient;
import com.ibatis.sqlmap.client.SqlMapClientBuilder;
public class TestInsertXMLType {
     * @param args
     public static void main(String[] args) throws Exception {
          // TODO Auto-generated method stub
          String resource="sql-map-config-xmlt.xml";
          Reader reader= Resources.getResourceAsReader(resource);
          SqlMapClient sqlMap = SqlMapClientBuilder.buildSqlMapClient(reader);
          OracleDataSource dataSource= new OracleDataSource();
          dataSource.setUser("test");
          dataSource.setPassword("test");
          dataSource.setURL("jdbc:oracle:thin:@localhost:1521:orcl");
          Connection connection=dataSource.getConnection();
          sqlMap.setUserConnection(connection);
          AqAost aqAost= new AqAost();
          aqAost.setFileNo(3);
          aqAost.setProgramNo("prg");
          Reader ostXMLReader= Resources.getResourceAsReader("ostXML.xml");
          Reader bltXMLReader= Resources.getResourceAsReader("bstXML.xml");
          aqAost.setOstXML(readerToString(ostXMLReader));
          aqAost.setBltXML(readerToString(bltXMLReader));
          sqlMap.insert("insert", aqAost);
          connection.commit();
     public static String readerToString(Reader reader) {
          StringWriter writer = new StringWriter();
          char[] buffer = new char[2048];
          int charsRead = 0;
          try {
               while ((charsRead = reader.read(buffer)) > 0) {
                    writer.write(buffer, 0, charsRead);
          } catch (IOException ioe) {
               throw new RuntimeException("error while converting reader to String", ioe);
          return writer.toString();
package com.sg2net.jdbc;
public class AqAost {
     private long fileNo;
     private String programNo;
     private String ostXML;
     private String bltXML;
     public long getFileNo() {
          return fileNo;
     public void setFileNo(long fileNo) {
          this.fileNo = fileNo;
     public String getProgramNo() {
          return programNo;
     public void setProgramNo(String programNo) {
          this.programNo = programNo;
     public String getOstXML() {
          return ostXML;
     public void setOstXML(String ostXML) {
          this.ostXML = ostXML;
     public String getBltXML() {
          return bltXML;
     public void setBltXML(String bltXML) {
          this.bltXML = bltXML;
I tested the insert and it works correctly
ciao,
Giovanni

Transporting large amounts of data from one database schema to another

Hi,
We need to move large amount of data from one 10.2.0.4 database schema to another 11.2.0.3 database.
Am currently using datapump but quite slow still - having to do in chunks.
Also the datapump files quite large so having to compress and move across the network.
Is there a better/quicker way?
Habe haerd about transportable tablespaces but never used them and don't know about speed - if quicker thana datapump.
tablespace names different in both databases.
Also source database on solaris opertaing system on sun box
target database on aix on ibm power series box.
Any ideas would be great.
Thanks
Edited by: user5716448 on 08-Sep-2012 03:30
Edited by: user5716448 on 08-Sep-2012 03:31

user5716448 wrote:
Hi,
We need to move large amount of data from one 10.2.0.4 database schema to another 11.2.0.3 database.
Pl quantify "large".
Am currently using datapump but quite slow still - having to do in chunks.
Pl quantify "quite slow".
Also the datapump files quite large so having to compress and move across the network.
Again, pl quantify "quite large".
Is there a better/quicker way?
Habe haerd about transportable tablespaces but never used them and don't know about speed - if quicker thana datapump.
tablespace names different in both databases.
Also source database on solaris opertaing system on sun box
target database on aix on ibm power series box.
It may be possible, assuming you do not violate any of these conditions
http://docs.oracle.com/cd/E11882_01/server.112/e25494/tspaces013.htm#ADMIN11396
Any ideas would be great.
Thanks
Edited by: user5716448 on 08-Sep-2012 03:30
Edited by: user5716448 on 08-Sep-2012 03:31Master Note for Transportable Tablespaces (TTS) -- Common Questions and Issues [ID 1166564.1]
HTH
Srini

Power BI performance issue when load large amount of data from database

I need to load data set from my database, which have large amount of data, it will take so many time to initialize data before I can build report, is there any good way to process large amount of data for PowerBI? As I know many people analysis data based
on PowerBI, is there any suggestion for loading large amount of data from database?
Thanks a lot for help

Hi Ruixue,
We have made significant performance improvements to Data Load in the February update for the Power BI Designer:
http://blogs.msdn.com/b/powerbi/archive/2015/02/19/6-new-updates-for-the-power-bi-preview-february-2015.aspx
Would you be able to try again and let us know if it's still slow? With the latest improvements, it should take between half and one third of the time that it used to.
Thanks,
M.

Bex Report Designer - Large amount of data issue

Hi Experts,
I am trying to execute (on Portal) report made in BEx Report Designer, with about 30 000 pages, and the only thing I am getting is a blank page. Everything works fine at about 3000 pages. Do I need to set something to allow processing such large amount of data?
Regards
Vladimir

Hi Sauro,
I have not seen this behavior, but it has been a while since I tried to send an input schedule that large. I think the last time was on a BPC NW 7.0 SP06 system and it worked OK. If you are on a recent support package, then you should search for relevant notes (none come to mind for me, but searching yourself is always a good idea) and if you don't find one then you should open a support message with SAP, with very specific instructions for recreating the problem from a clean input-schedule.
Good luck,
Ethan

Streaming large amounts of data of socket causes corruption?

I'm wrinting an app to transfer large amounts of data via a simple client/server architecture between two machines.
Problem: If I send the data too 'fast', the data arrives corrupted:
- Calls to read() returns wrong data (wrong 'crc')
- Subsequent calls to read() do not return -1 but allow me to read e.g. another 60 or 80 KBytes.
- available() returns always '0'; but I'll get rid of that method anyway (as recommended in other forum entries).
The behaviour is somewhat difficult to repeat, but it fails for me reliably when transferring the data between two separate machines and when setting the number of packets (Sender.TM) to 1000 or larger.
Workaround: Reduce number of packages send to e.g. 1; or intruduce the 'sleep' on the Sender side. Another workaround: Changing alone to java.nio.* did not help, but when I got rid of the Streams and used solely ByteBuffers, the problem disappeared. Unfortunately the Streams are required by other parts of my application.
I'm running the code on two dual-CPU machines connected via
Below are the code of the Sender and the Listener. Please excuse the style as this is only to demonstrate the problem.
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.channels.Channels;
import java.nio.channels.SocketChannel;
import java.util.Arrays;
public class SenderBugStreams {
    public static void main(String[] args) throws IOException {
        InetSocketAddress targetAdr = new InetSocketAddress(args[0], Listener.DEFAULT_PORT);
        System.out.println("connecting to: "+targetAdr);
        SocketChannel socket = SocketChannel.open(targetAdr);
        sendData(socket);
        socket.close();
        System.out.println("Finished.");
    static final int TM = 10000;
    static final int TM_SIZE = 1000;
    static final int CRC = 2;
    static int k = 5;
    private static void sendData(SocketChannel socket) throws IOException {
        OutputStream out = Channels.newOutputStream(socket);
        byte[] ba = new byte[TM_SIZE];
        Arrays.fill(ba, (byte)(k++ % 127));
        System.out.println("Sending..."+k);
        for (int i = 0; i < TM; i++) {
            out.write(ba);
//            try {
//                Thread.sleep(10);
//            } catch (InterruptedException e) {
//                // TODO Auto-generated catch block
//                e.printStackTrace();
//                throw new RuntimeException(e);
        out.write(CRC);
        out.flush();
        out.close();
import java.io.IOException;
import java.io.InputStream;
import java.net.InetSocketAddress;
import java.nio.channels.Channels;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
public class ListenerBugStreams {
    static int DEFAULT_PORT = 44521;
     * @param args
     * @throws IOException
    public static void main(String[] args) throws IOException {
        ServerSocketChannel serverChannel = ServerSocketChannel.open();
        serverChannel.socket().bind(new InetSocketAddress(DEFAULT_PORT));
        System.out.print("Waiting...");
        SocketChannel clientSocket = serverChannel.accept();
        System.out.println(" starting, IP=" + clientSocket.socket().getInetAddress() +
            ", Port="+clientSocket.socket().getLocalPort());
        //read data from socket
        readData(clientSocket);
        clientSocket.close();
        serverChannel.close();
        System.out.println("Closed.");
    private static void readData(SocketChannel clientSocket) throws IOException {
        InputStream in = Channels.newInputStream(clientSocket);
        //read and ingest objects
        byte[] ba = null;
        for (int i = 0; i < SenderBugStreams.TM; i++) {
            ba = new byte[SenderBugStreams.TM_SIZE];
            in.read(ba);
            System.out.print("*");
        //verify checksum
        int crcIn = in.read();
        if (SenderBugStreams.CRC != crcIn) {
            System.out.println("ERROR: Invalid checksum: "+SenderBugStreams.CRC+"/"+crcIn);
        System.out.println(ba[0]);
        int x = in.read();
        int remaining = 0;
        while (x != -1) {
            remaining++;
            x = in.read();
        System.out.println("Remaining:"+in.available()+"/"+remaining);
        System.out.println(" "+SenderBug.TM+" objects ingested.");
        in.close();
}

Here is your trouble:
in.read(ba);read(byte[]) does not read N bytes, it reads up to N bytes. If one byte has arrived then it reads and returns that one byte. You always need to check the return value of read(byte[]) to see how much you got (also check for EOF). TCP chops up the written data to whatever packets it feels like and that makes read(byte[]) pretty random.
You can use DataInputStream which has a readFully() method; it loops calling read() until it gets the full buffer's worth. Or you can write a little static utility readFully() like so:
    // Returns false if hits EOF immediately. Otherwise reads the full buffer's
    // worth. If encounters EOF in mid-packet throws an IOException.
    public static boolean readFully(InputStream in, byte buf[])
        throws IOException
        return readFully(in, buf, 0, buf.length);
    public static boolean readFully(InputStream in, byte buf[], int pos, int len)
        throws IOException
        int got_total = 0;
        while (got_total < len) {
            int got = in.read(buf, pos + got_total, len - got_total);
            if (got == -1) {
                if (got_total == 0)
                    return false;
                throw new EOFException("readFully: end of file; expected " +
                                       len + " bytes, got only " + got_total);
            got_total += got;
        return true;
    }

Large Amount of Data in JSF

Hello,
I am using the Table Group component for displaying data in my application designed in Java Studio Creator.
I have enabled paging on the component. I use CachedRowSet on the bean for the page for getting the data. This works very well at the moment in my development environment. At the moment I am testing on small amount of data.
I was wondering how does this component perform with very large amounts of data (>75,000 rows). I noticed that there is a button available for users to retrieve all the rows. So I was wondering apart from that instance, when viewing in a paged mode does the component get all the results from the database everytime ?
Which component would be best suited for displaying large amounts of data in a table format?
Thanks In Advance!!

Thanks for your reply. The table control that I use does have paging as a feature and I have enabled it. It still takes time to load the data initially.
I wonder if it is got to do with the logic of paging. How do you specify which set of 20 records to extract from SQL.
Thanks for your help!!

Advice needed on how to keep large amounts of data

Hi guys,
Im not sure whats the best way is to make large amounts of data available to my android app on the local device.
For example records of food ingredients, in the 100's?
I have read and successfully created .db's using this tutorial.
http://help.adobe.com/en_US/AIR/1.5/devappsflex/WS5b3ccc516d4fbf351e63e3d118666ade46-7d49. html
However to populate the database I use flash? So this kind of defeats the purpose of it. No point in me shifting a massive array of data from flash to a sql database, when I could access the data direct from the as3 array?
So maybe I could create the .db with an external program? but then how would I include that .db in the apk file and then deploy it to users android device.
Or maybe I create a as3 class with an xml object init and use that as a means of data storage?
Any advice would be appreciated

You can use any means you like to populate your SQLite database, including using external programs, (temporarily) embedding a text file with SQL statements, executing some SQL from AS3 code etc etc.
Once you have populated your db, deploy it with your project:
http://chrisgriffith.wordpress.com/2011/01/11/understanding-bundled-sqlite-databases-in-ai r-for-mobile/
Cheers, - Jon -

Sending Large amount of data (250 K +) from Oracle to flex client

Similar Messages

Maybe you are looking for