Improving performance on migrating data from one Oracle database to another

We are using hibernate to migrate complete tables from one database to the other. This takes hours to days to transfer a couple of tables, all having relationships between each other. The only difference is that the new tables have an extra column which stores the primary key of the source table as an alias key in the destination table and the destination table has only one extra column which is its own primary key. Another small difference is that the name of the columns is a little different.
Isn't there a simple way to use Oracle database's in built tools to migrate data if provided the source and destination tables and column names for mapping purposes? Please provide some suggestions for this Oracle newbie. Any other suggestions on improving the migration performance will be greatly appreciated.
Suggestions please!!!

Hello Justin,
Thanks for your prompt reply. Ours is a small company so we take up all kinda roles. In this particular scenario, I was asked to migrate all the tables from one database to another. Both of the databases are present on two different machines but on the same network. There are about 100,000 records in some of the tables and there are about 50 tables to migrate. As I said, both the tables have different columns to be mapped and then transferred and the new table has only one extra column which is a primary key in the new table. Under these constraints, Hibernate is taking way too much time.
I always felt that there must be a simpler and efficient way to do this. I would appreciate if you could you explain me the procedures in detail please.
Thanks in advance.

Similar Messages

  • Migrating Data from one BW system to another

    Hi Friends,
    I have a task where i need to replicate/migrate objects under specific InfoArea/s from one BW system to another BW system. Please suggest me the ways this can be done. Thanks in advance for all the help.
    Raj

    Hi
    to transfer the data is easy, use flat files or you can export the DS in the source sys and then load it in the target sys (the easiest way).
    to transfer structores look at these:
    Re: Transporting Queries
    I hope it helps.
    Edan

  • Copying database objects and data from one server database to another server database in AG group

    Hi,
    I am still trying to wrap my head around sql clusters and AGs and I have a project that requires I take a vendor's database and restore it weekly so its available on the production server which is clustered.
    The vendor's database on the cluster is in an AG group and encrypted.
    Right now, I plan to restore the database on a sql staging server and use the SSIS Transfer SQL Server Objects Task to copy the table structure and data from Stage to the Production database of same name and I would first drop the objects in production
    database using the same task.
    I am concerned that this might cause issues with the passive cluster due to "logging" from active to passive. The database is about 260 MBs and I am not sure how many tables.
    Has anyone run into this type of scenario before or have a better solution?
    Thanks
    Sue

    IF I understand anything about clustered sql and logging, the sql server should take the log file and recreate the same scenario on the passive side of the cluster.
    Is that correct?
    Hi Sue,
    Yes, for AlwaysOn Availability Group, the transaction log is basically replayed from the primary to all of the secondary's.
    Besides, from my point of view, as we cannot directly restore a database that is part of an Availability Group, it is a good way using SSIS task to drop and recreate all tables then transfer data from the restored database to the primary replica. Schema changes
    and data changes will also happen on the secondary  replica.
    There are some similar links for your reference.
    http://dba.stackexchange.com/questions/21404/do-schema-changes-break-sql-server-2012-alwayson-or-are-they-handled-transpare
    http://blogs.msdn.com/b/sqlgardner/archive/2012/08/28/sql-2012-alwayson-and-backups-part-3-restore.aspx
    Thanks,
    Lydia Zhang
    If you have any feedback on our support, please click
    here.
    Lydia Zhang
    TechNet Community Support

  • How to add data from one TS database to another?

    HI,
    This is really a simple and basic question.
    I'm collecting data in one local database (Access/Jet). I have the same database somewhere else and I would like to transfer the data I have collected locally to that other one.
    Is there a tool within the TS environment to help me do that or do I need to use MS Access in order to do it?
    Is it possible to do it programmatically?
    Thanks
    Rafi

    You have all of the database steps to work with. You can open one, do a SQL query in which you SELECT the data save it to TestStand variables and then open the second db and do a SQL INSERT to that. Tools for database synchronization are probably built into Access and that might be the more efficient method though. The TestStand ActiveX adapter can be used or maybe you can write a stored procedure that will do it for you.

  • HTTP post data from the Oracle database to another web server

    Hi ,
    I have searched the forum and the net on this. And yes I have followed the links
    http://awads.net/wp/2005/11/30/http-post-from-inside-oracle/
    http://manib.wordpress.com/2007/12/03/utl_http/
    and Eddie Awad's Blog on the same topic. I was successful in calling the servlet but I keep getting errors.
    I am using Oracle 10 g and My servlet is part of a ADF BC JSF application.
    My requirement is that I have blob table in another DB and our Oracle Forms application based on another DB has to view the documents . Viewing blobs over dblinks is not possible. So Option 1 is to call a procedure passing the doc_blob_id parameter and call the web server passing the parameters.
    The errors I am getting is:
    First the parameters passed returned null. and
    2. Since my servlet directly downloads the document on the response outputStream, gives this error.
    'com.evermind.server.http.HttpIOException: An established connection was aborted by the software in your host machine'
    Any help please. I am running out of time.
    Thanks

    user10264958 wrote:
    My requirement is that I have blob table in another DB and our Oracle Forms application based on another DB has to view the documents . Viewing blobs over dblinks is not possible. Incorrect. You can use remote LOBs via a database link. However, you cannot use a local LOB variable (called a LOB <i>locator</i>) to reference a remote LOB. A LOB variable/locator is a pointer - that pointer cannot reference a LOB that resides on a remote server. So simply do not use a LOB variable locally as it cannot reference a remote LOB.
    Instead provide a remote interface that can deal with that LOB remotely, dereference that pointer on the remote system, and pass the actual contents being pointed at, to the local database.
    The following demonstrates the basic approach. How one designs and implements the actual remote interface, need to be decided taking existing requirements into consideration. I simply used a very basic wrapper function.
    SQL> --// we create a database link to our own database as it is easier for demonstration purposes
    SQL> create database link remote_db connect to scott identified by tiger using
      2  '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=1521))(CONNECT_DATA=(SID=dev)(SERVER=dedicated)))';
    Database link created.
    SQL> --// we create a table with a CLOB that we will access via this db link
    SQL> create table xml_files( file_id number, xml_file clob );
    Table created.
    SQL> insert into xml_files values( 1, '<root><text>What do you want, universe?</text></root>' );
    1 row created.
    SQL> commit;
    Commit complete.
    SQL> --// a local select against the table works fine
    SQL> select x.*, length(xml_file) as "SIZE" from xml_files x;
       FILE_ID XML_FILE                                                                                SIZE
             1 <root><text>What do you want, universe?</text></root>                                    53
    SQL> --// a remote select against the table fails as we cannot use remote pointers/locators
    SQL> select * from xml_files@remote_db x;
    ERROR:
    ORA-22992: cannot use LOB locators selected from remote tables
    no rows selected
    SQL> //-- we create an interface on the remote db to deal with the pointer for us
    SQL> create or replace function ReturnXMLFile( fileID number, offset integer, amount integer ) return varchar2 is
      2          buffer  varchar2(32767);
      3  begin
      4          select
      5                  DBMS_LOB.SubStr( x.xml_file, amount, offset )
      6                          into
      7                  buffer
      8          from    xml_files x
      9          where   x.file_id = fileID;
    10 
    11          return( buffer );
    12  end;
    13  /
    Function created.
    SQL> --// we now can access the contents of the remote LOB (only in 4000 char chunks using this example)
    SQL> select
      2          file_id,
      3          ReturnXMLFile@remote_db( x.file_id, 1, 4000 ) as "Chunk_1"
      4  from       xml_files@remote_db x;
       FILE_ID Chunk_1
             1 <root><text>What do you want, universe?</text></root>
    SQL> --// we can also copy the entire remote LOB across into a local LOB and use the local one
    SQL> declare
      2          c               clob;
      3          pos             integer;
      4          iterations      integer;
      5          buf             varchar2(20);   --// small buffer for demonstration purposes only
      6  begin
      7          DBMS_LOB.CreateTemporary( c, true );
      8 
      9          pos := 1;
    10          iterations := 1;
    11          loop
    12                  buf := ReturnXMLFile@remote_db( 1, pos, 20 );
    13                  exit when buf is null;
    14                  pos := pos + length(buf);
    15                  iterations := iterations + 1;
    16                  DBMS_LOB.WriteAppend( c, length(buf), buf );
    17          end loop;
    18 
    19          DBMS_OUTPUT.put_line( 'Copied '||length(c)||' byte(s) from remote LOB' );
    20          DBMS_OUTPUT.put_line( 'Read Iterations: '||iterations );
    21          DBMS_OUTPUT.put_line( 'LOB contents (1-4000):'|| DBMS_LOB.SubStr(c,4000,1) );
    22 
    23          DBMS_LOB.FreeTemporary( c );
    24  end;
    25  /
    Copied 53 byte(s) from remote LOB
    Read Iterations: 4
    LOB contents (1-4000):<root><text>What do you want, universe?</text></root>
    PL/SQL procedure successfully completed.
    SQL> The concern is the size of the LOB. It does not always make sense to access the entire LOB in the database. What if that LOB is a 100GB in size? Irrespective of how you do it, selecting that LOB column from that table will require a 100GB of data to be transferred from the database to your client.
    So you need to decide WHY you want the LOB on the client (which will be the local PL/SQL code in case of dealing with a LOB on a remote database)? Do you need the entire LOB? Do you need a specific piece from it? Do you need the database to first parse that LOB into a more structured data struct and then pass specific information from that struct to you? Etc.
    The bottom line however is that you can use remote LOBs. Simply that you cannot use a local pointer variable to point and dereference a remote LOB.

  • Database link from one Oracle database to another

    Hi everyone!
    I have 2 databases. I need to create a database link to copy a table from one of the database to another.
    Both of them are Oracle 10g v2
    Please advice me on how can I go about it.
    I am trying to avoid import and export.
    What are the options left to me?
    Thanks

    user5160274 wrote:
    Hi Kanchana,
    I am using Oracle 10g/Windows Server 2008. I have created a dabaselink. The link works perfectly fine. I want to copy the table (Oracle 11g) from remote server in to my database. It is ongoing process ie, it has to be scheduled and I want the updated rows to be copied in to my database on a daily basis. Please help me out.
    Eagerly awaiting for your response.
    Regards
    Narayanan NAnd once again a new member of the forum, on their very first post, goes on an archeological dig and tries to raise the dead. This thread is well over a year old. You should have started your own thread for your own issue. You say you have created a link and it works, so whatever your problem is, it is NOT even related to the question the OP raised.
    At exactly what point are you unsure about the process? Scheduling a processes? The command to use? Exactly what have you tried? What results do you have so far?.
    Please, start a new thread explaining your issue.
    Moderator --- please lock this thread.

  • Is there a way to migrate data from one user account to another simply?

    Recently one of our employees has left the company. This user had pertinent images saved to their profile and we are needing to migrate those images to a new profile (Kiosk) so users can access the needed files. In InDesign I'm told it pulls file paths in order to pull up different images and documents. Is there a way to transfer these files without breaking InDesign? Or is there a simple way to recreate the file paths in InDesign so that we may move the user's information to a new account and recreate the file paths as soon as possible.
    I was referred here from - http://forums.adobe.com/thread/1397569

    It would be through scripting. If no one else answers here, try using the InDesign Scripting forum:
    InDesign Scripting

  • Migration from one oracle DB to another Oracle DB

    Hi ,
    We are migration data from one oracle DB to another Oracle DB. Lets take it as DB1 and DB2.
    Both are having many tables and many columns. So some tables contains common columns with different data types and same datatype with different datasize.
    I am trying to map the columns in both the DB's. Then I will retrieve what are the equal columns and what are columns have difference in data types and datatype lengths.
    for Eg : Please find the below tables and the structures.
    DB1                                                              DB2
    EMP                                                             EMPLOYEE
    ENO   VARCHAR2(10)                                  EMP_NO  Number(8)
    ENAME  Varchar2(25)                                   EMP_NAME  Varchar2(20)
    SAL   NUMBER(8)                                        JOb    VARCHAR2(20)
    Description  BLOB/CLOB                              Department   Varcgar2(20)
                                                                       SAL  number(8,3)
                                                                       JOB_DESC  varchar2(30)
    I would like to migrate DB1 to DB2
    Here I found the differences as follows.
    1. There is relation between EMP and EMPLOYEE tables.
    2. Datatype is different for EMPNO(DB2) compare to DB1. ( Here in DB1 datatype is Varchar and in DB2 it is Number )
    3. in DB1 Ename datatype length is 25 and in DB2 it is 20...
    So can any one suggest me any pl/sql automation code how can we map the columns,datatypes,datalengths and other required information.
    Can you please suggest me and give your suggestions to write automation code if possible or how can we resolve these type of issues.
    Thanks,
    Vas 

    So can any one suggest me any pl/sql automation code how can we map the columns,datatypes,datalengths and other required information.
    No such 'automation code' exists. You need to write your own mapping functions and exception handling processes.
    Can you please suggest me and give your suggestions to write automation code if possible or how can we resolve these type of issues.
    Finish what you started. The most important step you can take is to create a Functional Requirements document that includes, at a minimum:
    1. Mapping of each source table to each target table
    2. Lists of possible errors, or dirty data issues, that might be encountered
    3. The business rules that should be applied to each of those possible errors
    You already made a good start on the mapping for the one table above. And you made a good start of identifying the possible errors:
    2. Datatype is different for EMPNO(DB2) compare to DB1. ( Here in DB1 datatype is Varchar and in DB2 it is Number )
    Good - it should be easy for everyone in your org to agree that you can't put 'TEXT' data into a NUMBER column so now you need to document that issue:
    1. Identify data in DB1.ENO that is not numeric
    2. Remove, ignore, fix that data?
    What do the business users want to do with text data in that DB1.ENO column that is not numeric? They need to tell you. They may want a report or excel file that contains that 'dirty' data so they can review and/or fix it. They may want to query that data for themselves. They may want to know where that data came from - especially if the data is all supposed to be numeric.
    3. in DB1 Ename datatype length is 25 and in DB2 it is 20...
    Good - use the same process as above
    1. Identify data in DB1.ENAME that is longer than 20.
    2. Remove, ignore, fix that data? Truncate it?
    Again - it is usually the business users that need to actually determine what should be done. Your job is to identify that 'dirty' data and/or make it available to the business users so they can research it
    You don't need to wait for the business users to actually provide all of those answers. You can begin writing pseudo-code that contains steps to deal with those issues - you just can't write the actual code until you have the answers.
    You can certainly begin writing queries or procedures that look for 'dirty' data. Just based on the two problems you posted you can write queries to find data that has those two issues.

  • Import/export from one oracle db to anothe roracle db

    Hi All,
    I would like to import and export data from one oracle database to another oracle database. Can any one please suggest me/provide me any script if available.
    I am following Datapump IMPDP and EXPDP. I would like to do this data pump process every time. So I wolud like to do this task as automate. So please provide me any script for automating this job
    or let me know if there is any easy method to do this process.. I would like to have a script and am uising Oracle 11.2.0.3.
    Thanks

    Hi,
    Below Anonymous block Exports the 'HR' schema:
    DMPDIR -- Directory object where you want to create the dmp files
    DECLARE
      l_dp_handle      NUMBER;
      l_last_job_state VARCHAR2(30) := 'UNDEFINED';
      l_job_state      VARCHAR2(30) := 'UNDEFINED';
      l_sts KU$_STATUS;
    BEGIN
      l_dp_handle                          := DBMS_DATAPUMP.open( operation => 'EXPORT', job_mode => 'SCHEMA', remote_link => NULL, job_name => 'SCHEMA_EXPORT', version => 'LATEST');
      DBMS_DATAPUMP.add_file( handle        => l_dp_handle, filename => 'SCHEMA_EXPORT.dmp', directory => 'DMPDIR');
      DBMS_DATAPUMP.add_file( handle        => l_dp_handle, filename => 'SCHEMA_EXPORT.log', directory => 'DMPDIR', filetype => DBMS_DATAPUMP.KU$_FILE_TYPE_LOG_FILE);
      DBMS_DATAPUMP.metadata_filter( handle => l_dp_handle, name => 'SCHEMA_EXPR', value => '= ''HR''');
      DBMS_DATAPUMP.start_job(l_dp_handle);
      DBMS_DATAPUMP.detach(l_dp_handle);
    END;
    Now you can use this dmp file to import the 'HR' schema in the required database.
    Below Anonymous block imports the 'HR' schema:
    DECLARE
    dph NUMBER;
    BEGIN
      dph := dbms_datapump.open(operation => 'IMPORT', job_mode =>
      'SCHEMA',job_name => 'SCHEMA_IMPORT');
      dbms_datapump.add_file(handle => dph,filename =>
      'SCHEMA_EXPORT.dmp', directory => 'DMPDIR',filetype=>1);
      dbms_datapump.add_file(handle => dph,filename =>
      'SCHEMA_IMPORT.log',directory => 'DMPDIR',filetype=>3);
      dbms_datapump.start_job(dph);
      dbms_datapump.detach(dph);
    END;

  • Migrating data from one databaes to oracle?

    is there any way to migrate my data from my old/previous database to oracle db.
    ie. migrating data from one databaes to another.

    If you are using Oracle 9i, look at external tables. Otherwise, dumping your data to text files and using SQL*Loader to load it into Oracle tables is a good option. There are various tools and utilities you might look into, including ODBC and migration toolkit. I prefer the SQL*Loader method for DBase, FoxPro, and Access.

  • Migrate PY and TM cluster data from One SAP System to another

    Hi Experts,
    Could you please suggest me some options to migrate PY and TM cluster data from One SAP System to another? My requirement is to move all the cluster data (Active and Inactive ees).
    I see that PU12 is an option. Anybody used that to migrate between SAP systems. Please provide some details
    Points guaranteed
    I see that using PU12 you can export only PCL2 Clutsers. Are there any ways to move the other clusters like PCL4 ?

    Hi,
    As far as data transfer, If have all normal configuration settings in your targeted system you can use LSMW (Use T.code AS91 in LSMW for recording)  to transfer your data in your both cases.
    Hope it resolves your both the issues.
    Regards,
    Murali

  • How to put the data from one excel sheet in another excel sheet

    hi ,
    I want put the data from one excel sheet in another excel sheet in seq. order Eg: I have one excel sheet in which i have 3 col. Name , Sno. , Email along with data .I want to put data from this sheet to another excel sheet in the following orders of col. Sno,Name, Email .
    While loading data in another sheet , i have to perform validation like char field should n't contain numeric values and vice versa .
    Let me know on this soon ..
    regards
    Prashant

    Well, you can issue separate queries with the ordering you need from each tab in the spreadhseet. You can open an ODBC connection from a VBA macro, select a sheet, run a query, select another sheet and run another query. As for the validation, you can do this in Oracle via stored procedures or again in VBA code.

  • Using FDM to transfer data from one HFM app to another

    We are facing a project in which we need to load data from one HFM app to another. There're some mapping going so it was advised that FDM be used.
    However, the HFM extract files contain all data on the rows. While for the FDM multiload, the data from different periods need to be layed out on the columns. Also, we have multiple scenarios/years of data to handle. What would be the best approach? Has anyone had similar experience to load data from one HFM app to another using FDM? Thanks!

    It appears that this is some kind of migration project due to the scope of the data contained in the single file? If so whatever you do is like ly to be trow away once the migration of data is completed.
    You have a couple of options:
    1) Get the data extracted from HFM in multiple files instead of one bulk file, broken down by scanario,year & period
    2) Take the single data dump file produced by FDM and manipulate it yourself to get the data in a more usuable format for processing through FDM.
    Option 2 could be achieved via any ETL tool or a custom file parsing script. What may be more attractive to you and allow you to fully leverage your investment in FDM is that you could use the PULL adapter that ships as part of the FDM adapter suite to perform this transformation exercise. The PULL adapter takes a flat file input and allows you to use all the in built functionality of FDM to transform it and output a modified flat file (or series of flat files). You could use it to produce multioload files or a series of files broken down by scenario,year,period.
    Whatever you do I would suggest that break the single data file down into smaller chunks as this will help with the iterative debugging process you will inevitably have to undetake whislt migrating the data to the new application.

  • Pass data from one web application to another web application

    Hi,
    Please provide suggestion for following scenario:
    Scenario: Basically the aim is to transfer large amount of data from one application to another and display JSP of second application.
    User enteres data on a JSP of one web application. When he submits that data, another web application opens in new window and data from first application should be passed to this second web application. Another web application will display that data on its own JSP. User can perform whatever he wants on second application screen.
    Possible solutions:
    1) response.sendRedirect(): This makes GET request. But, there is large data to send. So, GET is not effective.
    2) forward(request, response): Can't use as I have to pass data to different application that is in another context.
    3) URLConnection: Here, I can make POST request and set attributes in HTTP request and make connection to another application. I can pass data using output stream. But, I can't display second application JSP to user even if I use input stream to read response. Because control will be ultimately in first application only.
    Is there any othe method to achieve this or any of the above options extendable?
    Please give your inputs.

    Hi,
    According to your post, my understanding is that you want to migrate list data from one web application to another.
    We can migrate list data programmatically, there are some articles for your reference.
    http://blogs.msdn.com/b/tejasr/archive/2007/11/12/code-snippet-copy-list-data-between-sites-programmatically.aspx
    http://www.fivenumber.com/copy-sharepoint-list-items-from-one-site-to-another-programmatically/
    http://geekswithblogs.net/AnneBougie/archive/2009/01/23/copy-a-sharepoint-list.aspx
    Thanks & Regards,
    Jason
    Jason Guo
    TechNet Community Support

  • Error while pulling data from an Oracle database. ORA-01858: a non-numeric character was found where a numeric was expected

    I'm trying to pull data from an Oracle database using SSIS. When I try to select a few fields from the source table, it returns the following error message:
        [OLE DB Source [47]] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E14.
        An OLE DB record is available.  Source: "OraOLEDB"  Hresult: 0x80040E14  Description: "ORA-01858: a non-numeric character was found where a numeric was expected".
        An OLE DB record is available.  Source: "OraOLEDB"  Hresult: 0x80004005  Description: "ORA-01858: a non-numeric character was found where a numeric was expected".
    The source columns are a combination of numeric and texts, and I've also tried selecting one of them, which didn't work. I'm using the Oracle client 11.2.0.1, and it works fine with any other data sources I have connected to so far. How can I resolve this
    error?

    Hi H.James,
    According to your description, the issue is a non-numeric character was found where a numeric was expected while pulling data from an Oracle database in SSIS.
    Based on the error message, the issue should be you are comparing a number column to a non-number column in a query. Such as the query below (ConfID is a number, Sdate is a date):
     where C.ConfID in (select C.Sdate
                       from Conference_C C
                       where C.Sdate < '1-July-12')
    Besides, a default behavior for the Oracle OleDb Provider that change the NLS Date Format of the session to 'YYYY-MM-DD HH24:MI:SS can also cause the issue. For more details about this issue, please refer to the following blog:
    http://blogs.msdn.com/b/dataaccesstechnologies/archive/2012/01/20/every-bug-is-a-microsoft-bug-until-proven-otherwise.aspx
    Thanks,
    Katherine Xiong
    Katherine Xiong
    TechNet Community Support

Maybe you are looking for