Need help / Advice ; manage daily millions of records;;plz help me:)

Hi all,
I've only 2 years of experience in Oracle DBA. I need advice from Experts:)
To begin, the company I work for, decide to daily save in our Oracle database about 40 millions of records in our only table (User tables). These records should be daily imported from csv or xml feeds into one table.
This 's a project that need :
- Study the performance
- Study What is required in terms of hardware
As a leader in the market, Oracle 's the only DBMS that could support this size of data, but what's the limit of Oracle in this case? can Oracle support and manage perfectly daily 40 millions of records and for many years, ie We need all data of this table, we can't consider after a period that we don't need history: we need to save all data and without purge the history and this for many years i suppose!!! you can imagine 40 daily millions of records and for many years!!!
Then we need to consolidate from this table different views (or maybe materalized view) for each department and business inside the company, one other project that need study!
My questions 're :Using Oracle Database 10g Enterprise Edition Release 10.2.0.1.0:
1-Can Oracle support and perfectly manage daily 40 millions of records and for many years?
2-Study the performance ; which solutions, technics could I use to improve the performance of :
- Daily Loading 40 millions of records from csv or xml file/files?
- Daily Consolidate / managing different views/ materalized view from this big table?
3- What is required in terms of hardware? features / Technologies( maybe clusters...)
Hope that experts help me and advice me! thank you very much for your atention :)

1-Can Oracle support and perfectly manage daily 40 millions of records and for many years?Yes
2-Study the performance ; which solutions, technics could I use to improve >>>the performance of :Send me your email, and I can send you a Performance tuning metodology pdf.
You can see my email on my profile.
Daily Loading 40 millions of records from csv or xml file/files?DIrect Load
- Daily Consolidate / managing different views/ materalized view from this big table?You can use table partitions, one partition for each day.
Regards,
Francisco Munoz Alvarez

Similar Messages

DELETE MILLIONS OF RECORDS;;;PLZ HELP ME

Hi all;
On my database I have an Unnecessary table ;
This table contains 38 millions of records; I want to purge this table but I afraid to destroy my database ;
this's the structure of my table :
-- Create table
create table MANAGEMENT.MC_COMPUTERS
GUID VARCHAR2(24),
GROUPID NUMBER(5),
QUOVACOUNTRY VARCHAR2(30),
CONNECTIONTYPE VARCHAR2(255),
CREA_MONTH VARCHAR2(10) not null,
ANTIVIRUS VARCHAR2(100),
WINVERSION VARCHAR2(100),
ACTF_1_DAY NUMBER,
ACTF_7_DAY NUMBER,
ACTF_14_DAY NUMBER,
ACTF_21_DAY NUMBER,
ACTF_28_DAY NUMBER,
INSTALL NUMBER,
DATE_CONSO VARCHAR2(24)
partition by (CREA_MONTH)
partition MC200701
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200702
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200703
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200704
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200705
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200706
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200707
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200708
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200709
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
partition MC200710
tablespace DATA
pctfree 10
initrans 1
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
-- Create/Recreate indexes
create index MANAGEMENT.INX_MC_MONTH on MANAGEMENT.MC_COMPUTERS (CREA_MONTH)
tablespace INDX
pctfree 10
initrans 2
maxtrans 255
storage
initial 64K
minextents 1
maxextents unlimited
Should I use this command truncate table ...; or delete from ... where CREA_MONTH ='' or maybe other sql statement ?

Well, you may not need to worry about destroy your database but you might find out if truncate this table will destroy your application.
A truncate statement is a regular DDL of Oracle, runing it will certainly not crush your database. However you need to talk to application user to figure out if this table is really unnecessary.
Before you do the truncate, better make a backup.

Inserting Millions of records-Please Help!

Hi All,
I have a scenario where I hvae to query MARA and filter out some articles and then query WLK1 table(Article/Site Combination) and insert the records to a Custom (z) table.the result maybe millions of records,
can anyone tell me a efficient way to insert large number of records? This is urgent.Please help.
Warm Regards,
Sandeep Shenoy

This is a sample code i am using in one of my programs. You can try similar way and insert into custom table with every loop pass.
I am considering 2000 records at a time. You can decide the no and code accordingly.
if not tb_bkpf[] is initial.
fetching the data from BSEG for each 1000 entries in BKPF to
reduce the overhead of database extraction.
    clear l_lines .
    describe table tb_bkpf lines l_lines.
    if l_lines >= 1.
      clear: l_start, l_end.
      do.
        l_start = l_end + 1.
        l_end = l_end + 2000.
        if l_end > l_lines.
          l_end = l_lines.
        endif.
        append lines of tb_bkpf from l_start to l_end to tb_bkpf_temp.
Populating the tb_bseg_tmp in the order of the database table
        select bukrs
               belnr
               gjahr
               buzei
               shkzg
               dmbtr
               hkont
               matnr
               werks
          from bseg
          appending table tb_bseg_tmp
          for all entries in tb_bkpf_temp
          where bukrs = tb_bkpf_temp-bukrs and
                belnr = tb_bkpf_temp-belnr and
                gjahr = tb_bkpf_temp-gjahr and
                hkont in s_hkont.
        refresh tb_bkpf_temp.
        if l_end >= l_lines.
          exit.
        endif.
      enddo.
    endif.

HT204382 music files that I'm trying to transfer from my external hard drive!!! i really need help! i am getting very frustrated. plz help me thank you

hi. i am trying to transfer my old pictures and music files from my external drive into my new laptop.and first of all only a coupleof my foldersare coming up.and even those few files/folders wont play ....i am new to mac.i REALLY need help. my old pictures and music are real important to me.plz help me.

IS the external HD from another Mac or a PC?

IPod Nano HELP!!!!!! PLZ HELP!!!!!!!!

Not all the time but sometimes when I charge my iPod nano mu iTunees will say iPod updated but when I unplug my ipod nano all the songs will erased of all iPod but there still on my iTunes. This has happened more then once. PLz HELP!!!!

Are you making sure to eject your iPod before disconnecting it?
While you are charging your Nano, you may want to click on the iPod icon under your playlists in iTunes, and see if any songs are on it. If not, your iPod may be set to manually update, which means you'd have to drag the songs you want to your iPod, as opposed to iTunes doing it automatically.

I need workflow advice with my green screen project! Help please!

Hello! I just shot a music video all in green screen. There are 5 different green screen shots and it is all HD. What is the best way to work with all of these green screen tracks? Should I key out all of the tracks and export them all with the alpha channel and then bring them back in? I just want to make sure when i layer them together to do effects, that it is not going to have to render everything again and take forever. Thanks!

Since this is your first production, you may need to transcode your "HD" to a frame-based codec before doing anything else.
Since this is your first chromakey shoot, we will assume you did not have the lighting and set optimized for proper green screen or you probably wouldn't be asking this question.
As Jim says, cut your show, get it close to locked. Take the green shots to Motion with some handles. Bring them back in as ProRes4444 movies.
bogiesan

Need help with ending of payment enrollment. Plz help as soon, as you can.

Hello, I bought a subscription for the iOS Developer, 11.03.13 deducted from my credit card $ 99. But the subscription has not joined. I received several letters, one of them written order number, but I can not see this number in your account - says error. Order No. 2442140328. Please help urgently to complete the process of registration and let me start downloading my app.
     P.S.Adj text messages. Do not rule out the possibility that the activation email just did not happen.
Thank you for your order.
We’ll let you know when your items are on their way.
Items to be Shipped
Shipment 1                         Available to ship within 24 hours via email
by Standard Shipping
                                                IOS DEVELOPER PROGRAM                                                                                       $99,00
Shipping Address                            Yaroslav Tretyakov/Andrey Tretyakov
52 Udaltsova str., app. 6
Moscow 77 119607
Russian Fed.
Payment
Billing Contact                   Yaroslav Tretyakov/Andrey Tretyakov
[email protected]
Billing Address                  52 Udaltsova str., app. 6
Moscow 77 119607
Russian Fed.
                                Subtotal              $99,00
Free Shipping    $0.00
Estimated Tax   $0,00
Order Total         $99,00
INVOICE RECEIPT
Dear Apple Customer,
Thank you for shopping at the Apple Store!
If you have already paid for your purchase, please retain this invoice receipt
for your records.
If you need to send payment to Apple, please reference Apple's Invoice Number on
your remittance. After remitting payment, please retain this invoice receipt for
your records.
Invoice Number:          4236349073
Invoice Date:            04/11/13
Reference Date:          04/11/13
Amount Due:              .00
Customer P.O. Number:    79859994401
Sales Order Number:      2442140328
Customer Number:         900001
Terms:                   Credit Card
Sold To:                                 Ship To:
Yaroslav Tretyakov/Andrey                Yaroslav Tretyakov/Andrey
Tretyakov                                Tretyakov
52 Udaltsova str., app. 6                52 Udaltsova str., app. 6
119607 MOSCOW                            119607 MOSCOW
RUSSIAN FED.                             RUSSIAN FED.
Item Product Product Description Total   Total   Unit Extended
      Number                              Ordered Shipped Price      Price
001 D4521G/A IOS DEVELOPER PROGRAM     1       1 99.00      99.00
Subtotal                 99.00
Tax                       0.00
Shipping Charges
TOTAL USD               99.00
Salesperson Contact Entry Date Ship Date Routing
900001 BD      04/11/13             Best Way
      Your Visa xxxx1634 has been charged   $ 99.00
      For a total of*********$                  99.00
Answers to many questions about Apple Store orders can be found in the online
Customer Service section. Visit http://store.apple.com/ and click "Customer
Service" in the navigation bar near the top.
If you have a question about an iTunes Store gift certificate purchase that was
made through the Apple Store, please visit the online Help section for the iTunes
Store at http://www.apple.com/support/itunes/musicstore/musiccard/.
If you require assistance beyond what is available online, please contact the
iTunes Store Team using the email forms available in the Help section.

Here is the link (at the bottom of the page there is a button) https://developer.apple.com/support/ios/enrollment.html When I click on it, I get redirected to the previous page https://developer.apple.com/contact/
Here is a link that sent me an e-mail to track order status. (Order No. 2442140328)https://secure2.store.apple.com/us/order/guest/2442140328/119607
I understand that it is not necessary to speak the information, but I'm in a desperate situation.

Need Some help in Developing an ALV report ..Plz help me

Hi Experts I am basic learner to ABAP Here I need some help in developing a Delivery *** Invoice Report....Please help me by spending a little time..
Tables are VBAK VBAP LIPS LIKP   and Document floe table is VBFA
SELECT VBELN VKORG VTWEG SPART
    FROM VBAK
    INTO TABLE I_VBAK
    WHERE VBELN IN S_VBELN.
IF I_VBAK IS NOT INITIAL .
    SELECT VBELN POSNR MATKL POSAR WERKS
      FROM VBAP
      INTO TABLE I_VBAP
      FOR ALL ENTRIES IN I_VBAK
      WHERE VBELN = I_VBAK-VBELN.
ENDIF.
IF I_VBAP IS NOT INITIAL.
    SELECT * FROM LIPS
      INTO CORRESPONDING FIELDS OF TABLE I_LIPS
      WHERE VGBEL = VBAP-VBELN
      AND VGPOS = VBAP-POSNR.
ENDIF.
IF I_LIPS IS NOT INITIAL.
    SELECT VBELN VSTEL VKORG KUNNR
    FROM LIKP
    INTO TABLE I_LIKP.
ENDIF.
Moderator message : Outsourcing is not allowed, don't expect others to correct your source code. Thread locked.
Edited by: Vinod Kumar on Aug 1, 2011 5:43 PM

hi ,
Look in this link
<a href="http://help.sap.com/saphelp_nw04/helpdata/en/b3/0ef3e8396111d5b2e80050da4c74dc/frameset.htm">http://help.sap.com/saphelp_nw04/helpdata/en/b3/0ef3e8396111d5b2e80050da4c74dc/frameset.htm</a>
<a href="http://help.sap.com/saphelp_nw04/helpdata/en/21/894eeee0b911d4b2d90050da4c74dc/content.htm">http://help.sap.com/saphelp_nw04/helpdata/en/21/894eeee0b911d4b2d90050da4c74dc/content.htm</a>
<a href="http://help.sap.com/saphelp_nw04/helpdata/en/80/1a62bfe07211d2acb80000e829fbfe/content.htm">http://help.sap.com/saphelp_nw04/helpdata/en/80/1a62bfe07211d2acb80000e829fbfe/content.htm</a>
Regards
Renjith Kumar

How to get the previous record value in the current record plz help me...

In my sql how to get the previous record value...
in table i m having the field called Date i want find the difference b/w 2nd record date value with first record date... plz any one help me to know this i m waiting for ur reply....
Thanx in Advance
with regards
kotresh

First of this not hte mysql or database forum so don;t repeate again.
to get diff between two date in mysql use date_format() to convert them to date if they r not date type
then use - (minus)to get diff.

Need Color from specific coord in canvas! plz help

not much more to say.. im a bloody newbie and cant seem to find the proper method to get the color of a single point in a canvas. thx very much for trying to help me..

If you have a new class extending Canvas, overide the paint method:
public void paint(Graphics g)
g.setColor(Color.red);
g.fillRect(30,30,1,1); // a red dot
}

Need an Oracle migration plan template or Sample;;;Plz Help Me

Hi all,
We prepare a migration from One database server to another. Both Databases have the same versions (10g) on Linux Servers and 're Large databases (greater than 200 GB of data).
My Boss ask me to prepare a migartion plan Data Sheet and I'm new in this kind of projects ;
Someone can help guide me and send me a template or plan doc migration sample just to know from where I should start.
thanks a lot
Message was edited by:
OracleJavaLinux

hello,
- 200 Go of data ; network load.
- New Server in France and Old one in US. (network traffic load)
- Old one 's on production and it's impossible to have many hours in Downtime.
- Don't have enougth disk space on the old server (the US one) (just only 14GO)
- Impossible to add disk on the old one since it's an old machine and this machine 's no more in grantee provider.
My only solution in my case 's an export / import. What could be the best procedure, in my case, to do this??. I'm not a senior DBA and Sometimes DBA should solve the company and protect its business , sould fix the conception errors of the others ,,,and it's my responsability in this case :))
Message was edited by:
OracleJavaLinux

I have been trying to restore my ipod on itunes but a lil box always pops up saying that my ipod software update server cannot be contacted what do i do i need help i feel like i tryed evrything plz help???

i been trying to restore my ipod on itunes but a lil box always pops up saying that my ipod software update server cannot contacted i feel like i tryed everything idk what to do i need help

Try:
- Powering off and then back on your router.
-iTunes for Windows: iTunes cannot contact the iPhone, iPad, or iPod software update server
- Change the DNS to either Google's or Open DNS servers
Public DNS — Google Developers
OpenDNS IP Addresses
- Try on another computer/network
- Wait if it is an Apple problem

Need code for the following chess game..plz help me out

Hi,
Problem:Move the chess piece "KNIGHT" from any location on a "3 x 3" Chess Board and make it go to the far right hand bottom corner.Chess Board in the problem is not the usual Chess Board of 8 x 8.KNIGHT starting position may be any position on board.Program should exit when knight moves to 3 x 3 corner.Here is how my Chess board looks.
|.... |.... |.... |
|.... |.... |.... |
|.... |.... |Queen |<<--The Queen should reach this position
Remember: KNIGHT moves in specific way such as 2 steps in one direction and 1 step left/right.If the KNIGHT starts are position (2,2) then it cannot move further and you have to throw exception with some error message.
Run command: "java <some class> x y", where x is x-coordinate and y is y-coordinate(starting position of Knight) on the chess board. For this problem x & y could be 1(min) and 3(max) values and any value(2) in between.
Deliver : 1) Send me the java code for above problem.
2) Java code should compile and run.
3) I will be interested in "Object Oriented Design" thaught process.
4) Your coding style
5) Javadoc
6) Makefile
I would be more interested in your class design, interface design and error handling.
please mail the solution to [email protected]
thanks n regards
Chinu

Hi, Here Knight moves like horse. Queen doesn't
exist. If knight moves to (3,3). Game will be over.
I am facing problem to move knight from one point
to another point. And also i have some confussion on
selecting interfaces and classes.<font size="10000">do your own homework</font>

URGENT ! Help to printing contents of a panel (Plz Help me)

hi guys,
i developed a application which consists of lot of information on a panel, i am using print utilities and printing the panel but its printing only the visible part of the panel and its printing a single page. is there any way i can print the whole contents of the panel
i really appreciate if anyone can help me out.
import java.awt.*;
import javax.swing.*;
import java.awt.print.*;
public class PrintUtilities implements Printable
private Component componentToBePrinted;
public static void printComponent(Component c)
new PrintUtilities(c).print();
public PrintUtilities(Component componentToBePrinted)
this.componentToBePrinted = componentToBePrinted;
public void print() {
PrinterJob printJob = PrinterJob.getPrinterJob();
printJob.setPrintable(this);
if (printJob.printDialog())
try
printJob.print();
} catch(PrinterException pe) {
System.out.println("Error printing: " + pe);
public int print(Graphics g, PageFormat pageFormat, int pageIndex)
if (pageIndex > 0)
return(NO_SUCH_PAGE);
else
Graphics2D g2d = (Graphics2D)g;
g2d.translate(pageFormat.getImageableX(), pageFormat.getImageableY());
disableDoubleBuffering(componentToBePrinted);
componentToBePrinted.paint(g2d);
enableDoubleBuffering(componentToBePrinted);
return(PAGE_EXISTS);
/** The speed and quality of printing suffers dramatically if
* any of the containers have double buffering turned on.
* So this turns if off globally.
* @see enableDoubleBuffering
public static void disableDoubleBuffering(Component c) {
RepaintManager currentManager = RepaintManager.currentManager(c);
currentManager.setDoubleBufferingEnabled(false);
/** Re-enables double buffering globally. */
public static void enableDoubleBuffering(Component c) {
RepaintManager currentManager = RepaintManager.currentManager(c);
currentManager.setDoubleBufferingEnabled(true);

not exactly so...
implementing Pageable means imlpementing methods that return the number of pages, PageFormat object, and Printable object for each page... look in the API for this interface. It's actually quite a lot of work to do, as you have to divide your panel programmatically on "good" - suitable for your purposes rectangles of the page size, then cretae printale object for each ... but that can be done.
I'm not that familiar with java printing, perhaps there is a much easier way to do it. Anyway, when I had to print one component on several pages, that's what I did - so it works.

I cant find my ipod plz help me, i cant find my ipod plz help me

so one stole it and they have it but wonht tell the real thing. so help me use locator to find my ipod

- If you previously turned on FIndMyiPod on the iPod in Settings>iCloud and wifi is on and connected go to iCloud: Find My iPhone, sign in and go to FIndMyiPhone. If the iPod has been restored it will never show up.
- You can also wipe/erase the iPod and have to iPod play a sound via iCloud.
- Change the passwords for all accounts used on the iPod and report to police
- There is no way to prevent someone from restoring the iPod (it erases it) using it.
- Apple will do nothing
Reporting a lost or stolen Apple product
- iOS: How to find the serial number, IMEI, MEID, CDN, and ICCID number

Need help / Advice ; manage daily millions of records;;plz help me:)

Similar Messages

Maybe you are looking for