Automated Database Partitioning

Hello,
I am looking for some insight on how to create a Perl script or Java program which would allow for automated partitioning of an Oracle 10g database. Presently, partitioning is being done manually and this process is taking quite some time to complete due to the size of the database. Therefore, it was suggested that perhaps creating a Perl script or Java program would serve to simplify this process. However, I'm fairly novice with writing Perl scripts and Java for the purposes of Oracle database administration, so any examples or guidance in this respect would be appreciated.
In addition, if there is a more efficient way to perform the task of automated partitioning via internal features of the 10g database, I'd like to pursue that route if possible.
Thanks in advance,
Scott.

After further analysis, I've decided that I would like to code a procedure, etc. in PL/SQL which can be sheduled via a UNIX crontab. With this in mind, I've included a "spec" of what I'd like to accomplish. I'm looking for some insight regarding the automation of table partitioning. Has anyone done this before and if so, am I on the right path? Any examples, advice would be appreciated.
Here's my "spec":
Several large tables need to be partitioned on a daily basis. The partitions are stored in their own tablespaces.
1. Create a table that will contain info about the partitioned tables.
Table Name: PARTITION_STAT
Columns: table_name, tablespace_name, partition_frequency (e.g. daily, weekly, etc), partition_name, max_partition_date
Note: This table will need to be udpated whenever a new table using partitioning is created.
2. Create a table that will contain the rowid of the last tablespace created.
Table Name: TABLESPACE_STAT
Column(s): last_tablespace_rowid
PL/SQL package requirements:
Need to create a procedure that will run automatically. The logic requirements are as follows:
1. Check to see if a new tablespace has been created. This is done by checking TABLESPACE_STAT.last_tablespace_rowid. All rowid's greater than last_tablespace_rowid are new tablespaces. Update the last_tablespace_rowid in the TABLESPACE_STAT table
2. Create partitions for the new tablespaces. For every new tablespace: create new partion and update the PARTITION_STAT table
3. Check to see if new tablespaces are required and send email notification to DBA if tablespaces are needed. This is accomplished by checking every max_partition_date and partition_frequency in PARTITION_STAT table. Send an email to DBA with list of tables that will require new tablespaces.
I realize that this is a very general overview of what I am looking to accomplish. Any assistance would be greatly appreciated.
Thanks and sorry for the long post.
Scott.

Similar Messages

  • Database partitions

    Hi,
    Please can anyone tell me the significance of data partitioning in terms of space issues?
    We have a table in our databse which will store upto 100 GB of data. This table is partitioned by range. The data from this table is not deleted anytime, but it keeps accumulating after every daily/monthly run.
    So my doubt is, how the data is maintained in this table in such huge amounts through database partitioning without posing any housekeeping problems? Also, if i want to take a certain amount of data from this table into a separate storage, say a disc, how database partitioning helps me in doing that?
    Thanks.

    Billy, I am really an amateur in this forum. I did not post it in multiple forums intentionally.
    First I posted both database partition and Business Objects queries in Database-General forum.
    When I did not get any responses there for some time, I decided to post Database partition in pl/sql forum and Business Objects in Objects forum.
    And if you notice, I have got the responses to my queries only after I posted them again in two different forums. This means I was in the wrong forum first time. Since I am using this forum newly, I was really not aware of the etiquettes to be followed here. And I apologise for that.
    But you should not assume everyone to be a prank. I am genuinely working in a Top Internaltional Software Consultancy Services company and I urgently needed the information on my query because of which I posted it again in another forum.
    I will keep in mind in future that I dont post queries in multiple forums. But it was really disappointing to see the culture of addressing people in this forum. Till now this was the only descent forum I came across where people are really serious. But it was really disappointing to address a mistake as a prank.

  • NW 7.3 specific - Database partitioning on top of logical partitioning

    Hello folks,
    In NW 7.3, I would like to know if it is possible to add a specific database partition rule on top of a logical partitioned cube. For example, if I have a LP cube by fiscal year - I would also like to specifically partition all generated cubes at DB level. I could not find any option in the GUI. In addition, each generated cube can be viewed only (cannot be changed in the GUI). Would anybody know if it is possible?
    Thank you
    Ioan

    Fair point! Let me explain more in details what I am looking for - in 7.0x, a cube can be partitioned at the DB level by fiscal period. Let's suppose my cube has only fiscal year 2011 data. If I partition the cube at the DB level by fiscal period in 12 buckets, I will get 12 distinct partitions (E table only) in the database. If the user runs a query on 06/2012, then the DB will search for the data only in the 06/2012 bucket - this is obviously faster than  browsing entire cube (even with indexes).
    In 7.3, cubes can be logical partitioned (LP). I created a LP by fiscal year - so far so good. Now, I would like to partition at the DB level each individual cube created by the LP. Right now I could not - this means that my fiscal year 2012 cube will have entire data residing in only 1 large partition, so a 06/2012 query will take longer (in theory).
    So my question is --> "Is it possible to partition a cube generated by a LP in fiscal period buckets"? I believe the answers is no right now (Dec 2011).
    By the way, all the above is true in a RDBMS environment - this is not a concern for BWA / HANA since data is column based and stored in RAM (not same technology as RDBMS).
    I hope this clarifies by question
    Thank you
    Ioan

  • Should I use Database Partitioning now ?

    Hi,
    I have an Employee table in an HR system, consist of data of 10 branch, total 3,5 million rows, size about 8-10 GB.
    Each branch only allowed to access data of their branch.
    BUT some department could access all data.
    Given this information, should I implement database partitioning on this table ?
    If yes, what are the reasons ?
    If No, also what are tke reasons ?
    Thank you for your help,
    xtanto

    Sometimes, Partitioning is not necessaryly increases performance as we thinks.
    We are you going to have DML regularly on these tables?
    Read the following
    http://asktom.oracle.com/pls/ask/f?p=4950:8:2270402190261165858::NO::F4950_P8_DISPLAYID,F4950_P8_CRITERIA:14188311731917

  • SQL Server CE 3.5 SP2 - SQLCE Toolbox / Automated Database Schema Upgrade Scripting / Drop Index

    I am provisioning for automated database schema upgrades in a Windows Forms Application (VS2010/VB) which uses SQL Server Compact 3.5 SP2.  A portion of the schema upgrade procedure uses TSQL commands passed to SQLCE like the following:
    cmd.CommandText = "DROP INDEX [SaleComps$ProjID]"
    cmd.ExecuteNonQuery()
    However, when this command is processed by the SQLCE engine, I get the following error:
    Command: DROP INDEX [SaleComps].[ProjID]
    Error: The specified index does not exist. [ ProjID ]
    But, if I try to recreate the index (or try to change the index column) I get the following errors which indicate that the index is still there:
    Command: ALTER TABLE [SaleComps] ALTER COLUMN [ProjID] nvarchar(20) NULL
    Error: Cannot alter a column that is part of a key or an index. [ Index Name = SaleComps$ProjID,Column = ProjID ]
    Command: CREATE INDEX "SaleComps$ProjID" ON "SaleComps" ("ProjID" ASC)
    Error: The specified index already exists. [ SaleComps$ProjID ]
    Any ideas why this would not be working? Thanks!
    -BGood

    No, I was guessing at the syntax from reviewing other web postings, some of which were in different SQL dialects. 
    While your SQLCE Toolbox did not provide the DROP INDEX command, it did provide most of the "Schema Diff" scripting commands.  In hindsight, maybe SQLCE Toolbox should recognize when schema differences involve a Column which is used in an index, and
    DROP the index before executing the ALTER COLUMN command, then re-CREATE the index with the modified column.
    Despite this shortcoming, I am very thankful for the help provided by SQLCE Toolbox, and learned a little more about SQL scripting in the process.
    Thanks, ErikEJ!
    -BGood

  • Database partitioning

    Hi All,
    database partitioning is for centralized enrollment or distributed or both?
    Is it reduce data that transfer from secondary storage to primary storage ?
    Many thanks

    Ayham wrote:
    database partitioning is for centralized enrollment or distributed or both?
    Is it reduce data that transfer from secondary storage to primary storage ?You need to clarify your question and put some context to it. Oracle has a feature called Partitioning Option and it deals with partitioning of tables. Oracle does not have a feature called database partitioning... which sounds more like a Hadoop kind of feature.
    Not does any of this have relevance to the SQL and PL/SQL languages within the basic context you asked your question. Database issues should be raised in {forum:id=61}.

  • Oracle Database Partitioning Feature on Azure VM

    Hi,
    we would like to know if Oracle Database partitioning feature is available and activable on Azure virtual machine with Oracle pre-installed database instance.
    Thanks

    Hi,
    Oracle Database 11g R2 EE with Advanced Options includes Partitioning.
    Create a virtual machine from here:, check this link:
    http://azure.microsoft.com/en-in/marketplace/partners/msopentech/oracle-db-11g-db-11g-ee-all/
    Regards,
    Azam khan

  • Automated Database Deployment

    Hi,
    I am working on automated database deployment. I have scripted all my database scripts and created a project and solution.
    If I want to implement the new environment from the script I can use msbuild/sqlpackage to apply the all my scripts into that environment.
    I just want to apply only changes into test/prod environment ie., changes like ALTER TABLE, ALTER PROCEDURE,ALTER VIEW to the test/prod environment.
    I am pretty well aware of Redgate SQL Source control explorer and don't want to use it. I looking for MS utilities which can do the same.
    Regards,
    kccrga
    -kccrga http://dbatrend.blogspot.com.au/

    I am worked on this topic for a while and found MS Build can perform everything that other external tools can do.
    MS Build is freeware and when we integrate it with Team city it does wonders!
    -kccrga http://dbatrend.blogspot.com.au/

  • Database Partitioning Problem

    hi
    i am to work out on transparent partition for one of our databases.
    During the partition set up, in the 1st step, i unchecked the Check box, so that my outline modifications should flow from both the sides, not exactly from source outline alone.
    but, once the outline is modified, say a member is added/deleted from the outline, in the partitioned area, may it be in the source/target database, and once i say "synchronise outline"
    a mesage pops up saying, both the outlines are in synchromisation but if i validate the same, it throws error as because the ouline are not really synchronised, either the source/target modified has extra members in the sharing/parition area.
    but during the 1st step, if the select the option, check the checkbox, saying the outline modifications flow in the same as dataflow, then outline sychronisation works but always, only source database is taken as refence and once i add a member in target partitioned database outline, it is getting deleted after synchronisation.
    please suggest me on this.
    thanks in advance

    Thanks. One question, if I export a table which will carry also its partitions. Is it possible, that i will drop the said table and recreate that said table but with a new partitions parameters then the backup that i made, i will import it to the newly created table with its new partition setup?
    Thanks,
    Hades

  • Automated Database dbsetup file does not exist in the 8.4 EPPM Installation media as suggested in documentation

    For a version 8.4 new installation the documentation states to run dbsetup from either the Professional OR EPPM media file.  The Professional media for EPPM has the dbsetup file in it, the EPPM media does not.  Is this intended?  Can I just run dbsetup from the Pro media for EPPM and it will create the correct database?  I've had issues with earlier versions doing that.
    This is all that is in the EPPM documentation:
    You can use either Oracle or Microsoft SQL Server on your database server for P6 EPPM installations. You must install the database server before you can create the P6 EPPM database.
    Note: When using an Oracle database, you must install Oracle Multimedia, along with these supporting components: Oracle JVM, Oracle XML DB (XDB), and XML Developer's Kit (XDK). Unless you specify otherwise, all these components automatically get installed with the latest versions of an Oracle database (11g and later). If you chose not to install these components, you will need to install them before you install P6 EPPM. See the Oracle Multimedia's User's Guide on the Oracle Technology Network (OTN) for information on how to install these components.
    PS:  I may have just answered it for myself - it appears that dbsetup is no longer the cmd being used, it is dbconfig instead.  I will post here if there an issues with this.
    Message was edited by: primaverawiz

    Hi,
    The dbsetup.bat exists in the PPM and EPPM media files. In PPM or professional client the path is:
    database\dbsetup.bat
    In EPPM:
    P6_R84\p6suite\database\
    To create an EPPM database, you must use the dbsetup.bat from the EPPM media folder (The one to install P6 Web). To create a PPM database, use the file located in the professional client media folder.
    Regards,
    Pablo
    ========
    Was our answer helpful?
    If so, highlight the information for other Community members by rating it or marking the answer as correct.
    Just hit the “Correct Answer” link on the answer, or rate it by clicking "Helpful Answer" or "Like".

  • Automating database creation

    Hiya, im trying to create a database of contacts from a website with about 3000 memebers. I want to write a java program which will 'click' on each member and read (and write to a txt file) each members name and email address (this info pops up into a new window), and once it gets to the end of page to move on to the next. Is this possible to do and if so whats the general structure?
    Thanks

    If your legitamate - wouldn't you have access to the websites database?
    Anyway - to do what you said you'd first download the source code, then parse it for the email addresses.
    For example if the source is similar to this:
    User1
    User2
    User3
    Your application would have to remove the and and split the other part - then it would do the same for 5654654.html.
    Sorry if I havn't explained it will. I dont actually know the code you would use for this just trying to explain the theory (well, my theory ;-))

  • Automating Database up & down when server start and shutdown in Linux

    Hi to all,
    Question:How to set in linux, whenever server is on the database up and server off the database down?
    Problem: whenever i am issue the command "chkconfig --add dbora"
    or "chkconfig --level 345 dbora on"
    The message is as follows
    service dbora does not support chkconfig.
    I use oracle 9i on Redhat linux as 3.
    please send reply as soon as possible

    http://tldp.org/HOWTO/Oracle-9i-RH8-and-RH9-HOWTO-5.html ??
    Maybe you can use this. Its probably also hidden somewhere inside manuals for unbreakable/redhat oracle linux on metalink. Dunno why they insist on NOT including this auto start/stop in installation - how hard can it be, if you know linux inside-out.... If they did, It should make them more comfortable competing with other rdbms vendors that dont object to making many such things working "out of the box" :) ..... (no pun intended)

  • Automating Database, Listener, Entr. Manager Startup & Shutdown in OEL5

    Hi All,
    bq. I am a quite newbie to Linux, I have installed Oracle 11g on Oracle Enterprise Linux 5 on VMWare by following [this article|http://www.oracle.com/technology/pub/articles/smiley-11gr1-install.html] everything went fine. But my problem is I need to explicitly start and shutdown database, listener & EM. Is there is anyway to automate this, start services on linux startup and stop them during linux shutdown. \\ If any linux script or links/articles available for this requirement, it will be very helpful to me?
    Thanks,
    Sumanth

    I am following the below steps
    1) edit the "/etc/oratab" file setting the restart flag for each instance to 'Y'.
    orcl:/u01/app/oracle/product/11.1.0:Y
    2) create a file called "/etc/init.d/dbora" as the root user, containing the following.
    #!/bin/sh
    # chkconfig: 345 99 10
    # description: Oracle auto start-stop script.
    # Set ORA_HOME to be equivalent to the $ORACLE_HOME
    # from which you wish to execute dbstart and dbshut;
    # Set ORA_OWNER to the user id of the owner of the
    # Oracle database in ORA_HOME.
    ORA_HOME=/u01/app/oracle/product/10.2.0/db_1
    #ORA_HOME=/u01/app/oracle/product/11.1.0/db_1
    ORA_OWNER=oracle
    if [ ! -f $ORA_HOME/bin/dbstart ]
    then
        echo "Oracle startup: cannot start"
        exit
    fi
    case "$1" in
        'start')
            # Start the Oracle databases:
            # The following command assumes that the oracle login
            # will not prompt the user for any values
            su - $ORA_OWNER -c "$ORA_HOME/bin/dbstart $ORA_HOME"
        'stop')
            # Stop the Oracle databases:
            # The following command assumes that the oracle login
            # will not prompt the user for any values
            su - $ORA_OWNER -c "$ORA_HOME/bin/dbshut $ORA_HOME"
    esac3) set the privileges to 750.
    chmod 750 /etc/init.d/dbora
    4) Associate the dbora service with the appropriate run levels and set it to auto-start using the following command.
    chkconfig --level 345 dbora on
    if am doing anything wrong do let me know
    thanks

  • Data Partitioning in Database

    Hi All,
    I have little bit confuse in data partitioning in database, I have read about it and i understand that there ate two type called vertical partitioning and another horizontal partitioning...
    I have three questions are
    1- is data partitioning used of in networks or can be in one PC?
    2- the data partitioning divided data that in table to partitions (groups) , according what ??? is to quantity or meaning of data that inside table?
    3- Is clustering that can be execute by Oracle using CTX_CLS.CLUSTERING type of it or partitioning not related to it?
    Please let any one give short answer about it.
    regards
    Dheya

    973907 wrote:
    I have little bit confuse in data partitioning in database, I have read about it and i understand that there ate two type called vertical partitioning and another horizontal partitioning...These are basic partitioning concepts. Vertical means taking something like a single 100 column row table, and splitting it up into 10 partitions/tables with 10 columns each (plus pk of course).
    Horizontal partitioning means taking a table like a year's invoices, and splitting that into 365 daily partitions, where each partition contains the invoice rows of for a specific day of the year.
    There is also other partitioning concepts like application partitioning, network partitioning (which happens to be a bad thing) and so on.
    I have three questions are
    1- is data partitioning used of in networks or can be in one PC?Neither. Database partitioning happens inside the database. The database uses different structures for storing data. It used hash tables, index organised tables, B+trees, bitmap indexes and so on. Partitioning is just another type of data structure, used by the database, for storing and managing data.
    2- the data partitioning divided data that in table to partitions (groups) , according what ??? is to quantity or meaning of data that inside table?That depends on the type of partitioning used. Oracle supports list, hash and range partitions. Each of these uses a different algorithm to determine what data goes into which partition. Each of these satisfy different partitioning requirements.
    3- Is clustering that can be execute by Oracle using CTX_CLS.CLUSTERING type of it or partitioning not related to it?Partitioning is a physical data structure and transparent to the user and application. The user and app see table EMPLOYEES. They do not need to know or understand the physical structure of the table. The table can be a hash table, an index organised table, a hash partitioned table - and the user and app will not even know that. It does not change the way the user and app uses the table, it does not make a difference to their SQL statements on the table.
    CTX_CLS is an interface for classifying text data/documents. It uses its own data structures to do that. It calls a collection of related text data, a cluster.
    This has nothing to do with the Oracle Partitioning Option feature.

  • Using partition in real time

    Hi i am new to essbase
    How we can use replicate and transperent partition option in the real time.
    kindly help me in understanding this concept.

    Hi,
    We will use transparent partition as an interface to read the data from the source cube
    A transparent partition allows users to manipulate their data that is stored in a target database as if it were part of the actual source database. The remote data is retrieved from the source database each time the users of the target database request it. Write backs to the target database also flow through back to the source database.
    We can use this technique for the improved maintenance of a large data bases by splitting them into smaller databases to allow the operations on the smaller databases and stiil facilitate the users as they see in a single data base.
    We can use this for the improved security maintenance.
    A replicated database partition copies a portion of the source database to be stored in a target database. Users can access the target database as if it were the source. The database administrator must occasionally refresh the target database from the source database.
    We can use this technique as a means of transferring data between the essbase cubes.
    I will explain you a classic example.
    In the current reporting cube we will maintain only two years data - previous year and current year data.
    As part of year end activity, we need to move the previous year data to the History cube and the same to be deleted from the current cube.
    we will create a replicated partition for the previous year b/w current and History cube.
    Replicate the data from current to History cube.
    drop the relicated partition and clear the prev year data in the current cube.
    I hope this will suffice.

Maybe you are looking for