What is the threshold to decide whether partitioning is needed or not?

Question

I have an oracle 10g enterprise edition but partitioning option is not purchased yet. I foresee the total amount of data by the end of year 2012 would be about 200 to 300 GB and by end of year 2013 will be 400 GB, year 2014 will be 500 GB. There may be four or five tables with about 2 million records (and these will be the biggest tables).
I have read that partitioning is definitely needed when the size of the database exceeds 500 GB. Are there any other criteria/threshold which suggests at what point in time the partitioning option needs to be recommended?

Answer 1

Hi,
whether or not you need partitioning depends not only on your data size, but mostly on your data structure. If you can think of "units" or "chunks" that your data can be broken in, and your business needs are such that they could be satisfied by one or a few "chunks", then partitioning may be the solution for you. For example, if your main application table stores data chronologically, and you often need to wipe off data for one month, or move it to cheaper storage, or take it offline for some kind of maintenance. Or if your data is distributed more or less uniformly across 20 different locations, and your users are often interested in seeing aggregate reports for just one location, etc.
If your data doesn't have a natural partitioning key, and cannot be broken into convenient "chunks" by this key, then not only partitioning may not improve your performance, it can make it worse.
There is a common misconception about partitioning: people tend to think of partitions as a way to complement indexing to get better selectivity. E.g. you have a table with columns a (indexed) and b (not indexed), and you have a report using WHERE a=:a and b>:b and think "ok, let me range-partition my table on b, then my report will run even faster with the combined power of indexing and partition pruning". In reality, your report will be running slower, maybe even much slower, depending on how many partitions will be covered by b>:b, because instead of doing an index unique scan once, you'll be doing it once per every index partition in your partition range. Of course, global indexes don't have this problem, but they sort of cancel the advantages of partitioning (what good is being able to operate on a small chunk of data if you have to follow up by rebuilding a huge global index?).
Another common misconception is urge to partition just because a table is becoming "too big". I've seen people doing partitioning on some obscure syntetic keys only used in table joins explaining that "well I know this is not a good partitioning key but I had to partition this table, it was becoming oh so big". In best case scenario, performance won't improve, but very likely it will get worse.
Best regards,
Nikolay