Azure SQL Federations Retired - Will Table Partitions replace?
My latest project was designed around Azure SQL Federations. Needless the frustration of the this feature leaving is high, I can't imagine the people that have already coded their projects around it.
With Azure SQL Federations gone, is table partitioning going to be in the road map soon?
Hi Mendokusai,
The features of SQL Server that are typically used for partitioning data in on-premises solutions include: Replication, Table Partitioning, Partitioned Views, Distributed Partitioned Views, File group Strategies, and Cross-database queries (Distributed Queries).
Azure SQL Database does not support all these features except Partitioned Views. Azure SQL Database does offer a scale out solution, which horizontally partitions data across multiple databases.
Azure SQL Database supports two methods to implement partitions, one is custom sharding, another is federations.
The current implementation of Federations will be retired with Web and Business service tiers. Consider deploying custom sharding solutions to maximize scalability, flexibility, and performance.
For more information, you can review the following articles.
http://msdn.microsoft.com/library/azure/dn495641.aspx
http://msdn.microsoft.com/en-us/library/jj156171.aspx
Regards,
Sofiya Li
Sofiya Li
TechNet Community Support
Similar Messages
-
Azure SQL Data Sync, LINQ optimization bug
How can I report the following Bug ?
var empty = new Set<int>();
var query = storage.table.Where(item => empty.Contains(item.id));
// Entity Framework Extended
query.DeleteAsync()
The query is optimized to something like this, because the set is empty, and therefore the result set is always empty:
SELECT
CAST(NULL AS uniqueidentifier) AS [C1],
CAST(NULL AS uniqueidentifier) AS [C2],
CAST(NULL AS int) AS [C3],
CAST(NULL AS int) AS [C4],
CAST(NULL AS datetime2) AS [C5],
CAST(NULL AS varchar(1)) AS [C6],
CAST(NULL AS int) AS [C7]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
WHERE 1 = 0
Now the problem is that update and delete trigger, introduced by Azure SQL Data Sync, will fail because they require the primary key, which is not part of the optimized query. Meaning that when the set is empty the triggers will fail, but if the set contains
an id, the query is not optimized and therefore will succeed.
(SELECT [i].[PrimaryKeyId] FROM DELETED AS [i])
Notice, the problem here is not Entity Framework Extended from my perspective, but the Azure SQL Data Sync Triggers!not sure if i understood your post correctly, but the columns available in the logical inserted and deleted tables available in the trigger are the actual columns of the table itself, not the columns in the query that caused the trigger to fire.
-
PL/SQL- Problem in creating a partitioned fact table using select as syntax
Hi All,
I am trying to create a clone(mdccma.fact_pax_bkng_t) of existing fact table (mdccma.fact_pax_bkng) using dynamic pl/sql. However, pl/sql anonymous block errors out with following error:
SQL> Connected.
SQL> SQL> DECLARE
ERROR at line 1:
ORA-00911: invalid character
ORA-06512: at "SYS.DBMS_SYS_SQL", line 1608
ORA-06512: at "SYS.DBMS_SQL", line 33
ORA-06512: at line 50
Here is pl/sql block:
-- CREATING FPB_T
DECLARE
v_owner VARCHAR2(32) := 'MDCCMA';
v_table_original VARCHAR2(32) := 'FACT_PAX_BKNG';
v_table VARCHAR2(32) := 'FACT_PAX_BKNG_T';
v_tblspc VARCHAR2(32) := v_owner||'_DATA';
CURSOR c_parts IS SELECT TABLESPACE_NAME, PARTITION_NAME,HIGH_VALUE, ROW_NUMBER() OVER (ORDER BY PARTITION_NAME) AS ROWNUMBER
FROM USER_TAB_PARTITIONS
WHERE TABLE_NAME = v_table_original
ORDER BY PARTITION_NAME;
v_cmd CLOB := EMPTY_CLOB();
v_cmd3 varchar2(300) := 'CREATE TABLE ' ||v_owner||'.'||v_table||' TABLESPACE '||v_tblspc
||' NOLOGGING PARTITION BY RANGE'||'(' ||'SNAPSHOT_DTM '||')' ||'(';
v_part VARCHAR2(32);
v_tblspc_name VARCHAR2(32);
v_row number;
v_value LONG;
v_tmp varchar2(20000);
v_cur INTEGER;
v_ret NUMBER;
v_sql DBMS_SQL.VARCHAR2S;
v_upperbound NUMBER;
BEGIN
v_cmd := v_cmd3;
OPEN c_parts;
FETCH c_parts INTO v_tblspc_name, v_part,v_value, v_row;
WHILE c_parts%FOUND
LOOP
IF (v_row = 1) THEN
v_tmp := ' PARTITION '||v_part||' VALUES LESS THAN ' ||'('|| v_value||')'||' NOLOGGING TABLESPACE '||v_tblspc_name;
ELSE
v_tmp := ', PARTITION '||v_part||' VALUES LESS THAN ' ||'('|| v_value||')'||' NOLOGGING TABLESPACE '||v_tblspc_name;
END IF;
v_cmd := v_cmd || v_tmp;
-- DBMS_OUTPUT.PUT_LINE(v_cmd);
FETCH c_parts INTO v_tblspc_name, v_part,v_value, v_row;
END LOOP;
-- DBMS_OUTPUT.PUT_LINE('Length:'||DBMS_LOB.GETLENGTH(v_cmd));
v_cmd := v_cmd||')'||' AS SELECT ' || '*'||' FROM ' || v_owner||'.'|| v_table_original ||' WHERE '||'1'||'='||'2'||';';
v_upperbound := CEIL(DBMS_LOB.GETLENGTH(v_cmd)/256);
FOR i IN 1..v_upperbound
LOOP
v_sql(i) := DBMS_LOB.SUBSTR(v_cmd
,256 -- amount
,((i-1)*256)+1 -- offset
END LOOP;
v_cur := DBMS_SQL.OPEN_CURSOR;
DBMS_SQL.PARSE(v_cur, v_sql, 1, v_upperbound, FALSE, DBMS_SQL.NATIVE);
v_ret := DBMS_SQL.EXECUTE(v_cur);
CLOSE c_parts;
DBMS_OUTPUT.PUT_LINE(v_cmd);
-- EXECUTE IMMEDIATE v_cmd ;
END;
The above pl/sql creates a DDL for partitioned fact table(new) based on an existing fact table and get executes through CLOB.
Please look into the issue and let me know any changes or modifications/suggestions that are required to fix the issue. Any help is appreciated.
Thank You,
SudheerThink this is your problem:
v_cmd := v_cmd||')'||' AS SELECT ' || '*'||' FROM ' || v_owner||'.'|| v_table_original ||' WHERE '||'1'||'='||'2'||';';Remove the SQL terminator ';' ... dynamic SQL doesn't require it, try this instead:
v_cmd := v_cmd||')'||' AS SELECT ' || '*'||' FROM ' || v_owner||'.'|| v_table_original ||' WHERE '||'1'||'='||'2';Thanks
Paul -
Does sql server could export a partition from a partition table ?
Dear :
Does sql server could export a partition from a partition table ?
For example, I need to export all old partition,which is '2013' and drop them. It is easy in oracle. but how to do with sql
server 2012 ?where do you want it to be exported to? Another server instance? if yes you can do piecemeal restore if partition is in a separate filegroup and you're on full recovery model
http://msdn.microsoft.com/en-IN/library/ms177425.aspx
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs -
Failing to create new SQL farm with FSconfig.exe for ADFS federation Services on an Azure SQL
I'm building ADFS in Azure. My plan is to use the Azure SQL. The problem is that FSconfig seams not to work with Azure SQL. Here's what i did untill now:
- I followed the documentation and created a new login and user (adfslogin, adfsuser) in a new
"adfsv21" database. Then i added the user to db_owner role.
- From c:\windows\adfs i run FSconfig: FSConfig.exe CreateSQLFarm /ServiceAccount mysa /ServiceAccountPassword ******** /SQLConnectionString "database=Adfsv21;server=[azureServer];User Id=mysa;Password=********" /AutoCertRolloverEnabled. I
tried it with both accounts, mysa and adfsuser with the same poor results.
The result is: "The following error occurred: The specified service account mysa is not valid. Specify
a different service account, and try again."
- I configured odbc and tested the connection. mysa can connect but adfsuser caanot. Why?
To be short, its not working. See this post with similar problems: google to "SQL server farm configuration error"...
So, is Azure SQL & ADFS not complatible? Really not?My apologies for take a super long time to reply. Are you still having issues here? How can we help?
Thanks Guy -
Connecting to an Azure SQL Database from a Windows C# Form
Hi,
Is it possible to connect to an SQL database in Azure from Visual Studio C# 2010 or 2012? If so how is this done. Is it the same as if you were connecting to SQL Express database with the use of a connection string? Any directions or tutorials would be of
great help!
Thanks!Hi,
>>Thanks for the reply Paul, That is disappointing. I am trying to get my students to create a C# application that will connect with a database in Azure. Is there another approach besides SQL that I could use?
Using SQL Azure will work fine (see below), if the students are familiar with SQL Server and SQL Databases it will be great for them to see how it works in the cloud. As an alternative you could explore Azure Storage, using Table Storage, Blob Storage or
the new Document Database features.
>>Azure enforces a firewall that prevents direct external access to the database server. You can add a firewall rule allowing external access, but the rules only support connecting from a fixed IP address because you have to specify the numeric IP
from which connection will be allowed.
You can specify ranges here, so you could cover a range of IP addresses. You can even specify 0.0.0.0 to 255.255.255.255 and allow all IP addresses, this could be an option if you don't know where the students will be connecting from, and you are using a
database for the exercise where you don't care too much about the security.
When I have run classes I typically use an IP address range, for example XXX.YYY.ZZZ.0 to XXX.YYY.ZZZ.255 if the students are accessing from the same classroom, and the external IP address may vary. You may find that they all have the same external IP address
if they are on the same internal network.
Also, with many students accessing the same database, you may get throttling if they place a lot of load on it. This is cool, as you can also teach them about transient fault handling and resource throttling :-).
Regards,
Alan
Free e-book: Windows Azure Service Bus Developer Guide. -
How do I set up Azure Sql Server to have multiple instances?
Hi all;
We are setting up a new SAAS application on Azure and while we have used Azure before, not at this level of multiple apps, etc.
We plan to have both a web app and a cloud app in both a US and EU data center. They need to hit a common database because requests will go to the closest data center via traffic manager and a lot of the data works off of the customer table. And a customer
(company) can have users in both the U.S. and E.U.
Is there a way to set up Azure Sql Server so it has instances in both data centers, and Azure keeps them synchronized? If I understand sharding right, that is not what we need as someone hitting either data center could be requesting any of the data in the
DB.
thanks - dave
What we did for the last 6 months -
Made the world's coolest reporting & docgen system even more amazingHi,
As far as I am aware, when you set the webapp/cloudapp to connect to a database you will have to specify the connection string in the app. The app will hit the database which is mentioned in the connection string you have specified. So it is not automatic
or Traffic manager. You have full control over it as you can tell your app to connect whichever database you want.The database could be located in any region, however the you have to keep latency in mind as the app in US Datacenter could be calling the Database
in EU, if that is they way it is setup.
You can have separate databases in each region and use Azure Data Sync
to have the database synchronized, however please keep in mind that Azure Data Sync is in Preview.
Regards,
Mekh. -
Automatic table partitioning in Oracle 11g
Hi All,
I need to implement automatic table partitioning in Oracle 11g version, but partitioning interval should be on daily basis(For every day).
I was able to perform this for Monthly and Yearly but not on daily basis.
create table part
(a date)PARTITION BY RANGE (a)
INTERVAL (NUMTOYMINTERVAL(1,'*MONTH*'))
(partition p1 values less than (TO_DATE('01-NOV-2007','DD-MON-YYYY'))
Table created
create table part
(a date)PARTITION BY RANGE (a)
INTERVAL (NUMTOYMINTERVAL(1,'*YEAR*'))
(partition p1 values less than (TO_DATE('01-NOV-2007','DD-MON-YYYY'))
Table createdBut if i use DD or DAY instead of YEAR or MONTH it fails......Please suggest me how to perform this on daily basis.
SQL>
1 create table part
2 (a date)PARTITION BY RANGE (a)
3 INTERVAL (NUMTOYMINTERVAL(1,'*DAY*'))
4 (partition p1 values less than (TO_DATE('01-NOV-2007','DD-MON-YYYY'))
5* )
SQL> /
INTERVAL (NUMTOYMINTERVAL(1,'DAY'))
ERROR at line 3:
ORA-14752: Interval expression is not a constant of the correct type
SQL> create table part
(a date)PARTITION BY RANGE (a)
INTERVAL (NUMTOYMINTERVAL(1,'*DD*'))
(partition p1 values less than (TO_DATE('01-NOV-2007','DD-MON-YYYY'))
); 2 3 4 5
INTERVAL (NUMTOYMINTERVAL(1,'DD'))
ERROR at line 3:
ORA-14752: Interval expression is not a constant of the correct typePlease suggest me to resolve this ORA-14752 error for using DAY or DD or HH24
-YasserYes, for differenct partitions for different months.
interval (numtoyminterval(1,'MONTH'))
store in (TS1,TS2,TS3)
This code will store data in partitions in tablespaces TS1, TS2, and TS3 in a round robin manner.
for Day wise day yes you can store
INTERVAL (NUMTODSINTERVAL(1,'day')) or
INTERVAL (NUMTODSINTERVAL(2,'day')) or
INTERVAL (NUMTODSINTERVAL(3,'day')) or
INTERVAL (NUMTODSINTERVAL(4,'day')) or
INTERVAL (NUMTODSINTERVAL(5,'day')) or
INTERVAL (NUMTODSINTERVAL(n,'day')) -
What is the best way to dynamically create table partition by year and month based on a date column?
Hi,
I have a huge table and it will keep growing. I have a date column in this table and thought of partition the table by year and month. Can any you suggest better approach so that partition will create automatically for new data also along with the existing
data? Nothing but automatically/dynamically partition should create along with file group and partition files.
Thanks in advance!
Palash Also this one
http://weblogs.sqlteam.com/dang/archive/2008/08/30/Sliding-Window-Table-Partitioning.aspx
Best Regards,Uri Dimant SQL Server MVP,
http://sqlblog.com/blogs/uri_dimant/
MS SQL optimization: MS SQL Development and Optimization
MS SQL Consulting:
Large scale of database and data cleansing
Remote DBA Services:
Improves MS SQL Database Performance
SQL Server Integration Services:
Business Intelligence -
Can't write right sql query by two tables
Hello
Everyone,
I am trying to get a sql query by two tables,
table:container
<pre class="jive-pre">
from_dest_id number
ship_from_desc varchar
to_dest_id number
</pre>
table: label_fromat (changeless)
<pre class="jive-pre">
SORT_ORDER number
PREFIX varchar2
VARIABLE_NAME varchar2
SUFFIX varchar2
LF_COMMENT varchar2
</pre>
the sql which i need is
a. table CONTAINER 's each column should have LABLE_FORMAT 's PREFIX before and SUFFIX back ,and these columns is connected
example : the query output should be like this :
<pre class="jive-pre">
PREFIX||from_dest_id||SUFFIX ||PREFIX||ship_from_desc||SUFFIX ||PREFIX|| to_dest_id||SUFFIX
</pre>
every PREFIX and SUFFIX are come from LABEL_FORMAT's column VARIABLE_NAME (they are different)
column SORT_ORDER decide the sequence, for the example above: Column from_dest_id order is 1, ship_from_desc is 2,to_dest_id is 3
b. table LABEL_FORMAT's column VARIABLE_NAME have values ('from_dest_id','ship_from_desc','to_dest_id')
If table CONTAINER only have one record i can do it myself,
But actually it is more than one record,I do not know how to do
May be this should be used PL/SQL,or a Function ,Cursor ,Procedure
I am not good at these
Any tips will be very helpful for me
Thanks
SavenHi, Saven,
Presenting data from multiple rows as a single string is called String Aggregation . This page:
http://www.oracle-base.com/articles/10g/StringAggregationTechniques.php
shows many ways to do it, suited to different requirements, and different versions of Oracle.
In Oracle 10 (and up) you can do this:
SELECT REPLACE ( SYS_CONNECT_BY_PATH ( f.prefix || ' '
|| CASE f.variable_name
WHEN 'FROM_DEST_ID'
THEN from_dest_id
WHEN 'SHIP_FROM_DESC'
THEN ship_from_desc
WHEN 'TO_DEST_ID'
THEN to_dest_id
END
|| ' '
|| f.suffix
, '~?'
, '~?'
) AS output_txt
FROM container c
CROSS JOIN label_format f
WHERE CONNECT_BY_ISLEAF = 1
START WITH f.sort_order = 1
CONNECT BY f.sort_order = PRIOR f.sort_order + 1
AND c.from_dest_id = PRIOR c.from_dest_id
saven wrote:If table CONTAINER only have one record i can do it myself,
But actually it is more than one record,I do not know how to do In that case, why did you post an example that only has one row in container?
The query above assumes
something in container (I used from_dest_id in the example above) is unique,
you don't mind having a space after prefix and before from_dest_id,
you want one row of output for every row in container, and
you can identify some string ('~?' in the example above) that never occurs in the concatenated data. -
Sliding Window Table Partitioning Problems with RANGE RIGHT, SPLIT, MERGE using Multiple File Groups
There is misleading information in two system views (sys.data_spaces & sys.destination_data_spaces) about the physical location of data after a partitioning MERGE and before an INDEX REBUILD operation on a partitioned table. In SQL Server 2012 SP1 CU6,
the script below (SQLCMD mode, set DataDrive & LogDrive variables for the runtime environment) will create a test database with file groups and files to support a partitioned table. The partition function and scheme spread the test data across
4 files groups, an empty partition, file group and file are maintained at the start and end of the range. A problem occurs after the SWITCH and MERGE RANGE operations, the views sys.data_spaces & sys.destination_data_spaces show the logical, not the physical,
location of data.
--=================================================================================
-- PartitionLabSetup_RangeRight.sql
-- 001. Create test database
-- 002. Add file groups and files
-- 003. Create partition function and schema
-- 004. Create and populate a test table
--=================================================================================
USE [master]
GO
-- 001 - Create Test Database
:SETVAR DataDrive "D:\SQL\Data\"
:SETVAR LogDrive "D:\SQL\Logs\"
:SETVAR DatabaseName "workspace"
:SETVAR TableName "TestTable"
-- Drop if exists and create Database
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL
BEGIN
ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE
DROP DATABASE $(DatabaseName)
END
CREATE DATABASE $(DatabaseName)
ON
( NAME = $(DatabaseName)_data,
FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',
SIZE = 10,
MAXSIZE = 500,
FILEGROWTH = 5 )
LOG ON
( NAME = $(DatabaseName)_log,
FILENAME = N'$(LogDrive)$(DatabaseName).ldf',
SIZE = 5MB,
MAXSIZE = 5000MB,
FILEGROWTH = 5MB ) ;
GO
-- 002. Add file groups and files
--:SETVAR DatabaseName "workspace"
--:SETVAR TableName "TestTable"
--:SETVAR DataDrive "D:\SQL\Data\"
--:SETVAR LogDrive "D:\SQL\Logs\"
DECLARE @nSQL NVARCHAR(2000) ;
DECLARE @x INT = 1;
WHILE @x <= 6
BEGIN
SELECT @nSQL =
'ALTER DATABASE $(DatabaseName)
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';
ALTER DATABASE $(DatabaseName)
ADD FILE
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'
EXEC sp_executeSQL @nSQL;
SET @x = @x + 1;
END
-- 003. Create partition function and schema
--:SETVAR TableName "TestTable"
--:SETVAR DatabaseName "workspace"
USE $(DatabaseName);
CREATE PARTITION FUNCTION $(TableName)_func (int)
AS RANGE RIGHT FOR VALUES
0,
15,
30,
45,
60
CREATE PARTITION SCHEME $(TableName)_scheme
AS
PARTITION $(TableName)_func
TO
$(TableName)_fg1,
$(TableName)_fg2,
$(TableName)_fg3,
$(TableName)_fg4,
$(TableName)_fg5,
$(TableName)_fg6
-- Create TestTable
--:SETVAR TableName "TestTable"
--:SETVAR BackupDrive "D:\SQL\Backups\"
--:SETVAR DatabaseName "workspace"
CREATE TABLE [dbo].$(TableName)(
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_scheme(Partition_PK)
) ON $(TableName)_scheme(Partition_PK)
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]
-- 004. Create and populate a test table
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key
--:SETVAR TableName "TestTable"
SET NOCOUNT ON;
DECLARE @Now DATETIME = GETDATE()
WHILE @Now > DATEADD(minute,-1,GETDATE())
BEGIN
INSERT INTO [dbo].$(TableName)
([Partition_PK]
,[RandomNbr])
VALUES
DATEPART(second,GETDATE())
,ROUND((RAND() * 100),0)
END
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
--=================================================================================
-- SECTION 2 - SWITCH OUT
-- 001 - Create TestTableOut
-- 002 - Switch out partition in range 0-14
-- 003 - Merge range 0 -29
-- 001. TestTableOut
:SETVAR TableName "TestTable"
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL
DROP TABLE [dbo].[$(TableName)Out]
CREATE TABLE [dbo].[$(TableName)Out](
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_fg2;
GO
-- 002 - Switch out partition in range 0-14
--:SETVAR TableName "TestTable"
ALTER TABLE dbo.$(TableName)
SWITCH PARTITION 2 TO dbo.$(TableName)Out;
-- 003 - Merge range 0 - 29
--:SETVAR TableName "TestTable"
ALTER PARTITION FUNCTION $(TableName)_func()
MERGE RANGE (15);
-- Confirm table partitioning
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.
The T-SQL code below illustrates the problem.
-- PartitionLab_RangeRight
USE workspace;
DROP TABLE dbo.TestTableOut;
USE master;
ALTER DATABASE workspace
REMOVE FILE TestTable_f3 ;
-- ERROR
--Msg 5042, Level 16, State 1, Line 1
--The file 'TestTable_f3 ' cannot be removed because it is not empty.
ALTER DATABASE workspace
REMOVE FILE TestTable_f2 ;
-- Works surprisingly!!
use workspace;
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2;
--Msg 622, Level 16, State 3, Line 2
--The filegroup "TestTable_fg2" has no files assigned to it. Tables, indexes, text columns, ntext columns, and image columns cannot be populated on this filegroup until a file is added.
--The statement has been terminated.
If you run ALTER INDEX REBUILD before trying to remove files from File Group 3, it works. Rerun the database setup script then the code below.
-- RANGE RIGHT
-- Rerun PartitionLabSetup_RangeRight.sql before the code below
USE workspace;
DROP TABLE dbo.TestTableOut;
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2;
USE master;
ALTER DATABASE workspace
REMOVE FILE TestTable_f3;
-- Works as expected!!
The file in File Group 2 appears to contain data but it can be dropped. Although the system views are reporting the data in File Group 2, it still physically resides in File Group 3 and isn’t moved until the index is rebuilt. The RANGE RIGHT function means
the left file group (File Group 2) is retained when splitting ranges.
RANGE LEFT would have retained the data in File Group 3 where it already resided, no INDEX REBUILD is necessary to effectively complete the MERGE operation. The script below implements the same partitioning strategy (data distribution between partitions)
on the test table but uses different boundary definitions and RANGE LEFT.
--=================================================================================
-- PartitionLabSetup_RangeLeft.sql
-- 001. Create test database
-- 002. Add file groups and files
-- 003. Create partition function and schema
-- 004. Create and populate a test table
--=================================================================================
USE [master]
GO
-- 001 - Create Test Database
:SETVAR DataDrive "D:\SQL\Data\"
:SETVAR LogDrive "D:\SQL\Logs\"
:SETVAR DatabaseName "workspace"
:SETVAR TableName "TestTable"
-- Drop if exists and create Database
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL
BEGIN
ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE
DROP DATABASE $(DatabaseName)
END
CREATE DATABASE $(DatabaseName)
ON
( NAME = $(DatabaseName)_data,
FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',
SIZE = 10,
MAXSIZE = 500,
FILEGROWTH = 5 )
LOG ON
( NAME = $(DatabaseName)_log,
FILENAME = N'$(LogDrive)$(DatabaseName).ldf',
SIZE = 5MB,
MAXSIZE = 5000MB,
FILEGROWTH = 5MB ) ;
GO
-- 002. Add file groups and files
--:SETVAR DatabaseName "workspace"
--:SETVAR TableName "TestTable"
--:SETVAR DataDrive "D:\SQL\Data\"
--:SETVAR LogDrive "D:\SQL\Logs\"
DECLARE @nSQL NVARCHAR(2000) ;
DECLARE @x INT = 1;
WHILE @x <= 6
BEGIN
SELECT @nSQL =
'ALTER DATABASE $(DatabaseName)
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';
ALTER DATABASE $(DatabaseName)
ADD FILE
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'
EXEC sp_executeSQL @nSQL;
SET @x = @x + 1;
END
-- 003. Create partition function and schema
--:SETVAR TableName "TestTable"
--:SETVAR DatabaseName "workspace"
USE $(DatabaseName);
CREATE PARTITION FUNCTION $(TableName)_func (int)
AS RANGE LEFT FOR VALUES
-1,
14,
29,
44,
59
CREATE PARTITION SCHEME $(TableName)_scheme
AS
PARTITION $(TableName)_func
TO
$(TableName)_fg1,
$(TableName)_fg2,
$(TableName)_fg3,
$(TableName)_fg4,
$(TableName)_fg5,
$(TableName)_fg6
-- Create TestTable
--:SETVAR TableName "TestTable"
--:SETVAR BackupDrive "D:\SQL\Backups\"
--:SETVAR DatabaseName "workspace"
CREATE TABLE [dbo].$(TableName)(
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_scheme(Partition_PK)
) ON $(TableName)_scheme(Partition_PK)
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]
-- 004. Create and populate a test table
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key
--:SETVAR TableName "TestTable"
SET NOCOUNT ON;
DECLARE @Now DATETIME = GETDATE()
WHILE @Now > DATEADD(minute,-1,GETDATE())
BEGIN
INSERT INTO [dbo].$(TableName)
([Partition_PK]
,[RandomNbr])
VALUES
DATEPART(second,GETDATE())
,ROUND((RAND() * 100),0)
END
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
--=================================================================================
-- SECTION 2 - SWITCH OUT
-- 001 - Create TestTableOut
-- 002 - Switch out partition in range 0-14
-- 003 - Merge range 0 -29
-- 001. TestTableOut
:SETVAR TableName "TestTable"
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL
DROP TABLE [dbo].[$(TableName)Out]
CREATE TABLE [dbo].[$(TableName)Out](
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_fg2;
GO
-- 002 - Switch out partition in range 0-14
--:SETVAR TableName "TestTable"
ALTER TABLE dbo.$(TableName)
SWITCH PARTITION 2 TO dbo.$(TableName)Out;
-- 003 - Merge range 0 - 29
:SETVAR TableName "TestTable"
ALTER PARTITION FUNCTION $(TableName)_func()
MERGE RANGE (14);
-- Confirm table partitioning
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.
The data in the File and File Group to be dropped (File Group 2) has already been switched out; File Group 3 contains the data so no index rebuild is needed to move data and complete the MERGE.
RANGE RIGHT would not be a problem in a ‘Sliding Window’ if the same file group is used for all partitions, when they are created and dropped it introduces a dependency on full index rebuilds. Larger tables are typically partitioned and a full index rebuild
might be an expensive operation. I’m not sure how a RANGE RIGHT partitioning strategy could be implemented, with an ascending partitioning key, using multiple file groups without having to move data. Using a single file group (multiple files) for all partitions
within a table would avoid physically moving data between file groups; no index rebuild would be necessary to complete a MERGE and system views would accurately reflect the physical location of data.
If a RANGE RIGHT partition function is used, the data is physically in the wrong file group after the MERGE assuming a typical ascending partitioning key, and the 'Data Spaces' system views might be misleading. Thanks to Manuj and Chris for a lot of help
investigating this.
NOTE 10/03/2014 - The solution
The solution is so easy it's embarrassing, I was using the wrong boundary points for the MERGE (both RANGE LEFT & RANGE RIGHT) to get rid of historic data.
-- Wrong Boundary Point Range Right
--ALTER PARTITION FUNCTION $(TableName)_func()
--MERGE RANGE (15);
-- Wrong Boundary Point Range Left
--ALTER PARTITION FUNCTION $(TableName)_func()
--MERGE RANGE (14);
-- Correct Boundary Pounts for MERGE
ALTER PARTITION FUNCTION $(TableName)_func()
MERGE RANGE (0); -- or -1 for RANGE LEFT
The empty, switched out partition (on File Group 2) is then MERGED with the empty partition maintained at the start of the range and no data movement is necessary. I retract the suggestion that a problem exists with RANGE RIGHT Sliding Windows using multiple
file groups and apologize :-)Hi Paul Brewer,
Thanks for your post and glad to hear that the issue is resolved. It is kind of you post a reply to share your solution. That way, other community members could benefit from your sharing.
Regards.
Sofiya Li
Sofiya Li
TechNet Community Support -
Fact and dimension table partition
My team is implementing new data-warehouse. I would like to know that when should we plan to do partition of fact and dimension table, before data comes in or after?
Hi,
It is recommended to partition Fact table (Where we will have huge data). Automate the partition so that each day it will create a new partition to hold latest data (Split the previous partition into 2). Best practice is to create partition on transaction
timestamps so load the incremental data into a empty table called (Table_IN) and then Switch that data into main table (Table). Make sure your tables (Table and Table_IN) should be on one file group.
Refer below content for detailed info
Designing and Administrating Partitions in SQL Server 2012
A popular method of better managing large and active tables and indexes is the use of partitioning. Partitioning is a feature for segregating I/O workload within
SQL Server database so that I/O can be better balanced against available I/O subsystems while providing better user response time, lower I/O latency, and faster backups and recovery. By partitioning tables and indexes across multiple filegroups, data retrieval
and management is much quicker because only subsets of the data are used, meanwhile ensuring that the integrity of the database as a whole remains intact.
Tip
Partitioning is typically used for administrative or certain I/O performance scenarios. However, partitioning can also speed up some queries by enabling
lock escalation to a single partition, rather than to an entire table. You must allow lock escalation to move up to the partition level by setting it with either the Lock Escalation option of Database Options page in SSMS or by using the LOCK_ESCALATION option
of the ALTER TABLE statement.
After a table or index is partitioned, data is stored horizontally across multiple filegroups, so groups of data are mapped to individual partitions. Typical
scenarios for partitioning include large tables that become very difficult to manage, tables that are suffering performance degradation because of excessive I/O or blocking locks, table-centric maintenance processes that exceed the available time for maintenance,
and moving historical data from the active portion of a table to a partition with less activity.
Partitioning tables and indexes warrants a bit of planning before putting them into production. The usual approach to partitioning a table or index follows these
steps:
1. Create
the filegroup(s) and file(s) used to hold the partitions defined by the partitioning scheme.
2. Create
a partition function to map the rows of the table or index to specific partitions based on the values in a specified column. A very common partitioning function is based on the creation date of the record.
3. Create
a partitioning scheme to map the partitions of the partitioned table to the specified filegroup(s) and, thereby, to specific locations on the Windows file system.
4. Create
the table or index (or ALTER an existing table or index) by specifying the partition scheme as the storage location for the partitioned object.
Although Transact-SQL commands are available to perform every step described earlier, the Create Partition Wizard makes the entire process quick and easy through
an intuitive point-and-click interface. The next section provides an overview of using the Create Partition Wizard in SQL Server 2012, and an example later in this section shows the Transact-SQL commands.
Leveraging the Create Partition Wizard to Create Table and Index Partitions
The Create Partition Wizard can be used to divide data in large tables across multiple filegroups to increase performance and can be invoked by right-clicking
any table or index, selecting Storage, and then selecting Create Partition. The first step is to identify which columns to partition by reviewing all the columns available in the Available Partitioning Columns section located on the Select a Partitioning Column
dialog box, as displayed in Figure 3.13. This screen also includes additional options such as the following:
Figure 3.13. Selecting a partitioning column.
The next screen is called Select a Partition Function. This page is used for specifying the partition function where the data will be partitioned. The options
include using an existing partition or creating a new partition. The subsequent page is called New Partition Scheme. Here a DBA will conduct a mapping of the rows selected of tables being partitioned to a desired filegroup. Either a new partition scheme should
be used or a new one needs to be created. The final screen is used for doing the actual mapping. On the Map Partitions page, specify the partitions to be used for each partition and then enter a range for the values of the partitions. The
ranges and settings on the grid include the following:
Note
By opening the Set Boundary Values dialog box, a DBA can set boundary values based on dates (for example, partition everything in a column after a specific
date). The data types are based on dates.
Designing table and index partitions is a DBA task that typically requires a joint effort with the database development team. The DBA must have a strong understanding
of the database, tables, and columns to make the correct choices for partitioning. For more information on partitioning, review Books Online.
Enhancements to Partitioning in SQL Server 2012
SQL Server 2012 now supports as many as 15,000 partitions. When using more than 1,000 partitions, Microsoft recommends that the instance of SQL Server have at
least 16Gb of available memory. This recommendation particularly applies to partitioned indexes, especially those that are not aligned with the base table or with the clustered index of the table. Other Data Manipulation Language statements (DML) and Data
Definition Language statements (DDL) may also run short of memory when processing on a large number of partitions.
Certain DBCC commands may take longer to execute when processing a large number of partitions. On the other hand, a few DBCC commands can be scoped to the partition
level and, if so, can be used to perform their function on a subset of data in the partitioned table.
Queries may also benefit from a new query engine enhancement called partition elimination. SQL Server uses partition enhancement automatically if it is available.
Here’s how it works. Assume a table has four partitions, with all the data for customers whose names begin with R, S, or T in the third partition. If a query’s WHERE clause
filters on customer name looking for ‘System%’, the query engine knows that it needs only to partition three to answer
the request. Thus, it might greatly reduce I/O for that query. On the other hand, some queries might take longer if there are more than 1,000 partitions and the query is not able to perform partition elimination.
Finally, SQL Server 2012 introduces some changes and improvements to the algorithms used to calculate partitioned index statistics. Primarily, SQL Server 2012
samples rows in a partitioned index when it is created or rebuilt, rather than scanning all available rows. This may sometimes result in somewhat different query behavior compared to the same queries running on SQL Server 2012.
Administrating Data Using Partition Switching
Partitioning is useful to access and manage a subset of data while losing none of the integrity of the entire data set. There is one limitation, though. When
a partition is created on an existing table, new data is added to a specific partition or to the default partition if none is specified. That means the default partition might grow unwieldy if it is left unmanaged. (This concept is similar to how a clustered
index needs to be rebuilt from time to time to reestablish its fill factor setting.)
Switching partitions is a fast operation because no physical movement of data takes place. Instead, only the metadata pointers to the physical data are altered.
You can alter partitions using SQL Server Management Studio or with the ALTER TABLE...SWITCH
Transact-SQL statement. Both options enable you to ensure partitions are
well maintained. For example, you can transfer subsets of data between partitions, move tables between partitions, or combine partitions together. Because the ALTER TABLE...SWITCH statement
does not actually move the data, a few prerequisites must be in place:
• Partitions must use the same column when switching between two partitions.
• The source and target table must exist prior to the switch and must be on the same filegroup, along with their corresponding indexes,
index partitions, and indexed view partitions.
• The target partition must exist prior to the switch, and it must be empty, whether adding a table to an existing partitioned table
or moving a partition from one table to another. The same holds true when moving a partitioned table to a nonpartitioned table structure.
• The source and target tables must have the same columns in identical order with the same names, data types, and data type attributes
(length, precision, scale, and nullability). Computed columns must have identical syntax, as well as primary key constraints. The tables must also have the same settings for ANSI_NULLS and QUOTED_IDENTIFIER properties.
Clustered and nonclustered indexes must be identical. ROWGUID properties
and XML schemas must match. Finally, settings for in-row data storage must also be the same.
• The source and target tables must have matching nullability on the partitioning column. Although both NULL and NOT
NULL are supported, NOT
NULL is strongly recommended.
Likewise, the ALTER TABLE...SWITCH statement
will not work under certain circumstances:
• Full-text indexes, XML indexes, and old-fashioned SQL Server rules are not allowed (though CHECK constraints
are allowed).
• Tables in a merge replication scheme are not allowed. Tables in a transactional replication scheme are allowed with special caveats.
Triggers are allowed on tables but must not fire during the switch.
• Indexes on the source and target table must reside on the same partition as the tables themselves.
• Indexed views make partition switching difficult and have a lot of extra rules about how and when they can be switched. Refer to
the SQL Server Books Online if you want to perform partition switching on tables containing indexed views.
• Referential integrity can impact the use of partition switching. First, foreign keys on other tables cannot reference the source
table. If the source table holds the primary key, it cannot have a primary or foreign key relationship with the target table. If the target table holds the foreign key, it cannot have a primary or foreign key relationship with the source table.
In summary, simple tables can easily accommodate partition switching. The more complexity a source or target table exhibits, the more likely that careful planning
and extra work will be required to even make partition switching possible, let alone efficient.
Here’s an example where we create a partitioned table using a previously created partition scheme, called Date_Range_PartScheme1.
We then create a new, nonpartitioned table identical to the partitioned table residing on the same filegroup. We finish up switching the data from the partitioned table into the nonpartitioned table:
CREATE TABLE TransactionHistory_Partn1 (Xn_Hst_ID int, Xn_Type char(10)) ON Date_Range_PartScheme1 (Xn_Hst_ID) ; GO CREATE TABLE TransactionHistory_No_Partn (Xn_Hst_ID int, Xn_Type
char(10)) ON main_filegroup ; GO ALTER TABLE TransactionHistory_Partn1 SWITCH partition1 TO TransactionHistory_No_Partn; GO
The next section shows how to use a more sophisticated, but very popular, approach to partition switching called a sliding
window partition.
Example and Best Practices for Managing Sliding Window Partitions
Assume that our AdventureWorks business is booming. The sales staff, and by extension the AdventureWorks2012 database, is very busy. We noticed over time that
the TransactionHistory table is very active as sales transactions are first entered and are still very active over their first month in the database. But the older the transactions are, the less activity they see. Consequently, we’d like to automatically group
transactions into four partitions per year, basically containing one quarter of the year’s data each, in a rolling partitioning. Any transaction older than one year will be purged or archived.
The answer to a scenario like the preceding one is called a sliding window partition because
we are constantly loading new data in and sliding old data over, eventually to be purged or archived. Before you begin, you must choose either a LEFT partition function window or a RIGHT partition function window:
1. How
data is handled varies according to the choice of LEFT or RIGHT partition function window:
• With a LEFT strategy, partition1 holds the oldest data (Q4 data), partition2 holds data that is 6- to 9-months old (Q3), partition3
holds data that is 3- to 6-months old (Q2), and partition4 holds recent data less than 3-months old.
• With a RIGHT strategy, partition4 holds the holds data (Q4), partition3 holds Q3 data, partition2 holds Q2 data, and partition1
holds recent data.
• Following the best practice, make sure there are empty partitions on both the leading edge (partition0) and trailing edge (partition5)
of the partition.
• RIGHT range functions usually make more sense to most people because it is natural for most people to to start ranges at their lowest
value and work upward from there.
2. Assuming
that a RIGHT partition function windows is used, we first use the SPLIT subclause of the ALTER PARTITION FUNCTIONstatement
to split empty partition5 into two empty partitions, 5 and 6.
3. We
use the SWITCH subclause
of ALTER TABLE to
switch out partition4 to a staging table for archiving or simply to drop and purge the data. Partition4 is now empty.
4. We
can then use MERGE to
combine the empty partitions 4 and 5, so that we’re back to the same number of partitions as when we started. This way, partition3 becomes the new partition4, partition2 becomes the new partition3, and partition1 becomes the new partition2.
5. We
can use SWITCH to
push the new quarter’s data into the spot of partition1.
Tip
Use the $PARTITION system
function to determine where a partition function places values within a range of partitions.
Some best practices to consider for using a slide window partition include the following:
• Load newest data into a heap, and then add indexes after the load is finished. Delete oldest data or, when working with very large
data sets, drop the partition with the oldest data.
• Keep an empty staging partition at the leftmost and rightmost ends of the partition range to ensure that the partitions split when
loading in new data, and merge, after unloading old data, do not cause data movement.
• Do not split or merge a partition already populated with data because this can cause severe locking and explosive log growth.
• Create the load staging table in the same filegroup as the partition you are loading.
• Create the unload staging table in the same filegroup as the partition you are deleting.
• Don’t load a partition until its range boundary is met. For example, don’t create and load a partition meant to hold data that is
one to two months older before the current data has aged one month. Instead, continue to allow the latest partition to accumulate data until the data is ready for a new, full partition.
• Unload one partition at a time.
• The ALTER TABLE...SWITCH statement
issues a schema lock on the entire table. Keep this in mind if regular transactional activity is still going on while a table is being partitioned.
Thanks Shiven:) If Answer is Helpful, Please Vote -
Update in table partition by hash
I have a table partitioned by hash. I want to make some updates (not on the partition key column and also not based on this column, i.e this column is not used in where clause).
Do I need to specify partition name in the query?
For a concrete example:
CREATE TABLE invoices
(invoice_id NUMBER NOT NULL,
customer_id NUMBER NOT NULL,
invoice_date DATE NOT NULL,
comments VARCHAR2(500))
PARTITION BY HASH (customer_id)
PARTITIONS 10
primary key is customer_id + invoice_date
I need to update invoice_id based on the rowid, like update invoices set invoice_id = MEMO_ID_1SQ.nextval where rowid = <variable>. I will be running this query from multiple processes and I want to make sure they are hitting different partitions, so they actually run in parallel.
Thanks in advance,
RaduRadu,
You can use parallel hint on your update statement.
update /*+ PARALLEL(invoices, 4) */ invoices set invoice_id = MEMO_ID_1SQ.nextval where rowid = <variable>;See following execution plan for both with and without using parallel option
SQL> update interfacerecords set status=1 where interfaceid=5;
200000 rows updated.
Elapsed: 00:00:03.75
Execution Plan
Plan hash value: 3154550297
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | UPDATE STATEMENT | | 200K| 3125K| 128 (3)| 00:00:02 |
| 1 | UPDATE | INTERFACERECORDS | | | | |
|* 2 | TABLE ACCESS FULL| INTERFACERECORDS | 200K| 3125K| 128 (3)| 00:00:02 |
Predicate Information (identified by operation id):
2 - filter("INTERFACEID"=5)
Statistics
262 recursive calls
458353 db block gets
1539 consistent gets
378 physical reads
91750476 redo size
655 bytes sent via SQL*Net to client
585 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
200000 rows processed
SQL> commit;
Commit complete.
Elapsed: 00:00:00.00
SQL> set autotrace traceonly;
SQL> set timi on;
SQL> set lines 400;
SQL> update /*+ PARALLEL(interfacerecords, 4) */ interfacerecords set status=5 where status=1;
200000 rows updated.
Elapsed: 00:00:02.48
Execution Plan
Plan hash value: 2940696107
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
| 0 | UPDATE STATEMENT | | 1 | 13 | 35 (0)| 00:00:01 | | | |
| 1 | UPDATE | INTERFACERECORDS | | | | | | | |
| 2 | PX COORDINATOR | | | | | | | | |
| 3 | PX SEND QC (RANDOM)| :TQ10000 | 1 | 13 | 35 (0)| 00:00:01 | Q1,00 | P->S | QC (RAND) |
| 4 | PX BLOCK ITERATOR | | 1 | 13 | 35 (0)| 00:00:01 | Q1,00 | PCWC | |
|* 5 | TABLE ACCESS FULL| INTERFACERECORDS | 1 | 13 | 35 (0)| 00:00:01 | Q1,00 | PCWP | |
Predicate Information (identified by operation id):
5 - filter("STATUS"=1)
Statistics
309 recursive calls
214609 db block gets
30287 consistent gets
0 physical reads
63356488 redo size
654 bytes sent via SQL*Net to client
617 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client
3 sorts (memory)
0 sorts (disk)
200000 rows processedRegards -
Hi,
our database contains XML documents in 11 languages, one table for each language. Besides, there is meta information about each document which is the same for each language. Therefore, this meta information has been placed in a separate table.
Now to the problem: common database queries search in the documents (Intermedia) as well
in the meta data, what means that there is a join between the table that contains the documents (columns: key, documenttext) and the table that contains the meta information (columns: key and 18 others). Unfortunately after having located documents via Intermedia index, Oracle reads the corresponding data blocks for extracting the key. Because the documents are up to 4K, Oracle reads much information that it does not need. Is it possible to have a vertical partition of the documents table in order to be able to efficiently read only the key information instead of the whole table rows?
Thank You for any hint,
MarkusThanks For your comment and sorry for delay my Internet got down yesterday. Well, As you said throw away existing table structure.. i agree with you but there is a limitation right now. Our application only understand the current structure. Any change in database result to change execution engine. Actually, Our application provide interface to do Ad hoc reporting. we certainly use dimensional modeling but unfortunately our CTO believe existing structure is much better than the dimensional model. it gives the flexibility to load any data like retail, click stream etc and provide reporting on data. Although, the architecture is bad but they we providing analytic to customer....
To cut the long story short, they have assigned me a task to improve its performance. i have already improve it performance by using different feature of oracle like table partitioning sub partitioning, indexing and many more... but we want more... i have observed during the data loading process the Update part took much time. if i could do this any other way it would reduced the overall data loading time.. we extract the file from client server and perform transformation according to business rule and out the result of transformation file in flat file.. finally our application use sql loader to load in flat table then divide it in different table. Some of our existing client have 250 column some are 350 plus...
i think i have explained much but if you have more question on it i will ready to answer them. Please suggest with your experience how could i improve performance of existing system. -
SQL statement for selecting multiple partitions
Hi All,
May i know how to Select data from a table having multiple partitions ?
Example : Owner = Scott, Table= Emp, Partitions (P1,P2,P3,P4,P5)
Thanksno...
SQL>
1 create table partition_test
2 (owner, object_name, object_id)
3 partition by list(owner)
4 (partition part_1 values ('SYS'),
5 partition part_2 values ('SYSTEM'),
6 partition part_3 values ('OUTLN')
7 ) as select owner, object_name, object_id from all_objects
8* where owner in ('SYS', 'SYSTEM', 'OUTLN')
9 /
Table created.
SQL> select count(*) from partition_test partition(part_1, part_2);
select count(*) from partition_test partition(part_1, part_2)
ERROR at line 1:
ORA-00933: SQL command not properly ended
SQL> select count(*) from partition_test partitions(part_1, part_2);
select count(*) from partition_test partitions(part_1, part_2)
ERROR at line 1:
ORA-00933: SQL command not properly ended
SQL> select count(*) from partition_test partitions(part_1 + part_2);
select count(*) from partition_test partitions(part_1 + part_2)
ERROR at line 1:
ORA-00933: SQL command not properly ended
SQL> select count(*) from partition_test partitions(part_1 between part_2
select count(*) from partition_test partitions(part_1 between part_2)
ERROR at line 1:
ORA-00933: SQL command not properly ended
SQL> select count(*) from partition_test partitions("part_1 part_2");
select count(*) from partition_test partitions("part_1 part_2")
ERROR at line 1:
ORA-00933: SQL command not properly endedand this is the documented behavior:
"partition_extension_clause
For PARTITION or SUBPARTITION, specify the name or key value of the partition or subpartition within table from which you want to retrieve data.
For range- and list-partitioned data, as an alternative to this clause, you can specify a condition in the WHERE clause that restricts the retrieval to one or more partitions of table. Oracle Database will interpret the condition and fetch data from only those partitions. It is not possible to formulate such a WHERE condition for hash-partitioned data."
from:
http://download.oracle.com/docs/cd/B28359_01/server.111/b28286/statements_10002.htm#i2076542
Amiel
Maybe you are looking for
-
Hi. The only save options for a full html type file on Firefox is as "web page, complete" and as I say when saving with Firefox the program is saving the usual yellow folder with links and artwork files etc, but is saving 2 copies of the HTML file wh
-
How do I find out what object is "instance 0xfc78f70"
At the point of, or after an exception, the debugger tells me there is a problem, such as: -[__NSCFArray length]: unrecognized selector sent to instance 0xfc78f70 *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[
-
Some literary agents want a chapter or two in the body of an email. They do NOT want an attachment. How do I do that?
-
Problem opening .PDF files in Safari and in MacBook :(
I'm having problems when I'm in Safari and I open a .PDF file this is what happens What is my Mac's problem?
-
Nokia C7's display suddenly dims out even in brigh...
I was using my C7 outdoors(in bright light) when I suddenly noticed that its display had totally dimmed out. Light sensor was not covered with anything. The other functions of the phone were working fine but the display stayed dim for approx 3 minute