Join between large tables

Hi,
It is likely that this question has already been asked in this forum. But I could not find such reference. So apologize for asking again.
I am using Oracle 11gR2 as my database. I have a table (say TABLE_X) containing nearly 440 million records (size about 22 GB). There is a requirement that TABLE_X has to be joined with TABLE_X itself on a column (say COLUMN_B). COLUMN_B has 13 million distinct values. The query looks roughly as below :
SELECT DISTINCT column_a, column_b, column_c
FROM table_x one, table_x two
WHERE one.column_b = two.column_b
There is nothing much complex about the query by it is simply not running. I have suggested proper housekeeping of the table (as it is a transaction table), but the application designer is against it.
What could be the most appropriate way to fetch the records faster ? I mean to ask if any kind of indexing or partitioning strategy will be helpful here. I would really appreciate any kind of help over here. Please feel free to ask if you require any further information.
Regards,
Mainak

user11183570 wrote:
Hi,
It is likely that this question has already been asked in this forum. But I could not find such reference. So apologize for asking again.
I am using Oracle 11gR2 as my database. I have a table (say TABLE_X) containing nearly 440 million records (size about 22 GB). There is a requirement that TABLE_X has to be joined with TABLE_X itself on a column (say COLUMN_B). COLUMN_B has 13 million distinct values. The query looks roughly as below :
SELECT DISTINCT column_a, column_b, column_c
FROM table_x one, table_x two
WHERE one.column_b = two.column_b
There is nothing much complex about the query by it is simply not running. I have suggested proper housekeeping of the table (as it is a transaction table), but the application designer is against it.
What could be the most appropriate way to fetch the records faster ? I mean to ask if any kind of indexing or partitioning strategy will be helpful here. I would really appreciate any kind of help over here. Please feel free to ask if you require any further information.
Regards,
MainakThread: HOW TO: Post a SQL statement tuning request - template posting
HOW TO: Post a SQL statement tuning request - template posting

Similar Messages

Join on Large Table

I have some queries that use an inner join between a table with a few hundred rows and a table that will eventually have many millions of rows. The join is on an integer value that is part of the primary key on the larger table. The primary key
on said table consists of the integer and another field which is a BigInt (representing Date/time to the millisecond). The query also has predicate (where clause) with an exact match for the BigInt.
The query take about a second to execute at the moment but I was wondering whether I should expect a large increase in execution time as the years go by.
Is an inner join on the large table advisable?
By the way, the first field in the primary key is the integer followed by the BigInt, so any thought of selecting on the BigInt into temp table before attempting the join probably won't help.
R Campbell

Just in case anyone wants to see the full picture (which I am not actually expecting) this is a script for all SQL objects involved.
The numbers of rows in the tables are.
Tags 5,000
NumericSamples millions (over time)
TagGroups 50
GroupTags 500
CREATE TABLE [dbo].[Tags](
[ID] [int] NOT NULL,
[TagName] [nvarchar](110) NOT NULL,
[Address] [nvarchar](80) NULL,
[DataTypeID] [smallint] NOT NULL,
[DatasourceID] [smallint] NOT NULL,
[Location] [nvarchar](4000) NULL,
[Properties] [nvarchar](4000) NULL,
[LastReadSampleTime] [bigint] NOT NULL,
[Archived] [bit] NOT NULL,
CONSTRAINT [Tags_ID_PK] PRIMARY KEY CLUSTERED
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[Tags] WITH NOCHECK ADD CONSTRAINT [Tags_DatasourceID_Datasources_ID_FK] FOREIGN KEY([DatasourceID])
REFERENCES [dbo].[Datasources] ([ID])
GO
ALTER TABLE [dbo].[Tags] CHECK CONSTRAINT [Tags_DatasourceID_Datasources_ID_FK]
GO
ALTER TABLE [dbo].[Tags] WITH NOCHECK ADD CONSTRAINT [Tags_DataTypeID_DataTypes_ID_FK] FOREIGN KEY([DataTypeID])
REFERENCES [dbo].[DataTypes] ([ID])
GO
ALTER TABLE [dbo].[Tags] CHECK CONSTRAINT [Tags_DataTypeID_DataTypes_ID_FK]
GO
ALTER TABLE [dbo].[Tags] ADD CONSTRAINT [DF_Tags_LastReadSampleTime] DEFAULT ((552877956000000000.)) FOR [LastReadSampleTime]
GO
ALTER TABLE [dbo].[Tags] ADD DEFAULT ((0)) FOR [Archived]
GO
CREATE TABLE [dbo].[NumericSamples](
[TagID] [int] NOT NULL,
[SampleDateTime] [bigint] NOT NULL,
[SampleValue] [float] NULL,
[QualityID] [smallint] NOT NULL,
CONSTRAINT [NumericSamples_TagIDSampleDateTime_PK] PRIMARY KEY CLUSTERED
[TagID] ASC,
[SampleDateTime] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[NumericSamples] WITH NOCHECK ADD CONSTRAINT [NumericSamples_QualityID_Qualities_ID_FK] FOREIGN KEY([QualityID])
REFERENCES [dbo].[Qualities] ([ID])
GO
ALTER TABLE [dbo].[NumericSamples] CHECK CONSTRAINT [NumericSamples_QualityID_Qualities_ID_FK]
GO
ALTER TABLE [dbo].[NumericSamples] WITH NOCHECK ADD CONSTRAINT [NumericSamples_TagID_Tags_ID_FK] FOREIGN KEY([TagID])
REFERENCES [dbo].[Tags] ([ID])
GO
ALTER TABLE [dbo].[NumericSamples] CHECK CONSTRAINT [NumericSamples_TagID_Tags_ID_FK]
GO
CREATE TABLE [dbo].[TagGroups](
[ID] [int] IDENTITY(1,1) NOT NULL,
[TagGroup] [varchar](50) NULL,
[Aggregates] [varchar](250) NULL,
[NumericData] [bit] NULL,
CONSTRAINT [PK_TagGroups] PRIMARY KEY CLUSTERED
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[TagGroups] ADD CONSTRAINT [DF_Tag_Groups_Aggregates] DEFAULT ('First') FOR [Aggregates]
GO
ALTER TABLE [dbo].[TagGroups] ADD CONSTRAINT [DF_TagGroups_NumericData] DEFAULT ((1)) FOR [NumericData]
GO
CREATE TABLE [dbo].[GroupTags](
[ID] [int] IDENTITY(1,1) NOT NULL,
[TagGroupID] [int] NULL,
[TagName] [varchar](150) NULL,
[ColumnName] [varchar](50) NULL,
[SortOrder] [int] NULL,
[TotalFactor] [float] NULL,
CONSTRAINT [PK_GroupTags] PRIMARY KEY CLUSTERED
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[GroupTags] WITH CHECK ADD CONSTRAINT [FK_GroupTags_TagGroups] FOREIGN KEY([TagGroupID])
REFERENCES [dbo].[TagGroups] ([ID])
ON UPDATE CASCADE
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[GroupTags] CHECK CONSTRAINT [FK_GroupTags_TagGroups]
GO
ALTER TABLE [dbo].[GroupTags] ADD CONSTRAINT [DF_GroupTags_TotalFactor] DEFAULT ((1)) FOR [TotalFactor]
GO
CREATE VIEW [dbo].[vw_GroupTags]
AS
SELECT TOP (10000) dbo.TagGroups.TagGroup AS TableName, dbo.TagGroups.Aggregates AS SortOrder, dbo.GroupTags.SortOrder AS TagIndex, dbo.GroupTags.TagName,
dbo.Tags.ID AS TagId, dbo.TagGroups.NumericData, dbo.GroupTags.TotalFactor, dbo.GroupTags.ColumnName
FROM dbo.TagGroups INNER JOIN
dbo.GroupTags ON dbo.TagGroups.ID = dbo.GroupTags.TagGroupID INNER JOIN
dbo.Tags ON dbo.GroupTags.TagName = dbo.Tags.TagName
ORDER BY SortOrder, TagIndex
CREATE procedure [dbo].[GetTagTableValues]
@SampleDateTime bigint,
@TableName varchar(50),
@PadRows int = 0
as
BEGIN
DECLARE @i int
DECLARE @ResultSet table(TagName varchar(150), SampleValue float, ColumnName varchar(50), SortOrder int, TagIndex int)
set @i = 0
INSERT INTO @ResultSet
SELECT vw_GroupTags.TagName, NumericSamples.SampleValue, vw_GroupTags.ColumnName, vw_GroupTags.SortOrder, vw_GroupTags.TagIndex
FROM vw_GroupTags INNER JOIN NumericSamples ON vw_GroupTags.TagId = NumericSamples.TagID
WHERE (vw_GroupTags.TableName = @TableName) AND (NumericSamples.SampleDateTime = @SampleDateTime)
set @i = @@ROWCOUNT
if @i < @PadRows
BEGIN
WHILE @i < @PadRows
BEGIN
INSERT @ResultSet (TagName, SampleValue, ColumnName, SortOrder, TagIndex) VALUES ('', NULL, '', 0, 0)
set @i = @i + 1
END
END
select TagName, SampleValue, ColumnName, SortOrder, TagIndex
from @ResultSet
END
R Campbell

How to provide joins between oracle tables and sql server tables

Hi,
I have a requirement that i need to generate a report form two different data base. i.e Oracle and Sql Server.
how to provide joins between oracle tables and sql server tables ? Any help on this
Regards,
Malli

user10675696 wrote:
I have a requirement that i need to generate a report form two different data base. i.e Oracle and Sql Server. Bad idea most times. Heterogeneous joins do not exactly scale and performance can be severely degraded by network speed and b/w availability. And there is nothing you can do in the application and database layers to address performance issue at the network level in this case - your code's performance is simply at the mercy of network performance. With a single glaring fact - network performance is continually degrading. All the time. Always. Until it is upgraded. When the performance degradation starts all over again.
If the tables are not small (few 1000 rows each) and row volumes static, I would not consider doing a heterogeneous join. Instead I would rather go for a materialised view on the Oracle side, use a proper table and index structure, and do a local database join.

[nQSError: 14044] Missing join between logical tables

Hi All,
I have three physical tables:
A- Dimension (Contact) B- Helper (Con-Prod) C- Dimension (Product). 'A' joins to facts.
Relationships are:
A:B=1:M and B:C=N:1
Currently a column of B table has been implemented as MLOV. As a result now I have one additional MLOV physical table 'D' that joins to 'B' (Since its B.MLOV_WID=D.MLOV_WID , its not a foreign key join).
The Logical Layer has three Logical tables: A, B and C as in the Physical Layer. B table has got one new LTS for 'D'.
Now the Problem is when I take a column that is sourced from D and another from C, it generates an error in Answers: Missing join between logical tables B and C. I have verified that the Logical and Physical joins exist.
I think this is something with the Logical level setup. So here's some more information on hierarchy setup. I have one Hierarchy (Contact) for all those Logical tables. I have setup Level for the MLOV column. I don't have level setup for most of the non-MLOV columns.
Can you please share your thoughts? I would like to avoid implicit join method.

Hi,
It seems like you are pulling a report from 2 tables with NO physical join.
Please check the physical layer diagram and join the 2 tables.
Thanks,
Vineeth

Sq02: define join between 2 tables as per your choice

Hi Team,
I am trying to create a join between 2 tables ekko & Lfa1. The sq02 tcode by default is joining it as
ekko-lblif = lfa1-lifnr in ecc 6.0
While in 4.7 i found that the same joint exists as
ekko-lifnr = lfa1-lifnr.
I want to create the joint as ekko-lifnr = lfa1-lifnr.
Can i do it ?
Please advice.

yes you can.
right click on the default link and delete it.. then start dragging from EKKO-LIFNR field to LFA-LIFNR. it will work.

Subject: How to do join between two tables using something like SE16

SE16, SE11 provide form based interface query information from a single table. Is there a way to do join between two tables without creating an infoset erc? I am looking for something similar to sql join but in SAP BI 7.0
Thanks.

Hi
Pls look into below links. Hope this helps you.
1. http://help.sap.com/saphelp_46c/helpdata/EN/d2/cb45bf455611d189710000e8322d00/content.htm
2. http://help.sap.com/saphelp_46c/helpdata/EN/d2/cb45a5455611d189710000e8322d00/content.htm
Regards
Sirigiri

Abap query, join between same tables

Hi,
I have an Abap Query (SQ01), I need to create a join between the same table (ESLL-ESLL) to obtain the services from a PO. The join is with the packno from ESLL to subpackno from ESLL (the same table). But I don't know how I can do that with Abap Query. Because the Infoset doesn't allow inserting the table two times.
Somebody can help me.
Thanks.
Victoria

Hi:
I was able to create a query to retrieve the service lines entries using tables ESSR (Header) (service entry sheet number as input parameter), linked to package number to view ML_ESLL and then from the view the sup-package number linked to ESLL. That way I was able to retrieve all the service lines information from table ESLL only using SQ02 and SQ01, no ABAP.
I Hope this help.
Juan
PS: I know the post is old but may be there are people out there with no ABAP access who needs to create reports for Service Entry Sheets lines. All the join conditions are.
Table             Table
ESSR            EKKO
ESSR            ML_ESLL
ML_ESLL      ESLL
ESLL             ESLH
Edited by: Juan Marino on Jan 23, 2012 10:53 PM

Doubt on creating join between two tables

Hi friends,
I have two tables , ekes and eine
my requirement is if Ekes-ebtyp = 'X1'.
then Ekes-ebtyp = EKES-EINDT + Eine-j_3alitra.
the doubt is there is no primary key common between this table . actually in my whole program for purchase order i used EBELN for extracting details when two tables are involved . in this two tables no two primary keys are common to extract data ...is there any ways please suggest..
the code i wrote is
loop at li_ekes assigning <fl_ekes>.
read table li_eine assigning <fl_eine> with table key (here i need to give some unique fields to join these two fields please guide me which field i can use.).
<fl_ekes>-ebtyp = <fl_ekes>-EINDT + <fl_eine>-j_3alitra.

Hi,
Both tables have field EBELN in common.
You can use it to extract data from both.
Hope this helps.
reward if helpful.
Regards,
Sipra

Outer join between logical tables

Hello,
This question is likely to be asked many times, but I failed to find the proper thread in the forum.
Assume there are 2 logical tables "Fact" and "Dim".
"Fact" has 1 LTS which consists of physical tables F, FX1, FX2 which are inner joined.
"Dim" has 1 LTS which consists of physical tables D, DX1, DX2 which are inner joined.
F and D tables are also joined together on physical layer.
I define left outer join between "Fact" and "Dim" on logical layer.
I create a request in Answers, querying columns from "Fact" and "Dim" which map to physical tables F and D only.
I expect OBIEE to build SQL query which uses F and D tables only, outer joined.
Instead of that all the physical tables used for logical tables "Fact" and "Dim" are joined together in a form like:
SELECT F.col, D.col
FROM (F inner join FX1 inner join FX2) left outer join (D inner join DX1 inner join DX2)
Is there any way to avoid this behavior or build data model in different way?
Thank you!

Hi Alex,
In BI Applications you never have a null in a facts foreign key to the dimension. Instead there is always a zero row wi record inserted with 'Unspecified' in all the columns. When the fact table is populated in the ETL, if the fact record doesn't have a corresponding dimension record the WID is poulated with zero.
This removes the problems with outer joins and helps considerably with performance.
This could be one practice you take from BI Apps and put into your own ETL's and data model.
Regards
Robin

Multiple foreign key joins between two tables

Hi,
I have a question about building a repository.
I have a date dimension and one fact. The fact table has about 10 foreign key columns that link to the date Dimension.
In this case should I create 10 aliases to create joins in the Physical and BMM layer or is there any other way to handle this situation.
I am asking this question because 10 aliases can get very confusing for me at the later point of time while creating reports.
Using OBIEE 10.1.3

Hi
I have a follow up question on this.
I am okay with not seeing the different date tables under the Subject area. Even if it just shows a it as a Simple DATE Dimension I am good with it.
In this case which is the efficient way, creating 10 aliases or creating 10 joins in the physical layer. I just figured out that we can create multiple joins between the same set of two tables but do not know how will that effect the way BI server works.
Please help me in understanding this concept.
thanks
This request id for OBIEE 10.1.3

How To Find joins between siebel tables

Hi Experts,
I am new in siebel and i am working on BI reports.
I need to write queries to fetch data from sibel database.
Any idea how do we find joins between table in siebel. Is there any user guide??
Cheers,
Andy

Hi Andy,
We can find Joins in Object Explorer. Expand the Business Component Object in Object Explorer and you can find Join. In Siebel generally we use only Equi Joins. ROW_ID of parent table will be stored in PAR_ROW_ID of child table. ROW_ID and PAR_ROW_ID are the columns of the table. Most of the tables have these two columns.
For Example: Take two tables S_CONTACT AND S_CONTACT_X, the ROW_ID of S_CONTACT will be stored in PAR_ROW_ID of S_CONTACT_X table.
If you want to join these two tables the query will be like
"SELECT * FROM S_CONTACT CON, S_CONTACT_X CONX WHERE CON.ROW_ID = CONX.PAR_ROW_ID"
This query will return all the records from S_CONTACT_X which matches S_CONTACT table.
Thanks & Regards,
Vinodhkumar

Join between 2 tables

All,
I am newbie to this forum.
A main query has a table say A along with other tables in a query. My requirement is to add a new table say D to fetch the records in the below fashion...
take grant_num from table A and join grant_num with table D and take deu_grant_num from same table D...Pass deu_grant_num from table D to table A and fetch the results with respect to the grant_num from table A...
Below are the table structures:
table A:
User_id, grant_num,...
Table B:
grant_num, deu_grant_num, user_id...
Appreciate ur help ASAP...

Try This Query
cursor abc is select grant_num from tableA
where grant_num in(select grant_num from tableB
where tableA.grant_num =tableB.grant_num)
begin
open abc;
ftech abc into :grant_num ;
close abc;
end;
or
select grant_num into grant_num from tableA
where grant_num in(select grant_num from tableB
where tableA.grant_num =tableB.grant_num)

Join between two tables in SAP ( actual sales vs budget sales)

I have one table with actual data sales in SAP .This table has a name   CE11000
This table contain amounts about : customers ,Products, Quantity of Sales ,amount of sales ,month witch did the sales and the year.
The other table from SAP has the budget sales . This table is CE21000
This table contain the same fields : customers ,Products, Quantity of Sales ,amount of sales ,month witch did the sales and the year but all this for budget .
In the budget table ( CE21000)there are no records for a particular make if it was not budgeted to make a sale.
Sometimes i have the same problem with the actual data table (CE11000) ...I have not records witch can use to join the results for a report witch take the comparison between actual sales and budget sales.
I would like to remind you that the report should have four "dimensions"
a) year
b) month
c) customer
d) type of product
I try to use left outer join with a customer fields in two tables but i have simultaneously the field with type of product .
I want to see in one year (for example february of 2011) for one customer the type of products (quantity and anmounts) witch buy (ACTUAL) and that to compare with the corresponding data if available
Any help would appreciate
thanks

Haggar
Your first query matches on two conditions BOTH being true. If there is a D and no matching E, you'll see the columns from D but none from E.
Your second query takes all Emplyees, and lines up 2 (possibly different) rows from D against each
E columns | D columns (match e.deptid=d.dept_id) | D columns (match ep_id=de_id)
So you may have any mix of:
E       | D1      | D2
E       | nothing | D2
nothing | D1      | D2
E       | D1      | nothing
E       | nothing | nothing
nothing | D1      | nothing
nothing | nothing | D2(NB I'm assuming FULL OUTER JOIN is commutative - ie the order is unimportant - but I may be wrong in which case the last row won't occur)
You will get a different number of rows (as well as different columns) in each case.
Try it!
Regards Nigel

Get highest value from 1 of the tables in a join between 2 tables

Hi,
i want to join 2 table:
- SALES
- MATERIAL
These table are joined by:
- Material ID
- Sales date -> start date
The problem is that it could be that 1 sales can have 2 records in material, but what i need to have is the latest record from material (highest date sequence number in material table)
I tried to aggregate and then put a max on sequence number, but that's not going to work. Does anyone have an idea how to get the highest number within a join?
regards,
Osman

Euh may be to speed.
Join it back first to your material and then to your sales table,
And also if you are in a SCD2, you can also simply made a simple select on the boolean column or on the end date.
Cheers
Nico

Join between fact table and master data table

Is it posible to join a Cube with a Characteristic? This is exactly what i need:
- In my cube i have date (0CALDAY) and (among others) a characteristic (ZCHAR) and a key figure (ZKEYF).
- I added a key figure (ZKFAT) as an attribute to ZCHAR. So the tables look something like this:
Fact table:
0CALDAY | ZCHAR | ZKEYF
12.10.2006 | CHAR1 | 10
12.10.2006 | CHAR2 | 20
12.10.2006 | CHAR3 | 30
Master data table for ZCHAR:
ZCHAR | ZKFAT
CHAR1 | 1000
CHAR2 | 2000
CHAR3 | 1500
I need to make a query with a table that looks like this:
0CALDAY | ZCHAR | ZKEYF | ZKFAT
12.10.2006 | CHAR1 | 10 | 1000
12.10.2006 | CHAR2 | 20 | 2000
12.10.2006 | CHAR3 | 30 | 1500
Finally, the query should result in something like this:
0CALDAY | ZKEYF | ZKFAT
12.10.2006 | 10 | 1000
12.10.2006 | 20 | 2000
12.10.2006 | 30 | 1500
Adding KFAT to the fact table is not an option, i need to read this information directly from ZCHAR.
I've tried using MultiProviders but didn't get the result i need.
Is there any way to achieve this? Please advice.
Thank you,

Thank you Ram C. i've tried your solution and it may be it, but i'm having one problem:
Since i'm reporting in web, i used the second solution you offered and almost got the desiered result. The problem is that some entries are correct but others ar duplicated! Using same example, my result table looks something like this:
0CALDAY | ZKEYF | Calculated KF (from ZKFAT)
12.10.2006 | 10 | 1000
12.10.2006 | 20 | 4000
12.10.2006 | 30 | 1500
Second row should be 2000, but instead it shows 4000. I added ZCHAR's attribute ZKFAT as a display attribute in order to compare the results. I found that the display attribute is correct (2000) but the Calculated KF still showed duplicated data (4000).
By the way, when i execute the query, i get this warning message:
"Calculated key figure ZRT_C02_2_CKF004 is not defined correctly"
Any ideas why this could be happening?
Thank you for your help.
Message was edited by: Gerardo Gaona

Join between large tables

Similar Messages

Maybe you are looking for