Duplicate records in Fact Tables

Hi,
We are using BPC 7.0 MS SP7. BPC created duplicate records in WB and Fac2 tables. We faced similar issue before and the solution was to reboot the server and cleanup the additional data created. I think it should not be an issue with the script logic files we have. We had the issue across all applications. Data is fine now after the server reboot and running the same logic files. I want to know if any one faced this issue and if there is any solution other than reboot. I appreciate your help.
Thanks
Raj

Hi Sorin,
I know this thread is rather old, but i have a problem which is pretty much related to this thread and appreciate if you could assist me. I have client running on 7.0 MS who has been using it for the past 3 years.
It is a heavily customized system with many batch files running daily to update dimensions, copy data and sort. And Yes we do use custom packages that incorporates stored procedures.
Recently, with no change in environment, we encountered our factwb ballooning up out of no where. fact table only contains less then 1 gb data but, factwb has 200 gb data and practically paralayzed the system. There is also equilavent 300 gb increase in the log files.
We are not able to find out what caused this? Or if even the 200gb records in wb are even valid records that are duplicated. Is there a way to troubleshoot this?

Similar Messages

How to delete the duplicate records in a table without promary key

I have a table that contains around 1 million records and there is no promary key or auto number coulums. I need to delete the duplicate records from this table. what is the simple effective way to do this.

Please see this link:
Remove duplicate records ...
sqldevelop.wordpress.com

Identifying duplicate records in a table

I am trying to identify duplicate records in a table - well they are broadly duplicated but some of the fields are changed on each insert whilst others are always the same.
I can't work out the logic and it is driving me #$%$#^@ crazy !

Here are a couple of other examples:
Method 1: -- Makes use of the uniqueness of Oracle ROWIDs to identify duplicates.
=========
To check for single column duplicates:
select rowid, deptno
from dept outer
where
outer.rowid >
(select min(rowid) from dept inner
where inner.deptno=outer.deptno)
order by deptno;
To check for multi-column (key) duplicates:
select rowid, deptno, dname
from dept outer
where
outer.rowid >
(select min(rowid) from dept inner
where inner.deptno| |inner.dname=outer.deptno| |outer.deptno)
order by deptno;
Method 2: -- Makes use of resultset groups to identify uniqueness
=========
To check for single column duplicates:
select rowid, deptno
from dept
where
deptno in
(select deptno from dept group by deptno having count(*) > 1)
order by deptno;
To check for multi-column (key) duplicates:
select rowid, deptno, dname
from dept
where
deptno| |dname in
(select deptno| |dname from dept group by deptno| |dname having count(*) > 1)
order by deptno;
null

Counting duplicate records in a table

Hi,
I have to display duplicate records in a table,but the table name and column should be passed dynamically(In procedure). Please let me know the query for this.

Try this one, is used to find the duplicate value from table
Type: I
SELECT * FROM employees e1 WHERE rowid> (SELECT min(rowid)
FROM employees e2 WHERE e1.department_id=e2.department_id);
Type: II
SELECT * FROM my_table t1 WHERE EXISTS (SELECT 'x' FROM my_table t2
WHERE t2.key_value1 = t1.key_value1
AND t2.key_value2 = t1.key_value2
AND t2.rowid > t1.rowid);
Type: III
SELECT count(*), empn FROM empmast_dum
GROUP BY empn HAVING count(*) >1 ORDER BY empn;
Type: IV
SELECT dep, name,net,RANK() OVER (PARTITION BY dep ORDER BY net) rank
FROM empmast_dump
WHERE dep=04;
Type: V
SELECT empn FROM empmast_dump
WHERE empn NOT IN(SELECT MIN(empn)
FROM empmast_dump GROUP BY NAME;
I have to display duplicate records in a
table,but the table name and column should be passed
dynamically(In procedure). Please let me know the
query for this.When you need to pass table name dynamically in your procedure ,
just enter table name as substitution variable or bind variable, if u want to know about
this, read here
venki
http://venki-hb.blogspot.com/2008/02/basic-sql-query-tips.html

Duplicate records in a table

i need to find duplicate records in a table that have the same name but a different id number.
only fields i am using is the id, first name, last name, middle name
any help would be great
thanks

Well, to just find the duplicate names, you can:
select first_name, middle_name, last_name, count(*)
from t
group by first_name, middle_name, last_name
having count(*) > 1;If you want to find the ids that are linked to these dupes, then:
select *
from t
where (first_name, middle_name, last_name) in
   (select first_name, middle_name, last_name
      from t
     group by first_name, middle_name, last_name
    having count(*) > 1);

How to delete duplicate records in all tables of the database

I would like to be able to delete all duplicate records in all the tables of the database. Many of the tables have LONG columns.
Thanks.

Hello
To delete duplicates from an individual table you can use a construct like:
DELETE FROM
    table_a del_tab
WHERE
    del_tab.ROWID <> (SELECT
                            MAX(dups_tab.ROWID)
                      FROM
                            table_a dups_tab
                      WHERE
                            dups_tab.col1 = del_tab.col1
                      AND
                            dups_tab.col2 = del_tab.col2
                      )You can then apply this to any table you want. The only differences will be the columns that you join on in the sub query. If you want to look for duplicated data in the long columns themselves, I'm pretty sure you're going to need to do some PL/SQL coding or maybe convert them to blobs or something.
HTH
David

To find the duplicate record in internal table

Hi,
i have a requirement to fine the duplicate record with 3 fields.
i am getting a flat file with 15 fields .
i need to check the duplaicate records of 3 fields . if i get any 2nd same record of 3 fields , the records will go to other internal table.
for ex :
1. aaa bbb ccc ddd eee fff ggg   hhh
2. aaa bbb ccf dde edd ffg ggh   hhj
3. aaa bbb cce ddd ees ffh ggu hhk
in that 1st record and 3rd record are same (aaa bbb ddd)
i need to find 3rd record
please help me
regrards
srinivasu

hi,
itab2[] = itab1[].
sort itab1 by f1 f2 f3.
sort itab2 by f1 f2 f3.
delete itab2 index 1.   "to delete the first record in itab2.
loop at itab1 into ws_itab1.
loop at itab2 into ws_itab2.
   if ws_itab1-f1 = ws_itab2-f1 and
     ws_itab1-f2 = ws_itab2-f2 and
    ws_itab1-f3 = ws_itab2-f3.
     ws_itab3 = ws_itab2.
     append ws_itab3 into itab3.   "Third internal table.
   endif.
endloop.
delete itab2 index 1.
endloop.
ITAB3 will have all the duplicate records.
Regards,
Subramanian

Duplicate records in Internal table

Hi All,
I want to find out the duplicate entry in the internal table. I have used,
Delete Adjacent duplicates from itab.
It is straight away deleting the record.
I want the user to correct that duplicate record.
May be some error message.
I have tried, with read,
Read table itab with key bzirk = itab-bzirk vkorg = itab-vkorg kunnr = itab-kunnr
matnr = itab-matnr comparing bzirk vkorg kunnr matnr.
But it's giving sy-subrc = 0 for the first record also.
Even, I have tried like, but it's giving syntax error.
Loop at itab where bzirk = itab-bzirk vkorg = itab-vkorg kunnr = itab-kunnr
matnr = itab-matnr.
Endloop.
Any method in case on Table control entries.
Thanks & Regards,
Kalyan Chandramouli
SAP Consultant

Hi,
Create a new internal table and assign the all the records of itab1 to itab2.
1.Sort Itab2.
2.delete adjacent duplicates.
3. loop at itab2.
     loop at itab1 where <conditon you want....>
     count = count + 1.
     endloop.
      if count GT 1.
        append the iatb2 records for user correction....
      endif.
   endloop.
If the hint is useful Say thanks by reward.
Regards,
Prabhu Rajesh

Importing and Updating Non-Duplicate Records from 2 Tables

I need some help with the code to import data from one table
into another if it is not a duplicate or if a record has changed.
I have 2 tables, Members and NetNews. I want to check NetNews
and import non-duplicate records from Members into NetNews and
update an email address in NetNews if it has changed in Members. I
figured it could be as simple as checking Members.MembersNumber and
Members.Email against the existance of NetNews.Email and
Members.MemberNumber and if a record in NetNews does not exist,
create it and if the email address in Members.email has changed,
update it in NetNews.Email.
Here is what I have from all of the suggestions received from
another category last year. It is not complete, but I am stuck on
the solution. Can someone please help me get this code working?
Thanks!
<cfquery datasource="#application.dsrepl#"
name="qryMember">
SELECT distinct Email,FirstName,LastName,MemberNumber
FROM members
WHERE memberstanding <= 2 AND email IS NOT NULL AND email
<> ' '
</cfquery>
<cfquery datasource="#application.ds#"
name="newsMember">
SELECT distinct MemberNumber
FROM NetNews
</cfquery>
<cfif
not(listfindnocase(valuelist(newsMember.MemberNumber),qryMember.MemberNumber)
AND isnumeric(qryMember.MemberNumber))>
insert into NetNews (Email_address, First_Name, Last_Name,
MemberNumber)
values ('#trim(qryMember.Email)#',
'#trim(qryMember.FirstName)#', '#trim(qryMember.LastName)#', '#
trim(qryMember.MemberNumber)#')-
</cfif>
</cfloop>
</cfquery>
------------------

Dan,
My DBA doesn't have the experience to help with a VIEW. Did I
mention that these are 2 separate databases on different servers?
This project is over a year old now and it really needs to get
finished so I thought the import would be the easiest way to go.
Thanks to your help, it is almost working.
I added some additional code to check for a changed email
address and update the NetNews database. It runs without error, but
I don't have a way to test it right now. Can you please look at the
code and see if it looks OK?
I am also still getting an error on line 10 after the routine
runs. The line that has this code: "and membernumber not in
(<cfqueryparam list="yes"
value="#valuelist(newsmember.membernumber)#
cfsqltype="cf_sql_integer">)" even with the cfif that Phil
suggested.
<cfquery datasource="#application.ds#"
name="newsMember">
SELECT distinct MemberNumber, Email_Address
FROM NetNewsTest
</cfquery>
<cfquery datasource="#application.dsrepl#"
name="qryMember">
SELECT distinct Email,FirstName,LastName,MemberNumber
FROM members
WHERE memberstanding <= 2 AND email IS NOT NULL AND email
<> ' '
and membernumber not in (<cfqueryparam list="yes"
value="#valuelist(newsmember.membernumber)#"
cfsqltype="cf_sql_integer">)
</cfquery>
<CFIF qryMember.recordcount NEQ 0>
<cfloop query ="qryMember">
<cfquery datasource="#application.ds#"
name="newsMember">
insert into NetNewsTest (Email_address, First_Name,
Last_Name, MemberNumber)
values ('#trim(qryMember.Email)#',
'#trim(qryMember.FirstName)#', '#trim(qryMember.LastName)#', '#
trim(qryMember.MemberNumber)#')
</cfquery>
</cfloop>
</cfif>
<cfquery datasource="#application.dsrepl#"
name="qryEmail">
SELECT distinct Email
FROM members
WHERE memberstanding <= 2 AND email IS NOT NULL AND email
<> ' '
and qryMember.email NEQ newsMember.email
</cfquery>
<CFIF qryEmail.recordcount NEQ 0>
<cfloop query ="qryEmail">
<cfquery datasource="#application.ds#"
name="newsMember">
update NetNewsTest (Email_address)
values ('#trim(qryMember.Email)#')
where email_address = #qryEmail.email#
</cfquery>
</cfloop>
</cfif>
Thank you again for the help.

How can I view duplicate records in a table?

Hello everyone,
My question is very straight forward. How can I view all duplicate records?
Thanks in advance,
Sonya

Easy enough,
select t1.n1,t1.c1,count(*) as "TOTAL_DUPS" from tableA t1 having count(*) > 1 group by t1.n1,t1.c1 order by count(*) desc
Tyler

To delete duplicate records from internal table

hi friends,
i have to delete records from internal table based on following criterion.
total fields are 7.
out of which if 4 fields are same and 5th field is different,then both records must be deleted.
in case all five fields are same,the program should do nothing.
for example.
if there are 3 records as follows
a1 b1 c1 d1 e1 f g
a1 b1 c1 d1 e2 w r
a1 b1 c1 d1 e1 j l
then first two records should be deleted as four fields are same but fifth(e) field differs.
but third record should remain as it is evenif first five fields are same for first and third record.
values of last two fields need not to be consider for deleting the records.

LOOP AT ITAB.
V_FILED5 = ITAB-F5. "to compare later
V_TABIX = SY-TABIX. "used to delete if condition not matches
READ TABLE ITAB WITH KEY F1 = ITAB-F1
                           F2 = ITAB-F2
                           F3 = ITAB-F3
                           F4 = ITAB-F4.
IF SY-SUBRC = 0.
    IF ITAB-F5 <> V_FIELD5.
*--both the records to be deleted,as Field5 is different.
      DELETE ITAB INDEX SY-TABIX. "deletes that record
      DELETE ITAB INDEX V_TABIX. "deletes the current record
    ENDIF.
ENDIF.
ENDLOOP.
Message was edited by: Srikanth Kidambi
added comments
Message was edited by: Srikanth Kidambi

Identify duplicate records in a table

I have this situation in the same table
GOREMAL_PIDM     GOREMAL_EMAL_CODE     GOREMAL_EMAIL_ADDRESS     GOREMAL_STATUS_IND     GOREMAL_PREFERRED_IND
2238954     REC1     [email protected]     A     Y
2238954     REC1     [email protected]     A     N
I need to identify those records (look they are the same(email address), just upper and lower case) then I need delete one the one with the GOREMAL_STATUS_IND = N
I am running this query
select
a.goremal_pidm    pidm_a,
b.goremal_pidm    pidmb,
a.goremal_emal_code emal_codea,
b.goremal_emal_code emal_codeb,
a.goremal_email_address email_addressa,
b.goremal_email_address email_addressb,
a.goremal_status_ind    status_inda,
b.goremal_status_ind    status_indb,
a.goremal_preferred_ind preferred_inda,
b.goremal_preferred_ind preferred_indb,
a.goremal_activity_date activity_datea,
b.goremal_activity_date activity_dateb,
a.goremal_user_id        user_ida,
b.goremal_user_id        user_idb,
a.goremal_comment        commenta,
b.goremal_comment        commentb,
a.goremal_disp_web_ind   web_inda,
b.goremal_disp_web_ind   web_indb,
a.goremal_data_origin    origina,
b.goremal_data_origin    originb
FROM
goremal a, goremal b, saradap
WHERE
--b.goremal_pidm = 2216086
b.goremal_preferred_ind = 'N'
AND a.goremal_preferred_ind = 'Y'
AND a.goremal_emal_code = 'REC1'
AND B.goremal_emal_code = 'REC1'
and a.goremal_email_address = b.goremal_email_address
and a.goremal_pidm = b.goremal_pidm
and a.goremal_pidm = saradap_pidm
AND Saradap_term_code_entry = 200990
AND Saradap_program_1 = 'UBA' to identify the records but it is not working sometimes give me recoirds that only have one row in the table (goremall)
what will be a good way to do this, again I need to identify all the records that have 2 rows, but the email is the same then I need to delete the ones with the
GOREMAL_STATUS_IND = N
tHANK YOU

Hi,
user648177 wrote:
Sorry but the previous answer won't work
I need to be able to identify (before the delete) all the records
like this one
GOREMAL_PIDM     GOREMAL_EMAL_CODE     GOREMAL_EMAIL_ADDRESS     GOREMAL_STATUS_IND     GOREMAL_PREFERRED_IND
2238954     REC1     [email protected]     A     Y
2238954     REC1     [email protected]     A     N
where the GOREMAL_EMAL_CODE are equal and the GOREMAL_EMAIL_ADDRESS is Y in one record and N in another record, then I will delete the records with N, but I want to de a select before I do the deleteWhat is wrong with Mathiasm's suggestion? What exactly "won't work" when you try it?
It would help if you formatted the data so that the columns lined up and posted it between {code} tags.
It could be that there was some confusion about what the column names were, and you only have to substitute the right names for it to work.
If you want to see the rows that will be deleted, rather than actually deleting them, change "DELETE" to "SELECT *".

Limiting duplicate records in secondary tables?

I have a report in CR2010 that links multiple tables. One of the linked tables has a many-to-one relationship with the primary table. I need a way to say, in effect, "Only the record with the latest date from the secondary table", thus pruning the earlier dated records. Con someone please provide a few pointers and/or some articles I can read to assist in this? Thanks!!
ETA: What's happening is that, because of the multiple records associated with each primary record, I'm getting multiple identical details, with only the one field from the other table different. I need to only have ONE detail line that is limited to that latest secondary table record.

you've got a couple of choices then...
A > Group Selection Formula
1) you can create a Group on the date field and have the Group for Every Day.
2) you can then create a Maximum on the date field
3) create a Group Selection Formula where you have = Max()
the problem with this is that you don't limit the number of records coming back to your report...you're just hiding them
B > Experiment with a SQL Expression to Use in Your Record Selection Filter
please note that this is not a supported functionality of SQL Expression fields and will sometimes work and sometimes not...hopefully it will. the following uses the Orders table in the sample Xtreme database for crystal.
1) create a new SQL Expression Field called "MaxOfMyDate" with syntax like the following...please note that i have aliased the Orders table that is used in the main report as Orders2...this is to avoid any errors by using the SQL Expression in the Record Selection Filter
(SELECT MAX(`Orders2`.`Order Date`) FROM `Orders` `Orders2`)
2) now go back to your Record Selection Filter and change it to something like
{Orders.Order Date} = {%MaxOfMyDate}
the above, if it works off of your database, will bring back only order dates that are equal to the very latest order date.
cheers,
jamie

QUERY - No Duplicate records from 2nd table ???

Hi Experts,
I really need help with this. I have 2 tables A and B. See below
<b>A -
B</b>
X -
XB1
XB2
XB3
Y----
YB
Z----
ZB
I select a record from A and get its record from B and give it on the output. For entry X I have 3 records in Table B. I want the output to give only 1 record from B for X. It can be any record. How can this be achieved in Query ?
Is there a special Join ? or can this be achieved by coding in Query ? If so how ?
I really appreciate your help.
Thanks !!

First get the data from table A.
select * from table (A) into table i_a.
loop at i_a.
select single * from table B into table i_b.
move b reord to final internal table
append final internal table.
endloop.

How can we select duplicate records from a table

Hi
I desire to use a query to select duplicate columns (based on all the columns of the table) ...
I am able to write queries which include column names, but when my table contains more than 50 columns, is it advisable for me to mention all the 50 column names in the query or is there any other query by which duplicate rows can be selected without column names being mentioned ?
Thnx in advance
Regards

Basic rule of relational design says that "thy table shalt have a primary key".
Now assuming that you do not have a primary key constraint and that resulted in duplicates, you should still have a couple of columns that uniquely describes a row in that table.
To select duplicate rows can then be done by joining rows with the primary key (same set of unique columns), where rows have different rowids (physical addresses).
E.g.
select
t1.*,
t2.*
from table t1,
table t2
where t1.col1 = t2.col1
and t1.col2 = t2.col2
and t1.rowid < t2.rowidIf you do not have a couple of columns (primary key) that uniquely identifies a row, then you do not have duplicates in the table - as there cannot be duplicates without having a means of unique identification.

Duplicate records in Fact Tables

Similar Messages

Maybe you are looking for