How to find duplicated values in rows obtained from multiple tables

Question

Hi.
I need to find the duplicates stored in different tables of my database. I have some tables like the following model (I know it could be nonsense, but that's because it's simplified):
table person { id, name, surname }.
table zoo {id, owner, name, city} (zoo.owner -> person.id)
table area {id, zoo, type, name} (area.zoo -> zoo.id).
table dog {id, area, name, colour} (dog.area -> area.id)
table elephant {id, area, name, height} (elephant.area -> area.id)
As ids are autoincremental, it could happen that a person has two zoos with identical areas (meaning, these areas has the same type and name, same dogs (same name and colour) and same elephants (same name and height)). In an example with data:
person
id     name     surname
p1     john     doe
zoo
id     owner     name          city
z1     p1          central     NY
z2     p1          central     NY
area
id     zoo     type     name
a1     z1     open     main
a2     z2     open     main
dog
id     area     name          colour
d1     a1     jaro          brown
d2     a1     chispa     white
d3     a2     jaro          brown
d4     a2     chispa     white
elephant
id     area     name          height
e1     a1     dumbo     5
e2     a1     elphy          4
e3     a2     dumbo     5
e4     a2     elphy          4
That is: John Doe has two zoos in the same city and with the same name. These two zoos, has one open main area each. Each of these areas has two dogs with the same names and colours and two elephants with the same names and heights. So this zoos would be identical. What I want is to delete z2 zoo.
I'd like to find a SQL function which returns me the id of one of these zoos, so it can respond to the question. Has the person called "John Doe" more than one area with the same type, name, dogs and elephants?
Is it possible?

Answer 1

Hi,
Interesting problem!
Two distinct zoos are duplicates of each other if they have the same owner, the same name, the same areas, and the same animals in each of those areas.
It's pretty easy to tell if two zoos are distinct, but have the same owner and name. We could do that with a self-join:
SELECT       z1.*
,       z2.id          AS id_2
FROM       zoo     z1
JOIN       zoo     z2  ON   z1.owner     = z2.owner
                AND      z1.name     = z2.name
              AND      z1.id          < z2.idBut how can we tell if two zoos that meet these criteria have the same aminals in the same areas? (It amuses me to call these the "critter criteria".)
One way is to make a list, for each zoo, of all the animals, with all their attributes (that is, the attributes that matter for determining if they are the same), in each area, and then see if the lists for those two zoos is identical. Like this:
WITH     all_animals     AS
     SELECT     d.id, d.area
     ,     d.name          -- || '~' || d.colour || ...
               AS attributes
     ,     'DOG'     AS animal
     FROM     dog     d
    UNION ALL
     SELECT     e.id, e.area
     ,     e.name          -- || '?`' || e.height || ...
                    AS attributes
     ,     'ELEPHANT'     AS animal
     FROM     elephant     e
-- UNION ALL   ... cat ... monkey ...
,     got_analytics     AS
     SELECT     z.id          AS zoo_id
     ,     ar.type          -- , ar.name, ...
     ,     an.animal
     ,     an.attributes
     ,     COUNT (*)     OVER ( PARTITION BY  z.id )     AS cnt
     ,     ROW_NUMBER () OVER ( PARTITION BY  z.id
                               ORDER BY          ar.type     -- , ar.name, ...
                         ,                an.animal
                                     ,                    an.attributes
                       )                    AS r_num
     FROM           zoo          z
     LEFT OUTER JOIN      area          ar  ON   ar.zoo          = z.id
     LEFT OUTER JOIN      all_animals     an  ON       an.area     = ar.id
SELECT       z1.*
,       z2.id          AS id_2
FROM       zoo     z1
JOIN       zoo     z2  ON   z1.owner     = z2.owner
                AND      z1.name     = z2.name
              AND      z1.id          < z2.id
WHERE     NOT EXISTS
          SELECT  type     -- , name, ...
          ,          animal, attributes
          ,          cnt, r_num
          FROM    got_analytics
          WHERE   zoo_id     = z1.id
       MINUS         
          SELECT  type     -- , name, ...
          ,          animal, attributes
          ,          cnt, r_num
          FROM    got_analytics
          WHERE   zoo_id     = z2.id
ORDER BY  z1.id
,            z2.id
;The first sub-query, all_animals, combines all the animals from all of the separate species tables. For each one, it makes a delimited list (the attributes column) of the columns that count in determining if 2 animals are the same. Different species may have different attributes and different numbers of attributes. The lists have to be delimited by some sub-string (not necessarily a single character) that can never occur in any of the attributes for that species. For example, dogs may never have '~' in their names (or their colours, if you expand the example to include colour). Elephants might have any character, but never have a '?' followed immediately by a grave accent '`'.
The next sub-query, got_analytics, joins the zoo and area tables to this combined list of animals. The joins are outer joins, so we can detect as duplicates two zoos that have no areas, or two areas that have no animals.
In the main query, I used MINUS to tell if the lists for two given zoos were identical. If x MINUS y produces no results, then every row in x has a matching row in y. To say that x and y are identical, we also have to know that every row in y has a matching row in x. Rather than do another MINUS, I counted the rows (got_analytics.cnt), so that if y has more rows than x, no rows will match when we do the MINUS.
Here's the output I got from your sample data:
`       ID      OWNER NAME             ID_2
         1          1 CENTRAL NY          2This shows the distinct ids, and the common attributes, of all distinct pairs of zoos that match. You can join this to the person table, if you want to see details about the owner.
If a query is correct, it will not only find all the results it is supposed to find; it will also not find all the results it is not supposed to find, for all the various reasons. I added some more sample data to test one such reason:
INSERT INTO ZOO (ID, OWNER, NAME) VALUES (91, 1, 'CENTRAL NY');
INSERT INTO AREA (ID, ZOO, TYPE) VALUES (92, 91, 'MAIN');
INSERT INTO DOG (ID, AREA, NAME) VALUES (93, 92, 'JARO');
INSERT INTO DOG (ID, AREA, NAME) VALUES (94, 92, 'CHISPA');Zoo 91 is identical to zoos 1 and 2, except that 91 has no elephants. To thoroughly test this solution, you need to add some more test data, especially zoos that will not be selected for various reasons: for example, two zoos that have all the same animals, but different numbers of areas, or that both have two areas, but the animals are distributed differently between the areas.