Mapping for "Junk dimensions"

Question

I'm trying to implement a mapping for some columns in what Kimball calls a "Junk Dimension." There are 25 columns in the dimension with cardinality's ranging from 2 to 10. There are 100,000,000 possible values, but only around 100,000 actually exist in the data set. I'd rather just populate the table with the 100,000 rows.
Here's a refernece to Kimball's discussion of this:
http://www.kimballgroup.com/html/designtipsPDF/DesignTips2003/KimballDT48DeClutter.pdf
Is the only way to do this in OWB to use a deduplicator (DISTINCT)?
If I have 2 dimensions like this, plus the fact table, am I going to fully scan the source table 3 times, once for each of the 2 distinct queries, and once for mapping into the fact table?
Is there a more efficient solution?

Answer 1

With “Statuses dimension” everything is clear – it’s classic junk dimension sourced from distinct values of different status fields.
About “Bands dimension”… Have you explicitly define (somewhere) bands for every value you are expecting to classify? If so, creating this dimension is much straightforward as previous – to join source table with “band definition tables” and collect distinct combinations of every “band id” into “band junk”.
If you are going to create bands implicitly (or dynamically – by applying expression like “trunc(x,-2)” ), approach may be the same. Just take additional care to not archive “useless banding” – remember that there are recommendation in statistics regarding optimal number of bands depending on number of measures (records) and magnitude of values (from min to max value).
I would not advise you to apply this “function lookup” approach. It will perform very poor on large dataset (because SQL <-> PLSQL context switching and additional exception handling – if key not found). Better try to browse source data during stage operation, find new key combinations for junk dimensions and load them. After that use simple join / lookup to get dimension key.
Sergey