Problems maintaning surrogate keys

Hi,
I am trying to load a dimension table. the initial load worked fine. now for the regular load if there is one a new record in the source table it does adds 1 new record in the dimension table. But the problem is the surrogate keys(I am using a oracle sequence for this) skips the number of record in the dimension table and assigs next value to the new record. The lode type is "insert/Update".Constraint used for maching is the primary key of source table.
e.g
--initial run.. table had 5 records before
-- Second run.. table has 6 records but the surrogate keys value for the 6th record will be '11' insted of being '6'
can you help me with this.....

Nawneet,
after I posted this problem on the forum i read an article that said i should do something like what you said. so i changed the seq to an function which returns only the next value when called. i imported it to the map and used it insted of the seq...but still no use....the map is doing the same thing during the second run. i also tried chaning the load type from insert/update to update/insert. also changed the default operating mode to row based....still no use...:((.....
what happens is during the first run 5 records get loaded perfectly and every thign is fine, but if I run it again with no new changes in the source records the map runs fine and the data is still good. but what happens is the nextval of the sequence jumps to 11...when i am expecting it to remain at 6 as no new records have been added to the dimension table. I think some how owb is seeing 5 records in the source table during the second run and uses the next val frmo the cache...but does not use them and then since it not used this values go waste.
Problem: I can not schedule the load of the dimension table for every day as the next value of this seq will keep on skeeping and when ever a new record is added to dimension table will have a large value .

Similar Messages

Problems Engineering Surrogate Primary Key with Unique Key

SDDM 3.3.0.747 with 2 problems (at least so far). I am hoping that the problem is with this SDDM rookie and I have overlooked some setting. PROBLEM 1 I don’t want to start a religious debate about surrogate vs. natural keys but I am having a problem engineering both correctly from the logical model. I am a rookie when it comes to SDDM but have many years of experience with Designer. By default I like to have both a natural UID (UK) and a surrogate key based primary UID (PK) which is used for foreign keys. The problem I am having with engineering is I can successfully engineer the surrogate PK’s, engineer the FK’s using the PK’s but cannot get the unique key to contain the surrogate keys in the child table. If I check the identifying property in the relations, the PK columns and the UK columns are included in the child PK and the UK contains no columns. The Setup I have defined two reference entities, PROBABILITY and SEVERITY with natural unique keys defined. I also have a child entity RISK_ASSESMENT with relationships back to the PROBABILITY and SEVERITY entities and both have the “Use surrogate keys:”: check box checked. The unique key for the RISK_ASSESMENT entity includes the relationships back to PROBILITY and SEVERITY. None of the entities have a PK or surrogate key defined and they all have the “Create Surrogate Key” check box checked. In addition the following preferences are set: Data Modeler/Model/Logical NOT Checked - Use And Set First Unique Key As Primary Key NOT Checked – Name – Keep as the name of the Originating attribute Checked – Entity Create Surrogate Key Checked – Relationship Use Surrogate Key PROBLEM 2 When the foreign key columns are engineered I want the names to have a prefix “FK_” but they don’t. Templates are set as follows: Data Modeler/Naming Standard/Templates Foreign Key: FK_{child}{parent} Column Foreign Key: FK_{ref column} Engineer to Relational Model/General Options Checked - Apply name translation Marcus Bacon

I have been switching between SD 4 EA1 and SDDM 3.3 trying to get things to work and trying out the template table for adding audit columns (really nice!).
Concerning Problem1. No matter what settings I use and whether I use SDDM 3.3 or SDI cannot get the FK columns to be included in the UK even though the relations are included in the UID in the entitty. When I open the properties of the child table and click on the naming standards button and click ok it complains that the UK is not complete. I add the FK columns to the UK and all is well including the naming standards.
Concerning Problem 2. Sometimes it engineers the names for FK's from the template and sometimes it doesn't. Didn't see a pattern. Gave up trying and used Naming Standards button. I still had to change a few.
The good new is, that after make changes needed in UK's and Column names of 18 tables, I know have everything deployed to Test except FK Indexes. I think I have to do those by hand.
Marcus Bacon

About Surrogate Key and Dimension Key on OWB 10.2

Hi, everyone.
I am using OWB 10.2 and I have a question about Surrogate key and Dimension Key.
I indicated the foreign key as VARCHAR2 type in Fact Table and Dimension Key as VARCHAR2 type is operated as Primary key in Dimension Table. I made Single Level in Dimension Table.
I know that Dimension Key stores the surrogate ID for dimension and is the primary key of the table. Also, Surrogate ID should be only NUMBER type.
So, in this case, Surrogate ID is NUMBER type
Dimension key should be NUMBER type to store the surrogate ID.
But, Dimension key also should operate the primary to relate Foreign key as VARCHAR2 type.
How I can solve this confusing condition?
Please let me know that.
JWS

Hi JWS,
From a SQL point of view it should not be a problem to join a NUMBER field to a VARCHAR2 field because during execution there will be an implicite cast for the NUMBER value to a VARCHAR2 value. See the example below.
SELECT * FROM DUAL
WHERE 1 = '1'From an OWB point of view it is not possible to have a Dimension with an NUMBER value Key that has a relation to a VARCHAR2 value Foreign key in a Fact table. This is caused due to the creation of a Fact table in OWB in which the Foreign keys in it are build from de Dimension tables that refer to them.
You will loose the reference to the Dimension when changing the type of the Foreign Key.
To resolve this issue I would advise you to use a Sequence that generates your Surrogate Key (NUMBER type) for the Dimension table and store it in the Primary Key Column (VARCHAR2 type).
When validating the mapping you will get a warning, but when executing this should give no problems.
Regards,
Ilona

Surrogate Key and Map for Cube

Hi
I am new to Data Warehousing and am trying to use OWB 11g.
I am trying to create dimensions with multiple levels. When I create more than one level it need to have surrogate as well business key for each dimension level. But I can create only one surrogate in the dimension, there is no option to create multiple surrogate keys in the same dimension. so what am I missing?
My second question is regarding cube. Do I need to create a Mapping for a cube? if yes, should I move the data to the cube from the dimensions? and where will the measures come from? do i need to load the measures or they will be calculated automatically?
please reply...
regards
Arif

hi
Got it, Yes that was the reason,
The table was not properly deployed after the dimension was modified.
Anyway, the describe of the table is as follows
describe arif.QUESTION_DIM
Name Null Type
DIMENSION_KEY NOT NULL NUMBER
IGV_ID NUMBER
PER_ID NUMBER
DIM_ID NUMBER
IGO_ID NUMBER
INQ_ID NUMBER
ID NUMBER
DIM_ORDEM NUMBER
DIM_AMBITO VARCHAR2(3)
DIM_NOME VARCHAR2(150)
10 rows selected
Now, I am having another problem,
when, I deploy the Map to load the data from three different tables, it gives the following problem
Name               Action               Status          Log
QUESTION_MAP          Create               Warning          ORA-06550: line 297, column 25:
                                        PLS-00302: component 'ID' must be declared
QUESTION_MAP          Create               Warning          ORA-06550: line 1153, column 11:
                                        PL/SQL: SQL Statement ignored
QUESTION_MAP          Create               Warning          ORA-06550: line 1155, column 15:
                                        PL/SQL: ORA-00904: "QUESTION_DIM"."ID": invalid identifier
QUESTION_MAP          Create               Warning          ORA-06550: line 1155, column 31:
                                        PLS-00302: component 'ID' must be declared
QUESTION_MAP          Create               Warning          ORA-06550: line 233, column 1:
                                        PL/SQL: SQL Statement ignored
QUESTION_MAP          Create               Warning          ORA-06550: line 2539, column 11:
                                        PL/SQL: SQL Statement ignored
QUESTION_MAP          Create               Warning          ORA-06550: line 2541, column 15:
                                        PL/SQL: ORA-00904: "QUESTION_DIM"."ID": invalid identifier
QUESTION_MAP          Create               Warning          ORA-06550: line 2541, column 31:
                                        PLS-00302: component 'ID' must be declared
QUESTION_MAP          Create               Warning          ORA-06550: line 297, column 9:
                                        PL/SQL: ORA-00904: "QUESTION_DIM"."ID": invalid identifier
Edited by: user643560 on Oct 22, 2008 9:38 AM

Apex and Natural Vs Surrogate keys

Hi
We've been using Apex for a few months now and there's a debate raging in our department over whether we should design our database tables using natural or surrogate (based on Oracle sequences / triggers) keys. Our experience as Apex developers shows that Apex itself looks to lean towards surrogate keys, a few examples are below:
- When creating forms on reports / tables Apex only allows 2 primary key columns without adding 'extras' in the background (see a previous post of mine).
- If we have a form on a table and our natural primary keys can be updated, the Apex-created DML statements break, as they look to do the update using the changed key values in the WHERE clause rather than the old ones. The only way around this seems to be to delete the inbuilt DML statements created by Apex and code your own, which is extra work.
- The Apex sample applications themselves seem to use sequences / surrogate keys.
What are people's opinions on this? In particular is there any guidance from the Apex development team on which is best to use with Apex?
Regards
Antilles

Hi Andrew,
As with abots_d, I only use "natural" keys for lookups.
>
1. the department names were here for 20 years and they never changed
>
But, can you guarantee that they never will? My firm has changed departmental names so many times, it's getting ridiculous! But other things also change over time - consider what happens if a person gets married and changes their name and you've used their previous names as the keys (and consider how much data in other tables may use those keys).
>
2. Server names are uniquely generated by special formula in excel to preciously avoid the duplication problem and guarantee the uniqueness within our glamorous bank.
>
SQL could probably recreate that formula and Unique Key constraints would handle the rest
>
3. no, we are not going to extend this app to cover any other banks
>
Given what's happening with the banking industry right now, who can say ;)
>
4. PK that means smth ( aka "server name" ) has a meaning, whereas meaningless - has no (business) value
>
Why does a bit of data have to have explicit "business value"? I would suggest that a surrogate key is a pointer to a record and allows you to easily create relationships. Once created, the key would never be changed regardless of what happens to the data on its record. Thus, the relationship is maintained. Using personnel (which our firm renamed as "Human Resources" a while back) as an example, it's likely that every employee would have an employee number. Does this number actually mean anything in itself, does it have "business value"? Most likely, it's just a convenient way to identify a person and relate records to them.
I would suggest that any non-numeric/date keys are relatively slow. As strings, the only way to check for their sort order would be to (A) convert to upper or lower case and (B) perform a string comparison left-to-right across the entire string. There's also the possibility of certain characters appearing in the strings that can cause issues - for example, quotes, apostrophes, colons, commas, question marks and percentage signs.
Also, consider the length of a VARCHAR2 that you would have to use - how big would it need to be to cover all possibilities? You may say 20 now but tomorrow you get data with 21 characters in it - do you want to update the table plus all related tables for that?
There are further issues with parent, child, grand-child etc relationships where the keys would have to be passed down in full through the relationships. Depending on how many levels you may have, a fair number of the fields on the bottom-most table would be there just for the keys.
It has been a standard industry practice for many years now to "normalise databases" to avoid lots of issues with keys and "repeating data". Apart from very simple lookup tables, I have stuck with those guidelines for years now without any problems at all.
Andy

Joe - Why was SURROGATE KEY left out from ISO table design?

The lack of SURROGATE KEY causes lots of confusion and ultimately loss of productivity. Common practice in SQL Server development to make the SURROGATE KEY the PRIMARY KEY, the source of all trouble because it is not really the "PRIMARY KEY" just
a meaningless integer identifier.
Example:
CREATE TABLE Products (
ProductID INT SURROGATE KEY,
ProductNumber char(12) PRIMARY KEY,
Name nvarchar(100) NOT NULL UNIQUE,
ListPrice DECIMAL (12,2) NOT NULL,
Color varchar(10) );
Is there a hope of correcting this issue?
Thanks.
Kalman Toth Database & OLAP Architect
SQL Server 2014 Database Design
New Book / Kindle: Beginner Database Design & SQL Programming Using Microsoft SQL Server 2014

Thanks Joe.
In SQL Server world we do use SURROGATE IDENTITY (or SEQUENCE object) INT in table design. That's like in our DNA even if it conflicts with Codd. AdventureWorks sample:
SELECT ProductID, ProductNumber, Name, ListPrice, Color
FROM Production.Product ORDER BY ProductNumber;
ProductID ProductNumber Name ListPrice Color
899 FR-T67Y-44 LL Touring Frame - Yellow, 44 333.42 Yellow
900 FR-T67Y-50 LL Touring Frame - Yellow, 50 333.42 Yellow
901 FR-T67Y-54 LL Touring Frame - Yellow, 54 333.42 Yellow
902 FR-T67Y-58 LL Touring Frame - Yellow, 58 333.42 Yellow
886 FR-T67Y-62 LL Touring Frame - Yellow, 62 333.42 Yellow
890 FR-T98U-46 HL Touring Frame - Blue, 46 1003.91 Blue
ProductID is the (SURROGATE) PRIMARY KEY
ProductNumber is the NATURAL KEY (created by the accounting department) - this is the real "PRIMARY KEY"
Name is the CANDIDATE KEY (too long to be a key)
In RDBMS theory when we talk about PRIMARY KEY we mean the ProductNumber column which is used in real life.
However, in reality the ProductID INT meaningless number is the PRIMARY KEY, while the meaningful ProductNumber has to settle for a UNIQUE KEY or unique index.
I understand your point that we should not use SURROGATES, but we do. It's like in our (SQL Server) blood. If I go to a company and design for them without SURROGATE IDENTITY/SEQUENCE, they would fire me. From an ORACLE forum: "Basically,
always use a surrogate key. There are a few special cases where a surrogate key really isn't any better than some "natural" key, and whatever effort is needed to create and populate a surrogate key would just be wasted. These situations
are pretty rare.
Here's one example: Say you have a many-to-many relationship between employees and departments, that is, each employee may be related to 0 or more departments, and each department may be related to 0 or more employees, but an given employee can only be
related to a given department 1 time. In that case, a primary key consisting of both columns, dept_id and emp_id, is about as good as a surrogate key. You'd need a unique constraint on (dept_id, emp_id) in any case, and I don't see any need to create a surrogate
key." LINK:
https://community.oracle.com/thread/2527771?tstart=0
I tell you Joe, 90% of the world running on SURROGATE PRIMARY KEY tables, so why should we care about Codd at this point? Even the perfect PRIMARY KEY candidate, social security number, may have problems such as stolen SSNo duplicates among others: "Natural
key. A key that is formed of attributes that already exist in the real world. For example, U.S. citizens are issued a Social Security Number (SSN) that is unique to them (this isn't guaranteed to be true, but it's pretty
darn close in practice). SSN could be used as a natural key, assuming privacy laws allow it, for a
Person entity (assuming the scope of your organization is limited to the U.S.)." LINK:
http://www.agiledata.org/essays/keys.html A good advice from the article: "Don't naturalize surrogate keys. As soon as you display the value of a surrogate
key to your end users, or worse yet allow them to work with the value (perhaps to search), you have effectively given the key business meaning. This in effect naturalizes the key and thereby negates some of the advantages of surrogate keys."
Kalman Toth Database & OLAP Architect
SQL Server 2014 Database Design
New Book / Kindle: Beginner Database Design & SQL Programming Using Microsoft SQL Server 2014

Surrogate Keys vs. Natural Keys

Hi All,
Is anyone aware of a recommendation regarding the use of surrogate keys vs. natural keys?
Regards,
Irfan Abdul Rehman

The Natural Keys approach was first. This was the approach used when Relational Databases were first discovered. But I believe, it was based on the premise that the design of the Database does not change over time. Most people seem to side with one or the other. People's opinions here are usually based on what they were brought up with or personal experiences where they have run into problems with one approach or the other.
When viewed within the context of object-oriented design, the Surrogate approach is more common. Consider: does the uniqueness of a table have any relevance to other tables? If the uniqueness of a table changes, why should this have to impact other tables? If a customer is associated with an Order, using Natural Keys, you need to know what columns make the Customer unique when inserting a record into the ORDERS table. With Surrogate Keys, you don't need to know what columns make the Customer unique, the customer is referenced by its Surrogate Key. With Surrogate Keys, you only need to know what makes a Customer unique when dealing with the CUSTOMERS table.
With SQL Server, the Surrogate approach is very straight forward. In Oracle, the appeal of Surrogate Keys is less than with SQL Server as there is a more work to implement them. In SQL Server you specify the Column as an identity column. In Oracle you need to additionally add a sequence. In addition, if you want the value automatic on inserts, you will need to create a trigger (unfortunately here, when you insert, there is no way to find out what you just inserted). If you don't choose to have the value automatic on inserts, your insert SQL statements will require extra SQL code and the Surrogate value is not enforced (i.e. someone could enter any value and this could lead to a Key Violation)
Here are some disadvantages of Natural Keys:
-     Almost always, more columns to join on. If Table B is a detail table of master A and C is a detail to B and D is a detail to C, you will need at least 3 columns to join D to C in an SQL query.
-     If your uniqueness of a table changes (Ex: the number of Columns making your Table unique changes from 2 to 3), with Natural Keys, all of your SQL (Stored Procedures, Reports, Views, SQL Scripts, Application Code) will have to be re-written and your foreign keys relating to that table will have to be changed. With Surrogate Keys usually as simple as modifying that table.
-     If the data type of a Primary Key column changes (Ex: you used a varchar(20), now it's not big enough and has to be changed to varchar(100)), with Natural Keys, all Foreign Keys related to that Table will have to be changed (may also impact SQL Code). With Surrogate Keys usually as simple as modifying that table.
-     The Classic: a value changes in the Primary Key (Like Last Name). Now you've got to update that in every Foreign Key. Which means you'll have a big headache when you have to temporarily drop the constraint(s).

How to Maintain Surrogate Key Mapping (cross-reference) for Dimension Tables

Hi,
What would be the best approach on ODI to implement the Surrogate Key Mapping Table on the STG layer according to Kimball's technique:
"Surrogate key mapping tables are designed to map natural keys from the disparate source systems to their master data warehouse surrogate key. Mapping tables are an efficient way to maintain surrogate keys in your data warehouse. These compact tables are designed for high-speed processing. Mapping tables contain only the most current value of a surrogate key— used to populate a dimension—and the natural key from the source system. Since the same dimension can have many sources, a mapping table contains a natural key column for each of its sources.
Mapping tables can be equally effective if they are stored in a database or on the file system. The advantage of using a database for mapping tables is that you can utilize the database sequence generator to create new surrogate keys. And also, when indexed properly, mapping tables in a database are very efficient during key value lookups."
We have a requirement to implement cross-reference mapping tables with Natural and Surrogate Keys for each dimension table. These mappings tables will be populated automatically (only inserts) during the E-LT execution, right after inserting into the dimension table.
Someone have any idea on how to implement this on ODI?
Thanks,
Danilo

Hi,
first of all please avoid bolding something. After this according Kimball (if i remember well) is a 1:1 mapping, so no-surrogate key.
After that personally you could use Lookup Table
http://www.odigurus.com/2012/02/lookup-transformation-using-odi.html
or make a simple outer join filtering by your "Active_Flag" column (remember that this filter need to be inside your outer join).
Let us know
Francesco

Mapping problem with compressed key update record

Hi, could you please advise?
I'm getting the following problem:
About a week ago replicat abened with "Error in mapping" error. I found in discard file some record looking like:
filed1 = NULL
field2 =
field3 =
field4 =
field5 =
datefield = -04-09 00:00:00
field6 =
field8 =
field9 = NULL
field10 =
Where filed9 = @GETENV("GGHEADER", "COMMITTIMESTAM"), field10 = = @GETENV("GGHEADER", "COMMITTIMESTAM"), others are table fields mapped by USEDEFAULTS
So I got Mapping problem with compressed key update record at 2012-06-01 15:44
I guess I need to mention that extract failed in 5 minuts before it with: VAM function VAMRead returned unexpected result: error 600 - VAM Client Report <[CFileInfo::Read] Timeout expired after 10 retries with 1000 ms delay, waiting to read transaction log or backup files. To increase the number of retries, use SETENV (GGS_CacheRetryCount = n) in Extract parameter file. To control retry delay time, use SETENV (GGS_CacheRetryDelay = n). handle: 0000000000000398 ReadFile GetLastError:997 Wait GetLastError:997>.
I don't know if it has ther same source as data corruption, could you tell me if it is?
Well, I created new extract, starting 2012-06-01 15:30 to check if there was something with extract at the time, but got the same error.
If I run extract beging at 15:52 it starts and works.
But well, I got another one today. Data didn't look that bad, but yet one column came with null value:( And I'm using it as a key column, so I got Mapping problem with compressed key update record again:(
I'm replicating from SQL Server 2008 to Oracle 11g.
I'm actually using NOCOMPRESSUPDATES in Extract.
CDC is enabled for all tables replicated. The only thing is that it is enabled not by ADD TRANDATA command, but by SQL Server sys.sp_cdc_enable_table, does it matter?
Could you please advise why does it happen?

Well, the problem begins somewhere in extract or before extract, may be in transaction log, I don't know:(
Here are extract parameters:
EXTRACT ETCHECK
TRANLOGOPTIONS MANAGESECONDARYTRUNCATIONPOINT
SOURCEDB TEST, USERID **, PASSWORD *****
exttrail ./dirdat/ec
NOCOMPRESSUPDATES
NOCOMPRESSDELETES
TABLE tst.table1, COLS (field1, field2, field3, field4, field5, field6, field7, field8 );
TABLE tst.table2, COLS (field1, field2, field3, field4 );
Data pump:
EXTRACT DTCHECK
SOURCEDB TEST, USERID **, PASSWORD *****
RMTHOST ***, MGRPORT 7809
RMTTRAIL ./dirdat/dc
TABLE tst.table1;
TABLE tst.table2;
Replicat:
REPLICAT rtcheck
USERID tst, PASSWORD ***
DISCARDFILE ./dirrpt/rtcheck.txt, PURGE
SOURCEDEFS ./dirdef/sourcei.def
HANDLECOLLISIONS
UPDATEDELETES
MAP tst.table1, t.table1, COLMAP (USEDEFAULTS , filed9 = @GETENV("GGHEADER", "COMMITTIMESTAMP"), filed10= @CASE(@GETENV("GGHEADER", "OPTYPE"), "SQL COMPUPDATE", "U", "PK UPDATE", "U",@GETENV("GGHEADER", "OPTYPE")) ), KEYCOLS (field3);
MAP dbo.TPROCPERIODCONFIRMSTAV, TARGET R_019_000001.TPROCPERIODCONFIRMSTAV, COLMAP (USEDEFAULTS , field5 = @GETENV("GGHEADER", "COMMITTIMESTAMP"), filed6= @CASE(@GETENV("GGHEADER", "OPTYPE"), "SQL COMPUPDATE", "U", "PK UPDATE", "U",@GETENV("GGHEADER", "OPTYPE")) ), KEYCOLS (filed1, field2, field3);
Rpt file for replicat:
Oracle GoldenGate Delivery for Oracle
Version 11.1.1.1 OGGCORE_11.1.1_PLATFORMS_110421.2040
Windows x64 (optimized), Oracle 11g on Apr 22 2011 00:34:07
Copyright (C) 1995, 2011, Oracle and/or its affiliates. All rights reserved.
Starting at 2012-06-05 12:49:38
Operating System Version:
Microsoft Windows Server 2008 R2 , on x64
Version 6.1 (Build 7601: Service Pack 1)
Process id: 2264
Description:
** Running with the following parameters **
REPLICAT rtcheck
USERID tst, PASSWORD ***
DISCARDFILE ./dirrpt/rtcheck.txt, PURGE
SOURCEDEFS ./dirdef/sourcei.def
HANDLECOLLISIONS
UPDATEDELETES
MAP tst.table1, t.table1, COLMAP (USEDEFAULTS , filed9 = @GETENV("GGHEADER", "COMMITTIMESTAMP"), filed10= @CASE(@GETENV("GGHEADER", "OPTYPE"), "SQL COMPUPDATE", "U", "PK UPDATE", "U",@GETENV("GGHEADER", "OPTYPE")) ), KEYCOLS (field3);
MAP dbo.TPROCPERIODCONFIRMSTAV, TARGET R_019_000001.TPROCPERIODCONFIRMSTAV, COLMAP (USEDEFAULTS , field5 = @GETENV("GGHEADER", "COMMITTIMESTAMP"), filed6= @CASE(@GETENV("GGHEADER", "OPTYPE"), "SQL COMPUPDATE", "U", "PK UPDATE", "U",@GETENV("GGHEADER", "OPTYPE")) ), KEYCOLS (filed1, field2, field3);
CACHEMGR virtual memory values (may have been adjusted)
CACHEBUFFERSIZE: 64K
CACHESIZE: 512M
CACHEBUFFERSIZE (soft max): 4M
CACHEPAGEOUTSIZE (normal): 4M
PROCESS VM AVAIL FROM OS (min): 1G
CACHESIZEMAX (strict force to disk): 881M
Database Version:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for 64-bit Windows: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
Database Language and Character Set:
NLS_LANG = "AMERICAN_AMERICA.CL8MSWIN1251"
NLS_LANGUAGE = "AMERICAN"
NLS_TERRITORY = "AMERICA"
NLS_CHARACTERSET = "CL8MSWIN1251"
For further information on character set settings, please refer to user manual.
** Run Time Messages **
Opened trail file ./dirdat/dc000000 at 2012-06-05 12:49:39
2012-06-05 12:58:14 INFO OGG-01020 Processed extract process RESTART_ABEND record at seq 0, rba 925 (aborted 0 records).
MAP resolved (entry tst.table1):
MAP tst.table1, t.table1, COLMAP (USEDEFAULTS , filed9 = @GETENV("GGHEADER", "COMMITTIMESTAMP"), filed10= @CASE(@GETENV("GGHEADER", "OPTYPE"), "SQL COMPUPDATE", "U", "PK UPDATE", "U",@GETENV("GGHEADER", "OPTYPE")) ), KEYCOLS (field3);
2012-06-05 12:58:14 WARNING OGG-00869 No unique key is defined for table table1. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.
Using the following default columns with matching names:
field1=field1, field2=field2, field3=field3, field4=field4, field5=field5, field6=field6, field7=field7, field8=field8
Using the following key columns for target table R_019_000001.TCALCULATE: field3.
2012-06-05 12:58:14 WARNING OGG-01431 Aborted grouped transaction on 'tst.table1', Mapping error.
2012-06-05 12:58:14 WARNING OGG-01003 Repositioning to rba 987 in seqno 0.
2012-06-05 12:58:14 WARNING OGG-01151 Error mapping from tst.table1 to tst.table1.
2012-06-05 12:58:14 WARNING OGG-01003 Repositioning to rba 987 in seqno 0.
Source Context :
SourceModule : [er.main]
SourceID : [er/rep.c]
SourceFunction : [take_rep_err_action]
SourceLine : [16064]
ThreadBacktrace : [8] elements
: [C:\App\OGG\replicat.exe(ERCALLBACK+0x143034) [0x00000001402192B4]]
: [C:\App\OGG\replicat.exe(ERCALLBACK+0x11dd44) [0x00000001401F3FC4]]
: [C:\App\OGG\replicat.exe(<RCALLBACK+0x11dd44) [0x000000014009F102]]
: [C:\App\OGG\replicat.exe(<RCALLBACK+0x11dd44) [0x00000001400B29CC]]
: [C:\App\OGG\replicat.exe(<RCALLBACK+0x11dd44) [0x00000001400B8887]]
: [C:\App\OGG\replicat.exe(releaseCProcessManagerInstance+0x25250) [0x000000014028F200]]
: [C:\Windows\system32\kernel32.dll(BaseThreadInitThunk+0xd) [0x000000007720652D]]
: [C:\Windows\SYSTEM32\ntdll.dll(RtlUserThreadStart+0x21) [0x000000007733C521]]
2012-06-05 12:58:14 ERROR OGG-01296 Error mapping from tst.table1 to tst.table1.
* ** Run Time Statistics ** *
Last record for the last committed transaction is the following:
Trail name : ./dirdat/dc000000
Hdr-Ind : E (x45) Partition : . (x04)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 249 (x00f9) IO Time : 2012-06-01 15:48:56.285333
IOType : 115 (x73) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 44 AuditPos : 71176199289771
Continued : N (x00) RecCount : 1 (x01)
2012-06-01 15:48:56.285333 GGSKeyFieldComp Len 249 RBA 987
Name: DBO.TCALCULATE
Reading ./dirdat/dc000000, current RBA 987, 0 records
Report at 2012-06-05 12:58:14 (activity since 2012-06-05 12:58:14)
From Table tst.table1 to tst.table1:
# inserts: 0
# updates: 0
# deletes: 0
# discards: 1
Last log location read:
FILE: ./dirdat/dc000000
SEQNO: 0
RBA: 987
TIMESTAMP: 2012-06-01 15:48:56.285333
EOF: NO
READERR: 0
2012-06-05 12:58:14 ERROR OGG-01668 PROCESS ABENDING.
Discard file:
Oracle GoldenGate Delivery for Oracle process started, group RTCHECK discard file opened: 2012-06-05 12:49:39
Key column filed3 (0) is missing from update on table tst.table1
Missing 1 key columns in update for table tst.table1.
Current time: 2012-06-05 12:58:14
Discarded record from action ABEND on error 0
Aborting transaction on ./dirdat/dc beginning at seqno 0 rba 987
error at seqno 0 rba 987
Problem replicating tst.table1 to tst.table1
Mapping problem with compressed key update record (target format)...
filed1 = NULL
field2 =
field3 =
field4 =
field5 =
datefield = -04-09 00:00:00
field6 =
field8 =
field9 = NULL
field10 =
Process Abending : 2012-06-05 12:58:14

Mapping problem with compressed key update record (target format)...

Hi Guys,
Getting below error while replication from Source to target. Source table is having NOT NULL Column, but on target replicat process giving error about some NULL value ??
How to overcome this issue, any idea...
2011-08-04 10:35:04 INFO OGG-00995 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: REPLICAT RMASTRK starting.
2011-08-04 10:35:05 INFO OGG-00996 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: REPLICAT RMASTRK started.
2011-08-04 10:35:06 WARNING OGG-00869 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: OCI Error ORA-01407: cannot update ("INFRA"."CUST"."CODE") to NULL (status = 1407), SQL <UPDATE "INFRA"."CUST" SET "ORD_ID" = :a2,"DP_ID" = :a3,"EXCHNG_CODE" = :a4,"ORD_QTY" = :a5,"ORD_PRICE" = :a6,"CODE" = :a7,"MKRT_CODE" = :a8,"CHANN>.
2011-08-04 10:35:06 WARNING OGG-01004 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: Aborted grouped transaction on 'INFRA.CUST', Database error 1407 (ORA-01407: cannot update ("INFRA"."CUST"."SCRP_CODE") to NULL).
2011-08-04 10:35:06 WARNING OGG-01003 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: Repositioning to rba 44132192 in seqno 68708.
2011-08-04 10:35:06 *WARNING OGG-01154 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: SQL error 1407 mapping INFRA.CUST to INFRA.CUST OCI Error ORA-01407:* *cannot update ("INFRA"."CUST"."SCRP_CODE") to NULL (status = 1407), SQL <UPDATE "INFRA"."CUST" SET "ORD_ID" = :a2,"DP_ID" = :a3,"EXCHNG_CODE"=:a4,"ORD_QTY"*
*= :a5,"ORD_PRICE" = :a6,"SCRP_CODE" = :a7,"MKRT_CODE" = :a8,"CHANN>.*
2011-08-04 10:35:06 WARNING OGG-01003 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: Repositioning to rba 44132192 in seqno 68708.
2011-08-04 10:35:06 ERROR OGG-01296 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: Error mapping from INFRA.CUST to INFRA.CUST.
2011-08-04 10:35:06 ERROR OGG-01668 Oracle GoldenGate Delivery for Oracle, rmastrk.prm: PROCESS ABENDING.
Oracle GoldenGate Delivery for Oracle process started, group RMASTRK discard file opened: 2011-08-04 10:35:05
Current time: 2011-08-04 10:35:06
Discarded record from action ABEND on error 1407
OCI Error ORA-01407: cannot update ("INFRA"."CUST"."SCRP_CODE") to NULL
(status = 1407), SQL <UPDATE "INFRA"."CUST" SET "ORD_ID" = :a2,"MKRT_CODE" = :a8,"CHANN>
Aborting transaction on ./dirdat/pm beginning at seqno 68708 rba 44132192
error at seqno 68708 rba 44132192
Problem replicating INFRA.CUST to INFRA.CUST
*Mapping problem with compressed key update record (target format)...*
ORD_QTY = 500
ORD_PRICE = 37430
SCRP_CODE =
MKRT_CODE = N
Oracle GoldenGate Delivery for Oracle process started, group RMASTRK discard file opened: 2011-08-
04 10:35:05
Current time: 2011-08-04 10:35:06
Discarded record from action ABEND on error 1407
OCI Error ORA-01407: cannot update ("INFRA"."CUST"."SCRP_CODE") to NULL
(status = 1407), SQL <UPDATE "INFRA"."CUST" SET "ORD_ID" = :a2,"MKRT_CODE" = :a8,"CHANN>
Aborting transaction on ./dirdat/pm beginning at seqno 68708 rba 44132192
error at seqno 68708 rba 44132192
Problem replicating INFRA.CUST to INFRA.CUST
Mapping problem with compressed key update record (target format)...
ORD_QTY = 500
ORD_PRICE = 37430
SCRP_CODE =
MKRT_CODE = N
Any inputs / help would be appreciated.
Regards,
Manish

The SCRP_CODE column has a NOT NULL constraint. The ORA-01407 error is telling you that you cannot update or set a value for this column to null because of the constraint. This has absolutely nothing to do with an index. You can use a marker/sentinel value in lieu of using NULL. For a numeric field, where everything is positive, a negative value (-1) can be decoded as meaning null. For a character field, a code such as NA can represent NULL.
This also has nothing to do (directly) with GoldenGate failing because of this error. The underlying SQL statement will fail everywhere, regardless of the tool or application. It is not a case of failing only in GoldenGate.

Should surrogate key be numeric?

1. While on a data modleing course at Oracle, I was advised NOT to use numeric data types unless you needed to do numeric/arithmetic operations on the attribute. E.g. Even though an account_id is all digits it should still be declared as alpha numeric ( VARCHAR2) as it is NOT really a numeric. This makes sense.
I would like opnions on the above
2. If above advice is taken, then even a surrogate key should be alpha numeric since it does not require arithmetic/numeric computations. However, I have also heard that numerics are better than VARCHAR2 for joins. If this join performance advantage is true, then using a NUMBER data type for surrogate keys should be preferred over VARCHAR2? This overrides the advice in 1 above for join performance reasons.
I would like comments on 1 and 2.
Thanks
Edited by: user4900730 on Jun 16, 2010 12:19 PM

user4900730 wrote:
1. While on a data modleing course at Oracle, I was advised NOT to use numeric data types unless you needed to do numeric/arithmetic operations on the attribute. E.g. Even though an account_id is all digits it should still be declared as alpha numeric ( VARCHAR2) as it is NOT really a numeric. This makes sense.
I would like opnions on the above
2. If above advice is taken, then even a surrogate key should be alpha numeric since it does not require arithmetic/numeric computations. However, I have also heard that numerics are better than VARCHAR2 for joins. If this join performance advantage is true, then using a NUMBER data type for surrogate keys should be preferred over VARCHAR2? This overrides the advice in 1 above for join performance reasons.
I would like comments on 1 and 2.
Thanks
Edited by: user4900730 on Jun 16, 2010 12:19 PMSybrand gave a pretty good analysis of the performance issues. The instructor you mention just carried away with the idea of 'not everything that LOOKS like a number IS a number'. I think his point is valid for things like SSN and telephone numbers, etc. But my take is that if you are using a "number" for a surrogate key, then it probably really is a number - even if it is true you'd never do arithmetic on it. Comes back to the first lesson of being a dba (quoted in the first two minutes of the first Oracle class I took at their facility in Atalanta, back at version 7.3). "The answer to almost every question is 'it depends'"

Best Practice loading Dimension Table with Surrogate Keys for Levels

Hi Experts,
how would you load an Oracle dimension table with a hierarchy of at least 5 levels with surrogate keys in each level and a unique dimension key for the dimension table.
With OWB it is an integrated feature to use surrogate keys in every level of a hierarchy. You don't have to care about
the parent child relation. The load process of the mapping generates the right keys and cares about the relation between the parent and child inside the dimension key.
I tried to use one interface per Level and created a surrogate key with a native Oracle sequence.
After that I put all the interfaces in to one big Interface with a union data set per level and added look ups for the right parent child relation.
I think it is a bit too complicated making the interface like that.
I will be more than happy for any suggestions? Thank you in advance!
negib
Edited by: nmarhoul on Jun 14, 2012 2:26 AM

Hi,
I do like the level keys feature of OWB - It makes aggregate tables very easy to implement if your sticking with a star schema.
Sadly there is nothing off the shelf with the built in knowledge modules with ODI , It doesnt support creating dimension objects in the database by default but there is nothing stopping you coding up your own knowledge module (use flex fields maybe on the datastore to tag column attributes as needed)
Your approach is what I would have done, possibly use a view (if you dont mind having it external to ODI) to make the interface simpler.

Having problem with dead key in Leopard

Hello. I'm having problem with dead key in Leopard.
In Tiger it was working, but in Leopard dead key "'" doesn't work.
In Tiger I could use OPTION key as well to get Latvian special characters (both worked - OPTION Key and dead-key), but in LEOPARD OPTION key is the only possibility to get these characters.
How can I get dead key "'" working?
I'm using Latvian language with Latvian keyboard layout.
Here are images how it looked in Tiger:
1) Before dead key is pressed:
http://amerix.no-ip.org/tmp/1.tiff
2) After dead key is pressed:
http://amerix.no-ip.org/tmp/2.tiff

ok, i got answer
so - first problem was that ukelele didn't work in leopard (it wasn't possible to start new layout), but it was possible to edit Tom Gewecke posted layout.
There it is possible to change every thing (dead-key as well).
So I edited it and now i have layout where dead-key works -
http://amerix.no-ip.org/tmp/LatvianT.keylayout
Just do what Tom Gewecke said with this layout. It should work!

DataModeler v3.3.0 - Naming standards template for surrogate keys creation

I'm using DM 3.3.0.734 and in the logical model we now can define in the entity properties to "Create Surrogate Key".
When we use Engineer to relational model, for each entity is automatically created a new column using the naming template {entity}_ID as NUMERIC (without precision) and is defined as primary key.
My questions are:
Is possible to define a different naming standard for surrogate key creation?. We define id_{entiry}
Is possible to set the precision of NUMERIC surrogate key?
If we define entity's column as Primary UID, these columns are included in an unique constraint, but are using the naming standards for PK.
As consequence are created the following:
Unique constraint name: entity_PK
Primary key(surrogate)name: entity_PKv1
There is any way to define naming standards like "{entity}_UID" for unique constraints, or even, "{entity}_SK" for surrogate primary key name?
Can anyone help with some of these topics?
Regards,
Ariel.

Hi Ariel,
Naming standards template for surrogate keys creation I logged enhancement request for that.
How to change those bad names (going to change them one by one is not an option):
1) If those "transformed" unique keys are the only ones you have in relational model then simply can apply naming standards
2) You can write transformation script to do that for you
3) you can use new functionality - search, export to excel file, change the names there (using find/replace will be faster) and return changed data back to relational model
you can find description for that here:
https://apex.oracle.com/pls/apex/f?p=44785:24:13179871410726::NO:24:P24_CONTENT_ID,P24_PREV_PAGE:6621,16
http://www.thatjeffsmith.com/archive/2012/11/sql-developer-data-modeler-v3-3-early-adopter-search/
http://www.thatjeffsmith.com/archive/2012/11/sql-developer-data-modeler-v3-3-early-adopter-collaborative-design-via-excel/
You should search for _PK, then filter result on Index and you can export result using report functionality (to XLS or XLSX output format). You can create template and include only table and name (of index) as properties to be included into report.
Regards,
Philip

How to create surrogate key in dimension without unique value

Hi, I have a dimension where there is no column with unique value. I want to add a surrogate key to replace the existing primary key which is derived from concatenating 3 columns(e.g. 'A'||'B'||'C'). I'm thinking of using sequence. But this won't allow me to link the dimension to fact table. How do I come up with surrogate key under this situation? Thanks. ~Tracy

I'm actually trying to accomplish something similar myself.
In my sources I've got two sorts of customers, ones that are directly reported, and ones whose information is provided with sales records (this is stored in module ODS).
Of course identification is different, but in the datamart (module DWH) I'm sort of forced to use an equivalent way of loading (due to the way it first used to work). To accelerate lookups on dimensions, I copy the ODS surrogate key to DWH dimensions, but this does not work for the 'inbuilt' customers because they do not have a surrogate key in the ODS.
They DO have means of unique identification, and at first I thought I could concatenate these (also 3) columns to use as identification code. Unfortunately this is VARCHAR2, where the surrogate key is (naturally) NUMBER.
So now it looks like I'm forced to first build a table in ODS especially for these 'inbuilt' customers and assign a surrogate key (by sequence) to it, this way it conforms to how 'normal' customers are loaded into DWH.
I guess you'll have to pull of the same trick, i.e. create a table with either only the 'translation' of D-code to a surrogate key or all information that is fed into the dimension, which then can be used as a lookup or as complete source when loading data into your datamart.
Good luck, Patrick

Problems maintaning surrogate keys

Similar Messages

Maybe you are looking for