Data Cleansing

Hi,
In my company there are duplicate vendors created under multiple company codes.
Is there any process or tool that is available to check for the duplicate records and then mark them for deletion.
If it is a process could you list them for me.
Thanks in advance.
raj

hi Raj
go to se11 give the table name LFB1 and click displey and click the contents, go to settings in the menu bar format list and select 'choose list' here you give your required fields (LIFNR BUKRS)and execute (F8),
Now goto EDIT in the menu bar select download option, select spread sheet option. It will ask path give the pathname, you will get the excel sheet, now you can do your R&D Work.
regs
arjun
if useful reward points
Message was edited by: Arjun
Message was edited by: Arjun

Similar Messages

Data Cleansing in CRM Web UI

Dear Experts,
I have activated Data Cleansing in CRM 7.0 and it is working.
Can anyone advise on the setting or configuration required to get the CRM Data cleansing functionality to copy communication data stored at the relationship from the Source record to the Master record.
Example:
Company A has a relationship with Contact person A - this relationship has a mobile number and email address maintained.
When merging the account of Company A (source) with a duplicate company A1(master), we need to copy the relationship (already occurs) between Contact Person A and Company A and the relationship level mobile number and email address to the master account which will be company A1 (this does not occur).
Currently the relationship level communication data is not copied.
Please provide step by step config settings to enable this. I already have the SAP Notes on Data Cleansing and have covered all WIKI's on the web. What i need are specific steps to overcome this issue.
Thanks!!!

In order to transfer communication data maitained at the Contact Person relationship level, you have to maintain the required node in transaction BUSWU02: BUP115 Contact's Work Address and BUP120 Contact's Communication Data
If data is still not transferred, SAP has released notes: 1243559, 1491950, and 1493240. to resolve. Check the relevance before implementing.
FK

BP_TASK Data Cleansing

Happy New Year Experts,
I have a question or three for you on Data Cleansing in the Web IC. I will explain what I have done and what I need answers to.
The setup of the Data Cleansing Cases and Account search is fine. We can search for Business Partners and Merge Now or Merge Later and then search for cases if we chose to Merge Later.
When we go into the Case to process it most of the functionality is fine. What isn't ok is this BP_TASK config .
I understand that in order to execute the 'START' button in the Case Processing screens you need to have the Task config setup. I have done this to an extent as described below:
1) Setup Number Ranges - IMG->CRM->Master Data->BP->DQA-> Maintain Number Ranges [Create line 01-0000000001-9999999999]
2) In Task (1003) add the action profile BP_TASK - IMG->CRM->Transactions->Basic Settings->Define transaction types
3) Create job CRM_BUPA_REALIGNMENT (periodic job 5-10 min)
Point 3 is what I am coming unstuck with. I cannot create a periodic job without assiging an ABAP Program to run etc. There is nothing anywhere that says 'use this program or method' when creating the job step.
Secondly, when I select the 'START' button in the Case Processing screen - after confirming the changes, the case errors with a vague message saying the case cannot be saved. However, if I actually hit the 'SAVE' button the case saves and the changes I confirmed are processed between the various accounts. So the whole process is about 95% great and 5% annoying.
Before the questions, all authorisation settings are good as well.
The questions are then:
1) What parameters are required above what I described for the periodic job?
2) Does the 'Task' transaction type need to be in the Business Transaction profile for the specific business role the Data Cleansing functionality is assigned within?
3) Once a task is created, I guess that the job will process these and that a user does not necessarily need to manual process these tasks?
4) Should I change the Action Definition and Condition at all over and above the standard setup that it is currently in?
Any help and guidance would be great - I'm afraid I have 'Wood for Trees' syndrome now
Many Thanks for any assistance.
Regards,
Mat.

Hi Gregor,
I have read your interesting blog. We have a similar kind of data cleansing activity. But when I tried to implement the same, we get a message that the Data Cleansing option is not available for the Insurance industry specific settings. Can you pls help us here ? When I analysed the same the FM 'FKK_CLEANSING_ALLOWED' has a hard coded code element as follows:
*4.71: Event 7500 does not exist yet*
IF 1 = 2.
     REFRESH: t_fbstab[].
     CALL FUNCTION 'FKK_FUNC_MODULE_DETERMINE'
       EXPORTING
          i_fbeve            = gc_event_7500
          i_only_application = gc_x
          i_applk            = applk
       TABLES
          t_fbstab           = t_fbstab[].
     LOOP AT t_fbstab INTO fbstab.
       CALL FUNCTION fbstab-funcc
          CHANGING
            c_cleansing_allowed = cleansing_allowed.
     ENDLOOP.
     IF 1 = 2.
*     für Verwendungsnachweis
       CALL FUNCTION 'FKK_SAMPLE_7500'
          CHANGING
            c_cleansing_allowed = cleansing_allowed.
     ENDIF.
ENDIF.
Our SAP version is as folows:
SAP_BASIS      620         0056 SAPKB62056             SAP Basis Component
INSURANCE     472           0010     SAPKIPIN10     INSURANCE 472 : Add-On Installation
Any help is much appreciated.
Thanks.
I Peter

Domain Names Not Appearing In Drop Down of Data Cleansing Task

For some reason, in several of my DQS data cleansing tasks, the domain names of the knowledge base I choose are not appearing. This is turn leads to an error message stating that "No domain field has been mapped to the input column...".
The task can see the DQS server and it can see the various knowledge bases on that server, but the domains are not appearing.
Has anyone else run into this issue?
Thanks!!
A. M. Robinson

I have just found the same issue.
For me, I believe it is because the domains are only used in matching rules, which the SSIS task does not support. Therefore they are not in the drop down as the SSIS DQS task cannot use them. Only map domains for data correction and you should be ok.

Information Steward 4.2.1 not installing Data Cleansing Advisor (DCA) database

Our situation is as follows:
We have a SAP BusinessObjects BI 4.1 Sp3 system on Windows Server 2012.
We are installing the following components onto a separate, new Microsoft Windows Server 2012 platform (in this order):
- Information Platform Services (IPS) 4.1 Sp3
- Data Services 4.2 Sp1 Patch 3
- BusinessObjects BI 4.1 Sp3 Client Tools (for the BOE Metadata Integrator)
- Information Steward 4.2 Sp1 Patch 3
We verified the cross compatibility between IPS, DS and IS with SAP Note 1740516.
SAP Note 1740516, as referenced in the PAM, lists the following: "BI 4.1 SP3 was officially certified with DS/IS 4.2 SP1 Patch 3".
Our repository databases are located onto a separate Microsoft SQL Server 2008 Enterprise instance and the domain "DS/IS" service account under which IPS, DS and IS are installed, and will operate the Tomcat, DS and SIA services, has full access to these databases.
All required repository databases, including the IS repository, were created at forehand.
IPS and Data Services install fine without any problems. IS also installs fine, without the "Addnode" or Metadata Integrator installation issues that I have seen with previous versions. There were no errors of any kind.
The "Information Steward" and "Data Services" nodes are available in the IPS CMC and can be accessed.
However - the EIM APS is shown as "Running with errors"!
Trouble shooting the EIM APS reveals that the Data Cleansing Advisor (DCA) Service is the culprit.
Selecting the properties of the EIM APS also shows that the DCA Service could not be started and is marked in red. All other entries seem fine.
Once the DCA Server is removed from the EIM APS, the EIM APS will return to a green "Running" status.
This seems to indicate that the DCA Service is faulty. Checking the ..\BusinessObjects Enterprise XI 4.0\Data\DCA-SA\Data directory shows further evidence of this issue, as this directory is EMTPY. The expected DCA database file and the DCA log files are not there!
We have tried a number of things:
1. Removing the DCA service from the EIM APS, restarting the SIA/IPS, adding the DCA service to the EIM APS again.
2. Uninstalling IS 4.2 Sp1 Patch 3, rebooting the server and repeating the installation again.
2. Uninstalling IS 4.2 Sp1 Patch 3, rebooting the server and repeating the installation with IS 4.2 Sp1 Patch 0 (GA release)
3. Upgrading the IS 4.2 Sp1 Patch 0 (GA release) with the IS 4.2 Sp1 Patch 3 version.
None of these have made any difference whatsoever.
I have found 1 post on SCN with the exact same problem: http://scn.sap.com/thread/3475164
Unfortunately, the listed solution of reinstalling has clearly not worked for us.
Has anybody seen this problem before? Any ideas what else we can do to resolve this issue? Our next port of call is opening a SAP support call but we really want to get IS up and running ASAP - so any suggestions would be most welcome!

Hi Erik and James,
This has been fixed in 4.2 SP2 and will be fixed in an upcoming patch for 4.2 SP 1, as well.
Below are steps you can take to work around the issue you are seeing:
1. Copy the files that are in <Install Directory>\InstallCache\im.lac.icc.cdq.cognitivedq.pjs.services.backend-4.0-core-nu\14.2.1.<your build number>\actions\plugins\14.1.2
To
<Install Directory>\SAP BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\java\pjs\services\ICC.DCAService\lib
2. Restart the EIMAdaptiveProcessingServer.
Thank you,
Lynne

Data archiving and data cleansing

hi experts,
Can any one tell me step by step guide for Data archiving and Data cleansing of SAP-ISU Object.
what is the difference between Data archiving and Data cleansing .
Thanx & Rgds

Data Archiving: So many objects are there you can look some of them..........
ISU_BBP IS-U Archiving: Budget Billing Plan
ISU_BCONT Business Partner Contacts (Contract A/R + A/P)
ISU_BILL IS-U Archiving: Billing Document Header
ISU_BILLZ IS-U Archiving: Billing Line Item
ISU_EABL IS-U Archiving: Meter Reading Results
ISU_EORDER IS-U Archiving: Waste Disposal Order
ISU_EUFASS Archiving of Usage Factors
ISU_FACTS Installation Facts
ISU_INSPEC IS-U Archiving: Campaigns for Inspection List
ISU_PPM Prepayment Documents
ISU_PRDOCH IS-U Archiving: Print Document Header
ISU_PRDOCL IS-U Archiving: Print Document Line Item
ISU_PROFV IS-U Archiving: EDM Profile Values
ISU_ROUTE IS-U Archiving: Waste Disposal Route
ISU_SETTLB Settlement Document
ISU_SWTDOC Archive Switch Document
Go to SARA t-code and enter the object CA_BUPA for business partner then Press F6 you will get step by step documentation. please follow the procedure for all the objects.
Regards,
Siva

Need detail information about Data Cleanse Transform.

Hi Every One,
I Need detail information about Data Cleanse Transform.
1. what is Local Cleansing Packages?
2. what are Directories and Dictioneries.
3. Data Cleanse Transform INPUT , OPTIONS, OUTPUT tabs and usage.
4. Details on Transform Input Field name in Data Cleanse INPUT tab.
5. What is PARENT_COMPONENT,FILED_NAME,FIELD_CLASS,CONTENT_TYPE in Data Cleanse OUT POT TAB.
I tried using "EnglishNorthAmericaDataCleanse" transform with NAME_OR_FIRM_LINE1. When I ran the job I got below warning message and my job was success full nut status it shown as failed.
Warning message:
"Trasform Data Cleanse is unable to retrieve cleansing packages from Cleansing Package Builder. Only local cleansing packages are available. Info: Unable to log on: Could not connect to server d184932040.sng1.attens.net:6400. Please check that the server name is correct, and that the server is running. "
Please make me understand, why did I get above message.
Thanks in Advance ...

Hello,
Cleansing package is a reference file that contains content and rules around person, firm, title, phone data and so on. Cleansing package used to be referred to as dictionaries. Directories are usually referred to address reference data or address directories. The input, output and option tabs are similar to other Data Quality transforms. For detail information on what each of the input fields and output fields are and how the options work can be found in the Data Services’ Designer Guide (http://help.sap.com/bods).
The warning message you are getting is stating that Data Services is not able to connect to the server to check to see if you have made modifications to the PERSON_FIRM cleansing package. You are not required to make any changes to the PERSON_FIRM cleansing package, but can change rules or content based on your organizational rules. If you want to make changes you will need to install SAP Information Steward. If you would like to install SAP Information Steward you can find documentation for installation on the Help Portal, http://help.sap.com/bods, along with some great Videos.
You can also find some great Data Services Product Tutorials on the SAP Community Network, at http://scn.sap.com/docs/DOC-54115.
Thank you,
Lynne Lintelman

Are there any set guidelines for customer master data cleansing.

1) I was wondering if there are any set guidelines for customer master data cleansing.
2) Part of the cleansing effort involves consolidating number of divisions etc. Any pointers to this regard?
3) Also how do we deal with open docs, billing plans etc?

The data cleansing requirement will take place, while move the data from legacy system to new target ERP / system. I.e. need to cleanse, enrich, de-duplicate, & standardize.
The existing data may not have consistent format since it is often being derived from various sources. It may contain duplicate information as well missing or incomplete information. Cleanse and normalize content to achieve accuracy, consistency and proper understanding of the data.
Also its a process of organizing the data, which will ensure enrichment of your data which are up-to-date, accurate, and complete. These processes are manual / labor intensive and require a fair bit of specialization.
If you need to have more details please feel free to interact with us as per below.
Thanks
Kumar
www deebrostech.com

Problem with universal data cleanse

I have problem with universal data cleanse.
I am using DS 3.2.x (12.2.2).
I created:
Dictionary: TEST
Classification: TEST_CL
Custom outputs: category: TEST_CAT
Dictionary Entry:
Primary: BO
Classification: TEST_CL
Gender: Unassigned
Secondary information:
When used as: TEST_CAT
Standard: Business Objects
Rule file:
DataCleanse Rule File v2.0;
TEST_CAT = TEST_CL;
action = TEST_CAT;
TEST_CAT = 1 : TEST_CAT : 1;
end_action
Data Cleanse:
Input: Multiline1
Options:
Parsing Dictionary: TEST
Rule file: .../test_rules.dat
Break On Whitespace Only: Yes
Parser Sequences Multiline1: TEST_CAT
Output:
Parent_component: TEST_CAT1
Generated_field_name: TEST_CAT and RULE_LABEL
Generated_field_class: parsed/standardized
Contenet_type: none
and EXTRA field
I wanted to replace word u201CBOu201D with the standard words u201CBusiness Objectu201D, but I have result: there is "BO" in the extra field,
there are "null" in others fields.
What am I doing wrong?
Thanks for all help!
P.S. I don't have cleansing packages installer.

There seem to be a couple of things going on here:
1. If you are using your custom dictionaries, then you have to map your input to MULTILINE1 and enable the custom parsers - just something to be aware of
2. You mentioned that you made some changes to the existing dictionary and you are not seeing any changes. To be clear, do you have different TEST and PRODUCTION environments? Or is it the same environment except that you have a local DS repository and for the dictionary you are pointing to another repository (using Dictionary --> Manage Connections)?
Having the dictionaries on a different repository should not make any difference as long as you point to them in your designer using the Dictionary --> Manage Connection option.
So I think there may be some issue with your job setup and/or dictionary values need to be looked at. You can start by adding another output field named "EXTRA" to see whether or not your data is getting parsed at all. Also, make sure the entry "CLEANME" is classified as FIRM_NAME_ALONE in the dictionary and that you are selecting the correct dictionary name in the Datacleanse Transform options.

Dependencies visible in Web UI for Data Cleansing

Hi All,
My client is using the functionality Data Cleansing and we are implementing CRM 2007. As of CRM 2007 we now have two variants in BUSWU02 - CLEARING and CLEAR_REP. The dependencies(objects) defined in CLEARING are visible in the web UI.
and objects defined in CLEAR_REP are not visible in the Web UI.
I need to see the object -Business Transaction in the web UI in the Data Cleansing Case page. For this i have added the object Business Transaction in BUSWU02 under variant CLEARING and removed it from CLEAR_REP. It is also actviated in BUSWU01 to be picked up in Data Cleansing. The problem is the Business transactions are still not visible in the Web-UI.
Anybody who faced this problem before, please do let me know the solution to this.
Thanks in advance
Shiloo

Dear Shiloo,
please see information given in note 900293 (especially point 4) and 1055480. Hopefully they are able to help and get your customizing correct!
Regards, Gerhard

Data Cleansing Terms Clarification

Folks,
Another question and thanks again to all those that have been helpful so far.
In a data flow that I am building I have both customer information and address information. I plan on running my data both through an English base data cleanse and a Canada address cleanse.
I notice that there are many options for the same output field. The difference being the generated field class. I am confused as to what is the difference between Parsed and Standardized in regards to the base data cleanse.
Also I am confused that there are extra types such as alternate or none. Also in dealing with address cleansing there are other items such as generated field category and generated field addrclass which also seem to have an impact on the data that is output from these transforms.
I would like my transforms to correct error data and also if it is possible add items such as city or province when they are missing based on the postal code which is there for example. I have gone through both the designer guide and reference guide in regards to these terms, it has left me more confused than anything.
Which way of implementing this would be the best.
Thanks in advance,
Bill
Edited by: William Grdovich on Oct 5, 2010 4:13 PM

Bill,
I notice that there are many options for the same output field. The difference being the generated field class. I am confused as to what is the difference between Parsed and Standardized in regards to the base data cleanse.
Parsed means that the address has been separated into its components (house numnber, street name, city, etc). Standardized means that the address has been parsed, and each component value has been corrected, updated or enhanced.
Also I am confused that there are extra types such as alternate or none.
An altenate type means that an alternate value is available. For example, in New York City, 6th Ave (the official name) is also known as Avenue of the Americas (the alternate name). If a field has type 'None', it means that there is only one type associated with this field.
Also in dealing with address cleansing there are other items such as generated field category and generated field addrclass which also seem to have an impact on the data that is output from these transforms.
A description of the field category columns is provided in the SAP Business Objects Data Services Reference Guide, Data Quality Fields, Global Address Cleanse fields.
I would like my transforms to correct error data and also if it is possible add items such as city or province when they are missing based on the postal code which is there for example.
Use Generated Field Class 'Best', Generated Field Category 'Component, Generated Field Addrclass 'Official'. If your selected output field has Generated Field Class 'None' then use Generated Field Category 'Standardized'.
Paul

Data Quality Data Cleanse Transform SQL Connection Not Open

Iu2019m working with the previously installed version of Data Services and there are errors encountered with Data Cleanse transforms not having all the metadata. When validating or double-clicking the NA Eng DC transform the following Option Group/connection not open message is returned.
Option Group Error (Option Group: Datacleanse transform): The SQL connection returned by the Metadata Repository is not open.
Iu2019ve verified substitution parameters (including using those from another install in the environment which is functioning correctly), re-synched repos, database connectivity, amongst other things. Connectivity to the repository is otherwise behaving normally. Does anyone recognized this problem, know the resolution?

Todd,
In Data Services Designer, there is a Dicitonary > Manage Connection option that is used to access the Cleansing Package (CP). Note: You must seed the CP within Repository Manager. It sounds as if you may not have established a valid connection inside of Designer causing the error you are seeing.
Thanks,
Paula

Missing Data Cleanse package option in Data services Repository Manager

Hello, I am trying to create a repository for data cleansing package repository in database using Repository Manager, but the drop down list box is not giving an option "Data Cleansing" to create repository. Only three values are available (Local,CEntral, and Profile).
Using Data Services 3.0 Version. Any help or any workarounds creating a repository with our Repository Manager.
Thanks,
Kiran

We searched log files and found no errors.
We then decided to attempt re-creating the local repository. We checked the Show Details box.
The local repository was reported has having been successfully created. This was the result we had before.
We then checked for canned functions and the list matched to those of our Oracle and SQL Server local repositories.
So, best guess is that something went wrong on the initial local repository creation that was either not reported or reported, but we missed.

Experimenting with Universal Data Cleanse

I am working with Universal Data cleanse and custom dictionaries for a project I am working on.
The goal is to clean the leagacy firm information with standards. The issue I am having is using the dicitonary, how to I replace words with the standard word.
For example, the text contains the word: CLEANME
I want to have the dictionary replace that with the standard: CLEAN ME
IBoth were added as dictionary entries, one at a time, and the other as the Alias or the Standard in that dictionary entry. I get no results.
Reading the documentation, it seems I should be setting it up like this:
Dictionary Entry: CleanMe
Classification: Firm_Name, Firm_Name_Only
Gender: Unassigned
Standard: Clean Me
Thanks for all help!

There seem to be a couple of things going on here:
1. If you are using your custom dictionaries, then you have to map your input to MULTILINE1 and enable the custom parsers - just something to be aware of
2. You mentioned that you made some changes to the existing dictionary and you are not seeing any changes. To be clear, do you have different TEST and PRODUCTION environments? Or is it the same environment except that you have a local DS repository and for the dictionary you are pointing to another repository (using Dictionary --> Manage Connections)?
Having the dictionaries on a different repository should not make any difference as long as you point to them in your designer using the Dictionary --> Manage Connection option.
So I think there may be some issue with your job setup and/or dictionary values need to be looked at. You can start by adding another output field named "EXTRA" to see whether or not your data is getting parsed at all. Also, make sure the entry "CLEANME" is classified as FIRM_NAME_ALONE in the dictionary and that you are selecting the correct dictionary name in the Datacleanse Transform options.

Customize Data Cleansing

Hi Everybody,
today I've started to customize Data Cleansing in our CRM System to get rid of all the Duplicate Accounts we have. I have two main problems:
1. I create a cleannsing case in Trasaction BP by marking two Business Partners and right-click to get the context menu and choose "create data cleansing case". The cleansing case is sucessfully created.
Now I start Transaction BUPA_CLEAR and open this case. In a popup I'm asked for "No Referencing Objects Are Registered for the Current Case. Should Determination Now Take Place Online? (All Changes to the Last Processed Case Are Saved)" I choose yes. Now a divided window appears which shows me both partners. What I expect now from the transaction is to show me also the Business Transactions related to the Business Partner. I've checked in TA CRMD_BUS2000126 that there are Transaction for that partner, but they don't appear.
What must I do in the customizing? The documentation at <a href="http://help.sap.com/saphelp_crm40/helpdata/en/35/16dc2c1f9911d6b1d200508b5d5211/frameset.htm">Data Cleansing</a> and in SPRO does not rearly help.
2. When I leave BUPA_CLEAR after I've opened a cleansing case I get the info message "Customizing is missing (table TBZ5*)" two times. What could be the case for that?
Regards
Gregor

HI Gregor.
Did you eventually find the solution? I think I have the same problem and I think I did what was to be done in customizing.
One thing about which I'm not sure are the tresholds.
the F1 says:
If the duplicate check is not called in dialog (e.g.BAPIs, Batch Input or direct input),
no dataset is created if a duplicate is found with a higher percentage match than the specified threshold.
but I don't see any difference if I put 99 instead 0.
The main problem is that in fact I can compare nothing.
There are no subtrees to expand.
Could you please take a quick look at customizing I did for it and its results?
I put some screenshots on my website: http://www.lucasdippel.vel.pl/
Any comment would be appreciated.
LUCAS

Running a Data cleansing project through ODI

Hello everyone,
I'm trying to run a data cleansing project in my odi package, i already exported the project and when i try to run the task in ODI operator looks fine but nothing happens to my output file in Data quality.
All the configurations seems to be ok.
Anybody can help?
thanks,
Questbr

Hi Balaji,
R u able to run that ODQ project independently ?
Reagrds,
Rathish A M

Data Cleansing

Similar Messages

Maybe you are looking for