Data Conversion Design Patters

I'm looking at building a conversion program that will import data from several different formats and convert into one common format. The convertor should simply be pointed to a database or a flat-file and it will extract the data and populate tables in the target database. All the logic is mind-numbingly simple, but as far as an overall design, what are your thoughts?
For example, should I build a separate tool that does validation of source data? Or should the validator be part of the convertor?
Are there design patterns out there that anyone can recommend?

This problem u can go for strategy pattern. A strategy is nothing but an algorithm to reach the soution.
Strategy pattern deals with different algorithms(strategies) to achieve the same result.The client will have some indicator. Using the indicator, stategy manager will decide which algorithm to invoke.
In ur case,the algorithm may differ depending on the data format. For (eg) if you have five different formats of data , then you may have to write five different type of logic to achieve the solution.
So firt of all u need to identify the strategies u r going to use.
Now you may require following classes
1) SourceReader -- > this class is responsible for getting the data either from file/database.It just reads and holds data.
It identifies which strategy to use by parsing the data . No data processing happens here
2) StrategyManager --> Strategy manager is the one who reads the data from source reader and decides and instantiate the strategy to use.
3) Strategy --> It could be an interface or an abstract class. It can be decided upon any common operation involved irrespective of data           format.
4)ConcreateStrategy --> This implemenattion class, here actual data extraction process will happens.
5) DataLoader --> this calss loads the data in to the destination database
6) YurEntity --> The data to be populated can be kept in the form object. if ur object contains many attributes, we can have data in the           form objects. (this class is required only if all the data are realted and fall in to same logical group).
Class SourceReader{
Vector data;
public void do(){
readData()
StrategyManager.getInstance().process(data,findStrategy());
public void readData()
     read from file or db and populate data
public int findStrategy()
     logic for identifying format
class StrategyManager
public static StrategyManager sm = new StrategyManger();
public StrategyManager getInstance()
return sm;
public void process(Vector data,int indicator)
     //read the indicator and instantiate the appropriate startegy class
     Strategy s = new Strategy1();
     s.process();
abstract class strategy{
public abstract void parse();
public void validate{};// if the validation logic is common, u can have implemenation here itself.
public updateDB()
     update destination db;
public void process()
     validate();
     parse();
     updateDB();
}

Similar Messages

  • Crystal Data Conversion Issue (Error converting data type varchar to datetime)

    Hi,
    I can run stored procedure without error in SQL Server using my personal credentials as well as database credentials.
    I can also run Crystal Report after connecting to Stored procedure without error on my desktop using my personal credentials as well as database credentials.
    But when I upload the crystal report in BOBJDEV and when I run using database credentials report fails saying that "Error in File ~tmp1d1480b8e70fd90.rpt: Unable to connect: incorrect log on parameters. Details: [Database Vendor Code: 18456 ]" but I can run the crystal report successfully on BOBJDEV using my personal credentials.
    I googled (Data Conversion Error Message) about this issue & lot of people asked to do "Verify Database" in Crystal Report. So I did that, but when I do it I am getting a error message like this:
    Error converting data type varchar to datetime.
    Where do you think the error might be occurring? Did anyone faced this kind of issue before? If so, how to resolve it?
    (FYI, I am using Crystal Reports 2008, & for stored procedure I have used SSMS 2012 )
    Please help me with this issue.
    Thanks & Regards.
    Naveen.

    hello Naveen,
    since the report works fine in the cr designer / desktop, we need to figure out where you should post this question.
    by bobjdev do you mean businessobjects enterprise or crystal reports server? if so please post this question to the bi platform space.
    -jamie

  • Data Conversion for BOMs and/or Routings

    I am looking for any examples for design spec for Data Conversion of Bill of Materials and/or Routings.
    Not sure if this would be in form of MD050 but any example would be appreciated

    Hi;
    Please check below links:
    Oracle Bill of Materials User's Guide
    http://download.oracle.com/docs/cd/A85683_01/acrobat/115bomug.pdf
    http://horizon.more4apps.com/doco/BMWRelease/output/index.htm
    Also see:
    Getting Started With R12 - Discrete Manufacturing Support Enhanced Release Content Document [ID 415922.1]
    Regard
    Helios

  • Data Services Designer Issue with Job Server

    I am on Business Objects Edge Pro using the Data Services Designer.  When I try execute a job I get "The job server you have selected is not working."  In addition to that when I try to make any personal change to the environment I get a BODI-1260016 error.  Finally when you go to the help and about they show both the Job Engine and Job Server not responding. 
    I have opened up my CMC and it shows all servers enabled and functioning.  I do not know where to go from here.
    Thanks
    Cassidy

    Voila. I know I am a bit late to the conversation, but here was my solution. I was running Designer on my local machine. We also have Designer on the server. So I decided to remote to the server and compare settings. When the server desktop came up, good old Windows was there, announcing that BODS had changed since the last time it was run, and would I like to allow it to run (Y/N/Cancel)? Thanks, Windows. I said Yes, then went back to my local workstation, tried again to run a job, and no problem.
    This has happened with other software services (scripted ftp for example) that run on servers. Seems it can happen after a Microsoft Tuesday, or after software is upgraded. Always remember to log on to the server and clear any security dialogs that are preventing your service from running. Not really a BO solution, but it was my solution. YMMV.

  • GL Data Conversion from Legacy to Oracle EBS

    Hi All,
    I am tasked with converting legacy data into GL. I have worked on other conversions like supplier, AR and AP conversions but never worked on GL conversion, so I don't know how and where to start, what are the pre-requisites for entering data into gl_interface table how to ensure that data is correct in the .csv file etc.
    Kindly request you all to guide me in right direction. Hope to hear from you all very soon!!! :)
    Regards

    Hi,
    When you talk about GL-Data it is the trial balance to be loaded from legacy to oracle applications.
    You can use Web-ADI to upload it ,the check list are
    The balance for each account comibnation in the legacy system to be mapped to GL-Oracle code combination balance
    With respect to open AR, AP Invoices if the invoices are converted using a control account for migration in GL then
    the the balance transfered from AP,AR need not be reversed in GL from the source receivables and payables.
    In case if the same account combination are for migrating the balances from AP,AR to GL ..Then the balances transferred from AP,AR should be reversed ..so that it does not affect the TB-GL-Balance
    Finally ensure that the balance for the TB-tallys with your legacy system and upload it using Web-ADI ,import and
    review and post it.
    Hope this points helps your GL-Data conversion.
    Regards,
    Ramaa

  • Data conversion strategy for new SOB

    Dear Viewers
    on 11.5.10
    We are creating a new SOB with a change in currency from Feb-11 as this is the requirement
    For the same, we need to do data conversion.
    I have a confusion for Purchase Orders and Sales Orders
    Purchase Orders:
    Open purchase orders will be converted, means the unfulfilled PO`s i.e the ones not received and are fully open.
    The PO`s which have been recieved but not delivered, Requested the users to clear the intransit receipts.
    The PO's which are partially received, what to be done for them?
    If a PO is fully received and Delivered will not me converted to the new SOB as its not an open PO
    but If invoice comes after Feb-11, than how the matching will be done?
    What if a return has to be made moving forward in FEB-11 under new SOB
    Sales Orders:
    Open sales orders will be converted, that is the ones that have been entered and not yet booked.
    Users have been requested to clear off the Sales order lines which are already pick confirmed but not yet shipped, hence they will be shipped and interfaced to AR
    For the Sales orders that have been booked, those lines that are not yet processed further will also be converted.
    Now what if a receipt comes after feb 11, how to handle this as the sales order wiould not have been converted?
    Please give your advise on the data migration strategy for PO`s and SO's.
    Please do add any point that may have been missed by me
    Appreciate your help
    Thanks
    Emm

    Hi David,
    for master data conversion you can use LSMW and the RE-FX BAPIs. (please refer to SAP note  [782947|https://service.sap.com/sap/support/notes/782947] ).
    Regards, Franz

  • Data conversion for new sob

    Dear Viewers
    on 11.5.10
    We are creating a new SOB with a change in currency from Feb-11 as this is the requirement
    For the same, we need to do data conversion.
    I have a confusion for Purchase Orders and Sales Orders
    Purchase Orders:
    Open purchase orders will be converted, means the unfulfilled PO`s i.e the ones not received and are fully open.
    The PO`s which have been recieved but not delivered, Requested the users to clear the intransit receipts.
    The PO's which are partially received, what to be done for them?
    If a PO is fully received and Delivered will not me converted to the new SOB as its not an open PO
    but If invoice comes after Feb-11, than how the matching will be done?
    What if a return has to be made moving forward in FEB-11 under new SOB
    Sales Orders:
    Open sales orders will be converted, that is the ones that have been entered and not yet booked.
    Users have been requested to clear off the Sales order lines which are already pick confirmed but not yet shipped, hence they will be shipped and interfaced to AR
    For the Sales orders that have been booked, those lines that are not yet processed further will also be converted.
    Now what if a receipt comes after feb 11, how to handle this as the sales order wiould not have been converted?
    Please give your advise on the data migration strategy for PO`s and SO's.
    Please do add any point that may have been missed by me
    Appreciate your help
    Thanks
    Emm

    emm wrote:
    Purchase Orders:
    Open purchase orders will be converted, means the unfulfilled PO`s i.e the ones not received and are fully open.
    The PO`s which have been recieved but not delivered, Requested the users to clear the intransit receipts.
    The PO's which are partially received, what to be done for them?
    If a PO is fully received and Delivered will not me converted to the new SOB as its not an open PO
    but If invoice comes after Feb-11, than how the matching will be done? <br>Business needs to take a decision whether they are fine with matching the POs manually (i.e. referring the documents and verify), in this case you may capture the PO information in a DFF in the Invoice distribution. Otherwise if it has to be converted identifying the POs under this scenario, you may consider converting those maintaining receipt close tolerance as 100% and matching type as 2 -way (again business approval needed to handle audit issues) in order to avoid the receipts/delivery conversion etc.<br>
    What if a return has to be made moving forward in FEB-11 under new SOB<br>Ideally returns can be done using Miscellaneous/Account Alias Issues specifying the appropriate transaction reasons to clarify the scenario.<br>

  • Data Conversion in SAP E-Recruiting.

    Hi SAP E-Recruiters,
    Our client is using a Legacy application for recruiting candidates where it has 5000 candidates data base (internal and external candidates' applications).
    Now they are asking for movement of this whole data to the newly implementing SAP E-recruiting system. We are exploring various data conversion methods and found that BDC, LSMW will not work for E-recruiting data upload as we have every thing in BSP/Web Dynpro pages here.
    We need your help in finding a solution for moving this data from legacy to SAP E-Recruiting. Pls do the needful.
    Thanks,
    Sudheer

    Sudheer,
    So far there are no SAP delivered migration methods or BAPI's available conversion of candidates/applications, as you have already specified BDC's wouldnot work. Internal candidates always can be brought over in eRecruitment with integration techniques between PA and eRecruit depending upon the landscape.
    However, for external candidates custom conversion programs have to be written by making use of methods/function modules that are used in BSP/WebDynpro.
    regards
    Sridhar

  • Billing Legacy master & transactional data conversion/migration in SAP ISU

    Hi Experts,
    Please let me know, in ISU implementation project, what is Billing related Legacy master & transactional data conversion/migration is done in SAP ISU system.
    Thanks in advance

    Pankaj,
    There is no straight forward transaction to download the Configuration data and export it to a file.
    However, you can use the BC set functionality to extract the IMG node with its contents.
    Check the following  SAP help portal link for more info on BC sets
    Business Configuration Sets (BC-CUS) - SAP Library
    The archive data that you are talking about; Is it master data or transaction data?
    Also, what this data is related to-Financial Documents, Billing related info?
    There is one standard transaction SARA through which based on the migration object you can archive the data and export it into a file in a particular location..
    Hope it helps..
    Thanks,
    AB

  • Data conversion for New GL - Going live during fiscal year

    Hi Experts,
    My client is going live on Oct 1 (fiscal year in Jan -Dec). So for data conversion we need to load P&L and BalanceSheet as of Sept 30, 2009.  I need your help to resolve few open issues. (we have activated Doc Splitting with Zero Balance and Segment )
    1) When we load P&L balances - how can I assign Profit Centers to all these P&L iitems .  
    2) What is the procedure (best practices) to assign Profit Centers to all the Balance Sheet items since we are going live during Fiscal Year
    3) When we process open items  (AR, AP) in month of October how the Profit Center assignment will work?
    (My client will be using Profit /Cost center terminology first time, so we need to map all the B&S and PnL items to Profit centers)
    4) What is the best way to map all the Balance Sheet GL accounts to Profit Centers.
    Sorry for asking too many questions. Any help would me very much appreciated.
    Thanks,
    Sam

    1) When we load P&L balances - how can I assign Profit Centers to all these P&L iitems .
    Each P&L account balance has to be upload with cost centre, then automatically PC will be triggered.
    2) What is the procedure (best practices) to assign Profit Centers to all the Balance Sheet items since we are going live during Fiscal Year
    Let client decide the profit centre for balance sheet items, since they knows which balance relates to which pc
    3) When we process open items (AR, AP) in month of October how the Profit Center assignment will work?
    While uploading AR,AP balances, each line item to be uploaded with profit centre, automatically in october PC assignment will work.
    (My client will be using Profit /Cost center terminology first time, so we need to map all the B&S and PnL items to Profit centers)
    4) What is the best way to map all the Balance Sheet GL accounts to Profit Centers
    Try to identify each balance sheet gl balance to each profit centre, this will help in future reports. otherwise put in common pc

  • Can anyone Explain about Data conversion for Material master In SAP MM

    Can anyone Explain about Data conversion for Material master, Vendor  In SAP MM
    Thanks

    Hi,
    Refer following link;
    [Data Migration Methodology|http://christian.bergeron.voila.net/DC_Guide/Data_Migration_Methodology_for_SAP_V01a.doc]

  • R/3 Date Conversion problem

    How to convert the R/3 Date to Array.

    If you are receiving a date field from R/3, please use <b>SimpleDateFormat</b>(class) to re-arrange your date in any fashion. Along with the method <b>format</b> you can just have your date in any format :
    DD-MM-YYYY
    MM-DD-YY
    DD-MMMMM-YYYY (Month will be spelled out as text)
    All you need to do is :
    SimpleDateFormat sf = new SimpleDateFormat("DD/MM/YYYY");
    sf.format(myDateField);
    Since R/3 date conversion is usually for this reason, I mentioned the above class.
    Regards,
    Subramanian V.

  • Question - new to BO Data Services Designer - how to automatically transform varchar column data to upper(varchar) across all columns in a table?

    Hello -
    New user to BO Data Services Designer. Company is using Data Services Version 12.2.
    I have many tables that all need the same transformation- converting varchars fields to upper(varchar) fields
    Example:
    I have a table called Items. It has 40 columns, 25 of which are of type varchar.
    What I have been doing is ....
    I make the Item table as the source table then create a Query transform that is then attached to a new target table called - ITEMS_Production.
    I can manually drag and drop columns from the source table to the query and then in the mapping area I can manually type in upper(table_name.column_name) or select the upper function and then select the table.column name from the function drop down list.
    Obviously, I want to do this quicker as I have to do this for lots and lots of tables.
    How can set up Data Services so that I can drag and drop an upper transform quickly and easily or automate this process.
    I also know Python-Java so am happy to script something if need be.
    the logic would be something like -
    if there is a column with a type varchar, add the upper() transformation to column. Go to next column.
    Excited to be using this new tool-
    Thanks in advance.
    Ray

    Use the DS Workbench.

  • Data Conversion Logic

    I am now trying to implement a data conversion logic(such as UOM field), I want to load all the data conversion rules in the cache from file and keep them in the cache. So when mapping starts it just picks up the data conversion rules from the cache.
    Using GlobalContainer it is ok for one mapping even several UOM fields(just load from file once). But it will load from file again when the mapping is triggered again.
    Is there any logic I can use to implememt the data conversion just load from file once?
    Regards,
    Nick

    Hi Arnold,
    normally every input should be made in LC, why you enter the data in group currency?
    However if you have to start from group currency the only solution to avoid this error is to put at the end of sprunconversion the ON_ERROR_CONTINUE clause.
    But remember he will stop to notify every type of error so you have to be very careful and have this clear in mind when some issue occurs with the currency conversion.
    Regards
         Roberto

  • Data conversion

    Hi,
    What does a data conversion mean. We are doing a full cycle implementaion.
    For example in Cost center accouting do we need to create cost center hierarchy structure as well as all the cost centers master data under the cost center groups or just need to create just the hierarchy not need to create the cost center master day will it be created while they do the data conversion.
    Please provide some help.
    Thank you!!!!

    It is csv file ,
    start_date            end_date
    01/08/2014         01/01/2015
    01/10/2014
        02/02/2015
    In SSIS Package , I tried to use Derived Columns i.e database date ,
    in expression  : (DT_DBDATE)((contract_start_date == (DT_DBDATE)"0001-01-01") ? (NULL(DT_DBDATE)) : ((DT_DBDATE)contract_start_date)) 
    but it failed
    What i need to do , please suggest .

Maybe you are looking for