What does the Merge Data Stream processor do when there are multiple input streams to it from the same reader?

Hi,
I have a process with a reader of master data that outputs 5 records that feeds simultaneously into 3 different lookup and return processors.
Each lookup and return processor brings some data back from a detail table. There can be multiple details so I follow each lookup processor with a split records from array processor. Hence I end up with 3 'streams' of data. Stream 1 has 8 records, stream 2 has 5 records and stream 3 has 6 records.
I join all these streams to a Merge Data Streams processor.
I end up with 9 records so although the help for the Merge Data Streams processor says 'Merge Data Streams does not perform any transformation, matching, or merging of records' there is clearly some merging going on.
What is the behaviour of the merge data streams processor in this scenario?
I have added attributes and flags into each of the streams. How many records should I see and what values should the added attributes/flags have (some records show attributes/flags from all 3 streams whereas others show just those attributes/flags from one stream).
I have developed a test case simply to understand what the processor is doing but it's not obvious and furthermore it's probably unwise to develop EDQ processes where the processor behaviour is not documented and guaranteed to remain consistent. What I am trying to achieve is to bring all of a person's (the master data) various details (assignments, employers, etc.) together so we can check the data (some rules require data from multiple details).
Thanks, Nik

Cheers Mike - and for the explanation of the terms.
I think I understand now how it's supposed to work.
What I'm finding however is that when I set a flag to Y at the beginning of a path (that includes a lookup and return and then split records from array processor) that flag is showing no (i.e. an empty) value in SOME of the records shown in the subsequent MDS processor (it's fine the very last split processor before we get to the MDS but then again there are fewer records in that split processor than the MDS).
In my case there are obviously more records in the MDS processor than there were in the original reader (because the lookup and returns are configured to have unlimited maximum matches). As mentioned, the different paths return different numbers of  records before being combined in the MDS. Say a reader has 5 records and path 1 returns 8 records in total including a path-specific flag (flag1, set to Y) but path 2 (that again adds its own path-specific flag (flag2, set to Y) returns just 5 records (since nothing was added from the lookups) are you saying that flag2 would show as 'Y' for all 8 records shown in the MDS?
Hopefully you would be able to see what I mean if you try to create a process like the one I've described (or I can upload a package).
Re. the purpose of the separate paths approach it is simply to allow the visualisation ('showing the working' as Neil puts it) of the different checks being carried out by the process.
This is considered one of the benefits of the tool over writing SQL queries (with outer joins, query criteria, etc.).
Also, as mentioned I was following an example that Neil put together for us to ensure that we are doing things in a 'proper' and supported way.
If we put all the lookups, etc. for all the checks into one datastream then it no longer becomes so understandable and the value of joining processors in a process over simply writing SQL becomes questionable; arguably the EDQ process in fact becomes less easy to understand than simply writing SQL.
Also, to go down this route I will need to revise the (what was previously substantially working until I revised it) processes that I have already developed.
Thanks, Nik

Similar Messages

  • Why does Lightroom append "-2" to file names when there are no duplicates.

    I have been using Lightroom since V. 1. I have noticed that many of my file names have been renamed in LR (now V5.3)  by changing the filename from "XXX_yyyy.NEF" to "XXX_yyyy-2.NEF". IF I look in the folder where the images are stored, both versions of the file are there. I went into LR and deleted the "-2" version and re-imported the original files. In some cases this worked and the LR file now has the original name. In some other cases, LR re-named on import and appended the "-2" to the filename. I checked and it turned out there were other files with the same name on the disk and in the LR catalogue, so I understand why these files were renamed. In other cases, however, there is no other file in LR with the same name. Yet, when I try to delete the "-2" file in LR and re-import the original file (as I did earlier),  LR again renames my file by appending the "-2" suffix to the filename. It does not copy the file, it actually renames the file in the folder where it is stored. I cannot understand why the filename is being changed in this case.
    Can anyon tell me why LR is doing this and how can I prevent it?
    I am runing LR under 64-bit Win 7 with 12 GB RAM, the problem seems to have occurred primarily with Nikon D200 RAW (.NEF) files.
    TIA
    Winslow

    Winslow98 wrote:
    Can anyon tell me why LR is doing this
    Yes - it's due to a bug, or if you prefer: a "design/implementation limitation". Sorry: I'm not qualified to provide details...
    Winslow98 wrote:
    how can I prevent it?
    Somebody else may be better qualified to answer - I never figured out exactly what causes it. Two ideas I have though (take with salt / refer to initial disclaimers...):
    * Export all photos into a new catalog (it's possible it'll leave the offending / problem-causing junk behind - dunno).
    * Use SQL to hack your catalog - look for clues in "Import Hash" column of 'AgLibraryFile' table (and if you see problem - fix it...).
    * Try deleting .agprefs file (in with your presets, 'Preferences' subfolder). Not sure if this has anything to do with it - I don't think so.
    Sorry I can't be more help (and consider holding out for a better answer),
    Rob

  • How to I toggle the adjustments from a single mask when there are multiple masks of the same kind?

    If I have multiple brush masks how can I toggle just one of them on and off; for instance just the active pin.  I know I can turn off all of the brush masks at once but I just can't figure out how to toggle them individually. Thanks!

    I think I know what you mean but if you do Reset it resets ALL the brushes instead of just the one.  The OP just wants to toggle one brush.
    To toggle one brush.
    Click on the brush you want to toggle.
    Double click the word "Effect:" int the upper righthand of the slider box,  This resets the sliders to zero
    To reverse, you can click undo
    To see the difference many times, just click back and forth in history to see the changes before you do the undo.

  • When there are multiple users can the email files that are on the left of the screen be hidden from other users?

    There are several people who might want to use Thunderbird on a work computer but I don't like how everyone's email files are open to public view. It looks like anyone can see emails without having to login to another person's account.

    Use different Windows user accounts for each person, and set-up Thunderbird for each person's email account underneath the respective Windows account.
    It is very easy to switch between Windows user accounts, and multiple people can be logged in simultaneously.

  • How do I make the slide "pause" when there are multiple buttons present?

    I'm creating a selection slide that will allow the user to click button 1, 2, 3 or 4 to jump to the first slide of a topic group or to click button 5 to exit.   The slide should just "stop" until any of the buttons are clicked.  Unfortunately, the slide ALWAYS goes to the jump for button 1 without waiting for any action.
    I've tried adding an invisible click box and an invisible text field with no success.
    What am I missing or doing incorrectly?
    Thanks in advance for any guidance.

    Hi there
    Yep, I can click the image and it scales back up.
    You need to look at the properties for your button objects. I can easily see on the timeline that you have apparently cleared the pause point for each of them. If they had a pause point, they might look like this:
    Cheers... Rick

  • Reporting on buttons in the LMS, when there are multiple buttons on the page

    I have a project, it is 2 slides in multiple languages adding up to 38 slides. The first slide appears in English . The user can click the language of their choice or  click I agree (reported action to the LMS for completion). If they click I agree, no problem.  They score 10 points and the requirement for completion is to score 5. 
    If the user clicks their language, they have the option to click I agree (reported action to the LMS) or they can go Back to English (Jump to first slide).  When I add the  reporting on the I Agree in the language slides.  The Back to English button will only advance to next slide even though it isn't set up that way.  The functionality works fine if I'm not trying to set up reporting.  Any suggestions?

    Hello and welcome to the forum,
    Not sure if I understand well, but think you are bumping into the Quiz scope... I have been trying to explain this in a blog post, with other functionalities of Question and Score slide:
    Question Question Slides
    Lilybiri

  • Minor bug with editing when there are multiple pages of reports on the list

    When I edit a report and save the changes, I am returned to a different page on the list of reports than I started from.
    Steps to reproduce:
    1. Sort reports by Type
    2. On the first page on the list of reports, find  a custom report I created
    3. click Edit, then click Save
    4. I am returned to page 2 of the reports instead of page 1.
    5. If I click Cancel instead of Save, I'm returned to page 1 as expected.
    Depending on which page I started from I may be returned to some other seemingly random page, i.e. starting from page 4 I might be returned to page 3.
    Expected behavior - it should either return me to the first page, or else return me to the page I started from.
    This topic first appeared in the Spiceworks Community

    Make sure your logo is not overlapped with any window... and i guess, your smartform has two page formats...
    Identify the next page attribute of you first page and do the amendments as stated above.
    Regards,
    SaiRam

  • Help! How do i make a button state maintain when there are other buttons on the page?

    I need to have the down state of pressed buttons stay in that state when there are multiple buttons being pressed on the same html page...

    Hello and welcome to the forum,
    Not sure if I understand well, but think you are bumping into the Quiz scope... I have been trying to explain this in a blog post, with other functionalities of Question and Score slide:
    Question Question Slides
    Lilybiri

  • On my mini ipad, I cleared the cookies/histories in the safari setting.  But what does the advance-setting of the clearing the website data does?  What's that used for? Thank you!

    On my mini ipad, when I'm on the internet -- a few times I will see a blue small pop-up boxes telling me to allow down/load or add something like the bubble-splash game or something similar.  I jusr click ignore and it goes away.. I'm not sure why it's showing up lately.. I'm not sure if it's a virus or hacked in or a web brower matter..
    Should I go to the cookies/histories in the safari setting and have it cleared out?  Will that help resolve that?? 
    And what does the advance-setting of the clearing the website data does?  Sorry, I'm not familiar.. what's is that used for? Should I use that feature as well?? Thank you!

    This indicates corrupt files.
    A restore should resolve.

  • Exactly what does the Data Drop-down List do?

    What does the Data Drop-down List do? Is it used to make a drop-down list from certain records in a column in a table of a data base?

    Yes, this is exactly what it does. However, it's a custom control, not a built-in control. JavaScript is used to set it up. In order to use it you have to have a data connection in the document and modify the "Initialize" script, which is were the list is populated from the DB. The "Initialize" code is a good example of how to use XFA JavaScript to access a DB. However, the code is also written in a very general manor and takes the long way around to hooking up to the DB. Much shorter code can be written for a specific DB.
    Thom Parker
    WindJack Solutions
    www.windjack.com

  • What does it mean by Stream and how does it is different from Class?

    What does it mean by Stream and how does it is different from Class?

    www.dictionary.com does
    Stream: An abstraction referring to any flow of data from a source (or sender, producer) to a single sink (or
    receiver, consumer).
    Class:
    The prototype for an object in an
    object-oriented language; analogous to a derived type in a
    procedural language. A class may also be considered to be a
    set of objects which share a common structure and behaviour.
    The structure of a class is determined by the class
    variables which represent the state of an object of that
    class and the behaviour is given by a set of methods
    associated with the class.

  • What does the recovery CD contain - Satellite A100-847

    Hello guys
    I had a few questions about the recovery CD:
    1- What does the recovery CD contain? and the other CD which is called Media something?
    2- Does it have Windows in it? So I can format my PC using it? And from where can i find the product key (windows key)
    3- What if i wanted to use the CD without formatting? etc. installing the softwares?
    Sorry for asking too much but my laptop started disturbing me. Just for knowledge my Laptop is satellite A100-847

    Hello
    I will try to answer your question.
    1 Recovery media contains recovery image. Recovery image is a kind of package that contains operating system, all necessary drivers, Toshiba tools and utilities and some additional software like WinDVD or Norton antivir application. Media something is not known to me. Maybe exact description can help.
    2 Yes it has Windows OS. Yes you can format your notebook using it. You do not need product key. Just install it.
    3 I do not know what you mean. The fact is that HDD will be formatted before you start recovery installation. Be sure all your important data are saved on some external device.
    Bye

  • What does this TJ data mean?

    Hello,
    I am implementing a PDF parser in Objective-C  (using CoreGraphics API's) that will aid in PDF search. I know that the Tj and TJ operators are used for displaying text, and that to retrieve the text information from a page I should be looking there.
    The problem comes in when there are weird operands in the TJ operator. Below is one such example:
    [<1C>]TJ
    0.7039 0 Td
    [<1D>]TJ
    What does the above highlighted red text data mean, in terms of the semantic information that it contains? During parsing, the output of the above 2 operands are the characters ˝ and ˛ respectively. Am I going to have to use the font encoding provided in the Resources dictionary of the page to decipher what these characters semantically mean?
    Thanks a lot,
    Michael

    I assume you have a copy of ISO 32000-1:2008 available, so that you are reading what is written there first before asking.
    The answer is that NOT ONLY do you need to read the font encoding information for ANY TEXT EXTRACTION, but you ALSO need to read the information from any embedded fonts for the embedded encoding information.  This is especially true for TrueType fonts vs. Type 1 fonts, and CID-based fonts.
    And don't forget ToUnicode tables!
    There is a section in ISO 32000-1 that describes the entire process of text extraction.

  • HT2729 what does the "other" category consist of when syncing iPhone

    what does the "other" category consist of when syncing iPhone

    "Other" data is real data that you put on your phone by using the built in apps. It is not music, not videos, not pictures and not App Store app data. It is "OTHER" meaning everything that does not fall into these categories. Such as:
    email messages and attachments
    Reminders
    Calendar entries
    Contacts
    Text messages
    iMessages
    MMS messages and attachments
    Genius data
    Music cover art
    Music library database index (but not the music itself)
    Operating system settings
    Safari cache
    bookmarks
    Game Center status
    Music catalog
    You will find posts from people who said they recovered over 3 GB of space by just deleting old text messages.
    If you want Other to be smaller you will have to keep less data on your phone; delete old MMS and texts, delete old email (especially from the "deleted" folder), clear the cache, eliminate cover art.
    Sometimes "Other" includes data corruption. If you suspect this is a possibility try connecting to iTunes and click the Restore iPhone button. First restore from backup; the Other doesn't get any smaller you can Restore again and set up as New, then add your content back separately from your backup.

  • What does the usage of CURSOR word mean in an SQL statement?

    Hey folks,
    Please check out the following query and do please explain me what does the usage of CURSOR keyword in an SQL statement mean.
    select deptno,cursor(select ename from emp a where a.deptno=b.deptno) from dept b;
    well, the output was like this,
    DEPTNO CURSOR(SELECTENAMEFR
    10 CURSOR STATEMENT : 2
    CURSOR STATEMENT : 2
    ENAME
    CLARK
    KING
    20 CURSOR STATEMENT : 2
    CURSOR STATEMENT : 2
    ENAME
    SMITH
    JONES
    SCOTT
    ADAMS
    FORD
    30 CURSOR STATEMENT : 2
    CURSOR STATEMENT : 2
    ENAME
    ALLEN
    WARD
    MARTIN
    BLAKE
    TURNER
    JAMES
    6 rows selected.
    40 CURSOR STATEMENT : 2
    CURSOR STATEMENT : 2
    no rows selected
    Your favour'll be deeply appreciated.
    Cheers,
    PCZ

    This returns a non-square result set. Each row of the result is a deptno and then a result set of the enames in that deptno.
    This can be useful when you need to send a lot of data to a client application in a single query that would otherwise contain a lot of redundancy. It tends to be a relatively unusual construct (I've only found one situation where it was appropriate in my career) and requires some relatively sophisticated understanding on both the database and client sides.
    Justin

Maybe you are looking for

  • Looking for a gaming controller that will work with iMac

    I'm not really into gaming, but I just bought a game and right now I have to use the keyboard keys and I'm finding it to be a not so enjoyable gaming experience. So I would like to purchase a gaming controller.... does anyone have any recommendations

  • Create directory on OpenVms

    Hi all i create a directory on ovms server CREATE OR REPLACE DIRECTORY MY_FILES AS '$DISK2:[ADG.DATI_SPOOL_1]'; How i can veriify if exists or if the directory is right? I create a procedure that read a file from this directory and i get an error on

  • Java Pet Store Deployathon - wls 6.1 deplyment problem

    Hi All, I get following exception when i run build.bat on step 6 for building petstore.ear file. I have checked all the required resources for compiling the ejb classes are there in the required structure. EXCEPTION org.xml.sax.SAXException: Could no

  • Why is my iTouch continuing to skip through songs

    My iTouch will not play songs, when I press "shuffle" it continues to skip through all my songs but doesnt pause when I press "pause".  If I select a song to play it doesn't play it and just brings me back to the song selection.  I don't understand w

  • ITunes stutters

    Ok, I recently downloaded the new version of iTunes. I also updated my anti-virus/anti-spyware program from McAfee to Norton (I'm a Comcast customer). Ever since then, my songs will sometimes stutter whenever I click on a link or load a page with a l