Architectural design for FTP batch processing

Hello gurus,
I would like your help in determining the design for the following.
We receive several HL7 messages as a text file and copied to a shared network folder. All these files are created into several different folders depending on the region, message type. We need to come up with a B2B process to read all the files from the netwrok folder using FTP (batch process) and translate if needed (depending on the scenario) and transfer the files over to other destination folder on the network (using FTP).
For this, we can create TPs with Generic FTP channel and this works without any issues. By doing this way, we need to create TP for each and every type of message which reads the files from its own specified directory location on the network based on the the polling interval.
My question is, instead of creating TPs for each and every type of file, is there a way by which I can write a common web service that reads the source files from the network and based on the type of the file route to the proper destination folders. If it is possible, I would like to know the architecture for accomplishing this task.
I really appreciate your kind help on this.
Thanks and regards,
Raghu

Hi Raghu,
Is it a B2B communication scenario?
By doing this way, we need to create TP for each and every type of message which reads the files from its own specified directory location on the network based on the the polling interval.Why cann't you have only one TP with multiple documents, channels and agreements?
My question is, instead of creating TPs for each and every type of file, is there a way by which I can write a common web service that reads the source files from the network and based on the type of the file route to the proper destination folders. If it is possible, I would like to know the architecture for accomplishing this task.Depends on your use case and products you want to use. You can very well use FTP adapter with BPEL and poll for files. Use DVM in composite to figure out the destination and send it there. You may use OSB if it is a typical routing case with heavy load and performance is a concern. You may use B2B as well here. So ultimately you need to figure out what you want and what tools you want to use.
Regards,
Anuj

Similar Messages

Need best Architecture design for the following scenario

Hi Experts,
I need best architecture design for the following scenario:
Sender: Mail
Receiver: if sender body contain u201DApproveu201D, then call SOAP adapter, If SOAP adapter execute successfully, then send Mail(SOAP adapter executed successfully) and SMS . So receiver is SOAP, Mail and SMS.
My current approach issue:
Three message mapping:
Mapping1: mail to SOAP
Mapping2: mail to Mail
Mapping3: mail to SMS
In interface determinant>u201DSelect all three operation mappingu201D> u201Cselect order at runtimeu201D.
Issue in current approach: For me first SOAP adapter should complete successfully, after that only my Mail and SMS operation mapping should execute:
But problem is before success of SOAP adapter engine, Mail and SMS mapping completed or integration engine executed.
Note: There is possible to fail the SOAP request in adapter engine.
Kindly help me --> u201CAm I going correct way or want to change the architectureu201D?
Thanks for advance!!
Regards, Kumar

What do you mean by successful execution of soap call? Are you talking about Successful response (happy flow) from the soap call instead of application error ? Then based on the response you want to decide sending mail and sms. How big call is your soap ? If your soap interface is very simple I can give the other possible way.
Sender sends the message, use mapping to read the content and then do soap call in the mapping itself and after seeing the soap response decide about two receivers (mail and SMS). If your soap call is very simple one, you can go for it. Otherwise I would not recommend this. The reason is you loose some visibility regards to monitoring during soap call in the mapping.
Other option is go for CCBPM. Here you receive the message, use send step to soap interface which is sync step. Then after seeing the response create another block with fork steps each for two senders mail and sms. If response bad then dont proceed or execute the next block. Simply use exeception or control step to jump out from the block,

Architecture Design for CORBA

hi,
i would like to know if it's possible to integrate CORBA afterwards in an application or do i have to make preparations in the current architecture if i decide that i want to use CORBA in my application?
thanks in advance.

wgctrey wrote:
thanks for you reply. i know that it depends on the design of the application and that i need a "good" design. but what is a good design to be prepared for CORBA?I wouldn't design for Corba. I would created a layered design where it doesn't matter if it's Corba at the lower layers.
Kaj

Global Architectural design for SharePoint 2013??

Hi Friends,
i am trying to pursue SharePoint 2013 global Architectural design before we implement SharePoint 2013 farm. FYI, Our Current SharePoint 2010 farm is three tier farm (2 web servers + 2 App servers + cluster Sql server) and total users 1,000 and its
located in one data center. From this farm we are serving now SP site to five different countries in the world via WAN. But as our company growing rapidly, we want share point will provide robust performance. As our company globalized, how many data center
we will need? Can we have one centralized farm in USA and other regional SharePoint farms into different countries? According to my business scenario, what will be the best architectural design?
Any help will be appreciated!!

"It depends" would be the answer with the information given. It depends on how much content is accessed at any one location, if a WAN accelerator could alleviate any issues end users at the remote locations were seeing, and if the company can take on the
added complexity, extra licensing, and hardware cost that are associated with SharePoint farms.
It would also depend if you want to replicate content (e.g. Metalogix Replicator) or if each farm will have its own unique content targeted at the specific region.
Trevor Seward
Follow or contact me at...
&nbsp&nbsp
This post is my own opinion and does not necessarily reflect the opinion or view of Microsoft, its employees, or other MVPs.

Idoc to XI to FTP BATCH process

Hi,
I have a scenario to send idoc from SAP R/3 to FTP server using XI.
In this scenario, the interface should be executed in batch once in every hour.
How can I schedule the interface in BATCH running for every hour?
Please explain in detail if possible. Thanks a lot
Thanks
Ricky

Ricky,
Is it ok if you can schedule your file adapter for every 1 hour. If yes then see this weblog:
/people/shabarish.vijayakumar/blog/2006/11/26/adapter-scheduling--hail-sp-19-
But what I see is one disadvantage. If you send Idocs from R3 it will reach to XI and will stay in XI till the adapter process it.
One other option is collect the Idocs in R3 and send to xi.
---Satish

Override action "save as" command with save for web batch processing

Hi everyone,
I've created an action that uses the save for web dialogue to optimise images for the web. When I use the batch command to process a full folder of images, even though I have "Override action "save as" command" checked, it ignores this and still uses the location that was used when the action was created.
Apparently this is a known issue in CS6 and previous versions but I wondered if anyone knows whether this has been fixed in Photoshop CC?
Appreciate any advice.
Thanks

Since save for web is an export plugin, none of the destination options have any effect on where the files are saved
or the file names.
Save for web only saves the files to the folder specified when you recorded the action.
(it's always been that way and still is in photoshop cc)
You might look at the Image Processor Pro
(included in Dr. Brown’s Services 2.3.1)
(has a save for web option)
http://www.russellbrown.com/scripts.html

Considerations when designing for spot vs. process color?

I'm helping a friend start up a clothing business. It's just him, and I'm providing some computing knowledge.
He is the designer and has been building his first set of graphics using AI.
He's selected the first two "printers". He ran into some price differences with a vendor that used spot colors vs. one that was process. I could give him a basic understanding of spot vs. process, but ran out of understanding when it came to questions like "Why the price difference?".
Would appreciate some explanation for things like "What kind of printers would shops with spot be using vs. those that do process?", "As the designer, are there reasons he would choose one method over the other?".
He will be producing both embroidered and printed media on different fabrics (cap, 2-3 different quality cloth). There will be different colors (for a particular design), and the size and location of the print/embroidery will vary (on the garment). All in all, there will be less than 12 variations each for color, size, and 12 location.
Along with the spot/process question, he's also being asked for different formats than he'd expect. Given my understanding that AI produces vector, if a vendor (the embroidery shop) asks for JPEGs, are they assuming he's done the scaling?
Will continue to do online searches and will post back if I figure this out.
Tom

The print shops should be able to explain the price difference and which one is more durable (depends on the printing process). Probably the one using spot colors does screen printing.
Regarding the embroidery: you should definitely clearly communicate which size the embroidery should be. It can be scaled within some limit, but you shouldn't rely on the service people interpreting the graphic's size and resolution.

E-Recruitment Technical architecture design for stand alone

Hello,
I looking at various options to implement E-Recruiting in a stand alone scenario.
One option i know is where my HR Data is in ECC6.0 and E-Rec component is in other ECC6.0 system and i can interface the data through ALE and RFC's.
When we talk about E-Rec in standalone (without ECC), how would be technical architecture look like.
Please help me with some inputs.
Best Regards
G Raj

Hi Raj,
Am not sure what are the exact details that you require. I have given a few pointers which may be of use.
Incase eRecruiting is on a different server then the transfer of data (PA & OM ) will be through ALE & idocs.
The transfer of data from eRecruiting to ECC is through RFC where only a set of 15 fields can be transferred back. There is also a newer option to use XI for this.
The standalone server usually has only the erec component & no other HR module (Pls chk the installation guide on this point)
A few BADIs need to be activated in the erec server
When an internal employee(object P) is transferred here, a corresponding object NA(candidate) & BP(business partner) is created.(This is because of the BADI activation)
Incase of an external employee , the moment he registers, an object NA ,BP ,CP is created.When he is hired into the system, there is an object P created with a New CP which then will have to be coonnected to the original NA.
Change pointers will take care of synchronisation of data between both the servers.
The relationship between the various objects are seen in infotype 1001 in the erec system.
The advantage of using this scenario is that core HR is not exposed to external people as the erec server talks with the external world.
Incase of webdynpro ABAP as UI, you can have separate fron-end & back-end.
Regards,
Sowmya

Architecture/design for high volume web service calls from on Demand

Hi,
We have hundreds of end users (less than 600 users) that will be initiating web service calls from On Demand that will deal with both querying (stateful transaction) and inserting/updating data (stateless). Currently, we see a limitation on the number of stateless sessions (33) that can initiate web service calls. We are afraid this will lead to poor performance with the number of sessions that are allocated for stateless sessions.
Can someone provide any design or framework information that can be implemented for high volume web service calls from On Demand? We anticipate 35K+ web service calls that will be initiated from On Demand per hour.
Thanks.

Hi,
We have hundreds of end users (less than 600 users) that will be initiating web service calls from On Demand that will deal with both querying (stateful transaction) and inserting/updating data (stateless). Currently, we see a limitation on the number of stateless sessions (33) that can initiate web service calls. We are afraid this will lead to poor performance with the number of sessions that are allocated for stateless sessions.
Can someone provide any design or framework information that can be implemented for high volume web service calls from On Demand? We anticipate 35K+ web service calls that will be initiated from On Demand per hour.
Thanks.

Architectural design for a forum

using servlets, we need to design a web forum. Question: How would you design the interface for the webmaster. How do you archive the website content, modify content etc.

this is another test

Is it possible to create a hot folder for batch processing?

I know that it's possible to make a droplet for preflight, but is it possible to create a droplet or a hot folder for another batch processes (mine crops the pages)? Or at least make a keyboard shortcut? It's so unproductive to go Advanced>Document Processing>Batch Processing>Run>Close (Adobe spoiled us with custom shortcuts in other programs)...

HenrikAppleJespersen wrote:
is it possible to create a hot spot with apple Airport express 2 generation and use it i a hotel with ethernet wall socket and use it with my Ipad 2?
Yes, it is one of the things AirPort Express was specifically designed for.

Batch process is started for DTP's-After Support package upgrate...

Dear All,
we upgraded with SP19 in our BI systems,
After patchs are completed when we run the dtp's either through process chain or normal triggering also dtp is shows still yellow for long time for all the cubes dtps...dso dtp....any dtps....
Message as follwos...
Extraction DataSource XXXXXXXX: 303 Data Records
Prepare for Extraction
Beginning extraction request REQU_D433FLO6MSZEOFHIR1SUOMBSP...
Extraction Completed
Waiting for free batch process...
Batch process started
then it is alwys shows yellow only even there is single records in Datasource...any help please...

Thanks for the quick reply!
We currently only have one of the notes implemented (1277550) and that one does not fix the problem, but it makes the yellow processes turn red (which is what it claims to do in the note description as well).
However, we have days where everything runs fine and we have days where we run into this issue. Therefore it is hard to test and before implementing this one additional note, I was curious whether there was anything else that you guys did to fix the problem. I am glad to hear that those two notes together should take care of the problem. We are implementing it right now.
Thank you!
Dennis

Batch process for setting document (language) and page (structure) properties

I'm looking for a batch process or script that will take a folder of PDFs and add some basic properties required by Section 508 (for accessibility) automagically--such as Document Properties where language=EnglishUS, and Page Properties Tab Order=Use Document Structure.
I'm not a programmer so any help is very much appreciated.
Thank you!
Mary

well on unix/Linux/solaris there are no drives but storage heirarchy starts with a root directory represented with ' / '. and all files and directories are created in root directoy like
/usr/local/bin
/export/home/san
etc
also filesystem of unix consists iNodes rathar than FAT entries as in case of Windows.
some basic information is available at
http://www.techonthenet.com/unix/index.php
Regards

Architecture advice for Effect and AEGP communication

Happy monday!
In my current plugin project I need to use an "helper" AEGP for tasks my main effect plugin cannot accomplish (delayed tasks via AEGP idle hook, project modifications that will crash if performed inside the Effect logic, etc.)
I'm trying to figure out an elegant and robust architecture design for this cross-communication.
Heres the way I see it:
Effect passes info along to AEGP via custom suite data.
AEGP reads the data at idle, and performs some required task.
The success/fail of this work and the resulting products are communicated back to the Effect via AEGP_EffectCallGeneric.
My question is basically surrounding the data/info handoff. What kind of objects should I be passing back and forth? Is it considered poor design to simply pass along a pointer to the sequence or custom arb data to the AEGP via the custom suite? And vice versa when the GP sends EffectCallGeneric to the effect and passes a pointer, it could just be the same pointer that the effect originally sent the AEGP using the custom suite.
Aside from undo/redo, is there an advantage to using arb parameter data over sequence data for this task?
It seems to me that AEGP plugins are loaded once and only once in the session, so if multiple instances of my effect plugin are used in the project, and they are all calling the AEGP there is the potential for a race condition if the custom suite data is used to pass along information. Is this an accurate assessment of AEGPs? A simple example of this would be that the custom data suite has a variable that stores which effect plugin is calling the AEGP. If another instance of the effect starts talking to the AEGP, it would overwrite the suite var telling the AEGP which effect instance is calling it, thereby making the AEGP perform the work of one effect instance but sending the results to different instance.
Even if the AEGP is extremely simple and just performs small atomic operations each time the custom suite data tells it to, how do I prevent n+1 Effect instances from colliding when using the AEGP worker? Am I missing some key part of the Effect->AEGP communication that prevents this race condition situation?
This is the first time I've tried my hand at writing inter-process communication where I have the option to exchange actual in-memory objects. I'm hoping someone with more experience with this sort of problem can give some pointers(hah) or at least a few cautionary words about designs to avoid.
Thanks, sorry for such an open ended question, but this is the place to talk to the pros
-Andy

yo andy! what's up?
i see you were up to no good... naughty.
you are correct in your assumptions.
AEGPs are indeed loaded only once per session, but if you're rendering with
the "multiple frame simultaneously" option, then the AEGP is loaded
separately for each AE instance.
(so would your plug-in. each AE instance calls it's own global setup)
now, plug-ins are called on the main thread. which should mean that only
one plug-in is is called at a time, which should mean that your AEGP may
only get one call at a time. i'm not absolutely sure that the base
assumption is correct here. it's possible that multiple effects are called
at the same time, so re-entrancy of your AEGP suite is possible.
how do we know? we can either test, or ask zac lam.
re-entrancy is a non issue when reading data (all can read at once. it
doesn't matter). but when writing data... that's a problem.
to prevent such issues you need to implement some mutex (mutually
exclusive) mechanism.
it only allows one caller access at a time, while the other caller stall
until their turn.
boost library offers such tools.
as for what data you should transfer.
you can either pass data by value, or by reference.
when a function gets it's data like so:
function(int val);
it get's a local copy of the data. no worries about scope here.
if it gets the data like so:
function(int *val);
or function(int &val);
then we're talking reference. in this case, the val variable is only valid
while the object to which it refers is valid.
that depends on stability of that memory chunk. what makes a chunk stable?
1. if the data is created as a local scope variable it will invalidate when
the code block finishes. make sure don't access that data at other times.
2. AE handles of a plug-in instance are only valid during calls to that
plug-in.
let's look at the two directions of communications you have.
1. effect calls AEGP.
in this case, the call order is as follows:
AE calls effect.
effect is now executing. at this point, sequence/global data are valid, and
anything locally allocated in the call handling function.
effect calls AEGP.
AEGP is now executing.
you can now pass anything you like from the effect to the AEGP.
if the AEGP returns data, it should be data that remains valid after the
AEGP finishes executing.
AEGP finishes it's thing and exits.
effect is now executing.
effect finishes the call, exits, and AE is executing.
at this point, seq/global data handles are no longer valid. (at least you
can't rely on that)
also, anything allocated in the effect's calling function as a local
variable, is no longer valid.
so now your AEGP gets called on idle time.
it should only rely on memory that has kept since the effect was executing.
what memory is kept?
anything in global scope of the AEGP, or pointers to memory that was not
deallocated or moved.
seq data handle may not have been deallocated, but you can't tell if it
moved or not. so you can't rely on it.
if the seq data has another pointer in it, which point to a non movable
piece of data, then you can rely on that pointer.
seq/global handles are locked an unlocked by AE. memory handles are managed
by the plug-in (so you know if they can be relied on or not).
2. AEGP calls effect.
AE is calling the AEGP on idle time.
AEGP is executing.
AEGP calls effect with data.
effect is now executing.
effect uses data gotten from AEGP.
effect optionally returns data to AEGP.
effect exits.
AEGP is now back executing.
AEGP uses data returned from effect. that data can't be references to local
variables in the effect as they are no longer valid.
AEGP exits, AE is now executing.
so in both directions you need to see that at the time the data is used,
that is is valid.
as for sequence data vs arb param.
yes, there is a difference besides undo/redo.
AE creates (multiple) copies of the data stored in arb params before each
call to the plug-in and also after.
if you're storing very large amounts of data in the arb param, then the
copy operations will start dragging your performance down.
it's usually a non-issue, but... it's a point to think about.

Why do I need to use GIMP to batch process?

Dear Adobe,
I'm grinding through thousands of display and template images for an online retailer. Photoshop layering and masks came in quite handy when I was making the templates
but now that it's got down to reusing those templates, I have to look elsewhere.
1 Why does every other piece of software on the market have a batch processing section, and yet you hide yours?
2. Why does Image Processor refuse to resize both horizontal and vertical, although it claims to? It actually refuses to distort the image to resize, which I actually want it to do. I had
to download GIMP to get this capability, as well as to specify the output file and change the dpi. These are all included in GIMP but not Pshop. Thanks for that. And don't tell
me to go to Actions. THat may be useful for something things, but it's hardly the optimal way to do it, recording your exact actions, and then saving or not saving, and then
seeing what happens. The way I wanted it was in GIMP, and its pathetic that 20 years in, Pshop still doesnt have a credible batch processor, or tell me where it is now.
Charge extra for that perhaps?
3.What has changed in Bridge CS6 since CS4? I see absolutely nothing different, but then output is still limited to PDF and 'WEB GALLERY' a dysfunctional glob
of code bested easily by jquery, but maybe you've impoved that end. My last experience with web gallery was so dysfunctional that I'll just stick to fixing what's
under the hood myself. I mean web gallery used to be flash, then something else that's not current. Let's just agree its a hobbyist option. Of course,
I don't know why 'output' might include other options for the batch processing of files, since batch rename does actually work, but you guys are the deciders.
4. Its too bad. This job has taught me a lot of new stuff about layering, masks, and the functionality of the pattern section, which I have no great qualms with, though,
considering how many companies you've bought, you could include tons of patterning capabilities that used to exist in these programs, but that would be too much
expenditure from MGMT. Maybe let's just fix the window views we broke in CS4, and call it a new version.
5. Thanks for forcing me to discover GIMP, who dont ask a thing. I dont really like being in there yet, but we'll see if that changes with time. They actually have some
different filters and rendering options that Adobe could've included years ago for gun, but again, there's just not enough money to go around at Adobe.

Antinet wrote:
Thanks for both of you proving both that the function doesnt exist.
When the Image Processor resizes image it used the Photoshop Plug-in Fit Image to do the resize. Both height and width are resized and the image aspect ratio is preserved so the image is not distorted. That means the image will fit within the height and width used in the dialog. Only image with the same aspect ratio will fill the area. It sounds to me like you want the image the size id the height and width you entered in the dialog. In which case you need to use Photoshop features the can accomplish that. If you do not care the the image is distorted you use Image size check resample uncheck constrain as set the height and width you want to resize to. If you do not want to much distortion you can try content aware resize. That will still distort the image some what but try to maintain some areas perspective it finds object in. You can also help that process by selecting areas. If You want no distortion you need to resize the image to fill the area and mask off the excess to virtually crop the image the the areas aspect ratio. To automate that you need to use some Photoshop scripting. Knowledge is required to get the most out of Programs like Gimp and Photoshop. The function doesn't exist because of your lack of knowledge.

Architectural design for FTP batch processing

Similar Messages

Maybe you are looking for