Duplicate File issue using FTP adapter - BizTalk 2010

Question

Hi We encountered an Issue of picking Duplicate files in BizTalk 2010 from the FTP Location. Need your assistance on this.
Mainframe sends multiple files a 0 KB file to FTP and BizTalk picks during a particular service window using FTP Adapter. 
Receive Location has a pipeline component which decodes the MF file(EBCDICRow format). 
Send Port transmits the file which is decoded in the Receive location. There is no mapping or orchestration involved. 
When 2 files are placed in FTP Location. BizTalk Transmits the files successfully
No suspended messages in BizTalk but we see below error in event log. we tried reproduce the issue but no luck. 
Host Instance running FTP Location is Clustered  
 There was a failure executing the receive pipeline: "XXXX.XX.Pipelines.Receive_XXX_TransactionsMC_passthru, XXX.XX.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=c2d1f476d5c2f97d"
Source: "EbcdicRowDeCode" Receive Port: "XXX.XX.MCSTransactionsMC" URI: "ftp://XXXXXXXX:21/'XXXX'/MCSDT.DEFKOP.R001.D*.T*" Reason: Unable to cast object of type 'Microsoft.BizTalk.Streaming.BasicStreamWrapper' to type 'XXX.BizTalk.Pipeline.Components.Streams.V3.VirtualStream
MF Puts below Files at FTP Location
PSNOX.MCSDT.DEFKOP.R001.D150406.T009000
PSNOX.MCSDT.DEFKOP.R001.D150406.T002100
BizTalk Picks ( at specified Service window) and Transmits as 
PSNOX.MCSDT.DEFKOP.R001.D150406.T009000
PSNOX.MCSDT.DEFKOP.R001.D150406.T009000
PSNOX.MCSDT.DEFKOP.R001.D150406.T002100
Regards
-Sri

Answer 1

Hi Sri,
There could be two reason for such a behavior:
1) Using Non-Clustered Hosts: It is always recommended to use the clustered host for FTP adapter. Because FTP don't allow
any locking mechanism on the files so in case of non-clustered with multiple host instances you might receive same file multiple times through different host instances.
2) If the original document is still being written to the FTP server by the host application, the FTP adapter cannot
delete the document and will retrieve another copy of the document at the next polling interval that is configured for the receive location. This behavior causes document duplication to occur. 
Workaround could be:
Configure the host application to write to a temporary folder on the same hard disk as the public FTP folder and to periodically move the contents of the temporary folder
to the FTP folder. The temporary folder should be on the same hard disk as the public FTP folder to make sure that the move operation is atomic. An atomic operation is an operation that is functionally indivisible. If you write data to the public FTP folder
by using the BizTalk Server FTP adapter, you can do this by specifying a Temporary Folder property in the FTP Transport Properties dialog box when you configure a send port. If you specify a Temporary Folder property, make sure that this folder is on the same
physical disk as the public FTP folder.
Configure the FTP receive location to operate within a service window when the host application is not writing data to the FTP server. You can specify the service window
when you configure the receive location properties.
Refer: Known Issues with the FTP
Adapter
Rachit
Please mark as answer or vote as helpful if my reply does