Large Files from FTP to Azure Storage Account.

         We required to transfer files from FTP to Azure Storage Account.
Our FTP contains Almost 4 TB data and in future it will be growing.
We require some automate process which will transfer data from FTP to Windows Azure Storage.
Currently Implemented Solution:  We have created
a windows service which will download files from FTP and converting to file stream and with Blob Storage object it will be uploaded to
Storage account.
With the mentioned approach bandwidth is utilized at 2 places
For downloading file from FTP
For Uploading to Azure Storage 
Can you please provide any solution/suggestion which will reduce the consumption of bandwidth and large data will transfer in less time

Hi,
Please have a look at below article, it is talking about Blob Transfer Utility tool. Blob Transfer Utility is a GUI tool to upload and download thousands of small/large files to/from Windows Azure Blob Storage.
#https://blobtransferutility.codeplex.com/
Disclaimer: This is not a Microsoft official tool and it is not supported by Microsoft. It's just a (very useful) sample.
Best Regards,
Jambor
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

Similar Messages

  • How to Delete Directory from Azure Storage Account?

    Hi All,
    Currently i am working on Microsoft Windows Azure (SaaS - Application).
    I am facing a problem with the AzureStorage.
    I want to delete a whole directory from the Azure Storage Account.
    For Example : I want to delete following Directory(14).
    https://azurestorage.blob.core.windows.net/blobname/Images/14
    Any help will be appriciated. 
    Thanks in Advance.
    Rakesh T. Gupta, Web Engineer, Ahmedabad, India

    Hi Rakesh,
    We have not heard you in days. Have you managed to write the code to delete a blob directory?
    If not, I'd like to share my code for your reference:
        public void RemoveBlobDirectory()
            string containerName = "blobname";
            string directoryName = "blobname/Images/14/";
            var storageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
            var blobStorage = storageAccount.CreateCloudBlobClient();
            // Ensure the container is exist.
            var blobContainer = blobStorage.GetContainerReference(containerName);
            blobContainer.CreateIfNotExist();
            foreach (IListBlobItem item in blobStorage.ListBlobsWithPrefix(directoryName))
                if (item.GetType() == typeof(CloudBlob) || item.GetType().BaseType == typeof(CloudBlob))
                    ((CloudBlob)item).DeleteIfExists();
    Please note that the ListBlobsWithPrefix method will scan through all blob entities in order to search the result. So it will potentially have low performance when you have large number of blobs in you storage account.
    Thanks,
    Wengchao Zeng
    Please mark the replies as answers if they help or unmark if not.
    If you have any feedback about my replies, please contact
    [email protected]
    Microsoft One Code Framework

  • How do I move large files from a Windows computer to my MacBook Pro? There are a lot of photos (tiffs

    How do I move large files from a Windows computer to my MacBook Pro? There are a lot of photos (tiffs, jpgs, some photoshopped files), iTunes library, word docs. Is there a particular type of external hard drive that can be used or am I limited to copying stuff onto z-drive and moving that way?

    If both computers are on the same network, you can share the files from the Windows host using SMB and mount the share on the Mac. Otherwise, the Mac should be able to read (not necessarily write) any storage device formatted in Windows.

  • Windows Azure Storage Account Location change

    Hi Experts,
    I have deployed my Web site on Windows azure. I have created Windows Azure Storage Account. By mistakely i have set the location wrong. How can we change the location of Windows storage account? 
    Thanks!

    Hello CRM Inn,
    I am sorry to know that you have chosen an incorrect location and would like to change the location for your storage account. Unfortunately, there is no option available to change the storage location for an existing storage account.
    Your only option would be to create a new storage account, migrate data to the new storage account and delete the existing one. You can refer to this article that will give you information on migrating data from one account to another:
    http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-asynchronous-cross-account-copy-blob.aspx
    I also suggest that you create affinity groups. By creating affinity groups you can be sure that all your related services are under the same data center. You can refer to this article to learn about Affinity groups:
    http://msdn.microsoft.com/en-us/library/azure/jj156209.aspx
    Syed irfan Hussain

  • Is there any way to automate purging Application Logging in azure storage account tables

    is there any way to automate purging Application Logging in azure storage account tables
    Rohit Pasrija

    hi Rohit,
    If you want to delete old data automatically, I think you need develop this feature by yourself. You could code the logic methods on your project, and set a timer to execute the methods to delete old data. Please refer to this thread (http://stackoverflow.com/questions/13602629/is-it-possible-to-acces-a-table-storage-in-azure-from-an-azure-web-site
    ), you could operate table storage data using azure sdk or REST API.
    Please try it.
    Regards,
    Will
    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click
    HERE to participate the survey.

  • Error while downloading PDF file from FTP Server

    Hi Friends,
    I have sent a PDF file with data to FTP , Then i want to check that uploaded pdf file , whether that is correct or not?
    for that , i have downloaded that file from FTP and i am trying to open the file . but it is giving this problem .
    "There was an error opening the document . The file is damaged and could not be repaired."
    will you suggest me regarding this.
    thanks in advance.
    balaji.T.

    CALL FUNCTION 'CONVERT_OTFSPOOLJOB_2_PDF'
              EXPORTING
           SRC_SPOOLID                    = spoolno
                src_spoolid                    = wa_file-rqident
                no_dialog                      = ' '
          DST_DEVICE                     =
          PDF_DESTINATION                =
              IMPORTING
                pdf_bytecount                  = numbytes
                pdf_spoolid                    = pdfspoolid
          OTF_PAGECOUNT                  =
                btc_jobname                    = jobname
                btc_jobcount                   = jobcount
              TABLES
                pdf                            = pdf
              EXCEPTIONS
                err_no_otf_spooljob            = 1
                err_no_spooljob                = 2
                err_no_permission              = 3
                err_conv_not_possible          = 4
                err_bad_dstdevice              = 5
                user_cancelled                 = 6
                err_spoolerror                 = 7
                err_temseerror                 = 8
                err_btcjob_open_failed         = 9
                err_btcjob_submit_failed       = 10
                err_btcjob_close_failed        = 11.
    because of this one PDF internal table is obtained.
           OPEN DATASET L_FILENAME  FOR OUTPUT in text mode  MESSAGE MSG.
      LOOP AT pdf.
        CONCATENATE pdf-tdformat       "Material group
                     pdf-TDLINE       "Basic Material
               INTO ITEXT-TLINE ..
        APPEND ITEXT.
        TRANSFER ITEXT TO L_FILENAME.
      ENDLOOP.

  • Problem while reading the file from FTP server

    Hi Friends,
    I have a problem while fetching files from FTP server.
    I used FTP_Connect, FTP_COMMAND function modules. I can able to put the files into FTP server.
    but I cant able to pick the files from FTP server.
    anyone have faced similar issues kindly let me know.
    Thanks
    Gowrishankar

    Hi,
    try this way..
    for reading the file using FTP you need to use different unix command ..
    Prabhuda

  • How to read .xls file from FTP server t oInternal table

    Hi
    am using the FTP_SERVER_TO_R3 to read xls file from FTP server to internal table
    but the data i get in LT_TEXT is special characters.
    CALL FUNCTION 'FTP_SERVER_TO_R3'
    EXPORTING
    handle = hdl
    fname = f_name "ProdDataFromCRM.xls.
    * CHARACTER_MODE = 'X'
    * IMPORTING
    * BLOB_LENGTH =
    TABLES
    BLOB = lt_text
    * TEXT = lt_text
    EXCEPTIONS
    TCPIP_ERROR = 1
    COMMAND_ERROR = 2
    DATA_ERROR = 3
    OTHERS = 4
    can any one help me out to get the exact data..
    Really appreciate your quick response..
    Thank You

    Hi, if you really retrieve an excel file, you can not see the data in ABAP. You may see them in Excel. For this you may use
    CALL METHOD document->open_document_from_table
    of the interface i_oi_document_proxy for OLE objects. You can access the data with reference to the interface i_oi_spreadsheet.
    Please check [Desktop Office Integration (BC-CI)|http://help.sap.com/printdocu/core/Print46c/en/data/pdf/BCCIOFFI/BCCIOFFI.pdf] for details.
    But, who knows, perhaps you want "And Now For Something Completely Different".
    Regards
    Clemens

  • Need to copy .txt file from FTP server and downloaded on local server directory.

    I need to figure out a way to copy .txt file from ftp server in local server directory using sql jobs.

    Below links will help achieving it:
    https://www.virtualobjectives.com.au/sqlserver/ftp_scripts.htm
    http://www.mssqltips.com/sqlservertip/2884/sql-server-integration-services-ssis-ftp-task-for-data-exchange/

  • How to read XL file from FTP server

    Hi all,
    I have a requirement like to read file from FTP server using path
    ftp:
    10.212......\DTR\DTR_ Accounted_Out
    and again save  other file in same location ,
    to doing this RFC connection is required?
    give a  procedure or program to do this requirement ..
    To create rfc for FTP which connection type can i use ?
    Give complete settings to create rfc destination

    Hi Rakhi,
    Use the below code to check if you have received the proper data.
    call function 'FTP_SERVER_TO_R3' "Get data as character instead of BLOB
        exporting
          handle         = hdl
          fname          = docid
          character_mode = 'X'
        tables
          text           = chardata.
    If the data is incorrect, it is possible that you are in the wrong directory.
    Use
    call function 'FTP_COMMAND'
        exporting
          handle        = hdl
          command       = 'cd mydir\mysubdir' "cd <space> your path
        tables
          data          = result
        exceptions
          tcpip_error   = 1
          command_error = 2
          data_error    = 3.
    Regards,
    Jovito.

  • Problem in download file from FTP server

    Hi
    I want to download a file from FTP server for that i am using the apache FTP module but i am not getting how to download a file with that api i can dispaly the list of files and folders but not able to download the file or folder can any one help me in this
    Thanks
    Ninad

    Hi
    I think you miss understood something I am writing a program to download the file in Java where i have used the jakarta.apache api for that and getting problem in that bellow is the code where I have written to print the directory & file names but I don't know how to download the file
    FTPClient ftpConnection = new FTPClient();
    ftpConnection.connect(host);
    ftpConnection.login(FTPConnection.userName,FTPConnection.password);
    FTPFile fileList[] = ftpConnection.listFiles();
    for(int i=0;i<fileList.length;i++)
         System.out.println(fileList.getName());
    thanks
    Ninad

  • How to get file from FTP Server using File Control

    Hi,
    Any one did getting file from FTP Server?
    Please let me know any one help me.
    I would need to get file from FTP Server.
    Thanks,
    Madhu

    Yes I have done that. But In FTP Server I cannt read file, because no previliges. Only I need to copy file from FTP Server to local server then only I can read that file.
    I tried all options using FileConrol(getFiles(),read()).
    getFiles() - It wont copy the file, it give information about file.
    read() - I dont have previliges to read the file.
    Please tell me any other procedure would be there for getting file from FTPServer.
    Thanks,
    Madhu

  • Downloading excel file from FTP Server to Application Server

    Hi,
    I have to get data from an excel file available on FTP server into an Internal table.Can I use FTP_SERVER_TO_R3 to do so.
    Please let me know if there are any function modules available to do this.
    Thanks,
    Prasuna.

    Dear Gayatri,
    You can get the file from FTP to internal table...
    I am sending you the code with inline comments ....Hope this will be helpful to you.
    Data: lv_key           TYPE i VALUE 26101957.
    Data: lv_password(30)  TYPE c.
      i_rfc_destination = 'SAPFTP'.
      lv_length = STRLEN( i_password ).
    CALL FUNCTION 'HTTP_SCRAMBLE'
        EXPORTING
          SOURCE      = i_password "give ftp server pwd
          sourcelen   = lv_length
          key         = lv_key
        IMPORTING
          destination = lv_password.
      CALL FUNCTION 'FTP_CONNECT'
        EXPORTING
          user            = i_user "give ftp user name
          password        = lv_password
          host            = i_host
          rfc_destination = i_rfc_destination
        IMPORTING
          handle          = lv_ftp_handle
        EXCEPTIONS
          not_connected   = 1
          OTHERS          = 2.
        CONCATENATE 'cd' i_folder_path INTO lv_cmd SEPARATED BY space.
    *i_folder path is the path in ftp server where file is  stored
        CALL FUNCTION 'FTP_COMMAND'
          EXPORTING
            handle        = lv_ftp_handle
            command       = lv_cmd
          TABLES
            data          = result
          EXCEPTIONS
            command_error = 1
            tcpip_error   = 2.
         lv_blob_length = 392.
         TRANSLATE i_filename TO LOWER CASE.
          CALL FUNCTION 'FTP_SERVER_TO_R3'
            EXPORTING
              handle      = lv_ftp_handle
              fname       = i_filename          "give required file name
            IMPORTING
              blob_length = lv_blob_length
            TABLES
              blob        = lt_dummy.
    Regards
    Sajid

  • [HELP] Download file from FTP server

    hi,
    I want to write a java program that can download and upload files from ftp server. Currently I only manage to upload a file to ftp server but i cant download file from ftp server. Here is the source code that only allow user to upload file. Anyone can give me some guidelines so that my program can download and also upload file? thx.
    import java.io.*;
    import java.net.*;
    public class FTPUpload {
    private static final int CTRLPORT = 21;
    private static Socket ctrlSocket;
    private static PrintWriter ctrlOutput;
    private static BufferedReader ctrlInput;
    private static byte[] localHostAddress;
    public final static String DIR = "C:\\zip\\";
    public static void main(String[] args) {
    try {
    String host = "192.168.1.1";
    String loginName = "testuser";
    String password = "password";
    String dirName = "/home/testuser";
    String fileName = "hello.zip";
    ctrlSocket = new Socket(host, CTRLPORT);
    localHostAddress = ctrlSocket.getLocalAddress().getAddress();
    ctrlOutput = new PrintWriter(ctrlSocket.getOutputStream());
    ctrlInput = new BufferedReader(new InputStreamReader(ctrlSocket.getInputStream()));
    ctrlOutput.println("USER " + loginName);
    ctrlOutput.flush();
    ctrlOutput.println("PASS " + password);
    ctrlOutput.flush();
    ctrlOutput.println("CWD " + dirName);
    ctrlOutput.flush();
    ctrlOutput.println("TYPE I");
    ctrlOutput.flush();
    FileInputStream fis = new FileInputStream(DIR + fileName);
    Socket dataSocket = dataConnection("STOR " + fileName);
    OutputStream outstr = dataSocket.getOutputStream();
    int n;
    byte[] buff = new byte[1024];
    while ((n = fis.read(buff)) > 0) {
    outstr.write(buff,0,n);
    dataSocket.close();
    fis.close();
    ctrlOutput.close();
    ctrlInput.close();
    ctrlSocket.close();
    }catch (Exception e) {
    e.printStackTrace();
    private static Socket dataConnection(String ctrlcmd)
    throws IOException,UnknownHostException {
    String cmd = "PORT ";
    ServerSocket serverDataSocket = new ServerSocket(0,1);
    for (int i=0;i<4;i++) {
    cmd = cmd + (localHostAddress[i] & 0xff) + ",";
    cmd = cmd + (((serverDataSocket.getLocalPort())/256) & 0xff)
    + ","
    + (serverDataSocket.getLocalPort() & 0xff);
    ctrlOutput.println(cmd);
    ctrlOutput.flush();
    ctrlOutput.println(ctrlcmd);
    ctrlOutput.flush();
    Socket dataSocket = serverDataSocket.accept();
    serverDataSocket.close();
    return dataSocket;
    }

    Or just use a java.net.URL("ftp://...) ..., get its input stream, and away you go ...

  • Upload XL file from FTP server

    Hi All,
    Can anybady help me, how to upload Excel file from FTP server.
    Thanks
    Sri
    Edited by: srikanthn on Apr 14, 2010 6:31 PM

    Hello
    How about using SAPFTP?
    I hope SAP note 130106 will guide you on this.
    Thanks
    koju

Maybe you are looking for