To extract duplicate files from a system

sir, i m working on project to find duplicate files from a system . can any one give me some idea how should I start ?

I have written one in Java and in Python.
My approach was to take the SHA1 hash of each file and use this as the key in a map with the file name being the value.
I traverse the file system building the hash mapping as I go along. When I find a SHA1 hash that is already in the map I add the pair of associated file names to a list of possible duplicate files.
After I have traversed the set of files I check all the potential duplicates for equality.

Similar Messages

  • How to extract  DB  FILE  FROM NONSAP  SYSTEM  IN BI-7

    how to extract  DB  FILE  FROM NONSAP  SYSTEM  IN BI-7

    hi,
    chk the links for extraction using DB
    Extraction using DB connect
    http://help.sap.com/saphelp_nw70/helpdata/EN/58/54f9c1562d104c9465dabd816f3f24/frameset.htm
    http://help.sap.com/saphelp_nw04/helpdata/en/c6/0ffb40af87ee6fe10000000a1550b0/frameset.htm
    Extract data from oracle DB to SAP BI 7.0
    Ramesh

  • Batch file extracting all files from nested archives

    I have managed to leverage a powerful
    forfiles command line utility with the mighty
    7z compression program.
    Below is a simple batch file extracting all files from nested archives hidden at any depth inside other archives and/or folders. After the extraction each archive file turns into a folder having the archive file name. If, for example, there was an "outer.rar"
    archive file containing nothing but an "inner.zip" archive with only "afile.txt" inside, "outer.rar" becomes "...\outer.rar\inner.zip\afile.txt" file system path.
    @echo off
    rem extract_nested_archives.bat
    move %1 "%TMP%"\%2
    md %2
    7z x -o%1 -y %TMP%\%2
    del "%TMP%"\%2
    for %%a in (zip rar jar z bz2 gz gzip tgz tar lha iso wim cab rpm deb) do forfiles /P %1 /S /M *.%%a /C "cmd /c if @isdir==FALSE extract_nested_archives.bat @path @file"
    ARCHIVES ARE DELETED DURING THE EXTRACTION! Make a copy before running the script!
    "7z.exe" and "extract_nested_archives.bat" should be in folders available via the %PATH% environment variable.
    The first parameter of extract_nested_archives.bat is the full path name of the archive or folder that should be fully expanded; the second parameter is just the archive or folder name without the path. So you should run "c:\temp\extract_nested_archives.bat
    c:\temp\outer.rar outer.rar" from the command line to completely expand "outer.rar". "c:\temp" must be the current folder.
    Best regards, 0x000000AF

    Incredibly useful!  Thank you so much.  I did make a couple of small changes to make the script a little easier to use from the end-user perspective.
    First - I don't like making the user input the redundant second parameter, so I added this snippet which extracts it from the first parameter.  The first line of the snippet enables delayed expansion so that special characters in our file name don't
    break anything.  The second line pulls the parameter into a variable, and the 3rd line uses delayed expansion on that new variable.  Before implementing delayed expansion I had problems with file paths which included parentheses.
    SetLocal EnableDelayedExpansion
    Set SOURCE=%1
    For %%Z in (!source!) do (
    set FILENAME=%%~nxZ
    set FILENAME=%FILENAME:"=%
    Anyway once that was done, I just used %FILENAME% everywhere in the script instead of
    %2 (making sure to correct quotes as needed)
    This way, to run my script all you need to run is:
    C:\temp\extract_nested_archives.bat C:\temp\Archive.zip
    Second - I didn't want to modify the Windows environment variable.  So I replaced
    7z with "%PROGRAMFILES%\7-zip\7z.exe"
    I also replaced extract_nested_archives.bat with "%~f0" (which represents the full path+filename of the current script).
    Here is my full script now.  Tested on Windows 8 with the 64-bit version of 7-zip installed:
    @echo off
    Setlocal EnableDelayedExpansion
    Set source=%1
    For %%Z in (!source!) do (
    set FILENAME=%%~nxZ
    set FILENAME=%FILENAME:"=%
    move /Y %1 "%TMP%\%FILENAME%"
    md "%FILENAME%"
    "%PROGRAMFILES%\7-zip\7z.exe" x -o%1 -y "%TMP%\%FILENAME%"
    DEL "%TMP%\%FILENAME%"
    for %%a in (zip rar jar z bz2 gz gzip tgz tar lha iso wim cab rpm deb) do (
    forfiles /P %1 /S /M *.%%a /C "cmd /c if @isdir==FALSE "%~f0" @path @file"

  • Utility to extract ADS file from Planning application

    Hi,
    We had a Planning System 9.3 Classic application. We wanted to create a replica of the same as an EPMA application. We used a utility to extract ADS file from classic application. Post that the classic application has locked i.e we are not able to edit it. All dimensions are in view mode. Kindly let us know how to revert back in edit mode for classic application.
    Thanks and Regards,
    Meenal Dhall

    Hi,
    I think you have come across the bug in the EPMA extractor, if I remember correctly it changes a value in a table which stops you editing in classic.
    You can resolve this looking at your planning system database.
    There is a table called HSPSYS_PROPERTIES.
    The field property_id is the application id, you find what application it relates to by looking at HSPSYS_APPLICATION
    Once you know the id for your application
    In the table HSPSYS_PROPERTIES look for a row with EDIT_DIM_ENABLED and change the value in the next field from false to true
    Restart planning and you should be able to edit again.
    Cheers
    John
    http://john-goodwin.blogspot.com/

  • No luck trying to extract .inf files from drivers downloaded as zip files

    Hi,
    I am trying to extract .inf files from drivers downloaded as zip files from the Toshiba website, but I'm having no luck.
    I am trying to deploy images using WDS on server 2008 and need the inf files for this to succeed, does anyone have the answer to this problem.
    thanks in advance.

    Hi
    I found this tip:
    Someone wrote this:
    +You have to run the installer, wait for it to get to the first point where you click Next to continue, and then leave it sitting there. Go into your system's Temp and find the particular one with the installation files inside it that were extracted, then copy it someplace, go back to the installer and cancel it.+
    You could also try this freeware application to extract the package:
    http://www.legroom.net/software/uniextract

  • Extracting .xls file from an FTP site

    Hi there,
    I'm trying to run a DI solution that is extracting a file from a FTP site. DI can see the file but it gives me an error saying that:
    FTP could not transfer file <FTP site/myfile.xls> from host <IP Address>: <>. Please ensure that the FTP relative path for the SAP R/3 working directory is set correctly such that the data file is accessible to the FTP
    Can someone please help with this.
    Thank you.
    Gsecure

    Hi,
    thats SP1 for DS XI 3.2
    I think there is no delta from XI 3.2 to XO 3.2 SP1 what means you have to do a fresh install and upgrade repo.
    But you better doublecheck before but i think there is no delta.
    Regards
    -Seb.

  • Delete a .csv file from desktop system

    Hi All,
    My requirement is to read the .csv file from the desktop system having the shared folder and delete the file after read successfully.
    Here I can read the .csv file from the location using the function RFC_REMOTE_FILE and updated the content into internal table.
    But I cant delete the file from the presentation server ( Desktop system).
    Can anyone tell me how to delete the .csv file from the desktop system on different location.
    Note:
    I followed this link to read file:
    http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/9831750a-0801-0010-1d9e-f8c64efb2bd2&overridelayout=true

    Hi Rob,
    Thanks. I solved this problem myself.
    The solution to delete the file from remote system is
    concatenate 'DEL' i_filename i_dirname into v_bkfile separated by space .
    call function 'RFC_REMOTE_EXEC'
      destination  c_dest
      exporting
        command               = v_bkfile
      exceptions
        system_failure        = 1  MESSAGE v_ermsg
        communication_failure = 2  MESSAGE v_ermsg.

  • How to delete file from other system(LAN)

    Hello folks,
    Cud u plz help me out in finding code for deleting a file from other system i.e in LAN,assuming that the connection is already established to that system.

    > Cud u plz
    sigh
    Please make the extra effort to write out words such as "could", "you", and "please". The extra keystrokes won't cost much in the way of time, and the enhanced clarity will be appreciated by those communicating on a forum with international readership. Also, it will give the appearance that you take your question seriously, which will in turn make your question look more interesting to answer.
    Thanks!
    ~

  • I have to extract all files from a hard drive of a macbook pro that is no longer working. All I have to access the hard drive is a PC, is this possible, and if so how?

    I have to extract all files from a hard drive of a macbook pro that is no longer working. All I have to access the hard drive is a PC, is this possible, and if so how?

    You could try Paragon HFS+ for Windows. That would allow you to read and write files from a Mac-formatted drive.
    Clinton

  • How to extract rpt file from .b1px file in SAP B1

    How to extract rpt file from .b1px file in SAP B1

    Hi Trupti,
    You will not be able to export .b1px file without importing in SAP B1.
    Please import .b1px file in SAP B1 and then export .rpt file from SAP B1 one by one.
    Hope this helps
    Regards::::
    Atul Chakraborty

  • How to extract single file from multipart zip archives?

    Hello,
    I have a 10 parts of zip files that contains single file. When I extracted these, each program taking an action seperatly to each parts and gave me 10 files in a different folders. In a Windows, it is so simple. Winzip,winrar automaticaly discovering that are the parts of a single file and give me that.
    My question is simple: How to extract single file from a multipart zip/rar archives ?
    I know a way in "Terminal" but it is not comfortable for big parts.

    I tried with UnArchiver but it extracted each file seperately like showen in a photo

  • Why is it that when i trash files from my system drive it asks for my permission but doesn't show up in the trash bin?

    Hey guys, need help.
    I've been trying to figure out why everytime i delete files from my system drive, it doesnt show up in my trash bin.
    Has anyone encountered the same issue?
    Would appreciate any help guys.
    Thanks.

    1. Triple-click the line below to select it:
    ~/.Trash
    2. Right-click or control-click the highlighted line and select
    Services ▹ Show Info
    from the contextual menu.* An Info dialog should open.
    3. The dialog should show "You can read and write" in the Sharing & Permissions section. If that's not what it shows, click the padlock icon in the lower right corner of the window and enter your password when prompted. Use the plus- and minus-sign buttons to give yourself Read & Write access and "everyone" No Access. Delete any other entries in the access list.
    4. In the General section, uncheck the box marked Locked if it's checked.
    5. From the action menu (gear icon) at the bottom of the dialog, select Apply to enclosed items and confirm.
    6. Close the Info window and test.
    *If you don't see the contextual menu item, copy the selected text to the Clipboard (command-C). Open a TextEdit window and paste into it (command-V). Select the line you just pasted and continue as above.

  • How to use utl_ftp package to transfer files from one system to another

    Hi all,
    How to use utl_ftp package to transfer files from one system to another using plsql procedure .
    I failed to find the appropriate document ,kindly help me .
    Thanks,
    P Prakash

    prakash wrote:
    I am sorry tell you, this blog is not enough to for me.It's not a blog, it's Chris' website where you download the package from. Once you download the package and unrar it, then the package spec gives instructions (and I think there's instructions on how to install it too).
    Kindly let me know if there is any oracle documentation for this .It's not an Oracle created package, it was created by Chris. It's based around the UTL_TCP package if you really want to get into the nitty gritty of how it works, but I don't think you do.

  • Transfering file from one system to another

    Hi,
    Is there any mean by which we can transfer file from one system to another without programing???
    Thanks and best Regards.
    Kusum.

    If you have Unix Utility program you can execute it with
    MV source destination
    Regards
    Shital

  • I'm trying to extract audio files from my OLYMPUS Digital Voice Recorder VN-6200PC I am getting this error whenever I try to play its file type on my MacBook Pro,"The document "VN622195.WMA" could not be opened. The movie's file format isn't recognized."

    I'm trying to extract audio files from my OLYMPUS Digital Voice Recorder VN-6200PC
    I am getting this error whenever I try to play its file type on my MacBook Pro,"The document “VN622195.WMA” could not be opened. The movie's file format isn't recognized."

    The mac has no native way to read .wma files (these are Windows Media Audio files).  Do a search on the internet for playing wma files on a mac.  You will find several links to solution (one is to use flip4mac).

Maybe you are looking for

  • HP Officejet Pro 8500 Wireless (Vista x64) Will not print after Sleep Mode

    Hello, As the title says the printer does not print after coming back from sleep mode. Before entering sleep mode the printer works fine and is responsive. After coming back from sleep mode the printer is still responsive (buttons on the touch screen

  • .pkf files opening in Adobe Reader

    I sincerely hope someone can help me rectify an awkward problem.  Yesterday, for some crazy reason, I clicked on a .pkf file and opened it with Adobe Reader. To make matters worse, that infamous little box was checked, and now ALL my .pkf files are o

  • Need help in HANA Coding

    Hi All, We have a requirement to calculate the Position Start/End Dates of an Employee. Logic: look for the earliest date when the position started and pick the Start date as the position start date,           look for the latest date when the positi

  • Urgent : req still running

    hello, i had strucked at ,one of ODS is uploading to three cubes ..where the req is still running the message is given as It is possible that data packets must be processed manually. error in inbox 00397 IDocs searched through for date 06.12.2006. 00

  • How do I uninstall PS 7 without a functional installer?

    Since I installed Mac 10.5.8 my PS 7 will not funtion, and an error message pops up that says it cannot be used. I purchased PSE 10 to update photoshop and it says there is not enough memory to install it.  I figured if I ununstalled PS 7 that should