Verify file copy to Isilon

We're about to move several terabytes of files & folders from six different shared volumes across two Xserves (running Mac OS X 10.6.8 Server) attached to two Apple xServe RAIDs and one Promise RAID. The files are being transfered to an EMC Isilon server that has a significant upgrade in available space for our users. When I've made a Mac to Mac copy I've used the find command piped into an xargs command to create an MD5 file of the original and the copy and then used the diff command to compare the two. Unfortunately, when I try to run the commands to create the MD5 file on the Isilon server the command just hangs and nothing every happens. (I've left the command running over night even when it should only take ten minutes on a smaller folder.) Are there any other utilities or command lines I can use to compare the two folders?
-Doug

Update: I've found the FileMerge utility from Xcode. Unfortunately, when discovering differences and merging them it doesn't maintain the correct dates and it doesn't maintain metadata like label colors.
-Doug

Similar Messages

Group Policy Preferences File Copy - Access is Denied on 2003 but not 2008 R2

Hello,
I have created a GPO which copies a file from a network share into a new folder under Program Files. This policy works just fine on a Windows 2008 box, but not on 2003. I've used "psexec -i -s cmd.exe" to verify system account permission to the
share. I am able to successfully browse and copy files from the share as the system account on both boxes.
However, when the GPO attempts to perform the file copy, it does not work, and generates the following error message:
Event Type: Warning
Event Source: Group Policy Files
Event Category: (2)
Event ID: 4098
Date: 8/28/2013
Time: 3:32:12 PM
User: NT AUTHORITY\SYSTEM
Computer: Server01
Description:
The computer 'file.txt' preference item in the 'TXT File Copy {9176122B-1A50-4AB8-91D9-6E8553727E18}' Group Policy object did not apply because it failed with error code '0x80070005 Access is denied.' This error was suppressed.
I am trying to avoid writing a login script, so I am hoping someone will be able to help me figure out why this works fine on Windows 2008 but not Windows 2003. Please note file names and paths are modified here for security reasons, but the principle is
the same.
My GPO is:
Computer Configuration\Preferences\Windows Settings\Files
File (Target Path: c:\Program Files\path\to\file\file.txt)
Source file: \\share\path\to\file\file.txt
Destination File: c:\program files\path\to\file\file.txt
Action: Update
Suppress errors on individual file actions: Disabled
Read-Only: Enabled
Hidden: Disabled
Archive: Enabled
Stop Processing items on this extension if an error occurrs on this item: No
Remove this item when it is no longer applied: No
Apply once and do not reapply: No
Item-level Targeting: None
Thanks

As a user, I am able to browse the share just fine using the alias. It is only when I try to access the share using the system account that I encounter a problem.
experiencing the exact same symptoms. in the tests below, server, client1, and client2 are all are native instances of microsoft windows server.
server: windows server 2008 R2 standard SP1
client1: windows server 2003 standard SP2
client2: windows server 2008 R2 standard SP1
registry setting "DisableLoopbackCheck": unconfigured on server; unconfigured on client1; unconfigured on client2
registry setting "DisableStrictNameChecking": configured as "1" on server; unconfigured on client1; unconfigured on client2
domain user on client1 attempt to access server by name: success
domain user on client1 attempt to access server by alias: success
domain user on client2 attempt to access server by name: success
domain user on client2 attempt to access server by alias: success
local system on client1 attempt to access server by name: success
local system on client1 attempt to access server by alias: failed (system error 5 has occurred. access is denied.)
local system on client2 attempt to access server by name: success
local system on client2 attempt to access server by alias: success
all tests done using "net view \\target", but similar results were seen when using "dir \\target\share" which the domain user and local system account have access to.
the differing behavior between client1 and client 2 suggests that server 2003 requires additional configuration to allow its local system account to access an SMB share by alias.
this problem prevents group policy features (such as software installation) from an aliased file server.

FAILURE ( 5640 ): FALSE: Verify File: \\SERVER\DeploymentShare\Operating Systems\Image 11-13-2014.wim\image.wim ERROR

Guys,
I need help with this error please. I have a WDS in building 13 and another 1 in building 31; Both servers have MDT 2013. At some point, both deployment shares were linked via MDT but I deleted that few days to solve some deployment share corruption in building
31 WDS.
Presently I have 2 32bit images (my 64bit images work fine) giving me the following MDT error in BUILDING 13 during the image deployment:
FAILURE ( 5640 ): FALSE: Verify File: \\BUILDING 31 SERVER\DeploymentShare\Operating Systems\Image 11-13-2014.wim\image.wim
Litetouch deployment failed, Return Code = -2147467259 0x80004005
Failed to run the action: Install Operating System.
Unknown error (Error:00001608; Source: Unknown)
The Execution of the group (Install) has failed and the execution has been aborted. An action failed....
Please note, BUILDING 13 deployment share is not linked to BUILDING 31 deployment share. I don't understand why this error is pointing to BUILDING 31 deployment share.
\\BUILDING 31 SERVER\DeploymentShare\Operating Systems\Image 11-13-2014.wim\image.wim has been removed. I need the task sequence to point to
\\BUILDING 13 SERVER\DeploymentShare\Operating Systems\Image 11-13-2014.wim. BUILDING 13 deployment Share task sequences are pointing to
\\BUILDING 13 SERVER\DeploymentShare as well as my CustomSettings.ini and Bootstrap.ini
1. I have diskpart and cleaned the disc, no luck.
2. I have checked the Install action for both images task sequences and even verified each action on both tasks and cant find any error.
3. I have re-generated the boot images and retyped
\\BUILDING 13 SERVER\DeploymentShare on its CustomSettings.ini and Bootstrap.ini
4. I am in the process of copying Image 11-13-2014.wim back to
\\BUILDING 31 SERVER\DeploymentShare\Operating Systems BUT again I don't understand why these task sequences are pointing to building 31. If this step doesn't work, I will restore both images from a backup and re-added them to building 13 MDT.
But I don't think these steps are the solution.I am out of ideas here...

I disabled McAfee AV, moved images boot files to a different location, regenerate boot images files in MDT, update deployment share, re-add boot files to WDS & refresh WDS. This works for me.

Getting "File copy Config Sync Failed. Commit unsuccessful! " on CSS-11155

Getting a strange error "FILE copy Config Sync Failed. Commit unsuccessful!" from the following and I am not sure what it exactly means:
cce01-c35-in# sh app
APP CONFIGURATION:
Enabled PortNumber: 5001 MaxFrameSize: 10240
cce01-c35-in# sh app session
App Session Information 'no hostname':
Session ID: 839f0990 IP Address: 10.10.10.2 State: APP_SESSION_UP
css-c09-nz# commit_redundConfig "10.10.10.2"
Verifying app and redundancy configs ... -
Checking local and remote switch versions ...|
Checking Backup app session up.... \
Checking redundancy state.... -
Working /
Waiting for completion signal from remote switch .../
Verifying running-config copy success ...-
File copy Config Sync Failed. Commit unsuccessful!
localconfig: 6238 bytes
remoteconfig : 6150 bytes
css-c09-nz#

there is at least one know bug :
CSCdx89818 - config sync fails
I could not check if this applies to you since you did not provide your version.
This bug was fixed in the following release :
006.010(000.004) 05.3(00.18)S 05.0(00.54)S 05.10(01.03)S
Gilles.

Issues with File Polling and File Copy in a Cluster

Hi,
I have a 2 node cluster of SOA 11.1.1.5. Running into an issue.
I have a File Poll Process which picks up new files and then copy them over to another directory [using COPY operation of file adapter]. After that using FTP Adapter to ftp the file [using MOVE operation of FTP]. Now if I place a file Poll is creating 2 instances. I have configured HAFile Adapter and HAFTPAdapter. Now if I remove the file COPY operation I see only one instance as soon as I put back file copy it creates multiple instances again. Any idea what might be wrong.
I am following the document for file copy and ftp move http://docs.oracle.com/cd/E23943_01/integration.1111/e10231/adptr_file.htm#BABEBJAF section
4.5 Oracle File and FTP Adapters Use Cases
But in a cluster if I use file copy I see multiple instances of Polling instances.
Thanks

"puneet" <[email protected]> wrote in message
news:3fcf4fa4$[email protected]..
>
Hi I use WLI studio to add a user in a clustered environment (admin + 2managed
wli servers). I see the following message in the logs:
####<Dec 3, 2003 1:05:02 PM MST> <Warning> <Security> <suznd265><eflow_2323_wli
managed_2> <ExecuteThread: '9' for queue: 'default'> <kernel identity><14:5a2cc
dff10338da2> <090048> <Made an in-memory only change to the FileRealm.>
As a result, the user, that I added is not actually visible to the othermanaged
servers in the cluster. I have verified this by having only one managedserver
up when adding the user. Once I have added the user, I boot the othermanaged
server and bring the first managed server down. The user I have justcreated gets
lost.
This occurs if you make the change on the managed server. Can you make the
change on
the admin server? It should get pushed to the managed servers.

File Copy Dialog box hangs in mid-copy; Cancel does nothing.

Hi all,
I'm having the aforementioned issue fairly often with Windows 7. It seems to happen when copying to network shares. It is more than annoying--it is literally stopping production at our company. I would really appreciate an realistic fix
for this--that is, one that isn't the catch-all "Reinstall windows".
Let me further state that it has nothing to do with hardware--I can reliably copy the same files in XP. It is a Windows 7/Explorer thing.
It also seems to be size-related--but I haven't had the time to do scientific testing to verify this. I know that if I'm copying an Access .mdb file from my development machine (Windows 7) to a client machine (mostly windows XP) with a mapped drive,
it seems to hang. But if I compact/repair the .mdb file and bring it from, say 25 meg to 12 meg, it seems to copy fine.
Worse, is that when it hangs, I'll finally hit "Cancel" after a few minutes, and then the entire Windows 7 locks!! I can't do anything until I ctrl-alt-del and kill the Explorer.exe process, which wipes out my desktop, etc, until Explorer.exe auto-starts.
This is such a huge problem that I have to think there's a fix upcoming.
Thanks,
--Jim

Hi,
May I know how it works if you copy files between local drives? Please also boot the system to Safe Mode with networking and check this issue.
In addition, may I know more information about the machine which hosts the share folder you mapped is located, a Windows based computer? Please also
let us know the operating system.
Based on my research, I would like to suggest the following:
1.
Try disabling Receive Side Scaling, Chimney Offload, and NetDMA support and see if it works:
Information about the TCP Chimney Offload, Receive Side
Scaling, and Network Direct Memory Access features in Windows Server 2008
2.
Also run the following commands in Windows 7:
netsh interface tcp set global autotuninglevel=disabled
netsh int ip set global taskoffload=disabled
3.
Try Robocopy:
Get to Know Robocopy for More Powerful File Management
For detail information about the usage of the Robocopy command, please also refer to:
Robocopy
Meanwhile, I would like to share the following with you for your reference:
Slow Large File Copy Issues
Windows Explorer and SMB Traffic
Hope this helps.
Thanks.
Nicholas Li - MSFT
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

Existing files on drive connected to latest Airport Extreme do not show up, however files copied over the air to the drive do. Any ideas?

I have the latest Airport Extreme with a Seagate drive connected. I copied files to the drive while it was connected to my MacBook Pro. The files do not show up when connected to the AE, but they appear when connected to the Macbook, Files copied over the network to the drive appear. The drive is formatted HFS+.

After playing with this for the last few days, I found a solution. Under the "Disks" tab in Airport Utility, the "Secure Shared Disks" drop down needs to be set to "With device password". Mine was set to "With acounts", which even after correct authentication, none of the files would appear.
Another interesting note: The files that I could see, the ones I copied over the network, do not appear now. I can copy over new files, and they show up, but the ones copied prior to changing the "Secure Shared Disks" option, do not.

File Copy times

My newsreader is acting funny and dropping posted messages, so I
apologize if this shows up twice.
My comments on the file speed are that the times posted by other just go
to show how difficult it sometimes is to make good timing measurements.
I suspect that the wide variations being posted are in large part to
disk caching. To measure this, you should either flush the caches each
time or run them multiple times to make sure that the cache affects them
more equally.
Here is what I'd expect. The LV file I/O is a thin layer built upon the
OS file I/O. Any program using file I/O will see that smaller writes
have somewhat more overhead than a few large writes. However, at some
size, either LV or the OS will break the larger writes into smaller
ones. The file I/O functions in general will be slower to read and
write contents than making a file copy using the copy node or move node.
Sorry if I can't be more specific, but if you have a task that
seems way to slow, please send it to technical support and report a
performance problem. If we can find a better implementation, we will
try to integrate it.
Greg McKaskle

Maybe this is because of the write buffer?
Try mounting the media using the -o sync option to have data written immediately.

How to measure progress of file copy?

One of the requirements for my AIR application is to copy
large files from one location to another on the client computer.
I've been using the copyToAsync() method in AIR, but unfortunately
this does not trigger progress events. Copying gigabyte files that
take several seconds becomes pretty uncomfortable for users, when
they can't see the progress.
Does anyone know of an alternative or workaround? All I need
to do is copy a file from one location to another, and measure the
progress while that is happening.
Since I control the client machines (it is a closed
environment), I have the option to setup file servers on every
machine and use the file download control with Flash instead of
file copy. However, when I investigate this option it seems that
the file download control will always trigger user interaction to
select the target path (which I don't want -- I want to
programatically select the source files and the target directory).
If someone knows a solution to that problem I would happily take it
and move on :)

Hi,
Use 2 FileStream objects in async mode (openAsync). One to
read and another to write. When you read in async mode, you get
progress events. When they fire, you can read bytesAvailable
amounts of data and write that to the output filestream.
The output filestream can listen for output_progress events
to ensure that new data can be written to it or not.

Network speed affected by large file copy operations. Also, why intermittent network outages?

Hi
I have a couple of issues on our company network.
The first is thate a single large file copy imapcts the entire network and dramatically reduces network speed and the second is that there are periodic outages where file open/close/save operations may appear to hang, and also where programs that rely on
network connectivity e.g. email, appear to hang. It is as though the PC loses it's connection to the network, but the status of the network icon does not change. For the second issue if we wait the program will respond but the wait period can be up to 1min.
The downside of this is that this affects Access databases on our server so that when an 'outage' occurs the Access client cannot recover and hangs permamnently.
We have a Windows Active Directory domain that comprises Windows 2003 R2 (soon to be decommissioned), Windows Server 2008 Standard and Windows Server 2012 R2 Standard domain controllers. There are two member servers: A file server running Windows 2008 Storage
Server and a remote access server (which also runs WSUS) running Windows Server 2012 Standard. The clients comprise about 35 Win7 PC's and 1 Vista PC.
When I copy or move a large file from the 2008 Storage Server to my Win7 client other staff experience massive slowdowns when accessing the network. Recently I was moving several files from the Storage Server to my local drive. The files comprised pairs
(e.g. folo76t5.pmm and folo76t5.pmi), one of which is less than 1MB and the other varies between 1.5 - 1.9GB. I was moving two files at a time so the total file size for each operation was just under 2GB.
While the file move operation was taking place a colleague was trying to open a 36k Excel file. After waiting 3mins he asked me for help. I did some tests and noticed that when I was not copying large files he could open the Excel file immediately. When
I started copying more data from the Storage Server to my local drive it took several minutes before his PC could open the Excel file.
I also noticed on my Win7 client that our email client (Pegasus Mail), which was the only application I had open at the time would hang when the move operation was started and it would take at least a minute for it to start responding.
Ordinarlily we work with many files
Anyone have any suggestions, please? This is something that is affecting all clients. I can't carry out file maintenance on large files during normal work hours if network speed is going to be so badly impacted.
I'm still working on the intermittent network outages (the second issue), but if anyone has any suggestions about what may be causing this I would be grateful if you could share them.
Thanks

What have you checked for resource usage during one of these copies of a large file?
At a minimum I would check Task Manager>Resource Monitor. In particular check the disk and network usage. Also, look at RAM and CPU while the copy is taking place.
What RAID level is there on the file server?
There are many possible areas that could be causing your problem(s). And it could be more than one thing. Start by checking these things. And go from there.
Hi, JohnB352
Thanks for the suggestions. I have monitored the server and can see that the memory is nearly maxed out with a lot of hard faults (varies between several hundred to several thousand), recorded during normal usage. The Disk and CPU seem normal.
I'm going to replace the RAM and double it up to 12GB.
Thanks! This may help with some other issues we are having. I'll post back after it has been done.
[Edit]
Forgot to mention: there are 6 drives in the server. 2 for the OS (Mirrored RAID 1) and 4 for the data (Striped RAID 5).

Does the last updated backup in Time Machine have all my files for "standard" file copy? (i want to copy and paste files to another computer that doesn't have TM)

My personal Macbook pro died, but I did have a Time Machine backup. I have a new iMac that I would like to use as a family computer. I would like to transfer some files to the new computer (and set-up a new Time Machine backup), but archive the remainder on a non-TM drive. I was planning to:
1. File copy my last backup folder from my old TM drive to the new computer
2. Copy (and then delete) the files I want to archive on another HD drive
3. Wipe clean my current TM drive
4. Set-up TM on my new computer
Is this a good way to proceed?
Is copying the last backup folder (vs. all backup folders) enough to move my files over?
Any better approaches appreciated!

I suggest you visit MicroCenter and go to there Apple section and ask one of there guys if they are still offering the 2 terabyte Backup For the iMac, Mine was only $100.00.
So the iMac has Time machine plus the USB 2 Terabyte Backup for less than your $130.00

File copying problems

Copying some files using the Finder results in a dialogue box error message: "The Finder cannot complete the opeeration because some data in +(file name)+ could not be read or written. (Error code -36). The same files copy without error when using SilverKeeper (v.2.0.2) backup program.
I am also experiencing a problem when quitting SIlverKeeper. A second or so after selecting Quit, and after the program has actually quit, a system dialogue box appears saying that the program has unexpectedly quit and offering choices of Reopen, Report to Apple or Close.
The system also seems somewhat sluggish.
Can anyone offer suggestions?

Hi John, At this point I think you should get Applejack...
http://www.versiontracker.com/dyn/moreinfo/macosx/19596
After installing, reboot holding down CMD+s, (+s), then when the DOS like prompt shows, type in...
applejack AUTO
Then let it do all 5 of it's things.
At least it'll eliminate some questions if it doesn't fix it.
The 6 things it does are...
Correct any Disk problems.
Repair Permissions.
Clear out Cache Files.
Repair/check several plist files.
Dump the VM files for a fresh start.
Trash old Log files.
First reboot will be slower, sometimes 2 or 3 restarts will be required for full benefit... my guess is files relying upon other files relying upon other files!
Disconnect the USB cable from any UPS so the system doesn't shut down in the middle of the process.

Is it possible to have media files copied to a new location as you import them into a project?

I have roughly 800gb of footage i've taken over the past 2 years of traveling, and I am making a single video montage... The files are located on an external drive, making them slower to work with. I have a 1tb ssd but not enough free space to copy all the files to it for sorting. What i want to do is have the files copy over to the SSD as i add them to the project, so that only the footage i'm going to use gets copied over to the SSD. The only way i can determine if i am going to use the footage or not is to scrub through and mark Ins and Outs in premiere...
So i guess my question, in the simplest terms i can ask is, Is there a setting in Premiere, like the project manager tool which allows you to consolidate used media files to one destination, but one that will consolidate files as you add them to the project? or could i do this manually by repeatedly using the project manager tool to consolidate files periodically?
Hope that makes sense...
Oh, and yes, i have considered other methods to my madness, like reviewing the footage outside of premiere before deciding if i will use it or not, frankly, i have over 4000 pieces of footage and it's just much faster to scrub through them in premiere than to open each one in vlc to review. It's also much more convenient to mark my ins and outs as i review the footage, so premiere is my only option.
Thanks!

You can easily (and better, IMHO) do what you want in Adobe Prelude. It's part of the CC subscription, and may have been included with CS6.
Open the Ingest panel, navigate to the external drive where you have your source clips, make the thumbnails in the Ingest panel comfortably large, click a video clip to enable scrubbing, and use the J-K-L keys to navigate playback through the clips. Put a check mark on the clips you want and be sure to select and set up the Transfer option on the right side of the panel before ingesting. Don't select the Transcode option.
Cheers,
Jeff

File copy speeds to CSV vs non-CSV

I'm working on bringing up a 2012 R2 cluster and doing a basic test. In this cluster, I have two adapters for iSCSI traffic, one for network traffic, and one for the heartbeat. Cluster node has all the current updates on it. Everything
is set up correctly as far as I can see. I'm taking a folder with 1GB of random files in it and copying it from the C: drive of a node to an iSCSI LUN. If I have the LUN set up as a non-CSV disk, the copy happens about three time faster than if
I have it set up as a CSV disk. All I'm doing is using FCM to change the disk from CSV to non-CSV (right-click, Remove from CSV, right-click, Add to CSV). I can swap it back and forth and each time the copy process is about three time slower when
it's a CSV. Am I missing something here? I've been through all the usual stuff with regard to the iSCSI adapters, MPIO, drivers, etc. But I don't think that would have anything to do with this anyway. The disk is accessed the same with
regard to all that whether it's CSV or not, unless I'm missing something. Right now, I only have a single node configured in the cluster, so it's definitely not anything to do with the CSV being in redirected mode.
I'm not trying to establish any particular transfer speed, I know file transfers are different than actual workloads and performance tools like iometer when it comes to actual numbers. But it seems to me like the transfers should be close
to the same whether the disk is a CSV or not, since I'm not changing anything else.

Which system owns the CSV? If the system from which you are copying does not own the CSV then all the metadata updates have to go across the network to be handled by the node that does own the CSV. If you are copying a lot of little
files, there is more metadata.
Actually, metadata updates always happen in redirected IO from what I'm reading, that has been the part that I was missing. This explains it.
https://technet.microsoft.com/en-us/library/jj612868.aspx?f=255&MSPPError=-2147217396 "When certain small changes occur in the file system on a CSV volume, this metadata must be synchronized on each of the physical nodes that access the
LUN, not only on the single coordinator node... These metadata update operations occur in parallel across the cluster networks by using SMB 3.0. "
So a file copy, even when done on a coordinator node, does the metadata updates in redirected mode. Other articles seem to say the same thing, though not always clearly. So it's still accurate to say that a file copy isn't the best way to measure
CSV performance, but there doesn't seem to be a lot of pointing to the (I think) important distinction regarding how the metadata updates work. From what I can see, that distinction is probably trumping anything else such as who is the
coordinator node, CSV cache, etc. For me anyway, it makes a 3X performance difference, so I think that's pretty significant.

Files copied (backed up) on an external hard drive can NOT be opened

MacBook Pro (2.5GHz, intel Core i5)
OS X Yosemite, 10.10.2
I was running low on storage space on my Mac Pro. I copied all items on Desktop to my external hard drive. After all files were copied from MacBook Pro to the external hard drive, "get info" suggested that the amount of space taken entirely was the same between when all items were on the Desktop and when they were on the external hard drive. The number of items copied over were also identical. I should have been more careful, but I then deleted all items from the Desktop. It was until a few days later (today) that I noticed majority of the files copied over onto the external disk can NOT be opened now. The error message was "The alias “<filename>” can’t be opened because the original item can’t be found.".
I really don't want to lose these files, but where are they? They got their file sizes correct but just can not be opened. I tried copying these files back to Desktop on my MacBook Pro, that's not helpful. I tried accessing the files on a PC, still can't open the files. I tried repairing disk of the external hard drive (under Disk Utility), still can't open the files. I tried repairing disk permissions of my MacBook Pro hard drive, still can't open the files.
One last thing I tried in terminal (I am not terminal savvy at all, just googled the "The alias..." error message and copied the instruction): /usr/bin/SetFile -a a /Volumes/MAC This did not solve the problem, either.
Thank you for your help in advance!

Hi, eddie from pataskala ohio.
We love pro-active thinking here at the Apple Discussions and you're off to a great start.
Even though upgrading iTunes shouldn't affect your Library, one of the funny things about computers is that sometime, somehow, somewhere when you least expect it , they can do some not so funny things, like losing data that they're not supposed to. Which is why it's a good idea to have a back up copy of any data (such as your music) that you value in reserve ... just in case.
The most straightforward way, is to go to the iTunes Edit menu > Preferences > Advanced tab > General sub-tab and make sure your iTunes Music folder is assigned to its default location: C:\Documents and Settings\User Name\My Documents\My Music\iTunes .
Next make sure all your music is stored in that folder by using the iTunes Advanced menu > Consolidate Library option.
Now you can copy the entire iTunes folder - the iTunes Music folder and the iTunes library files in particular - to your external drive.
If anything does go wrong with the upgrade, you'll be all set to recreate your Library.

Verify file copy to Isilon

Similar Messages

Maybe you are looking for