Filesystem Copy Slowdown

I work with animations, so I produce many thousands of frame files. The frames are rendered on separate machines, and then copied to a central hard drive where they can be locally accessed.
I have continually run into the issue of bad performance while copying those files to a local, external hard drive. I am not referring to typical "bad performance." I am well aware of the physical limitations of 7600 RPM USB-connected hard drive. The performance I am experiencing, however, must be due to something else.
The average file size of a single frame is 10MB. Copying a single 10MB file to the external hard drive takes less than a second. Copying 10 files, or perhaps even 50 files, will produce the same performance per file. However, if I copy a large chunk of files, perhaps just 500, it will start out at the expected performance and then very quickly die down to the most hideous pace you could imagine. If I am lucky, it will copy one file every five seconds. Sometimes it will stall on a single file for several minutes.
The Macs' internal drives are all SATA II, formated in HFS+, with 500GB of capacity. The external hard drive is 1.5Tb formated in FAT32, with a USB2.0 connection.
When I use the word "copy," I am referring to the Mac operating system's copy function. By this I mean the copy that takes place when I drag and drop files into the external hard drive. However, I also tried different copy functions. I created a Python script that copies files. The same problem occurs. I also tried rsync, but it also starts fast and then dies to a snail's pace.
What is causing this hideous performance problem?
It is not dependent on the machine being used. I have seen exactly this behavior across five Mac Pros and three Mac Minis. Each machine is running Mac OSX 10.5.8.
Is it the filesystem? Why would I be getting such horrible FAT32 performance on a Mac?
Many Thanks.

This is a WAG, but most likely USB and FAT-32 format. What kind of performance do you get with a Mac OS X Extended formatted ext HD over FireWire? That's where I'd be looking for bottlenecks.

Similar Messages

FileSystem Copy for BW3.5

Hi,
We have installed NW04 for BW3.5 on a HP Host. We plan to move it to a new HP UX server. All we will be doing is copying the file system from old host to the new host as the DB/System name will remain same.
My question is: Since NW/WAS640 also has J2EE component, do we need any additional steps for that? or copy anything from old to new for any J2EE filesystem?
Thanks

Hi
Its the usual copy back, so just follow the standard procedure. That should be fine.
if it is a production environment, takelittle more caution step by having a very good backup.
Regards,
GPK.

OS Filesystem Copy of EM Job Output Log?

Is there a predictable location & filename to view OEM (Database Control: dbconsole) output logs for jobs that run? I have a nightly backup that runs and I want to use a simple korn shell script to do some evaluation on the output log of the RMAN backup.
Thanks,
Keith

I too was wanting to look at the output of the jobs. OEM was reporting the job status as successful, but that just meant that it was able to attempt to run my script.
I put this together to give me more info on failures. Test an modify to meet your needs, particulary the last line.
select E.START_TIME,D.JOB_NAME,D.JOB_DESCRIPTION,D.JOB_TYPE,C.STEP_NAME,B.STEP_ID,B.PARENT_STEP_ID,B.STEP_NAME,A.OUTPUT
from sysman. MGMT_JOB_OUTPUT A, sysman.MGMT_JOB_HISTORY B,sysman.MGMT_JOB_EXECUTION C, sysman.MGMT_JOB D,sysman.MGMT_JOB_SCHEDULE E
where A.OUTPUT_ID=B.OUTPUT_ID
and B.JOB_ID=C.JOB_ID
and C.JOB_ID=D.JOB_ID
and E.SCHEDULE_ID=D.SCHEDULE_ID
and rownum<10 and output like '%ORA-00942%'

Database copy on same server

Hi guys,
I want to copy the existing 6.40 system TBW to a new instance TBX on the same server. I've created all filesystems, copied the content, changed SIDs and paths in environment scripts and profiles and restored the database TBW to TBX with brdb6brt. Now my problem is to understand how to set the permissions and users correctly. the authorizations are stored in the DBM(SYSADM,SYSCTRL and SYSMAINT group) and their names contain the SID of the instance. So what do I do? Copy the db2tbw user to db2tbx and assign the "wrong" old groups to it? Or use the same user db2tbw also for db2tbx? Or is it possible to have more than one DBM per server? What are the best practices about more than one SAP DB/2 instance on one server? Any docus around?
Regards,
Pascal

Hi Pascal
use everything separate
[except Database Connect Users since Database schemata cannot be renamed .]
There is nothing special when performing a systemcopy on the same server .
Hint : The group names need to be adjusted in the DBM config
>>>snip<<<
Copy the db2tbw user to db2tbx and assign the "wrong" old groups to it?
>>>snip<<<
No , better
create an new user db2tbx
create the according groups
dbtbxadm , dbtbxctl , dbtbxmnt and assign db2tbx to these groups
create an new database instance db2tbx and restore your
database into this instance
>>>snip<<<
Or use the same user db2tbw also for db2tbx?
>>>snip<<<
This is not possible , you need an new separate database instance "db2<dbsid>" for your new Database TBX .
>>>snip<<<
Or is it possible to have more than one DBM per server?
>>>snip<<<
Yes of course you can have more than one database instance per server .
Best regards
dirk

Computer to computer copy slows down over internal airport network over time

This is a strange problem but here goes...
Some useful background info:
I have recently successfully downloaded a 212MB file from a site on the internet through a new Apple Airport Express network I just created as a troubleshooting method in about 12 minutes or so and I don't seem to have any slowing down issues with internet connections to websites and downloads.
The issue I seeing with internal network copies:
When I try to copy files from an Apple PowerBook G4 running Leopard 10.5.2 to an iMac G5 running the same version of the OS, copies over the internal network exhibit a very odd behavior. The copy is from the Power Book G4's Public folder to any folder on the iMac.
The copy starts in The Finder just fine and seems to be going very fast as expected by an internal network copy but after some short arbitrary amount of time, the copy slows down, the estimated times grows slowly but consistently for the copy to complete and eventually the OS says there was a problem and the copy fails. There's no error number of message other than the copy couldn't be completed. As I mentioned above, the time to complete the transfer id initially estimated as a very short amount of time but eventually, the copy action hits a brick wall where it slows down further and further until it fails.
Other notes: the copy I've been attempting consists of files and folders that are of various sizes but I've tried it with small and large copies and this has worked fine in the past with Leopard and other, older OS' over the WRT54G's wireless network. I created the Airport Express network to see if that would solve this problem but it hasn't which makes me think the issue is with the router and not the wireless network.
Additional details:
Sometimes, the reported copy slowdown occurs at 5MB into the copy, other times it doesn't happen until it gets about 50MB into the copy. What's strange to me is, large file downloads over the Internet are just fine. There does appear to be some burst that degrades a bit over time but not like the internal network copy's speed which just drops off really quickly.
I have been successful in the past copying from Mac to Mac over this exact same router using the WTR54G's wireless network but I started having problems fairly recently and now the copy doesn't finish over the WRT54G or the Airport Express network that I recently created to see if the issue was with the WRT54G's wireless network settings or hardware.
There's plenty of free space on the machine that I'm copying to but this happens repeatedly when I try to copy from one computer running Leopard to another.
I don't think this issue is related to the Airport Express because I recently created a new network to test this issue called Bedroom Airport Express. Previously, when I'd been successful performing these types of transfers, I was using the LinkSys WRT54G's built in wireless capabilities and had no issues using Go To and entering the computer's IP address.
The router is still the WRT54G on ATT's DSL network using DHCP and there's no special config., other than that I had to re-IP the internal network to 192.168.199.1 for work related reasons but that's about it.
The MTU had to be increased to 1492 for the internet to work properly when I got my first WRT54G; an older version of this device that I replaced recently with a new one [although this copy worked with the new one as well prior to the issue I'm posting about coming up].
The firmware version of the WTR54G router is v8.00.2 and the Airport Express has been recently updated to the latest firmware available (this issue was occurring over the WRT54G's wireless network before the firmware upgrade to the Airport Express).
If anyone knows what if anything I can do to troubleshoot what the issue is, I would greatly appreciate the assistance.
This networking stuff is a bit of a black art to those of us that are dilettantes in this arena and I certainly qualify as a dabbler in the art of network configuration.
I've been using a LinkSys router for a long time with a Mac/PC internal network that connects to the internet and computers inside the internal network for many years now and have had few issues.
Thanks in advance,
Darin

I'm new to the Apple discussions, although not new to Apple computers and I've seen that this post has received quite a few views but no responses. Maybe someone could direct me to the right place to post this, if this is not that place, or, suggest some terminal tools that I could use to troubleshoot the issue with the Airport network or LinkSys Router I'm using [WTR54G v.8.0.0.x]? Thanks in advance for any assistance anyone can provide to a networking noobie.

$ORACLE_HOME/appsutil/clone folder missing on DB Tier

Env - APPS Tier - SLES 10 - SP3 - 12.1.2
DB Tier - zO/S - s390 - SLES 11-SP1
I'm missing my $ORACLE_HOME/appsutil/clone folder on the DB tier
Here's the catch - this instance was upgraded to 11gR2 in Sept - and I followed the steps to "Implement Autoconfig on a Recently Upgraded ORACLE_HOME".
Today is the first time we're re-cloning from PROD - so I've laid down a new APPS tier filesystem (copied from PROD) and the Senior Oracle DBA is in the process of bring up the DB after a flash copy.
When I log into my DB server, I cannot find the above folder - but find the following folders under appsutil
drwxr-xr-x 2 oracle dba 80 2012-08-19 05:10 sql
drwxr-xr-x 3 oracle dba 104 2012-08-19 05:10 java
drwxr-xr-x 2 oracle dba 176 2012-08-19 05:10 media
drwxr-xr-x 4 oracle dba 96 2012-08-19 05:10 perl
drwxr-xr-x 3 oracle dba 440 2012-08-19 05:10 html
drwxr-xr-x 2 oracle dba 48 2012-08-19 05:20 temp
drwxr-xr-x 2 oracle dba 3536 2012-08-19 05:20 template
drwxr-xr-x 5 oracle dba 144 2012-09-25 13:35 out
drwxr-xr-x 5 oracle dba 304 2012-09-25 13:35 log
drwxr-xr-x 5 oracle dba 144 2012-09-25 13:35 scripts
drwxr-xr-x 5 oracle dba 144 2012-09-25 13:35 outbound
drwxr-xr-x 5 oracle dba 176 2012-09-25 13:35 install
drwxr-xr-x 5 oracle dba 1080 2012-09-25 13:35 bin
I know I have to run the perl <AD_TOP>/bin/admkappsutil.pl (829343.1) again - but my apps tier is not yet configured for use. So I don't have an '.env' or 'environment' to source.
Can I cd to the AD_TOP on the APPS tier and run the above command or is there any other way to create (rather re-create) the appsutil/clone folder?
Thanks

864641 wrote:
Env - APPS Tier - SLES 10 - SP3 - 12.1.2
DB Tier - zO/S - s390 - SLES 11-SP1
I'm missing my $ORACLE_HOME/appsutil/clone folder on the DB tier
Here's the catch - this instance was upgraded to 11gR2 in Sept - and I followed the steps to "Implement Autoconfig on a Recently Upgraded ORACLE_HOME".
Today is the first time we're re-cloning from PROD - so I've laid down a new APPS tier filesystem (copied from PROD) and the Senior Oracle DBA is in the process of bring up the DB after a flash copy.
When I log into my DB server, I cannot find the above folder - but find the following folders under appsutil
drwxr-xr-x 2 oracle dba 80 2012-08-19 05:10 sql
drwxr-xr-x 3 oracle dba 104 2012-08-19 05:10 java
drwxr-xr-x 2 oracle dba 176 2012-08-19 05:10 media
drwxr-xr-x 4 oracle dba 96 2012-08-19 05:10 perl
drwxr-xr-x 3 oracle dba 440 2012-08-19 05:10 html
drwxr-xr-x 2 oracle dba 48 2012-08-19 05:20 temp
drwxr-xr-x 2 oracle dba 3536 2012-08-19 05:20 template
drwxr-xr-x 5 oracle dba 144 2012-09-25 13:35 out
drwxr-xr-x 5 oracle dba 304 2012-09-25 13:35 log
drwxr-xr-x 5 oracle dba 144 2012-09-25 13:35 scripts
drwxr-xr-x 5 oracle dba 144 2012-09-25 13:35 outbound
drwxr-xr-x 5 oracle dba 176 2012-09-25 13:35 install
drwxr-xr-x 5 oracle dba 1080 2012-09-25 13:35 bin
I know I have to run the perl <AD_TOP>/bin/admkappsutil.pl (829343.1) again - but my apps tier is not yet configured for use. So I don't have an '.env' or 'environment' to source.
Can I cd to the AD_TOP on the APPS tier and run the above command or is there any other way to create (rather re-create) the appsutil/clone folder?Does the directory exists on the source database node? If yes, copy it from the source server to the target one.
If the directory is missing on the source database node, then it has to be created there then it can be copied to the target server.
Thanks,
Hussein

Regarding Migration from Weblogic 10.3.0 to 10.3.3

We have configured a domain in our weblogic 10.3.0 installation. With in this domain A J2EE application has been deployed with datasources configuration, JAAS configuration.
Is it possible to migrate to Weblogic 10.3.3 without making any changes in our existing application configuration and domain setup?
If yes can any one please tell us what is the procedure?
Thanks in advance.

Hi,
first of all could you please clarify the need for migration from BO XI 3.0 to 3.1, as you can simply apply SP1 on top of 3.0 and it becomes 3.1.
if you have considered this and still needs to migrate, following are the 3 posible options.
1. Live-Live Migration - Import Wizard is the tool
2. BIAR - Live Migration - Import Wizard is the tool
Please reffer the Import Wizrd User guide for the option 1 and 2 link provided in the above thread.
Also the 3rd option is to perform the Databse and Filesystem copy, find the step below.
Launch CCM from the Destination System
Copy all of the file store from your Source implementation to the file store folder for your Destination implementation
Stop SIA - right click and select properties - Go to the Configuration tab - Click on the button to specify the data source -
Select "Copy Data from another Data Source". This will copy the data from your source to the Destination.
Regards,
Ramu.

Migration from BOE Xi 3.0 to BOE Xi 3.1

Hi ,
Can any share the documents / views about to Migration from BOE Xi 3.0 to BOE Xi 3.1
thanks in advnace,
Reddepap K

Hi,
first of all could you please clarify the need for migration from BO XI 3.0 to 3.1, as you can simply apply SP1 on top of 3.0 and it becomes 3.1.
if you have considered this and still needs to migrate, following are the 3 posible options.
1. Live-Live Migration - Import Wizard is the tool
2. BIAR - Live Migration - Import Wizard is the tool
Please reffer the Import Wizrd User guide for the option 1 and 2 link provided in the above thread.
Also the 3rd option is to perform the Databse and Filesystem copy, find the step below.
Launch CCM from the Destination System
Copy all of the file store from your Source implementation to the file store folder for your Destination implementation
Stop SIA - right click and select properties - Go to the Configuration tab - Click on the button to specify the data source -
Select "Copy Data from another Data Source". This will copy the data from your source to the Destination.
Regards,
Ramu.

OS and DB migration

Hi Guys,
Please share with me if any one having SAP EP migration
OS- Windows 2003 Server to Solaris
DB-SQL Server to Oracle
Thanks.

Hi ,
Please migrate the portal system ( or any Java based systems ) using SAPINST as the java system stores persident data in the filesystem as well as data in the database.
For java based systems it is mandatory to run the sapinst for any system copy as the migration controller needs to be invoked .
The migration controller deletes all information of the source system from the database after the database is built by the Jload.
The instance ID of a java instance is a unique id and the migration controller is responsible for the correctness of the instance for a particular combination of hostname, instance number and SID. Hence a db restore with a filesystem copy from source to target in case of a homogeneous system copy will not suffice .
The application specific data and SDM data (in the filesystems ) are copied as SAR archives by sapinst and uncarred in the target.
Hope this answers your querries.
Regards
Ratnajit

Oracle Upgrade interrupted inbetween, How to start the database?

Hi All,
Oracle upgrade is interrupted in between during the migration.
could you please provide me with the detail steps to start the database again with the 9i.
Thanks,
Anu.

Where is "in the middle"? Using dbua? During filesystem copy? Without specifying where it broke the only thing one can suggest is to restore everything from before the upgrade (Oracle binaries and database).
Markus

OS Error 665(failed to retrieve text for this error. Reason: 15105) while performing backup

Hi everyone,
I am performing a backup of sharepoint 2010 content database on my sql 2008 std edition which has file stream enabled and I experience the following problem.
When the backup reaches 40% (both when I do it via SSMS or via Maintenance plan) I get the following error:
BackupIoRequest::ReportIoError: write failure on backup device 'K:\SQL Backup\SP_Content2010.bak'. Operating system error 665(failed to retrieve text for this error. Reason: 15105).
Error: 18210, Severity: 16, State: 1.
I also tried backup the database on a different disk to make sure the problem is not there, but the same error appeared when the backup reached 40 %. I also executed DBCC CHECKDB against the database but it run successfully.
I could not find any other relevant source what the problem might be.
Any help will be very much appreciated.
Thanks

Hello,
Honestly it's going to come down to testing and root cause analysis. I don't have a single shot answer for you unfortunately.
I would start by doing the hotfix on that KB since you had OS Error 665 (again, you can run net helpmsg 665 to see the os error message). The SQL Server error 3041 (select * from sys.messages where message_id = 3041 and language_id = 1033) is "BACKUP failed
to complete the command %.*ls. Check the backup application log for detailed messages." Error 18210 is "%s: %s failure on backup device '%s'. Operating system error %s.". This points me back to the filesystem or storage.
Could you P->V the server during a maintenance window so you can test? The biggest thing here is that you don't have any valid backups so you're running the risk of losing data.
While not a method I use, you could stop SQL Server and filesystem copy the database and filestream share in the interim.
-Sean
Sean Gallardy | Blog |
Twitter

CP command to Windows machine

If I use cp to copy the bulk of my user directory to a Windows machine is there anything I need to worry about with regard to symbolic links, hard links, and the like? I don't know much at all about resource forks - but thought I read that CP under Tiger changed the way they were handled.
Also, if I did the copy from the Windows side using a tool like robocopy, are there worries there?
My user directory contains pretty standard stuff: iPhoto libraries, Final Cut projects, Word docs, music...
Thanks for any advice.
Dual 1.8 G5 Mac OS X (10.4.5) 1.5 GB RAM, iPod, iPod Mini

Hi bjw,
> I don't know much at all about resource forks - but
I read that CP under Tiger changed the thought way
they were handled.
   Yes but does the remote filesystem handle resource forks? I would tend to doubt it. In that case, what happens depends on how you are connected to the computer, which you don't bother to tell us. However, probably each resource fork would be copied to a separate file with the same name as the original but with "._" prepended to the name.
   Softlinks that contain absolute paths would be broken and if they contain relative paths they would work only if the path is contained completely within the portion of the filesystem copied. Hardlinks would cease to be links but the data would be copied to a new file.
Gary
~~~~
   QOTD:
      I love your outfit, does it come in your size?

Official Systemcopy Guides

Hi,
I am in the process of developing a concept for a BOBJ system landscape.
Now I am totally new to BOBJ, but quite aged in SAP ERP environments, and thinking
about some standard tasks in ERP, I am missing some standard procedures in the BOBJ documentation.
Are there already some "officially" supported systemcopy procedures documented somewhere for BO XI 3.1?
I did not find any yet. If there are, they might be worth to be put into the "sticky" thread at the forums top
(i.e. SAPNET search with "business objects systemcopy" goes without any good hit)
I worked through the admin guide and I at least supose one can install onto a single box
several BOBJ servers. i.e. by utilizing multi IPs so you can keep the ports for diffrent servers
on the same machine the same.
I find the way to assign diffrent ports very uncomfortable esp. as the admin guide states only
some of them to be configurable but quite a couple of ports to be needed in addition in the firewall admin chapter.
So I suspect there might be potential possible port conflicts in the later life of a system, when installing
multiple instances of a BOBJ server onto a single box.
So I am just thinking of a setup like
hostname       bobphys
virtual ip         bobprog   with Server installed in /usr/sap/BO1
virtual ip         bobqual   with Server installed in /usr/sap/BO2
virtual ip         bobsand   with Server installed in /usr/sap/BOS
All servers using the same ports but using bindings to the seperate virtual Ips utilizing the
"server host identification options" from the admin guide.
Some kind of standard task would be to periodicly copy BO2 to BOS.
Has anybody insights, if this approach would be a usefull one and how any needed adoptions for
hostnames / pathnames might be needed to be performed after a BO2 -> BOS copy.
Thanks for insights
Volker

Hi,
The System Copy is between two different versions is reffered as Migration in Business Objects Enterprise.
If the versions are different the follwing options are available.
1. Live-Live Migration - Import Wizard is the tool
2. BIAR - Live Migration - Import Wizard is the tool
Please reffer the [Import Wizrd User guide |http://help.sap.com/businessobject/product_guides/boexir31/en/xi3-1_bip_importwiz_en.pdf]for the option 1 and 2.
Also the 3rd option is to perform the Databse and Filesystem copy, find the step below.
Launch CCM from the Destination System
Copy all of the file store from your Source implementation to the file store folder for your Destination implementation
Stop SIA - right click and select properties - Go to the Configuration tab - Click on the button to specify the data source -
Select "Copy Data from another Data Source". This will copy the data from your source to the Destination.
If you are on the same version and using XI 3.1 or above you can use [LCM|http://help.sap.com/businessobject/product_guides/boexir31/en/xi31_LCM_User_en.pdf] for the Promotion of the objects from one system to another in case both source and destination are on the same level.
Regards,
Ramu.

Is following message due to 'Java' setting left unchecked? Mac OSX supportsUser Authentication Mechanism (UAM) plug - ins from other manufacturers to control access to servers.To use a UAM, copy the plug - in to: Library/ Filesystems/ AppleShare/ Authenti

Is following message due to 'Java' setting left unchecked?
Mac OSX supportsUser Authentication Mechanism (UAM) plug - ins from other manufacturers to control access to servers.To use a UAM, copy the plug - in to: Library/ Filesystems/ AppleShare/ Authenti

Man that is an ancient message.
The last time I saw that was using Mac clients connected to a Microsoft (Windows) Server running 'Services for Macintosh' which included the ability to act as an AppleShare compatible file server. Because Microsoft have a different security system for defining accounts which includes the 'domain' as well as username, the standard Mac AFP client did not know how to send that information.
Therefore Apple made it possible to installed a plugin in the form of a UAM or User Authentication Mechanism which added the ability to send this information to login to the fileserver.
See http://support.microsoft.com/kb/101747
However Microsoft have long discontinued 'Services for Macintosh' and now the only way for a Mac to connect to a standard Windows Server is via SMB not AFP. I don't believe this plugin is available to download anymore.

Copy BLOB column to filesystem

Hello everybody.
Im working with RedHat Linux and Oracle Database 10gR2.
My problem , I have some tables with Blob columns those Blob columns have pictures (jpg, gif etc) , and I need copy those pics ,blob colums, to filesystem.
Question : Can anybody send me a link where I can see a example of how to ?.
Thanks in advanced and regards everybody.

You can check this asktom post
Retrieving Data from BLOB into a file
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:6379798216275
or search for more demo on asktom.

Filesystem Copy Slowdown

Similar Messages

Maybe you are looking for