Any ideas on how to do a local mirror for this situation?
I'm starting a project to allow ArchLinux to be used on a Cluster environment (autoinstallation of nodes and such). I'm going to implement this where I'm working right now (~25 node cluster). Currently they're using RocksClusters.
The problem is that the connection to internet from work is generally really bad during the day. There's a HTTP proxy in the middle. The other day I tried installing archlinux using the FTP image and I took more than 5 hours just to do an upgrade + installing subversion and other packages, right after an FTP installation (which wasn't fast either).
The idea is that the frontend (the main node of the cluster) would hold a local mirror of packages so that when nodes install use that mirror (the frontend would use this also, because of the bad speed).
As I think it should be better to only update the mirror and perform an upgrade not very often (if something breaks I would leave users stranded until I fix it), I thought I should download a snapshot of extra/ and current/ only once. But the best speed I get from rsync (even at night, where an HTTP transfer from kernel.org goes at 200KB/s) is ~13KB/s this would take days (and when it's done I would have to resync because of any newer package that could have been released in the meantime).
I could download extra/ and current/ at home (I have 250KB/s downstream but I get like ~100KB/s from rsync), record several CDs (6!... ~(3GB + 700MB)/700MB) but that's not very nice. I think that maybe this would be just for the first time. Afterwards an rsync would take a lot less, but I don't know how much less.
Obiously I could speed things a little If I download the full ISO and rsync current using that as a base. But for extra/ I don't have a ISOs.
I think this is a little impractical (to download everything) as I wouldn't need whole extra/ anyways. But it's hard to know all packages needed and their dependencies to download only those.
So... I would like to know if anyone has any ideas on how to make this practical. I wouldn't wan't my whole project to crumble because of this detail.
It's annoying because using pacman at home, always works at max speed.
BTW, I've read that HOWTO that explains how to mount pacman's cache on the nodes to have a shared cache. But I'm not very sure if that's a good option. Anyway, that would imply to download everything at work, which would take years.
V01D wrote:After installation the packages that are in cache are the ones from current. All the stuff from extra/ won't be there until I install something from there.
Anyway, if I installl from a full CD I get old packages which I have to pacman -Syu after installation (that takes long time).
Oh, so that's how is it.
V01D wrote:
I think I'm going to try out this:
* rsync at home (already got current last night)
* burn a DVD
* go to work and then update the packages on DVD using rsync again (this should be fast, if I don't wait long time after recording it)
And to optimize further rsync's:
* Do a first install on all nodes an try it out for a few days (so I install all packages needed)
* Construct a list of packages used by all nodes and frontend
* Remove them from my mirror
* Do further rsync updates only updating the files I already have
This would be the manual approach of the shared cache idea I think.
Hmm... but why do you want to use rsync? You'll need to download the whole repo, which is quite large (current + extra + testing + community > 5.1GB, extra is the largest). I suggest you to download only those packages and their dependencies that you use.
I have similar situation. At work I have unlimited traffic (48kbps at day and 128kbps at night), at home - fast connection (up to 256kbps) but I pay for every megabyte (a little, but after 100-500 megabytes it becomes very noticeable). So I do
yes | pacman -Syuw
or
yes | pacman -Syw pkg1 pkg2 ... pkgN
at work (especially when packages are big), then put new downloaded files on my flash drive, then put them into /var/cache/pacman/pkg/ at home, and then I only need to do pacman -Sy before installing which takes less than a minute.
I have 1GB flashdrive so I can always keep the whole cache on it. Synchronizing work cache <-> flash drive <-> home cache is very easy.
P.S.: Recently I decided to make complete mirror of all i686 packages from archlinux.org with rsync. Not for myself but for my friends that wanted to install Linux. Anyway I don't pay for every megabyte at my work. However it took almost a week to download 5.1 GB of packages.
IMHO for most local mirror solutions using rsync is overkill. How many users are there that use more than 30% of packages from repos? So why to make full mirror with rsync when you can cache only installed packages?
Similar Messages
-
My 3gs wont restore after ios 5 update. I have tried firmware restore and restore from backup, neither works. All I get is "iphone restore failed because backup session failed" Any ideas on how to make a successful restore with this ios 5 update?
I was having the same problem, I backed up my phone and then updated to iOS 5 and then it kept telling me it could not restore from my backup as it had failed. After reading a bunch of forums and posts and trying everything, I did manage to get it to work. I had tried the Time zone, creating a new computer user account, disabling anti-virus, etc.
It took a combination of things, one of which was disabling the anti-virus completely and copying the contents of the backup folder to my desktop and then deleting everything in the backup folder. Then I rebooted the PC, entirely disabled the anti-virus, copied the one backup I wanted to restore from the desktop to the backup folder and then trying to restore. Low and behold the phone said restoring from backup and I am back in business, so when in doubt, trying combining some of the fixes together. -
I cannot downlaod and install any apps. How do I change the settings for this?
I cannot downlaod and install any apps. How do I change the settings for this?
Use this article: iOS 7: If you're asked for the password to your previous ...
-
I have an iPhone 3GS and was trying to restore it...
Now it won't let me activate my phone and my itunes tells me I have no SIM card inserted when I do.
Any ideas on how to get it working again?This means that the phone was previously hacked to carrier unlock (unauthorized unlock), and when you restored, the hack was removed and it became re-locked to its original carrier. It will now accept a SIM card only from that carrier.
Your only option is to contact the carrier it's locked to and enquire if they will provide authorized unlocking. Not all carriers will unlock and those that do usually have requirements and qualifications - i.e. you have to be a current or former customer in good standing. -
My iphone (ios 7.1) is not showing the apple tv icon from my pull up control center? They are both running on the same wifi network... Any ideas on how to fix this problem?
Was it working before? Is your Apple TV enabled for airplay?
See
http://support.apple.com/kb/TS4215?viewlocale=en_US&locale=en_US -
I cannot get a dvd to play on my computer. The error message says in iDVD is "media type not supported".
The DVD was produced with a video camera and is in HD format. I think the the format is .m2ts?
Any ideas on how to get this to play?Check out > Mac App Store - Free MTS M2TS Converter
and/or others at > playing .m2ts format in OS X - Google Search -
i have purchased itunes match but the cloud will not load on my ipad 4.it works on my iphone 5 & it originally worked on my ipad.the only music that will show up is stuff that i bought from the itunes store.any ideas on how to get this to work?
On the MBP, in iTunes, switch to Song view and enable the iCloud Status column by pulling down View > View options and selecting the option for "iCloud Status." Close the small window. Look for any tracks that have a status of "waiting." If you see several of them disable iTunes Match while holding down the Option key. Quit iTunes, wait a few seconds, then open it again and turn iTM back on. Let it compelete the scan. Wait until all tracks that are "waiting" have a status of either "matched" or "uploaded."
On the iPad, turn off iTunes Match then launch the Music app and let the contents clear out. Power cycle the iPad for good measure, then re-enable the service once all the tracks in iTunes have been processed. -
I have duplicates of a song on an album. One of the duplicates is grey with no album cover. When you attempt to swite and delete, the track will not delete and will not even play. I've contacted iTunes store and the person I received the support email from told me that it may not be downloaded. I emailed him back and explained that no there is no download for that song in my itunes store on my phone.
One I never received an answer back which shows you how well these people do their job.
Two: I really would prefer to have a solution to this problem without having to delete all of my music from my phone or resetting my phone. Because this duplicate isn't even showing up in my iTunes, it's only showing up on my phone.
When I attempt to find it in my itunes it plays fine. But when I try to find the file to delete it, it is no where to be found. I've tried everything that I could think of to get rid of it. And I don't want to leave the album/artist off my phone because I do enjoy their music... And I have the microsoft sync in my car.
Anyone got any ideas on how to do this without resetting the phone and/or taking my music off?If you have all your music on the cmputer then i would connect up my phone uncheck the Sync Music option and apply.
This should take all the music off your phone .Then you can resync the music back on to it.
If this doesn't work maybe a restore to factory settings and start from scratch is the way to go -
My Iphone 5 won't show up in itunes on my computer? Any idea on how to fix this problem?
My Iphone 5 won't show up in itunes when i plug it into my windows 8 laptop but it shows up on my computer under devices and drivers as an iphone. Any idea on how to fix this problem so I can use my phone in itunes?
http://support.apple.com/kb/ts1538
-
I'm trying to connect my 30" Cinema Display to my new Mac Pro along with a new Apple 23'" monitor. The new Apple monitor is fine. On the cinema monitor everything is blown up pretty large. Any ideas on how to solve this?
The 30" display needs a DUAL-LINK adapter.
<http://store.apple.com/us/product/MB571Z/A/mini-displayport-to-dual-link-dvi-ada pter> -
After connecting my Apple TV this morning, I have no sound. It is hooked up to play through my television speakers. Any ideas on how to get sound? Also, I tried hooking it up through my surround receiver, but could not get it to come up on my ty screen, what can I do to correct this problem?
Have you tried adjusting the "Audio Out" and the "Dolby Digital" settings in settings > audio and video? It may be worth trying several combinations of these settings, although typically it is the "Dolby Digital" setting that causes most problems for users.
Also make sure that you have correctly mapped the input port on your sound system to the output, if you are unsure about how to do this, refer to the manual that came with your sound system.
Have you tried playing various content sources (i.e. podcasts, music, iTunes Store video, other video etc), do you have the same problem with all types of content? If you use optical you might also check that you can see a red light at the end of your optical audio cable. Is your optical audio cable plugged straight into your sound system, or does it go through a switch or other device? -
I am trying to play a DVD on my MacBook Pro and transmit it to my TV (using my TV as the monitor) utilizing a VGA cable and adapter for the Mac but the only image that shows up on the TV is the MacBook wall paper, any ideas on how I can transmit the movie from my MacBook to the TV? I also tried an HDMI cable and had the same issue.
you are probably seeing the extended display.. to see ur main display go to pefrences then display steeings and turn mirroring on.
this will display ur mbp main screen. good luck. -
When I switch my time capsule to 5.0ghz mode the range of my wireless network drops significantly. Near the Time Capsule my Macbook Pro works fine, in my bedroom I have almost no signal. In mixed mode it works fine both placed. Any ideas on how to get 5.0 working? I thought 5.0Ghz was supposed to have a wider range. Do I need to change a setting on the Macbook Pro?
Thanks, WI'm not sure if you understand that 5 GHz has some upsides, but also some very significant downsides.
The upside is that the 5 GHz signal will allow faster speeds.
The downside is that you must always give up something in order to gain something else.Unfortunately, the marketing hype about 5 GHz does not go into very many details about this.
In the case of 5 GHz, the 5 GHz signals are much weaker than 2.4 GHz signals, so they do not travel as far or penetrate any obstructions like walls or ceilings nearly as well as 2.4 GHz signals. In many cases, in order to get good 5 Ghz performance, you literally have to have almost a line-of-sight relationship between your computer and the wireless router.
If you have a new dual band router, which broadcasts both a 2.4 GHz and 5 GHz signal simutaneously, your Mac will connect to the best signal quality automatically depending on its location in relation to the router. In the same room, without any obstructions, it will connec to 5 GHz. But, take the laptop down the hall a few rooms and around the corner and the Mac will connect to the 2.4 GHz signal because at that location, the 2.4 GHz signal is not only stronger, it is also faster. -
I have a new Mac Mini and I found AdWare called TakeOverSearchAssetsMac and SearchProjects on it. I have been unable to remove it. Does anyone have any ideas on how to remedy this?
Please review the options below to determine which method is best to remove the Adware installed on your computer.
The Easy, safe, effective method:
http://www.adwaremedic.com/index.php
If you are comfortable doing manual file removals use the somewhat more difficult method:
http://support.apple.com/en-us/HT203987
Also read the articles below to be more prepared for the next time there is an issue on your computer.
https://discussions.apple.com/docs/DOC-7471
https://discussions.apple.com/docs/DOC-8071 -
I am trying to record a meditation from my Macbook Air and an annoying loud sound, like a burp, will suddenly erupt from the MacBook, necessitating I start again. Any ideas on how I may stop this, or what it is??
Click on the blue Internet Recovery in nbar's post. That is a link to what computers can run Internet Recovery.
Do a backup, preferable 2 separate ones on 2 drives. Boot to the Recovery Volume (command - R on a restart or hold down the option/alt key during a restart and select Recovery Volume). Run Disk Utility Verify/Repair and Repair Permissions until you get no errors. Reformat the drive using Disk Utility/Erase Mac OS Extended (Journaled), then click the Option button and select GUID. Then re-install the OS.
OS X Recovery
OS X Recovery (2)
When you reboot, use Setup Assistant to restore your data.
Maybe you are looking for
-
I downloaded firefox 4.0 and attempted to use business objects. A screen comes up saying that a plugin is required but when I click on the screen, it say's no plugin is found. I am using the latest version of Java but it seems that the plugin is not
-
Updating TOC erases all document text
Hello all. I am working with a document in cs4 that I placed in from Word. All the heading styles were set in InDesign. There are graphics and text. I inserted a TOC on the first page early on. I then went to Update the TOC, and it asked if I wanted
-
Weird basic authentication behavior on safari
I'm a PHP developer, and testing web app compatibility among browsers. I found that safari behaves differently from other major browsers(IE,firefox,chrome) when it comes to basic authentication. for example, i have a http://asdf.com/a.php, that is pr
-
Hi All I know this question has been asked before. I've done the searches and still can't get the attached vi to operate correctly Would one of the LV knights look at the vi and tell me why the two constants on the BD (values 104 and 360) are not sho
-
J2sdk1_3_1_07 not running on up-to-date patched Solaris 8
I have installed all public Solaris 8 patches as well as all public J2SE patches for 1.4.1_02 (should make stable for 1.3.1_07) and the bin/java program won't launch on my machine. The 1.4.1_02 bin/java will launch. - G