Local-repo ~ a local repository manager with AUR support

Hey people,
i want to announce the release of local-repo 1.6.2. I made it to easily build and manage my own repos with fancy features like:
caching & logging
config file support
AUR support - add & upgrade packages from the AUR
an ABS like PKGBUILD store
a buildlog store
package and database signing
localization
Maybe this is something for you, too.
Visit the projects website [1], if you want to know more about it.
The PKGBUILD is available in the AUR [2].
Much fun, ushi
[1] http://ushi.wurstcase.net/local-repo/
[2] https://aur.archlinux.org/packages.php?ID=56876
Edit:
The first run should be something like this:
$ mkdir testrepo && local-repo testrepo -A vim-html5-git
This will create a new repo in the directory testrepo and add vim-html5-git from the AUR.
Last edited by ushi (2012-03-31 12:04:40)

Awesome! Looks like what I need, let's see if I can get it to work.

Similar Messages

How to develop repository manager with initial folder structure

Hi 2 all!
I wonder if it is possible to create new repository programmatically with defined folder structure. I'm interested in solutions when repository manager is works with db or file system. Can you provide any example or tell me where can I define my structure?
thnx

Hi Nata!
Basicly this task consists of two steps:
1. Deploy a new configuration object for the repository manager
2. Create the intial folder structure
Step one is simple. Here is the configuration object of the repository manager for /documents. It's been exported from the KM configuration by using the standard export mechanisms. This can be included into a portal application project in Netweaver Studio.
documents.co.xml
<?xml version="1.0" encoding="UTF-8" ?>

<Configurable configclass="CmRepositoryManager" owner="km.appl.rep.mgr.prjconfig">
<property name="ro-expiry-time" value="0"></property>
<property name="fsconttrack" value="true"></property>
<property name="poolid" value="dbcon_wcmdb_cm"></property>
<property name="securitymgr.aclcacheid" value="ca_rsrc_acl"></property>
<property name="localroot"></property>
<property name="serverfarm" value="false"></property>
<property name="propertysearchmgr.class" value="com.sapportals.wcm.repository.manager.cm.CmPropertySearchManager"></property>
<property name="keepVersionHistory" value="false"></property>
<property name="securitymgr.aclcfg" value="acl_jdbc_rep"></property>
<property name="versioning.localroot"></property>
<property name="persistence.mode" value="db"></property>
<property name="hidden" value="false"></property>
<property name="fsnosync" value="false"></property>
<property name="cacheid.versionhistory"></property>
<property name="prefix" value="/documents"></property>
<property name="fsContentCacheMaxEntries" value="-1"></property>
<property name="fsContentCache"></property>
<property name="name" value="documents"></property>
<property name="cacheid.content"></property>
<property name="compressLimit" value="-1"></property>
<property name="cacheid.foldercontent"></property>
<property name="cacheid.properties"></property>
<property name="active" value="true"></property>
<property name="cacheid.rid"></property>
<property name="ignorerootdisplayname" value="false"></property>
<property name="class" value="com.sapportals.wcm.repository.manager.cm.CmRepositoryManager"></property>
<property name="cclist"></property>
<property name="intLinksDefaultDynamic" value="true"></property>
<property name="contentmgr.class" value="com.sapportals.wcm.repository.manager.cm.CmNamespaceAndContentManager"></property>
<property name="cacheid.documents"></property>
<property name="versioningmgr.class" value="com.sapportals.wcm.repository.manager.cm.CmVersioningManager"></property>
<property name="securitymgr.w2kcfg.systemid"></property>
<property name="lockmgr.class" value="com.sapportals.wcm.repository.manager.cm.CmLockManager"></property>
<property name="cacheid.locks"></property>
<property name="securitymgr.class"></property>
<property name="persistence.db.instance-id" value="documents"></property>
<property name="cacheid" value="ca_cm"></property>
<property name="namespacemgr.class" value="com.sapportals.wcm.repository.manager.cm.CmNamespaceAndContentManager"></property>
<property name="autoversioning" value="false"></property>
<property name="description"></property>
<property name="securitymgr.ref" value="AclSecurityManager"></property>
<property name="fsContentCacheLimit" value="100"></property>
<property name="propertymgr.class" value="com.sapportals.wcm.repository.manager.cm.CmPropertyManager"></property>
<property name="services" value="accessstatistic, com.dkv.DKVApprovalAndTranslation, com.dkv.jobtool.repository.JobToolCreationService, comment, discussion, feedback, layout, personalnote, properties, rating, statemngt, subscription, svc_acl, tbp"></property>
<property name="sendevents" value="true"></property>
</Configurable>
This is a DB repository. The config object for a filesystem repository might differ.
Step 2, the creation of the initial file structure may be done with in two different ways:
a) Include a Repository Service with your repository, which creates the folder structure on the first access of your repository.
b) Include a Portal app which has to be called once to create the folder structure.
For more info about using the API to create km folders take at look at this:
https://www.sdn.sap.com/irj/sdn/thread?threadID=933638&messageID=5663711#5663711
Cheers,
Jürgen

Can't install packages from local repository created with repo-add

I did the following
repo-add /mnt/packages/repo.db.tar.gz /var/cache/pacman/pkg/*
cp /var/cache/pacman/pkg/* /mnt/packages
on one computer
connected the hard drive to another computer and did the following:
added the pacman.conf configurations as follows
[repo]
SigLevel = PackageRequired
Server = file:///mnt/packages
then ran
pacman -Sy
which worked.
and then I tried to install a package using pacman -S.but all the packages give me :
error: packagename: missing required signature,File:///var/cache/pacman/pkg/packagename is corrupted(invalid or corrupted package(PGP Signature))
then i tried:
pacman-key --init
pacman-key --populate archlinux
pacman -Syy
which didn't work and made no difference.what can I do?

anatolik wrote:
One thing that comes to my mind is that repo directories have *.sig files next to the packages, see file structure here http://mirrors.kernel.org/archlinux/testing/os/x86_64/
But cache directory /var/cache/pacman/pkg/ does not have these sign files and it is why pacman complains. Where to find the sign files locally? I do not know the recommended way, but I see that /var/lib/pacman/sync/$REPO.db contains signature information, so you can write a script that extracts it.
Or you can use `repo-add --sign` and sign packages with your key, but it is not very secure.
PS Instead of using repo-add you might want just use repo files from /var/lib/pacman/sync/
I will try the above and report.
but it shouldn't be necessary according to https://wiki.archlinux.org/index.php/Pacman_tips

CM Repository manager in FSDB mode

Hi ,
I am creating a CM repository manager with persistent mode as FSDB.I have created a network path and given a username which is having full control in the remote machine.
But when I am checking in component monitor it is giving the folowing error and the repository manager fails to start.
Startup Error: Error occured while retrieving canonical name for folder '
172.17.1.249\Portal_KM_Store\Data' (java.io.IOException: Logon failure: unknown user name or bad password
Please help
Regards
Nayeem

You might have a look at
<a href="http://help.sap.com/saphelp_nw2004s/helpdata/en/e3/92322ab24e11d5993800508b6b8b11/frameset.htm">this documentation</a>.
The CM repository in mode FSDB keeps the namespace structure of the repository in the database just as it does in the other modes. Any change is done parallel in the file system and in the internal database.
Since we have mode FSDB and the file system might change at any time by external processes, there is a component called synchronizer that checks with every lookup if the file system and the internal structure are still in sync. To determine this it looks at the modification date for folders and at modification date and size for plain files.
If any of these do not match then the synchronizer overwrites and modifies the internal data to reflect again the status of the external file system.
Now in your portal the synchronizer itself is working OK. That's because your repositories run fine locally.
With your remote file systems I can think of some possible explanations.
The caches might deliver responses after immediate file system changes. We ruled out this one already.
The synchronizer might not recognize the changes in the file system's meta data. This is why I'd like you to try and compare the behaviour of the file system repository, see above.
The synchonizer might run into some error when it tries to access the remote file system. This should be visible in the logs.
Michael

CM Repository manager in FSDB mode failed to start

Hi ,
I am creating a CM repository manager with persistent mode as FSDB.I have created a network path to access a remote server which will act as the file system repository.I am giving a user which has full control in that machine.
When I am checking in the component monitor the repository manager is failing to start and gives the following error message.
Startup Error: Error occured while retrieving canonical name for folder '(java.io.IOException: Logon failure: unknown user name or bad password
Please help.
Regards
Nayeem

You might have a look at
<a href="http://help.sap.com/saphelp_nw2004s/helpdata/en/e3/92322ab24e11d5993800508b6b8b11/frameset.htm">this documentation</a>.
The CM repository in mode FSDB keeps the namespace structure of the repository in the database just as it does in the other modes. Any change is done parallel in the file system and in the internal database.
Since we have mode FSDB and the file system might change at any time by external processes, there is a component called synchronizer that checks with every lookup if the file system and the internal structure are still in sync. To determine this it looks at the modification date for folders and at modification date and size for plain files.
If any of these do not match then the synchronizer overwrites and modifies the internal data to reflect again the status of the external file system.
Now in your portal the synchronizer itself is working OK. That's because your repositories run fine locally.
With your remote file systems I can think of some possible explanations.
The caches might deliver responses after immediate file system changes. We ruled out this one already.
The synchronizer might not recognize the changes in the file system's meta data. This is why I'd like you to try and compare the behaviour of the file system repository, see above.
The synchonizer might run into some error when it tries to access the remote file system. This should be visible in the logs.
Michael

CM Repository manager is not appearing for other users

Hi Friends,
I have configured CM repository manager with the user having super admin role.
After configuring it am able to see the KM content folder in the Content Administration->KM Content only with the user from where i have configured it. But if i login with the other non-super users the KM content folder is not appearing.
when i open the permissions of the KM content folder through the details of the context menu am getting as below.
child permissions
read permissions
write permissions
parent permissions etc
where all above are in display mode. So with this i cannt give read or write permission to any other user.
Could anyone please help me out to make it visible to all users.
I have activated all repository services while creating CM repository manager.
Is there any other permissions to be set to make KM folder visible to all users or do i need to any other repository service?
Regards
Sireesha.

Hi Lakshmi,
Content manager role has been assigned to other users.
Regards
Sireesha.

How to control permissions in KM using File System Repository Manager

Hello everybody
I have a scenario with 15+ file servers which content needs to be mapped into Portal KM as read-only.
Windows Authentication is configured on the portal, so the users don't need to supply the portal with username/password.
Now the users are of course only allowed to see the same content in the portal as they can see on the file servers. I know that the permission mapping can be done using a File System Repository Manager with W2KSecurityManager, but in that case the users will have to maintain the mapping of rights from windows to portal manually. And with ever-changing passwords this is too big an administrative task. So I am looking for another solution.
My question is therefore:
How can you configure the File System Repository Manager in a way where it the user does not need to maintain "user mapping" manually, but only need to logon to the portal (through windows authentication) and can see the same files from the portal as he/she can see from the file servers.
Can this information in some way be taken from the configured Windows Authentication? Or is the only solution to enable WebDav on the file servers and and follow the "Integration of Windows File Servers into the SAP KM platform Using SSO and the WebDAV Repository Manager"-guide?
Any help will be rewarded.
Best regards,
Martin Søgaard

Hi again Dirk
I just studied the "Integration of Windows File Servers into the SAP KM platform using SSO and the WebDAV repository manager"-guide in depth and I don't quite understand what rights the cmadmin_service and notificator_service are supposed to be given in the Active Directory. In the guide it says:
"A number of global services and repository services need to perform operations on resources in various repositories. For example, the index management service must read all resources it is meant to index. When an index is created, the service user index_service is used. Therefore, this user has to have at least read access to all repositories that are to be indexed. Other users that are used by the SAP KM platform are the service users notificator_service and cmadmin_service. When a search is taking place, it is the user that is currently logged on to the portal."
I dont quite understand what the two service users are doing but more importantly, I need to give the two service users the correct permissions in the Active Directory (or through user mapping). I understand that index_service only needs read_only permissions, but what permissions should the cmadmin_service and notificator_service be given?
Best regards,
Martin

Local repository with dependencies from aur

I have local repository and I some packages there. For one package I want to specify dependence from AUR package, but when I try to install it I receive warning:
warning: cannot resolve "ruby-compass", a dependency of "local-company"
Is it possible to ask yaourt check dependencies in AUR when installing package from local repository?

Nope. Many "AUR helpers" do this though. In my limited experience yaourt seems to be one that handles this seemlessly, but I'd say it comes with some costs of hiding what is doing from the user.

System update configuration issue with local repository related to machine type

Hi,
Sorry if the issue has been posted earlier, i tried searching but cudnt find relevant post. Here is my issue:
My machine is Thinkpad R52. Type as printed on label is: 1858-CLG
While adding machine type in update retriever, I added machine type as 1858 (as it doesnt accept 1858CLG). After connecting to the support site, it downloaded all required updates onto a local(shared) repository.
I installed system update, configured it to pickup update from local (network) repository in the registry.
Now when I launch system update, it determines my machine type as 1858CLG & it connects to local repository only to display message as "No updates were found for this machine type"
Though I cud see updates if I directly connect to help center.
I think the problem is with machine type identification, not sure though.
Please advice on what I'm doing wrong. Is it with update retriever configuration or system update configuration.
Thanks much,

Solution for me was to change status of packages from "Test" to "Active" using Update Retriever.
Since this is not documented in Best Practices (but is stated in chapter 5 - modify settings) I spent several hours figuring it out..
-brian

Repository Manager - Edit Locally / Edit Online

Hi All,
Repository Manager gives the facility to edit the document through Edit Online and Edit Locally, can anybody suggest me how can I achieve this using RF client API.
Thanks,
Sharad

This is covered in the NetWeaver documentation at help.sap.com. Please try searching before posting questions.

Oracle VM 3.1.1 and local repository

Hi there,
I have created one LUN hardware RAID 10 for 4x300G SAS disks, and then installed Oracle VM Server 3.1.1, during the installation I determined the remaining space on sda for Oracle VM repository on partition sda2 mounted on directory /OVS OCFS2 file system, installation completed successfully, and then I installed Oracle VM Manager 3.1.1 on a separate machine it is installed and working fine.
Now, Oracle VM Manager discover the Oracle VM server smoothly but unable to detect that remaining space on /OVS, my question how to use this remaining space for Oracle VM repository as I read on the new release document "Release Notes" : said that this issue is fixed with this release and mentioned it as bug in their previous version. I wish if any ideas or workaround to fix this issue as I always hope, there is a manual intervention to solve this issue and want at the same time to keep performance benefit of RIAD 10 configuration.
Thanks in advance,

Sorry, you get to do it over.
You must install the Local Repository on an unused, complete, unpartitioned LUN of it's own. Build about a 50G LUN for Oracle VM and and additional LUN for the remaining terabyte. OVM will then see the space and utilize the repo. Create a non-clustered Server Pool.
You can put OVM in much smaller space, but I like to have "extra" (add a /dev/sda4 and mkfs on it) stuff on my OS disks. If you don't want to do that, OVM fits nicely in 5G. I"d give it 10G for growth with later upgrades, however.

Best way to upgrade the local repository from 11.7.3 to 12.2.3.3 DS 3

Hi, we are going to upgrade our DI 11.7.3 to DS3 - 12.2.3.3 in Sep. I found out there are different ways to upgrade the local repository from the SAP doc and tech support. I try to find a best and safe way to do it. If anyone had experince and inputs on this. It would be great appreciated. It seems there are these options:
we all agree that need to back up the repostiry db, and export the repository to atl file.
Option 1:
Upgrade the repository during the Data Services installation. But the tech support said there are times where this does not work as expected and Option 2 below needs to be implemented.
Option 2:
a) Export the local repository, via 11.7.3 Designer, to an ATL file.
b) Uninstall 11.7.3.
c) Create a new database in SQL Server to use with 12.2.3.3. (Optionally, use the existing 11.7.3 database. If you do, make a backup of it before making any changes).
d) Install Data Services 12.2.3.3 and reference the database created in step "c" above during installation (select the Create option).
e) Log in to the new local repo, via 12.2.3.3 Designer, and import the ATL that was exported in step "a" above.
Option 3:
a. Create a new database u2013 20 MB space
b.Create a new repo in 11.7.3 point to ds_local via Repository Manager (click create)
c.Log in 11.7 designer and Repository import from out.atl
d.Install 12.2.3.3 without upgrde the repostory
e.use epository manager in 12.2.3 to upgrade the new repository

Hi.
If you import the 11.7 atl into a 12.2 Designer you will face a warning message like this:
"The version 11.7 of the ATL file <c:\export.atl> is lower than the repository version <12.2>.
To update an ATL file, import it into a repository of the same version then ugrade that repository"
So, the correct method to upgrade the Data Integrator 11.7 repository to Data Services 12.2.x is the following:
- Export the complete Data Integrator 11.7 repository to ATL.
- Create a NEW Data Integrator 11.7 Local Repository.
- Import the ATL created in step 1 to the NEW Data Integrator 11.7 Repository. At this point you have two identical local repositories.
- From the Data Services 12.2.x system, open Repository Manager.
- Enter the Database connection information to the NEW repository created in step 2.
- Select UPGRADE, to upgrade this repository to support Data Services 12.2.x.
- Log into Data Services Designer to start using this Local Repository.
Regards,
Leo.

[SOLVED]Make a local repository

I wonder if there is a way to make an archlinux repository for AUR packages, I think it'll be a lot easier to share these packages to other "Archers" with packages available in local repository like /var/cache/pacman/pkg
Can anyone tell me how to make it, because i cannot find it anywhere so probably with asking in the forum i will find the answer.
Thanks to you guys!
Last edited by ch1pz (2010-04-08 14:29:11)

first what you call AUR packages are not in fact binary packages but rather builds for making those packages. this means that is no sense for doing a repository.
if you want to redistribute some of the builds out there, the tool for creating a repository is repo-add which is distribute with pacman.
repo-add repo_name.db.tar.gz package-pkgver-pkgrel.pkg.tar.gz

[SOLVED] Create local repository

Hello everyone!
First of all, I'd like to apologize for the terminology. I'm still not sure about what are the names I should use. That said, here's my problem: I want to set up a local repository to place all the packages I've ever built with makepkg. The main reason that has motivated me to do so is that, when building a package, I often need to install make dependencies, such as, for example, xorg-server-devel. That package in particular comes from xorg-server, which I have built by myself. Thus, if a package required it, I should install the correspondent package (the one I've built), and not the generic one coming from the repositories (this case may not be a good example, but, well, just think about kernel headers).
In order to set up the repository, I read Allan McRae's articles about makepkg, repo-add and package signing. I generated a key with GPG as explained here, taking the defaults, and added it to the pacman's keyring:
gpg --export Kalrish > Kalrish.key
pacman-key -a Kalrish.key
I also edited /etc/makepkg.conf as follows:
BUILDENV=(fakeroot !distcc color !ccache check sign)
Note that I've enabled sign.
I decided to maintain my "local repository" (please, correct me if I'm not making use of the correct terminology) in /var/abs/local. For that purpose, I edited /etc/makepkg.conf:
$ grep -m1 PKGDEST /etc/makepkg.conf
PKGDEST=/var/abs/local
I then built dhcpcd (which is relatively small), just to try out. makepkg put the package file in /var/abs/local as I intended; then, I did this:
repo-add 'local.db.tar.gz' 'dhcpcd-5.6.8-3-x86_64.pkg.tar.xz'
It asked me for the password to unlock my secret key. After entering it, all went well:
==> Adding package 'dhcpcd-5.6.8-3-x86_64.pkg.tar.xz'
-> Adding package signature...
-> Computing checksums...
-> Creating 'desc' db entry...
-> Creating 'depends' db entry...
==> Creating updated database file 'local.db.tar.gz'
And finally added the repo in /etc/pacman.conf (I'm only showing the relevant part of the file):
[local]
Server = file:///var/abs/local
I updated the package databases just fine with pacman -Syy. But, after removing the previously-downloaded packages from /var/cache/pacman/pkg, when trying to install the package:
$ pacman -Sy dhcpcd
:: Synchronizing package databases...
local is up to date
core is up to date
extra is up to date
community is up to date
multilib is up to date
:: dhcpcd is in IgnorePkg/IgnoreGroup. Install anyway? [Y/n] y
warning: dhcpcd-5.6.8-3 is up to date -- reinstalling
resolving dependencies...
looking for inter-conflicts...
Packages (1): dhcpcd-5.6.8-3
Total Installed Size: 0.23 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring [######################] 100%
(1/1) checking package integrity [######################] 100%
error: dhcpcd: signature from "Kalrish (Kalrish Bäakjen) <[email protected]>" is invalid
:: File /var/cache/pacman/pkg/dhcpcd-5.6.8-3-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] n
error: failed to commit transaction (invalid or corrupted package (PGP signature))
Errors occurred, no packages were upgraded.
So it seems that I have not added my key properly. Can anybody please help me? Any answer, even if it explained why I shouldn't be trying to do this or why I should try other alternatives, will be welcomed.
Best regards,
Kalrish
Last edited by Kalrish (2013-06-28 18:26:30)

drcouzelis wrote:... if you built the package on that machine yourself then all of the dependencies are already installed.
I was talking about make dependencies, not runtime dependencies. I always build with makepkg -sr :)
drcouzelis wrote:I hope I'm not being too critical. I just don't want you to work harder than you need to. :)
Of course not, don't worry.
Kalrish wrote:Any answer, even if it explained why I shouldn't be trying to do this or why I should try other alternatives, will be welcomed.
I love thought-provoking answers as well as personal opinions. Who knows when an unexpected explanation may change my mind and make me think different? You're right: it may not be worth it. But, well, you yourself pointed the "elegant" reason out:
drcouzelis wrote:I think the only benefit of creating a local repository is so you can use "pacman -S" instead of "pacman -U".
If pacman found the packages by itself, I wouldn't have to execute pacman -U --asdeps for every make dependency that is required by a package - but rather execute makepkg -sr and simply let pacman pick those packages I've built by myself from my local repository, and the rest from the official repos.

OVM 3.0.1 local repository problem

Good morning all, i am really new in OVM and i am facing a big issue that stops me evaluating this product.
I have a couple of servers, connected to a S.A.N. array. I can see from both the servers i added to a clustered pool, and i am able to create a shared repository without problems.
I am not able to see local disks in the OVM manager administration and therefore i can't create local repositories. I tried all i found in this forum, but without success.
Let's focus on server1: it has a couple of 146GB disks. I used one of them for OVS installation leaving the second disk alone, without partitioning it.
Tried to create local repository in the clustered pool, but no way...
So i created a single full-disk partition and retried to create repo: still no way
Then i created an ocfs2 filesystem in the new partition but, again, i couldnt see physical local server1 disk.
Every time i changed partitions configuration, i obviously did rescanning of physical disks.
I all my tests, local physical disks selection list in Generic Local Storage Array @ node1 is always empty.
Any hint about solving this issue? Any good pointer to an hands-on guide (official docs are not so good)? Any suggestion about what to look at in log files for debugging?
Any answer is welcome...
Thank you all!

I was able to do this as follows
1. have an untouched unformatted disk (no partitions, no file system)
2. in hardware under the vmserver name , scan for the disk and it should show in the list
3. in the repository section of home, add the repository as physical disk
4. "present" (green up and down arrows) the physical disk on the vmserver itself (dont ask me why you have to do this but if you dont it wont find its own disk)

Local-repo ~ a local repository manager with AUR support

Similar Messages

Maybe you are looking for