Hosting a Gentoo Layman repository on Amazon S3

I’ve recently had to create a lot of custom Gentoo ebuilds, and distribute them out to separate machines. Luckily Gentoo has the layman system for adding experimental overlays, so we can use this to add our own custom one, hosted on Amazon S3.

Setting up the repository is a two part process – creating the actual repository, and enabling it on the machine(s) it needs to be setup on:

Creating The Repository

The first part to creating a S3 hosted repository is creating an S3 bucket. You can create an S3 bucket using the Amazon AWS Management Console.

Now to create the actual repository; this is as simple as changing directory into your repository tree, and running:

tar cjf /tmp/overlay.tar.bz2 *

You can then upload this file into your S3 bucket. Once it is uploaded, find the URL and ensure it is publicly accessible.

The next step is to create a repository XML file. This file lists the repositories for layman, so you can use this single XML file for multiple repositories. This is an example of the XML file, you will need to replace the example data with your own. Most of the parameters should be obvious, the only non-obvious thing is the source line, which will be the location of the tar file we have previously uploaded:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE repositories SYSTEM "/dtd/repositories.dtd">
<repositories xmlns="" version="1.0">
 <repo quality="experimental" status="unofficial">
 <name><![CDATA[overlay name]]></name>
 <description><![CDATA[overlay description]]></description>
 <homepage>http://www.example.com</homepage>
 <owner>
 <email>Email</email>
 <name><![CDATA[Your Name]]></name>
 </owner>
 <source type="tar">http://reponame.s3.amazonaws.com/overlay.tar.bz2</source>
 </repo>
</repositories>

Upload it to S3, and take a copy of the path to this file. We will need it when setting it up on the Gentoo machines we are adding it to. Our repository is now ready for use.

Adding the repository to each machine

Obviously the first step is to install layman :

emerge -va layman

We should now have a config file in /etc/layman/layman.cfg , in this file should be a section called ‘overlays’, simply add your repository XML path, so the overlay section looks something like this:

overlays  : http://www.gentoo.org/proj/en/overlays/repositories.xml
            http://repository.s3.amazonaws.com/repositories.xml

Have layman update it’s repositories with our new repo, and add our repository by name:

layman -f
layman -a repository

Make sure your /etc/make.conf file has this line at the end, and you will be able to install packages from your own repository:

source /var/lib/layman/make.conf

If you are using eix , make sure to run eix-update , so you can search your packages using eix:

eix-update