Jens Krämer

Automatic mirroring of Gitolite repositories to Github

 |  gitolite, automation, sysadmin, linux, git  |  2 comments

I’m a big fan of self-hosting my stuff. That’s why all the Git repositories where I keep the code I’m writing and any other things I want to have versioned live in a Gitolite instance on a rented server.

However for various reasons (not having to provide public git access on my own server, Travis CI, ease of collaboration via pull requests) it still makes sense to mirror the open source stuff on Github.

Obviously it’s cumbersome to manually push to two remotes all the time, so I set out to automate this. Here’s how.

On the Gitolite server

Configure Gitolite

There’s a config file named .gitolite.rc in /home/git (or whatever your Gitolite user’s home dir is). Here you need to set the LOCAL_CODE variable (search for it in th file, there’s two suggested settings already and you just need to pick one):

/home/git/.gitolite.rc
# suggested locations for site-local gitolite code (see cust.html)

# this one is managed directly on the server
# LOCAL_CODE                =>  "$ENV{HOME}/local",

# or you can use this, which lets you put everything in a subdirectory
# called "local" in your gitolite-admin repo.  For a SECURITY WARNING
# on this, see http://gitolite.com/gitolite/cust.html#pushcode
LOCAL_CODE                =>  "$rc{GL_ADMIN_BASE}/local",

# remove the no-agent-forwarding restriction which is present by default
AUTH_OPTIONS => 'no-port-forwarding,no-X11-forwarding,no-pty',

Regarding the LOCAL_CODE there’s two possiblities - you can either manage the custom code directly on the server or through the gitolite-admin repository. As you can see the latter is the route I went with.

The AUTH_OPTIONS line originally wasn’t present in my config file. It determines what ssh options are set for keys managed by gitolite in /home/git/.ssh/authorized_keys and defaults to no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty. We remove the part disabling agent forwarding so our hook script can use the ssh keys from your local machine to connect to Github.

Also make sure the repo-specific-hooks are enabled in the ENABLE array further down in the same file. Thanks to Lucas Camargo who pointed that out in his comment!

After making these modifications you have to update the gitolite installation:

# su - git
$ gitolite compile
$ gitolite trigger POST_COMPILE

In your gitolite-admin repo

Add a hook script

Create a directory local/hooks/repo-specific and in there a simple shell script that mirrors a given repository to github:

gitolite-admin/local/hooks/repo-specific/github-mirror`
git push --mirror git@github.com:jkraemer/$GL_REPO.git

Don’t forget to replace jkraemer with your own Github username . If you set the LOCAL_CODE variable above to something outside the gitolite-admin repo, you have to create the hook script in that location.

Configure which repositories to mirror

I have my gitolite config organized by declaring variables (groups of repositories and users) on the top, and rules using these variables further down below. This way it’s easy to make changes without having to hunt around for the right places.

So for the mirroring, I have a group of repositories to mirror declared at the top, and a rule setting the mirror script for these repositories later on:

gitolite-admin/conf/gitolite.conf
# repositories to mirror to github
@mirrored-repos = some-repo another-repo

# other variables and configuration...

repo @mirrored-repos
  option hook.post-update = github-mirror

Commit all changes (don’t forget to git add the hook script) and push them to your server.

That’s it! Now everytime something is pushed to a repository in your @mirrored-repos list, the hook script will be called and do a push to github.

Comments

Lucas Camargo

You forgot to tell us all to uncomment the "repo-specific-hooks" enable string on the gitolite rc file. Other than tha, super useful!

Here is a simple hook script that uses a github API token to create the repo if it does not exist yet:

!/bin/bash

echo "Mirroring ${GL_REPO} to GitHub" echo "It will go into url $URL"

USERNAME=github-username TOKEN=cat /home/git/github-api-token GITHUB_REPO=${USERNAME}/${GL_REPO//"prefix/"/""} GITHUB_REPO_SHORT=${GL_REPO//"prefix/"/""} URL=git@github:${GITHUB_REPO}.git

curl=$(curl --fail https://api.github.com/repos/${GITHUB_REPO}) if [ $? -ne 0 ]; then echo "Failed to get repo info at ${GITHUB_REPO}, trying to create it" DESCRIPTION=cat description curl -i -H "Authorization: token ${TOKEN}" \ -d "{ \ \"name\": \"${GITHUB_REPO_SHORT}\", \ \"description\": \"${DESCRIPTION}\", \ \"auto_init\": false, \ \"private\": false \ }" \ https://api.github.com/user/repos fi

git push --mirror ${URL}

remember to change github-username

# a file at the location /home/git/github-api-token has to exist (or maybe just put the token there) # if there is no "prefix/" substring on the repo name to remove, just change ${GL_REPO//"prefix/"/""} for ${GL_REPO} (empty string) # otherwise, change the prefix accordingly

Jens

Thanks, I added that info to the article! Nice idea about automatically creating the Github Repositories.

You can use Markdown here.

For the sake of spam checking any data you submit, including your IP address, will be transferred to the US based Akismet web service (akismet.com). If that's not acceptable for you, you can also reach me by other means.