Posts Tagged ‘ subversion ’

experiments w/ upstream tracking and git

WordPress-MU uses subversion for their code repository, but I’m starting to become a bit of a git-addict. I’ll outline the steps I’ve taken to keep track of upstream while maintaining my (hopefully minimal) fedora customizations. Brain-patches and better methods cheerfully accepted!

Step 1: Create a remote repo

You don’t really need to do this, but tend to prefer having some off-site backup of my work… many a drive has failed for me in the past 😉

You’ve got a number of options for hosting sites; I’ll highlight katzj’s work w/ fedorapeople.org, as well as point folks to fedorahosted.org.

For now, I’m using fedorapeople.org, and here’s what the setup looked like:

[bretm@koom ~]$ ssh fedorapeople.org
Last login: Fri Sep  5 14:30:11 2008 from somewhere.net
[snip]
This system is puppet managed!  Local changes may be overwritten:
http://fedoraproject.org/wiki/Infrastructure/SOP/Puppet

[bretm@people1 ~]$ mkdir -p public_git/foo.git ; cd public_git/foo.git
[bretm@people1 foo.git]$ git --bare init
Initialized empty Git repository in /home/fedora/bretm/public_git/foo.git/
[bretm@people1 foo.git]$ exit

fedorapeople.org note: the .git segment of foo.git is important, you need that or gitdaemon won’t find your repo

That’s it, you’ve created a bare, off-site repo. Remember that, in git, bare repos contain only diffs, and binary blobs, and nothing else. We’ll need some place to actually do our work…

Step 2: Create a local repo

Head to your working directory of whatever upstream source that you need to track & package. I’ll just create a quick text file and we can use that for an example:

[bretm@koom ~]$ mkdir -p devel/git/foo; cd devel/git/foo
[bretm@koom foo]$ echo "some stuff to track" > README
[bretm@koom foo]$ git add README
[bretm@koom foo]$ git commit -m "initial commit" -a
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README
[bretm@koom foo]$ git branch
* master

So now we have a local repo, w/ a branch named master with a little bit of content. We’re now going to shove that into the remote repo with the branch name upstream:

[bretm@koom foo]$ git remote add fedorapeople git+ssh://fedorapeople.org/~bretm/public_git/foo.git
[bretm@koom foo]$ git push git+ssh://fedorapeople.org/~bretm/public_git/foo.git master:refs/heads/upstream
Counting objects: 3, done.
Writing objects: 100% (3/3), 230 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git+ssh://fedorapeople.org/~bretm/public_git/foo.git
 * [new branch]      master -> upstream
[bretm@koom foo]$ git remote show fedorapeople
* remote fedorapeople
  URL: git+ssh://fedorapeople.org/~bretm/public_git/foo.git
  New remote branches (next fetch will store in remotes/fedorapeople)
    upstream
[bretm@koom foo]$ git fetch fedorapeople
From git+ssh://fedorapeople.org/~bretm/public_git/foo
 * [new branch]      upstream   -> fedorapeople/upstream
[bretm@koom foo]$ git branch -a
* master
  fedorapeople/upstream

The git fetch fedorapeople is important; without that, you would only see your local master branch, and not the remote fedorapeople/upstream one. So, now we have a remote branch with which to track upstream’s trunk, but we also want one to hold the Fedora customizations:

[bretm@koom foo]$ git push git+ssh://fedorapeople.org/~bretm/public_git/foo.git master:refs/heads/fedora
Total 0 (delta 0), reused 0 (delta 0)
To git+ssh://fedorapeople.org/~bretm/public_git/foo.git
 * [new branch]      master -> fedora
[bretm@koom foo]$ git fetch fedorapeople
From git+ssh://fedorapeople.org/~bretm/public_git/foo
 * [new branch]      fedora     -> fedorapeople/fedora
[bretm@koom foo]$ git branch -a
* master
  fedorapeople/fedora
  fedorapeople/upstream

Now comes the interesting part… interacting w/ upstream’s svn repo.

Step 3: Use git-svn to fetch upstream’s Subversion repository

The general workflow for maintaining the Fedora package will look something along the lines of:

  1. fetch updates from upstream
  2. apply updates to fedorapeople/upstream branch
  3. rebase the fedorapeople/fedora branch upon fedorapeople/upstream
  4. roll a new srpm and build through koji

To make our lives easier, we’re going to use git-svn.

[bretm@koom foo]$ git-svn init http://svn.foo.com/foo
[bretm@koom foo]$ git-svn fetch
        A       trunk/htaccess.dist
        A       trunk/Changelog-old.txt
        [snip]
r1 = 4634bd6d2b37808f9ff43839411131dffdf067c6 (git-svn)
        M       trunk/README
r2 = de60b7d4aaab3fc45b2c1548062c21265873719b (git-svn)
        A       trunk/blah.txt
[snip]
[bretm@koom foo]$

We now have local cached information from upstream’s Subversion repository.

Step 4. Rebase your patchset given new updates on trunk

We’ll step away from foo for a bit and use my wordpress-mu package as a more concrete example. Right now, fedorapeople/upstream is a pristine copy of wordpress-mu’s 2.6.1 tag. The Fedora-specific config customizations reside in fedorapeople/fedora. But it has been several days, and there’s a bit of a difference upstream between trunk & 2.6.1 (maybe some 2.7 stuff creeping in?):

[bretm@koom wordpress-mu]$ git diff tags/2.6.1 trunk | diffstat | tail -n1
 89 files changed, 1741 insertions(+), 1403 deletions(-)

Wow, that’s quite a bit of deltas. I want to pull these updates in, but I’m a little nervous at the scope. Ideally, I’d like to avoid tainting my upstream and fedora branches, yet still pull in the fedora-specific patch-set so I can run this through Koji, etc. In this case, it seems like the best strategy is to branch fedorapeople/fedora, and rebase that new branch from trunk.

[bretm@koom wordpress-mu]$ git checkout -b experimental fedorapeople/fedora
Branch experimental set up to track remote branch refs/remotes/fedorapeople/fedora.
Switched to a new branch "experimental"
[bretm@koom wordpress-mu]$ git merge trunk
Auto-merged htaccess.dist
CONFLICT (content): Merge conflict in htaccess.dist
Auto-merged index-install.php
Auto-merged wp-admin/admin.php
CONFLICT (content): Merge conflict in wp-admin/admin.php
Auto-merged wp-admin/includes/mu.php
CONFLICT (content): Merge conflict in wp-admin/includes/mu.php
Auto-merged wp-admin/menu-header.php
CONFLICT (content): Merge conflict in wp-admin/menu-header.php
Auto-merged wp-content/blogs.php
Auto-merged wp-includes/functions.php
Auto-merged wp-includes/l10n.php
Auto-merged wp-includes/version.php
CONFLICT (content): Merge conflict in wp-includes/version.php
Auto-merged wp-includes/wpmu-functions.php
Auto-merged wp-settings.php
Auto-merged wpmu-settings.php
CONFLICT (content): Merge conflict in wpmu-settings.php
Automatic merge failed; fix conflicts and then commit the result.

Hm. That doesn’t look great.

[bretm@koom wordpress-mu]$ git status
wp-admin/admin.php: needs merge
wp-admin/menu-header.php: needs merge
wp-includes/version.php: needs merge
# Not currently on any branch.
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       modified:   wp-admin/admin-ajax.php
#       modified:   wp-admin/admin-header.php
#       deleted:    wp-admin/bookmarklet.php
#       modified:   wp-admin/css/colors-classic-rtl.css
#       modified:   wp-admin/css/colors-fresh-rtl.css
#       modified:   wp-admin/css/dashboard-rtl.css
#       modified:   wp-admin/css/global-rtl.css
#       modified:   wp-admin/css/ie-rtl.css
#       modified:   wp-admin/css/install-rtl.css
#       modified:   wp-admin/css/login-rtl.css
#       modified:   wp-admin/css/media-rtl.css
#       modified:   wp-admin/css/press-this-ie-rtl.css
#       modified:   wp-admin/css/press-this-rtl.css
#       modified:   wp-admin/css/press-this.css
#       modified:   wp-admin/css/theme-editor-rtl.css
[snip]
#       modified:   wp-includes/wp-diff.php
#       modified:   wp-settings.php
#       modified:   xmlrpc.php
#
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#       unmerged:   wp-admin/admin.php
#       modified:   wp-admin/admin.php
#       unmerged:   wp-admin/menu-header.php
#       modified:   wp-admin/menu-header.php
#       unmerged:   wp-includes/version.php
#       modified:   wp-includes/version.php
[bretm@koom wordpress-mu]$

Hm. A few vi sessions later, I’ve got the conflicts merged, and experimental is now representative of trunk plus my fedora patch-set.

Is there an easier/better way to do this?