experiments w/ upstream tracking and git

WordPress-MU uses subversion for their code repository, but I’m starting to become a bit of a git-addict. I’ll outline the steps I’ve taken to keep track of upstream while maintaining my (hopefully minimal) fedora customizations. Brain-patches and better methods cheerfully accepted!

Step 1: Create a remote repo

You don’t really need to do this, but tend to prefer having some off-site backup of my work… many a drive has failed for me in the past 😉

You’ve got a number of options for hosting sites; I’ll highlight katzj’s work w/ fedorapeople.org, as well as point folks to fedorahosted.org.

For now, I’m using fedorapeople.org, and here’s what the setup looked like:

[bretm@koom ~]$ ssh fedorapeople.org
Last login: Fri Sep  5 14:30:11 2008 from somewhere.net
[snip]
This system is puppet managed!  Local changes may be overwritten:
http://fedoraproject.org/wiki/Infrastructure/SOP/Puppet

[bretm@people1 ~]$ mkdir -p public_git/foo.git ; cd public_git/foo.git
[bretm@people1 foo.git]$ git --bare init
Initialized empty Git repository in /home/fedora/bretm/public_git/foo.git/
[bretm@people1 foo.git]$ exit

fedorapeople.org note: the .git segment of foo.git is important, you need that or gitdaemon won’t find your repo

That’s it, you’ve created a bare, off-site repo. Remember that, in git, bare repos contain only diffs, and binary blobs, and nothing else. We’ll need some place to actually do our work…

Step 2: Create a local repo

Head to your working directory of whatever upstream source that you need to track & package. I’ll just create a quick text file and we can use that for an example:

[bretm@koom ~]$ mkdir -p devel/git/foo; cd devel/git/foo
[bretm@koom foo]$ echo "some stuff to track" > README
[bretm@koom foo]$ git add README
[bretm@koom foo]$ git commit -m "initial commit" -a
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README
[bretm@koom foo]$ git branch
* master

So now we have a local repo, w/ a branch named master with a little bit of content. We’re now going to shove that into the remote repo with the branch name upstream:

[bretm@koom foo]$ git remote add fedorapeople git+ssh://fedorapeople.org/~bretm/public_git/foo.git
[bretm@koom foo]$ git push git+ssh://fedorapeople.org/~bretm/public_git/foo.git master:refs/heads/upstream
Counting objects: 3, done.
Writing objects: 100% (3/3), 230 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git+ssh://fedorapeople.org/~bretm/public_git/foo.git
 * [new branch]      master -> upstream
[bretm@koom foo]$ git remote show fedorapeople
* remote fedorapeople
  URL: git+ssh://fedorapeople.org/~bretm/public_git/foo.git
  New remote branches (next fetch will store in remotes/fedorapeople)
    upstream
[bretm@koom foo]$ git fetch fedorapeople
From git+ssh://fedorapeople.org/~bretm/public_git/foo
 * [new branch]      upstream   -> fedorapeople/upstream
[bretm@koom foo]$ git branch -a
* master
  fedorapeople/upstream

The git fetch fedorapeople is important; without that, you would only see your local master branch, and not the remote fedorapeople/upstream one. So, now we have a remote branch with which to track upstream’s trunk, but we also want one to hold the Fedora customizations:

[bretm@koom foo]$ git push git+ssh://fedorapeople.org/~bretm/public_git/foo.git master:refs/heads/fedora
Total 0 (delta 0), reused 0 (delta 0)
To git+ssh://fedorapeople.org/~bretm/public_git/foo.git
 * [new branch]      master -> fedora
[bretm@koom foo]$ git fetch fedorapeople
From git+ssh://fedorapeople.org/~bretm/public_git/foo
 * [new branch]      fedora     -> fedorapeople/fedora
[bretm@koom foo]$ git branch -a
* master
  fedorapeople/fedora
  fedorapeople/upstream

Now comes the interesting part… interacting w/ upstream’s svn repo.

Step 3: Use git-svn to fetch upstream’s Subversion repository

The general workflow for maintaining the Fedora package will look something along the lines of:

  1. fetch updates from upstream
  2. apply updates to fedorapeople/upstream branch
  3. rebase the fedorapeople/fedora branch upon fedorapeople/upstream
  4. roll a new srpm and build through koji

To make our lives easier, we’re going to use git-svn.

[bretm@koom foo]$ git-svn init http://svn.foo.com/foo
[bretm@koom foo]$ git-svn fetch
        A       trunk/htaccess.dist
        A       trunk/Changelog-old.txt
        [snip]
r1 = 4634bd6d2b37808f9ff43839411131dffdf067c6 (git-svn)
        M       trunk/README
r2 = de60b7d4aaab3fc45b2c1548062c21265873719b (git-svn)
        A       trunk/blah.txt
[snip]
[bretm@koom foo]$

We now have local cached information from upstream’s Subversion repository.

Step 4. Rebase your patchset given new updates on trunk

We’ll step away from foo for a bit and use my wordpress-mu package as a more concrete example. Right now, fedorapeople/upstream is a pristine copy of wordpress-mu’s 2.6.1 tag. The Fedora-specific config customizations reside in fedorapeople/fedora. But it has been several days, and there’s a bit of a difference upstream between trunk & 2.6.1 (maybe some 2.7 stuff creeping in?):

[bretm@koom wordpress-mu]$ git diff tags/2.6.1 trunk | diffstat | tail -n1
 89 files changed, 1741 insertions(+), 1403 deletions(-)

Wow, that’s quite a bit of deltas. I want to pull these updates in, but I’m a little nervous at the scope. Ideally, I’d like to avoid tainting my upstream and fedora branches, yet still pull in the fedora-specific patch-set so I can run this through Koji, etc. In this case, it seems like the best strategy is to branch fedorapeople/fedora, and rebase that new branch from trunk.

[bretm@koom wordpress-mu]$ git checkout -b experimental fedorapeople/fedora
Branch experimental set up to track remote branch refs/remotes/fedorapeople/fedora.
Switched to a new branch "experimental"
[bretm@koom wordpress-mu]$ git merge trunk
Auto-merged htaccess.dist
CONFLICT (content): Merge conflict in htaccess.dist
Auto-merged index-install.php
Auto-merged wp-admin/admin.php
CONFLICT (content): Merge conflict in wp-admin/admin.php
Auto-merged wp-admin/includes/mu.php
CONFLICT (content): Merge conflict in wp-admin/includes/mu.php
Auto-merged wp-admin/menu-header.php
CONFLICT (content): Merge conflict in wp-admin/menu-header.php
Auto-merged wp-content/blogs.php
Auto-merged wp-includes/functions.php
Auto-merged wp-includes/l10n.php
Auto-merged wp-includes/version.php
CONFLICT (content): Merge conflict in wp-includes/version.php
Auto-merged wp-includes/wpmu-functions.php
Auto-merged wp-settings.php
Auto-merged wpmu-settings.php
CONFLICT (content): Merge conflict in wpmu-settings.php
Automatic merge failed; fix conflicts and then commit the result.

Hm. That doesn’t look great.

[bretm@koom wordpress-mu]$ git status
wp-admin/admin.php: needs merge
wp-admin/menu-header.php: needs merge
wp-includes/version.php: needs merge
# Not currently on any branch.
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       modified:   wp-admin/admin-ajax.php
#       modified:   wp-admin/admin-header.php
#       deleted:    wp-admin/bookmarklet.php
#       modified:   wp-admin/css/colors-classic-rtl.css
#       modified:   wp-admin/css/colors-fresh-rtl.css
#       modified:   wp-admin/css/dashboard-rtl.css
#       modified:   wp-admin/css/global-rtl.css
#       modified:   wp-admin/css/ie-rtl.css
#       modified:   wp-admin/css/install-rtl.css
#       modified:   wp-admin/css/login-rtl.css
#       modified:   wp-admin/css/media-rtl.css
#       modified:   wp-admin/css/press-this-ie-rtl.css
#       modified:   wp-admin/css/press-this-rtl.css
#       modified:   wp-admin/css/press-this.css
#       modified:   wp-admin/css/theme-editor-rtl.css
[snip]
#       modified:   wp-includes/wp-diff.php
#       modified:   wp-settings.php
#       modified:   xmlrpc.php
#
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#       unmerged:   wp-admin/admin.php
#       modified:   wp-admin/admin.php
#       unmerged:   wp-admin/menu-header.php
#       modified:   wp-admin/menu-header.php
#       unmerged:   wp-includes/version.php
#       modified:   wp-includes/version.php
[bretm@koom wordpress-mu]$

Hm. A few vi sessions later, I’ve got the conflicts merged, and experimental is now representative of trunk plus my fedora patch-set.

Is there an easier/better way to do this?

Advertisements

Fedora packages for WordPress-MU

It’s taken me a lot longer to get through all the details, but I’ve finally finished the first-pass of WordPress-MU packages for the upcoming Fedora 10 beta:

http://koji.fedoraproject.org/koji/buildinfo?buildID=61416

These packages will form the basis for internal blogs at Red Hat, as well as some blogging work going on in the Fedora community. Stay tuned, and patches cheerfully accepted 🙂

Time Machine & NFS or Samba backup

Janel has been very pleased with her Mac so far; mostly she’s been working mostly with iPhoto and iMovie. I have to say: I’m damn impressed with the machine.

I’ve been doing a bit of research on the integrated backup program, Time Machine, that’s included in Mac OS X 10.5. At first, I was disappointed to hear it required the HFS filesystem, but I did a bit more digging, and found these two links:

http://freakymousemats.com/blog/posts/2007/11/01/time-machine-over-smb/

http://forums.bit-tech.net/showthread.php?t=141960

These suggest that folks have had success using both SMB as well as NFS to serve as the persistent storage. I haven’t tested it out, but plan to use it with the 300gb HP Media Vault that serves as our on-site backup. Wish me luck.

Oh, and the critical command appears to be:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

UPDATE (14-July-2008): so, this didn’t work out too well.  The machine complained that it wasn’t able to create the image.  I’ve heard that other folks have the same issue, and will be trying to find and document the solution.  Then I’ll start testing restoring files =)

UPDATE (22-July-2008):  Some initial success… Wes’s information got me past the sparsebundle failure, thought I had to dig through the link comments to find the right parameters for Janel’s Intel-based Mac.  Here’s what I used:

hdiutil create -size $SIZEg -fs HFS+J -type SPARSEBUNDLE -volname “Backup of $MACHINENAME” $MACHINENAME_$MACADDRESS.sparsebundle

You then recursively copy that sparse bundle to the network share, and it should become something you can select during Time Machine setup.  Time Machine is going to try backing up her Mac in an hour, I’ll post if it succeeds.

One thing I am concerned about is keeping the network connection available to the Mac… I haven’t found the equivalent of “reconnect,” etc.

Blackberry vs iPhone

Ok, sure, the iPhone now has all three of the drool-over functions I knew I wanted in a phone:

  1. GPS
  2. WiFi
  3. Quality calendaring w/ syncing

However, I’ve just found the uber-feature I never knew the Blackberry Curve’s had; honest-to-god voice dialing.

I used to try to use this w/ my Motorola Razr; it would record a sound tag of me saying something, then try to match it when I was placing a call. The results were similar to the Newton’s handwriting recognition attempts as immortalized in Doonesbury. I was a bit skeptical when I saw it in the Blackberry’s menus, but I eventually got around to giving it a shot. I hit the voice dial button, and the phone ordered me to “say a command.” Disconcerted, I said the first thing that came to mind: “Call.” The phone responded: “Say a number or a name.” I rattled of the ten digits of the home phone, and was soon talking to my wife.

It was a little scary.

I think I’ve dialed about 20 unique numbers this way, and the phone has only made mistakes on ~3. And when it screws up, it guesses two alternative numbers before giving up completely, and only 1 of the 3 recognition failures proved ultimately unsuccessful.

If the iPhone doesn’t have this functionality, I flat-out do not want one.

MacBook Pro

We finally did it; we made the switch. Janel’s HP was on its last legs, and while I did try to get her onto Fedora, it proved a bit to rough-around-the-edges for her. So, we picked her up a 15″ MacBook Pro. So far, I’m envious… it is pretty. Plenty of horsepower in it for her music, photos, and now gaming 🙂

    Blackberry Curve 8310

    Well, I finally won a new Blackberry Curve 8310 off of EBay. So far, I love it. The Google applications are beautiful on the BB compared to what I’m used to on my old Treo. Palm’s days are numbered.

    Coolest feature: integrated GPS + Google Maps. I’ll never be lost again.

    All these innovations in the wireless field are getting me pretty interested in what kind of applications might be most needed. Pair open handset platforms, with the new ease of developing distributed web services (App Platform, EC2, S3, etc), and there’s a new world heading our way…

    Open-source blogging software :/

    I’ve been playing around w/ WordPress-MU’s code-base a bit, helping evaluate it for potential in fedora-land and other places… so far the experience has been a bit frustrating. While I think it has more potential than our current solution (lyceum), I would have expected a much easier path.

    For someone that is rather intimately familiar with the construction of web applications, the installation process is *horrible*. It requires the schema to be there prior to installation; why not simply allow your installer to create your datastore? Once I finally got myself through the installer process, the 1.3.3 codebase is blowing up on some unknown column error (“blog_public”).

    They do have an interesting approach in terms of trying to keep some level of compatibility with the core WordPress code-base… I have to wonder, though, why they just didn’t move towards -MU as the new common platform? Cut a branch for the single-user code-base, maintain it for awhile, but do active development in the multi-user space? If anyone can shed some light on the reasoning here, I’d appreciate it 🙂

    I also briefly looked at livejournal & Movable Type; both of these may merit more of a look.

    Has anyone found a good python or java FOSS blogging system? If so, *please* let me know.