As we came out of beta, we continued to host on our virtual infrastructure however we needed to make a significant change to the way we served out Git repositories.

• We pulled away from Gitosis. This was never going to be an acceptable long-term solution for hosting any number of repositories or even supporting more than one directory level’s worth of repos. Replacing Gitosis was our own connector which would process all incoming SSH git requests and execute the appropriate git-receive|upload-packcommand. This removed the need for us to generate a gitosis config file and we could call our database directly – a massive performance boost for the application and users could assign permissions in real time rather than having to wait for the file to be generated by a background task.

This was pretty much it for this version. The reason we made the jump from 1 to 2 was simply because this marked us removing our beta flag and starting to accept any signups. At the time, we had plans to launch another of our web applications, so our entire signup process was handled through our main aTech Media website which managed all user data and simply used API calls to create actual accounts in Codebase.

Version 3 – February 2009 – present

Version 3 saw a complete rewrite of the entire application – from the ground up we re-wrote every view, controller, model and internal library. The only thing which remained the same from v2 was our SSH git connector (which, incidentally, has now gone too). With almost a years experience of running the app we’d found a number of things which we could have done better and the interface left much to be desired. The principle changes between v2 and v3 included:

• A completely new interface
• Multiple repositories per project
• Complete account separation – the early days saw user accounts shared between accounts but this was confusing and we had many enquiries from users who didn’t understand this.
• Ability to take payments from clients and start charging for the service
• Signups happened directly in the application rather than being farmed out to the aTech Media website

Hosting wise when we released version 3 we migrated everything onto its own dedicated hardware which included a single application server (which also doubled up as the background task runner) and another box for storage. Never underestimate the power of a single bare-metal server – they are extremely powerful and you can run a lot of traffic through just one and it’s often preferable than adding another and dealing with clustering them (which will always add extra network latency to your file system requests).

• Removed our own task runner and replaced it with delayed job
• Moved from an Nginx/Mongrel stack to an Apache/Passenger one
• Removed much of the Grit usage in favour of our own internal library
• Removed all view logic and moved into specific presenters which offer a simple class to a view
• Added a number of “containers” to provide inherited functionality to controllers.
• Added a gravatar cache
• All views are written in HAML
• We use the jQuery & jQuery-UI javascript frameworks

Since this version went live, we’ve been adding new features all the time including the wiki, time tracking, commit comments and much much more – you can see by looking back in the blog or using the application itself.

Version 3.3 – Summer 2009

Version 3.3 saw a major milestone in Codebase’s history – adding support for two extra repository types in the form of Subversion & Mercurial. It saw the final demise of Grit – which, although Grit is awesome, was a great feeling for us.

Since we launched in April 2008, we always thought I’d be cool to support Subversion in our application but ruled this out early on as “too much” and we didn’t want to overburden ourselves + our infrastructure would never have supported it. In July 2009, we finally bit the bullet and started planning how we’d add support for Subversion & Mercurial.

Tripod – our repository library

Tripod is our library which supports unified access to any git, subversion or mercurial repository. As an example example:

repo = Tripod.initialize_repository('path/to/my/repo') repo.scm_type #=> 'Git' repo.commits #=> [<Tripod::Git::Commit: be71f1>, <Tripod::Git::Commit: 3e910e] repo.commit('f368ef') #=> <Tripod::Git::Commit: f368ef> repo = Tripod.initialize_repository('path/to/my/svn_repo') repo.scm_type #=> 'Subversion' repo.commits #=> [<Tripod::Subversion::Commit: 4>, <Tripod::Subversion::Commit: 3] repo.commit(2) #=> <Tripod::Subversion::Commit: 2> 

That only gives a tiny taste of what we can run with Tripod and, because the API is exactly the same for any repository type, it makes our lives so much easier when making our presenters & controllers.

Tripod is completely separate to Codebase’s application so can easily be re-used in any other system. I’m sure we’ll be posting more about Tripod in the future.

SCAM – Source Control Access Manager

SCAM is another of our libraries which works with the Codebase database to establish whether or not users have access to a specific requested repository. We use for authenticating all Git & Mercurial requests over SSH as well as when authenticating Subversion requests over HTTP. Git & Mercurial support was easy because it was much the same as what we were already doing, however Subversion posed a slight issue. For Subversion support, we’d already decided to support HTTP as the official protocol because we wanted a low barrier to entry for SVN users – no public key nonsense etc… We ended up writing an Apache module which accepted the user’s normal Codebase username & password and used SCAM to establish whether or not the connection should be accepted – it works very well (if I do say so myself) – kudos to Charlie for getting this working so well.

Next time on “Codebase: the technical story”…

• What we did when we upgraded to our new multi-server cluster
• Details about our new partnership with VPS.NET
• Information about our new background runner: WorkHorse
• Our new SSH key server