The One Minute Build

The project I am working on has a build which used to take under a minute and has slowly crept up to around one minute forty five seconds. Recently I have begun to feel the pain of this longer build; it is surprising just how much a paltry forty-five seconds increases those periods of sitting and waiting.

The build is an essential part of the code base, it is what gives you the developer and the rest of the team feedback on the state of your code. Continous Integration tell us that we should make as small changes as possible, verify them using the build, then check in (then verify the build on the build server). Under these rules we should probably aim for a optimum integration rate - the frequency of check-ins (per pair). I believe a optimum integration rate should be around once per hour.

A slow build can be a blocker to good CI practice: the longer a build takes to run the less often people will run it, the less they will check in and the bigger the change list they create. Martin Fowler's original article on CI states that "the XP guideline of a ten minute build is perfectly within reason" however I want to contest that guideline and bring it down to a minute. Why? because the difference between a build that takes under a minute and one that is over a minute is quite considerable in trying to keep your change list down.

On average a developer will run three builds per change, the first being on the change itself - which is usually run in the IDE and will often have some failing tests (though not always) - the second to verify the change and the third to verify the update. So the general rhythm goes: make a change, run the tests, fix the broken tests, verify the build on the command line, update, verify the build again, commit. Of course it is possible to make a change and have no tests failing (meaning two builds per change) but even then most devs will run the testsuite in the IDE and then verify the full build against the command line.

In the case of a merge conflict (either in code or failing tests on update) we add an extra two builds per conflict: the first to verify the fix, the second to verify the fresh update before check-in.

Three changes per build means that the total amount of time spent building is the build time multiplied by three: therefore any build which takes one minute will consume three minutes of developer productivity. If we check-in once every hour that is 3/60 or 5% of your devs time spent building. Take the build up to just two minutes and we have 6/60 or 10%. Take a build of ten minutes and we have 30/60 or 50% of devs time spent building! If there was a merge conflict on a ten minute build it will consume 50/60 or over 80% of that hour.

On a ten minute build a productivity ratio of 1:2 is not acceptable, so to compensate we adjust the optimum integration rate. This is achieved, essentially, by trying to bring the ratio back to an acceptable level, so if 10% loss (equivalent to a two minute build) is acceptable the check-in rate becomes once every five hours. Comparatively 10% on a one minute build is a check-in every 30 minutes verses once every five hours!

Decreasing check-ins is a dangerous game; the longer changed code is out of the repository the more out-of-date it is. The difference between your repository and your code becomes greater (assuming, of course, that you are making changes over that period) and the difference between your code and your fellow developers also becomes greater. By checking in frequently the codebases are constantly re-aligned minimizing the impact of future changes. The most obvious side effect to diverging code bases is the chance of conflict increases - small, frequent check-ins reduce this to an extent where it can become a non-issue - big check-ins mean big conflicts, big conflicts mean loss of productivity, stress, more time your code is out of sync and more times you have to run the build.

To demonstrate this lets create a concept of a "window of opportunity", this window is the space between doing the update and checking in and the opportunity is for another pair to check-in before you do! The size of this window is equivalent to the length of the build; we all know that if we are going to check in we try to grab this window (how often do we say "hang on a minute I've just got to run the tests and check in this build"?). The reason for this is we don't want to find that someone else has checked in between the last update and us running the build. The greater the length of the build, the greater the chance that someone else has committed (which increases based on the size of the team) and the more times you have to run your build.

There are other side-effects to long builds, you can't keep the change list small and people will find ways around running the build; from only running it once out of the three times (increasing the chance of broken builds) or avoiding full runs by only running the tests they believe to be impacted (again increasing the chance of broken builds). None of these is a good practice. Big builds also mean loss of attention: running the build, doing an update, running the build again, then checking in takes just two minutes on a one minute build but twenty on a ten. Two minutes won't disrupt to flow of development and probably works as a nice gap to discuss things with your pair or just give the brain a rest. The ten minute build equals a wait of twenty minutes: enough time to eat lunch! At the end of twenty minutes any focus you had has gone.

A one minute build may sound unrealistic but I believe it is possible. In some ways it is an indication to healthiness of your code base and by that I mean both production and test - poorly written and bad performing test code is equally as bad as poorly written, bad performing production code. It is very easy to be slightly relaxed about build times, treat them as an inevitability and the watch them slowly go from one minute builds to ten.

It's all very easy to preach about one minute builds when I am in the privilliged position to be working on a code base that only takes one minute forty five seconds. The speed of the build is due to the drive, early on in the project, not to tolerate long builds: several times it's been looked at when it started to get a bit overgrown and it's been hacked back down to size. If we had been more relaxed I'm sure it would have crept up a lot higher.

Rawhide

I found out yesterday that I was being rolled of the project I am currently on. I was very ill prepared and I had been guilty of kidding myself the project was going on forever. I had made the classic mistake of inaction believing that denial resolves into desire. It has been quite a strange experience seeing a project draw to a close. I have learnt there are two pains to consultancy on a project of this kind: people roll off and you roll off. It was strange to come back after Christmas to see the team halved. People you'd worked very closely with - whole days pairing together side by side - good friends who are suddenly not there. Though it isn't literally here today gone tomorrow the period between finding out someone is going and them being gone is very short - there isn't the one month notice period and the usual "good luck with your next job" as essentially nobody left ThoughtWorks, they've just left the team, but TW being such a vast place it feels as if they get sucked up back into the unseen world of clients.

The biggest shock is when you yourself get your papers. It was a moment I was inwardly dreading: I knew I rolled off mid-January but I was hoping that my term would be increased, I was hoping the project would go on but the cogs that do turn leave my assignment end date landing in a void where there is currently no commitment to extend the project meaning I've popped up as a resource to be used somewhere else. In someways the great Resource Management machine can feel quite heartless and random, as I guess it should be, but the first time you experience it - especially after ten years of in-house employment - you realise how it's dangerous to ignore it.

It's been a great project: the best I've ever worked on. It's renewed my faith in development; I've never enjoyed my work as much as I have over the last 5 months. Although not everything has been perfect - the pace too aggressive, the project too short, a feeling of uncertainty over it's lifespan - it has proven to me how much better agile is especially when you apply all the best practices from stands-ups, story walls, retrospectives, continuous integration, testing and of course pairing (I have developed a terror of having to solo again: back to the days of insoluar, procrastinating, under productive depression). From it's earliest days I wanted to drag old colleagues in and show them how this project has lived and breathed in a way so unlike all the bureaucratic process driven drivel that I have had the unpleasure to work on or the other political chaos zero management departments out there. The people have renewed my faith in development too: working with people who seem to get *it*; a whole team (QAs, BAs, developers, project managers etc.) who care about what they do, bother to learn and understand and discover, has been mightily refreshing.

So a farewell to a great project and the great team I have had the please to work with. I hope I come across your paths and get to work with you again. A special mention to Sukita and Seema who are not only great role models for that all too rare bread of women in IT but have character of Dickensian proportions and, in the highly unlikely event of media type people reading this, deserve their own TV show (Sukita already has broadcasting experience). Also special mentions to the other half of the fire team: Greg, and to Alex for all our discussions on Russian literature and the church.

Your Feet's Too Big

Every piece of code written has a refactoring footprint which is essentially its proliferation through the code base and its level of difficulty to refactor. Some code can have a high proliferation but the refactoring may be simple, for example renaming a variable in modern IDE's is a simple matter regardless of the number of usages. On the other hand changing a variables return type or changing the signature of a method both create a large refactoring effort, especially if that requires knock-on refactorings of other classes and the greater the consumption of the code the greater the footprint. There are other things that can have an impact on the ease of refactoring: using libraries that create code that is difficult to refactor (for example having to place variable names etc. in strings), using external libraries directly without encapsulation, having similar routines across code which isn't encapsulated, the number of entry points to a piece of code (the number of different ways of achieving the same thing) all of these can make refactoring difficult and risk the introduction of bugs. Another area, much neglected when considering ease of refactoring, is test code which very often has a high consumption of production code (sometimes higher than the production code itself) and can make the simplest refactorings painful because large chunks of test code are dependent on your class.

TDD states that code that is easy to test is good code and the same is true for refactoring: writing code which is easier to refactor leads to cleaner code. Refactoring footprints have, in very real terms, an impact on productivity and therefore managing them is quite important. There are various techniques - most of them basic OOP principles - that can help keep the refactoring footprint down. However, due to the natural evolution of code, this can be quite hard to keep track of and unless you have an in-depth knowledge of the codebase - which is increasingly unlikely the bigger and older the codebase is - it can be difficult to 'see' a footprint.

I think it would be interesting to explore the use of metrics to measure refactoring footprints. These metrics would give a very interesting and valuable insight into the codebase and its quality. The interesting thing is that unlike many other code metrics refactoring footprints need to take into account test code as well. From a very basic level counting the number of usages across both production and test code can give a rudimentary indication to a footprint. More sophisticated metrics would measure the impact of a refactoring (are there knock-on refactorings). Different types of refactoring would also require different metrics and some refactorings may be uncommon in certain codebases or may have a degree of acceptable pain (package refactorings for example). As with most metrics they will be open to a degree of interpretation based on codebase and class but none the less the metrics should be useful - especially in recommending refactorings!

I haven't done much research on this so I do not know if any tools already exist to calculate such metrics or even what metrics they provide, though I am sure there will be. I shall try and track some down and have a play to see what sort of data they provide and establish what sort of insight they can give into a codebase.

Seperating the Men from the Morts

I'm a bit slow on the uptake here (maybe I'm too busy to be a true alpha geek) but if you're one of those .NET developers who've always felt a little bit uncomfortable sitting in the MS Fanboy Hummer on the way to the latest .NET launch (and secretly was wishing you were at RailsConf) and because every time you mentioned test first, agile, a new OS project, regret that a Java library wasn't in .NET - or god forbid you actually deployed open source non-MS certified libraries on a production machine! - you got funny looks, then maybe you are an ALT.NETer and never knew it.

The opposite is MORT.NET: those old VB6ers or VisualC++ers who just can't shake the habit of databound controls with a business layer of static methods which shuttle anemic data objects to and from the database and UI, and tests by creating a test app of datagrids and command buttons. The MORT.NETer is a drag-and-drop all star who wouldn't have known what AJAX was until MS released ATLAS (but are slightly cautious as it's FREE) and believe NHibernate is pure evil because it doesn't use stored procedures.

Join the movement comrades: if you're the kind of developer that's prepared to try something because it's cool rather than asking if it is MS certified you're already there!

Blogger: stop stealing my space

Does anyone know why Blogger is adamant on removing spaces when you post? When I post code snippets it always decides to cut down all my spaces making my code snippets look a bit short on the indentation.

What's more annoying is that everytime I edit a post it takes a few more spaces away leaving the code without indentation and I have to re-indent everytime I do an edit!

I've searched the site for options but can find nothing that says "stop stealing my spaces". Any clues?

Loopy loops

Encapsulation and information hiding tells us that we encapsulate an objects internal data structure and expose methods, a client can see all the methods but none of the data. Tell, don't Ask extends this to say we should always tell an object to do something, not ask it for some information and then do something for it. The Law of Demeter helps to clarify this by telling us that we should avoid invoking methods of a member object returned by another method. By employing Demeter we avoid "train-wreck"s e.g. a.B.C.DoSomething().

I have seen - and written - a lot of code that does this:

foreach(Object obj in myObject.Objects)
{
 obj.DoSomething();
}

But doesn't this break the three rules? First we are exposing the objects internal structure (the fact that myObject has a collection of Objects inside it). Secondly we break Demeter in three ways by:

1. invoking the GetEnumerator (in C#) method of IEnumerable which was returned by myObject (via Objects)
2. invoking the MoveNext (which incidently breaks Command-Query seperation) and Current methods of IEnumerator (in C#) which was returned by IEnumerable which was returned by myObject
3. invoking the DoSomething() method of obj which was returned by the Current method of IEnumerator which was returned by IEnumerable which was returned by myObject (see why it's called a train wreck?)

Lastly we break "Tell, Don't Ask" by asking all the above objects (three in total: myObject, IEnumerable, IEnumerator) in the collection for something and then doing something to obj.

Looping arrays and collections is very common and in most programs exposing the collection is the norm. But what happens when you need to change the way the collection works? What if you decide to replace your objects array with a HashTable or a Set? Thankfully most OOP languages gives you some sort of iterator so, in C# for example, you can just expose IEnumerator and you might be alright. But this still breaks the three rules and what do you do if your underlying data structure doesn't have an IEnumerable? What if you're dealing with some legacy COM+ object and you have revert back to the old for or while? Or even worse, the data structure doesn't even seem to have a simple structure (not so uncommon)? Just exposing an enumerator isn't so easy but we still work around it by doing something like this:

IEnumerator Objects
{
 IList list;
 for(int i = 0; i < comObject.Count; i++)
 {
  list.Add(comObject[i]);
 }
 
 return list.GetEnumerator();
}

And then in the client we have:

foreach(Object obj in myObject.Objects)
{
 // here we go again!
}

Hang on a second: we loop so we can loop? Doesn't that sound a bit strange? To make matters worse we seem to do this as well:

void SaveEverything()
{
 foreach(Object obj in myObject.Objects)
 {
  database.Save(obj);
 }
}

... Somewhere else in the program ...

void DisplayEverything()
{
 foreach(Object obj in myObject.Objects)
 {
  ui.Display(obj);
 }
}

... Again somewhere else ...

void SendEmail()
{
 foreach(Object obj in myObject.Objects)
 {
  email.Send(obj);
 }
}

How many times do you want to write that damn loop? So what's the alternative? To implement the three rules: hide the collection and stop exposing it and then tell the myObject to loop it for you. Quite simply:

public class MyObject
{
 void Loop(IReceiver receiver)
 {
  foreach(Object obj in objects)
  {
   receiver.Recieve(obj);
  }
 }
}

All you need to do is pass your receiver to MyObject and it will do all the work for you:

myObject.Loop(databaseSaver);

public class DatabaseSaver : IReceiver
{
 void Receive(Object obj)
 {
  database.Save(obj);
 }
}

Or if you prefer you can use delegates and anonoymous methods:

myObject().Loop(delegate(Object obj) { database.Save(obj); });

public delegate void LoopDelegate(Object obj);

public class MyObject
{
 public void Loop(LoopDelegate loopDelegate)
 {
  foreach(Object obj in objects)
  {
   loopDelegate(obj);
  }
 }
}

Now if you need to change the objects internal data structure of the way you iterate it (for performance reasons etc.) you can do so with ease and without breaking any of your clients.

Although with simple loops it all looks straightforward enough, it is with more complex structures that you gain a real advantage. In a parent child structure for example you may find a loop such as this:

foreach(Parent parent in foo.Parents)
{
 foreach(Child child in foo.Children)
 {
  // do something
 }
}

Writing this loop all over the place can be fairly cumbersome, and what if you don't want to expose the structure in this way? By using the pattern above you have more control over the semantics of the object. For example you could present the list flat and merle passes each object across as above (and thus loosing nested loops all over your code) or you could expose some structure:

public interface ParentReceiver
{
 ChildReceiver CreateChildReceiver();
 void ReceiveParent(Parent, ChildReceiver);
}

public interface ChildReceiver
{
 void ReceiveParent(ParentReceiver);
}

public class Parents
{
 public void Loop(ParentReceiver receiver)
 {
  foreach(Parent parent in parents)
  {
   ChildReceiver childReceiver = receiver.CreateChildReceiver();
   parent.Loop(childReceiver);
   receiver.ReceiveParent(parent, childReceiver);
  }
 }
}

public class Parent
{
 public void Loop(ChildReceiver receiver)
 {
  for(int i = 0; i < children.count; i++)
  {
      // don't pass any illegitimate children!
      if(children[i].IsNotIllegitimate)
      {
    receiver.Receive(children[i]);
   }
  }
 }
}

As you can see by following the three rules and telling the object what to do, and only the immediate object what to do, we can not only express the structure of the objects better and force clients to work with that structure, we also loosen the object couplings.

Tip:
For those pre .NET 2.0 people out there, or those who use languages that don't have generics, by using the above pattern you can strongly type your objects even if you can't strongly type the internal collections.

If only you want to dance to your tune you'll leave an empty dance floor

Last year my wife and I were married (hence the fact she is now my wife) and being the independent thinking, creative couple we are, we planned and organised the whole wedding ourselves, right down to the music. Both of us being passionate and snobbish about our music - and having the luxury of a very large collection - we didn't want some spotty DJ ruining our day by playing some dreadfully cheesy tracks and mumbling his way through the evening. So we managed what many said was impossible: a completely cheese free wedding playlist which got everyone boogieing the night away (not to blow our own trumpets but we had to hold back drunken protesters when we turned the music off!).

This weekend was my best man's wedding and so impressed was he with our music he wanted me to emulate it for him; so I accepted and invited him over to peruse my record collection. Finally, a few days before the wedding, he gave me his list and I looked at it with horror: it was clear he had just given me a list of his favourite personal tracks. His wedding being a typical big family and friends affair (over 150 guests compared to our 50) I could tell instantly the list wouldn't work, but no matter how much I tried to explain this he was adamant that on his wedding he wanted his favourite tracks. So I conceded and took his laptop gave him copies of the tracks I had and installed the latest WinAmp and the excellent SqrSoft Advanced Crossfading plugin to give it that ultimate disco feel and used WinAmp's built in ReplayGain to level the volumes of the tracks out (to prevent quiet track then sudden loud track).

The wedding reception went as I had expected: the groom loved every track that came on and a few close friends got up a jumped around the dance floor with him. But after half and hour or so the groom was off mingling at the bar and the dance floor was empty. His new bride was soon begging us groomsmen to get people up and dancing to fill the depressingly empty dance floor and although we did drag a few people off the sides, quickly the floor was vacant again.

About an hour later the groom came running up to me waving his arms and shouting in distress that his new wife had only gone and changed the playlist: she had loaded her emergency backup playlist instead! Before I could even react she had rushed over, the train of her dress picked up in her hands, shouting at him that she'd changed it because no-one was dancing while he shouted back something to the effect that her girl band music was rubbish (I can't print the exact words) and as I turned to leave them to their first domestic as husband and wife both their eyes looked pleadingly into mine. So there I stood with both looking to me to take their side; the groom playing to my taste as his wife’s playlist was full of offensively bad music and his wife looking to my reasonable side and the fact that people were now dancing. It was an easy decision for me to make: I asked them "do you want people to dance?" they both replied instantly "YES", "were people dancing before you changed the playlist?", "no" the wife replies and a sharp look at the groom squeezes out a squeaky "no", "OK then", I firmly state "if we go into the dance room and people are dancing then we leave her playlist on and if they're not then we change it". So we marched into the dance room to find a mass of badly dancing relatives and the decision was made.

There is a moral to this humorous story of marital conflict; my friend had placed his personal preferences before his objective: to ensure people dance and have a good time at his wedding. When my wife and I did our playlist we didn't just dump our favourite tracks down, we made lots of compromises - which included a strict no Smiths or she'll annul our wedding - to ensure that the songs we played people would want to dance to but we used our principles of good taste to guide us. Our principles meant that there would be no cheese at the objective meant that although The Rolling Stones Beast of Burden may be on our personal playlist we knew that Brown Sugar is the one that get's everyone dancing and that people would bounce around to The Undertones Teenage Kicks way before Radiohead's Exit Music (For a Film). When we put together our playlist we carefully considered whether each track was appropriate (which is why we didn't have Super Furry Animals Man Don't Give a Fuck) and listened to the track trying to imagine our guests dancing to it and ultimately remembering that although every song wouldn't please everyone that everyone should be pleased by most of the songs.

Development is like choosing a playlist: a good developer will always remember the objective. We've all been there: developing our own little features, the one's we think are really cool, and then getting upset when the customer says they don't like it and they want to change it. Developers are very guilty of doing what they think the application should do, forgetting what they've been asked to deliver and instead of doing the best thing to meet the customer's requirements they end up writing code to solve problems that don't exist or aren't important. It's a hard thing to avoid and I am as guilty as anyone of doing it: I remember all too clearly arguing that a feature shouldn't be developed the way it was requested as I felt it was functionally incomplete forgetting that what the customer was trying to achieve was to solve a problem quickly but not perfectly (what I call developing Word when all they want is Notepad).

As developers we find it difficult to compromise the 'perfect solution' for what the customer wants and we get stuck in thinking that to not write a technically or functionally complete solution means writing a technically or functionally bad solution (as my friend believed that not playing his favourite music was to play bad music). I had a conversation with a developer the other day who, like my friend, didn't want to compromise his design to meet the customer's objective. He wanted to build the feature in a way that was more generic and technically clever than was required and saw any sacrifice to this principle as bad coding. So entrenched was he in this view point he was prepared to sacrifice the customer's requirements and the deadline as he couldn't conceive that writing good, flexible code which was specific to the customer's requirements was a good thing. I struggled to find a way around this loggerhead and in the end I tried to explain the emphasis should be on usability before reusability and not on over-engineering.

The difference between my friends playlist and ours was that although both of us stuck to our principles of what we believe is "good music" (in the same way as a developer I would stick to mine: Agile, TDD etc.) my wife and I always checked ourselves against the objective and adapted to the situation, where as my friend just hung on to his principles for principles sake, forgetting what he was trying to achieve, and ultimately it was this that separated our full dance floor from his empty one.