Friday, 7 March 2008

Balancing Greed with Anemia

I am many things to many people. If I go to the doctors I am a patient, when at work I am an employee, when at home, with my wife, I am a husband. I cannot be all these things at the same time and I need to behave differently with different people and what's more my doctor isn't interested in me as an employee, my employee cares not for my detailed medical history and my wife certainly doesn't care what languages I can program in let alone the content of my blog!

Readers of Domain Driven Design will be familiar with the concept of Aggregates: simply put an aggregate provides a boundary around concepts which are closely related and provides an entry point to those other entities. If you were modelling people the Person class would be an aggregate, within the person class would be other classes and methods to do with my medical state, languages I can program in and package private methods between my wife and I. Using a behaviour centric domain finding we can write code like this:

class Client 
def do
john = employeeRespotory.find("John")

if(john.is_not_skilled_in(:AnyJava)) do
john.train(:BeginnersJavaCourse)
end

john.can_do_job(Job.new(:SqlServer, :DotNet))
end
end

class Person < Skilled
skills

def is_skilled_in(skill)
if(skills.not_contains(skill.type)) do
return false
end

return skills.get(skill.type).is_at_level(skill.level)
end

def is_not_skilled_in(skill)
!is_skilled_in(skill)
end

boolean train(Training)
if(skills.not_contains(Training.skill) do
skills.add(Training.skill)
else
skills.get(Training.skill).upgrade(Training.level)
end
end

def can_do_job(Job)
Job.matches_skills(this)
end
end

class Job
def matches_skills(skilled)
Skills.each | skill | do
return false if skilled.is_not_skilled_in(skill)
end
end
end

class Test
def can_be_skilled_in_something
skills = Skills{:BasicJava}
testPerson = new TestPerson(skills)

assert_true(testPerson.is_skilled_in(:BasicJava))
end

def can_be_trained_in_something
testPerson = TestPerson.new(:NoSkills)
skill = Skill.new(Java, Basic)

testPerson.train(Training.new(skill))
assert_true(testPerson.is_skilled_in(skill))
end
... plus all the other tests
end

The problem is if we placed all of this behaviour and data in one Person class it will quickly get on the big side. This is what I call a Greedy Aggregate: in the same way I don't try to be all things to all people a class shouldn't try to be all things to all clients.

The biggest criticism I have heard about placing behaviour on Domain objects is the problem of Greedy Aggregates: huge classes that end up with bags of methods and test classes as long as your arm. The solution often presented is to move to a Service paradigm instead. I agree with the complaint of Greedy Aggreates but not the solution as the service paradigm moves code away from being object orientated and towards procedural. The above in service orientated code would be:

class Client 
def do
john = employeeRespotory.find("John")

if(employeeService.is_not_skilled_in(john, :Java)) do
employeeService.train(john, :BeginnersJavaCourse)
end

employeeService.can_do_job(john, Job.new(:SqlServer, :DotNet))
end
end

class EmployeeService
skillsRespository
def is_skilled_in(person, skill) do
if(skillsRespository.not_contains(person.id, skill.type)) do
return false
end

return skillsRespository.get(skill.type).is_at_level(skill.level)
end

def is_not_skilled_in(person, skill)
!is_skilled_in(person, skill)
end

def train(person, training) do
if(skillsRepository.not_contains(person.id, training.skill)) do
skillsResponsity.add(person.id, Training.skill)
else
skillsResponsity.get(prson.id, taining.skill).upgrade(training.level)
end
end

boolean can_do_job(person, job) {
job.eachSkill |skill| do
return false if is_not_skilled_in(person, skill)
end

return true
end
end

def Test
def canBeSkilledInSomething
testPerson = TestPerson.new
skill = new Skill(Java, Basic)
MockRepository {
expect.contains(testPerson.id, skill.type)
will.return(true)
expect.get(testPerson.id, skill.type)
will.return(new Skill(:Java, :Basic))

}

assert_true(employeeService.is_skilled_in(testPerson, :BasicJava))
end

def can_be_trained_in_something
testPerson = TestPerson.new
Skill = Skill.new(:Java, :Beginner)

MockRepository {
expect.contains(testPerson.id, skill.type)
will.return(false)
expect.add(testPerson.id, skill.type)
expect.get(testPerson.id, skill.type)
will.return(new Skill(:Java, :Basic))

}

employeeService.train(testPerson, Training.new(skill))
assert_true(employeeService.is_skilled_in(skill))
end
... plus all the other tests
end

The service client code feels more difficult to read: it flows less as a sentance making the Service the subject and not John who has disapeared somewhere in the parantheses. The Service code itself is more difficult to read than the behaviour class and the service test class is significantly more complex. Also the service requires a more complex repository and the Person class is reduced to being nothing more than a data holder. Another problem with service based is you often have to break encapsulation to get it to work and, as a result, you can end up with a lot of repetation in code.


I believe the issue with Greedy Aggregates mainly stems from making code centric to O/R mapping tools. To simply model our Person domain we end up with:

  • A Person object which maps to a Person table

  • A Medical History class with a table with a Person_Id foreign key

  • A Skill class with a table with a Person_Id foreign key


This makes it difficult to split the Person into smaller more specific classes as the O/R mapper requires a definate class to map to. In the above code we may want to split Person into two classes: one which represents the core aspects of a Person (name, age etc.) and another for their skills lets call it PersonWithSkills. In many O/R mappers this is difficult because we cannot create a mapping for PersonWithSkills as two classes cannot map to the same table. However we can create a repository which ties them together:

class PersonWithSkillsRepository
def get(id)
person = personRepository.get(id)
skills = skillsRepostitory.get(person)
return PersonWithSkills.new(person, skills)
end
end

This is how I've tended to advocate it in the past but recently I was playing with Ruby, and using its cool dynamic powers I approached it in a different way: essentially I merley extended the Person class using a module like so:

class Person 
has_many :skills

def with_skills
self.extend PersonWithSkills
end
end

module PersonWithSkills
def is_skilled_in
... # all skills based methods here
end

The only issue with this method is all of the ActiveRecord declarations have to be on the Person class - there may be a way around this but I don't know Ruby well enough to say for sure. It also has the disadvantage of not working for any classes outside of the core package (though knowing Ruby there may be a way around this to). However if you want to do more complex logic between role based aggregates you can. For example:

class Person
def as_traveller
self.extend PersonAsTraveller
end
end

module PersonAsTraveller < Locatable ... end

module TravellingPersonWithSkills < Skilled
def can_do_job(job)
job.matches_location(self.as_traveller) && job.matches_skills(skills)
end
end

class Job
def matches_location(locatable)
return locatable.matches(location)
end
end

Dynamic languages such as Ruby make it nice and easy to do the above as it has the ability to mix in methods. That doesn't mean we can't do something similar in a static langauge such as Java or C# it simply means that we have to jump through a few more hoops and use delegation instead.

So now that we are using behaviour based domain objects and moved away from services and have managed to cut those Greedy Aggregates down to size when is it right to use a service? A service will need to be introduced when there is something normal domain objects cannot be trusted to deal with something by themselves and require some co-ordination or greater authority to do it for them. The service must be stateless. A good example of this is transferring money between accounts: you don't want to leave the accounts to sort it amongst themselves so you will need a service to deal with the interaction:

class AccountTransferService
def transfer(account_from, account_to, amount)
if(account_from.has_funds(amount)) do
account_from.debit(amount)
account_to.credit(amount)
end
end
end

There are a number of ways of dealing with Greedy Aggregates in both dynamic and static langauges. What is important is being able to clearly identify aggregates and not be to restricted by the O/R mapper or simply give up and rely on services.

Saturday, 19 January 2008

The One Minute Build

The project I am working on has a build which used to take under a minute and has slowly crept up to around one minute forty five seconds. Recently I have begun to feel the pain of this longer build; it is surprising just how much a paltry forty-five seconds increases those periods of sitting and waiting.

The build is an essential part of the code base, it is what gives you the developer and the rest of the team feedback on the state of your code. Continous Integration tell us that we should make as small changes as possible, verify them using the build, then check in (then verify the build on the build server). Under these rules we should probably aim for a optimum integration rate - the frequency of check-ins (per pair). I believe a optimum integration rate should be around once per hour.

A slow build can be a blocker to good CI practice: the longer a build takes to run the less often people will run it, the less they will check in and the bigger the change list they create. Martin Fowler's original article on CI states that "the XP guideline of a ten minute build is perfectly within reason" however I want to contest that guideline and bring it down to a minute. Why? because the difference between a build that takes under a minute and one that is over a minute is quite considerable in trying to keep your change list down.

On average a developer will run three builds per change, the first being on the change itself - which is usually run in the IDE and will often have some failing tests (though not always) - the second to verify the change and the third to verify the update. So the general rhythm goes: make a change, run the tests, fix the broken tests, verify the build on the command line, update, verify the build again, commit. Of course it is possible to make a change and have no tests failing (meaning two builds per change) but even then most devs will run the testsuite in the IDE and then verify the full build against the command line.

In the case of a merge conflict (either in code or failing tests on update) we add an extra two builds per conflict: the first to verify the fix, the second to verify the fresh update before check-in.

Three changes per build means that the total amount of time spent building is the build time multiplied by three: therefore any build which takes one minute will consume three minutes of developer productivity. If we check-in once every hour that is 3/60 or 5% of your devs time spent building. Take the build up to just two minutes and we have 6/60 or 10%. Take a build of ten minutes and we have 30/60 or 50% of devs time spent building! If there was a merge conflict on a ten minute build it will consume 50/60 or over 80% of that hour.

On a ten minute build a productivity ratio of 1:2 is not acceptable, so to compensate we adjust the optimum integration rate. This is achieved, essentially, by trying to bring the ratio back to an acceptable level, so if 10% loss (equivalent to a two minute build) is acceptable the check-in rate becomes once every five hours. Comparatively 10% on a one minute build is a check-in every 30 minutes verses once every five hours!

Decreasing check-ins is a dangerous game; the longer changed code is out of the repository the more out-of-date it is. The difference between your repository and your code becomes greater (assuming, of course, that you are making changes over that period) and the difference between your code and your fellow developers also becomes greater. By checking in frequently the codebases are constantly re-aligned minimizing the impact of future changes. The most obvious side effect to diverging code bases is the chance of conflict increases - small, frequent check-ins reduce this to an extent where it can become a non-issue - big check-ins mean big conflicts, big conflicts mean loss of productivity, stress, more time your code is out of sync and more times you have to run the build.

To demonstrate this lets create a concept of a "window of opportunity", this window is the space between doing the update and checking in and the opportunity is for another pair to check-in before you do! The size of this window is equivalent to the length of the build; we all know that if we are going to check in we try to grab this window (how often do we say "hang on a minute I've just got to run the tests and check in this build"?). The reason for this is we don't want to find that someone else has checked in between the last update and us running the build. The greater the length of the build, the greater the chance that someone else has committed (which increases based on the size of the team) and the more times you have to run your build.

There are other side-effects to long builds, you can't keep the change list small and people will find ways around running the build; from only running it once out of the three times (increasing the chance of broken builds) or avoiding full runs by only running the tests they believe to be impacted (again increasing the chance of broken builds). None of these is a good practice. Big builds also mean loss of attention: running the build, doing an update, running the build again, then checking in takes just two minutes on a one minute build but twenty on a ten. Two minutes won't disrupt to flow of development and probably works as a nice gap to discuss things with your pair or just give the brain a rest. The ten minute build equals a wait of twenty minutes: enough time to eat lunch! At the end of twenty minutes any focus you had has gone.

A one minute build may sound unrealistic but I believe it is possible. In some ways it is an indication to healthiness of your code base and by that I mean both production and test - poorly written and bad performing test code is equally as bad as poorly written, bad performing production code. It is very easy to be slightly relaxed about build times, treat them as an inevitability and the watch them slowly go from one minute builds to ten.

It's all very easy to preach about one minute builds when I am in the privilliged position to be working on a code base that only takes one minute forty five seconds. The speed of the build is due to the drive, early on in the project, not to tolerate long builds: several times it's been looked at when it started to get a bit overgrown and it's been hacked back down to size. If we had been more relaxed I'm sure it would have crept up a lot higher.

Thursday, 10 January 2008

Rawhide

I found out yesterday that I was being rolled of the project I am currently on. I was very ill prepared and I had been guilty of kidding myself the project was going on forever. I had made the classic mistake of inaction believing that denial resolves into desire. It has been quite a strange experience seeing a project draw to a close. I have learnt there are two pains to consultancy on a project of this kind: people roll off and you roll off. It was strange to come back after Christmas to see the team halved. People you'd worked very closely with - whole days pairing together side by side - good friends who are suddenly not there. Though it isn't literally here today gone tomorrow the period between finding out someone is going and them being gone is very short - there isn't the one month notice period and the usual "good luck with your next job" as essentially nobody left ThoughtWorks, they've just left the team, but TW being such a vast place it feels as if they get sucked up back into the unseen world of clients.

The biggest shock is when you yourself get your papers. It was a moment I was inwardly dreading: I knew I rolled off mid-January but I was hoping that my term would be increased, I was hoping the project would go on but the cogs that do turn leave my assignment end date landing in a void where there is currently no commitment to extend the project meaning I've popped up as a resource to be used somewhere else. In someways the great Resource Management machine can feel quite heartless and random, as I guess it should be, but the first time you experience it - especially after ten years of in-house employment - you realise how it's dangerous to ignore it.

It's been a great project: the best I've ever worked on. It's renewed my faith in development; I've never enjoyed my work as much as I have over the last 5 months. Although not everything has been perfect - the pace too aggressive, the project too short, a feeling of uncertainty over it's lifespan - it has proven to me how much better agile is especially when you apply all the best practices from stands-ups, story walls, retrospectives, continuous integration, testing and of course pairing (I have developed a terror of having to solo again: back to the days of insoluar, procrastinating, under productive depression). From it's earliest days I wanted to drag old colleagues in and show them how this project has lived and breathed in a way so unlike all the bureaucratic process driven drivel that I have had the unpleasure to work on or the other political chaos zero management departments out there. The people have renewed my faith in development too: working with people who seem to get *it*; a whole team (QAs, BAs, developers, project managers etc.) who care about what they do, bother to learn and understand and discover, has been mightily refreshing.

So a farewell to a great project and the great team I have had the please to work with. I hope I come across your paths and get to work with you again. A special mention to Sukita and Seema who are not only great role models for that all too rare bread of women in IT but have character of Dickensian proportions and, in the highly unlikely event of media type people reading this, deserve their own TV show (Sukita already has broadcasting experience). Also special mentions to the other half of the fire team: Greg, and to Alex for all our discussions on Russian literature and the church.

Monday, 29 October 2007

Your Feet's Too Big

Every piece of code written has a refactoring footprint which is essentially its proliferation through the code base and its level of difficulty to refactor. Some code can have a high proliferation but the refactoring may be simple, for example renaming a variable in modern IDE's is a simple matter regardless of the number of usages. On the other hand changing a variables return type or changing the signature of a method both create a large refactoring effort, especially if that requires knock-on refactorings of other classes and the greater the consumption of the code the greater the footprint. There are other things that can have an impact on the ease of refactoring: using libraries that create code that is difficult to refactor (for example having to place variable names etc. in strings), using external libraries directly without encapsulation, having similar routines across code which isn't encapsulated, the number of entry points to a piece of code (the number of different ways of achieving the same thing) all of these can make refactoring difficult and risk the introduction of bugs. Another area, much neglected when considering ease of refactoring, is test code which very often has a high consumption of production code (sometimes higher than the production code itself) and can make the simplest refactorings painful because large chunks of test code are dependent on your class.

TDD states that code that is easy to test is good code and the same is true for refactoring: writing code which is easier to refactor leads to cleaner code. Refactoring footprints have, in very real terms, an impact on productivity and therefore managing them is quite important. There are various techniques - most of them basic OOP principles - that can help keep the refactoring footprint down. However, due to the natural evolution of code, this can be quite hard to keep track of and unless you have an in-depth knowledge of the codebase - which is increasingly unlikely the bigger and older the codebase is - it can be difficult to 'see' a footprint.

I think it would be interesting to explore the use of metrics to measure refactoring footprints. These metrics would give a very interesting and valuable insight into the codebase and its quality. The interesting thing is that unlike many other code metrics refactoring footprints need to take into account test code as well. From a very basic level counting the number of usages across both production and test code can give a rudimentary indication to a footprint. More sophisticated metrics would measure the impact of a refactoring (are there knock-on refactorings). Different types of refactoring would also require different metrics and some refactorings may be uncommon in certain codebases or may have a degree of acceptable pain (package refactorings for example). As with most metrics they will be open to a degree of interpretation based on codebase and class but none the less the metrics should be useful - especially in recommending refactorings!

I haven't done much research on this so I do not know if any tools already exist to calculate such metrics or even what metrics they provide, though I am sure there will be. I shall try and track some down and have a play to see what sort of data they provide and establish what sort of insight they can give into a codebase.

Thursday, 2 August 2007

Seperating the Men from the Morts

I'm a bit slow on the uptake here (maybe I'm too busy to be a true alpha geek) but if you're one of those .NET developers who've always felt a little bit uncomfortable sitting in the MS Fanboy Hummer on the way to the latest .NET launch (and secretly was wishing you were at RailsConf) and because every time you mentioned test first, agile, a new OS project, regret that a Java library wasn't in .NET - or god forbid you actually deployed open source non-MS certified libraries on a production machine! - you got funny looks, then maybe you are an ALT.NETer and never knew it.

The opposite is MORT.NET: those old VB6ers or VisualC++ers who just can't shake the habit of databound controls with a business layer of static methods which shuttle anemic data objects to and from the database and UI, and tests by creating a test app of datagrids and command buttons. The MORT.NETer is a drag-and-drop all star who wouldn't have known what AJAX was until MS released ATLAS (but are slightly cautious as it's FREE) and believe NHibernate is pure evil because it doesn't use stored procedures.

Join the movement comrades: if you're the kind of developer that's prepared to try something because it's cool rather than asking if it is MS certified you're already there!

Wednesday, 18 July 2007

Blogger: stop stealing my space

Does anyone know why Blogger is adamant on removing spaces when you post? When I post code snippets it always decides to cut down all my spaces making my code snippets look a bit short on the indentation.

What's more annoying is that everytime I edit a post it takes a few more spaces away leaving the code without indentation and I have to re-indent everytime I do an edit!

I've searched the site for options but can find nothing that says "stop stealing my spaces". Any clues?

Tuesday, 17 July 2007

Loopy loops

Encapsulation and information hiding tells us that we encapsulate an objects internal data structure and expose methods, a client can see all the methods but none of the data. Tell, don't Ask extends this to say we should always tell an object to do something, not ask it for some information and then do something for it. The Law of Demeter helps to clarify this by telling us that we should avoid invoking methods of a member object returned by another method. By employing Demeter we avoid "train-wreck"s e.g. a.B.C.DoSomething().

I have seen - and written - a lot of code that does this:

foreach(Object obj in myObject.Objects)
{
obj.DoSomething();
}
But doesn't this break the three rules? First we are exposing the objects internal structure (the fact that myObject has a collection of Objects inside it). Secondly we break Demeter in three ways by:
  1. invoking the GetEnumerator (in C#) method of IEnumerable which was returned by myObject (via Objects)
  2. invoking the MoveNext (which incidently breaks Command-Query seperation) and Current methods of IEnumerator (in C#) which was returned by IEnumerable which was returned by myObject
  3. invoking the DoSomething() method of obj which was returned by the Current method of IEnumerator which was returned by IEnumerable which was returned by myObject (see why it's called a train wreck?)
Lastly we break "Tell, Don't Ask" by asking all the above objects (three in total: myObject, IEnumerable, IEnumerator) in the collection for something and then doing something to obj.

Looping arrays and collections is very common and in most programs exposing the collection is the norm. But what happens when you need to change the way the collection works? What if you decide to replace your objects array with a HashTable or a Set? Thankfully most OOP languages gives you some sort of iterator so, in C# for example, you can just expose IEnumerator and you might be alright. But this still breaks the three rules and what do you do if your underlying data structure doesn't have an IEnumerable? What if you're dealing with some legacy COM+ object and you have revert back to the old for or while? Or even worse, the data structure doesn't even seem to have a simple structure (not so uncommon)? Just exposing an enumerator isn't so easy but we still work around it by doing something like this:

IEnumerator Objects
{
IList list;
for(int i = 0; i &lt; comObject.Count; i++)
{
list.Add(comObject[i]);
}

return list.GetEnumerator();
}
And then in the client we have:

foreach(Object obj in myObject.Objects)
{
// here we go again!
}
Hang on a second: we loop so we can loop? Doesn't that sound a bit strange? To make matters worse we seem to do this as well:

void SaveEverything()
{
foreach(Object obj in myObject.Objects)
{
database.Save(obj);
}
}

... Somewhere else in the program ...

void DisplayEverything()
{
foreach(Object obj in myObject.Objects)
{
ui.Display(obj);
}
}

... Again somewhere else ...

void SendEmail()
{
foreach(Object obj in myObject.Objects)
{
email.Send(obj);
}
}
How many times do you want to write that damn loop? So what's the alternative? To implement the three rules: hide the collection and stop exposing it and then tell the myObject to loop it for you. Quite simply:

public class MyObject
{
void Loop(IReceiver receiver)
{
foreach(Object obj in objects)
{
receiver.Recieve(obj);
}
}
}
All you need to do is pass your receiver to MyObject and it will do all the work for you:

myObject.Loop(databaseSaver);

public class DatabaseSaver : IReceiver
{
void Receive(Object obj)
{
database.Save(obj);
}
}
Or if you prefer you can use delegates and anonoymous methods:

myObject().Loop(delegate(Object obj) { database.Save(obj); });

public delegate void LoopDelegate(Object obj);

public class MyObject
{
public void Loop(LoopDelegate loopDelegate)
{
foreach(Object obj in objects)
{
loopDelegate(obj);
}
}
}


Now if you need to change the objects internal data structure of the way you iterate it (for performance reasons etc.) you can do so with ease and without breaking any of your clients.

Although with simple loops it all looks straightforward enough, it is with more complex structures that you gain a real advantage. In a parent child structure for example you may find a loop such as this:

foreach(Parent parent in foo.Parents)
{
foreach(Child child in foo.Children)
{
// do something
}
}
Writing this loop all over the place can be fairly cumbersome, and what if you don't want to expose the structure in this way? By using the pattern above you have more control over the semantics of the object. For example you could present the list flat and merle passes each object across as above (and thus loosing nested loops all over your code) or you could expose some structure:

public interface ParentReceiver
{
ChildReceiver CreateChildReceiver();
void ReceiveParent(Parent, ChildReceiver);
}

public interface ChildReceiver
{
void ReceiveParent(ParentReceiver);
}

public class Parents
{
public void Loop(ParentReceiver receiver)
{
foreach(Parent parent in parents)
{
ChildReceiver childReceiver = receiver.CreateChildReceiver();
parent.Loop(childReceiver);
receiver.ReceiveParent(parent, childReceiver);
}
}
}

public class Parent
{
public void Loop(ChildReceiver receiver)
{
for(int i = 0; i &lt; children.count; i++)
{
// don't pass any illegitimate children!
if(children[i].IsNotIllegitimate)
{
receiver.Receive(children[i]);
}
}
}
}

As you can see by following the three rules and telling the object what to do, and only the immediate object what to do, we can not only express the structure of the objects better and force clients to work with that structure, we also loosen the object couplings.

Tip:
For those pre .NET 2.0 people out there, or those who use languages that don't have generics, by using the above pattern you can strongly type your objects even if you can't strongly type the internal collections.

About Me

My photo
West Malling, Kent, United Kingdom
I am a ThoughtWorker and general Memeologist living in the UK. I have worked in IT since 2000 on many projects from public facing websites in media and e-commerce to rich-client banking applications and corporate intranets. I am passionate and committed to making IT a better world.