Immutable Wrapper

I am a huge fan of immutables, especially for value objects. Even in clumsy static languages like C# and Java that seem to throw every obstacle in your way to deter you from using them I find the benefits outweigh the costs.

Sometimes though a mutable makes sense. Whether that's because of the restrictiveness of a framework or the paradigm of the language. But what if we don't want to loose the power of immutables behind the scenes?

Introducing The Immutable Wrapper Pattern!

The Immutable Wrapper is a wonderfully simple pattern because there are only two things you need to do.

1. Create a class with only one mutable field
2. Wrap the immutables functions.

Here's an example. You've got a copy-on-write list implementation but your binding it to some GUI control that gives you a big headache if you try and point it to a different data source.

Here's your immutable:

  class CopyOnWriteList
 {
   private readonly IEnumerable items;

   CopyOnWriteList Add(item)
   {
     return new CopyOnWriteList(new List(items, item));
   }
 }

Now wrap it:

  class WrappedCopyOnWriteList
 {
   CopyOnWriteList list;

   void Add(item)
   {
     list = list.Add(item)
   }
 }

There, it couldn't be simpler. So simple in fact I don't know what else to say. Enjoy!

The Singleton Killer

We all hate singletons so here's another useful refactoring pattern: The Singleton Killer.

Here's a common snippet of code we all love to hate:

class Transfer
 def funds(from_number, to_number, amount)
   from = AccountRepository.instance.get(from_number)
   to = AccountRepository.instance.get(to_number)
   from.balance -= amount
   to.balance += amount
 end
end

So how do we kill this beast? Simply follow these idiot proof steps:

Step One: Introduce Field

Stop asking the singleton for it's reference and store your own by creating a field:

class Transfer
 @repository = AccountRepository.instance
 def funds(from_number, to_number, amount)
   from = repository.get(from_number)
   to = repository.get(to_number)
   ...
 end
end

Wow: it's starting to look like normal code!

Step Two: Initialize in constructor

Move the initialization to the constructor:

def initialize()
 @repository = AccountRepository.instance
end

This is good but wouldn't it be better if the dependancy could be injected?

Step Three: Chain constructor

In languages where dependency injection is useful you will need to chain the constructors:

public Transfer() : this(AccountRepository.instance)
public Transfer(AccountRepository repository)
{
 this.repository = repository
}

Now the AccountRepository can be injected which of course means we can mock it out!

Step Four: Extract interface (for statically typed languages)

Except statically languages: they'll need to extract an interface:

public interface IAccountRepository
{
 Account Get(string number);
}

public Transfer(IAccountRepository repository) ...

Good: now you can start mocking and testing.

Step Five: Introduce container

If you haven't already got a container then get one (else ensure you a wiring up your dependencies in one place). Then get that to wire up the Transfer object for you:

container.Add(AccountRepository)
container.Add(Transfer)
transfer = container.Get(Transfer)
transfer.funds("YOUACCNUM", "MYACCNUM", 50000)

Step Six: Remove singleton

Once a container is doing all your injection for you that singleton is a waste of space: get rid of him by removing that horrible instance accessor.

Choose your strategies with this refactoring wisely: if your objective is to get a class tested then first repeat steps one-four within the same class until it is clean of all singletons. This is a good exercise in it's own right - even if you're not going to move on to test or remove the singleton completely - as you start to move all the dependancies to the constructor you gain a better picture of the classes collaborators (and the quality of design). On the other hand if your objective is to remove the singleton repeat steps one-three on every usage you can find then finish up with steps four-six. Another technique is to keep pushing the useage of the singleton up through the code so each client then takes on responsibility for passing the singleton down when it creates an instance of the class. Like so:

  Transfer.new(AccountRepository.Instance)

Then repeat steps one-three on the class you've just pushed into, then push the singleton up again up and so on until you can't push it anymore. This can be an interesting exercise in itself as it starts to push out the paths through the code base as you move through the layers and may also give you a clear point where your container needs to be. Be warned though: it can get messy (though strangely satisfying)!

On larger, messier codebases none of these approaches may be realistic in which case opt for the guerilla approach and pick them off as you find them. Though I would encourage trying to at least clean out a whole class using steps one-three even if you won't end up testing it right away. With all your dependancies in one place you'll still have a better picture of the design.

Beware Metanarratives

Now my darlings gather round while I tell you a story. But not just any story. A story about stories. About BIG stories.

Not so many decades ago, when the French still measured their GDP on the combined weight of their philosophical works, a particular man, with the amusing name of Lyotard, came up with the concept of Metanarratives (or Grand Narratives).

A Metanarrative, simply put, is a big story, often idealistic in tone, essential a grand, universal theory which is then applied to every aspect of life. There are many good examples of metanarratives from religious narratives, humanist, to economical and social theories communist, capitalist, freemarket etc.

The IT industry is brimming with metanarratives. Whether it is Stallman's completely open source future, no software patents, where everyone collaborates and all is wonderful, or the visions of an open web which overflows with information causing the end of totalitarianism, poverty etc. or the whole proprietary vs Open Source battle of the metanarratives.

Even as we move closer to home in the ever rational world of Enterprise Software Development we too find it overspilling. Waterfall has a wonderfully convincing metanarrative. And if SOA isn't a metanarrative (of Joycean composition) with it's land of milk and honey then I don't know what is. Probably the fastest growing metanarratives in IT is that of the Agile and Lean movement.

We humans love a good metanarrative to pin our life story on. They make us feel safe and cosy; it's nice to comprehend the world through the view of a single cohesive story with a beginning, middle and happy end (which is why marketing departments love forming them). We look around us finding more and more things to embellish and support our narratives. As we gain experience we see it as evidence to the greater truth of our chosen metanarrative.

But metanarratives are not a good thing - especially in an ever shifting landscape like IT. They make us lazy and prevent us from considering other stories. And if they are considered it is as a threat (e.g. MS' campaign against Open Source). Which is a problem in it's own right: if someone has subscribed to a metanarrative then it is almost impossible to win them round to any other story. Put one metanarrative against another and your blocked - queue visions of an evolutionist and a creationist locked in a room for eternity - monkeys would have written the works of Shakespeare before either changes their mind.

Metanarratives ultimately disappoint - especially in the ever shifting landscape of IT. Not so much because they are wrong but because they cannot be right. Everything, especially in IT, cannot fit into one nice story. The world is too culturally rich and diverse. As with the thousands of varying religions, The British Empire, Communism, the American Dream, DNA architecture, eventually the world shifts into a state where the narrative becomes ridiculous to apply. Yet people will keep doing so - the continued prevalence of Waterfall in some businesses being a case in point.

My concern is that we are guilty of promoting Agile as a metanarrative. Evangelists have moved it, from being a local story about a collection of problems in software development, to a model for the entire way businesses are run. Now, let's be clear, there are many elements of Agile I strongly subscribe to and promote, and I often agree with the basis of many of these claims. However the danger occurs when the Agile movement (or any other movement) seeks to displace any other narrative and claim itself universal. That ALL software should be developed this way, and ALL businesses should model themselves to support this. That's just crazy speak!

Interestingly (in my opinion) in their original forms Agile and Lean are post-modern in their attitudes towards metanarratives. Both attempt to promote fresh thinking and new ideas and move away from grand prescriptivism and dogma, especially of those found in the industry at the time. Most importantly both movements promote the fact that there isn't one universal way. For many people it was this breaking away from the metanarrative of the time and creating space to form our own stories which made Agile so attractive. Perhaps it is the inherent paradox in the post-modernists view of metanarratives that have caused them to, ironically, become metanarratives themselves.

The world of IT is too rich and diverse and full of too many stories (some old, some new) to support metanarratives. There is nothing wrong with some of these stories (such as Lean and Agile) when viewed in isolation; some are full of rich culture and ideas, some are simply misconceived and naive. On their own they are harmless and together with other stories they are invaluable. But as soon as people start turning them into something with which to narrate the whole Industry they are deadly.

So my warning is this: beware of metanarratives. If you notice that you are tending to frame the IT world around one story, whether that is Scrum, SOA, BDUF etc. please consider whether you have been caught in a metanarrative. Equally if you find yourself embattled due to someone's particular metanarrative, don't try and slam another one around their head. Instead try and appreciate their story at a local level whilst keeping your own stories localized and nonthreatening - they may even begin to fit it into theirs.

Killing the Config

Most applications require config of some description and it is considered good practice to wrap any calls which retrieve configured values in a config class and pass that around rather than splattering the codebase with direct calls to the frameworks inbuilt static methods.

With all the applications configs neatly wrapped in one place the code starts to become more maintainable. Yet are we just replacing one evil with another, if somewhat lesser, one? The problem I often see is this Config class finds itself passed from class to class flouting it's promiscuity and leading writers of articles on config objects down paths dangerous and full of inappropriate metaphors more commonly favoured by the ruby crowd.

There is no doubt that it is useful to collate all of the configurable elements together in one (or more) common classes. What should be avoided is leaking this concept through the whole domain and kicking it from object to object like a dirty old data bag which everyone rumages through, unchecked, pulling out stuff on their own whim. Config is an implementation detail which the majority of the application should have no concept of and (like any other such implementation details) it needs to be locked firmly away like a lump of kryptonite in a lead cased layer so it can no longer leak and permeate and weaken your SuperApp.

Let's think about this in a BDD fashion. When your class want's a ghostbusting url who's it gonna call (they have a webpage now dinyuknow)? string Config.GhostBusterUrl? Or perhaps the GhostBusterUrl class? If your class needs a typed collaborator with certain logic and behaviour (even if it is just supplying a valid url) then that's what your class should get. Not some smelly generic config object that dishes out dirty unparsed strings. The tests should be enough to tell you I'm telling you the right thing: several lines of priming the mockConfig are replaced with the simple supply of a GhostbusterUrl.

So how do we get the GhostBusterUrl from a line in config to those poor desperate classes who are being terrorized by spooks? With a little Tell don't ask and a container. Turn the config object inside out, make its role to inject into the application the dependencies it wants to supply. If your config believes it has a GhostBusterUrl then get it to construct one and register it (along with anything else it thinks it might have of use) into the container when the application tells it to. Not only does the config object gain some of its dignity back (some other classes were calling it nasty names, I know, it's the new millennium, but some people still hold the old fashioned ideas) and is safely kept right to the very edges of your application where no one can get at it.

Another positive side effect is that now your config object is giving birth to loads of lovely typed objects that actually do real stuff rather than having wandering hands pluck at its primitives (I know but I did warn it was a dangerous path laden with tasteless innuendos, I was bound to trip). Though of course, be careful not to pollute your codebase with a million classes that simply represent strings and do nothing else (it's a fine line and one you'll need to argue between me, myself and I). It's also one of the easiest refactorings you can do to your code. Give it a go (and if you object on the grounds of their being loads of small types then please find the closet crumbling procedural bridge and throw yourself into the polluted sequential river below).

Value Driven Development

At the beginning of the year I blogged about using value as a measure of productivity. Since that time I have had the opportunity to refine my thinking and be influenced by some other ideas. I want to revisit the original idea of using value as a measure. I may be guilty of repeating myself a little but it's a small cost for clarity.

The concept focuses around one very simple idea: the primary objective of a project is to deliver value. The more value we deliver the more successful the project is. If you agree with this then my next idea shouldn't be too hard to grasp: the primary measure of a project is value. Not velocity, not story points, flow, throughput, cycle time, capacity, cyclic complexity, code quality, but value, pure and simple. Of course we can use the other measurements to help us discover more effective ways of delivering value but on their own they tell little about how much value is being delivered.

Strangely, despite the fact that agile and Lean and Kanban and all of it's derivatives tell us how important value is none of them propose explicitly measuring it. Agile proponents often argue that it is their focus on delivering value early and often which makes them unique over conservative methodologies but how do they know? I've never picked a story off the wall and been able to tell what worth the customer gains by delivering it - ten, a hundred, a thousand, a millions pounds? Hell, who knows? The BA? The PM? The product owner? And how many projects out there can raise their hands to the question "How much value has your project delivered in the last week/month/year?" So if we're all so very value focused in our lean and agile worlds how are we not answering these simple questions? Are the statements about delivering more value pure assumption? How do we know that when we make 'improvements' to our processes the effect is to deliver more value? Or is at case of substituting value with story points mapped against velocity? If so how do you know that twenty-five story points to improve online advertising generated any revenue? Or is it a case of take the points and run?

Before we can confidently improve the amount of value we deliver we need to be measuring the value we are delivering. The first problem that presents itself is how do we do this? Value is hard to quantify and even harder to estimate. Which sound awfully similar to the problems with size so why not transfer the techniques used to estimate of story size to estimating value. Start simple, use relative sizes, but instead of t-shirts maybe use precious metals (aluminium, bronze, silver, gold, platinum etc.) or any other metaphor that helps the team visualize features in relative worth and as with size estimation it's an estimate, it doesn't have to be exact, the important thing is that we have some sort of gage and that we track it and refine it.

Once stories have value estimates they can go through the normal delivery process: estimate size, queue them up, stick them in iterations (or do some funky Kanban pulling on them). Except now every story card has it's most important piece of information branded to it: it's value. During prioritisation the platinum card which is the size of a small T-Shirt should obviously be delivered before the XXL bronze one. When stories are worked on people know that it's worth putting that extra effort into putting the plating on the platinum one but perhaps the aluminium story requires a more pragmatic approach than spending two days of cross functional refactoring to perfect it's design. During standups if someone says a platinum card is blocked the whole team knows it's important to get this sorted and coming up to a release with three gold stories complete but a stubborn bronze proving difficult to get across the line you may be more tempted to make the call to leave it out this time and hell to the size points. Very quickly every part of the process focuses around the amount of value being delivered, every decision is being influenced by the value of the story, not by it's size, not by how cool it is, not how much can be crammed into an iteration, but by how much value this piece of functionality is actually going to bring to the customer. Now that's being agile!

The purpose of putting values on stories is to measure how much value has been delivered. Now let's be clear about what delivered means: out there, in production, available to everyone who should have it, in full use. Basically, not dev complete, not in UAT, not in live pilot to a handful of select users, but out there live and in concert, fully functional, fully delivered, bringing value. Until then the functionality is worthless; until it is delivering value then you ain't delivered any value. Once it is though start totting it up: add the value points onto the delivery (note: delivery not iteration - iterations cannot deliver value unless they release), repeat for the next delivery plotting the value delivered on a chart and voila: the project's true velocity: how much value, for each delivery, it has been delivering.

But what if the value didn't quite get delivered? What if the functionality was released but with a bug which somehow hindered it's full capability? Maybe it is too slow, too difficult to use, doesn't do quite the right thing etc. Anything that prevents the application from delivering all of the value it promised is an issue (regardless of whether it is a bug or 'a feature'). However by resolving the issue the lost value can be reclaimed. By fixing the bug, performance or usability problems, the project claws back the value it lost when it didn't get it quite right first time.

It's important the project's value velocity reflects this. So when a fix is raised it is given a value which is then deducted from the velocity: this is because the full value of that feature was not delivered. Once the fix is in use and active then the value points go back on. Personally I find this one of the most powerful ideas behind measuring the velocity of value as the bad habits of delivering bug ridden functionality iteration after iteration by dismissing fixes in order to keep earning the points will not get you anywhere as the value velocity either struggles to move or even, in extreme cases, goes backwards.

I think there are many other benefits to be gained from using value as the primary measure that are yet to be discovered (possibly placing value onto technical debt? Though I'm not yet convinced of that one). Explicitly placing value at the forefront of the project gives teams the power to optimize themselves and discover new ways to improve their efficiency. And it isn't necessarily limited to individual teams: projects themselves should be measured in this way with teams 'contributing' to value - not simply earning points for delivering 'their part' - breaking down silo attitudes by finding ways to effectively work as a whole so value can be delivered.

I think it's pretty exciting stuff. Of course all the other metrics out there may or may not find use, dependant on whether they are contributing to the delivery of value or not, and holes will be found in the technique, but overall I think it shifts the debate and discussion forward to a entirely new level of thinking. It will allow us to reach a whole new level in the Value Enlightenment.

Getting the measure of Value

In my previous post I made the claim that we should not be measuring velocity (to the customer at least) in terms of functionality and effort delivered but in terms of value delivered. In this post I will expand on this by explaining how value and velocity work together.

The idea behind measuring velocity in terms of value is simple and, at its core, doesn't greatly differ from the way in which we measure velocity now. Functionality is still broken up into stories (or MMFs if you prefer the Kanban approach) and they are still assigned points. The subtle change comes from what the points represent: not the size of the story but instead the value of delivering it.

The first problem with this is how to assign points of value to a story? Value is something that can be hard to define, that is variable and may be measuring different things (monetary value, visitors to a web site, time saved). Story sizing has similar difficulties so it makes sense that the solution is the same: estimation. Value can be estimated by having the team discuss a story and assign points to it. It doesn't have to be accurate - it's an estimate - but it does have to be representative and the more the project goes on the better it will get. The important thing is to decide on what a unit of value represents and as a team (customer, users etc.) agree to it. But guess what? Estimating the value of something is a core activity of most businesses and they should be both used to it and good at it to.

Which begs the question what should a unit of value be? This is highly dependent on the domain and type of business. In most domains value could simply be represented in terms of money: a story to allow people to purchase your widget online is the profit of the widget * the increase in the number of sales. In some domains it might be different, for example a web site (wikipedia for example) may measure value in terms of how many visitors it gets. Some domains may be more complex and have a combination of the two (or more): number of visitors and the money raised in advertising. In these cases the solution would be to find a common representation: maybe one visitor is worth £10 in advertising revenue (for example). Some domains may even express value in a way entirely different (schools: exams passed, hospitals: lives saved etc.). The important thing is that what the unit of value represents is consistent and meaningful.

Now stories have been assigned points which represent their potential value how does a team earn (or realize) them? If you read my last post you may remember that I started by explaining that in Lean you don't earn value until a customer receives it. This rule is extremely important and velocity should work the same way. Until the customer (or end user) has the functionality out there, working and in use the team does not get the points. Functionality not delivered, only partially delivered (delivered only to a pilot group etc.) or released but unused is not considered delivered - it is inventory - and therefore does not realize its value. Until the customer has realized the value then it is not recorded on the velocity.

Recording velocity in this way ensures that it represents the reality as perceived by the customer and aligns it with the rules of Lean. It should also change the way teams are measured and motivated: their focus switches from getting functionality out of the door to genuinely delivering value into the hands of the end user.

Measure for measure (why we're doing it wrong)

In this post I will outline that the way in which we measure productivity in software development is still flawed - even in agile.

One tenet of Lean is that the core activity of production is to deliver value to your customer. A second tenet of Lean is that something is not finished until the customer receives it. Only then do you record it on your balance sheet. Combining the two is a simple statement: the purpose is to deliver value to the customer; it is not delivered until the customer receives it.

Everything else is unimportant. Anything that is not contributing to the first tenet is waste (muda) anything that is produced but isn't delivered to the customer is inventory and inventory is also waste.

The relationship between productivity and value in Lean is fundamental. If something which is produced yet not delivered doesn't have any value then we have not produced. Regardless of how many hours worked, how many items output, how much blood, sweat and tears, if all you have created is inventory then you have as good as done nothing.

Measuring productivity has been a holy grail of software development. It sits at the heart of many of Agile methodologies. Thanks to essays such as The Mythical Man Month and with tools such as estimation, iterations, velocity and burn down charts Agile has moved us beyond measuring productivity with the highly flawed method of counting hours worked. Instead it breaks projects into units of functionality (stories) which are given sizes (points) relative to their complexity/effort. Productivity is then measured by totaling up the measure of effort once the functionality is completed and 'signed off'.

Using velocity to measure progress and productivity has resulted in a revolution in thinking. Using velocity teams are better placed to manage and schedule workload (scope) and much more able to accurately report what has been done and what is left to do. The result has been much more predictable and achievable project timelines.

However there is a disconnect between the customer (and user) and the producing team. The producing team is measuring its productivity based on how much effort is realized into working software. The customers success will be ultimately measured (whether they are aware of it or not) on how much value they have delivered (note the tense: not how much they potentially will/may deliver but HAVE delivered).

The result of this disconnect is that teams responsible for production are working within a model which is not based on reality. The real value of what they are producing is nowhere to be seen in any of the measurements or reports output by the team. The result is their measure of success being out of line with that of the customers. The team ends up working to its goal to maximize output of functionality but not to producing value.

Teams need to change the way they measure productivity to bring it inline with the way the customer is measured. To achieve this is simple: measure velocity in terms of value, not effort. And by keeping with the Lean tenet to only record value when it is delivered to the end user.