I am NOT a geek

I am writing this due to provocation by an number of colleagues who seem intent on labelling me as a Geek. Making claim to bullying of this character in a technical organization such as ThoughtWorks may sound implausible but my public derision finds root in the rallying call I made (on an internal mailing list): I AM NOT A GEEK.

So why would a respected developer such as myself risk a well earned reputation with such a perilous statement? And then why would I go onto circulate the claim amongst the wider community of readers on the ThoughtWorks blog roll (I am still on it aren't I?)?

It has been a dark secret that I have been keeping from all but my closest colleagues; only my friends and family have really ever known the truth. Coming out of the closet on an internal mailing list may have been tough, but - apart from the odd comment from the usually insensitive and ignorant (well emotional intelligence was never the geeks strong point) - it has been a liberating experience. No longer do I have to pretend that I actually care about the resolution of iPhones versus HTCs, no longer do I have to feign interest in the difference between the generics implementation in Java and dotnet. And, for the first time in my career, I have felt that I could style my hair in a moderately fashionable manner. And the support has been incredible: BAs and QAs and even other devs have quite literally wrapped their arms around me in deep sympathy to confess that they too couldn't work out what pub everyone was meeting at every night until they twigged that all the socializing occurred within the realms of the World of Warcraft servers.

Though one cannot make such a claim as I have without some hard evidence. After all, over a decade in the IT industry would suggest that my declaration is fraudulent. But no, the truth is that my participation in the geek subculture is limited at best. I don't like science fiction or fantasy, I find Lord of the Rings far fetched and ridiculous, I own not a single gadget (no honestly not even a fancy phone), I perceive the playing of computer games to be a waste of my life, I don't like Star Wars (but I want to ride my bicycle). On the other hand I like being outside - especially if involved sport - adore music (and no, not Heavy Metal), enjoy the company of others and I can relate to women. Some may go as far as describing me as a "people person".

To sum it up, once out of the workplace, technology plays a very minor role in my life, except in the places where I can leverage it as an enabler (for typing this blog on, or finding cinema times on the Internet). Careful not to misconstrue me: I believe technology is a great enabler for many things, from opening information to social justice. As a power in society it is arguably the single greatest development since the printing press. However, if it all disappeared overnight would I cry? If I was born fifty years ago would I have struggled to find my calling in life? No, no and definitely no. So hence my assertion: I AM NOT A GEEK.

Please, before you make your judgement to find my decision wise or foolish, let me impart you with my reasoning. Firstly I hope my actions will path the way for others, but also, I believe this selfless acts of hari kari will be the catalyst to make our industry a better place.

Within our great industry there is a template, an expectation, a stereo-type if you like, of what a typical IT worker should be. So deep it is within the culture I should not need to draw this caricature in too much detail but, let's throw some words out there: spotty, socially awkward, coffee drinking, light hating, arrogant, emotionally unintelligent male that spits when he talks and is probably wearing a t-shirt - for several days and smelling a bit - with some witticism along the lines of '/me 0mwz j00' or something equally as mystical - possibly even written in Klingon - to the majority of society. For cultural references consult The IT Crowd, that Russian guy in that James Bond film, Big Bang Theory or any other computer guy you can think of in any mainstream film or TV comedy.

Before continuing let me state, on the record, that the number of individuals I find that meet this template in IT are ever decreasing. Of course it was considerably greater a decade ago when they locked us in the basement but since we've been allowed to occupy floors that let in natural light, like Gremlins, they've bubbled up and exploded (I believe getting them wet was equally dangerous).

Regardless, the stereotype still persists. My concern is this stereotype is a barrier to the IT industry reaching its full potential. It has a number of negative effects: it deters potentially able people from the industry (women for example, but not exclusively); it normalizes a set of behaviours which are damaging; and it prevents peoples development by narrowing their horizons to a limited template.

Though there are people out there who not only relate to the stereotype but will defend its place within the unique culture of IT. They will state that the attributes to be good with technology are exclusive to geeks and therefore they are the best people to run IT - yet looking at the state of the industry I'd say you've not done a very good job so get out of the way and let someone else have a go. But this is a complete fabrication. Without trying too hard to burst the whole indulgent We are Special myth and point out that lots of normal people in other industries have these skills in abundance too there truth is that there are many skills that IT people, in particular developers need, to enable technology in a manner that brings value to the business which are both arguably more important and that the stereotypical geek lacks; whether communication skills, awareness of business, empathy and understanding for users, prioritization, even the ability to solve problems WITHOUT technology. Of all of the key skills those attributed to the geekier sensibilities are, in modern development, the least critical. Wizzy maths and clever algorithms do not form the heart of the vast majority of software. Thanks to the Agile movement and things like BDD and Domain Driven Design, the business is at the heart of software development.

Modern software places the emphasis on those other skills that the stereotypical geek doesn't tend to have. Normal people on the other hand do. And what's more, plenty of normal people have those other problem solving skills that geeks believed made them stand out - I'm sorry but attention to detail, inquisitive nature, analytical skills, logical thinking, problem solving etc. are not, and never have been, attributes exclusive to the introverted middle class white male plagued by OCD and touched by autism. The simple truth is that non-geeks can and do make great developers and often even better developers than geeks (in fact some of the worst developers I've worked with in my career have been the geekiest and the inverse has also been true).

Let me restate this for crystal clarity: you do NOT need to be a geek to be a great developer; you don't even need to be a bit of a geek - not even give into the odd geeky impulse and no, no, no you don't even need to try and fit in and do geeky stuff - 'cos it doesn't make one tiny bit of difference to how good you are as a developer.

In a by-gone age when geeks were the main consumers of technology then the natural geek instinct of the developer was enough to satisfy any demands the audience may have. One bunch of geeks produced stuff for other geeks to consume. What the first geeks produced made perfect sense to the other geeks. But that isn't true any more. Geeks are a minority of users. Do you want evidence? Go and look at how many users are running Linux (no true geek would run Windows): the number is insignificant. The majority of technology in the majority of the world is used by non-geeks.

To write effective software for non-geeks you need people who understand them, and unfortunately geeks tend not to. The things that geeks prioritize are often not what non-geeks prioritize. Remember when everyone derided Macs for being 'simple' and how dumb Apple users were because they couldn't manage more than one mouse button, whilst MS covered every possible surface of the mouse with different ways to click and Opera invented 'mouse gestures' (yes, only a geek could have done that). And then Apple go and produce a music product with only one button and a wheel? What geek would have come up with that idea? No, that required someone who understood how normal people want to interact with a music player (the geek stood back in shock that normal people could manage to rip, manage and sync their CD collection without any guidance just because were given an interface that made sense).

Now my tirade is over, my blows made, wounds inflicted, oil spilt and lit, I wish to make my peace. The future of IT and software development needs to be one where the creativity, the passion, the human element of building brilliant software and the skills and attributes required to achieve this are encouraged and nurtured regardless of it's source, geek or non-geek. The industry must stop holding the geek up as the exemplar and alienating, or degrading, those that do not meet or wish to conform with this caste. With each generation IT moves further into the consciousness of the mainstream and in doing so its appeal will grow beyond those great pioneers, the geeks, upon which today's industry is founded.

So here's to an industry which attracts the brightest and the best from the entire psychological and sociological spectrum and a long overdue fairwell to that which limits it to that one tiny, specific character with its limited range of skills.

UK politics and the battle between Old and Social Media

Firstly this content is about politics.  Though the purpose of the post is not to 'push' any specific political agenda but to examine the role social media has played in the sudden, and unexpected rise, of the Liberal Democrats and how that too has impacted the 'old' media.

For those of you reading this who are not in touch with the current affairs in the UK, currently there is a general election being held it appeared that the 13 year majority of the current Labour government would be seriously contested by the opposition Conservative party.  Then, on the 15th April, the UKs first live televised debate went out with the three main leaders.  The result was that the previously marginal Liberal Democrat party were declared the outright winners.  All hell broke loose.

The TV debates are an historic first for the UK but something else that is also a historic first is the use of social media.  Of course, over in the US, the effective use of social media gave it claim to Obama's victory.  I don't think that this was a fact ignored by the election teams in the UK as politicians ran to pick up their own Twitter accounts and Facebook profiles but the nature of on the ground campaigning hasn't altered quite so dramatically.

Traditionally, in the UK, the main stream press has held the single greatest influence over elections and this sudden swing is evidence that their grip is breaking.  There is no doubting that the televised debates have been the single biggest influencer in the dramatic shift in political ground over the last two weeks however there is a real war between social media and the mainstream press.  After all, the UK has taken fondly to social media, being only second behind the US and with London having the highest percentage of Twitter users than any other city.

Although it is easy for some in the press to dismiss social media users the intensity of UK usage compared to other countries is no doubt going to have an impact on the end result.

Real evidence of this was available even before the first leaders debate had finished.  Looking at what was trending on Twitter gave a solid prediction of the outcome.  Nick Clegg was far in front with Cameron and Brown just nipping in at the bottom of the top ten terms.  Ironically Clegg was kept off the top spot by the hash tag #iagreewithnick.  A few minutes after the debate had finished Nick Robinson delivered the catchphrase live on BBC.  For Twitter users he was only confirming what they'd known for the last ninety minutes.

It was in the week after the first leaders debate when the power of social networking was truly demonstrated.  After Cleggs surpirse win the right wing press went into overdrive to discredit the new upstart.  What resulted was a simultaneous barrage of front page smear campaigns across the four main right wing papers (Daily Mail, Daily Express, Daily Telegraph, Sun).  Twitter users reacted instantly with disdainful satire by manufacturing a series of ridiculous claims and blaming them on Clegg.  Before lunch time the same day #nickcleggsfault was the highest trending hashtag.  And it stayed there, trumping even #leadersdebate, for several days.  Even if the press couldn't pick up on the subtleties of that then the fact that "murdoch lose britain" was trending a few positions  behind should have been a clear enough message.

Then the Murdochractic media made its second mistake when  Sky TVs host Adam Boulton questioned Clegg on one of those very front pages.  It was outside the rules and hundreds of tweets popped up within seconds protesting against it.  These soon translated into over 100 complaints to Ofcom and of course news coverage of the slight.  From that moment Twitter had a microscope over Sky's coverage of Clegg and was even accusing Sky of deliberately 'cutting' Cleggs shots so he couldn't 'look into the camera' whilst giving Cameron more opportunity to do so.  Of course this is pure conspiracy but it was a small enough seed of doubt that people believed deserved to be retweeted.

And this seed of doubt had serious consequences.  Once Sky announced that YouGov had David Cameron in first place Twitter reacted with disbelief.  Within seconds people were posting that they'd taken part in the YouGov poll and felt it was biased, they were even posting screenshots of the survey to prove it.  This lead to accusations of 'push-polling'.

And it didn't end there.  The day after the Daily Mail were claiming on their home page that their polls showed that Cameron came out on top.  But a Twitter user had observed that the Mail had withdrawn their original survey which had Clegg top and republished it.  The result was a Twitter call for poll-jacking to put the numbers 'right' (which they eventually did).

What is interesting about all the above is how UKs Right Wing press inadvertently waged war with social media users.  The press forced people into camps: with us or against us, and to against us was to be with Clegg.   And so social media users reacted in the same way they had against what they perceived as the mainstream bullying and manipulation of Simon Cowell by campaigning to deny him the Christmas number one by replacing it with Rage Against The Machine anthem "Fuck You I won't do what you tell me".  Except this time they are reacting against the Right Wing Press and Clegg is their Rage.  Evidence?  Well a facebook group called "We got Rage Against the Machine to #1, we can get the Lib Dems into office!" that has already collected over 150,000 members - which is more than any political party.

Why did they make such an error?  Well perhaps it's because the press like to dismiss social networking as something for 'young people' and 'young people' don't read newspapers and, so they thought, young people don't vote.  But, thanks to social networking, a kick-back against the 'old' media and a some dramatic TV coverage it looks like that all might change.

Of course the truth will out on May 6th when people turn out to vote.   The Right Wing press still hold considerable influence over the British public, let's not forget that social networkers are a specific, though not insignificant, demographic. But, though it's hard, look behind the politics and you'll see, just like the XFactor Christmas single, there is a clear theme from the 'internet generation' that they won't be dictated to by the traditional media and now, thanks to social media, they can quickly mobilize and push back.  Perhaps the press should take warning from the Rage campaign, because when they do push back, they push back on mass, and they push back hard.

nPower and my UX nightmare

I saw npowers new Wallace and Gromit ads for their energy monitoring device.  Apparently just ask and you'll get one.  Brilliant.  After lots and lots of web searching and scratching of heads I found out the way to do it is register for a paperless billing (which I was already on) - this is of course once I'd worked out how to log in (tip: tiny link top left corner in a sea of red).  More confusion and then I find you just have to edit your details (it took me a couple of reads of the clearly written English to work this out):

To request a FREE smartpower electricity monitor, set up paperless billing, or activate other smartpower options, simply add or edit your account details now 

First, what's wrong here?  Do an ad campaign and then make it hard for existing customers to follow through? You'd at least thought that all you have to do is log into your account and see a picture of Wallace and Gromit inviting you to click the big Apply for Smart Power Monitor?  No?  Well, obviously the marketing guys at nPower don't think so.

But it gets worse.  I try to edit my details and the page blows up with a load of error messages about missing information.  But there's only check boxes, the stuff that's missing is your account details? Huh? There's no place to enter them either (and why do I need to enter something you should already know)?   So I try to re-register for paperless (I think I'm smart here, I'll disable it then re-enable it).  This time the fields it moaned about not being filled in are there and blank and apparantly I have to refer to the bill npower sent me for the details (even though they are all on the previous page next to the link I clicked to get here).  But it blows up, this time horredously, logs me out and gives me a 'sorry, our system appears to be experiencing problems'.  You don't say.

So I email npower to ask them to sort it out.  Explain clearly that the website keeps blowing up every time I try register for the monitor and ask them if they could just sort it out for me behind the scenes.  A few days later I get this reply:

I have lokked into this for you and customer do have to apply for the smart power monitor on line please follow the information below to apply ofyou still have issues please call our web support team direct on 08451663443.  

Did they read my message?  Did they read there's before sending it? Can they read?  They certainly can't write.  Great customer service nPower.  I feel really valued.  Well done.

A Thoroughly Modern Developer

The world of development has changed rapidly over the last decade or so.  Thanks to Agile, great tools such as TDD, O/R mapping, dynamic and functional languages and a million other little things, the way companies approach development is changing.  A bright new future awaits where only the true veterans wince as the CEO discusses going waterfall when promoting the merits of the new hydroelectric generator.

Inevitably, as development changes so will developers need to adapt.  The stereotype of a socially awkward, green screen loving, mouse hating, hacker who wears black-now-grey jeans and t-shirts that state "127.0.0.1 is where the heart is" for a week at a time and mumbles through the pizza crumbs that drop in piles from his beard onto his ever rounding belly, is not going to cut it in this new world.  Future projects will be run on the basis of success and that means you can't forgive someone's shortcomings because they are a "code wizard".

So what sort of developer does cut it at the beginning of this new decade?  What sort of skills are you going to need?  Well, funny you should ask-

Domain Knowledge

Or, knowledge of the business you are working for.  Systems are more complex than ever and businesses increasingly rely on them.  Regardless of the general mistrust of IT it has moved right into the heart of business, providing the engine, one without which many businesses couldn't survive.
To build successful systems The Thoroughly Modern Developer has a thorough knowledge of the businesses intention and the value being delivered; it isn't good enough to rely on a BA and a Development Manager to 'translate' business speak into dev syntax.

Eric Evans goes into this in detail in Domain Driven Design.  Designing and building a system is a collaborative effort between the domain experts and the developers to create a common model (or a ubiquitous language).  If you don't understand the business how can you model it?  But it goes further than that; if you don't understand the business value how can you deliver it?

QA

Back in the old days testing meant running up the app, clicking a couple of times and then waiting a month or two before the testing team raised a list of bugs for the junior devs to pick up and fix.  It took XP to change our attitudes on this.  TDD meant we wrote unit tests and verified our systems with at least some code.  Agile put QAs at the heart of the development process and bugs where fixed at the end of every iteration, but the 'throw it over the wall' principle was still there, just shorter.

The Thoroughly Modern Developer takes responsibility for her own quality, she cares more about meeting the acceptance criteria in a bug free fashion than anything else.  This makes the QA's role even more critical as they must continuously guide and help the dev but they will no longer be reduced to simply checking they've done the work.

Usability

The average developers idea of building something usable is akin to [the car Homer designed].  For some strange reason even the simplest of tasks, such as getting a column of text boxes to line up, seems to be a feat of incredible endurance.

But usability is crucial.  The first developer I ever worked for told me The users don't care about how clean and beautiful your code is, they never see it, but the smallest spelling mistake on the UI and they're on the phone.

The Thoroughly Modern Developer builds systems with usability in mind from the start.  Sure, she's no expert - but she knows enough simple rules to get her by - so she works closely with the UX person to ensure what is being produced is usable not just functional.

Polyglot

The Thoroughly Modern Developer does not define her role or skillset around a single language (or worse, a single toolkit - i.e. ASP.NET Developer).

She is language agnostic, and she has experience of a number of different languages, using different paradigms (OOP, functional, dynamic etc.) and her level of understanding goes beyond syntax.
The Thoroughly Modern Developer will choose the best tool for the job or circumstance.  Throw her a language she's never worked in and she has no issues about picking it up.  Or put her on a project where she's expected to work in two or more different languages and she isn't phased.

In Code Complete Steve McConnell talks about Programming "into" a langague over programming "in" a language.  The Thoroughly Modern Developer does the former.

Value Driven

To every new feature, ever request, every line of code, the Thoroughly Modern Developer asks the same question: how does this deliver value or what's the value of doing this?  She's obsessed, she keeps going on about it, it's all most as if it's all she cares about.

Which it is of course.  To the Thoroughly Modern Developer value is the sole purpose of her job.

A People Person

Oh yes, it's that horrible phrase, one that causes many devs of old to run away and hide behind a wall of cabinets filled with specification documents.  The Thoroughly Modern Developer, on the other hand, likes people, gets on with people, can talk to people.  She doesn't need 'Relationship Managers' or 'Business Interfacers'; put her in a room full of real people and she'll hold her own without spitting when she talks or snorting Beavis and Buthead style when someone uses any word, or collection of words, which bear a vague resemblance to bodily functions.

Why is the Thoroughly Modern Developer such a people person?  Because she understands that in order to build quality software, that delivers business value she needs to talk to people, all different sorts of people, all the time.  Whether it's to find out whether the button should say Save or Create or to explain to non-technical people why it took longer to integrate the zobertron with the phlargbleg initiator (of course the Thoroughly Modern Developer would never have come up with those names but she's still got to get along with the old skool) and for the client to be confidant.

People are what makes a software project successful and if you can't do people you can't do software.

Facilitation

The Thoroughly Modern Developer often finds herself in the middle of difficult and complex situations.  Because she wants to get the system right she has to raise difficult questions about the way the business works.

The Thoroughly Modern Developer needs basic facilitation skills.  She needs to be able to lead a group of people through creative and difficult exercises. To get the right answers you have to keep people on track, resolve conflicts, remove distractions, know when to call time-out, get them to make a decision.

Has "other" interests

For athletes cross-training (training in your non-core sport) is a essential technique to ensuring you excel in your core discipline.  This is no less true for intellectual disciplines and even more true for creative ones (artists/writers/musicians have known for centuries the importance of pursing other arts - think Da Vinci).  If your entire existence is writing software then you are greatly narrowing your reference points and are more likely to suffer from boredom or stagnation.  For example many prominent developers have blogged on the strange relation between development and music (as a failed musician I entirely concur).

Personally, I have found that long distance sports have allowed me to strengthen and develop a lot of essential development skills: focus, pace, general discipline; not to mention the health benefits that keep my brain active and my energy levels high.

But not only does it benefit your work but it makes you a more interesting person, which is always useful when talking to 'real' people like the users. So do your self a favour, when you get home do something that doesn't involve the computer.

Understands that technology isn't important

The Thoroughly Modern Developer has a healthy cynicism towards technology. If something can be done without technology that's her preference and she'll push for it.  She actually wants to write less software; complex clever gadgetry and features fill her with a great sense of foreboding.

If she was a developer at Timpsons (who have no centralized till system), she'd be strong in resisting all efforts to introduce one.  She only cares about technology if it offers real benefit, if it provides genuine value or is essential to the business or user.


She's great is the Thoroughly Modern Developer.  She's so awesome people high five her every time she gets up to make a cup of tea.  And yet she's so humble with it.  If only I could be just like her (sigh).

Fear of the new (how TDD and DI encourage bad practice)

When I first started doing TDD back in the mid-naughties it changed the way I programmed dramatically.  It took me a while to get it at first, how can you test one unit? until I discovered the tricks of dependency injection.  Now my code was shaped in loosely-coupled wonderness and I felt invincible.  I thought I had reached the pinnacle of OOP. Everything suddenly became so easy, it was well tested, easy to change, easy to wrap, swap implementations, decorate, whatever, I was throwing the best design patterns out with such ease that I barely needed to refer to the Gang-of-Four anymore.

I went on in this vein for a good year or so.  And as the millenium's first decade entered early evening and its bright lights began to fade I took a step back and looked at my code.  My god, I thought, this is terrible, what the hell have I been doing? And it was terrible, hundreds of interfaces named somebody's Manager, Persister, Service (oh so many classes called Service), Validator etc. etc., all with a single namesake class implementing them.  There were tens of classes whose purpose seemed to be to co-ordinate these little buggers - for they were little, ten or so lines each (oh how I thought I was doing a grand job) - and they would hand off to other similar classes who co-ordinated other classes and it was turtles all the way down.  And in amongst all of it there were about a dozen classes that had real names like Customer, Account (but even they had interfaces too) and, aye how the sting of memory pains me to this day, they only had getters and setters (it's the shame, you never get over the shame).

It was with great pain I realized for the last year of my professional life I had been producing procedural code with all the idealistic misguided joy of a card holding Russian worker pouring cheap cement to build pointless roads.  I knew the culprit alright: TDD and DI.  I had gone decoupling, unit testing mad.  The result was an irrational fear of the new keyword.  As far as I was concerned the new keyword was banished to the dark depths of Containers, it was a clumsy language feature never to be used again.  But what I had instead was a dependency graph made up almost entirely of stateless singletons.  A simple Ruby script could have replaced every injected dependency with calls to static methods and sure the application would have lost its testability, it would no longer be decoupled, but essentially, at it's essence, and in terms of structure, it would be the same.  TDD and DI had simply given me a fancy way of decoupling static classes.

The new keyword is a wonderful thing.  It lies at the heart of OOP.  Its sole objective is to create a new instance of an object.  If you don't use new you can't create new objects and if you can't create new objects you can't write object orientated code.  And the other fundamental thing about objects is they encapsulate behavior and state.  Singletons don't.  Singletons are procedural.  Regardless of whether they are loosely coupled and injectable.  Real OO classes have private fields (the fewer the better) and methods which do things based on those private fields, sometimes based on arguments passed in, but always, always in the context of those private fields.  And the only way, the one single true way you get values into those private fields is via the constructor and that means using the new keyword.  Let's put it clearly: this 'new ValidatedCustomer(customer).IsEmailAddressValid' is OOP. This 'customerValidator.Validate(customer).IsEmailAddressValid' is procedural.

Now I was writing code using TDD and DI that was object orientated code.  But my fear of the new was still there.  In order to maintain the injectable, loosely-coupled gorgeousness I had begun to do a horrible thing.  I started injecting factories everywhere.  Sure it was an improvement but there was still something horribly smelly going on.  After all the factories were essentially static, singletons whose sole purpose was to delegate to the new statement.  I mean there's single responsibility and then there's craziness!

So I started doing something I thought was really bad: I created objects in my classes! But I wanted to keep loosely coupled and all of that wonderfulness.  This required a dramatic shift in thinking.  Remember when you first did TDD and how it really hurt because you had to structure programs in a different way?  Well it was like doing that all over again.  Every time I started doing something I had to spend half an hour thinking, how do I make this work?  I know I'm right but how?

I learnt a few lessons:

• Not all classes are about interactions, sometimes they are about results.  Mocking everything out for the sake of it doesn't gain you anything.  Use state based testing to assert the end results not interaction testing to assert what is happening to achieve those results.  
• It's OK for your collaborators to new up something if you ask them, so you can say fileSystem.NewFile(name) if you need to.  
• Collaborators don't always have to be passed in the constructor: they can be passed as arguments to methods as well, so you can say new File('myfile.txt', 'Some text').SaveTo(filesystem).  
• If a class is unavoidably all boilerplate and co-ordination consider using integration tests, after all mocking the interactions of boilerplate code doesn't tell you anything useful at all.  
• The container shouldn't contain everything, it should be the things that generally require configuration or are likely to change: databases, filesystems, web services etc. Rarely do core domain classes or business principles need to be 'switchable'.

Desktop Mashup

The concept of mashups generated a leap forward for web applications. By exposing data and functionality in an open and standard way other sites could use it, mashed up with other data using the functions from a different site, to provide services beyond the imagination of the original owners. It presented a significant shift in thinking: that data and function was more powerful - and arguably more valuable - if applications outside your own could leverage it.

This shift in thinking is not only limited to the web; desktop applications can also leverage these ideas. Locally running rich client desktop applications can be enabled to expose their features to other local, or even distributed, applications.

In fact, in Unix, this concept has been around for decades in the very simple, yet immensely powerful, form of software pipelines. By chaining the output of one program's data (via stdout) another program, with it's own distinct functionality, can then manipulate the result. This simple technique has enabled Unix developers and administrators to solve an infinite number of problems from a small set of tools.

Between these two extremes is the idea of desktop mashups. But instead of following the pull nature of the web, or the push nature of software pipelines, desktop applications can take advantage of the event driven nature of GUI systems and thus create a rich desktop experience. Thus separate programs, with distinctly different functional offerings, can leverage each other to provide functionality beyond the others original intention.

For clarification consider this simple example: you have two separate applications: one which rips DVDs, another which catalogues them ready for playbook (e.g. iTunes). When the DVD ripping applications completes it sends a message to say a new movie is available. The cataloging application - which has no direct knowledge of the first, receives the event and interrogates the new movie and adds it, and its metadata to the catalogue.

Of course this is nothing more than Event Driven Architecture which is hardly a new concept. The move to mashups comes when we create new applications from the existing apps functionality. So, in our simple example, we create a new application which fetches film reviews from imdb. When the user inserts a DVD from ripping it fetches the review so the user can decide whether they should go ahead. When the user does rip the DVD the review application updates the catalogue application so the review is part of the movie's meta.

Enterprises are ripe for desktop mashups. A significant number of problems for users is the lack of integration between applications, whether they are different internal apps or even third party. Another advantage, especially in large enterprises, is that development teams can focus on developing small applications that do their jobs well, or integrating with legacy apps, and mashing them together rather than struggling to develop all-in-one god apps.

Immutable Wrapper

I am a huge fan of immutables, especially for value objects. Even in clumsy static languages like C# and Java that seem to throw every obstacle in your way to deter you from using them I find the benefits outweigh the costs.

Sometimes though a mutable makes sense. Whether that's because of the restrictiveness of a framework or the paradigm of the language. But what if we don't want to loose the power of immutables behind the scenes?

Introducing The Immutable Wrapper Pattern!

The Immutable Wrapper is a wonderfully simple pattern because there are only two things you need to do.

1. Create a class with only one mutable field
2. Wrap the immutables functions.

Here's an example. You've got a copy-on-write list implementation but your binding it to some GUI control that gives you a big headache if you try and point it to a different data source.

Here's your immutable:

  class CopyOnWriteList
 {
   private readonly IEnumerable items;

   CopyOnWriteList Add(item)
   {
     return new CopyOnWriteList(new List(items, item));
   }
 }

Now wrap it:

  class WrappedCopyOnWriteList
 {
   CopyOnWriteList list;

   void Add(item)
   {
     list = list.Add(item)
   }
 }

There, it couldn't be simpler. So simple in fact I don't know what else to say. Enjoy!

The Singleton Killer

We all hate singletons so here's another useful refactoring pattern: The Singleton Killer.

Here's a common snippet of code we all love to hate:

class Transfer
 def funds(from_number, to_number, amount)
   from = AccountRepository.instance.get(from_number)
   to = AccountRepository.instance.get(to_number)
   from.balance -= amount
   to.balance += amount
 end
end

So how do we kill this beast? Simply follow these idiot proof steps:

Step One: Introduce Field

Stop asking the singleton for it's reference and store your own by creating a field:

class Transfer
 @repository = AccountRepository.instance
 def funds(from_number, to_number, amount)
   from = repository.get(from_number)
   to = repository.get(to_number)
   ...
 end
end

Wow: it's starting to look like normal code!

Step Two: Initialize in constructor

Move the initialization to the constructor:

def initialize()
 @repository = AccountRepository.instance
end

This is good but wouldn't it be better if the dependancy could be injected?

Step Three: Chain constructor

In languages where dependency injection is useful you will need to chain the constructors:

public Transfer() : this(AccountRepository.instance)
public Transfer(AccountRepository repository)
{
 this.repository = repository
}

Now the AccountRepository can be injected which of course means we can mock it out!

Step Four: Extract interface (for statically typed languages)

Except statically languages: they'll need to extract an interface:

public interface IAccountRepository
{
 Account Get(string number);
}

public Transfer(IAccountRepository repository) ...

Good: now you can start mocking and testing.

Step Five: Introduce container

If you haven't already got a container then get one (else ensure you a wiring up your dependencies in one place). Then get that to wire up the Transfer object for you:

container.Add(AccountRepository)
container.Add(Transfer)
transfer = container.Get(Transfer)
transfer.funds("YOUACCNUM", "MYACCNUM", 50000)

Step Six: Remove singleton

Once a container is doing all your injection for you that singleton is a waste of space: get rid of him by removing that horrible instance accessor.

Choose your strategies with this refactoring wisely: if your objective is to get a class tested then first repeat steps one-four within the same class until it is clean of all singletons. This is a good exercise in it's own right - even if you're not going to move on to test or remove the singleton completely - as you start to move all the dependancies to the constructor you gain a better picture of the classes collaborators (and the quality of design). On the other hand if your objective is to remove the singleton repeat steps one-three on every usage you can find then finish up with steps four-six. Another technique is to keep pushing the useage of the singleton up through the code so each client then takes on responsibility for passing the singleton down when it creates an instance of the class. Like so:

  Transfer.new(AccountRepository.Instance)

Then repeat steps one-three on the class you've just pushed into, then push the singleton up again up and so on until you can't push it anymore. This can be an interesting exercise in itself as it starts to push out the paths through the code base as you move through the layers and may also give you a clear point where your container needs to be. Be warned though: it can get messy (though strangely satisfying)!

On larger, messier codebases none of these approaches may be realistic in which case opt for the guerilla approach and pick them off as you find them. Though I would encourage trying to at least clean out a whole class using steps one-three even if you won't end up testing it right away. With all your dependancies in one place you'll still have a better picture of the design.

Beware Metanarratives

Now my darlings gather round while I tell you a story. But not just any story. A story about stories. About BIG stories.

Not so many decades ago, when the French still measured their GDP on the combined weight of their philosophical works, a particular man, with the amusing name of Lyotard, came up with the concept of Metanarratives (or Grand Narratives).

A Metanarrative, simply put, is a big story, often idealistic in tone, essential a grand, universal theory which is then applied to every aspect of life. There are many good examples of metanarratives from religious narratives, humanist, to economical and social theories communist, capitalist, freemarket etc.

The IT industry is brimming with metanarratives. Whether it is Stallman's completely open source future, no software patents, where everyone collaborates and all is wonderful, or the visions of an open web which overflows with information causing the end of totalitarianism, poverty etc. or the whole proprietary vs Open Source battle of the metanarratives.

Even as we move closer to home in the ever rational world of Enterprise Software Development we too find it overspilling. Waterfall has a wonderfully convincing metanarrative. And if SOA isn't a metanarrative (of Joycean composition) with it's land of milk and honey then I don't know what is. Probably the fastest growing metanarratives in IT is that of the Agile and Lean movement.

We humans love a good metanarrative to pin our life story on. They make us feel safe and cosy; it's nice to comprehend the world through the view of a single cohesive story with a beginning, middle and happy end (which is why marketing departments love forming them). We look around us finding more and more things to embellish and support our narratives. As we gain experience we see it as evidence to the greater truth of our chosen metanarrative.

But metanarratives are not a good thing - especially in an ever shifting landscape like IT. They make us lazy and prevent us from considering other stories. And if they are considered it is as a threat (e.g. MS' campaign against Open Source). Which is a problem in it's own right: if someone has subscribed to a metanarrative then it is almost impossible to win them round to any other story. Put one metanarrative against another and your blocked - queue visions of an evolutionist and a creationist locked in a room for eternity - monkeys would have written the works of Shakespeare before either changes their mind.

Metanarratives ultimately disappoint - especially in the ever shifting landscape of IT. Not so much because they are wrong but because they cannot be right. Everything, especially in IT, cannot fit into one nice story. The world is too culturally rich and diverse. As with the thousands of varying religions, The British Empire, Communism, the American Dream, DNA architecture, eventually the world shifts into a state where the narrative becomes ridiculous to apply. Yet people will keep doing so - the continued prevalence of Waterfall in some businesses being a case in point.

My concern is that we are guilty of promoting Agile as a metanarrative. Evangelists have moved it, from being a local story about a collection of problems in software development, to a model for the entire way businesses are run. Now, let's be clear, there are many elements of Agile I strongly subscribe to and promote, and I often agree with the basis of many of these claims. However the danger occurs when the Agile movement (or any other movement) seeks to displace any other narrative and claim itself universal. That ALL software should be developed this way, and ALL businesses should model themselves to support this. That's just crazy speak!

Interestingly (in my opinion) in their original forms Agile and Lean are post-modern in their attitudes towards metanarratives. Both attempt to promote fresh thinking and new ideas and move away from grand prescriptivism and dogma, especially of those found in the industry at the time. Most importantly both movements promote the fact that there isn't one universal way. For many people it was this breaking away from the metanarrative of the time and creating space to form our own stories which made Agile so attractive. Perhaps it is the inherent paradox in the post-modernists view of metanarratives that have caused them to, ironically, become metanarratives themselves.

The world of IT is too rich and diverse and full of too many stories (some old, some new) to support metanarratives. There is nothing wrong with some of these stories (such as Lean and Agile) when viewed in isolation; some are full of rich culture and ideas, some are simply misconceived and naive. On their own they are harmless and together with other stories they are invaluable. But as soon as people start turning them into something with which to narrate the whole Industry they are deadly.

So my warning is this: beware of metanarratives. If you notice that you are tending to frame the IT world around one story, whether that is Scrum, SOA, BDUF etc. please consider whether you have been caught in a metanarrative. Equally if you find yourself embattled due to someone's particular metanarrative, don't try and slam another one around their head. Instead try and appreciate their story at a local level whilst keeping your own stories localized and nonthreatening - they may even begin to fit it into theirs.

Killing the Config

Most applications require config of some description and it is considered good practice to wrap any calls which retrieve configured values in a config class and pass that around rather than splattering the codebase with direct calls to the frameworks inbuilt static methods.

With all the applications configs neatly wrapped in one place the code starts to become more maintainable. Yet are we just replacing one evil with another, if somewhat lesser, one? The problem I often see is this Config class finds itself passed from class to class flouting it's promiscuity and leading writers of articles on config objects down paths dangerous and full of inappropriate metaphors more commonly favoured by the ruby crowd.

There is no doubt that it is useful to collate all of the configurable elements together in one (or more) common classes. What should be avoided is leaking this concept through the whole domain and kicking it from object to object like a dirty old data bag which everyone rumages through, unchecked, pulling out stuff on their own whim. Config is an implementation detail which the majority of the application should have no concept of and (like any other such implementation details) it needs to be locked firmly away like a lump of kryptonite in a lead cased layer so it can no longer leak and permeate and weaken your SuperApp.

Let's think about this in a BDD fashion. When your class want's a ghostbusting url who's it gonna call (they have a webpage now dinyuknow)? string Config.GhostBusterUrl? Or perhaps the GhostBusterUrl class? If your class needs a typed collaborator with certain logic and behaviour (even if it is just supplying a valid url) then that's what your class should get. Not some smelly generic config object that dishes out dirty unparsed strings. The tests should be enough to tell you I'm telling you the right thing: several lines of priming the mockConfig are replaced with the simple supply of a GhostbusterUrl.

So how do we get the GhostBusterUrl from a line in config to those poor desperate classes who are being terrorized by spooks? With a little Tell don't ask and a container. Turn the config object inside out, make its role to inject into the application the dependencies it wants to supply. If your config believes it has a GhostBusterUrl then get it to construct one and register it (along with anything else it thinks it might have of use) into the container when the application tells it to. Not only does the config object gain some of its dignity back (some other classes were calling it nasty names, I know, it's the new millennium, but some people still hold the old fashioned ideas) and is safely kept right to the very edges of your application where no one can get at it.

Another positive side effect is that now your config object is giving birth to loads of lovely typed objects that actually do real stuff rather than having wandering hands pluck at its primitives (I know but I did warn it was a dangerous path laden with tasteless innuendos, I was bound to trip). Though of course, be careful not to pollute your codebase with a million classes that simply represent strings and do nothing else (it's a fine line and one you'll need to argue between me, myself and I). It's also one of the easiest refactorings you can do to your code. Give it a go (and if you object on the grounds of their being loads of small types then please find the closet crumbling procedural bridge and throw yourself into the polluted sequential river below).

Value Driven Development

At the beginning of the year I blogged about using value as a measure of productivity. Since that time I have had the opportunity to refine my thinking and be influenced by some other ideas. I want to revisit the original idea of using value as a measure. I may be guilty of repeating myself a little but it's a small cost for clarity.

The concept focuses around one very simple idea: the primary objective of a project is to deliver value. The more value we deliver the more successful the project is. If you agree with this then my next idea shouldn't be too hard to grasp: the primary measure of a project is value. Not velocity, not story points, flow, throughput, cycle time, capacity, cyclic complexity, code quality, but value, pure and simple. Of course we can use the other measurements to help us discover more effective ways of delivering value but on their own they tell little about how much value is being delivered.

Strangely, despite the fact that agile and Lean and Kanban and all of it's derivatives tell us how important value is none of them propose explicitly measuring it. Agile proponents often argue that it is their focus on delivering value early and often which makes them unique over conservative methodologies but how do they know? I've never picked a story off the wall and been able to tell what worth the customer gains by delivering it - ten, a hundred, a thousand, a millions pounds? Hell, who knows? The BA? The PM? The product owner? And how many projects out there can raise their hands to the question "How much value has your project delivered in the last week/month/year?" So if we're all so very value focused in our lean and agile worlds how are we not answering these simple questions? Are the statements about delivering more value pure assumption? How do we know that when we make 'improvements' to our processes the effect is to deliver more value? Or is at case of substituting value with story points mapped against velocity? If so how do you know that twenty-five story points to improve online advertising generated any revenue? Or is it a case of take the points and run?

Before we can confidently improve the amount of value we deliver we need to be measuring the value we are delivering. The first problem that presents itself is how do we do this? Value is hard to quantify and even harder to estimate. Which sound awfully similar to the problems with size so why not transfer the techniques used to estimate of story size to estimating value. Start simple, use relative sizes, but instead of t-shirts maybe use precious metals (aluminium, bronze, silver, gold, platinum etc.) or any other metaphor that helps the team visualize features in relative worth and as with size estimation it's an estimate, it doesn't have to be exact, the important thing is that we have some sort of gage and that we track it and refine it.

Once stories have value estimates they can go through the normal delivery process: estimate size, queue them up, stick them in iterations (or do some funky Kanban pulling on them). Except now every story card has it's most important piece of information branded to it: it's value. During prioritisation the platinum card which is the size of a small T-Shirt should obviously be delivered before the XXL bronze one. When stories are worked on people know that it's worth putting that extra effort into putting the plating on the platinum one but perhaps the aluminium story requires a more pragmatic approach than spending two days of cross functional refactoring to perfect it's design. During standups if someone says a platinum card is blocked the whole team knows it's important to get this sorted and coming up to a release with three gold stories complete but a stubborn bronze proving difficult to get across the line you may be more tempted to make the call to leave it out this time and hell to the size points. Very quickly every part of the process focuses around the amount of value being delivered, every decision is being influenced by the value of the story, not by it's size, not by how cool it is, not how much can be crammed into an iteration, but by how much value this piece of functionality is actually going to bring to the customer. Now that's being agile!

The purpose of putting values on stories is to measure how much value has been delivered. Now let's be clear about what delivered means: out there, in production, available to everyone who should have it, in full use. Basically, not dev complete, not in UAT, not in live pilot to a handful of select users, but out there live and in concert, fully functional, fully delivered, bringing value. Until then the functionality is worthless; until it is delivering value then you ain't delivered any value. Once it is though start totting it up: add the value points onto the delivery (note: delivery not iteration - iterations cannot deliver value unless they release), repeat for the next delivery plotting the value delivered on a chart and voila: the project's true velocity: how much value, for each delivery, it has been delivering.

But what if the value didn't quite get delivered? What if the functionality was released but with a bug which somehow hindered it's full capability? Maybe it is too slow, too difficult to use, doesn't do quite the right thing etc. Anything that prevents the application from delivering all of the value it promised is an issue (regardless of whether it is a bug or 'a feature'). However by resolving the issue the lost value can be reclaimed. By fixing the bug, performance or usability problems, the project claws back the value it lost when it didn't get it quite right first time.

It's important the project's value velocity reflects this. So when a fix is raised it is given a value which is then deducted from the velocity: this is because the full value of that feature was not delivered. Once the fix is in use and active then the value points go back on. Personally I find this one of the most powerful ideas behind measuring the velocity of value as the bad habits of delivering bug ridden functionality iteration after iteration by dismissing fixes in order to keep earning the points will not get you anywhere as the value velocity either struggles to move or even, in extreme cases, goes backwards.

I think there are many other benefits to be gained from using value as the primary measure that are yet to be discovered (possibly placing value onto technical debt? Though I'm not yet convinced of that one). Explicitly placing value at the forefront of the project gives teams the power to optimize themselves and discover new ways to improve their efficiency. And it isn't necessarily limited to individual teams: projects themselves should be measured in this way with teams 'contributing' to value - not simply earning points for delivering 'their part' - breaking down silo attitudes by finding ways to effectively work as a whole so value can be delivered.

I think it's pretty exciting stuff. Of course all the other metrics out there may or may not find use, dependant on whether they are contributing to the delivery of value or not, and holes will be found in the technique, but overall I think it shifts the debate and discussion forward to a entirely new level of thinking. It will allow us to reach a whole new level in the Value Enlightenment.

Getting the measure of Value

In my previous post I made the claim that we should not be measuring velocity (to the customer at least) in terms of functionality and effort delivered but in terms of value delivered. In this post I will expand on this by explaining how value and velocity work together.

The idea behind measuring velocity in terms of value is simple and, at its core, doesn't greatly differ from the way in which we measure velocity now. Functionality is still broken up into stories (or MMFs if you prefer the Kanban approach) and they are still assigned points. The subtle change comes from what the points represent: not the size of the story but instead the value of delivering it.

The first problem with this is how to assign points of value to a story? Value is something that can be hard to define, that is variable and may be measuring different things (monetary value, visitors to a web site, time saved). Story sizing has similar difficulties so it makes sense that the solution is the same: estimation. Value can be estimated by having the team discuss a story and assign points to it. It doesn't have to be accurate - it's an estimate - but it does have to be representative and the more the project goes on the better it will get. The important thing is to decide on what a unit of value represents and as a team (customer, users etc.) agree to it. But guess what? Estimating the value of something is a core activity of most businesses and they should be both used to it and good at it to.

Which begs the question what should a unit of value be? This is highly dependent on the domain and type of business. In most domains value could simply be represented in terms of money: a story to allow people to purchase your widget online is the profit of the widget * the increase in the number of sales. In some domains it might be different, for example a web site (wikipedia for example) may measure value in terms of how many visitors it gets. Some domains may be more complex and have a combination of the two (or more): number of visitors and the money raised in advertising. In these cases the solution would be to find a common representation: maybe one visitor is worth £10 in advertising revenue (for example). Some domains may even express value in a way entirely different (schools: exams passed, hospitals: lives saved etc.). The important thing is that what the unit of value represents is consistent and meaningful.

Now stories have been assigned points which represent their potential value how does a team earn (or realize) them? If you read my last post you may remember that I started by explaining that in Lean you don't earn value until a customer receives it. This rule is extremely important and velocity should work the same way. Until the customer (or end user) has the functionality out there, working and in use the team does not get the points. Functionality not delivered, only partially delivered (delivered only to a pilot group etc.) or released but unused is not considered delivered - it is inventory - and therefore does not realize its value. Until the customer has realized the value then it is not recorded on the velocity.

Recording velocity in this way ensures that it represents the reality as perceived by the customer and aligns it with the rules of Lean. It should also change the way teams are measured and motivated: their focus switches from getting functionality out of the door to genuinely delivering value into the hands of the end user.

Measure for measure (why we're doing it wrong)

In this post I will outline that the way in which we measure productivity in software development is still flawed - even in agile.

One tenet of Lean is that the core activity of production is to deliver value to your customer. A second tenet of Lean is that something is not finished until the customer receives it. Only then do you record it on your balance sheet. Combining the two is a simple statement: the purpose is to deliver value to the customer; it is not delivered until the customer receives it.

Everything else is unimportant. Anything that is not contributing to the first tenet is waste (muda) anything that is produced but isn't delivered to the customer is inventory and inventory is also waste.

The relationship between productivity and value in Lean is fundamental. If something which is produced yet not delivered doesn't have any value then we have not produced. Regardless of how many hours worked, how many items output, how much blood, sweat and tears, if all you have created is inventory then you have as good as done nothing.

Measuring productivity has been a holy grail of software development. It sits at the heart of many of Agile methodologies. Thanks to essays such as The Mythical Man Month and with tools such as estimation, iterations, velocity and burn down charts Agile has moved us beyond measuring productivity with the highly flawed method of counting hours worked. Instead it breaks projects into units of functionality (stories) which are given sizes (points) relative to their complexity/effort. Productivity is then measured by totaling up the measure of effort once the functionality is completed and 'signed off'.

Using velocity to measure progress and productivity has resulted in a revolution in thinking. Using velocity teams are better placed to manage and schedule workload (scope) and much more able to accurately report what has been done and what is left to do. The result has been much more predictable and achievable project timelines.

However there is a disconnect between the customer (and user) and the producing team. The producing team is measuring its productivity based on how much effort is realized into working software. The customers success will be ultimately measured (whether they are aware of it or not) on how much value they have delivered (note the tense: not how much they potentially will/may deliver but HAVE delivered).

The result of this disconnect is that teams responsible for production are working within a model which is not based on reality. The real value of what they are producing is nowhere to be seen in any of the measurements or reports output by the team. The result is their measure of success being out of line with that of the customers. The team ends up working to its goal to maximize output of functionality but not to producing value.

Teams need to change the way they measure productivity to bring it inline with the way the customer is measured. To achieve this is simple: measure velocity in terms of value, not effort. And by keeping with the Lean tenet to only record value when it is delivered to the end user.

Examples as tests

We often talk about tests being documentation but what about using tests for usage examples for APIs? It's something I rarely (if ever) see and yet tests are far more succint and effecient at communicating behaviour.

To demonstrate this I am going to take a very simple function from MSDN on the C# ++ operator . Here is the extract of MS' example:

The increment operator (++) increments its operand by 1. The increment operator can appear before or after its operand:
RemarksThe first form is a prefix increment operation. The result of the operation is the value of the operand after it has been incremented.
The second form is a postfix increment operation. The result of the operation is the value of the operand before it has been incremented.
Numeric and enumeration types have predefined increment operators. User-defined types can overload the ++ operator. Operations on integral types are generally allowed on enumeration.Example

// cs_operator_increment.cs
using System;
class MainClass
{
    static void Main()
    {
        double x;
        x = 1.5;
        Console.WriteLine(++x);
        x = 1.5;
        Console.WriteLine(x++);
        Console.WriteLine(x);
    }
}

Output
2.5
1.5
2.5

Although we are accustomed to these types of examples have a think about how hard you have to work to undertand it. The example on it's own gives no indication of what behaviour it is demonstrating (which is explained in complex english in the preceding text) and the call and the result are nowhere near each other - how often do my eyes have to refocus (look at the first Console.Writeline, then the first line of the output, then the explanation, match them up, look at the second Console.Wr... phew)? And this is just a simple example with a few behaviours.
Here is the same thing using tests (I've tried to closely match MS' own text for test names):

using NUnit.Framework;

[TestFixture]
public class IncrementOperatorBehaviour
{

    [Test]
    public void IncrementAsPostfixIncrementsValueByOne
    {
      double x = 1.5;
      x++;
      Assert.AreEqual(2.5, x);
    }

    [Test]
    public void IncrementAsPrefixIncrementsValueByOne
    {
      double x = 1.5;
      ++x;
      Assert.AreEqual(2.5, x);
    }

    [Test]
    public void IncrementAsPrefixGivesResultAfterIncrement
    {
      double x = 1.5;
      Assert.AreEqual(2.5, ++x);
    }

    [Test]
    public void IncrementAsPostfixGivesResultBeforeIncrement
    {
      double x = 1.5;
      Assert.AreEqual(1.5, x++);
    }
}

How much clearer is that? You even have the satisfaction of seeing it all go green when you run them!

Agile welcomes friendly aliens

It is a common complaint amongst agilists that teams label themselves as Agile simply because they do all the practices such as TDD etc. yet they are not what they would call agile. They argue that the practices alone aren't enough, that there is something else, something unquantifiable which agile is all about: the so called 'spirit' of agile, the 'people factor', or to quote "[it] may look the same as an agile methodology, but it won’t feel the same”. What they are trying to describe is a lack of alienation.

Anyone who did philosophy or sociology - or was simply a sixth form socialist revolutionary - would probably have looked at Marx's theory of alienation. Whilst most people tend to think of Marx's economic theories the problem of alienation was of primary importance to Marx; to quote Ernesto "Che" Guevara:

"One of the fundamental objectives of Marxism is to remove interest, the factor of individual interest, and gain, from people’s psychological motivations"

Marx believed that all humans are naturally motivated by the creation of value and self development, thriving on challenges, regardless of class or situation. For example, take the stereotype of a football obsessed couch potato in a dead end job. Despite lacking in motivation in their work they seek pleasure from a study of football which is shared and developed with like minded friends. This study is so intense and focused it would put the most dedicated of academics to shame. How comes, under his own steam he can achieve so much yet is so despondent in his employment?

Marx argues the reason is that capitalism's methods of maximizing productivity actually create a system which alienates the worker and therefore fails in its aims. Inversely an environment without alienation is productive and produces value for all.

Marx described four types of alienation in labour:
1) From our product: we are separated from what we produce and its value.
2) From our productive activity: the product is outside of our control so work simply becomes a meaningless activity.
3) From other human beings (or other workers): we are isolated from each other, made to work independently, despite the fact that productivity is social in nature.
4) From our species being (or species essence): we are forced to specialize which removes the freedom to develop our capabilities therefore we loose stimulation and fail to be challenged.

Agile and Lean (though I am not sure if intentionally) recognize the problems of alienation and have strategies for removing them:

From our product
The importance of our role in providing value and creating a successful product are fundamental to Agile and Lean. Many practices focus directly on reducing the gap between the team and the end product such as keeping the customer close, delivering regularly and maintaing constant feedback. AgileLean is also focused on our role in creating value rather than simply meeting a set of personal objectives.

From our productive activity
AgileLean places us in control of the process and the system. Lean emphasis the importance we all have in the productive activity and empowers us to improve the process and eliminate waste. In Agile we are responsible and involved in all productive activities from planning and estimation, to picking up stories, to delivery and improving the agile process itself.

From other human beings
Agilists have always been keen on the social nature of development believing it to be imperative to the success of the project. Communication, knowledge sharing and working together is actively encouraged with techniques such as pairing, stand-ups etc. AgiLean environments are made up of people working closely together as a team rather than individuals in a group.

From our species being:
AgileLean is often critical of specialization. Instead agilean prefers cross-functional teams made up of multi (highly) skilled generalists. This creates an environment were people are challenged and are free to develop, on a day to day basis, in the directions they choose and not reduced to the criteria their performance management chart dictates.

The S and M labels often make people feel nervous, in some cases to the point of rejecting ideas completely because of the connotations. Socialism isn't short of its critics but agile and lean prove how we can address some of the most important concerns Marx had with the capitalist system in a way that makes business stronger (just look a Toyota's huge success in a very aggressive capitalist market compared to its more communist fearing cousins in the States). The success of agile and lean prove how much more productive we can be when we tackle the problems of alienation. This is why many of us find it so frustrating to see teams labelled agile who still operate in an alienated way.

Agile and lean teaches us that teams who recognize alienation and address it are ultimately more successful. By explicitly seeking to address alienation we can create healthier working environments which offer greater value at a higher rate of productivity. Now what right minded capitalist wouldn't want that?

That's right: blame Agile

Before we start a small warning. This is a rant and as such contains certain properties common to rants such as emotionally overstating unimportant points, glossing over and confusing important ones, over decorative language and taking the odd innocent as victim.  Bearing this in mind please remember though serious my tongue is sometimes placed firmly in cheek for dramatic effect.

There has been a great debate happening over the airwaves of the blogosphere recently. Sparked of by James Shore's post The Decline and Fall of Agile it has gotten to some people declaring the death of agile and others calling for the rejection of the term altogether [link]. It's as if the Nazi party has just announced that the Third Reich was built on Scrum and suddenly everyone is trying to distance themselves: "no we never agreed with that agile stuff, what we were talking about is something completely different".

Around the same time another debate is picking up steam started by Roy Osherove's post Goodbye Mocks, Farewell Stubs. Apparently people find testing too difficult to grok so we need to reject terms like mocks and stubs ('cos though accurate they are apparently confusing) and replace it with another: Fakes. Oh the glaring irony as one side of the movement is rejecting marketing terms because too many people who use it don't know what it means while the other side try to introduce more marketing language to attract the people who don't know what it means.

Uncle Bob has dropped in like Batman at the Jokers party and delivered his direct, powerful blow: "the problem is you're all f*ing lazy". Although I feel that Uncle Bob is much closer to the truth than many other commentators he's not really offering a realistic solution out of the mess.

So what exactly is the problem we've got here? Both of these posts have something in common. James Shore's issue stems from teams picking up words like Scrum and Sprint and forgetting that they have to write decent code as well. Roy Osherove's post stems from the fact that getting teams to even write tests is met with constant failure. Both posts talk about a general lack of engineering skill in IT. In reaction different parties are offering up different solutions but they all fail to address the core issue.

For those jumping the agile ship and getting onto the next coolest speedboat what exactly is that going to prove? Rather than improving the industry it leaves it helpless and sinking. This is a rotten attitude that resolves nothing: when every PHP and VB programmer converts to Ruby what you gonna do then? Oh it's OK Ola's writing Ioke and if that doesn't work we've always go Lisp (they'll never get that). Roy Osherove's solution of lowering barriers to entry is more commendable but will achieve just as little and is guilty of another great crime: dumbing down (though he's spot on about tool support). I don't think the solution to poor engineering skills are to withdraw important technical terms: can you imagine the Bar rebranding legal terms because, y'know Lawyers just find it too difficult to understand this stuff or Doctors calling for less names for the organs 'cos med students these days just can't get the point of having the terms stomach and intestine and they keep operating on the wrong one. Why can't we just call it the food processor? Jay Fields' novel idea is to ask them all to politely leave  (50% of them in fact). And if they don't? Well then I guess we'll all get together and go around with a big seal club and sort it out that way (it's what I refer to as "The Great Culling" - there's those Nazi's marching past again).

The real problem, the root cause to all this is simple (and on this most agree especially Uncle Bob and Jay): the gross under valuing of the skills which enable you to do your job. The thing is it's nothing new. It was there before agile and TDD had the impact they do today (they just laughed then). Remember those not so long ago days when people thought the solution to delivering software wasn't skills or focus on quality but getting some glorified secretary to type up several hundred pages of requirements and then handing them to some supposed genius to design everything for the monkeys to punch in? Well we're still there people: it's just they thought the solution to the mess that disaster caused was to do something called iterations, standing up and getting the monkeys to skip around the genius guy wrapping him in his own ribboned UML diagrams chanting "architect, architect, architect".

Nothing's changed, agile or not. The majority of the industry still doesn't value the basic, fundamental skills it takes to write software of acceptable quality. It didn't before and it still doesn't now. Instead it's obsessed with solving the problem by bringing in the right highly paid manager with the right powerpoint presented methodology. Until the industry gets that ain't the way it will drag every shining beacon of light (agile, Ruby whatever) down into Hades with it.

Writing software is about people People! That's what agile and Lean and all their friends tell you again and again. The fundamental principles are about people, not iterations, continuous integration and story boards. Go check out the Agile Manifesto and tell me where you see any of that stuff? This bastard form of agile which only talks about tools and processes is like someone got the manifesto and spun it round (People OVER Process read it Again). The practices and tools are there to actively encourage developers who are skilled, committed and know what they're doing and make them more empowered and productive and ultimately deliver MORE value and success, they are not there to replace them.

So what is the solution? The simple truth is that those of us who understand the problem have got to keep doing a great job, we've got to help those who want to do a great job but aren't sure how to do a great job and we've got help those who don't understand how to do a good job and may never will and may not even want to. Sure it's depressing sometimes, it's frustrating but we can't just give up on the industry and jump away on the next term or simply dumb the whole thing down so the next generation can get it a little more but ultimately still fail to get it.

Agile was never, is never going to be an quick, faultless success forever preserving an eternal purity. Our industry is sick, real sick. It's a beer guzzling, chain smoking, couch potato who's been told by his doctor to eat healthily, give up smoking and booze and get some exercise but instead just swallows the agile pill and hopes that's enough to save his sorry fat arse. And surprise surprise when the casualties start coming in their gonna blame those pills.

We have generation after generation bought up on bad practices and incorrect assumptions (smoking isn't bad for you, exercise gives you heart attacks) and what's worse most of them don't even know how bad things are: they think this is NORMAL (sadly because IT IS), they've adjusted to a certain degree of failure. IT projects are like lifts: sure your pissed off there broken down but your not surprised, you even kinda expected it.

The mess is big and it's gonna take years to sort it out and a lot of hard work. And yes, along the way people are going to buy the agile weight loss pill from some dodgy site on the Internet without seeking professional advice, get it wrong. And why? Because they're desperate!

It was totally delusional to think agile was going to result in everyone suddenly rejecting their bad habits and come crawling on their knees and go XP Kosher. And when that didn't work well there's always the prophesy of the True agile warriors standing in the post-apocolyptic world looking down on the dying as the last breath of those Tragic Lost sigh the now infamous regretful words "we wish we'd read our Domain Driven Designs". Well don't worry because to become agile all you have to do is close your eyes and say "please Kent forgive us our hacks and deliver us not into Waterfall" and mean it, really mean it, and wait, above the mountain there what's that? is it who I think it is? It is: the heavens are open and Martin Fowler is ascending into them.

Let's get real here: developers who have never come across TDD before, who've never experienced it's value first hand, who've spent years doing things a certain way are struggling to grasp the concept and oh sorry that's surprising why? I guess you were doing TDD from that first Hello World in Pascal? And those people who go in and try Scrum for the first time and find they are riddled with technical debt because gave into pressure and didn't invest in quality. Now stand up and be counted all you who never made that mistake.

I'm not saying that we should just let these things go, hold our noses, ignore the smells and step over the shit. Quite the opposite. But we've got to accept that we're not going to walk into a room of professionals and get them to convert overnight (or year or possibly even decade). How many years did it take before doctors accepted germ theory and started washing their hands?

Though there is hope: the most effective weapon we have is success. By being successful and then telling people about why and how we were successful. That's how this whole agile thing started, that's how it built up it's great reputation and that's how it's going to survive and get better. The more success the more people will start to realize that what they're doing now isn't as good as they think. Right now not testing, not caring about quality, high technical debt: that's the norm, that's expected. Keep being successful and people will start asking the others those awkward questions: why does this application crash when my last team were virtually bug free, why does it take a weekend to get a release out when my last team did it in a few minutes? Why does it take six months to implement a new feature when my last team took a week? What do you mean there are ramifications and complications? The more we deliver quality software the less room people will have to worm out of those questions and then the tipping point will come.

Let's not be mistaken, it's going to be hard work: but isn't that the whole point? Writing software is hard, agile doesn't change that and at first TDD certainly doesn't. Yes it's frustrating and somedays you really want to scream and kick in the can but what's the alternative IT ghettos sucking in grads and with no way out?

If, in the years to come, we want to work in a different industry then we've got to take some responsibility for helping create it.

Make me Whole

This is one of my favourite refactoring patterns and one I find I use a great deal so I thought I'd share it. The basic premise of this refactoring is to encapsulate logic into its own class using small, easy steps. The trick is each step should always leave the code in good working order i.e. the tests should always pass.

To help explain I'm going to walk through the refactoring by encapsulating some simple procedural logic and by making the concept explicit by giving it it's own class. Here's our starting point:

  def say_hello()
    if @friend == "Jeff" || @friend == "Robert" do
       put "Hello my good friend #friend"
    else
       put "Hello #friend"
    end
  end

The first thing we notice is there is some behaviour which depends on the state of friend (is the value "Jeff" or "Robert"). This is a good sign that there's a concept called Friend which needs encapsulating. So here's how:

Step One: Extract method

The first step is easy: extract the logic into it's own method like so:

  def say_hello()
    if is_good_friend do ...
  end

  def is_good_friend
    @friend == "Jeff" || @friend == "Robert"
  end

This is good, now the code is clearer to read and we've encapsulated the logic (albeit only in a method). That's already a big improvement.

Step Two: Introduce parameter

The problem is our method is dependent on its class for the @friend variable. We need to sever this connection so the method can stand on its own two feet. The simple way is to pass friend through as a parameter.

  def say_hello()
    if is_good_friend(friend) do ...
  end

  def is_good_friend(friend)
    friend == "Jeff" || friend == "Robert"
  end

That's better: is_good_friend is decoupled from the class. Its free!

We still need to get that method onto a new class. The problem is it can't be done in one easy non-breaking step. So we're going to have to take a tiny intermediate step to get there:

Step Three: Make method static

By making the method static it severs all connections to the original class: it is explicitly non-dependant on any instance variables the original class has.

  def say_hello()
    if self.is_good_friend(friend) do ...
  end

  def self.is_good_friend(friend) ...

Making the method static is also a good safety check to ensure that you've not got any references to the original class and it is properly isolated before moving it.

Step Four: Move method

Now the method is static move it to the new class:

  def say_hello()
    if Friend.is_good_friend(friend) do ...
  end

  class Friend
    def self.is_good_friend(friend) ...
  end

Excellent, we're really getting somewhere! Though now we've got a static method sitting on a helper class. Yuk: let's get that sorted before someone copies us and that nasty anti-pattern starts to proliferate through our code base:

Step Five: Create instance

This is where the real magic starts to happen: we create an instance of our new class and use it to store the value of friend:

  class Friend
    @value

    def self.is_good_friend(old_friend)
      friend = Friend.new(old_friend)
      friend.value == "Jeff" || friend.value == "Robert"
    end 
  end

Still, this is a bit ugly. What we really need to do is get the client to do the work of creating the instance.

Step Six: Introduce parameter

Rather than have our static method new up Friend get the client to do it and pass it across:

  def say_hello()
    if(Friend.is_good_friend(@friend, Friend.new(@friend)) ...
  end

  class Friend
   ...
    def self.is_good_friend(old_friend, friend)
      friend.value == "Jeff" || friend.value == "Robert"
    end
  end

Step Seven: Remove parameter

The old_friend parameter is redundant and ugly: let's get rid of it!

  def say_hello()
    if(Friend.is_good_friend(Friend.new(@friend)) ...
  end

  class Friend
    ...
    def self.is_good_friend(friend) ...
  end

Excellent, this is starting to look a lot better now. Though look at that repetition: the static method on Friend passes an instance of it's own class! Only one way to sort that out:

Step Eight: Make method non-static

Now the beauty of this is that if you are using a modern IDE it will work out that the method is being passed an instance of its own type and it will magically do this:

  def say_hello()
     if(Friend.new(@friend).is_good_friend) ...
  end

  class Friend
     def is_good_friend
         @value == "Jeff" || @value== "Robert" 
     end
  end

Brilliant: Friend is now nicely encapsulated and ready to grow into a responsible object with a bright future. From here we can start pushing more behaviour onto the object and eventually make @friend reference an instance of the Friend class.

Overall the pattern is very simple: extract the logic into its own method then work to detach it from the class by making in static, then simply move the static method and reattach it to the new class by returning it to an instance. It is also possible to handle more complex behaviour with several variables involved, simply pass them through as parameters and then push them onto the new instance.

Although this example is simple the pattern can be repeated again and again to build up classes into a number of methods (sometimes these classes can again be refactored into smaller classes still).

if(conditional) push.down(to.origin)

Conditionals (ifs, switches) are a plague in many code bases and although there are campaigns to prevent their spread many developers are unclear on how to control these pests. Conditionals are one of the most prevalent, yet tolerated, sources of repetition and bugs. Why? Well let's start with one simple yet common example:

if(thing.is_valid) do
 other_thing.do_something_with(thing)
end

This sort of code spreads like a disease because every time the client calls do_something it must ensure it checks for validity. If we look closer at the object in use we find that calling do_something in an invalid state not only requires the same conditional but also throws an error:

def do_something_with(thing)
 raise "Thing must be valid" if not thing.is_valid
 // do lots of wonderful things with a valid thing
end

The repetition is not only in the client code but in tests:

def calls_other_thing_when_thing_is_valid
 mock_thing.expects(:is_valid).returns(true)
 mock_other_thing.expects(:do_something).with(mock_thing)
 ...
end

def does_nothing_when_thing_is_invalid
 mock_thing.expects(:is_valid).returns(false)
 ...
end

The simplest solution to remove the glut of these conditions (in terms of lines of codes, not calls) is to push it down off the client and to the object who knows his state. If we go back to the tests and use BDD we find that we would rather define the interactions as such:

def tells_thing_to_do_something_when_valid
 mock_thing.expects(:when_valid).with(block)
end 

Thing now takes a block (in ruby), delegate (in C#) or anonymous class (in Java):

thing.when_valid({other_thing.do_something_with(thing)})

def when_valid(&block)
 &block.call if valid
end

Now the conditional is controlled by the object responsible for its state: we've removed an Ask and replaced it with a Tell. This is a vast improvement, all the repetition in the clients has gone and the chances of bugs due to accidental calls when in an invalid state are removed.

This presumes that Thing doesn't really do much but chances are it's got lots of behaviour and the chances are that behaviour changes based on being valid or not. Of course all the methods could delegate to its own when_valid method but I bet you also need a when_not_valid and oh boy that code's getting difficult to read. Also, the repetition, in terms of lines of code may be lower but how many times are conditionals executed during run time?

class Thing
 def when_valid(&block)
   &block.call if valid
 end

 def other_method
  if valid do
    // do some stuff
  else
    // do some different stuff
  end
 end
end

The conditional can still be pushed further down right to its origin: when Thing changes between a valid and invalid state. This can be achieved by using the Replace Conditional With Polymorphism refactoring to employ the state pattern:

NOTE: I want to keep this example clear for developers of other languages though there are cleverer ways of achieving this in Ruby

class Thing
 def validate
   valid = // check if valid
   @state_behaviour = valid ? Valid.new : Invalid.new
 end

 def when_valid(&block)
   @state_behaviour.when_valid(&block)
 end

 def other_method
   @state_behaviour.other_method
 end

 class Valid
   def when_valid(&block)
     &block.call
   end

   def other_method
      // do some stuff
   end
 end

 class Invalid
   def when_valid(&block)
      // ignore
   end

   def other_method
      // do some different stuff
   end
 end
end

Now there is only one point where the conditional is called and all of the behaviour for each state (valid or invalid) is in one easy to read place.  Of course, this is just a simple example so just imagine what it would do for more complex solutions (and don't forget: the same can be applied to switches).