Agile welcomes friendly aliens

It is a common complaint amongst agilists that teams label themselves as Agile simply because they do all the practices such as TDD etc. yet they are not what they would call agile. They argue that the practices alone aren't enough, that there is something else, something unquantifiable which agile is all about: the so called 'spirit' of agile, the 'people factor', or to quote "[it] may look the same as an agile methodology, but it won’t feel the same”. What they are trying to describe is a lack of alienation.

Anyone who did philosophy or sociology - or was simply a sixth form socialist revolutionary - would probably have looked at Marx's theory of alienation. Whilst most people tend to think of Marx's economic theories the problem of alienation was of primary importance to Marx; to quote Ernesto "Che" Guevara:

"One of the fundamental objectives of Marxism is to remove interest, the factor of individual interest, and gain, from people’s psychological motivations"

Marx believed that all humans are naturally motivated by the creation of value and self development, thriving on challenges, regardless of class or situation. For example, take the stereotype of a football obsessed couch potato in a dead end job. Despite lacking in motivation in their work they seek pleasure from a study of football which is shared and developed with like minded friends. This study is so intense and focused it would put the most dedicated of academics to shame. How comes, under his own steam he can achieve so much yet is so despondent in his employment?

Marx argues the reason is that capitalism's methods of maximizing productivity actually create a system which alienates the worker and therefore fails in its aims. Inversely an environment without alienation is productive and produces value for all.

Marx described four types of alienation in labour:
1) From our product: we are separated from what we produce and its value.
2) From our productive activity: the product is outside of our control so work simply becomes a meaningless activity.
3) From other human beings (or other workers): we are isolated from each other, made to work independently, despite the fact that productivity is social in nature.
4) From our species being (or species essence): we are forced to specialize which removes the freedom to develop our capabilities therefore we loose stimulation and fail to be challenged.

Agile and Lean (though I am not sure if intentionally) recognize the problems of alienation and have strategies for removing them:

From our product
The importance of our role in providing value and creating a successful product are fundamental to Agile and Lean. Many practices focus directly on reducing the gap between the team and the end product such as keeping the customer close, delivering regularly and maintaing constant feedback. AgileLean is also focused on our role in creating value rather than simply meeting a set of personal objectives.

From our productive activity
AgileLean places us in control of the process and the system. Lean emphasis the importance we all have in the productive activity and empowers us to improve the process and eliminate waste. In Agile we are responsible and involved in all productive activities from planning and estimation, to picking up stories, to delivery and improving the agile process itself.

From other human beings
Agilists have always been keen on the social nature of development believing it to be imperative to the success of the project. Communication, knowledge sharing and working together is actively encouraged with techniques such as pairing, stand-ups etc. AgiLean environments are made up of people working closely together as a team rather than individuals in a group.

From our species being:
AgileLean is often critical of specialization. Instead agilean prefers cross-functional teams made up of multi (highly) skilled generalists. This creates an environment were people are challenged and are free to develop, on a day to day basis, in the directions they choose and not reduced to the criteria their performance management chart dictates.

The S and M labels often make people feel nervous, in some cases to the point of rejecting ideas completely because of the connotations. Socialism isn't short of its critics but agile and lean prove how we can address some of the most important concerns Marx had with the capitalist system in a way that makes business stronger (just look a Toyota's huge success in a very aggressive capitalist market compared to its more communist fearing cousins in the States). The success of agile and lean prove how much more productive we can be when we tackle the problems of alienation. This is why many of us find it so frustrating to see teams labelled agile who still operate in an alienated way.

Agile and lean teaches us that teams who recognize alienation and address it are ultimately more successful. By explicitly seeking to address alienation we can create healthier working environments which offer greater value at a higher rate of productivity. Now what right minded capitalist wouldn't want that?

That's right: blame Agile

Before we start a small warning. This is a rant and as such contains certain properties common to rants such as emotionally overstating unimportant points, glossing over and confusing important ones, over decorative language and taking the odd innocent as victim.  Bearing this in mind please remember though serious my tongue is sometimes placed firmly in cheek for dramatic effect.

There has been a great debate happening over the airwaves of the blogosphere recently. Sparked of by James Shore's post The Decline and Fall of Agile it has gotten to some people declaring the death of agile and others calling for the rejection of the term altogether [link]. It's as if the Nazi party has just announced that the Third Reich was built on Scrum and suddenly everyone is trying to distance themselves: "no we never agreed with that agile stuff, what we were talking about is something completely different".

Around the same time another debate is picking up steam started by Roy Osherove's post Goodbye Mocks, Farewell Stubs. Apparently people find testing too difficult to grok so we need to reject terms like mocks and stubs ('cos though accurate they are apparently confusing) and replace it with another: Fakes. Oh the glaring irony as one side of the movement is rejecting marketing terms because too many people who use it don't know what it means while the other side try to introduce more marketing language to attract the people who don't know what it means.

Uncle Bob has dropped in like Batman at the Jokers party and delivered his direct, powerful blow: "the problem is you're all f*ing lazy". Although I feel that Uncle Bob is much closer to the truth than many other commentators he's not really offering a realistic solution out of the mess.

So what exactly is the problem we've got here? Both of these posts have something in common. James Shore's issue stems from teams picking up words like Scrum and Sprint and forgetting that they have to write decent code as well. Roy Osherove's post stems from the fact that getting teams to even write tests is met with constant failure. Both posts talk about a general lack of engineering skill in IT. In reaction different parties are offering up different solutions but they all fail to address the core issue.

For those jumping the agile ship and getting onto the next coolest speedboat what exactly is that going to prove? Rather than improving the industry it leaves it helpless and sinking. This is a rotten attitude that resolves nothing: when every PHP and VB programmer converts to Ruby what you gonna do then? Oh it's OK Ola's writing Ioke and if that doesn't work we've always go Lisp (they'll never get that). Roy Osherove's solution of lowering barriers to entry is more commendable but will achieve just as little and is guilty of another great crime: dumbing down (though he's spot on about tool support). I don't think the solution to poor engineering skills are to withdraw important technical terms: can you imagine the Bar rebranding legal terms because, y'know Lawyers just find it too difficult to understand this stuff or Doctors calling for less names for the organs 'cos med students these days just can't get the point of having the terms stomach and intestine and they keep operating on the wrong one. Why can't we just call it the food processor? Jay Fields' novel idea is to ask them all to politely leave  (50% of them in fact). And if they don't? Well then I guess we'll all get together and go around with a big seal club and sort it out that way (it's what I refer to as "The Great Culling" - there's those Nazi's marching past again).

The real problem, the root cause to all this is simple (and on this most agree especially Uncle Bob and Jay): the gross under valuing of the skills which enable you to do your job. The thing is it's nothing new. It was there before agile and TDD had the impact they do today (they just laughed then). Remember those not so long ago days when people thought the solution to delivering software wasn't skills or focus on quality but getting some glorified secretary to type up several hundred pages of requirements and then handing them to some supposed genius to design everything for the monkeys to punch in? Well we're still there people: it's just they thought the solution to the mess that disaster caused was to do something called iterations, standing up and getting the monkeys to skip around the genius guy wrapping him in his own ribboned UML diagrams chanting "architect, architect, architect".

Nothing's changed, agile or not. The majority of the industry still doesn't value the basic, fundamental skills it takes to write software of acceptable quality. It didn't before and it still doesn't now. Instead it's obsessed with solving the problem by bringing in the right highly paid manager with the right powerpoint presented methodology. Until the industry gets that ain't the way it will drag every shining beacon of light (agile, Ruby whatever) down into Hades with it.

Writing software is about people People! That's what agile and Lean and all their friends tell you again and again. The fundamental principles are about people, not iterations, continuous integration and story boards. Go check out the Agile Manifesto and tell me where you see any of that stuff? This bastard form of agile which only talks about tools and processes is like someone got the manifesto and spun it round (People OVER Process read it Again). The practices and tools are there to actively encourage developers who are skilled, committed and know what they're doing and make them more empowered and productive and ultimately deliver MORE value and success, they are not there to replace them.

So what is the solution? The simple truth is that those of us who understand the problem have got to keep doing a great job, we've got to help those who want to do a great job but aren't sure how to do a great job and we've got help those who don't understand how to do a good job and may never will and may not even want to. Sure it's depressing sometimes, it's frustrating but we can't just give up on the industry and jump away on the next term or simply dumb the whole thing down so the next generation can get it a little more but ultimately still fail to get it.

Agile was never, is never going to be an quick, faultless success forever preserving an eternal purity. Our industry is sick, real sick. It's a beer guzzling, chain smoking, couch potato who's been told by his doctor to eat healthily, give up smoking and booze and get some exercise but instead just swallows the agile pill and hopes that's enough to save his sorry fat arse. And surprise surprise when the casualties start coming in their gonna blame those pills.

We have generation after generation bought up on bad practices and incorrect assumptions (smoking isn't bad for you, exercise gives you heart attacks) and what's worse most of them don't even know how bad things are: they think this is NORMAL (sadly because IT IS), they've adjusted to a certain degree of failure. IT projects are like lifts: sure your pissed off there broken down but your not surprised, you even kinda expected it.

The mess is big and it's gonna take years to sort it out and a lot of hard work. And yes, along the way people are going to buy the agile weight loss pill from some dodgy site on the Internet without seeking professional advice, get it wrong. And why? Because they're desperate!

It was totally delusional to think agile was going to result in everyone suddenly rejecting their bad habits and come crawling on their knees and go XP Kosher. And when that didn't work well there's always the prophesy of the True agile warriors standing in the post-apocolyptic world looking down on the dying as the last breath of those Tragic Lost sigh the now infamous regretful words "we wish we'd read our Domain Driven Designs". Well don't worry because to become agile all you have to do is close your eyes and say "please Kent forgive us our hacks and deliver us not into Waterfall" and mean it, really mean it, and wait, above the mountain there what's that? is it who I think it is? It is: the heavens are open and Martin Fowler is ascending into them.

Let's get real here: developers who have never come across TDD before, who've never experienced it's value first hand, who've spent years doing things a certain way are struggling to grasp the concept and oh sorry that's surprising why? I guess you were doing TDD from that first Hello World in Pascal? And those people who go in and try Scrum for the first time and find they are riddled with technical debt because gave into pressure and didn't invest in quality. Now stand up and be counted all you who never made that mistake.

I'm not saying that we should just let these things go, hold our noses, ignore the smells and step over the shit. Quite the opposite. But we've got to accept that we're not going to walk into a room of professionals and get them to convert overnight (or year or possibly even decade). How many years did it take before doctors accepted germ theory and started washing their hands?

Though there is hope: the most effective weapon we have is success. By being successful and then telling people about why and how we were successful. That's how this whole agile thing started, that's how it built up it's great reputation and that's how it's going to survive and get better. The more success the more people will start to realize that what they're doing now isn't as good as they think. Right now not testing, not caring about quality, high technical debt: that's the norm, that's expected. Keep being successful and people will start asking the others those awkward questions: why does this application crash when my last team were virtually bug free, why does it take a weekend to get a release out when my last team did it in a few minutes? Why does it take six months to implement a new feature when my last team took a week? What do you mean there are ramifications and complications? The more we deliver quality software the less room people will have to worm out of those questions and then the tipping point will come.

Let's not be mistaken, it's going to be hard work: but isn't that the whole point? Writing software is hard, agile doesn't change that and at first TDD certainly doesn't. Yes it's frustrating and somedays you really want to scream and kick in the can but what's the alternative IT ghettos sucking in grads and with no way out?

If, in the years to come, we want to work in a different industry then we've got to take some responsibility for helping create it.

Make me Whole

This is one of my favourite refactoring patterns and one I find I use a great deal so I thought I'd share it. The basic premise of this refactoring is to encapsulate logic into its own class using small, easy steps. The trick is each step should always leave the code in good working order i.e. the tests should always pass.

To help explain I'm going to walk through the refactoring by encapsulating some simple procedural logic and by making the concept explicit by giving it it's own class. Here's our starting point:

  def say_hello()
    if @friend == "Jeff" || @friend == "Robert" do
       put "Hello my good friend #friend"
    else
       put "Hello #friend"
    end
  end

The first thing we notice is there is some behaviour which depends on the state of friend (is the value "Jeff" or "Robert"). This is a good sign that there's a concept called Friend which needs encapsulating. So here's how:

Step One: Extract method

The first step is easy: extract the logic into it's own method like so:

  def say_hello()
    if is_good_friend do ...
  end

  def is_good_friend
    @friend == "Jeff" || @friend == "Robert"
  end

This is good, now the code is clearer to read and we've encapsulated the logic (albeit only in a method). That's already a big improvement.

Step Two: Introduce parameter

The problem is our method is dependent on its class for the @friend variable. We need to sever this connection so the method can stand on its own two feet. The simple way is to pass friend through as a parameter.

  def say_hello()
    if is_good_friend(friend) do ...
  end

  def is_good_friend(friend)
    friend == "Jeff" || friend == "Robert"
  end

That's better: is_good_friend is decoupled from the class. Its free!

We still need to get that method onto a new class. The problem is it can't be done in one easy non-breaking step. So we're going to have to take a tiny intermediate step to get there:

Step Three: Make method static

By making the method static it severs all connections to the original class: it is explicitly non-dependant on any instance variables the original class has.

  def say_hello()
    if self.is_good_friend(friend) do ...
  end

  def self.is_good_friend(friend) ...

Making the method static is also a good safety check to ensure that you've not got any references to the original class and it is properly isolated before moving it.

Step Four: Move method

Now the method is static move it to the new class:

  def say_hello()
    if Friend.is_good_friend(friend) do ...
  end

  class Friend
    def self.is_good_friend(friend) ...
  end

Excellent, we're really getting somewhere! Though now we've got a static method sitting on a helper class. Yuk: let's get that sorted before someone copies us and that nasty anti-pattern starts to proliferate through our code base:

Step Five: Create instance

This is where the real magic starts to happen: we create an instance of our new class and use it to store the value of friend:

  class Friend
    @value

    def self.is_good_friend(old_friend)
      friend = Friend.new(old_friend)
      friend.value == "Jeff" || friend.value == "Robert"
    end 
  end

Still, this is a bit ugly. What we really need to do is get the client to do the work of creating the instance.

Step Six: Introduce parameter

Rather than have our static method new up Friend get the client to do it and pass it across:

  def say_hello()
    if(Friend.is_good_friend(@friend, Friend.new(@friend)) ...
  end

  class Friend
   ...
    def self.is_good_friend(old_friend, friend)
      friend.value == "Jeff" || friend.value == "Robert"
    end
  end

Step Seven: Remove parameter

The old_friend parameter is redundant and ugly: let's get rid of it!

  def say_hello()
    if(Friend.is_good_friend(Friend.new(@friend)) ...
  end

  class Friend
    ...
    def self.is_good_friend(friend) ...
  end

Excellent, this is starting to look a lot better now. Though look at that repetition: the static method on Friend passes an instance of it's own class! Only one way to sort that out:

Step Eight: Make method non-static

Now the beauty of this is that if you are using a modern IDE it will work out that the method is being passed an instance of its own type and it will magically do this:

  def say_hello()
     if(Friend.new(@friend).is_good_friend) ...
  end

  class Friend
     def is_good_friend
         @value == "Jeff" || @value== "Robert" 
     end
  end

Brilliant: Friend is now nicely encapsulated and ready to grow into a responsible object with a bright future. From here we can start pushing more behaviour onto the object and eventually make @friend reference an instance of the Friend class.

Overall the pattern is very simple: extract the logic into its own method then work to detach it from the class by making in static, then simply move the static method and reattach it to the new class by returning it to an instance. It is also possible to handle more complex behaviour with several variables involved, simply pass them through as parameters and then push them onto the new instance.

Although this example is simple the pattern can be repeated again and again to build up classes into a number of methods (sometimes these classes can again be refactored into smaller classes still).