German General von Manstein about Agile

Fifth principle of Agile Manifesto says: 

"Build projects around motivated individuals. 
Give them the environment and support they need, 
and trust them to get the job done."

Before you build a project or to be more specific a team, you have to choose 'motivated individuals'. Above rule is just a general guideline, not telling you anything how to do it. So the question, related to the way of choosing motivated people, remains open and is somewhat worrisome, as we still do not have a simple and clear protocol how to do it. 
To be fairly honest, it is always hard to give one, neat and concise rule describing what to do. However, instead of that I will quote German's general, von Manstein of the German Officer Corps, rule to be used as a tool for choosing motivated employees. 
Von Manstein used to divide his soliders into four groups:

1. Lazy and stupid - leave them alone, as they are doing no harm
2. Hardworking and intelligent - they would be an excellent staff officers, they can ensure that every detail is properly introduced and considered
3. Lazy and intelligent - these are people suited for the highest office
4. Hardworking and stupid - they are a menace, who should be fired at once, as they create and add irrelevant work for everybody aka "cannon fodder"

The rule is rigid and acute not living any space for people being in between.
It is not the most detailed approach, however we can leverage its features, anyway. It can be treated more like a filter, rather than just a simple rule. It gives us some view on who we should avoid. It also tells us about rough activities and positions people may hold.
It is not the best rule allowing you to choose 'motivated individuals' , but it is definitely the very first step on that path. 

The Pareto Principle
It's also worth to know Pareto Principle, which says: 

"80% of your results will come from 20% of your efforts".

It means that you need to work hard to identify 20% of things you do, which generate 80% of what you do. In other words, this principle teaches us how to be lazy and not to fall into the activity trap. Be honest with yourself, nobody cares how busy you are, they just do care what you produce. You don't have to work hard. It's better to decide, what NOT to do, so that you have more time for valuable 20% of your core activities, which gives you 80% of return of your time investment (ROI).

Self Assesment
So what you can do now, is to assess your position against von Manstein and Pareto principles and see where you are. Sometimes it might be very painful process, but definitely it's worth doing that. If you are finding yourself too low in hierarchy and you are willing to pull yourself up a bit try below clues:

• Don’t try to keep all people happy all the time
• Have a work plan managing your time
• Practice saying “no”
• Don't stuck in one square, say Smart and Hardworking. Try to improve yourself, change your character, so that you will be eligible for another square: Smart and Lazy. Train your laziness in an intelligent and smart way.

Awaitility - testing asynchronous calls in Java

Asynchronicity plays an important role in systems we develop, nowadays. Below are examples, where we are using asynchronous operations, almost on a daily basis, in our systems:

• writing to cache with asynchronous write to database behind the scenes or otherwise
• JMS queues or topics, which are asynchronous itself
• retrieving data from external systems in asynchronous way

Asynchronous operations are often used for caching data in order to decrease the number of expensive calls to: database, webservice etc. There are also mutations of above caching approach. For instance, one may combine sending time consuming or complicated algorithmic operations to JMS queue (asynchronous system). Later on, when results of such an intensive operation/algorithm would be available, it might be simply cached in a local in-memory storage. Next time, when user would try to fetch same data, it will be fast and easy for her to retrieve previously calculated results from cache. Such an approach, gives you an ability to build very fast, scaleable, flexible and user friendly applications/GUIs.
However, there is one caveat here. It is quite difficult to test such an asynchronous system. If we like to do it from scratch, we will need to provide some threads' handlers, timeouts and generally deal with concurrency, which makes the whole test more obscure and less focused on business principles ruling the entire system. If above situation is true, it means that we found a niche and basically a need for some library to handle our problem. 
The answer for our headache is Awaitility. It's a quite simple and powerful, Java based library, to test asynchronous calls. Moreover, it does have a concise and expressive DSL to define expectations.

Now, let's see Awaitility in action. I wrote a simple application, which is available on GitHub. The basic idea is that there is a DelayedFileCreator class, which is responsible for creating a file on a filesystem. It also tries to mimic an expensive and time consuming calculations, by time delay. Important thing is a result of that operation. This sort of asynchronous calls are having either a state returned or they cause a change of state somewhere else - in our case it is a newly created file on a filesystem. 

package asynchronousexample;

import java.io.File;
import java.io.IOException;

public class DelayedFileCreator implements Runnable {

 private static final int THREE_SECONDS = 3000;

 private File file;
 private Timer timer;

 public DelayedFileCreator(Timer aTimer, File aFile) {
  timer = aTimer;
  file = aFile;
 }

 @Override
 public void run() {
  sleepBeforeCreatingFile();
  createNewFile();
 }

 private void sleepBeforeCreatingFile() {
  try {
   timer.sleep(THREE_SECONDS);
  } catch (InterruptedException ie) {
   throw new RuntimeException(ie);
  }
 }

 private void createNewFile() {
  try {
   file.createNewFile();
  } catch (IOException ioe) {
   throw new RuntimeException(ioe);
  }
 }
}

 
Ideally, we would like to be able to write a test, which invokes overridden method run(), waits until it is done and asserts the result of asynchronous operation. Awaitility hits the nail on the head. Below example shows an integration test, using nice and readable DSL, to assert the result.

package integration;

import asynchronousexample.AsynchronousTaskLauncher;
import asynchronousexample.DelayedFileCreator;
import asynchronousexample.Timer;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

import java.io.File;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import static com.jayway.awaitility.Awaitility.with;
import static com.jayway.awaitility.Duration.ONE_HUNDRED_MILLISECONDS;
import static com.jayway.awaitility.Duration.TEN_SECONDS;
import static com.jayway.awaitility.Duration.TWO_HUNDRED_MILLISECONDS;
import static org.hamcrest.Matchers.equalTo;

public class CreateFileAsynchronouslyIntegrationTest {

 private static final int THREAD_POOL_SIZE = 3;
 private static final String FILENAME = "sample.txt";

 @BeforeTest
 public void deleteFileFromFileSystem() {
  File file = new File(FILENAME);
  if (file.exists()) {
   file.delete();
  }
 }

 @Test
 public void shouldAsynchronouslyWriteFileOnDisk() throws Exception {
  AsynchronousTaskLauncher launcher = prepareAsynchronousTaskLauncher();
  Runnable delayedFileCreatorTask = prepareDelayedFileCreatorWith(FILENAME);

  launcher.launch(delayedFileCreatorTask);

  with().pollDelay(ONE_HUNDRED_MILLISECONDS)
    .and().with().pollInterval(TWO_HUNDRED_MILLISECONDS)
    .and().with().timeout(TEN_SECONDS)
    .await("file creation")
    .until(fileIsCreatedOnDisk(FILENAME), equalTo(true));
 }

 private Runnable prepareDelayedFileCreatorWith(String filename) {
  Timer timer = new Timer();
  File file = new File(filename);
  return new DelayedFileCreator(timer, file);
 }

 private AsynchronousTaskLauncher prepareAsynchronousTaskLauncher() {
  ExecutorService executorService = Executors.newFixedThreadPool(THREAD_POOL_SIZE);
  return new AsynchronousTaskLauncher(executorService);
 }

 private Callable fileIsCreatedOnDisk(final String filename) {
  return new Callable() {
   public Boolean call() throws Exception {
    File file = new File(filename);
    return file.exists();
   }
  };
 }
}

 
The whole beauty lies in readability and expressiveness of Awaitility. We do not have to take care about threads' handling, concurrency aspects etc. Everything is being done by Awaitility. 

I really encourage you to use this small, but very handy library to test your asynchronous calls. 

Uncle Bob - Let's Learn a Little Clojure

Here is a link to Uncle Bob's talk about Clojure. As always, he explains basic stuff in a cool way. Have fun as I had. 

The A Team has A players

I think, we all know that universities' programmes do not conform well to  industry requirements. I would be even tempted to state that computer science and IT are the most vulnerable subjects, in terms of coherence between what is taught and what is required. Basically, teaching too much science can affect industry skills. What I mean is a proportion of industry to science skills (I/S). To my mind, it should be close to 1:1 proportion. In fact, we often observe much higher contribution of science, comparing to industry one. My guess, based on my private research, would be that nowadays the ratio is floating somewhere between 1:4 to 2:3.

To be clear, I have nothing against science. I am even extremely keen on science from my childhood. What's more, I always saw a huge value in being open minded and having analytical skills. 

On the other hand, some knowledge of current, top industry technologies, approaches and tricks is of at most importance. These are your tools. You will be using them on a daily basis. You have to know how to use them!

Okay, rather than grumbling and ranting, let's go straight to the definition of set of requirements and skills necessary in industry. Incidentally, these might be things necessary in science, as well. 
Below requirements were initially defined and put together by Wojtek Seliga, but then enriched in few places by me:

Basic programming skills:

- java.util.concurrent package

- GC (how it works, types)

- Socket programming and threads

- TCP/IP, HTTP (Basic Auth, Cookies, Session)

- Scalability - how to build scalable apps

- Performance - how and when to pay attention for performance

- Transactions - types of transactions

- CAP rule

Java Core:

- interface, class

- composition, inheritance

- collections

- complex types, algorithms

- hash code, searching complexity, putting complexity

- concurrency (thread, monitor, synchronization different approaches, semaphore, cyclic barrier, latch)

- streams

- immutability (thread safe, object pools)

- reflection, AOP, byte code manipulation, generation, dynamic proxy - it explains how dependency injections frameworks, testing libraries work

- Web technologies stack: Struts, Filter, Servlet, Server socket (socket bind accept)

Libraries:

- JDK a lot of classes, on the other hand, not so many to scroll down and eyeball check

- Guava - very good library

- Apache Utils

- Yoda Time - don't use Date and Calendar
- Spring, Nano, Pico, Guice - as dependency injection frameworks

Tools:

- know some IDE (keyboard shortcuts, IDE shouldn't slow you down)

- debugger (not System.out.println all over the place)

- profiler (bottleneck, deadlock)

- internet traffic analyzer (WireShark, Fiddler), FireBug

Books:

- Effective Java 2nd edition (Joshua Bloch) - after reading you think you know everything

- Java concurrency in practice - after reading you think you know nothing

Personality:
- intelligent, smart, active

- willing to change and go out from his comfort zone
- looks for new technologies, experience, A players are not worried to suggest something new, because even if new idea wouldn't work they learn on their own mistakes
- they have high self esteem (often times it's correct)
- they can often find a job
- pragmatic
- there is some 'public track' of them in the internet (forums, blogs, conferences, twitter etc.)

So these are things and topics required by companies from people they are hiring. Now think, how many of these you were taught at university and how many of them you were taught and you know on a decent level. 

Good Recruiter
We were talking about proper set of skills for new joiner. Now let's think about person I call a good recruiter and let's analyze his approach to a candidate. A good recruiter is somebody, who is able quickly and honestly estimate the level of developer. So how they think?
First of all, small companies pays attention for knowledge. Secondly, the most important language in IT is English. It is a sort of must be nowadays and at least good command of English is essential. The candidate should also have a correct financial self esteem and knowledge of market trends at his work. A good recruiter should also check skills, which developer will be using all the time at work i.e.:
- naming variables and methods,
- IDE knowledge, 
- way of writing tests
- refactoring

These are things, which new developer will be coming across on a daily basis and these sort of things should be checked. Please, give me a break with intelligence tests and questions about falling bulb from the skyscraper etc. It's all pointless! Check real skills, which will be used in real life, not some artificially invented questions to show off to the recruited person.

The best boss is the most stupid person in company
There is also a concept of A players. Companies should hire only A and A+ players. 
A players have rightly high self esteem. That's why the best boss is the most stupid boss, as he knows that only by hiring people better than himself, he will be able to build valuable company.
B players are scared a bit. They don't know what other will say. They are a bit lost. B player is hiring weaker, than himself.
C players - misery.

People tend to blindly repeat that developer is the most precious asset of the company. 
If yes, why companies are rising salary when precious developer is quitting?
If developer is so important, stop your most significant project and put the best people (A and A+) to interview, because you can loose cool guy, who is waiting for his turn out there. He will pay off 100 times anyway.

What we see in reality is that companies are sending whoever to interview new, fresh blood to the company. Is it the right move? Is it how it should be done? Definitely not.
If company really truly believes that developer is the most important asset for them, they should use their best people to recruit others. 

Remember: The A team has only A players.

Code Retreat in ThoughtWorks 2012

This time I was in taking a part in Code Retreat organized by London branch of ThoughtWorks and it was awesome !!

I decided to spent half of a day writing code in languages other than Java. I have chosen Groovy, Ruby and C#. I have to say that all sessions went very well. Okay, so let me describe them briefly: 

#session 1: Groovy with no constraints - just a warm up in a language similar to Java.

#session 2: Java and baby steps - this one was very good. The quite nice and neat design was emerging from the code driven by our tests.

#session 3: Ruby and no more than 4 lines of code per method - actually 4 lines of code per method wasn't hard to achieve in Ruby. To be fairly honest, the longest method we had, had literally 2 lines. What's more, Ruby guy, I was pairing with, has shown me some magic in Vim. I thought I knew Vim quite good, however it appeared I am a barely novice :P

#session 4: Java and immutable objects - this was actually quite interesting one, as well. Immutable objects were affecting our design in a way we did not expect, at the very beginning. 

#session 5: Java and no primitives - again another interesting design emerged.

#session 6: C# and tell don't ask (our choice) - that was actually quite a challenge to use only procedures.

All in all, it was very educational, eyes opening and enlightening day. Although there were couple of tricks I learned, I think the most valuable one is related to the way of coping & pasting code. If you really need to do it, do it as below:
- copy & paste the code
- use question marks to denote parts of code you have to amend
- replace question marks with valid code starting from the top

Just to make things clear, let's have a look on a below example:

"Any live cell with fewer than two live neighbours dies, as if caused by under-population."

@Test
public void liveCellWithFewerThanTwoLiveNeighboursDies() {
	int oneNeighbour = 1;
	Cell liveCell = new Cell(ALIVE, oneNeighbour);

	liveCell.tick();

	assertThat(liveCell.getState(), is(equalTo(DEAD)));
}

And here you decided to copy above test in order to tweak it slightly, so that you can get another valid test.

"Any live cell with more than three live neighbours dies, as if by overcrowding."

@Test
public void liveCellWith???NeighboursDies() {
	int ???Neighbour??? = ???;
	Cell liveCell = new Cell(ALIVE, ???Neighbour???);

	liveCell.tick();

	assertThat(liveCell.getState(), is(equalTo(DEAD)));
}

And here is the result:

@Test
public void liveCellWithMoreThanThreeLiveNeighboursDies() {
	int fiveNeighbours = 5;
	Cell liveCell = new Cell(ALIVE, fiveNeighbours);

	liveCell.tick();

	assertThat(liveCell.getState(), is(equalTo(DEAD)));
}

As simple as that, but helps you to avoid silly errors.

Manager in Agile team.

There is no manager in Agile team.

It might be hard to believe, but developers know what needs to be done, as they have backlog and their own brain!
They do not need another guy, who is cultivating seagull management. 

And now shit load of stuff is getting a new meaning :P

Which game are you playing ?

Few days back, I spoke with my friend about pretty "funny" situation. He told me that developers at his project are playing a game, which my friend called Pair Programming vs. Code Review. If developers like each other, they play Pair Programming. If they do not like each other, then they play Code Review.

As you can see those developers found a problem, where it was not even existing. That's a rare and undesired skill, which often manifest itself in pseudo pragmatic teams, who are probably not sticking to 5th Agile manifesto principle: 
"Build projects around motivated individuals. 
Give them the environment and support they need, and trust them to get the job done."
Generally, it is called over-engineering! But not only that, they are loosing time for guerrilla warfare, rather than focusing on helping each other. They also seem not to have any respect to each other. 

The question is: should it be this way? Of course, no!
I would like to be clear about Pair Programming and Code Review, so let's start from some definitions.

Scratching Tiger, Hidden Dragon - Pair Programming

One developer types in code at the keyboard. He works at the tactical level. The other one looks for mistakes and thinks strategically about whether the code is being written right and whether the right code is being written. 

You can also describe pair programming in a more holistic and tao oriented way, as a yin-yang or a unity and harmony arousing from diversity. There are many approaches explaining principles of pair programming like coder and observer, driver and navigator, however I am always keen on scratching tiger and hidden dragon parallel. 

In other words, it is a conjunction of opposite features like active and passive, linear and non-linear or specific and general in a collective action such as solving a problem - programming in this case.

To be fairly honest, one can extend and generalize this approach to any sort of task you have got to do. 

The overall idea is very easy to grasp and simple. Let's look at pair programming from two, different perspectives: tiger and dragon.

Scratching Tiger:

1. Active person writing the code, almost scratching it
2. Keeps on focusing on syntax and details
3. Uses his linear part of brain, logical (left) hemisphere to solve arousing problems

Hidden Dragon:

1. Passive person pays attention to code quality and acceptance criteria fulfillment, almost like dragon, flying in the sky and observing what tiger does
2. Can see whole solution, has an overall idea of subsequent tasks and steps
3. Uses his non-linear part of brain, abstract (right) hemisphere to imagine a flow and grasp the bigger picture

You would probably agree that above description is somewhat static and simply a bit boring. Frankly, you are right, but the key here is movement and synergy you can get from it.

How can we get the movement ?

Again, this is simple. You need a proper space. Comfortable desk, two chars and two keyboards. These things are really important, so do not neglect any of it, especially two keyboards. These prerequisites enable higher level of collaboration between developers. Apart from that they are not loosing time and ideas, while swapping keyboards.

And now comes the most important part. Developers are creating a collective mind, which is buzzing and boiling due to new, fresh ideas. The worst thing you can do, is not switching between Tiger and Dragon role. You have to do it frequently, without any hesitation and freely. You cannot be afraid of your partner saying leave that keyboard, etc. When you see Tiger did a typo, initialized a loop in a wrong way or did not do refactoring, which might be done - simply take over and use your own keyboard to fix it. You need to change perspectives very often and switch your brain from linear to non-linear mode and vice versa. That is the only way you can solve difficult problems. 

Why pairing on solving tasks?

Mainly because of four reasons:

• Readability - two people take care about readable code, both of them have to understand what was written

• Code quality - percentage of defects found 40% - 60% (Steve McConnell "Code Complete")

• Bus factor - knowledge sharing and mitigating the risk of single point of failure

• Pressure - group shields developers from external pressure much better than single developer can protect himself. What's more, in stressful situations people tend to cluster to face difficult situation.

Okay, so when it is worth to pair? 

It is definitely more expedient to pair on crucial and strategically important tasks. It is also useful to pair on both very hard, non-trivial implementations and medium level difficulty tasks. There is no use of pairing on relatively easy tasks, as they do not add so much value, comparing to effort.

Always remember about rotating pairs, not giving an opportunity to get used to your partner. The efficiency is decreasing then. Moreover, it is also good not to pair newbies, as there is not master-apprentice relation, which negatively affects pairing process. Definitely people need to like each other.

Code Review - No junk in a trunk

It is a sort of semi formal walkthrough the code aiming to:

1. share knowledge (mitigate bus factor)
2. check design from high level perspective
3. check at least SOLID rules appliance
4. check definition of done (DoD)
5. increase resilience before holidays

When can you perform code review?

Of course after development and testing, but before merging to master or trunk, so that there is no junk in a trunk.

The programmer or pair, who developed a solution, walks some other team member through new code. They start from telling the story and acceptance criteria, then they are explaining applied design and showing working, green, CI  system tests. There is also a place for Q&A and potentially some code review remarks or changes. Code review is able to discover between 20% to 40% of defects (Steve McConnell "Code Complete").

Summary

Code Review and Pair Programming are two, complementary techniques improving the general quality of the provided solution. Apart from that they mitigate the risk associated with bus factor, by knowledge sharing.

Both of them create a useful unity in development cycle. 

You may even use scratching tiger, hidden dragon parallel, in this case, as well. Pair Programming would be a tiger and code review a dragon looking at the solution from the distance. 

It is much cheaper (sic!) to let developers be tigers and dragons, rather than giving them a wigging that they loose time for crap.

Rarely you have to choose between these two approaches. Then do it wisely and choose pairing. If your management is trying to be clumsy lean oriented and tells you should not do pairing, as it is a pure waste, take at least code review. Remember, half pairing is still better than not paring at all. It is always good to discuss design, start doing the most tricky bit together and then split and finish what you started separately, if time is so valuable. 

However, in fact it does not hold true. You are writing robust and clean solutions faster while paring and reviewing, than not. 

Agile Manifesto Signatory

Few days ago, I have become an Agile Manifesto Signatory :)