Friday, 2 May 2014

Watch 'Crazy House', on Friday, after 3 pm, at your work.

If you are lucky, while working on an open space, sometimes, you may see people running back and forth, impinging each other and behaving exactly like headless chickens - especially after 3 pm on Friday ;) The reason why you may see that somewhat peculiar picture is because of either somebody's birthday (i.e. cookies and doughnuts) or production issue ;) Check your mailbox and if there is no email about doughnuts, it has to be an issue on prod!


Usually, managers refrain from communicating a problem to business, before getting some initial knowledge about it. However, do not be mislead by lack of email. That state would not last long. That is actually the time, when managers start to create an Asterix and Obelix 'Crazy House' cartoon atmosphere, harass developers, L3 or any other support team. Of course, they think they do their best to behave in a calm and professional way, but subconsciously they do what all people do, when they are jeopardized - they are clustering into harassment groups behind the back of (un)lucky developer, creating totally unnecessary pressure. 

Then, there is a time for usual questions cannonade: where are we, what we know, what is the risk, what is happening, what are our options etc. You may have an ironic answer for all of them, but you must keep a poker face.



In fact, calmness is your bless. The more clam and methodic your approach would be, the sooner you would be at your home sipping whisky. So calm down and follow below steps:
  1. Ask for a little bit of context and clarification what they think is all that fuss about.
  2. Do not easily believe what 'headless chickens' are saying ;) Check it yourself. If they knew what is going on, they would not harass you ;) Simply ask for evidence based on which they raised an alert. It might be a monitoring screen shoot, DB query, logs, whatever. You may see something, they may have skipped. Also, it is pretty common that people misinterpret what they see and hit 'panic alarm' straight away. It happens, you must live with that.
    You may also ask a couple of context enriching questions like 'who said that' and 'based on what premises you thing so'.
  3. If you really can smell the issue, do an initial investigation and try to estimate:
    - impact
    - risk
    - also it would be marvelous, if you could apply some trends checking here.
    If possible, check whether same or similar situation was happening in the past. Perhaps, it is a regular behaviour.
  4. If there was something similar happening in the past, try to dig out the knowledge about that issue and its remediation. JIRA ticket, conversation with other peers in the project, basically anybody or anything might be helpful.
  5. If not, then it is a genuine issue, which has to be investigated.
  6. If you finally come across the solution and it involves a fix on prod, try to mitigate its risk, by doing the smallest possible change. 
    Remember: small change = small impact (according to stable system definition).
    Assess minimal, optimal and maximal downtime of the system vs. you SLAs. Communicate it clearly to your business stakeholders, prod services and managers.
    Also do not let them suggest solution, unless it has reasonable base.
  7. Test your solution on prod like environment (including starting and stopping your app) and write full sequence of steps. Do not let anybody add/change/remove any of steps, before agreeing it with devs, prod services and testing them properly.
    It's better to have a broken system, where you know what is going on, rather than possibly fixed one, where nobody is entirely sure, if and how it's gonna work.





Saturday, 22 March 2014

Growing applications handled by micro services approach

Let's have a look on characteristics of systems we are building. In general, clients spend only about 10% of their money on building application. The rest 90% of their money is the cost of ownership. Our systems are getting less maintainable. They grow in size and from time to time we are witnesses of IT failures, like 'the biggest IT failure ever seen'. Also, for some reason, there is a tendency to rewrite the entire application every 2 to 5 years.

Above observations and concerns are leading us to one of the great bugaboos of software applications over the years, which is their size. The problem is that we simply tend to build too big systems! Although above statements seem to be quite obvious, we still do same mistakes all over again. 


Let's think for a while and ask ourselves a question: what we do, when a given class is doing too many things? We simply go and refactor it, so that it sticks to SRP principle. Second question: what we do when our system is doing too much? Basically, we keep adding new features. It is extremely counter intuitive, but that is what we do. In fact, I am not even surprised, as the most common way of teaching TDD is to apply it together with SOLID principles on a unit level. Of course it is true, but it might be extended to all levels of abstraction. However, having a tangled application, it is our duty to apply SRP principle on a system level. Such an approach may lead us, but is not limited, to micro services architecture.


Micro services characteristics

There is no official definition of micro services, but they do have their own, specific characteristics. Micro services might be treated as a result of applying refactoring, with an SRP principle in mind, on a system level. They enable easier testing and re-usage of services. Also, you may treat them as fine-grained SOA approach. Moreover, micro services create real options, as you may adapt different (right) tools for different jobs (Unix system analogy). Exploiting micro services style we are getting independent deployment and scale, easier test and a change (side effect of small systems!) and very important option - flexibility.

Architect and his job

Let's step back for a second and describe what an architect does? An architect is a person who is the 'chief builder' of a system. His role is to satisfy clients' needs. In addition, he should do 'just enough architecture and design upfront'. 

In terms of architect's work, it might be compared to city planning. All major requirements have to be envisaged in the plan and embraced accordingly. For instance, there has to be a place for industrial and housing zones, possibly some sub-zones for light and heavy industry, too. I am sure, people would also appreciate nice city centre with greenery and leisure bits. On the flip side, it would not be a good idea to build a playground for children in the middle of industrial zone, instead of housing area. What is more, such a city should have shared utilities like: electricity, waterworks and gas pipelines. 



All these decisions and many others must be done upfront. However, they cannot restrain or impose too tight constraints on zones themselves. That is why, an architect should do 'just enough design upfront'.

Getting maximum flexibility with upfront design
'Just enough' is in fact very fuzzy quantifier of 'upfront design', so in real life we need more concrete guidelines. We should be starting from asking a question, how we can provide just enough upfront design in order to provide flexibility in our architecture? 

Having city planning example in mind, it may lead us to a conclusion that such a high level plan is actually something what business people call a capability map. In simple terms, it is a diagram, which maps business needs and requirements into a set of services talking to each other, where services might be zones and interfaces between services might be utilities, in our city planning example.

In order to get a flexible design, apparently contradictory, but in fact complementary concepts are used i.e. evolutionary architecture and emergent design.


In a nutshell, evolutionary architecture is 'all the stuff that is hard to change later' ;) - usually these are interfaces and contracts between components/systems. That is why one of the core rules of evolutionary architecture suggests to defer decisions as late as possible (aka keep your options open).
On the other hand, there is a concept of emergent design. It is an idea of deriving a system design in the continuous process of adding incremental changes.
Here comes another, tricky question, how we should use these techniques? Luckily, the rule itself is fairly simple. We apply emergent design on a level of bounded contexts/services (zones) and evolutionary architecture on a level of gaps between bounded contexts i.e. interfaces.

Hexagonal architecture

Also, while designing the general architecture and communication between components/services, it is very important not to obstruct the natural, creative flow of the thoughts by distorted diagrams.



We should be using right tools from the very beginning - hexagonal architecture approach is one of them. It gives us extremely clear view on what should happen in the core of a given application and how that application should communicate with external systems. Apart from that hexagonal architecture introduces more generic concept of looking at a system. In contrary to common approach, we are able to freely express more than two integration points in our model (i.e. UI and DB integrations). 



Incidentally, if you were imaging a system that is described in terms of hexagonal architecture and sticking to SRP principle, instantly you would get a handy tool for depicting micro services and making SRP fully visible on a system level! Apart from that such a micro service also conforms to Bovine's conjecture: "objects should be no bigger than one's head", which in our case applies to the system.


Integration

Assuming we have micro services in place, we can start introducing well-defined interfaces (contracts) between them. It will give us a power of communication with external world e.g. micro services.

There are two, very important attributes of decent software:
  • high cohesion
  • loose coupling
High cohesion relates to a service. Basically it means service should have a single reason to change - that is where SRP comes from. In terms of loose coupling, it might be treated as description of components dependencies to other components. In this case it should be as loose, as possible.

Over the years, in software design we went through many integration patterns and anti-patterns, as well. It would be worth going through majority of them and pointing out theirs characteristics.

Data oriented integration (dependency on DB level):
  • no loose coupling
  • hard to reason about
  • brittle approach
  • difficult to maintain and change

Procedure oriented integration (RCP, CORBA, WSDL-binding, JAXB, Java Serialization):
  • method calls across boundaries
  • sharing serialized object (tight dependency)
  • adding methods with different input parameters doing similar things
  • above leads to 'God classes' syndrome 
  • above leads to very difficult change
  • above leads to 'objects explosion'
  • above leads to SRP and interface segregation principle (ISP) violation

Document oriented integration (JMS, messaging in general):
  • asynchronous (decoupled)
  • allows additive changes without breaking existing clients
  • adding a field in document without break
  • renaming a field in document, by adding new field - no breaks
  • requires middleware

Resource oriented integration (REST, not HTTP):
  • language agnostic 
  • expose state
  • noun oriented, not verb oriented like HTTP

When we talk about integration, it would be hard to forget about Postel's law: "Be liberal in what you do, conservative in what you expect". It might be translated into slightly more practical hint: "only bind to what you need, to reduce breaking service consumption".

As we see, there are many integration styles. However, RESTful approach looks like the most promising one. It is a fairly light concept, providing desired components' properties like loose coupling and it does not repeat errors from the past.

Summary
I believe I have outlined main issues in software design, prodding us to better understanding of the nature of the problem and finding out some better solutions. Searching for a right design is a continuous process and starts from the very beginning. Collecting and defining clients' requirements, the time architects spend on thinking about all use cases and satisfying client defined criteria, developers correctly modelling and implementing ideas can never be neglected. All these actions are crucial in the process of building anti-fragile and well-designed software.

Monday, 13 January 2014

Class naming convention and information theory

How many times have you seen classes named with one of these suffixes: manager, util, helper, common, impl etc.? How many times have you seen columns in same DB table called *_id? Probably far too many. 

Have you ever thought about informativeness of above suffixes? Isn't it loosing its point of passing a piece of information to other developers, when it is overused? To be frank, it looks like a noise, a message which is blurred by useless words. The more often those funny words occur in code base, the less information they carry. That is actually, what Shannon said when he was building keystones for information theory.

In fact, it is very important, to give a proper name for a class, DB column name etc. On the other hand, it is also very significant not to overuse same words all over again in names. It is not only getting less readable, but also adding a kind of boilerplate, giving no informativeness value, at all. There is of course a value in pissing other developers off, but I guess it is rather doubtful pleasure ;)