Did you say agile?

Did you say agile?

What can we learn from the latest evolutions of software organization?

The recent success stories in the new economy are often small start-ups that begun from scratch in their "garage", working with agile processes. "Small is beautiful" was their motto.
The interesting question is to know if these are only exceptions due to some brilliant founders, or if there are lessons that can be learned from it and applied to larger companies’ organizations. It is interesting how companies like Google, Facebook or Amazon keep being flexible and innovative despite a consequent size (32 000, 3 000, 22 000 employees respectively).
At a time when "offshore" is as fashion as "lean", how to adjust organization and process to reach maximal efficiency?

When 1 is better than 10

The famous iPhone software (operating system) was developed by 60 developers, while Motorola has not been able to develop a competing operating system with 1 500 people: the quality of developers cannot be compensated by the quantity. Besides, an oversized team is often counterproductive. We saw projects taking several months of delay for 15 man days of development only.
Why? First, at the simple developer’s level: "a good developer is worth 10 mediocre" (study of Sackman, Erickson and Grant). Good developers code better and faster. Code quality impacts further development phases: testing, debugging, maintenance and upgrade. Then, at the team level: skills or incompetence are leveraged because the actions are not parallel (where a mistake in an action would only impact its own results) but interrelated:  an error impacts everyone and delays the entire project.
These laws apply to many other sectors, as long as we are not in pure execution: craft industry, creation, analysis, etc. Management must be adapted and enable initiative taking. Indeed, pure execution gets much better along with a "military" management, where a clear and detailed executable order is dictated to the workers. As soon as we expect some initiatives from the workers, we are not in pure execution. Initiatives are expected from all the managers, but also from the technicians and the other populations which are not pure executors, and in a broader sense, to any big project.

Simplicity and lightness

Let’s focus back on the software to understand what made these teams efficient. First we have to admit that describing what a software shall do (the so-called specification) can be as complicated as to actually coding it! Besides, the specifications are always subject to interpretation, thus, always false (questionable/incomplete/not precise enough). This assessment leads to limit as much as possible the phase of specification and to merge it with the development. But it is not about dashing first and thinking afterward either. It is simply about limiting the scope of the specifications to the overall architecture and interfaces, and then cutting into coherent pieces. Thanks to these methods, Facebook publishes a new version every week. Firefox has reduced the delivery time for a new version from several months to six weeks.
What tools to set to achieve this?
The teams are small and accountable for the delivery, with a functionality manager (product owner), a unique representative of the client, a Scrum Master aiming at facilitating cooperation, supporting the team for its external relations and reliving all the little pains not directly related to the product. And last but not least, a small competent and accountable team of developers with a broader vision of the project and a witty sense of initiative. Just the opposite of what we can find in certain projects where there are more decision-makers, project managers, approvers, etc. than of actual actors.

Dare flexibility

So, specifications shall be light, evolving and easily adaptable to the requirements of development: they are only a tool in the software production, not a bible written in the stone. They must be reduced to the strictly necessary, shall only give general orientation and objectives in order not to influence the development and they must be able to evolve during the project according to the contributions of the participants.
The detailed specifications contract is no longer needed: it was indeed reassuring but inadaptable! In fact, they are always false and become a burden from which comes the classic conflict between the customer and the supplier over the delivery deadlines. They shall be only a support for a dialogue, one of the tools allowing sharing the vision of the product with the developers. The dialogue shall not end until final delivery.

Detailing is not winning

This is a teaching widely generalizable: how many hyper-detailed schedules have not been respected before they even started? How many 50 pages contracts, hardly read, will be useless until we finally realize a fundamental misunderstanding?
Details can be reassuring but may make you miss the main part. Is the 25th decimal of Pi exact? Pi = 31,41592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117067? I let you check... 
Culture of simplicity is very difficult to obtain. Simplicity is highly complicated… 1/2gt²… Maybe you remember this formula of gravity. It will be necessary to wait until Galilee and Newton to have a simple formula, which neglects the frictions of the air, but is right in general. Everybody is not Newton. But, by looking for simple and concise results, we are more likely to head toward the right direction rather than by trying to produce a block of 200 pages.

The internal customer/supplier culture

The internal customer/supplier culture has many benefits. Obviously one of them is to make internal processes much clearer and understandable. Another one is to cut the process and the project into intermediate measurable "deliverables", in the spirit of what is described above. Nevertheless, in many cases, it leads to extra costs by accumulating the “whishes” as well as by ignoring existing constraints. That’s s what we called technical complexity:
•    “The product is too expensive because those lazy guys from purchase department failed to find my piece at the right price. They did not respect their SLA!” “But of course, you might have forgotten to say them that 3m was an approximate length, and that you would have been well satisfied with the standard of 3,02m.
•    “I specified a column for months and another for dates. In fact, this is not was really need, but the project has been delayed by a month and now I cannot change it in the specifications.”
Does it remind you of something? Product co-writing by marketing and developers is essential to the overall efficiency of the project. The discussion between both teams about the interfaces (software, physical) should be bilateral negotiation rather than one-sided orders.

Iterations

Along with these flexible specifications, flexibility requires iterative processes adapted to changes: the software is developed by successive and periodic increments called sprints.
A validation is made at the end of each sprint (rather than at the end of project); something assessable is to be delivered at the end of the sprint. New features can be added to the specifications in every sprint. The previous codes can also be optimized (re-engineering). The sprint (both its progress and its result) is analyzed to prepare potential trainings, improve processes and allow the transfers of in-house knowledge.
Sprints typically last a fortnight. But of course this must be adapted to the size of the project: 2 hours to prepare a presentation, few weeks for a large industrial project. The right duration balance between seeing something concrete and measurable on one hand, and a certain stability of the expressed needs as well as a certain initiative leeway on the other hand.
These methods can truly be applied on projects as diverse as the preparation of a seminar or the launch of a new product. We saw customers asking for minute by minute schedules of a three-day seminar, although it was clear that the schedule would have to be adapted to participants’ reactions.

You may wonder why estimates are almost always wrong

You may wonder why estimates are almost always wrong. The boundaries of the project are not clearly defined yet. The difficulties are not immediately visible. The estimates are made by the management or the marketing rather than by the operational people, who usually have a better grasp of those insignificant details that might prove to be extremely time consuming. The learning from a previous project is transposed to another one, without taking into account the specificity of each of these projects…
Project hazards go against our rationality and our optimism. But technical challenges, unexpected changes, setbacks and mistakes are inevitable. If there is none, we can start wondering if the project really creates any value! One must accept unpredictability and narrow it down by focusing estimation on portions of projects and regularly reassess them.

Get-it-right-from-first-time

Then comes the question of quality. The “get-it-right-from-first-time” is always widely above the "statistical quality". If the flask of shampoo is not put out of line because a simple bar detects that the stopper is not well screwed, you can correct it immediately and you ensure a 100% quality for cheap. This would be impossible to reach with an end-of-line control.
How to translate it into acts?
•    First via the culture:  try to avoid the "we will correct that later", from the spelling mistake which would eventually be detected by a proofreader to the serious "bug" which is supposed to be detected by the validation/quality team.
•    Protect your teams from "crunch mode": the peak of hard work to meet the deadline. The increase of pace is made at the expense of quality, which can cause setbacks.
•    Set up immediate simple checks: consistency of the results, sprint validation by the client
•    Build clean and legible tools: the IT code must be as clean as the Toyota workstations! 
Having your work (plans, code, presentation, business case, etc.) challenged by your peers is a good habit: you work much better when you know you will be assessed by your peers. We deliver a clear, and correctly presented material, out of respect for our peers. The return on investment is usually immediate.

Technology and market watch

Finally, the essential of good management and personal organization books: start with what is important rather than what is urgent. People who will remain above the fray are those who have managed to remain constantly on the watch for the latest developments of the state of the art, and thus be able to choose the right model, tool, language, framework, depending on the problem to solve.
It is often tempting and comforting to spend time dealing with the current affairs, while technology and market watch or investment seem more random. However, it is usually more efficient to find a way to finally solve a problem, finding an existing solution, automating it or delegating it ("teach a man how to fish and you feed him all his life"), than resolving it on your own. While the search for a lasting solution takes two or three times the time to find a simple solution, profitability comes usually very fast.
Finally, the so-called “agile" revolution in the IT world is full of teachings for the management in many other projects. We could say that this revolution is the continuation of "quality" methods initiated in the 70s. Yet these "quality" methods were so badly implemented, and generated so many reports and reassuring manual that never left the cupboards, that we thought it was worth to (re)draw an overview of these "disruptive" methods enabling to launch a new product every week (and not every year), and without any additional cost.