Making architecture decisions with distributed teams

When it comes to software architectures I am with the agile manifesto that states “The best architectures, requirements, and designs emerge from self-organizing teams.”. As a consequence no single individual should be making all the important (that is the architectural) decisions for a software. I like to see teams tackling these decisions by a quorum of team members, deciding on the important stuff in consensus. While this is a hard thing to do for on-premise teams, it’s even harder to pull off when your team is highly distributed and you can’t just gather everyone around a whiteboard. In recent weeks I’ve been trying a few things to achieve this way of collaborative decision making with a highly distributed team. Here’s what worked for us.

The architect as a moderator

Moderation is a huge part of the technical leadership skills I have come to expect from Software Architects. Even if you don’t have an explicit architect role in your organisation or team, you should make explicit who will be moderating the discussion. Their responsibility will be to drive the flow of ideas and the discussions towards a common solution. You can’t imagine how? Let me give you some examples.

Imagine you’re the moderator for this discussion, when you notice that your quorum is stuck discussing minute or irrelevant details of a single approach. It’s your job to shift the collective attention into a more productive direction. You tell the team “I feel like we painted us into a corner discussing the downsides of eventual consistency. Can we find any more upsides of this approach, or ways to deal with it or even another approach that doesn’t have this downside at all?”. Your team gets unstuck. Well done!

Then you notice, that one of your team members is very quiet and has not been contributing. You decide to try and activate them by asking “What do you think about this approach?”. On the other side of the spectrum when someone is dominating the discussion it is your job to gently remind them that their ideas are not the only ones that count. “I feel like we talked a lot about this approach, but I would like to have at least three alternatives to decide upon, so we can say we looked at all the possibilities we could think of. So what other approaches should we discuss?”

Silence ensues. No new ideas are flowing. Not it’s your job to get them started again. You try with “Okay, what is the simplest thing that could possibly work?” or “Has anyone solved something like this in the past? If so, what did you do?”. By now your head is probably full of ideas of your own. But you don’t push them. Instead you try to put your ideas into their heads by asking the right questions. “Is there a way we can get some of the benefits of messaging without requiring a message oriented middleware? Maybe we can do this over HTTP instead?”. Make sure your team did not take your bait and has nothing left to contribute, before you start explaining your own ideas.

Don’t be afraid of awkward silences, especially when you are working in a distributed team. Sometimes people need some silence to think. But note that these awkward silences may also be an indicator of low trust in your team. If that is the case, I recommend you check out the Five Disfunctions of a Team. However the silence could also be a sign that your team lacks the sufficient knowledge to tackle decision like this. Which brings us to the next point.

Make sure everyone is in the know

So you are building a distributed application. Does your team know about common ways to integrate distributed systems? What about their unique advantages, challenges and downsides? If not, you should take the time to bring everyone up-to-speed before tackling important decisions, so everyone is discussing things on eye level.

You can either present things yourself, or you can assign the research work to different team members and have them present their topic along with associated advantages and challenges to each other before going into a decision meeting. We have found this to be an essential prerequisite to a fruitful discussion, and it is well worth the time you invest.

If you even have the opportunity to invest in a learning spike, that is a tiny independent application using the approach or technology you want to evaluate to get some hands-on experience, that’s even better. I learned from James Shore to even keep these learning spikes in my repository as a reference for later (“How do you write a multipart form submit in JAX-RS again? Oh I’ll just check my learning spike!”).

Use visual models

Now that everyone can discuss the subject on eye level. But you still need to make sure, that everyone is talking about the same thing. In my experience that’s proven even more difficult in distributed teams than with co-located teams. That’s why visual models and structured record keeping of the things that were discussed have proven invaluable to us. I’ll address the visual models first.

We think about abstract things in mental models. That is, when we imagine structures of things like classes or objects or even systems we tend to visualise these and their relationships and attributes in our head. And when we talk about these abstract things we usually construct our arguments based on the mental model in our mind. But nothing guarantees that these mental models are shared between all team members. That’s why it’s easy to argue for long periods of time without noticing that there’s actually no disagreement, except for the mental models that differ. This may happen because there’s some kind of uncertainty or you are using imprecise or ambiguous language for things.

At that point it’s essential you start drawing some shared models in the form of a diagram. These don’t need to be fancy or beautiful diagrams. You don’t want your coworkers to wait while you try to find the right UML element in a palette of 238 boxes and arrows. All you need now are some basic boxes, lines and labels.

If you are distributed you have a several choices to share these diagrams with your team mates. I have used a whiteboard in front of a webcam to communicate my designs (very crude, but works if you make sure your colleagues can actually see what you’re drawing and reflection or bad camera resolution don’t render the whiteboard illegible). I have also used something like Paint on a Tablet PC where you can draw things on the screen. If you don’t have a Tablet PC, you can get a cheap one on eBay just for these discussions or you can get a document camera and use that as your webcam image instead. Just find something that works for you.

Keep track of the alternatives

Last but not least, we found it’s essential to keep track of the alternative approaches you discussed. As with the diagrams, keep this very simple. I have used Word documents screen shared over Skype to record all the alternatives we discussed. We also note down their respective advantages and shortcomings we identified.

You will probably need this log of your trade-off anyway, if record this as part of your architecture documentation. This will also help you moderate the discussion by referring to different approaches by their number. “Solution 3 looks good, but I don’t like that drawback here. Can we find a way to achieve the same advantages without the drawback by combining it with solution 2?” or “I think we discussed the drawbacks of solution 5 in length, do these also apply to one of the other approaches or can we find an upside, because currently there are none?”.

When you finally agree on a way to do things, you can record the decision in this document as well as the main reasons why you chose to go with it. And then post it somewhere public, for all that couldn’t take part in the discussion to see.

It works

I have seen this approach work well both with on-premise and very distributed teams. The main challenges I found when it comes to the distributed teams are bound to the limited bandwidth of the communication channels we usually use to share our thoughts over the internet. By using more of the senses and getting some form of visual aid by sharing your screen or whiteboard, you can increase this bandwidth by some and prevent some of the most costly issues I’ve seen distributed teams struggling with. Give it a try and drop me a line if it worked for you. Also if you have another tip what we should try, feel free to drop me a line on Twitter!