Mastering Team Autonomy: A CTO's Dilemma

Exploring approaches CTOs and Engineering Leaders can deploy to strike the ideal balance between autonomy and adhering to coherent standards across their teams

One of the key challenges a CTO or Tech Leader usually faces is defining the ideal level of autonomy within their organization. Particularly on how to do that while keeping their stack and setup manageable, consistent, and coherent.

In today's article, we'll explore this non-trivial aspect of our lives. I'll guide you through some of the underlying theories first, and then we'll look at some approaches I recommend to help you navigate this challenge.

💡 Why Autonomy Matters

There is a conspicuous body of research that points to autonomy as one of the key drivers for job satisfaction, motivation, and performance.

Many of you might be familiar with the very influential book Drive: The Surprising Truth About What Motivates Us by Daniel Pink. The book builds on the results from decades of research that led him to build his model for Intrinsic Motivation.

According to Pink, Intrinsic Motivation is achieved through 3 main drivers:

• Autonomy

• Mastery

• Purpose

One of the key theories that inspired Pink, is the Self-Determination Theory (SDT).

This theory was developed in the 80s by Deci and Ryan and states that humans have innate psychological needs for competence, relatedness, and autonomy.

Autonomy in this context is defined as the need to be in control of one’s own behavior and goals.

It should be of no surprise then that the need for team's and individual's autonomy at work, specifically among software engineering teams and organizations, has been gaining a lot of popularity and traction in the past years.

Though this approach can lead to increased performance and motivation, it's not without challenges. Autonomy is not a discrete entity, but rather a continuum. As such, different people will likely have different representations of what degree of autonomy is “right” for their situation.

If the challenges with total lack of autonomy seem to be well understood, the opposite scenario is not without its own set.

That's what we're going to explore next.

💢 The Challenges with Full Autonomy

I grew up in Italy, and in the early 2000s Berlusconi and his political party were a the peak of popularity.

Please stay with me, I’m not going into a long political digression here.

At that moment his party was called “La Casa delle Libertà” - literally, The House of Liberties.

In those same years a comedy TV show called “L’Ottavo Nano” - literally, The Eight Dwarf - that was openly anti-Berlusconi became quite popular.

Among other things, these comedians had come up with a very effective sketch to poke fun at Berlusconi's party. It was a fake advertisement for the party, that had a very simple tagline: “La casa delle libertà - Facciamo un po’ come cazzo ci pare.” - literally, We are the house of liberties - We do whatever the fuck we want.

You don't need to side with a specific political party to grasp the fundamental concept behind that satire:

The complete absence of rules can lead to the normalization of deviant behaviors. These behaviors can lead to unpredictably bad results.

Shifting our focus away from the political landscape in Italy at the turn of the millennium, we can encounter the concept of normalization of deviant behaviors in a famous book that explores failures in complex systems, such as aviation, healthcare, or nuclear power: Drift into Failure: From Hunting Broken Components to Understanding Complex Systems by Sidney Dekker.

At the core of this book - which is not a lightweight read, you've been warned - is the idea that the normalization of deviance is a major cause of failures and disasters.

The normalization of deviance refers to the process by which individuals within organizations become increasingly accustomed to deviant practices or standards that stray from the originally designed safety protocols or operational best practices.

These deviations may initially arise as shortcuts, under pressures of efficiency, or through a lack of awareness regarding their potential consequences.

Over time, as these practices do not immediately lead to negative outcomes, they become normalized within the organizational culture, effectively lowering the bar for acceptable behavior or performance.

My theory is that a situation of full autonomy will expose your organization to the pressures of efficiencies and short-term gains to the point that it will incentivize the normalization of deviant behaviors.

I will cause a broader set of issues as well, such as stack fragmentation, insular culture, and friction in internal mobility. But in essence, I believe they could all be grouped under the general concept of normalization of deviance.

So, what level of autonomy is the right one for your organization?

As a CTO you're left to deal with this dilemma.

Which one should you pick?

🤔 The CTO Dilemma

Regardless of whether you're joining a mature organization or an early-stage startup, at some point, you'll have to confront the challenging task of establishing a clear and explicit level of autonomy for teams in your organization.

On the one hand, you want to motivate people and promote innovation. As such you will want to ensure a your teams benefit from a reasonable level of autonomy.

On the other hand, going towards full autonomy will increasingly expose your team to the natural forces that any business is subject to: growing faster and making more money. In the absence of back-pressure mechanisms, these can push your team to lower their standards and take shortcuts.

Unsurprisingly, the latter is often what happens by default as leaders might struggle to find a way to set boundaries as they're afraid to be perceived as too strict, dismissing people's seniority and expertise, and micro-managing.

This poses a non-trivial challenge for engineering leaders, as they are left to resolve this ambiguity in a way that balances both individual and company needs; both short-term impact and long-term sustainability.

What I advocate for first is to take a proactive approach to the problem. It is your responsibility and it is in your own interest to create clarity by setting explicit boundaries of autonomy.

It's through the act of setting such boundaries that you can unleash the full power of delegation and decision-making at scale. If you do a good job, you allow individuals and teams to make decisions quickly, without having to ask for permission, in a way that ensures consistency, quality, and interoperability.

How do you figure that out?

Let's look at some tactics and approaches you can deploy in your organization for that purpose.

🌐 Unlocking Coherent Distributed Decision-Making CDDM

As usual, there isn't a single silver bullet that will allow you to unleash autonomy in your team. Instead, you should leverage multiple tools and approaches, complementing each other to support you in striking the right balance.

These are all approaches I've seen used in practice with positive results. Your mileage may vary, and you don't necessarily need to deploy all of them. Adapt them to the level of maturity, size, and complexity of your own reality.

Let's look at them in detail.

🧭 Establish a Tech Strategy

This is often the biggest missing piece in small to medium-sized technical organizations. Even though there might be some implicit guidelines for how to direct the efforts of the engineering team, usually a structured, crisp, and well-documented strategy is missing.

Engineering leaders who have never developed a strategy before might feel intimidated by the task, struggling to figure out where to get started.

Without a solid and well-documented strategy, you can only hope that people in your team will organically come up with the right decisions for the whole company. But as the SREs at Google famously said, Hope is not a strategy.

As a CTO it is your responsibility to ensure the collective efforts of your teams are directed towards what matters most for the company.

By establishing a solid tech strategy, you'll provide your teams with both directions and boundaries to help them make decisions autonomously, as long as they stay within the general directives of the strategy.

This is one of the most effective tools to implement coordinated autonomy within your organization.

Defining an effective tech strategy is work that requires the involvement of many key people across the entire team. If you try to set the strategy alone, not only do you come up with a suboptimal result, but you'll also have a hard time getting buy-in from the people you lead.

The nice side effect of its collaborative nature is that the act of defining the strategy itself will increase the feeling of agency - and therefore of autonomy - of all the people involved.

This is in line with the definition of autonomy provided in STD: the need to be in control of one’s own behavior and goals.

🥇 Define Common Engineering Principles

This might or might not be part of the overall tech strategy. I generally prefer to include common principles and practices in the strategy document(s), but decoupling here can have its benefits.

Especially with a team that has never worked with tech strategies before, it could be beneficial to start by establishing these common principles. It will allow you to start with a smaller scope, leveraging a lot of common wisdom already available in the industry at large.

In other words, while the tech strategy will likely be very tailored to your own context and organization, engineering principles tend to be more universally applicable. As such, you will spend more time selecting among available choices rather than having to invent your own. An easier task, and one that can be completed more quickly.

These principles will guide members of your teams on how to make decisions in their day-to-day jobs. There will still be room for interpretation and minor deviations, but the agreed-upon principles will allow you and the team to have informed and objective discussions whenever they face disagreements on how to interpret or apply them.

As for the overall tech strategy, the recommendation of coming up with Engineering Principles for your team collaboratively also applies.

You can and you probably should start small. Pick an area that is particularly relevant for your team - test automation, resiliency, code reviews, etc - and define the first set of principles you want to start testing within the team. Once you have reached a good level of adoption, you can start adding more, ideally combining them in a single Manifesto-style document that provides a coherent view and narrative around your choices.

🚫 Establish Clear Boundaries of Ownership

Tech Strategy and Engineering Principles will provide your team with guidelines on the Why, What, and How of decision-making.

Setting clear boundaries of ownership will provide clarity on the Where part of it, i.e. the scope.

You want your teams to know exactly the scope of work that is under their responsibilities, with as little ambiguity as possible.

You want to have unambiguous answers to the following types of questions:

• Which team is responsible for dealing with this bug/issue/incident?

• Which team is responsible for budgeting this service/tool?

• Which team is responsible for setting reliability targets for this service?

If the answers to some of those questions are unclear, you will want to resolve them.

In some cases, there won't be an optimal resolution available.

I believe in sub-optimal being better than just ignoring the problem. That's because once you have assigned ownership, one team will be in a position to evolve the underlying system to improve their ability to fully own it. Without that first step, these orphans will linger around for an indefinite amount of time, often causing a whole bunch of headaches.

Some corollaries to this approach are that:

• Every line of code should have a clear owner

• Every service running in your architecture should have a clear owner

• Every domain name, public IP address, or third-party tool should have a clear owner

You will not get this right at the first iteration.

Think about this as approaching asymptotically the ideal scenario while never fully reaching it.

It's OK that your first iteration will have a lot of suboptimal boundaries. Don't make that an excuse to not get started.

Use it as an opportunity to make those problems more evident, and give a clear mandate to people to start addressing them.

💾 Introduce a Structured Process for Tech Innovation

One of the most common arguments against establishing standards and common practices is that it could stifle innovation.

Besides the fact that sometimes we confuse introducing a new language or framework with innovation - something I'll write about in a later moment - I don't believe this is necessarily true.

I also think that the act of introducing a new piece of technology in an existing stack is something that you should evaluate carefully as it brings with it a non-trivial baggage of additional complexity that you and your team will have to live with for a potentially long time.

For that reason, my recommendation is to establish a process that regulates how a new technology should be introduced in the company. This process should be as lightweight as possible, but not more.

A bit of up-front work establishes a healthy discipline of thinking about the benefits and the costs of any decision, and can significantly reduce the risk of falling prey to the Shiny Object Syndrome.

This process should require that the team or individual formulates a proposal, starting from the problem they're trying to solve, the benefits of the new solution, and the potential implications. It should also include a rough timeline for evaluating the new technology in a small scope, as well as the evaluation criteria that will be used to decide whether or not to promote this new technology to an internal standard.

You can start small, but over time you will need to cover more aspects related to lifecycle management, including deprecation.

This last point is particularly important as you want to make sure you don't keep growing the list of technologies in use indefinitely.

🃏 The Delegation Poker & Board

All of the approaches we've looked at so far generally apply at the team level. In your position, you're not only leading teams but also individuals.

One tool that I've found very effective in managing expectations around autonomy for individuals reporting to me, be they managers or individual contributors, is the Delegation Poker & Board from the Management 3.0 framework.

I'll focus here on the gist of the approach while leaving the detailed instructions to the official page.

In essence, it is an engaging way to bring clarity and set expectations for effectively delegating tasks and decisions to members of your team, in a controlled way.

What's powerful is that you can have different Delegation Boards for different people, taking into consideration not only the scope of their work but also their level of seniority as well as their own appetite for more autonomy.

It will not be uncommon to end up with a higher degree of delegation with more senior members of your team while retaining a higher degree of control for those who are still lacking the required level of experience. This will evolve as people gain more experience and seniority through your guidance.

This implies that you should review the delegation board with each one of your direct reports regularly, but not too often either.

Doing it twice a year and/or every time there is a significant change in scope for their role could be a good starting point.

🏁 Conclusions

In today's article we've looked at a common challenge many CTOs and engineering leaders are facing in their organization: establishing the ideal level of autonomy for their teams.

We explored the general concept and the research behind the importance of autonomy and then looked at some practical approaches you can deploy to strike the right balance in your team. These include:

🧭 Establish a Tech Strategy

🥇 Define Common Engineering Principles

🚫 Establish Clear Boundaries of Ownership

💾 Introduce a Structured Process for Tech Innovation

🃏 The Delegation Poker & Board

I'm thinking about doing one or more follow-up articles to dive deeper into how to deploy some or all of the solutions I'm suggesting.

Which one are you struggling the most with?

Which one do you think I should focus on more?

Please share your answers in the comments section below. They will be extremely valuable to inform my decision.

Thanks in advance, and see you all next week!

📚 References

If you want to go deeper on some of the topics mentioned in the article, here is a list of useful references that you can consult:

• Daniel H. Pink (2009). Drive: The Surprising Truth About What Motivates Us

• Sidney Dekker (2011). Drift into Failure: From Hunting Broken Components to Understanding Complex Systems

• Deci, E.L., & Ryan, R.M. (2017). Self-Determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness.

• Management 3.0