How to Structure Tech Innovation in Your Team
Dive into the essence of balancing autonomy with standardisation in tech innovation. Explore practical insights and notable industry examples like Apple and Netflix to craft a successful innovation pr
In a recent article, we discussed the common CTO dilemma of balancing Autonomy and Standardisation in your teams.
One of the practical recommendations from that article was to establish a structured process to manage technical innovation.
Today we'll go deeper into this specific topic, introducing a set of considerations required to be successful.
This article will be followed by more of a how-to guide on how to set up your process.
Let's start by looking at some notable examples in the tech industry.
💡Some sources of inspiration in the industry
Much has been written and discussed about the innovation process in tech companies.
Some notable examples include:
The “10 to 3 to 1” design process at Apple
“Let 1000 Flowers bloom, then rip 999 of them” from Twitter (pre-Musk)
“The Paved Road” approach at Netflix.
🍏 Apple “10 to 3 to 1” process
Apple follows a rigorous design process internally where they first develop 10 high-fidelity prototypes and designs for each new product. These 10 designs are then reviewed, and the 3 “most promising” ones are selected.
The team works refining these 3 options further, and then eventually the team ends up selecting the one option that will be turned into a final product.
This is very peculiar to Apple, which is a product design-centric organization.
It's well known that Apple doesn't do user testing or other form of experimentation involving real users. Instead, they focus on having these very intense and rigorous internal processes of natural selection, to ensure the final product will be of a very high standard.
Their process is focused on product innovation rather than technical or tooling innovation.
The key principles could be also applied to the selection of the next language or framework to use.
I'd like to thank
for introducing me to Apple's approach in the previous article's comments section!🌷Let a 1000 Flowers bloom, then rip 999 of them
In a notable 2015 article, a past member of Twitter's Engineering Effectiveness group shared insights. They explained that, for a long time, Twitter's tech stack developed in an organic and somewhat chaotic manner. Recently, however, there's been a shift towards emphasizing standardization. By focusing on uniform tools and programming languages, the goal is to significantly enhance quality, efficiency, and overall joy for the engineering team.
Things might have changed significantly since Musk took over, and I don't have visibility on the current efforts around Engineering Effectiveness at Twitter.
Many of the considerations you'll find in that article apply to organizations of a significant size, which represent a minority of all the companies out there.
Though the article is almost 10 years old, many of the concepts it introduces are still relevant today.
🛣️ The Paved Road approach at Netflix
Netflix is a company that has always focused on the concepts of blitzscaling, not having rules, and focusing on the Freedom and Responsibility mantra for their teams.
Despite that emphasis on autonomy, they are making non-trivial investments in what they call the paved road.
There are teams in charge of making the experience of using well-supported tools as efficient and effective as possible. Their use is not enforced by any means. The investment made in supporting them is supposed to make the experience of going down this path significantly better for engineering teams.
The approach at Netflix focuses on positive reinforcement: rather than coercing people to adhere to a standard, they make the standard so compelling that many teams would willingly adopt it due to the massive benefits it brings.
Teams that decide to follow an unpaved road instead and use a different approach are left with the burden of being on their own. They are trading freedom and flexibility for the support they could get from other teams.
🎨 How to design your process
The worst thing you could do is simply copy one of the mentioned approaches, and go down the tempting yet often deceiving cargo-culting alley.
What works for Apple, Twitter or Netflix is highly context-dependent. It's part of a complex system and balance that includes company size, business success, and internal culture.
Trying to adopt a piece of their setup in our organization hoping it will work is akin to trying to fit an airplane jet engine into a small car. You might be lucky that it works for a few miles, but in most cases, you'll just blow yourself up.
If I were to establish a process for tech innovation in a new environment, I'd start by evaluating the following aspects.
Organization Size
Company's Runway
Culture and Maturity of the Engineering Team
These observations should help you orient your thinking in one direction or another.
📏 Organization Size
This is one of the cases where I think size matters. And it matters a lot!
That's why I'm always very cautious when applying learnings from famous names in the industry. While they tend to capture the public discourse that is pushing the boundaries of technology, FAANG, and other big tech companies are just the tip of the iceberg.
While a lot of people are employed in big tech firms, most of us don't.
More importantly, most of those looking to implement an effective process for innovation at their firm are unlikely to work at big tech.
When you're in charge of a small team, you should be extra careful in managing cognitive load and complexity. When running a team of 10 to 20 engineers, for example, adding a new language or framework could have more downsides than benefits.
You might increase effectiveness in one specific area (local maximum), while negatively impacting the efficiency of the entire team as it now has to deal with greater complexity.
Furthermore, new technologies you introduce at the early stages have a high chance of being left orphaned as the key sponsor or expert moves on to another job.
I've seen many cases of teams in the 10-20 size range dealing with a very fragmented stack: A combination of Angular and React on the frontend, Java, Python, and Go on the backend, and sometimes even multiple cloud providers.
All these are often the relics of innovation work that was started but never finished, due to the low bus factor in the team.
This is to say that if your team is small, I'd be very very very careful whenever introducing a new piece of technology.
Every new tool adds to the overall complexity your team has to deal with. More often than not, you'll end up carrying that load longer than you originally intended.
In such a situation I'd recommend establishing a process that is aimed at optimising for the following aspects:
Replacing rather than Adding. You want to keep your stack simple, therefore you should favor innovation that allows you to replace a component with a new one, be it a language, a vendor, or a framework. Be wary of attempts to introduce new technologies for narrow use cases.
Learning Curve. You want to onboard technologies that will be easy to pick up for the team so that you avoid depending on just a few experts. If only one person on the team seems excited about a new piece of tech, it's usually a sign that you should be extra careful.
End-to-end migration time. With such a small team you won't often have the luxury of an extensive amount of time available for technical migrations. The risk of not being able to finish the work increases more or less exponentially with the complexity of the migration task itself.
When applying these principles you will likely realize that in most cases where you have a small team, it's not wise to switch your primary programming language or database engine.
💸 Company's Runway
One more factor I think is essential is the Company's Runway. Simply put, if your company has less than 12 months of runway, you should focus all your efforts on finding product market fit and keeping operational costs as low as possible.
In other words, you should give the company a future before you focus on building the future of the company.
In a situation where your long-term horizon is quite short-term, keeping the stack simple can be an advantage. You are also likely to have a small team at this stage.
This means all the considerations from the previous point apply, with two additions:
Focus on very short-term impact. Any technological innovation you introduce should have a lead time or time-to-impact measured in days or weeks. If it takes more than a few weeks to deliver results, chances are you can't afford it now.
Cost reduction can be both under- or over-emphasized. Innovation aimed at reducing costs could be a double-edged sword. On the one side, the potential for reducing operating costs, especially cloud spending, is often underestimated. Especially in early-stage startups that don't have sophisticated tools for keeping an eye on their cost base. On the other hand, you might fall into the trap of being penny-wise and pound-foolish. This happens when you spend an excessive amount of human capacity trying to reduce costs by a tiny fraction. Always express the impact in terms of extension or Runway and Opportunity cost.
An exception could be an early-stage startup that is burning through its seed round. In this case, I'd consider it normal to see an almost frantic pace of technical innovation as you and the team are iterating through different prototypes. You'll see then a case of “100 flowers blooming”, though hardly in parallel. You'll most likely be blooming them and then throwing them away one after the other in rapid succession until you find the one that you can start building a whole business on top of.
🥊 Culture and Maturity of the Engineering Team
This element is the most fluid and intangible as it can't be captured by hard metrics and numbers. It also becomes more important to consider and handle properly the bigger the size of your organization.
A bigger organization often correlates with a higher risk of certain teams or individuals focusing on technology for the sake of technology. As their distance from the end user increases, so does the potential disconnect from the work someone is doing and how that makes the company more successful.
This is often the realm of massive platforms being built that struggle to get internal adoption or the realm of fragmentation and duplication of work.
I've seen my dose of teams building their container orchestration stack, CI/CD pipelines, or observability tools because they weren't happy with the one offered by internal platform teams.
These initiatives tend to fall into the category of trying to solve people's problems with technology.
When you have a widespread insular culture in your company, you should focus on the root causes rather than the symptoms.
Leadership role modeling is key. Once that is in place, you can use a combination of shared goals, enabling team, and accountability to help move the culture towards a better place.
Shared goals will help create alignment across teams that build and consume internal technology. A tight collaboration model as defined in team topologies can also help bridge the gap with existing tools.
Enabling teams can be instrumental in bridging the knowledge gap, ensuring engineers in your firm know how to get the best out of the available technology stack. At the same time, they can serve as a powerful feedback loop by collecting observations on the ground and making them available to the teams in charge of specific technology.
Accountability will help you ensure that your teams are consistently focusing on what's most impactful. That means, for example, that before investing a significant portion of the team's time to build an alternative solution to something already available, the team leaders will have to build a strong enough case for the expected impact, and they should be held accountable for delivering such results.
🏁 Conclusions
In today's article, we looked more in-depth at the complex problem of establishing an internal process for managing tech innovation.
As you might have noticed though, the article didn't provide a guide on how to set up a process, but rather a set of considerations that I'd recommend anyone to evaluate when thinking about how to set one up.
This might be a bit disappointing but don't worry, as that's exactly what we'll do in next week's article.
We'll look at a guide that you will not be able to simply git clone into your situation though. Instead, you'll need to adjust it to your specific context.
That's why I wanted first to expose you to some of the underlying thinking, as this will increase your chances of designing a successful process for your team.
As usual, I'm looking forward to your contributions in the comments section, especially examples, and approaches to this problem that you have seen both working and not working well.
☝️One More thing
After the success of the initiative I ran in Q1 to offer free coaching & mentoring sessions to people affected by layoffs, I’ve just announced my pay-it-forward initiative for Q2.
I will be offering free coaching and mentoring sessions for Women in Tech during Q2. You will find all the details in this post, but in summary:
Subscribe this newsletter
Fill in this application form
The deadline is April 5th, 2024!
If you know someone who could be interested, please forward them the link to this article.
Thanks in advance and see you all next week!
Great follow up, Sergio! I believe that context matters quite a lot and any approach has trade offs that need to carefully considered on a regular basis. I love the Netflix paved road approach but that only works if you have good runway and mature teams.
Complexity is bad, we should put gate keepers to avoid raising it when not absolutely necessary. We should always start small and keep it simple for as much as possible.