How do you measure the success and productivity of an engineering team? Which metrics should you use? What is engineering excellence, and how can you achieve it?
You must ask yourself these questions. The key performance indicator we aim for is delivered business value and customer satisfaction. However, these might be subjective, so let us consider some metrics and tools to help you develop a strong engineering culture and achieve high software quality.
Engineering excellence and well-architected are two concepts we talk about a lot. And they feature in our book, The Value Flywheel Effect. The term engineering excellence encompasses a lot of broader topics. Well-architected is emerging, and people are becoming more aware of it in our organizations because we are big fans! Engineering excellence, enabling constraints, and engineering performance, in general, are thought about by companies. We would like to know if individual engineers have a good understanding.
It may be something that needs to mature in the industry. The Accelerate book and the work that Jez Humble and others have done with continuous delivery have helped. And Dave Hardy and others have helped to bring forward a culture of seeking engineering excellence. But it has yet to be embedded across all organizations. You see it in pockets.Some teams are confident that they have implemented the four key metrics or focus on deployment frequency. But there’s a lot more to it. Teams are starting to understand the value of having a fast feedback loop by baking engineering disciplines and guardrails into their pipelines. But teams still need to start doing it holistically or joined manner.
Let’s dig into what engineering excellence is
Having two phrases or two terms for these things is essential. And that there’s a good definition for them. Let’s dig into them one by one.
Engineering excellence is a funny one. I first heard that term in the United States. I know that Amazon uses it. But two things are interesting for me. Few companies will say they don’t want good engineering. And we are happy with bad engineering. But they don’t define what good looks like.
And then the second thing is that most organizations expect engineers to write lots of code. For us, engineering excellence is not writing loads of code. It’s about getting into code reliability, feedback loops, and how well engineering functions. And how your team is performing and meeting bigger goals. That’s what we find super interesting. When we are doing this, we tie it to a business goal. And that makes the difference.
It’s not feature, feature, feature.
You can start articulating this in your prioritization conversations with customers or business partners. Those conversations should not be just feature, feature, feature! Or delivery, delivery, delivery. You need to talk about value and investing in better testing or pathway to production or reducing code liabilities by doing a well-architected review or a threat model. Then you start to have a language that can explain these things’ value to people who aren’t on the team. So the stakeholders can see the benefit.
In other words, if we do these things, we can prevent problems down the line. Investing in engineering excellence can help the flywheel turn faster in the short and medium term. We describe this in our book ‘The Value Flywheel Effect.’ And Nicole Forsegen also covers this in ‘Accelerate.’ It can help you make a case for investing in these things. We hope it is easier now than it has been to get buy-in and backing to invest in engineering excellence. So it’s not just feature, feature, feature anymore.
Slow is smooth, and smooth is fast.
The ‘Accelerate’ book and DORA metrics appeal to technical leadership. Teams want to do stuff fast and get things out. They are keen to solve problems or address business or customer needs. Technological leadership needs to make sure teams are doing it well. And down the line, how are we performing? We’re delivering fast. And we’re delivering with quality. But how many errors are coming back with each of these releases? What was the team’s approach to the solution? Are we moving fast enough?
We love saying, ‘Slow is smooth, and smooth is fast.’ So how do we set ourselves up for that? Initially, we like to look at our process by asking, ‘How are we going to test that this is successful in production?’ And how do we limit errors? What do we know, and what do we not know? Also, what do we need to do upfront? Leaders must go slow and put those things in place to enable teams to operate more effectively. We discuss baking problem prevention in the value flywheel’s long-term value section. You either pay upfront, do discovery, and put rigor into pathways to production.
Or else you are going to pay for it in the long term! You have to decide what’s right. We prefer to do things in the right way.With engineering excellence, engineering maturity, and experience come into play. You have to control flow and observe how people are approaching things. So it’s a critical topic for Tech Leadership in the industry.
What is the fifth DORA metric?
With enabling constraints, you’re making decisions for the engineering teams. We use Dan Pink’s motivation factors of autonomy, mastery, and purpose to define engineering excellence. We translate those motivation factors to technical excellence, autonomy, and customer focus. It is an excellent way to phrase it so people understand what it means. The DORA metrics are also excellent. DORA added a fifth one last year, and it is reliability.We talk about engineering excellence, which is your flow and feedback cycles. And how you think you are performing. That bleeds into well-architected dimensions, so we are starting to see an overlap.
Let’s dig into well-architected.
We have looked at Engineering Excellence. So what about well-architected? We have defined well-architected better than engineering excellence. But I find that a lot of people have yet to hear the term well architected and they think you’ve just made it up! We see lots more talks in the community. People are beginning to understand what we mean. Certainly within the organizations we’re in.There is a need for discipline. You have to assess the best way to go about something:
- Have you considered the various dimensions?
- What are we good at, and what can we improve upon?
- And what would we like to change?
Well-architected is a vehicle for continuous improvement. It’s evaluating where we’re at and what we could do better.
Self-serve and learn
There’s a body of evidence, supporting material, videos, tutorials, workshops, and labs to help you with well-architected. You can google well-architected for AWS, Azure, or Google, and you will get a consistent response. You can self-serve and learn, and there are courses and certificates also.It’s starting to permeate through. Maybe we’re in an echo chamber or little bubble, but it’s resonating much more than ever. Teams love that we’re starting to focus on these things because they’ve always wanted to do them. And they didn’t have the capacity, the space, or the language to fight to deliver well-architected.
A separate body defines well-architected. Years ago, when you talked to architects, they said good architecture is a non-functional requirement. It’s performability, scalability, extendibility, and every uninformed word ending in ‘ility.’ Everyone had a different list. And if you talked to somebody a month later, they had another list again. So you never knew what someone was going to say. It became folklore and black magic. It is well documented and understood; you can rhyme them off as SCORPS, which makes it easy.
Be empowered and know what good architecture looks like
There’s something that clicks with well-architected. When you run well architected with squads for the first time, it’s interesting what sorts of reactions you get. Something always comes up: maybe if we had time, perhaps if we had been thinking about that, or we should do this. But ultimately, a team that wants to move fast has little time to stop or pencil in a two-week architectural review. Or bounce something off someone three weeks down the line to make a decision.
If you want to move fast, you’ve got to be empowered and know what good architecture looks like. When you have guidance or patterns implemented well architected, that sets you up for rapid delivery, moving with speed and in an engineering excellence way. You move fast, get things out quicker, and are more competitive.
The trail of evidence
With both of these approaches, you can give all your stakeholders competence. Not only are you going faster, but you’re also going faster in a good way with the data, metrics, and evidence to back it up. So teams are rapidly delivering product value multiple times a day. And they have proof that they’re going the right way because of these practices. They have dashboards, logs, alarms, alerts, run books, and playbooks. And the key business indicators (KPIs) at their fingertips. So any stakeholder can challenge them on what they are doing, how they are operating, or what they are deploying. They have information that teams can review to know what’s happening.
Back in the day, we aimed to make sound engineers who said the engineering was good because they were good! I have talked to a lot of C Suites and Board members. And when you bring up well-architected or engineering excellence, they want that. And you can rattle off the salient points and direct them to the dashboard. That’s hugely powerful to convince people that this stuff’s important.
You need a commoditized set of standards.
There’s also another dimension to this. I’ve experienced squads that work with well-architected and commit to work through all the processes. And I have worked with teams that don’t engage. When you compare the two, they’re very different.
At an organizational level, you need a set of commoditized standards. So that teams can move from one area to another and transfer to and from an engagement with a certain level of proficiency, quality, engineering excellence, or well-architected. These sorts of things facilitate that.If you arrive in an area that needs to be well-architected, you’ve got to spend three months trying to understand what went on before or the decision-making. Or you’ve got 12 months of technical debt before you moved there. That is very difficult from a personal perspective.These patterns enable fluidity to move one team to a different workload. And they’ll be effective quickly, instead of going to an area where they have to absorb a lot of cognitive burden.
Problem Prevention Culture
There’s a whole bunch of advantages. One is a problem-prevention culture, which is an exciting concept. Do companies understand that they need a problem-prevention culture in their engineering and product areas? It doesn’t exist in many organizations; It hasn’t cut through. They’re still on the feature, factory, and server mindset. Deliver, deliver deliver. Feature, feature feature.
Then they have a tipping point where everything grinds to a halt for months or maybe years. And it may never recover. But if you have a continuous improvement mindset, invest in fraud prevention, well-architected engineering excellence, and continuous learning. And you are enabling and applying this to your teams; you get ahead of problems before they become problems. You’re investing in performance, security, resilience, and high availability daily and weekly. And those things compound on each other. It’s like compound interest! You’re getting better returns as you go.
We don’t want superheroes.
Problem prevention culture is critical. But one of the significant challenges is that silent excellence can be recognized or unrewarded. So you must be careful that you have enough people who understand what good looks like, reward it, and realize it appropriately. You don’t want to fall into a superhero ‘save the day’ mindset! If you invested in problem prevention culture, superheroes do not need to come in and save the day!
Another type of superhero is Mickey, who stays up all night. So he’s a legend. You should have put more prep into the design review on Monday, and you would not have had to do that! From a problem prevention perspective, do you want to spend all your time babysitting a non-well-architected workload and dealing with all that? You want your teams moving, adding value, and moving on to the next thing. That’s only possible if your build has quality. And your system is not always down, requiring you to upgrade constantly.
Build things that run and increase in value.
With well-architected and engineering excellence, you can build things that run and increase in value. And there’s less work over the long term while your teams move on to what the business needs. I don’t think a lot of orgs think in that way. By the time they do get thinking that way, they’re already experiencing a lot of problems.
We talk about creating space for innovation. If you have a problem-prevention culture, you’re removing a lot of busy work. Your teams can focus on innovative things and help your business succeed because they’re not fixing disasters, managing outages, or chasing their tail trying to keep things alive.That’s quite hard to explain to senior people. If you want more innovation, you need to do more boring architecture. Because one feeds the other. Do you want all your best people firefighting? Or do you want them to help you grow in the marketplace? Or have your best people do architecture so everybody can innovate?
That’s the craic! Follow us on Twitter @ServerlessEdge and at TheServerlessEdge.com for our blog. And subscribe to Serverless Craic on YouTube.