Dave Anderson
We were chatting earlier in the week about two things that I think are quite interesting: engineering excellence and well architected. They are two phrases we kick about a lot. And they’re heavily featured in the book, The Value Flywheel Effect coming out soon. So do you think engineering excellence and well architected are well understood in the wider industry?
Michael O’Reilly
That’s a good question.
Dave Anderson
It depends.
Michael O’Reilly
I think it depends. The term engineering excellence is something we would use quite a lot. And it emcompasses a lot of broader topics. Well architected is emerging. People are becoming more aware in the organisations we work with. Because we are obviously big fans! Engineering excellence, enabling constraints and engineering performance in general are thought about by companies. But on an individual engineering level, I am not sure if it’s well understood.
Mark McCann
It’s something that needs to mature and in the industry. The ‘Accelerate’ book and the work that Jez Humble and others have done with continuous delivery has helped. And Dave Hardy and others have helped to bring forward a culture of seeking engineering excellence. But it hasn’t embedded across all organisations yet. You see it in pockets.
Some say they have the four key metrics or we are focused on deployment frequency. But i’s not just that. There’ a lot more to it. I think it’s improved. Teams are starting to understand the value of having a fast feedback loop. And baking engineering disciplines and guardrails into their pipelines and getting faster feedback and quality issues. But again, I don’t think it’s done in an holistic manner. Or a joined up manner yet.
Let’s dig into what engineering excellence is
Dave Anderson
I think having two phrases or two terms for these things is important. And that there’s a good definition for them. Let’s dig into them, one by one.
So let me start with engineering excellence. It’s a funny one. I first heard that term from people in the States. I know that Amazon use it. But two things are interesting for me. Few companies will say they don’t want good engineering. And we are happy with bad engineering. But they don’t define what good looks like.
And then the second thing is that engineers are expected to write lots of code. For me, engineering excellence is not writing loads of code. It’s about getting into code reliability, feedback loops and how well engineering is functioning. And how your team is performing and meeting bigger goals. That’s what we find super interesting. When we are doing this, we tie to a business goal. And that make the difference. As opposed to thinking, my complex is really low, aren’t we really great! Lift it up a level.
It’s not feature, feature, feature
Mark McCann
Also they can start to articulate this when they’re having prioritisation conversations with customers or business partners. It’s not just feature, feature, feature! Or delivery, delivery, delivery. They’re able to have a well articulated conversation about value. And investing in better testing or pathway to production. Or reducing code liabilities, doing a well architected review or a threat model. They’re starting to have a language that can explain the value of these things to people that aren’t on the team. So the stakeholders can see the benefit.
In other words if we do these things, we can prevent problems down the line. We can help the flywheel turn faster in the short or medium term if we invest in engineering excellence. The work involved is described in ‘Accelerate‘ and in our own book ‘The Value Flywheel Effect‘. They can help you make a case for investing in these things. So hopefully it is easier now than it has been in the past to get buy in and backing to invest in engineering excellence. So it’s not just feature, feature, feature anymore.
Slow is smooth and smooth is fast
Michael O’Reilly
The ‘Accelerate’ book and DORA metrics are for technical leadership . Teams want to do stuff fast and get stuff out. They are keen to solve problems or address business or customer need. Technical leadership is there to make sure it’s done well. And down the line, how are we actually performing? We’re delivering fast. And we’re delivering with quality. But how many errors are coming back with each of these releases? What was the team’s approach to solution that? Are we are we moving fast enough?
I love to say ‘slow is smooth and smooth is fast’. So how do we set ourselves up for that? At the beginning, I like to go slowly to look at our process. How are we going to test that this is successful in production? And how do we limit the the errors? What do we know and what do we not know? Also, what do we need to do up front? Leaders have got to be prepared to go slow and put those things in place to enable teams to operate more effectively.
We talk about problem prevention being baked into the long term value section of the value flywheel. You either pay upfront. And do discovery and put rigour into pathways to production. Or else you are going to pay for it in the long term! You have to decide what’s right. Obviously, we prefer to do things in the right way.
With engineering excellence; engineering maturity and experience come into play. You have to control flow and observe how people are approaching things. So it’s a really important topic for Tech Leadership in the industry.
What is the fifth DORA metric?
Dave Anderson
With enabling constraints, you’re making decisions for the engineering teams. To define engineering excellence, we use Dan Pink’s motivation factors of autonomy, mastery, and purpose. We translate those motivation factors to technical excellence, autonomy, and customer focus. It is a nice way to phrase it to explain to people what it means. The DORA metrics are also excellent. Herie is a question for you. Can you name the fifth DORA metric?
The fifth one was added last year and it is reliability.
Michael O’Reilly
We talk about engineering excellence, which is your flow and feedback cycles. And how you think you are performing. That bleeds into well architected dimensions so we are starting to see an overlap.
Let’s dig into well architected
Dave Anderson
We talked about Engineering Excellence. So that what about well architected? I think well architected is better defined than engineering excellence. But I find that a lot of people haven’t heard the term well architected and they think you’ve just made it up! It’s actually quite specific.
Michael O’Reilly
I’m starting to see lots more talks, And I’m starting to see it more in the community. People are starting to know what what what we mean, when we’re talking about well architected. Certainly within the organisations we’re in.
This is where a need for discipline comes in. You have to assess the best way to go about something:
- Have you considered the various dimensions?
- What are we good at and what can we improve upon?
- And what would we like to change?
Well architected is a vehicle for continuous improvement. It’s evaluating where we’re at and what we could do better.
Self serve and learn
Mark McCann
We haven’t created this ourselves. There’s a body of evidence, supporting material, videos, tutorials, workshops and labs to support well architected. People can google well architected, whether they’re talking about AWS, Azure or Google. And they’re going to get a fairly consistent response. They self serve and self learn. They can don’t need to go to anyone to learn about well architected. There are courses and certificates also.
It’s starting to permeate through. Maybe we’re in an echo chamber or little bubble, but it’s definitely resonating a lot more than it ever has. Teams are loving the fact that we’re starting to focus on these things, because they’ve always wanted to do them. And maybe they didn’t have the capacity, the space or the language to fight to deliver well architected.
Dave Anderson
Well architected is actually defined by a separate body.
I remember years ago, when you talked to architects, they said that good architecture is a non functional requirement. It’s performability, scalability, extendibility and every stupid word ending in ‘ility’. Everyone had a different list. And if you talked to somebody a month later, they had a different list again. So you never knew what someone was going to say. It became folklore and black magic. Now the fact is that it is well documented, understood and you can rhyme them off as SCORPS makes it easier.
Be empowered and know what good architecture looks like
Michael O’Reilly
There’s something that clicks with well architected. When you run well architected with squads for the first time, it’s interesting what sorts of reactions you get. There’s always stuff that comes up: maybe if we had time, maybe if we had been thinking about that or we should definitely do this. But ultimately a team that wants to move fast does not have time to stop or pencil in a two week architectural review. Or bounce something off someone three weeks down the line to make a decision.
If you want to move fast, you’ve got to be empowered and know what good architecture looks like. When you have guidance or patterns that are implemented in a well architected fashion, that sets you up for rapid delivery, moving with speed, and in an engineering excellence way. It’s fast flow aimed to facilitate you moving fast, getting things out quicker and being more competitive.
The trail of evidence
Mark McCann
With both of these approaches you are able to give all your stakeholders competence. Not only are you going faster, you’re going faster in a good way with the data, metrics and evidence to back it up. So teams are rapidly delivering product value multiple times a day. And they have a trail of evidence that they’re going the right way because of these practices. They have dashboards, logs, alarms, alerts and the run books and the playbooks. And the key business indicators (KPIs) at their fingertips.
So any stakeholder can challenge them on what they are doing, how they are operating or what they are deploying. They have information radiators that teams can look at so they know what’s happening
Dave Anderson
That’s really important. Back in the day we aimed to make good engineers who said the engineering was good because they were good! I have talked to a lot of C Suites and Board members. And when you say well architected or engineering excellence they want that. And you can rattle off the salient points and direct them to the dashboard That’s hugely powerful to convince people that this stuff’s important.
You need a commoditized set of standards
Michael O’Reilly
There’s also another dimension to this. I’ve experienced squads that work with well architected, and they are fully bought in. They work through all the processes. And I have worked with squads that aren’t bought in. When you compare the two, they’re very different.
At an organisational level, you need a set of commoditized standards. So that teams can move from one area to another, and move to and from an engagement with a certain level of proficiency, quality, engineering excellence or well architected. These sorts of things facilitate that.
If you arrive in an area that isn’t well architected, you’ve got to spend three months trying to understand what went on before or trying to understand the decision making. Or you’ve got 12 months of technical debt that should have been taken on board before you moved there. That very difficult from a personal perspective.
These patterns enable fluidity to move one team to a different workload. And they’ll be effective quickly, as opposed to them going to an area where they have to absorb a lot of cognitive burden.
Problem Prevention Culture
Dave Anderson
There’s a whole bunch of advantages. One is a problem prevention culture, which is an interesting concept. Do companies understand that they need a problem prevention culture in your engineering and product areas?
Mark McCann
I don’t think it exists in a lot of organisations, It hasn’t really cut through. They’re still on the feature, factory, server mindset. Deliver, deliver deliver. Feature, feature feature. And then they have a tipping point where everything grinds to a halt for months or maybe years. And it may never recover.
But if you have a continuous improvement mindset and you are investing in fraud prevention, well architected, engineering excellence and continuous learning. And you are enabling and applying this to your teams, you get ahead of problems before they become problems. You’re investing in performance, security, resilience, high availability, every day and every week. And those things compound on each other. It’s like compound interest! You’re getting better returns as you go.
We don’t want super heros
I think that problem prevention culture is critical. But one of the big challenges is that silent excellence can go unrecognised or unrewarded. So you need to be careful that you have enough people who understand what good looks like, and reward it and recognise it appropriately. You don’t want to fall into a superhero ‘save the day’ mindset! If you invested in problem prevention culture, there’s no need for superheroes to come in and save the day!
Dave Anderson
Another superhero is Mickey who stayed up all night. So he’s a legend. Well you should have put more prep into the design review on Monday, and you would not have had to do that!
Michael O’Reilly
From a problem prevention perspective, do you want to spend all your time babysitting a non well architected workload and dealing with all of that? You want your teams moving, adding value and moving on to the next thing. That’s not possible if your build lacks quality, is always down or if you’re having to constantly upgrade. This is why service is such a big concept.
Build things that run and increase in value
With well architected and engineering excellence, we can build things that run and increase in value. And there’s less work over the long term, while your teams move on to what the business needs. I don’t think a lot of orgs think in that way. By the time they do get thinking that way, they’re already experiencing a lot of problems.
Mark McCann
We talk about creating space for innovation. If you have a problem prevention culture, you’re removing a lot of busy work. So your teams are truly able to focus on things that are innovative and help your business succeed. Because they’re not fixing disasters, managing outages or chasing their tail trying to keep things alive.
Dave Anderson
That’s quite hard to explain to senior people. If you want more innovation, you need to do more boring architecture. Because one feeds the other.
Mark McCann
Do you want all your best people firefighting? Or do you want them helping you grow in the marketplace?
Dave Anderson
Or have your best people do architecture so everybody can innovate?
That’s the craic! Follow us on Twitter @ServerlessEdge and at TheServerlessEdge.com for our blog. And subscribe to Serverless Craic on YouTube. Thanks very much.
Transcribed by https://otter.ai