The team talk through the Top 8 Tenets needed to transform Cloud Software Engineers into high performance serverless first teams.
Our 8 tenets for high performance serverless first teams
We put together the original list in 2019 by writing 8 tenets. And we joke that it took 10 years to think up the tenets. But they took only 10 minutes to write. I thought I might be useful to go through that list and see if it still resonates three years later.
They apply to a ‘high performance serverless first team’. But they could also apply to a high performance modern cloud team.
1. Chase a business outcome or a KPI
So the first one is: ‘Chase a business outcome or a KPI’. Behind that is the fact that a team should know what business KPI they’re working towards.
Mark McCann
You should be able to tap a cloud software engineer on the shoulder and have them tell you what they’re working on. And what business impact the work they’re doing is going to have. They should be able to articulate that.
Mike O’Reilly
A guardrail to allow you move with speed and velocity relies on making good decisions. And really understanding your priorities and how you prioritise. The only way to do that is by tapping into the success of the product.
Use North Stars to track your business success
We talk about North Stars to track your way from profitability or business success to what you’re doing. And how you’re having an impact. That’s how to make good decisions and move fast. So this tenet stands up today.
Mark McCann
We don’t share these principles or tenets with teams, without giving them guidance on how to achieve or align to them. Northstar is a great one. We did a lot of Northstar workshops. To help teams understand if they didn’t have a good grasp of their KPIs or Northstar for their business.
Dave Anderson
I’m not sure if we explained this. But there’s a really simple thought behind that. You ask a team what their KPI is. If the team says ‘I don’t know’, then you run a Northstar workshop. After the Northstar workshop if nobody can think of a KPI then the next step is to ask if the team should be doing this work. And that happened a few times. It’s not that they are a bad team. They are being asked to do the wrong stuff. Yes, that still stands up.
2. Be secure by design
Number two on the list of principles is ‘Be secure by design’. That worked to secure development for us for a long time. And then AWS came out with ‘Secured by Design’. So we borrowed it. I really liked this tenet. Don’t do security afterwards. Bake it in from the start. It’s everyone’s job. Period/full stop.
Mike O’Reilly
It’s such a difficult thing to retrofit. Use threat models and get it done early. Try to solve for what you can and what you know.
Mark McCann
Bake it into all your engineering practices. And bake it into your pipelines. Shift it all left and help to enable teams to be more secure.
Dave Anderson
Don’t say that it’s too hard. We’re not doing it. Because for the next project you won’t have done it. So start today and bake it in!
Mark McCann
Are you aligned with the business? Is business success number one? And being secure is number two. Because security has a risk profile, if you don’t do it right. And it can be an existential risk for the businesses if you don’t have a secure solution. Number one and two are in the right order there.
3. Keep a high throughput of work
Dave Anderson
That’s very deliberate. The third principle or tenet is ‘Keep a high throughput of work’. That is borrowed from the DORA metrics in the ‘Accelerate‘ book by Nicole Forsgren. I always liked the 4 ‘accelerate’ metrics. Nobody usually understands that two are for throughput. And the other two are for stability. This principle looks at high throughput, which is deployment frequency and lead time.
Mark McCann
Around the time we were trying to articulate this, the ‘Accelerate’ book was published. It gave us the language and external validation for what we were saying to teams. We could point to the ‘Accelerate’ book and the DORA metrics as actionable to quantify velocity. As well as development time, deployment frequency and lead time. These principles are well chosen. And everything that has come since then really supports and reinforces the need for these principles.
Mike O’Reilly
For serverless teams, it is key to make changes fast and frequent. And always be learning and driving observability. As you say Mark it is really core to moving with speed.
Mark McCann
It drives the right behaviour for removing impediments. The fast flow and really questioning your dependencies. Such as why can’t we be in the elite category? Why do we have this dependency on this group? Or another group? And why can’t we deploy on demand? Why can’t we deploy multiple times a day? It really helps teams to think in the right way.
Speed is stability
Dave Anderson
I have lots of funny stories in relation to this. I remember talking to a monthly release team who were angry and didn’t want to do extra work. They felt that their throughput was one per month or 12 a year. And they did not want to measure it. But they already had measured it! So it wasn’t going to be much work for them. And number two, what would happen if they got a zero day security vulnerability? They would have to break everything because they didn’t know how to release it! So they were able to see what I meant. And added to that was the fact that they didn’t know if business wanted anything else for another month. That was a very interesting and amusing conversation.
Mark McCann
As Charity Majors says, “speed is stability”. The more frequently you do something, the more you deploy to production. You’re actually improving your stability. You smooth out the pathways and the error conditions. And you bake it into your pipelines. Which means that you automate a lot of the stuff that could go wrong.
4. Reliably run high stability systems
Dave Anderson
And that leads us on to why these things get repaired. Number four is ‘Reliably run, high stability systems’. That’s the other two DORA metrics of throughput and stability.
Mark McCann
A lot of discussions with test teams, QA and software engineers drive the need for investment in world class quality and testing capabilities/practices. If you’re not stable, where’s the gap? What scenarios and behaviours have you not covered? Are there chaos engineering items you have missed? What gaps do you have in your test suites? You need to make sure that stability is there. So again, that drives the right behaviour.
Mike O’Reilly
Well, it drives the right evolution. When you look at three and four, you can’t achieve any of them if you’ve got things in the middle. Or handoffs, or if you are dumping things over walls. It’s about promoting ownership. To get elite scores, you need to know what you’re doing and embrace that approach. So, 100 per cent for those principles. They help to modernise, shape and move teams towards that way of working.
5. Rent or reuse with build as a final option
Dave Anderson
We set a high bar with these principles. Number five is: ‘Rent or reuse with build as a final option’. How do you do that? Serverless!
Mike O’Reilly
Even with Serverless and SaaS, with our background you’re used to going straight to the workspace. And with the FORESEE diagram, we find out what we are doing and it is coding. It’s a mindset thing. And it’s a very healthy principle to embrace.
Mark McCann
It’s back to knowing your business purpose. And then knowing your business KPIs. If you can achieve business outcomes without doing code you are at your most optimal. If you can leverage a SaaS offering that does what you need, that’s probably the next thing. And finally you need to build following a serverless first mindset and approach. Using all the serviceful offerings and managed services.
Mike O’Reilly
What do you think about this? I’m not a big fan of hero developers or a hero mentality. Over the years, with this principle, we’ve learned to use ‘off the shelf’. Less code!
Dave Anderson
A democratised engineer rather than a superhero who builds something that no one understands.
6. Continuously optimise the total cost
Number six is: ‘Continuously optimise the total cost’. I think this is the best question to ask any team. Because good teams will tell you how much their cloud costs are. But loads of teams have no idea. This is a great measure of a good team. They have a cost in mind.
Mike O’Reilly
I would add that they also tell you how much they cost. And how much they cost to do what you’re asking them to do. They’ll give you good advice and guidance as well. And then get straight on to ROI and good projected ROI as well. So it’s a solid principle.
How can we mature teams?
Mark McCann
A good team will tell you the run cost. And a great team will tell you the total cost. But really good teams will get into a worst case development conversation about how much features cost. And how much revenue they’re bringing in.
In other words, how impactful they are to the business. I always add a fantastic question: ‘how can we mature the teams?’ How can we evolve the team so that they can answer new questions readily? For example, total cost is going to include carbon footprint and sustainability costs. When your team is optimising travel cost, they are not only optimising for financial culture, they need to optimise for carbon footprint too. They have to drive conversation on finding the most ecologically friendly region for their workload.
7. Build event driven via strong APIs
Dave Anderson
Number seven is: ‘Build event driven via strong API’s’. This sounds very easy. But from talking to Sam Dengler, nobody is doing this properly. We’ve been talking about this for 20 years. Proper integration is still a mystery to most people.
Mike O’Reilly
It’s moving fast. It is about making sure you’ve got the right things in the right places. But also at the right size. And having things that are composable. It’s about breaking things up into their smallest constituent parts. And change things as frequently as possible.
I find that this one takes a lot of evolution and yields through different levels of complexity. And it takes time. You should always be thinking about it. Teams who are new to serveless and that way of working will reinvent what they know.
The principles balance upon each other
Mark McCann
The principles on improving stability, move you into the elite categories and drive you towards loosely coupled event driven architecture. That gives you more autonomy and freedom. And gives you the ability to deploy when you’re ready. Because you are event driven and loosely coupled with strong API’s that give you autonomy. Architecturally that autonomy is baked in.
With the right team alignment, you can go fast and be in those elite groups. A lot of these principles balance on each other. If you’re trying to influence with one principle you have to have some of the elements of the other principles in place.
Dave Anderson
People like to think in layers. But when they try to do ‘events driven’ they try to go through layers. But that’s not event driven.
Mark McCann
A lot of the facilitator practices have come to the fore in the last couple of years. There’s lots of good stuff from the DDD career. And there’s lots of great ‘event storming’ from Alberto. As well as event bridge storming. So there’s good hands on facilitated techniques that demystify this. And make it more approachable for squads to benefit from an event driven architecture.
8. Build solutions that fit in their heads
Dave Anderson
Number eight is one of my favourites: ‘Build solutions that fit in their heads’. This principle is borrowed from Dan North. In other words, don’t build crazy systems that are too complicated. This has a nice nod towards Team Topologies and setting proper boundaries. We’ve seen teams become victims to crazy architectures. Where there’s too much to fit in your head and the cognitive load breaks people.
Mike O’Reilly
I think this one will evolve over time. When we are getting teams going my manta is ‘just enough design’. Some teams want to design everything up front and go into huge amounts of detail. But it is better to keep your world small. Focus on what you’re doing today. When you’re moving with rapid development and continuous architecture, you should always be refactoring and changing.
Limit cognitive burden
You won’t design the end state upfront. You have got to be prepared to change and move direction. In many serverless projects we’ve nuked what we’ve done after two or three weeks and started again. It’s just the way of work. But the point of the principle goes back to domain driven design and limiting cognitive burden. And making sure your groups and classifications are well defined.
Mark McCann
The Team Topologies guys nailed this one by optimising for cognitive burden. And that’s where all the other principles really come in. We can design systems that are small. And are loosely coupled, event driven and deployed frequently. That helps reduce cognitive burden. It’s not easy to get there. It’s hard work, and you have to evolve. And you have to edit and incrementally go after it. But you can start to really optimise for solutions that do fit in the heads of your teams.
Other tenets or principles to consider for cloud software engineers
Dave Anderson
We originally wrote that as a serverless first thing. But we haven’t mentioned lambda or serverless on the list! It wasn’t intentional, but it’s certainly how we did and do think. Do you have anything to add to that list?
Mark McCann
We talk a lot about well architected. So whenever I talk about these principles, I’m always referring to modern cloud or serverless first. Because well architected leverages these principles. If you do adopt these principles, you’re going to be heading towards well architected.
The other thing is operating with situational awareness. There’s got to be a principle on Wardley maps or other techniques to really understand the direction. And business KPIs lend to that understanding. But there might be something more on operating with good situational awareness. If teams are following these principles. It would be good if they also have situational awareness.
What about Developer Experience?
Dave Anderson
What about developer precision with developer experience? Is that within the team’s remit?
Mike O’Reilly
I look at these principles at a team level. Situational awareness feeds into these principles. And well architected underpins those principles also. Collaboration and working as a broader group is important. Looking at the 8th principle, if you and all the other teams work to the well architected standards, you will have portability and mobility of teams.
Dave Anderson
Well architected is the ‘how’. These principles are the direction. Well architected looks security, costs, operational excellence, reliability, performance and sustainability. So they are threaded through the principles. And the ‘how’ is well architected.
Mark McCann
In relation to your question about developer experience, I think cloud teams with a high throughput of work solve the challenge of developer experience. You cannot have a high throughput of work and deployment frequency if your developer experience is terrible.
Dave Anderson
I don’t think we were brave enough to write that at the time. Don’t accept poor developer experience. I didn’t want to be responsible for a bunch of developers leaving the company! We had to keep our jobs as well. We didn’t want to entice people to leave.
Create an environment for success
Mark McCann
Creating an environment for success is a critical enabler.
Mike O’Reilly
What I am trying to get to is contribution. In order to have good developer experience, they need to be able to contribute.
Dave Anderson
Give back through an inner source programme. One we didn’t put in is the idea of learning. You need to be learning as a team? So maybe there’s a curiosity principle there. So that the team are always questioning stuff. But again, sometimes that goes wrong. Because when you ask a team to do well architected, they respond by asking why?
Mark McCann
Continuous learning applies to the ‘optimise total cost’ principle. Have you looked at other options and involved your stock? Are you continuously learning about new features and capabilities that are available?
Dave Anderson
Be curious with the growth mindset! All right. I’ll leave it at that. Give us a like out on TheServerlessEdge.com. Or @ServerlessEdge on Twitter or on YouTube. Thanks very much. Cheers. Bye bye, everybody.
Transcribed by https://otter.ai