At AWS re:Invent, one of the final sessions of the week addresses a core challenge in software development and platform engineering: balancing the freedom to innovate with the consistency needed for scalability. Presented by Matt Meckes, Senior Serverless Specialist AWS, Roland Barcia, Director Specialist SA and Tech AWS and our own Dave Anderson, the session explores how organisations can manage this balance effectively while maintaining agility and operational efficiency.
The Challenge: Innovation vs. Consistency
Developers thrive on freedom—exploring new ideas, creating innovative solutions, and delivering impactful products. However, as organisations scale, consistency across teams and systems becomes critical to maintain security, efficiency, and manageability. This session examines the delicate art of navigating these competing priorities, drawing on AWS’s tools, frameworks, and organisational strategies.
Key Insights
1. The Evolution of Software Development
- Traditional Models: Developers previously focused solely on coding, handing their work over to infrastructure teams for deployment and management.
- DevOps and Cloud: Modern cloud development shifts responsibility to developers, requiring them to manage deployment, security, cost optimisation, and sustainability.
- Cognitive Load: Developers now face greater complexity, managing not only the code but also broader operational concerns.
2. Effective Team Structures
- Two-Pizza Teams: Small, autonomous teams with clear goals remain a cornerstone of innovation.
- Beyond Two Pizzas: Supporting structures, such as enabling teams, platform teams, and facilitating teams, are vital for scaling. The Team Topologies framework categorises teams into:
- Stream-Aligned Teams: Deliver customer-focused value.
- Complicated Subsystem Teams: Handle specialised or complex systems.
- Platform Teams: Provide core tools and services.
- Enabling Teams: Support and coach other teams to adopt best practices.
3. Mechanisms for Success
- Automation: Frameworks like the AWS Well-Architected Framework help embed best practices into development processes.
- Developer Portals: Tools such as Backstage offer self-service access to templates, documentation, and automated workflows, reducing friction.
- Continuous Improvement: Regular reviews and feedback loops support ongoing optimisation and team development.
4. Platform Engineering in Practice
- Platform as Product: Internal platforms must be treated like external products, with dedicated ownership and a focus on user needs (developers and data scientists).
- Standardisation with Flexibility: Provide default patterns while allowing room for innovation through escape hatches.
- Empowering Developers: Enable teams to work autonomously while adhering to organisational standards through coaching and well-designed tools.
Case Study: Globalization Partners
Dave Anderson highlights practices from Globalization Partners (GP), a leader in global employment solutions. The company uses AWS to scale its operations across over 180 countries, focusing on compliance and operational efficiency.
GP’s Approach:
- Service-First Strategy: Teams use a core set of AWS services (e.g., Lambda, DynamoDB, Step Functions) as a starting point for new workloads.
- Domain Ownership: Single teams own specific domains (e.g., customer data, billing), ensuring accountability and autonomy.
- Event-Driven Architecture: Domains communicate via events, reducing dependencies and enhancing scalability.
- Continuous Reviews: Teams track performance, cost, and reliability using lightweight dashboards, encouraging regular improvement.
Lessons for Scaling Software Development
- Clarify of Purpose: Ensure teams understand the business outcomes driving their work.
- Foster Psychological Safety: Create an environment where experimentation and learning thrive.
- Empower with Tools: Leverage automation and self-service platforms to reduce barriers.
- Encourage Collaboration: Strong interactions between development and infrastructure teams ensure balance.
- Measure and Improve: Use frameworks like Well-Architected as tools for continuous learning and refinement.
Final Thoughts
Scaling software development requires thoughtful strategies, robust platforms, and a commitment to continuous improvement. By combining innovative structures like two-pizza teams with enabling frameworks and tools, organisations can foster an environment where innovation and consistency coexist. As the session concludes, the speakers emphasise that successful platforms require vision, collaboration, and an unwavering focus on developer empowerment.
For more information, visit AWS Serverless Land and explore tools such as AWS PowerTools and Backstage to support your journey.
Serverless Craic from The Serverless Edge
Check out our book, The Value Flywheel Effect
Follow us on X @ServerlessEdge
Follow us on LinkedIn