Platform-as-Infrastructure

1. Overview

The Platform-as-Infrastructure (PaI) pattern represents a paradigm shift in how organizations provision, manage, and consume infrastructure resources. At its core, PaI is the practice of treating infrastructure as a product, delivered through a self-service platform that abstracts away the underlying complexity. This approach combines the flexibility and control of Infrastructure-as-a-Service (IaaS) with the ease of use and developer-friendliness of Platform-as-a-Service (PaaS). By providing a curated set of tools, services, and automated workflows, PaI empowers development teams to build, deploy, and operate their applications with greater autonomy and speed, while ensuring that infrastructure and operations (I&O) teams can maintain governance, security, and cost-efficiency. This paradigm is not merely a technological solution but a socio-technical one, demanding changes in organizational structure, culture, and mindset to fully realize its benefits. It is about creating a symbiotic relationship between the builders of the platform and the consumers of the platform, fostering a culture of collaboration and shared ownership.

The significance of the PaI pattern lies in its ability to address the growing friction between development and operations teams in the era of cloud-native technologies and agile methodologies. As organizations strive to accelerate their software delivery cycles, traditional infrastructure provisioning models, often characterized by manual processes and lengthy lead times, have become a major bottleneck. PaI tackles this challenge by creating a “golden path” for developers, offering a standardized and streamlined way to access the resources they need, without requiring deep expertise in infrastructure management. This not only improves developer experience and productivity but also enables organizations to achieve greater consistency, reliability, and scalability in their IT operations. The result is a virtuous cycle of innovation, where developers are empowered to experiment and iterate more quickly, leading to better products and services for customers.

The historical origins of PaI can be traced back to the evolution of cloud computing and the rise of the DevOps movement. The emergence of IaaS platforms like Amazon Web Services (AWS) and Google Cloud Platform (GCP) provided the foundational building blocks for programmable infrastructure. Concurrently, the DevOps philosophy emphasized collaboration, automation, and shared responsibility between development and operations teams. PaI builds upon these foundations by introducing the concept of an “internal developer platform” (IDP) – a dedicated product team responsible for building and maintaining the platform that serves the needs of internal developers. This product-centric approach, inspired by the success of large technology companies like Google, Netflix, and Spotify, has now become a widely recognized best practice for organizations of all sizes seeking to optimize their software development and delivery capabilities. The evolution from artisanal, handcrafted infrastructure to a more industrialized and productized approach is a natural response to the increasing complexity and scale of modern software systems.

2. Core Principles

1. Infrastructure as a Product: The fundamental principle of PaI is to treat infrastructure as a product that is designed, developed, and maintained to meet the needs of its users (i.e., developers). This involves applying product management disciplines to the infrastructure domain, such as user research, roadmap planning, and iterative development based on feedback. By adopting a product mindset, platform teams can create a more user-centric and value-driven offering. This means having a product manager for the platform, who is responsible for understanding the needs of the developer community, prioritizing features, and communicating the platform’s vision and roadmap. It also means having a dedicated engineering team that is responsible for building, testing, and operating the platform with the same rigor and discipline as any other software product.

2. Developer Self-Service: PaI aims to empower developers with a high degree of autonomy and control over their application environments. Through a self-service portal or API, developers can provision and configure infrastructure resources on-demand, without having to file tickets or wait for manual intervention from the operations team. This not only accelerates the development process but also fosters a sense of ownership and responsibility among developers. The goal is to create a “vending machine” experience for infrastructure, where developers can easily select the services they need and have them provisioned automatically.

3. Abstraction and Standardization: The platform abstracts away the underlying complexity of the infrastructure, presenting a simplified and standardized interface to developers. This allows developers to focus on writing code and delivering business value, without getting bogged down in the intricacies of infrastructure management. Standardization also helps to ensure consistency, security, and compliance across all application environments. For example, the platform might provide a standard way to provision a new microservice, with pre-configured CI/CD pipelines, logging, and monitoring.

4. Automation as a Foundation: Automation is a cornerstone of the PaI pattern, underpinning everything from infrastructure provisioning and configuration to application deployment and monitoring. By automating repetitive and error-prone tasks, platform teams can improve efficiency, reduce the risk of human error, and enable a more scalable and resilient infrastructure. This includes using tools like Terraform and Ansible to automate infrastructure provisioning, and using CI/CD pipelines to automate the build, test, and deployment of applications.

5. Golden Paths, Not Golden Cages: While PaI promotes standardization, it should not be overly prescriptive or restrictive. The platform should provide “golden paths” – recommended and well-supported ways of doing things – but also allow for flexibility and customization when needed. This approach encourages adoption and innovation, while still providing a safety net of best practices and guardrails. For example, the platform might provide a default set of supported programming languages and frameworks, but also allow developers to bring their own tools if they have a good reason to do so.

6. Separation of Concerns: PaI establishes a clear separation of concerns between the platform team and the application development teams. The platform team is responsible for providing a reliable and secure infrastructure platform, while the application teams are responsible for building, deploying, and running their applications on top of that platform. This division of labor allows each team to focus on their core competencies and work more effectively. It also creates a clear contract between the platform team and its users, with well-defined service level agreements (SLAs) and support channels.

7. Continuous Improvement: A PaI is not a one-time project but an ongoing journey of continuous improvement. The platform team should constantly gather feedback from users, monitor key metrics, and iterate on the platform to better meet the evolving needs of the organization. This commitment to continuous improvement is essential for ensuring the long-term success and relevance of the platform. This can be done through regular user surveys, feedback sessions, and by tracking metrics such as developer satisfaction, lead time for changes, and service reliability.

3. Key Practices

1. Establish a Dedicated Platform Team: The first step in implementing PaI is to create a dedicated platform team with a clear mission and mandate. This team should be staffed with a mix of skills, including software engineering, infrastructure engineering, and product management. The platform team is responsible for building, operating, and evolving the internal developer platform. This team should be treated as a first-class citizen within the engineering organization, with its own budget, roadmap, and backlog.

2. Define the Platform’s Value Proposition: The platform team should work closely with its target users (i.e., developers) to understand their pain points and needs. Based on this research, the team should define a clear value proposition for the platform, outlining the key benefits it will provide, such as increased speed, improved reliability, or enhanced security. This value proposition should be communicated clearly to all stakeholders to ensure alignment and buy-in.

3. Build a Minimum Viable Platform (MVP): Instead of trying to build a comprehensive platform from the outset, the platform team should start with a minimum viable platform (MVP) that addresses the most pressing needs of its users. This iterative approach allows the team to deliver value quickly, gather feedback, and learn from its mistakes. The MVP should focus on a small number of high-impact services, such as a container orchestration platform and a CI/CD pipeline.

4. Develop a Platform-as-Code Strategy: The entire platform, including the infrastructure, the platform services, and the application configurations, should be managed as code. This enables the platform team to apply software engineering best practices, such as version control, automated testing, and continuous integration/continuous delivery (CI/CD), to the management of the platform. This also makes the platform more transparent, auditable, and reproducible.

5. Curate a Set of Platform Services: The platform should provide a curated set of services that are designed to meet the common needs of application developers. These services may include things like container orchestration (e.g., Kubernetes), databases, message queues, and observability tools. The platform team is responsible for selecting, integrating, and maintaining these services. The goal is to provide a “paved road” of well-supported and easy-to-use services, while still allowing for flexibility and choice.

6. Provide a Self-Service Interface: The platform should provide a self-service interface, such as a web portal or a command-line interface (CLI), that allows developers to easily provision and manage their application environments. This interface should be intuitive, well-documented, and backed by a robust set of APIs. The goal is to provide a seamless and frictionless experience for developers, so they can focus on what they do best: writing code.

7. Promote and Support the Platform: Building a great platform is only half the battle; the platform team also needs to promote and support it to drive adoption. This includes creating clear documentation, providing training and support, and actively engaging with the developer community to gather feedback and build a sense of shared ownership. The platform team should also market the platform internally, highlighting its benefits and success stories.

4. Application Context

Best Used For:

• Large and Complex Organizations: Organizations with a large number of development teams, a diverse application portfolio, and a complex technology landscape are prime candidates for PaI. In these environments, a centralized platform can help to tame the complexity, reduce duplication of effort, and enforce consistency and standards.
• High-Growth Companies: Companies that are experiencing rapid growth and need to scale their engineering organizations quickly can benefit greatly from PaI. A platform can help to onboard new developers more quickly, and to ensure that the infrastructure can keep pace with the growing demands of the business.
• Digital Transformation Initiatives: Organizations that are undergoing a digital transformation and are looking to modernize their application development and delivery practices are also good candidates for PaI. A platform can be a key enabler of this transformation, providing the foundation for a more agile, cloud-native, and DevOps-oriented way of working.
• Regulated Industries: Organizations in regulated industries, such as finance and healthcare, can use a PaI to enforce security, compliance, and governance policies in a consistent and automated way. This can help to reduce the risk of non-compliance and to simplify the audit process.

Not Suitable For:

• Small and Simple Organizations: For small organizations with only a handful of developers and a simple application architecture, the overhead of building and maintaining a PaI may not be justified. In these cases, a simpler approach, such as using a public cloud provider’s PaaS offering, may be more appropriate.
• Organizations with a Homogeneous Technology Stack: If an organization has a very homogeneous technology stack and a small number of well-defined application patterns, the benefits of a PaI may be limited. In these cases, a more lightweight approach, such as a set of shared libraries and templates, may be sufficient.
• Organizations with a Strong “Not Invented Here” Culture: A PaI requires a willingness to adopt a more standardized and centralized approach to infrastructure. If an organization has a strong “not invented here” culture, where every team wants to build its own infrastructure, it will be difficult to get the buy-in and adoption needed for a PaI to be successful.

Scale:

The Platform-as-Infrastructure pattern is highly scalable and can be applied to organizations of all sizes, from startups to large enterprises. However, the complexity and scope of the platform will vary depending on the size and needs of the organization. For smaller organizations, a simple PaI built on top of a public cloud provider’s PaaS offering may be sufficient. For larger enterprises, a more sophisticated PaI with a custom-built platform and a wide range of services may be required. The key is to start small, with an MVP, and to iterate and evolve the platform over time as the needs of the organization change.

Domains:

The PaI pattern is applicable across a wide range of industry domains, including:

• Technology: Software companies, e-commerce platforms, and other technology-driven businesses.
• Finance: Banks, insurance companies, and other financial institutions that are undergoing digital transformation.
• Healthcare: Healthcare providers and life sciences companies that are developing new digital health solutions.
• Retail: Retailers that are building e-commerce platforms and other customer-facing applications.
• Government: Government agencies that are modernizing their IT systems and delivering digital services to citizens.
• Manufacturing: Manufacturing companies that are embracing Industry 4.0 and are building smart factories and connected products.

5. Implementation

Implementing a Platform-as-Infrastructure requires a strategic and phased approach. The journey typically begins with a thorough assessment of the organization’s current state, including its existing infrastructure, tools, processes, and culture. This assessment helps to identify the key pain points and opportunities for improvement. Once the assessment is complete, the organization can begin to define a vision and roadmap for its internal developer platform. This roadmap should outline the key capabilities that the platform will provide, the target users, and the expected business outcomes.

The next step is to build a dedicated platform team and empower it to execute on the roadmap. The platform team should start by building a minimum viable platform (MVP) that delivers a small set of high-value services to a limited number of early adopters. This allows the team to validate its assumptions, gather feedback, and iterate on the platform in a controlled and agile manner. As the platform matures, the team can gradually add more services and expand its user base.

Technology selection is a critical aspect of implementing a PaI. The platform team will need to choose a set of tools and technologies that are well-suited to the organization’s specific needs and constraints. This may include a mix of open-source and commercial products, as well as custom-built components. Some of the key technology choices include the underlying infrastructure (e.g., public cloud, private cloud, or hybrid cloud), the container orchestration platform (e.g., Kubernetes), the CI/CD pipeline, and the observability stack. The choice of technologies should be driven by the needs of the users and the long-term vision for the platform, rather than by the personal preferences of the platform team.

Finally, it is important to remember that implementing a PaI is not just a technical challenge but also a cultural one. The organization will need to foster a culture of collaboration, shared responsibility, and continuous learning to ensure the success of the platform. This may involve providing training and coaching to developers, establishing clear communication channels between the platform team and its users, and celebrating successes along the way. It also requires a shift in mindset, from a project-based approach to a product-based approach, and from a focus on cost-cutting to a focus on value creation.

6. Evidence & Impact

The adoption of the Platform-as-Infrastructure pattern has had a profound impact on the software development and delivery landscape. Numerous organizations have reported significant improvements in their ability to innovate, compete, and respond to market changes after implementing an internal developer platform. For example, at Spotify, the adoption of a PaI has enabled the company to scale its engineering organization to thousands of developers, while still maintaining a high level of productivity and autonomy. Similarly, at Netflix, the company’s “paved road” platform has been instrumental in its ability to deliver a reliable and scalable streaming service to millions of users worldwide.

The impact of PaI can be measured across a number of key dimensions. In terms of speed, organizations that have adopted PaI have seen a dramatic reduction in the time it takes to provision infrastructure and deploy applications. This has enabled them to release new features and services to market more quickly and to respond more rapidly to customer feedback. In terms of quality, PaI has helped to improve the reliability, security, and compliance of applications by providing a standardized and automated way to manage the application lifecycle. And in terms of cost, PaI has enabled organizations to optimize their infrastructure spending by improving resource utilization and reducing the operational overhead associated with managing a complex and distributed infrastructure.

Another notable example is Adidas. The company embarked on a platform engineering journey to modernize its e-commerce platform and improve its ability to deliver new digital experiences to its customers. By building an internal developer platform on top of Kubernetes, Adidas was able to significantly reduce its lead time for changes, improve the reliability of its services, and empower its development teams to work more autonomously. The platform provided a standardized set of tools and services for building, deploying, and managing microservices, which enabled the company to move away from a monolithic architecture and embrace a more agile and scalable approach to software development. The success of Adidas’s platform engineering initiative has been widely cited as a model for other large enterprises that are looking to embark on a similar journey.

7. Cognitive Era Considerations

The advent of the cognitive era, characterized by the widespread adoption of artificial intelligence (AI) and machine learning (ML), is poised to have a significant impact on the Platform-as-Infrastructure pattern. AI and ML can be used to enhance the capabilities of the platform in a number of ways. For example, AI-powered observability tools can help to proactively identify and diagnose performance issues, while ML-based capacity planning tools can help to optimize resource allocation and reduce costs. Furthermore, AI and ML can be used to automate more complex and cognitive tasks, such as security threat detection and remediation.

As AI and ML become more deeply integrated into the software development lifecycle, the role of the PaI will become even more critical. The platform will need to provide the specialized infrastructure, tools, and services that are required to build, train, and deploy AI and ML models at scale. This may include things like GPU-accelerated computing, distributed data processing frameworks, and MLOps pipelines. By providing a robust and scalable platform for AI and ML, organizations can accelerate their adoption of these transformative technologies and unlock new sources of business value. The concept of a “self-driving” platform, which can automatically optimize its own performance, security, and cost-efficiency, is also emerging as a key area of research and development in the cognitive era. This will require a new generation of platform engineers who are skilled in both infrastructure engineering and data science.

8. Commons Alignment Assessment

• Shared Resource Potential: High - A Platform-as-Infrastructure is a shared resource by its very nature. It is a common platform that is used by all development teams within an organization to build, deploy, and operate their applications. By providing a shared set of tools, services, and best practices, a PaI can help to break down silos and foster a sense of collective ownership. This shared infrastructure can be seen as a form of digital commons within the organization, enabling a more efficient and collaborative use of resources.

• Democratic Governance: Medium - The governance of a PaI can be more or less democratic, depending on how it is implemented. In a more democratic model, the platform team would work closely with the developer community to co-create the platform and to make decisions about its future direction. This can be achieved through mechanisms such as open forums, request for comments (RFC) processes, and community-driven development. In a less democratic model, the platform team would make decisions in a more top-down manner. However, even in a less democratic model, the platform team is still accountable to its users and must be responsive to their needs to ensure the platform’s success.

• Equitable Access: High - A PaI can help to ensure that all development teams have equitable access to the resources they need to be successful. By providing a standardized and self-service way to access infrastructure, a PaI can level the playing field and reduce the “shadow IT” phenomenon, where teams go outside of the official channels to get the resources they need. This promotes fairness and equal opportunity for all teams to innovate and deliver value.

• Sustainability: Medium - The sustainability of a PaI depends on a number of factors, including the organization’s commitment to investing in the platform team, the platform’s ability to evolve to meet the changing needs of the organization, and the platform’s impact on the organization’s overall environmental footprint. A well-designed PaI can help to improve sustainability by optimizing resource utilization and reducing energy consumption. However, the constant need for maintenance, updates, and innovation can also create a significant resource burden.

• Community Benefit: High - A PaI can provide significant benefits to the developer community within an organization. By improving developer experience, increasing productivity, and fostering a culture of collaboration, a PaI can help to create a more engaged and empowered developer community. Furthermore, by open-sourcing parts of the platform or by contributing to open-source projects, the organization can share its knowledge and expertise with the broader community, creating a positive feedback loop of innovation and collaboration.