Database-Per-Service Pattern
Also known as: Database per Microservice, Service-Specific Database
1. Overview
The Database-per-Service pattern is a fundamental architectural principle in the design of microservices-based systems. It dictates that each microservice should have its own private database, accessible only by that service [1]. This stands in contrast to the traditional monolithic approach where multiple services share a single, large database. The historical origins of this pattern are tightly coupled with the rise of microservices architecture, which emerged as a solution to the scalability and maintenance challenges of monolithic applications. By decentralizing data ownership, the Database-per-Service pattern enables the core tenets of microservices: loose coupling, independent deployment, and technological diversity.
2. Core Principles
The pattern is defined by a set of clear and concise principles:
- Data Encapsulation: Each microservice encapsulates its own data. Other services cannot access the database directly and must interact with the data through the service’s public API.
- Independent Scalability: By having separate databases, each service’s data store can be scaled independently based on its specific needs, without impacting other services.
- Technological Heterogeneity: Each service can choose the database technology that is best suited for its specific requirements. For example, a user service might use a relational database, while a product catalog might use a NoSQL database.
- Loose Coupling: Services are not tied together by a shared database schema. This allows services to evolve independently, as changes to one service’s database do not directly impact others.
3. Key Practices
In a monolithic architecture or a microservices architecture with a shared database, several problems arise:
- Tight Coupling: A shared database creates tight coupling between services. A change in the database schema required by one service can break other services.
- Reduced Autonomy: Development teams are not ableto work independently. Changes to the database need to be coordinated across teams, slowing down development and deployment.
- Scalability Bottlenecks: A single database can become a performance bottleneck. It is difficult to scale the database to meet the conflicting requirements of multiple services.
- Technology Lock-in: A shared database forces all services to use the same database technology, even if it is not the best fit for all of them.
4. Implementation
The Database-per-Service pattern provides a clear solution to these problems. By assigning each microservice its own private database, the pattern enforces a strong boundary between services. This boundary ensures that each service is the sole owner of its data, and all data access occurs through a well-defined API. This approach promotes loose coupling and service autonomy, as each team can manage its own data model and technology stack without interfering with other teams. Consequently, services can be developed, deployed, and scaled independently, leading to a more agile and resilient system.
5. 7 Pillars Assessment
| Pillar | Score (1-5) | Rationale |
|---|---|---|
| Purpose | 3 | Serves a clear technical purpose in system design |
| Governance | 3 | Can be governed through standard engineering practices |
| Culture | 3 | Supports engineering culture of reliability and quality |
| Incentives | 3 | Aligns incentives toward system stability |
| Knowledge | 4 | Well-documented pattern with extensive community knowledge |
| Technology | 4 | Directly applicable to modern technology stacks |
| Resilience | 4 | Contributes to overall system resilience |
| Overall | 3.4 | A valuable technical pattern that supports commons infrastructure |
While the Database-per-Service pattern offers significant advantages, it also introduces a new set of challenges:
| Pros | Cons |
|---|---|
| Loose Coupling and Autonomy | Data Consistency |
| Independent Scalability | Transactional Complexity |
| Technology Flexibility | Increased Operational Overhead |
Data Consistency: Maintaining data consistency across multiple databases is a significant challenge. Since ACID transactions cannot span multiple databases, eventual consistency models are often adopted, which can add complexity to the application logic.
Transactional Complexity: Implementing business transactions that involve multiple services requires a different approach, such as the Saga pattern, which coordinates a series of local transactions.
Increased Operational Overhead: Managing multiple databases increases operational complexity. Each database needs to be provisioned, monitored, and backed up, which can be a significant undertaking without proper automation and infrastructure.
6. When to Use
Many large-scale, successful companies have adopted microservices architectures and, by extension, the Database-per-Service pattern:
- Netflix: One of the most well-known examples of a company that has successfully implemented a microservices architecture. Each of their services, from user authentication to content recommendation, has its own data store.
- Amazon: Amazon’s e-commerce platform is composed of hundreds of microservices, each with its own database. This allows them to innovate and scale different parts of their platform independently.
- Uber: Uber’s ride-sharing platform is another example of a complex system built on microservices. Each service, such as trip management, billing, and driver tracking, has its own dedicated database.
7. Anti-Patterns & Gotchas
In the cognitive era, where AI and machine learning are becoming increasingly prevalent, the Database-per-Service pattern remains highly relevant. AI/ML models often have unique data storage and processing requirements. This pattern allows for the use of specialized databases (e.g., vector databases for embeddings, graph databases for knowledge graphs) for specific AI/ML services without impacting the rest of the system. Furthermore, the isolation provided by this pattern can be beneficial for data governance and security, which are critical when dealing with sensitive training data.
8. References
The Database-per-Service pattern aligns with several of the Commons principles:
- Shared Resource: While each service has its own database, the overall system of interconnected services can be seen as a shared platform. The APIs of the services are the shared resources that enable interaction and data exchange.
- Democratic Governance: The pattern promotes decentralized governance. Each team has autonomy over its own service and data, which aligns with the principle of distributed decision-making.
- Equitable Access: Access to data is managed through well-defined APIs, which can be designed to provide equitable access to different services and users based on their roles and permissions.
- Sustainability: The ability to scale services independently can lead to more efficient resource utilization, which contributes to the long-term sustainability of the platform.
- Community Benefit: By enabling the creation of more scalable, resilient, and evolvable systems, the pattern ultimately benefits the community of users who rely on these systems.
References
[1] Microservices.io. Pattern: Database per service. https://microservices.io/patterns/data/database-per-service.html [2] AWS Prescriptive Guidance. Database-per-service pattern. https://docs.aws.amazon.com/prescriptive-guidance/latest/modernization-data-persistence/database-per-service.html [3] Medium. The Database-per-Service Pattern. https://medium.com/design-microservices-architecture-with-patterns/the-database-per-service-pattern-9d511b882425