What is CQRS?

CQRS, which stands for "Command and Query Responsibility Segregation," is a pattern that separates the read and write operations of data storage. Implementing CQRS in an application can greatly improve its performance, scalability, and security. The flexibility created by migrating to CQRS allows the system to evolve better over time and prevents update commands at the domain level from causing merge conflicts.

In traditional architectures, the same data model is used for both querying and updating the database. This approach is simple and suitable for basic CRUD operations. However, in more complex applications, this method becomes difficult to handle. For example, on the reading side, the application may perform a large number of different queries that return data transfer objects (DTOs) with different shapes. Object mapping can become complex. On the writing side, the model may implement complex validation and business logic. As a result, the model becomes too overloaded and overly complex.

Workloads for reading and writing data are often asymmetric, with significant differences in performance and scalability requirements. For example, in common scenarios like flash sales, the database often faces a huge reading load at the start of an event, while the writing load becomes relatively smaller after going through message queue processing.

There can be a mismatch between the representation of data for reading and writing, such as additional columns or properties that need to be updated correctly (even if they are not part of the operation).

Data contention can occur when performing operations on the same set of data in parallel.

Traditional approaches can have a negative impact on performance due to the workload on data storage and data access layers, as well as the complexity of queries required to retrieve messages.

Managing security and permissions can become complex as each entity is simultaneously affected by both read and write operations, potentially exposing data in the wrong context.

CQRS separates the read and write operations into different models, using commands to update data and queries to read data.

Commands should be based on tasks rather than being data-centric ("book a hotel room" instead of "set ReservationStatus to Reserved"). Commands can be queued for asynchronous processing instead of synchronous processing. Queries should never modify the database. The DTOs returned by queries should not encapsulate any domain content. Using separate query and update models can simplify design and implementation. However, one drawback is that it is not possible to generate CQRS code automatically from the database schema using framework mechanisms such as O/RM tools (however, generated code can be used to build custom ones).

To achieve better isolation, reading data can be physically separated from writing data. In this case, the read database can have its own data schema optimized for queries. For example, it can store materialized views of the data, avoiding complex joins or ORM mappings. It may even use a different type of data storage. For example, the write database could be a relational database while the read database could be a document database.

If using separate read and write databases, they need to be kept in sync. Typically, when the write model updates the database, it publishes an event, which allows for synchronization. Since message brokers and databases usually cannot participate in a single distributed transaction, there are some challenges in ensuring consistency when updating the database and publishing events.

The read store can be a read-only copy of the write store, or the read and write stores can have completely different structures. Using multiple read-only copies can improve query performance, especially in distributed scenarios where the read-only copies are located closer to the application instances.

The separation of read and write stores also allows them to be scaled appropriately to match the workload. For example, the read store often encounters a higher load than the write store.

Some CQRS implementations use the event sourcing pattern. In this pattern, the application state is stored as a sequence of events. Each event represents a series of changes to the data. The current state is constructed by replaying the events. In the context of CQRS, one benefit of event sourcing is that the events can be used to notify other components, specifically the read models. The read models use the events to create snapshots of the current state, which is more efficient for queries. However, event sourcing adds complexity to the design.

Independent scalability: CQRS allows the read and write workloads to be scaled independently, reducing lock contention. Optimized data schemas: The read side can use an architecture optimized for queries, while the write side can use an architecture optimized for updates. Security: It becomes easier to ensure that only the appropriate domain entities perform write operations on the data. Separation of concerns: Separating the read and write sides makes the models easier to maintain and more flexible. Most complex business logic is delegated to the write model, while the read model remains relatively simple. Simplified queries: By storing materialized views in the read database, the application can avoid complex joins when querying.

There are some challenges when implementing this pattern, including:

Complexity: The basic idea behind CQRS is simple, but it can lead to increased complexity in the design of an application, especially when combined with the event sourcing pattern.

Messaging: Although not required, CQRS often uses messaging for handling commands and publishing update events. In this case, the application needs to handle message failures or duplicate messages. Refer to guidance on priority queues for handling commands with different priorities.

Eventual consistency: If the read and write databases are separated, the read data may become outdated. The read model store must be updated to reflect changes in the write model store, which can be challenging when dealing with requests based on outdated read data.

CQRS can be considered in the following situations:

When many users collaborate over the same data in a co-operative domain. CQRS allows defining commands with sufficient granularity to minimize domain-level merge conflicts, and the conflicts that do occur can be resolved through command merging.

Task-based user interfaces that guide users through complex processes composed of a series of steps or complex domain models. The write model has a full command handling stack, including business logic, input validation, and domain validation. The write model treats a set of related objects as a single unit of data change (an aggregate in DDD terms) and ensures these objects remain in a consistent state. The read model has no business logic or validation stack, only returning DTOs for use in view models. The read model stays consistent with the write model.

Scenarios where data read performance needs to be tweaked independently of data write performance, especially when the number of reads is significantly higher than the number of writes. In this scenario, the read model can be scaled horizontally, but the write model runs on a small number of instances. Having a small number of write model instances helps minimize merge conflicts.

Application scenarios where one development team can focus on complex domain models as part of the write model, while another team can focus on the read model and user interface.

Application scenarios where the system evolves over time and may include multiple versions of the model or regularly changing business rules.

Integration with other systems, especially when integrating with event sourcing, where a temporary failure in one subsystem should not affect the availability of other subsystems.

It is not recommended to use this pattern in the following cases:

When the domain or business rules are very simple.

Simple CRUD-style user interfaces and data access operations.

Apply CQRS to the limited parts of the system where it can realize its value the most.

Relationship with Event Sourcing

CQRS is often used in conjunction with the event sourcing pattern. In a CQRS-based system, separate models are used for reading and writing data, each tailored to their respective tasks and often residing in physically separated storage. When used in conjunction with event sourcing, the event store becomes the write model and the official source of information. The read model in a CQRS-based system provides materialized views of the data, often highly denormalized views customized for the interfaces and display requirements of the application, improving display and query performance.

Using an event stream as the write store (instead of actual data at a particular point in time) allows avoiding update conflicts on a single aggregate and maximizes performance and scalability. Events can be used to generate data-specific materialized views for the read store asynchronously.

Since the event store serves as the official source of information, the materialized views can be discarded and all past events can be replayed to create a new representation of the current state during system upgrades or when a change in the read model is needed. Materialized views are essentially persistent read-only caches of the data.

When using CQRS and event sourcing together, the following points should be noted:

In a system where the read and write stores are separated, a CQRS-based system is eventually consistent. There is a certain delay between the generation of events and the updating of the data storage.

This pattern introduces complexity, as code needs to be created to handle events and compose or update views or objects needed for queries or the read model. When combined with the event sourcing pattern, the complexity of the CQRS pattern makes it difficult to implement smoothly and requires the use of other techniques in designing the system. However, event sourcing makes it easier to create models for the domain, allowing for easy regeneration of views or the creation of new ones, as it retains the data changes that need to be performed.

Generating materialized views or data projections for the read models by replaying and processing events specific to entities or collections of entities can require a significant amount of processing time and resources, especially when long-running aggregations or analysis values are needed, as all relevant events need to be examined. This can be addressed by implementing data snapshots at planned intervals, such as the total count of specific operations that have occurred or the current state of an entity.