Join the DZone community and get the full member experience.
It is intended for architects and developers to use this information to guide their integration solutions. Once an integration style has been selected, they should review the associated integration style section to understand the typical patterns relevant to that integration style, abide by the constraints and review the implementation considerations.
There is an important distinction around the responsibilities of an integration system and applications. Integration facilitates the transmission of information between systems and may be realized through the use of technologies such as messaging systems, API gateways, and their associated protocols (MQ, AMQP, HTTP, FTP). Over the years, many technology vendors have implemented additional capabilities within their integration products that have enabled developers to add business logic embedded within the integration solution. This has resulted in business logic being spread outside of the application layer and integration an integration system that is often managed and maintained by a single team resulting in a resource bottleneck.
In following a microservice-based architecture that supports a decentralized development model, applications are implemented as independently developed and operated modules that encapsulate business logic. Whilst microservices may connect with other microservices or other systems of record they do so via a communication mechanism that does not hold business logic or manage business state.
Business logic is the programmatic encoding of real-world business rules that govern the access and lifecycle of business domain data. Simply, if an action requires the processing of business rules against business information then it constitutes business logic.
Business logic includes:
Business logic should only be implemented within an application, not within an integration layer. The integration layer facilitates the exchange of information between applications and includes the following technical components: API gateways, ESBs, File Transfer Systems and Messaging Systems. Syntactic transformation, the straight through mapping of attributes with the same business meaning, may be implemented within the integration layer.
Synchronous communication involves the consumer submitting a request to a service and waiting for the service to respond before the consumer can continue executing its process. The technical implementation of this form of communication has evolved over time with sophisticated frameworks and computing capabilities so that an application does not necessarily need to block in whole. From a user experience perspective, a User Interface (UI) may continue to be responsive to interaction whilst a synchronous request has been submitted.
Whilst a synchronous process may utilise different integration styles a common synchronous communication protocol is HTTP, used with the API style of integration. HTTP by nature is synchronous in that a request is submitted (either requesting information or submitting information) and the submitting application waits for the response (either a simple OK or some data in the payload is returned). Modern frameworks ensure the UI or process can continue to operate in parallel to perform other tasks however some thread of execution is blocked to await the response. Typical request/response time frame is ideally measured in milliseconds to a few seconds with timeout set to 30-60 seconds.
Asynchronous communication involves the consumer submitting a message to a service but not needing to wait for the service to respond. A consequence of the message in the form of a new message may be returned to the consumer at some point in the future or not at all. The consumer often does not know the destination service and simply submits the message to a messaging system which then delivers the message to one or more destination services. Asynchronous communication may be realised through different integration styles but is typically implemented using asynchronous protocols such as AMQP, MQ, MQTT, Web Sockets or proprietary binary protocols over TCP sockets such as with Kafka.
Web browsers largely use the HTTP protocol and web applications will include JavaScript that execute within the browser to make API calls over HTTP. Modern JavaScript frameworks can ensure the UI is not blocked when making these synchronous calls. For cases where an asynchronous interaction is required, polling is used by the JavaScript code to determine the presence of response data from the service however asynchronous integration may also be used such as web sockets and MQTT.
An overview of the identified styles and patterns adopted within the Digital Platform are shown in Figure 1 using the definition of style and pattern as:
Style — High level, abstract, approaches to integration that represent a number of specific patterns
Patterns — Technology agnostic, generic designs that address common business problems/scenarios
The following integration styles will be supported by the Digital Platform going forward:
For each of these styles there are various patterns and constraints that will be defined. There are situations where some or all participants in an integration do not support directly connecting with a messaging system. These participants may utilise an intermediary service to perform a translation between an API integration to a messaging interface.
Figure 1 Integration Styles and Patterns
The following table provides a set of scenarios under each integration style to help select a suitable style. Identify a matching scenario to select the matching integration style.
| API | Messaging | File Transfer |
| Application submits data, requests data or issues command to an internal or external application or service that requires immediate processing of submitted data or command and subsequent near-immediate response is expected. | Application submits data or issues command to an internal application or service that does not require a near-immediate response. | Transfer large data set to one or more applications as a bulk data set as part of a batch process. |
| Application submits data or issues command to an internal application or service that is likely to have an on-going planned or unplanned outages and does not require a near-immediate response. | Existing application only supports file transfer-based integration. | |
| Application submits data or issues command to multiple internal applications or services but does not require near-immediate response. | ||
| Application submits a high volume of data or commands to one or more internal applications or services and does not require a near-immediate response. | ||
| Example | Example | Example |
| Customer requests an account via web application in digital channel. The customer has submitted their details and is awaiting a response. The response time requirement is < 3 seconds. A number of components are integrated through a series of API-based calls. This example illustrates a synchronous interaction with an end user and supporting systems that follow the API style integration. | An update to a customer’s profile is made via a web application in the digital channel. A number of systems must be notified of the changes in order for them to change their own records. This example illustrates an asynchronous update of multiple systems that does not require a response to be returned to the initiating system. | A large number of charging requests are generated on a nightly basis and must be issued to the financial system once a night for processing. A file transfer style of integration may be used to transfer the payment requests within one or more files that are transferred as part of a batch process to the financial system. This example illustrates a file transfer style integration. |
| Principle | Condition | Guidelines | Reasoning |
| Business logic should not be implemented within the integration technology layer. The integration layer includes the following technical components: API gateways, ESBs, File Transfer Systems or Messaging Systems. (Think dumb pipes, smart end points) | All | Integration should facilitate the exchange of information between applications. |
Integration components should not include any business logic or state.
Integration adapters used as a façade to API providers should not include any business logic or state.
Syntactic transformation for the purpose of implementing an integration adapter is not considered business logic.
Data enrichment is considered business logic and must not be implemented within the integration layer. This may be implemented by a package application or a microservice.
Integration technologies are primarily designed for data transfer and have significantly less capability in solving the complex and broad set of business problems when compared with an application technology that is programmable using a general programming language.
| Principle | Condition | Guidelines | Reasoning |
| Capability that exists within a strategic packaged application should not be hollowed out into an independent microservice if the capability does not need to be independently managed or developed and at the expense of significant cost and supportability of the package. | All | Packaged applications should not be modified to extract and disable business capability that is central to the package operation if the capability does not need to be independently managed or developed. |
Package applications should not be modified to extract and disable business capability if by doing so significant cost and supportability is incurred.
Capabilities within a packaged application may be enhanced or extended within a reasonable cost (balance cost/return).
Minimise the impact on cost and supportability of modifying strategic packaged solutions.
API may be invoked to initiate behaviour and/or exchange information. A consumer may call a target system API directly or through an intermediary.
This style of integration is synchronous in nature and is used where a response is required within a short time frame, typically as a result from a user action or a step in an executing process. API calls may also be used to trigger processes, expose interfaces externally in a secure and restricted manner or act as an intermediary translation style for the messaging integration style.
| Pattern | Description | When to use | Considerations |
| Request/response | A consumer submits a request to an application that processes the request and returns the result over the same client connection. | When the response from the request is required within a short time frame and the consumer cannot continue without the result. |
The consumer may periodically submit a request for the status or result of the processing of the request.
The consumer requires to be called by the provider when the result is available.
| Constraint | Condition | Rules | Reasoning |
| APIs must confirm to the REST architectural style | Developing a new API as part of bespoke application or microservice | APIs must be stateless. |
APIs must conform to a uniform interface based on HTTP operations.
Resources must be identified in the request.
Support Hypermedia as the engine of application state (HATEOS).
Resources may be cacheable.
The API intermediary will perform syntactic and/or semantic transformation from the independently defined information model and schema to a downstream provider system’s information model and schema.
Service contracts must have defined schemas.
A connection between applications via a common messaging system is used to exchange data or invoke behaviour using messages of a pre-defined format. The messaging style maximises decoupling between systems not only from the interface perspective but also from a time-based perspective. The messaging system includes queueing to allow for a consumer reading a message from the messaging system when it is ready, allowing for the handling of high-volume spikes.
| Pattern | Description | When to use | Considerations |
| Publish-Subscribe | An application submits a message of which one or more applications may receive the message. | One or more applications need to receive the message. |
The publisher does not need to know who will consume the message.
High volume load on target system may impact system’s capacity to process messages.
The emitter of the event does not need to know who will consume the event.
| Constraint | Condition | Rules | Reasoning |
| Message schema must be participant independent. | Multiple Consumers or Multiple Providers | The schema must be based on a system independent information model. | Provides decoupling between applications. |
| Message schema may be consumer or provider based. | Consumer and Provider interfaces are custom developed and unlikely to introduce other participants. | The schema may be based on a system independent information model if the effort to prepare the schema is cost effective. |
Information is exported from a source application to a file system as a file in a format such as CSV and picked up by the target system for processing. The file can be directly imported by the consuming file system or transported to a destination file system for the consuming system to retrieve. This style of integration is used where a bulk amount of information may be transferred as part of a batch process often executing at pre-defined times. File Transfer must not be used where API or messaging styles of integration are available and suitable for participant applications.
| Pattern | Description | When to use | Considerations |
| Point-to-point | An application creates one or more files to be consumed by a single target system. | Only one target is required to consume the file and it is highly unlikely any other target will require the file in the future. | |
| One-to-Many | An application creates one or more files to be consumed by multiple target systems. | Multiple applications need to receive the file. |
| Constraint | Condition | Rules | Reasoning |
| File format must be defined by the source system. | All | The file format must be defined by the source system. |
In a microservices application architecture style an application is composed of many discrete network-connected components termed microservices. Each component is a small application itself that implements a part of the overall application. They are independently developed, packaged, deployed, scaled and managed. This is in contrast to monolithic applications that are developed as a single application that is packaged and deployed as a single unit.
In order to provide some level of organisational alignment in terms of ownership and accountability, a collection of microservices may be logically grouped to form an application boundary. The application boundary may align to a domain or subdomain, defined by a bounded context through Domain Driven Design. Within this grouping a single or small selection of microservices may be exposed as service interfaces for consumption outside the domain/subdomain. The service interfaces may be API or messaging based. Some service interfaces may be exposed to external parties via an external facing API gateway or even exposed internally via an internal API gateway as managed APIs. Within an application boundary, microservice to microservice communication may be established through a service mesh under a cloud deployment environment where the service mesh provides capabilities such as service discovery and load balancing.
Ideally microservices should be independent, with their own persisted data where appropriate and not relying on calling downstream services in order to fulfil their responsibilities. However, there are cases where some microservices do integrate with other packaged applications or microservices some of which may be provided by external parties.
Packaged applications are typically implemented as monolithic applications. Even though their internal structure may be divided as modules that communicate through some form of inter-process communication or internal messaging system. They are more or less deployed as a single deployment unit and require a vendor to incorporate changes through a re-deployment of the entire package as a whole or through patches. Microservices and packaged applications may co-exist simple through decisions around the allocation of domain responsibilities and associated service composition.
A packaged solution may be responsible for an entire domain of which it provides one or more service interfaces as part of its bounded context. In these cases, an adaptor is often used as a façade to avoid directly coupling to the packaged application’s specific interface. This provides a level of protection from having to change consumer integrations if the packaged application is eventually changed. A packaged application may also be “hollowed out” where certain modules are disabled and bypassed, or it lacks some capability that may be provided by one or more custom built microservices. The microservices complement the packaged application and collectively they form the logical application represented by the dotted bounded box in the diagram below. The packaged application could expose an interface as of which the microservice consumes. The microservice’s interface would be exposed a business API at the application boundary.
The diagram below illustrates a generalised example of an application architecture that comprises applications integration through the invocation of APIs and/or event based.
Figure 2 - Packaged Application & Microservices Architecture Example
The patterns defined in the table below provide a refence for commonly used arrangements of microservices with their associated consumers/providers. Whilst a persistent storage is included with each pattern and is typical of an ideal microservice it does not mandate the need for one as some microservices may not need to hold state.
There are a number of microservice implementation patterns however the focus of this section is around the integration between the microservices and consumers/provider. The purpose is to clarify the options for how they are arranged and highlight areas where data transformation is required and where it should be implemented.
Other microservice implementation patterns include:
In an event-driven architecture, applications interact by producing and consuming events. An event is an immutable notification of the change of state of a business entity or user action within an application. One or more applications may subscribe to events of interest. Events are typically communicated via a messaging system. A number of application architecture patterns may be built upon events including:
An event may be transferred between applications using an asynchronous, messaging style of integration following a publish-subscribe pattern. The level of event detail may vary within the message payload. The publishing application is not aware of the consuming applications and the consuming applications are not aware of the publisher. This provides the basis of decoupling between the applications. The messaging system does not provide a long-term persistence of the messages but may provide an optional persistence to facilitate delivery assurance in case of a breakdown in communication. Once a message is consumed by the subscribers the message is typically removed from the messaging system. It is the responsibility of the subscribers to persist and/or perform the necessary action upon receiving the event.
The diagram below illustrates an application publishing an event (or emitting an event) to a topic through a messaging system. One or more applications may subscribe to the topic and receive the event.
Figure 3 - Application Architecture - Events Pattern
In separating the read and write models CQRS requires a mechanism to provide updates between them. This can be achieved through the use of an asynchronous messaging style of integration following a point-to-point pattern or publish/subscribe pattern. A command may be submitted via a synchronous API style o integration following a request/response pattern to update the write-optimised data store. Alternatively, an event may also be submitted to update an event store. When the data or event store are updated an event is emitted comprising the change details from which a read optimised data store will receive. The read-optimised store will persist the change in a materialised view appropriate for supporting queries. An application may perform a query against the read optimised store using an API style of integration following a request/response pattern.
Figure 4— Application Architecture - CQRS Pattern
An event sourced system persists a sequence of events rather than repeatedly overwriting a single state. The state is kept in-memory and is updated by events generated as a consequence of processing commands. A command may be submitted via a synchronous API style of integration following the request/response pattern where it is checked if it can be applied. If it can an event is created from which the state is changed. The event is persisted in an event journal and published to a topic using an asynchronous message style of integration following a publish/subscribe pattern from which other subscribers may also receive the event. The in-memory state machine may be recreated through replaying of the event history. Event Sourcing is typically complemented by the CQRS pattern to provide materialised views from the update events.
Figure 5 — Application Architecture - Event Sourcing Pattern
• UI Application makes API Backend for Frontend service that in turn makes API call to Microservice.
• User is waiting for response (e.g. submitted scheduled payment – you want to know it was successfully scheduled)
• Microservice either retrieves data from its own data store or calls a downstream provider via API call.
• Suitable for low volume scenarios.
• Direct coupling between components.
Transactions executed across microservices as part of a long running process can be implemented using the Saga pattern. The Saga pattern relies on each microservice performing a step in the process, updating its own data from which triggers the subsequent step to be completed by another microservice. There are two types of implementations for the Saga pattern: Choreography and Orchestration.
Opinions expressed by DZone contributors are their own.
