Microservices interactions in Ballerina

Many complex applications have to interact with various other applications when performing their operations. For example, a logistics application may have to interact with a vehicle tracking system to arrange optimal delivery schedules. It may also have to access warehouse management systems for checking stock levels. In addition, it may have to get customers’ delivery addresses and other details CRM systems. Furthermore, it has to interact with some kind of user store to authenticate users and determine which operations are allowed for them. Such logistics application may also expose its functions to multiple external components such as web portals, mobile apps running on drivers’ phones, data analytics and auditing systems, etc. Therefore, high level interactions of a such application can be modeled as follows:

When moving towards the microservices architecture, applications are decomposed into more granular components in order to facilitate independent teams and development processes, as well as to improve scalability. However, as a result each application becomes a collection of interacting microservices. Therefore, from the communications point of view, an application can be though of as a collection of microservices that call each other and external services, and expose services to external entities (e.g. web portals and other applications). For example, if the above logistics application is developed using microservices, interactions among application components could look like below, where each green rectangle represents a microservice.

These interaction involving microservices can vary considerably based on many factors such as number of communicating parties, whether a response is expected, expected delay for a response, how failures have to be handled, communication frequency, etc. In the rest of this post, we will go through some commonly used microservice interaction scenarios and see how those can be implemented using the Ballerina language.

We can start with a simple scenario, where a microservice S2 exposes an interface for consuming its services and another microservice S1 consumes services exposed by S2. Further, let’s assume that operations provided by microservice S2 needs some data and responds within a short period of time. This could be because the requested operation can be completed within a short period of time or S2 acknowledges the request and continue executing a long-running operation (we shall see this in detail later).

As the response is sent within a short period of time, usually such interactions can be completed using a single TCP connection (by keeping the connection open until the response is written). HTTP is a suitable application layer protocol for facilitating this type of request/response interactions. We can use a Ballerina HTTP service for implementing S2 as shown below:

Above service can be started using the below command (if the ballerina file is named as S2_http.bal):

ballerina run S2_http.bal

Once S2 service is started, it can be invoked from the command line as below:

curl localhost:9090/S2/operation1 -d "Some data"

Microservice S1 is the client application in this scenario. We can implement S1 in Ballerina to consume S2 as follows (Note that the term microservice is used here for both service and client applications. However in many scenarios, the client code we are using here would be just a part of a service, which wants to consume other services).

In above interactions, we assumed that the service S2 wants to send a response to the client immediately. Furthermore, we assumed that both client application (S1) and the service (S2) are running during the interaction. We also assumed that clients send requests at a reasonable rate so that the service can process them without causing any internal errors (e.g. OOM) or message loses. Although above assumptions can be true for some applications, there can be some situations where those will not hold. For example, what happens to requests if S2 went down for 5 minutes? Or if a client sends a burst of requests that exceeds S2’s capacity? In such situations we can employ a message broker in between clients and the service, so that the broker acts as a temporary store for messages as shown below:

If the receiving application (i.e. S2) is not available (e.g. due to a temporary failure) at the time of receiving a message, the message will be stored in the message broker and will be delivered once the application becomes available. If there is a spike in requests during a certain time period, message broker will store messages until being consumed by the target application, thus avoiding any message loses. Microservices written in Ballerina can consume requests from message brokers as follows:

Note that we are specifying the details necessary to connect to the broker (ActiveMQ in this example) when declaring the service. Whenever a message is received by the broker, onMessage(...) method will be triggered.

Although this approach can avoid message loses due to temporary server failures, you may have noticed that the Ballerina program does not have control over when to receive messages. Therefore, the consumption rate is controlled only by the number available threads in the Ballerina runtime. If more control over message consumption is required, receive action of the JMS receiver can be called at any point within a Ballerina program as shown below. Calling the receive action blocks the Ballerina program until a message is available or the given timeout occurs.

Microservice S2 exposed via a queue can be consumed by sending messages to the associated broker. This can be done in S1 as follows:

Another possible use case is that an application wants to broadcast messages to multiple listeners. For example, when a price of an item is changed, it has to be notified to services running at each outlet. Such interactions can be implemented using a construct named topics, which are usually available in message brokers (e.g. ActiveMQ, RabbitMQ, etc). A microservice interaction via a topic is depicted below:

Ballerina programs can listen for messages published to a topic using the below code (i.e. implementations of S2, S3 and S4):

Microservice S1, which sends messages to a topic is similar the one used for sending messages to a queue. Only difference is that we have to use a TopicPublisher instead of a QueueSender as below:

Another approach for achieving such publish/subscribe behavior among microservices is to use websub protocol. In the previous JMS based interactions, message broker acts as a central component to which publishers send messages and consumers subscribe to receive messages. Similarly, websub interactions has a component named hub, which acts as a central location for publishing and receiving messages. In this case, instead of using a third-party component, a Ballerina program can act as a hub as shown below. This hub program has to be started before starting publisher or subscriber programs.

Note that the hub.bal program registers a topic named “http://topic1.com” after starting the hub, so that publishers and subscribers can use this topic for exchanging messages. Ballerina programs can subscribe to this topic using the below code:

Now any Ballerina program can publish messages to the topic “http://topic1.com” as shown below:

All methods we have discussed so far allows microservices to consume messages over a network (via a higher level protocol implemented on TCP/IP). There is another simple method for consuming messages. That is, via files. Basically, the idea is that we specify a folder to which a microservice can listen. Whenever a new file (containing a message payload) is dropped into that folder, the corresponding microservice is notified, so that it can read and process the message contained in the new file.

Such file-based microservices can be implemented in Ballerina as follows:

Microservices interactions we have considered under HTTP have request/response semantics. That means, each request has an associated response and the client does not expect any message from the server other than a response. Further, the messaging based interactions discussed under JMS and websub have one-way communication semantics, which means that the client sends a message and does not expect a response (a client may expect a response via a different channel, but that is outside of the interaction we are considering). However, there can be situations where two microservices want to exchange messages frequently and either of those microservices can send messages whenever required. An example would be a chat application where a web browser and a chat server want to exchange messages. If we consider participants of a such interaction, each participant can act as a client (i.e. send messages whenever necessary) and a server (i.e. receive messages from the other participant). This type of message exchanges among microservices can be implemented using the websocket protocol. Usually, both participants of a websocket interaction has a callback handler to listen for incoming messages. Then any participant can send messages to the other party whenever necessary and receiver’s callback handler will process them.

Here, we assume that S1 initiates the websocket interaction and S2 is triggered by incoming messages. Microservice S2 can be implemented in Ballerina as follows:

S1’s implementation is similar to S2 in this case as both are using callbacks to handle incoming messages. In addition to the callback service, S1 sends a series of messages to S2 within the main() function.

There can be some microservices interactions where a service S1 wants to invoke another service S2, which takes a long time to respond. Note that although S2 takes a long time to process, it eventually responds. If S2 takes a long time to process but does not respond, we can simply use one-way messaging mechanisms discussed earlier. However, as S2 responds in this case, S1 should be ready to receive a response. An important point to note here is that we have no limitation on processing time of S2. It can take 2 minutes, 3 hours, 5 days or 3 months. Therefore, we cannot keep a TCP connection open until a response is available, which rules out HTTP or websockets for this scenario.

Basically, the request and response have to use two separate TCP connections. One option would be to use a callback service in S1, which will be called by S2 (in a separate TCP connection) once processing is complete. However, there is a problem here.. S1 may send multiple requests to S2. Then once S2 sends responses in new connections, S1 cannot match those responses with requests. Furthermore, S1 may need the response from S2 to proceed from a certain point (e.g. S1 may be waiting for item availability information from S2 to complete a sales order). If S1 receives responses using a callback service, S1’s service invocation logic is disconnected from the response processing logic. In order to overcome these problems, asynchronous service invocation patterns can be used with message correlations where the client (S1) can wait for a message with certain parameters. The invoked service (S2) responds with relevant parameters to facilitate the identification of a waiting client (S1).

Below is an implementation of a asynchronous service (S2) in Ballerina. Note that the service responds immediately (line 21) and starts processing later (line 27 simulates long processing time). Once processing is complete, it sends the reply by invoking the callback service in S1 (line 30).

Asynchronous client (S1) implementation is given below. Client’s main logic runs in the main() function. It invokes the long-running service S2 (line 10) and waits for a response using a channel (line 11). S1’s callback service listens for any responses from S2 and injects responses with the corresponding correlation parameter to the channel (line 26). When a message with a matching correlation parameter is received from the channel, waiting client will be unblocked and the message is returned (in this as the s2Res json variable).

We have looked at various possibilities of microservices interactions and how those can be implemented in Ballerina. As we have discussed, when building microservices based applications, these patterns can be used to facilitate interactions among application components as well as with external applications depending specific requirements. However, interaction scenarios are just one area of consideration for microservices applications. In addition, we have to consider aspects such as how to communicate securely among microservices, how to monitor and control message exchanges, how to deal with communication failures, etc. Therefore, technologies such as API gateways, message encryption (e.g. TLS), observability tools, authentication and authorization mechanisms (e.g. OAuth2, OIDC, SAML) and resiliency patterns may have to be integrated with above interaction scenarios as necessary when building microservices based applications.