Microservices communication using gRPC protocol

 

Hello everyone. In this article, we are going to learn about the gRPC protocol. gRPC has found huge success because of using HTTP/2 instead of HTTP/1.1 and protocol buffers instead of XML and JSON. 

gRPC Protocol = HTTP/2 + Protobuf

gRPC is widely used for communication between internal microservices majorly due to its high performance and its polyglot nature.

gRPC uses HTTP/2 as its transfer protocol and hence it inherits the benefits like binary framing from HTTP/2. As I mentioned in my previous article, HTTP/2 is robust, lightweight to transport, and safer to decode compared to other text-based protocols. Due to its binary nature, HTTP/2 forms a good combination with the Protobuf format.

gRPC works in the same way as old Remote Procedure Calls(RPC). It is API oriented as opposed to REST protocol which is resource-based. Protbuf gives it support for 11 languages out of the box and all the benefits of binary protocol superpowers in terms of performance, throughput, and flexibility.

gRPC for microservices

One of the biggest advantages of microservices is the ability to use different technologies for each independent service i.e polyglot. Each microservices agrees on API to exchange data, data format, error patterns, load balancing, etc. Since gRPC allows for describing a contract in a binary format, it can be used efficiently for microservices communication independent of the languages.

Image Source: https://miro.medium.com/max/1400/1*HRO6F1VnuK_3YOo_oPgxvg.png

In the above diagram, a microservices architecture is shown. The request from the client (web or mobile) reaches the API Gateway and then goes to the aggregator service.

The Shopping Aggregator microservice calls the other microservices internally and aggregates the data to send to the front end. As you can see, the communication between aggregators and other services happens through the gRPC protocol.

In a real-world scenario, microservices will talk to each other a lot to aggregate the data and send it to the client. If we use a normal REST API it will tend to be slower and the client will experience more latency. If we use gRPC between microservices, it will be much faster and have low latency for the client.

gRPC internals

Let us look at the steps needed for communication between 2 microservices using gRPC. The service that provides the service is the gRPC server and the other service which requires data from the provider is the gRPC consumer. A gRPC server can also be a consumer when it needs data from another microservice.

In this example, let us assume that the gRPC is implemented in the Go language and the consumer is implemented using Java. The network communication between the service and consumer takes place over HTTP/2 using the gRPC protocol.

Image Source: https://www.oreilly.com/library/view/grpc-up-and/9781492058328/assets/grpc_0101.png

There are 3 components as follows

1. Service Definition
2. gRPC Server
3. gRPC Client
   
   
   
   
   
   

1. Service Definition

1. The first step of building a gRPC service is to create the service interface definition with the methods that are exposed by that service along with input parameters and return types.
   
   The service definition is specified in the ProductInfo.proto file, which is used by both the server and client sides to generate the code and this serves as the API contract between them.
   
   These proto files are generated on the gRPC server and used in the client code irrespective of the language.
   2. gRPC Server
   
   With the service definition file, the source code can be generated using the protocol buffer compiler protoc. As mentioned in my previous article, protoc is supported by a lot of languages.
   
   With the gRPC plug-in for protocol buffers, you can generate gRPC server-side as well as the regular protocol buffer code for populating, serializing, and retrieving your message types.
   
   On the server side, the server implements that service definition and runs a gRPC server to handle client calls.
   3. gRPC Client
   
   On the client side, we generate the client-side stub using the service definition. The client stub provides the same methods as the server and it converts them to remote function invocation network calls that go to the server side. Since gRPC service definitions are language-agnostic, client stubs can be created from any language.
   
   
   gRPC Communication Patterns

gRPC supports four communication patterns: simple RPC, server-streaming RPC, client-streaming RPC, and bidirectional RPC. Each communication pattern addresses unique practical problems

1. Simple RPC

This is the simplest communication pattern by gRPC. The client sends a single message to the server and receives a single message. As shown in the diagram below, only one binary message is going in the stream on both request and response.

Image Source: https://cdn.thenewstack.io/media/2021/08/bb5ced3a-image1.png

2. Server-Streaming RPC

In server-streaming RPC, once the client sends a message to the server, the server sends back a sequence of responses. This sequence of response messages is sent inside the same HTTP stream initiated by the client.

As shown in the diagram below, the server waits until it receives the message from the client and sends multiple response messages as framed messages. The server concludes the stream by sending the trailing metadata with the call status details.

Image Source: https://cdn.thenewstack.io/media/2021/08/c6826dd4-image3.png

3. Client-Streaming RPC

In this pattern, the client sends multiple messages to the server and the server sends only one message in return. The diagram below shows how multiple messages flow through the stream. This stream can last until the end of RPC.

Image Source: https://cdn.thenewstack.io/media/2021/08/74a4c5f9-image4.png

4. Bidirectional Streaming RPC

In bi-directional streaming RPC, both the client and server send a stream of messages to each other. The client sets up the HTTP stream by sending header frames. Once the connection is established, both the client and server can send messages simultaneously without waiting for the other to finish.

Image Source: https://cdn.thenewstack.io/media/2021/08/414e501c-image5.png

gRPC Use Cases

gRPC can be used for the following use cases

• Microservices: gRPC is designed for low latency and high throughput communication. As discussed above, it works very well for microservices where efficiency and latency are critical.
• Point-to-point real-time communication: gRPC has excellent support for bi-directional streaming. gRPC services can push messages in real-time without polling.
• Polyglot environments: gRPC tooling supports all popular development languages, making gRPC a good choice for multi-language environments.
• Low-power low-bandwidth networks: gRPC messages are serialized with Protobuf, a lightweight message format. A gRPC message is always smaller than an equivalent JSON message.
• Inter-process communication (IPC): IPC transports such as Unix domain sockets and named pipes can be used with gRPC to communicate between apps on the same machine.

Benefits of gRPC

1. Lightweight messages
2. High performance
3. Built-in code generation
4. Multiple language support
5. Different Communication Patterns

Drawbacks of gRPC

1. Limited Browser Support:
2. Non-human Readable Format
3. No Edge Caching
4. Steeper Learning Curve

gRPC vs REST API

As gRPC heavily uses HTTP/2, it is impossible to call a gRPC service from a web browser directly. Modern browsers do not provide the control needed over web requests to support a gRPC client. Hence, a proxy layer and gRPC-web are required to perform conversions between HTTP/1.1 and HTTP/2.

So REST APIs are still well suited for client-server communication and gRPC is now mainly used for server-to-server communication between microservices

Image Source: https://cdn-gcp.marutitech.com/wp-media/2022/01/1cbf6ec0-grpc_vs._rest_2_copy.png

In this article, we saw about gRPC internals and how microservices communicate using the gRPC protocol. Then we saw about 4 types of communication patterns. We also saw the use cases for gRPC and the benefits and drawbacks of gRPC. We have concluded this article by comparing gRPC with the REST framework

Hope you had good learning and thanks for reading!!!