In the last few years, we saw a great shift in technology, where projects are moving towards “microservice architecture” vs the old "monolithic architecture". This approach has done wonders for us.
As we say, “smaller things are much easier to handle”, so here we have microservices that can be handled conveniently. We need to interact among different microservices. I handled it using the HTTP API call, which seems great and it worked for me.
But is this the perfect way to do things?
The answer is a resounding, "no," because we compromised both speed and efficiency here.
Then came in the picture, the gRPC framework, that has been a game-changer.
Quoting the official documentation-
“gRPC or Google Remote Procedure Call is a modern open-source high-performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication.”
Credit: gRPC
RPC or remote procedure calls are the messages that the server sends to the remote system to get the task(or subroutines) done.
Google’s RPC is designed to facilitate smooth and efficient communication between the services. It can be utilized in different ways, such as:
- HTTP/2 based transport - It uses HTTP/2 protocol instead of HTTP 1.1. HTTP/2 protocol provides multiple benefits over the latter. One major benefit is multiple bidirectional streams that can be created and sent over TCP connections parallelly, making it swift.
- Auth, tracing, load balancing and health checking - gRPC provides all these features, making it a secure and reliable option to choose.
- Language independent communication- Two services may be written in different languages, say Python and Golang. gRPC ensures smooth communication between them.
- Use of Protocol Buffers - gRPC uses protocol buffers for defining the type of data (also called Interface Definition Language (IDL)) to be sent between the gRPC client and the gRPC server. It also uses it as the message interchange format.
Let's dig a little more into what are Protocol Buffers.
Protocol Buffers like XML, are an efficient and automated mechanism for serializing structured data. They provide a way to define the structure of data to be transmitted. Google says that protocol buffers are better than XML, as they are:
Protobuf are defined in .proto files. It is easy to define them.
1. Unary RPCs:- This is a simple gRPC which works like a normal function call. It sends a single request declared in the .proto file to the server and gets back a single response from the server.
CODE: https://gist.github.com/velotiotech/d2938c90ee7948186e7a3848f3558577.js
2. Server streaming RPCs:- The client sends a message declared in the .proto file to the server and gets back a stream of message sequence to read. The client reads from that stream of messages until there are no messages.
CODE: https://gist.github.com/velotiotech/0bdb7a50673c97745b37995a83f74ba3.js
3. Client streaming RPCs:- The client writes a message sequence using a write stream and sends the same to the server. After all the messages are sent to the server, the client waits for the server to read all the messages and return a response.
CODE: https://gist.github.com/velotiotech/757cef3a558b6ffbd38ff6eee37ab8ab.js
4. Bidirectional streaming RPCs:- Both gRPC client and the gRPC server use a read-write stream to send a message sequence. Both operate independently, so gRPC clients and gRPC servers can write and read in any order they like, i.e. the server can read a message then write a message alternatively, wait to receive all messages then write its responses, or perform reads and writes in any other combination.
CODE: https://gist.github.com/velotiotech/3e64bbe6b9e15c13feb31b2204f27ec0.js
**gRPC guarantees the ordering of messages within an individual RPC call. In the case of Bidirectional streaming, the order of messages is preserved in each stream.
Currently, gRPC provides support for many languages like Golang, C++, Java, etc. I will be focussing on its implementation using Python.
CODE: https://gist.github.com/velotiotech/bb3daedb9e213985122dde02190653ac.js
This will install all the required dependencies to implement gRPC.
Unary gRPC
For implementing gRPC services, we need to define three files:-
CODE: https://gist.github.com/velotiotech/28d88d9bbf29c86e0f548cb73eeaa965.js
In the above code, we have declared a service named Unary. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a MessageResponse. Below the service declaration, I have declared Message and Message Response.
Once we are done with the creation of the .proto file, we need to generate the stubs. For that, we will execute the below command:-
CODE: https://gist.github.com/velotiotech/bc5fbd828ba23019161c8fd25566f1da.js
Two files are generated named unary_pb2.py and unary_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and the client.
CODE: https://gist.github.com/velotiotech/3e6812a7277cc765dde2e4c77a707a67.js
In the gRPC server file, there is a GetServerResponse() method which takes `Message` from the client and returns a `MessageResponse` as defined in the proto file.
server() function is called from the main function, and makes sure that the server is listening to all the time. We will run the unary_server to start the server
CODE: https://gist.github.com/velotiotech/8d067e1d1ae747b03121255492bde7af.js
CODE: https://gist.github.com/velotiotech/75f6f2f53e722db2a7343c03782a74aa.js
In the __init__func. we have initialized the stub using ` self.stub = pb2_grpc.UnaryStub(self.channel)’ And we have a get_url function which calls to server using the above-initialized stub
This completes the implementation of Unary gRPC service.
Let's check the output:-
Run -> python3 unary_client.py
Output:-
message: "Hello Server you there?"
message: "Hello I am up and running. Received ‘Hello Server you there?’ message from you"
received: true
CODE: https://gist.github.com/velotiotech/bbabd8c23f18d1da0c480339de226eb7.js
In the above code, we have declared a service named Bidirectional. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a Message. Below the service declaration, I have declared Message.
Once we are done with the creation of the .proto file, we need to generate the stubs. To generate the stub, we need the execute the below command:-
CODE: https://gist.github.com/velotiotech/b33906ac7adb8a51311b58f952ff8cd8.js
Two files are generated named bidirectional_pb2.py and bidirectional_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and client.
CODE: https://gist.github.com/velotiotech/81b63c1a92f23b9c4478d09433a2f281.js
In the gRPC server file, there is a GetServerResponse() method which takes a stream of `Message` from the client and returns a stream of `Message` independent of each other. server() function is called from the main function and makes sure that the server is listening to all the time.
We will run the bidirectional_server to start the server:
CODE: https://gist.github.com/velotiotech/11e327c95e9fed1fb1be84357ee0566a.js
CODE: https://gist.github.com/velotiotech/ad7b026cad3b523de876cf131d52d4d2.js
In the run() function. we have initialised the stub using ` stub = bidirectional_pb2_grpc.BidirectionalStub(channel)’
And we have a send_message function to which the stub is passed and it makes multiple calls to the server and receives the results from the server simultaneously.
This completes the implementation of Bidirectional gRPC service.
Let's check the output:-
Run -> python3 bidirectional_client.py
Output:-
Hello Server Sending you the First message
Hello Server Sending you the Second message
Hello Server Sending you the Third message
Hello Server Sending you the Fourth message
Hello Server Sending you the Fifth message
Hello from the server received your First message
Hello from the server received your Second message
Hello from the server received your Third message
Hello from the server received your Fourth message
Hello from the server received your Fifth message
For code reference, please visit here.
gRPC is an emerging RPC framework that makes communication between microservices smooth and efficient. I believe gRPC is currently confined to inter microservice but has many other utilities that we will see in the coming years. To know more about modern data communication solutions, check out this blog.
Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.
We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.
In the last few years, we saw a great shift in technology, where projects are moving towards “microservice architecture” vs the old "monolithic architecture". This approach has done wonders for us.
As we say, “smaller things are much easier to handle”, so here we have microservices that can be handled conveniently. We need to interact among different microservices. I handled it using the HTTP API call, which seems great and it worked for me.
But is this the perfect way to do things?
The answer is a resounding, "no," because we compromised both speed and efficiency here.
Then came in the picture, the gRPC framework, that has been a game-changer.
Quoting the official documentation-
“gRPC or Google Remote Procedure Call is a modern open-source high-performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication.”
Credit: gRPC
RPC or remote procedure calls are the messages that the server sends to the remote system to get the task(or subroutines) done.
Google’s RPC is designed to facilitate smooth and efficient communication between the services. It can be utilized in different ways, such as:
- HTTP/2 based transport - It uses HTTP/2 protocol instead of HTTP 1.1. HTTP/2 protocol provides multiple benefits over the latter. One major benefit is multiple bidirectional streams that can be created and sent over TCP connections parallelly, making it swift.
- Auth, tracing, load balancing and health checking - gRPC provides all these features, making it a secure and reliable option to choose.
- Language independent communication- Two services may be written in different languages, say Python and Golang. gRPC ensures smooth communication between them.
- Use of Protocol Buffers - gRPC uses protocol buffers for defining the type of data (also called Interface Definition Language (IDL)) to be sent between the gRPC client and the gRPC server. It also uses it as the message interchange format.
Let's dig a little more into what are Protocol Buffers.
Protocol Buffers like XML, are an efficient and automated mechanism for serializing structured data. They provide a way to define the structure of data to be transmitted. Google says that protocol buffers are better than XML, as they are:
Protobuf are defined in .proto files. It is easy to define them.
1. Unary RPCs:- This is a simple gRPC which works like a normal function call. It sends a single request declared in the .proto file to the server and gets back a single response from the server.
CODE: https://gist.github.com/velotiotech/d2938c90ee7948186e7a3848f3558577.js
2. Server streaming RPCs:- The client sends a message declared in the .proto file to the server and gets back a stream of message sequence to read. The client reads from that stream of messages until there are no messages.
CODE: https://gist.github.com/velotiotech/0bdb7a50673c97745b37995a83f74ba3.js
3. Client streaming RPCs:- The client writes a message sequence using a write stream and sends the same to the server. After all the messages are sent to the server, the client waits for the server to read all the messages and return a response.
CODE: https://gist.github.com/velotiotech/757cef3a558b6ffbd38ff6eee37ab8ab.js
4. Bidirectional streaming RPCs:- Both gRPC client and the gRPC server use a read-write stream to send a message sequence. Both operate independently, so gRPC clients and gRPC servers can write and read in any order they like, i.e. the server can read a message then write a message alternatively, wait to receive all messages then write its responses, or perform reads and writes in any other combination.
CODE: https://gist.github.com/velotiotech/3e64bbe6b9e15c13feb31b2204f27ec0.js
**gRPC guarantees the ordering of messages within an individual RPC call. In the case of Bidirectional streaming, the order of messages is preserved in each stream.
Currently, gRPC provides support for many languages like Golang, C++, Java, etc. I will be focussing on its implementation using Python.
CODE: https://gist.github.com/velotiotech/bb3daedb9e213985122dde02190653ac.js
This will install all the required dependencies to implement gRPC.
Unary gRPC
For implementing gRPC services, we need to define three files:-
CODE: https://gist.github.com/velotiotech/28d88d9bbf29c86e0f548cb73eeaa965.js
In the above code, we have declared a service named Unary. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a MessageResponse. Below the service declaration, I have declared Message and Message Response.
Once we are done with the creation of the .proto file, we need to generate the stubs. For that, we will execute the below command:-
CODE: https://gist.github.com/velotiotech/bc5fbd828ba23019161c8fd25566f1da.js
Two files are generated named unary_pb2.py and unary_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and the client.
CODE: https://gist.github.com/velotiotech/3e6812a7277cc765dde2e4c77a707a67.js
In the gRPC server file, there is a GetServerResponse() method which takes `Message` from the client and returns a `MessageResponse` as defined in the proto file.
server() function is called from the main function, and makes sure that the server is listening to all the time. We will run the unary_server to start the server
CODE: https://gist.github.com/velotiotech/8d067e1d1ae747b03121255492bde7af.js
CODE: https://gist.github.com/velotiotech/75f6f2f53e722db2a7343c03782a74aa.js
In the __init__func. we have initialized the stub using ` self.stub = pb2_grpc.UnaryStub(self.channel)’ And we have a get_url function which calls to server using the above-initialized stub
This completes the implementation of Unary gRPC service.
Let's check the output:-
Run -> python3 unary_client.py
Output:-
message: "Hello Server you there?"
message: "Hello I am up and running. Received ‘Hello Server you there?’ message from you"
received: true
CODE: https://gist.github.com/velotiotech/bbabd8c23f18d1da0c480339de226eb7.js
In the above code, we have declared a service named Bidirectional. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a Message. Below the service declaration, I have declared Message.
Once we are done with the creation of the .proto file, we need to generate the stubs. To generate the stub, we need the execute the below command:-
CODE: https://gist.github.com/velotiotech/b33906ac7adb8a51311b58f952ff8cd8.js
Two files are generated named bidirectional_pb2.py and bidirectional_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and client.
CODE: https://gist.github.com/velotiotech/81b63c1a92f23b9c4478d09433a2f281.js
In the gRPC server file, there is a GetServerResponse() method which takes a stream of `Message` from the client and returns a stream of `Message` independent of each other. server() function is called from the main function and makes sure that the server is listening to all the time.
We will run the bidirectional_server to start the server:
CODE: https://gist.github.com/velotiotech/11e327c95e9fed1fb1be84357ee0566a.js
CODE: https://gist.github.com/velotiotech/ad7b026cad3b523de876cf131d52d4d2.js
In the run() function. we have initialised the stub using ` stub = bidirectional_pb2_grpc.BidirectionalStub(channel)’
And we have a send_message function to which the stub is passed and it makes multiple calls to the server and receives the results from the server simultaneously.
This completes the implementation of Bidirectional gRPC service.
Let's check the output:-
Run -> python3 bidirectional_client.py
Output:-
Hello Server Sending you the First message
Hello Server Sending you the Second message
Hello Server Sending you the Third message
Hello Server Sending you the Fourth message
Hello Server Sending you the Fifth message
Hello from the server received your First message
Hello from the server received your Second message
Hello from the server received your Third message
Hello from the server received your Fourth message
Hello from the server received your Fifth message
For code reference, please visit here.
gRPC is an emerging RPC framework that makes communication between microservices smooth and efficient. I believe gRPC is currently confined to inter microservice but has many other utilities that we will see in the coming years. To know more about modern data communication solutions, check out this blog.
.svg)
.svg)
