We use cookies to ensure that we give you the best experience on our website.

gRPC – Load balancing with HAProxy in docker

Define protobuff file

Protocol buffer file is the base template for the communication between the server and the client.

syntax = "proto3";
package greeting;
option java_package = "com.thebytecloud.greeting";
option java_multiple_files = true;
message HelloRequest {
    string name = 1;
}
message HelloResponse {
    string response = 1;
}
service GreetingService {
    rpc sayHello(HelloRequest) returns (HelloResponse);
}

Generating the java classes

Running below command in the terminal will generate java files in package com.thebytecloud.greeting (option java_package)

mvn clean generate-sources

Implementing protobuff service methods

package com.thebytecloud.server;
import com.thebytecloud.greeting.GreetingServiceGrpc;
import com.thebytecloud.greeting.HelloRequest;
import com.thebytecloud.greeting.HelloResponse;
import io.grpc.stub.StreamObserver;
public class GreetingServiceImpl extends GreetingServiceGrpc.GreetingServiceImplBase {
    @Override
    public void sayHello(HelloRequest request, StreamObserver<HelloResponse> responseObserver) {
        final HelloResponse response = HelloResponse.newBuilder()
                .setResponse("Hello " + request.getName() + ", from " + System.getenv("HOSTNAME"))
                .build();
        responseObserver.onNext(response);
        responseObserver.onCompleted();
    }
}

Register GreetingServiceImpl with server

GreetingServiceImpl should be added in the server builder.

final Server server = ServerBuilder.forPort(serverBindPort)
                .addService(new CalculatorServiceImpl())
                .addService(new GreetingServiceImpl())
                .build();

Client Implementation

package com.thebytecloud.client;
import com.thebytecloud.calculator.*;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.grpc.StatusRuntimeException;
import io.grpc.stub.StreamObserver;
import java.util.Arrays;
import java.util.Iterator;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
public class CalculatorClient {
    private final ManagedChannel managedChannel;
    public CalculatorClient(ManagedChannel managedChannel) {
        this.managedChannel = managedChannel;
    }
    public static void main(String[] args) throws InterruptedException {
        String server = "localhost";
        int serverPort = 7070;
        if(System.getenv("SERVER_PORT") != null)
            serverPort = Integer.parseInt(System.getenv("SERVER_PORT"));
        if(System.getenv("SERVER") != null)
            server = System.getenv("SERVER");
        System.out.println("server = " + server+":"+serverPort);
        Thread.sleep(2000);
        ManagedChannel managedChannel = ManagedChannelBuilder.forAddress(server, serverPort)
                .usePlaintext()
                .build();
        CalculatorClient calculatorClient = new CalculatorClient(managedChannel);
        calculatorClient.sayHello();
    }
    private void sayHello() throws InterruptedException {
        final GreetingServiceGrpc.GreetingServiceBlockingStub
                blockingStub = GreetingServiceGrpc.newBlockingStub(managedChannel);
        final HelloRequest request = HelloRequest.newBuilder().setName("Nantha").build();
        while (true){
            HelloResponse response = blockingStub.sayHello(request);
            System.out.println("response = " + response.getResponse());
            Thread.sleep(2000);
        }
    }
}

Executing server and client

Executing server and client can be done via IDE or command line. Following are the commands to execute in terminal.

gRPC Server

grpc-java-examples$ mvn clean install
grpc-java-examples$ java -cp target/grpc-java-examples-1.0-SNAPSHOT-jar-with-dependencies.jar com.thebytecloud.server.CalculatorServer
serverBindPort = 7070
Starting gRPC Server...!

gRPC Client

grpc-java-examples$ java -cp target/grpc-java-examples-1.0-SNAPSHOT-jar-with-dependencies.jar com.thebytecloud.client.CalculatorClient
server = localhost:7070
response = Hello Nantha, from <hostname>
response = Hello Nantha, from <hostname>
response = Hello Nantha, from <hostname>

Here client is sending name and server returned with hostname. With this we can identify which server returns the response. In detail will look in next post.

Docker

Docker is a container platform which allows to run in a server environment. Once converting to docker, we are going to test load balancing with haproxy.

Create client and server docker file in the main directory grpc-java-examples

Client Docker file

FROM openjdk:8-jdk-alpine
VOLUME /tmp
ADD target/grpc-java-examples-1.0-SNAPSHOT-jar-with-dependencies.jar grpc-docker-app.jar
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -cp /grpc-docker-app.jar com.thebytecloud.client.CalculatorClient" ]

Server Docker file

FROM openjdk:8-jdk-alpine
VOLUME /tmp
ADD target/grpc-java-examples-1.0-SNAPSHOT-jar-with-dependencies.jar grpc-docker-app.jar
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -cp /grpc-docker-app.jar com.thebytecloud.server.CalculatorServer" ]

HAProxy configuration

Create haproxy.cfg in the path grpc-java-examples/haproxy.

global
  tune.ssl.default-dh-param 1024
  log stdout local0
  debug
defaults
    log global
    maxconn 3000
    mode http
    timeout connect 10s
    timeout client 30s
    timeout server 30s
    option httplog
    option http-use-htx
    option logasap
listen stats    #Listen on all IP's on port 9000
    bind *:9000
    mode http
    stats enable    #Enable stats page
    #stats hide-version  # Hide HAProxy version
    #This is the virtual URL to access the stats page
    stats uri /stats
    stats realm HAProxy\ Statistics      # Title text for popup window
    #The user/pass you want to use. Change this password!
    stats auth admin:admin
    #This allows you to take down and bring up back end servers.
    #This will produce an error on older versions of HAProxy.
    stats admin if TRUE
frontend fe_http
  bind *:8000 proto h2
  mode http
  default_backend be_grpc
  # Redirect to https
  #redirect scheme https code 301
#frontend fe_https
#  mode tcp
#  bind *:8443 npn spdy/2 alpn h2,http/1.1  #when negotiating with multiple protocols
#  default_backend be_grpc
# gRPC servers running on port 7070-7171
backend be_grpc
  balance roundrobin
  server srv01 grpc-server1:7070 check proto h2
  server srv02 grpc-server2:7171 check proto h2   #when there is only one protocol   proto h2

Docker compose file

Here with docker-compose.yml file, we will create separate network with two instance of grpc server, one instance of grpc-client and one instance of haproxy latest.

compose file itself a self explanatory with ip address of the instances.

version: '3.1'
networks:
  thebytecloud:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/24
services:
  haproxy:
    image: haproxy
    container_name: haproxy
    depends_on:
      - grpc-server1
      - grpc-server2
    volumes:
      - ./haproxy:/usr/local/etc/haproxy:ro
    ports:
      - "8000:8000"
      - "9000:9000"
    networks:
      thebytecloud:
        ipv4_address: 172.20.0.2
  grpc-server1:
    build:
      context: .
      dockerfile: server.docker
    image: thebytecloud/grpc-server:latest
    container_name: grpc-server1
    ports:
      - "7070:7070"
    environment:
      - BIND_PORT=7070
    networks:
      thebytecloud:
        ipv4_address: 172.20.0.3
  grpc-server2:
    build:
      context: .
      dockerfile: server.docker
    image: thebytecloud/grpc-server:latest
    container_name: grpc-server2
    ports:
      - "7171:7171"
    environment:
      - BIND_PORT=7171
    networks:
      thebytecloud:
        ipv4_address: 172.20.0.4
  grpc-client:
    build:
      context: .
      dockerfile: client.docker
    image: thebytecloud/grpc-client:latest
    container_name: grpc-client
    depends_on:
      - haproxy
    environment:
      - SERVER=172.20.0.2
      - SERVER_PORT=8000
    networks:
      thebytecloud:
        ipv4_address: 172.20.0.11

Running Docker containers

grpc-java-examples$ mvn clean install
.........
........
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.639 s
[INFO] Finished at: 2019-10-10T16:41:44+05:30
[INFO] Final Memory: 29M/447M
[INFO] ------------------------------------------------------------------------
grpc-java-examples$ docker-compose up
WARNING: The Docker Engine you're using is running in swarm mode.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
Starting grpc-server2 ... done
Starting grpc-server1 ... done
Starting haproxy      ... done
Starting grpc-client  ... done
Attaching to grpc-server1, grpc-server2, haproxy, grpc-client
grpc-server1    | serverBindPort = 7070
grpc-server1    | Stating gRPC Server...!
grpc-server2    | serverBindPort = 7171
grpc-server2    | Stating gRPC Server...!
haproxy         | Note: setting global.maxconn to 524269.
haproxy         | Available polling systems :
haproxy         |       epoll : pref=300,  test result OK
haproxy         |        poll : pref=200,  test result OK
haproxy         |      select : pref=150,  test result FAILED
haproxy         | Total: 3 (2 usable), will use epoll.
haproxy         | 
haproxy         | Available filters :
haproxy         | 	[SPOE] spoe
haproxy         | 	[COMP] compression
haproxy         | 	[CACHE] cache
haproxy         | 	[TRACE] trace
haproxy         | Using epoll() as the polling mechanism.
haproxy         | <133>Oct 10 11:12:15 haproxy[1]: Proxy stats started.
haproxy         | <133>Oct 10 11:12:15 haproxy[1]: Proxy fe_http started.
haproxy         | <133>Oct 10 11:12:15 haproxy[1]: Proxy be_grpc started.
haproxy         | [NOTICE] 282/111215 (1) : New worker #1 (6) forked
haproxy         | <129>Oct 10 11:12:15 haproxy[6]: Server be_grpc/srv01 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy         | [WARNING] 282/111215 (6) : Server be_grpc/srv01 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy         | [WARNING] 282/111216 (6) : Server be_grpc/srv02 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy         | [ALERT] 282/111216 (6) : backend 'be_grpc' has no server available!
haproxy         | <129>Oct 10 11:12:16 haproxy[6]: Server be_grpc/srv02 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy         | <128>Oct 10 11:12:16 haproxy[6]: backend be_grpc has no server available!
grpc-client     | server = 172.20.0.2:8000
haproxy         | [WARNING] 282/111220 (6) : Server be_grpc/srv02 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
haproxy         | <133>Oct 10 11:12:20 haproxy[6]: Server be_grpc/srv02 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
haproxy         | 00000000:fe_http.accept(0007)=0013 from [172.20.0.11:55118] ALPN=<none>
haproxy         | 00000000:fe_http.clireq[0013:ffffffff]: POST /greeting.GreetingService/sayHello HTTP/2.0
haproxy         | 00000000:fe_http.clihdr[0013:ffffffff]: content-type: application/grpc
haproxy         | 00000000:fe_http.clihdr[0013:ffffffff]: te: trailers
haproxy         | 00000000:fe_http.clihdr[0013:ffffffff]: user-agent: grpc-java-netty/1.15.1
haproxy         | 00000000:fe_http.clihdr[0013:ffffffff]: grpc-accept-encoding: gzip
haproxy         | 00000000:fe_http.clihdr[0013:ffffffff]: grpc-trace-bin: 
haproxy         | 00000000:fe_http.clihdr[0013:ffffffff]: host: 172.20.0.2:8000
haproxy         | [WARNING] 282/111221 (6) : Server be_grpc/srv01 is UP, reason: Layer4 check passed, check duration: 0ms. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
haproxy         | <133>Oct 10 11:12:21 haproxy[6]: Server be_grpc/srv01 is UP, reason: Layer4 check passed, check duration: 0ms. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
haproxy         | 00000000:be_grpc.srvrep[0013:0014]: HTTP/2.0 200 
haproxy         | 00000000:be_grpc.srvhdr[0013:0014]: content-type: application/grpc
haproxy         | 00000000:be_grpc.srvhdr[0013:0014]: grpc-encoding: identity
haproxy         | 00000000:be_grpc.srvhdr[0013:0014]: grpc-accept-encoding: gzip
haproxy         | <134>Oct 10 11:12:21 haproxy[6]: 172.20.0.11:55118 [10/Oct/2019:11:12:21.425] fe_http be_grpc/srv02 0/0/129/127/+256 200 +109 - - ---- 1/1/1/1/0 0/0 "POST /greeting.GreetingService/sayHello HTTP/2.0"
haproxy         | 00000000:be_grpc.srvcls[0013:0014]
haproxy         | 00000000:be_grpc.clicls[0013:0014]
haproxy         | 00000000:be_grpc.closed[0013:0014]
grpc-client     | response = Hello Nantha, from 41fc078a5bb3
haproxy         | 00000001:fe_http.accept(0007)=0013 from [172.20.0.11:55118] ALPN=<none>
haproxy         | 00000001:fe_http.clireq[0013:ffffffff]: POST /greeting.GreetingService/sayHello HTTP/2.0
haproxy         | 00000001:fe_http.clihdr[0013:ffffffff]: content-type: application/grpc
haproxy         | 00000001:fe_http.clihdr[0013:ffffffff]: te: trailers
haproxy         | 00000001:fe_http.clihdr[0013:ffffffff]: user-agent: grpc-java-netty/1.15.1
haproxy         | 00000001:fe_http.clihdr[0013:ffffffff]: grpc-accept-encoding: gzip
haproxy         | 00000001:fe_http.clihdr[0013:ffffffff]: grpc-trace-bin: 
haproxy         | 00000001:fe_http.clihdr[0013:ffffffff]: host: 172.20.0.2:8000
haproxy         | 00000001:be_grpc.srvrep[0013:0014]: HTTP/2.0 200 
haproxy         | 00000001:be_grpc.srvhdr[0013:0014]: content-type: application/grpc
haproxy         | 00000001:be_grpc.srvhdr[0013:0014]: grpc-encoding: identity
haproxy         | 00000001:be_grpc.srvhdr[0013:0014]: grpc-accept-encoding: gzip
haproxy         | <134>Oct 10 11:12:23 haproxy[6]: 172.20.0.11:55118 [10/Oct/2019:11:12:23.710] fe_http be_grpc/srv02 0/0/0/7/+7 200 +161 - - ---- 1/1/1/0/0 0/0 "POST /greeting.GreetingService/sayHello HTTP/2.0"
haproxy         | 00000001:be_grpc.srvcls[0013:0014]
haproxy         | 00000001:be_grpc.clicls[0013:0014]
haproxy         | 00000001:be_grpc.closed[0013:0014]
grpc-client     | response = Hello Nantha, from 41fc078a5bb3
haproxy         | 00000002:fe_http.accept(0007)=0013 from [172.20.0.11:55118] ALPN=<none>
haproxy         | 00000002:fe_http.clireq[0013:ffffffff]: POST /greeting.GreetingService/sayHello HTTP/2.0
haproxy         | 00000002:fe_http.clihdr[0013:ffffffff]: content-type: application/grpc
haproxy         | 00000002:fe_http.clihdr[0013:ffffffff]: te: trailers
haproxy         | 00000002:fe_http.clihdr[0013:ffffffff]: user-agent: grpc-java-netty/1.15.1
haproxy         | 00000002:fe_http.clihdr[0013:ffffffff]: grpc-accept-encoding: gzip
haproxy         | 00000002:fe_http.clihdr[0013:ffffffff]: grpc-trace-bin: 
haproxy         | 00000002:fe_http.clihdr[0013:ffffffff]: host: 172.20.0.2:8000
haproxy         | 00000002:be_grpc.srvrep[0013:0015]: HTTP/2.0 200 
haproxy         | 00000002:be_grpc.srvhdr[0013:0015]: content-type: application/grpc
haproxy         | 00000002:be_grpc.srvhdr[0013:0015]: grpc-encoding: identity
haproxy         | 00000002:be_grpc.srvhdr[0013:0015]: grpc-accept-encoding: gzip
haproxy         | <134>Oct 10 11:12:25 haproxy[6]: 172.20.0.11:55118 [10/Oct/2019:11:12:25.722] fe_http be_grpc/srv01 0/0/126/82/+208 200 +109 - - ---- 1/1/1/1/0 0/0 "POST /greeting.GreetingService/sayHello HTTP/2.0"
haproxy         | 00000002:be_grpc.srvcls[0013:0015]
haproxy         | 00000002:be_grpc.clicls[0013:0015]
haproxy         | 00000002:be_grpc.closed[0013:0015]
grpc-client     | response = Hello Nantha, from 5ca8611ed2fc
haproxy         | 00000003:fe_http.accept(0007)=0013 from [172.20.0.11:55118] ALPN=<none>
haproxy         | 00000003:fe_http.clireq[0013:ffffffff]: POST /greeting.GreetingService/sayHello HTTP/2.0
haproxy         | 00000003:fe_http.clihdr[0013:ffffffff]: content-type: application/grpc
haproxy         | 00000003:fe_http.clihdr[0013:ffffffff]: te: trailers
haproxy         | 00000003:fe_http.clihdr[0013:ffffffff]: user-agent: grpc-java-netty/1.15.1
haproxy         | 00000003:fe_http.clihdr[0013:ffffffff]: grpc-accept-encoding: gzip
haproxy         | 00000003:fe_http.clihdr[0013:ffffffff]: grpc-trace-bin: 
haproxy         | 00000003:fe_http.clihdr[0013:ffffffff]: host: 172.20.0.2:8000
haproxy         | 00000003:be_grpc.srvrep[0013:0014]: HTTP/2.0 200 
haproxy         | 00000003:be_grpc.srvhdr[0013:0014]: content-type: application/grpc
haproxy         | 00000003:be_grpc.srvhdr[0013:0014]: grpc-encoding: identity
haproxy         | 00000003:be_grpc.srvhdr[0013:0014]: grpc-accept-encoding: gzip
haproxy         | <134>Oct 10 11:12:27 haproxy[6]: 172.20.0.11:55118 [10/Oct/2019:11:12:27.981] fe_http be_grpc/srv02 0/0/0/3/+3 200 +161 - - ---- 1/1/1/0/0 0/0 "POST /greeting.GreetingService/sayHello HTTP/2.0"
haproxy         | 00000003:be_grpc.srvcls[0013:0014]
haproxy         | 00000003:be_grpc.clicls[0013:0014]
haproxy         | 00000003:be_grpc.closed[0013:0014]
grpc-client     | response = Hello Nantha, from 41fc078a5bb3

grpc-client received response from both container.

Share your love
Nanthakumar
Nanthakumar

I’m a curious engineer, interested in various aspects of software engineering.

Articles: 7

Related Posts

Home Scroll Top