Connecting to Hazelcast from Outside Kubernetes

latest-snapshot 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5

Hazelcast supports connections from both smart and unisocket clients that are outside your Kubernetes cluster. To allow these clients to connect to Hazelcast, you need to configure the `exposeExternally` field of the Hazelcast custom resource.

For a working example, see this tutorial.

Environmental Constraints

The kubectl command-line tool must be configured to communicate with your cluster.

Google Kubernetes Engine (GKE)

If you use GKE, consider the following:

  • Make sure the Kubernetes cluster is public.

  • If you plan on using NodePorts, (discoveryServiceType: NodePort or memberAccess: NodePortExternalIP), set up a firewall rule for each Hazelcast member to allow TCP traffic on node ports.

Amazon Elastic Kubernetes Service (EKS)

If you use EKS, consider the following:

  • Having the VPC of the EKS cluster configured only with private subnets is not recommended.

  • If you plan on using NodePorts, (discoveryServiceType: NodePort or memberAccess: NodePortExternalIP), the VPC of the EKS cluster should be configured only with public subnets. Additionally, inbound rules for the node ports should be added to the nodes' security groups.

Azure Kubernetes Service (AKS)

If you use AKS, consider the following:

  • Make sure the Kubernetes cluster is public.

  • If you plan on using NodePorts, (discoveryServiceType: NodePort or memberAccess: NodePortExternalIP), the node pools should be created by using the --enable-node-public-ip flag. Additionally, the required inbound rules should be added for network security groups (NSGs) to allow access to node IPs and ports.

Configuring exposeExternally

To allow connections from clients, you need to configure the exposeExternally field, of the Hazelcast custom resource.

If using Istio, you cannot use exposeExternally.

This field has the following properties:

Property Description

type

The way in which Hazelcast members are exposed to clients:

  • Unisocket: Members are exposed through one Kubernetes service that automatically load balances the traffic to Hazelcast members. Use this property if your clients use the unisocket operation mode.

  • Smart: Each member is exposed through a separate Kubernetes service that allows clients to send requests directly to the members that own a particular partition. Use this property if your clients use the smart operation mode.

discoveryServiceType

Type of Kubernetes service used for discovering members in a Hazelcast cluster:

  • LoadBalancer (default): Hazelcast client configuration <load-balancer-external-ip>:5701.

    Your Kubernetes cluster should be able to assign public IPs to LoadBalancer services.

  • NodePort: Hazelcast client configuration <node-address>:<node-port>.

memberAccess

The way in which clients access Hazelcast members:

  • LoadBalancer: Hazelcast clients are outside the Kubernetes network and can access members, using the external IP that is assigned to the LoadBalancer service.

  • NodePortExternalIP (default): Hazelcast clients are outside the Kubernetes network, so a client can access members only via the public IP address that’s assigned to their Kubernetes nodes.

  • NodePortNodeName: Hazelcast clients are inside the same network as the Kubernetes nodes, so clients can access a member by the hostname of its Kubernetes node.
If NodePorts are used (discoveryServiceType: NodePort or memberAccess: NodePortExternalIP), the Kubernetes nodes must allow external TCP traffic on node IPs and ports.

Exposing Hazelcast to Smart Clients

Smart clients access partitioned data in a cluster by communicating with the partition owner. This operation mode increases the overall throughput and efficiency of client-cluster communication.

For non-partitioned data, smart clients can communicate with any cluster member.

To support smart clients, you should configure the type property as smart.

In this configuration, the standard Kubernetes mechanism routes client traffic to the correct member. Or, if the client is not accessing partitioned data, the traffic is automatically load balanced.

Exposing Hazelcast to Unisocket Clients

Unisocket clients communicate with a single member in the cluster. To enable these clients to connect to the Hazelcast cluster, you should configure the type property as unisocket.

In this configuration, the standard Kubernetes mechanism automatically load balances client traffic across the Hazelcast cluster.

Example Configuration

In this example, the Hazelcast cluster is configured to load balance connections from smart clients.

Example configuration
apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast-sample
spec:
  clusterSize: 3
  repository: 'docker.io/hazelcast/hazelcast-enterprise'
  version: '5.5.2-slim'
  licenseKeySecretName: hazelcast-license-key
  exposeExternally:
    type: Smart
    discoveryServiceType: LoadBalancer
    memberAccess: NodePortExternalIP

This configuration creates a load balancer for the discovery service and one service for each member.

kubectl get service
NAME                 TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
hazelcast-sample     LoadBalancer   10.219.246.19    35.230.92.217   5701:30560/TCP   2m11s (1)
hazelcast-sample-0   NodePort       10.219.254.192   <none>          5701:31890/TCP   2m11s (2)
hazelcast-sample-1   NodePort       10.219.247.43    <none>          5701:32310/TCP   2m11s (2)
hazelcast-sample-2   NodePort       10.219.243.16    <none>          5701:32585/TCP   2m11s (2)
1 Discovery service
2 Member access service

The command below allows you to list the external addresses of the cluster.

kubectl get hazelcastendpoint --selector="app.kubernetes.io/instance=hazelcast-sample"
NAME                   TYPE        ADDRESS
hazelcast-sample       Discovery   35.238.20.134:5701
hazelcast-sample-0     Member      34.122.120.18:30108
hazelcast-sample-1     Member      34.27.85.200:32012
hazelcast-sample-2     Member      34.173.81.209:30512
hazelcast-sample-wan   WAN         35.238.20.134:5710

Smart clients can use the external IP of the load balancer to connect to the cluster.

  • Java

  • Node.js

  • Go

  • Python

ClientConfig config = new ClientConfig();
config.getNetworkConfig().addAddress("35.230.92.217");
config.getProperties().setProperty(ClientProperty.DISCOVERY_SPI_PUBLIC_IP_ENABLED.toString(), "true");
HazelcastInstance client = HazelcastClient.newHazelcastClient(config);
const { Client } = require('hazelcast-client');

const clientConfig = {
  network: {
    clusterMembers: [
      '35.230.92.217'
    ]
  },
  properties: {
    ['hazelcast.discovery.public.ip.enabled']: true
  }
};
const client = await Client.newHazelcastClient(clientConfig);
import (
	"context"

	"github.com/hazelcast/hazelcast-go-client"
)

func main() {
	config := hazelcast.Config{}
	cc := &config.Cluster
	cc.Network.SetAddresses("35.230.92.217")
	cc.Discovery.UsePublicIP = true
	ctx := context.TODO()
	client, err := hazelcast.StartNewClientWithConfig(ctx, config)
	if err != nil {
		panic(err)
	}
}
import logging
import hazelcast

logging.basicConfig(level=logging.INFO)

client = hazelcast.HazelcastClient(
    cluster_members=["35.230.92.217"],
    use_public_ip=True,
)
lifebuoy

Help and support