Multi-node clusters with Embedded Cluster air-gapped

This guide walks you through setting up a multi-node cluster using the Embedded Cluster air-gapped method.

Configure multi-node clusters Copied

The sample setup below shows a Small High Availability (HA) deployment with a cluster configured with three controller nodes.

Here’s a summary checklist of the steps you need to complete:

• Set node labels and taints for timescale nodes, if applicable.
• Create and initialize the first node.
• Download and extract ITRS Analytics bundle on each node.
• Create all additional nodes in the same network.
• Execute join command on each node.
• Enable HA (after setup of 3 controllers).

Configure node labels and taints for timescale Copied

For Medium and Large-sized deployments, you must assign the appropriate labels and taints to nodes designated for timescale to ensure proper scheduling and workload isolation.

For label command:

kubectl label nodes <node_name> instancegroup=timescale-nodes;

For taint command:

kubectl taint nodes <node_name> dedicated=timescale-nodes:NoSchedule;

Set up the main node Copied

1. Create your first node for the Embedded Cluster. Disk size depends on your deployment type. Refer to the t-shirt sizing guide for appropriate specifications.

2. Use the following command to download and extract the ITRS Analytics bundle.

   curl -f "https://app.itrsgroup.com/embedded/itrs-analytics/stable" -H "Authorization: <insert auth token>" -o itrs-analytics-stable.tgz
   tar -zxvf itrs-analytics-stable.tgz
   

3. Install ITRS Analytics on the first node.

   sudo ./itrs-analytics install --license license.yaml --airgap-bundle itrs-analytics.airgap
   

Configure multi-node Copied

In this step, you need to add the additional nodes to the same network using the first node. Make sure all nodes are connected to the same network.

1. In the KOTS Admin Console, go to the Configure the Cluster page and copy the command for adding worker or timescale nodes. Take note of the role token in the screenshot below as an example; it changes depending on the roles you select.

   

2. Download and extract the ITRS Analytics bundle (itrs-analytics-stable.tgz) and run the join command shown in the example below:

   sudo ./itrs-analytics join --airgap-bundle itrs-analytics.airgap 172.31.21.149:30000 <insert role token>
   

   The new node should appear once the process is complete. Repeat the process for the remaining nodes.

   

   Note

   Wait for each new node to become ready. Please note that only one node can be joined at a time. For every node you add, make sure the primary node is accessible.

3. Once three controllers are set up, HA can be enabled. See the example message below for reference:

   <user>@ip-172-31-17-41:~/iax$ sudo ./itrs-analytics join --airgap-bundle itrs-analytics.airgap 172.31.21.149:30000 <insert role token>
   Do not join another node until this join is complete.
   ✔  Host files materialized!
   ✔  Host preflights succeeded!
   ✔  Node installation finished!
   ✔  Node has joined the cluster!
   
   You can enable high availability for clusters with three or more controller nodes.
   Data will be migrated so it is replicated across cluster nodes.
   When high availability is enabled, you must maintain at least three controller nodes.
   
   ? Do you want to enable high availability? Yes
   
   ✔  High availability enabled!
   

4. Perform a verification of the multi-node setup.

   

Add new nodes after deployment Copied

You can add new nodes after deployment by navigating to the Cluster Management tab in the admin console.

Preflight check for multi-node HA clusters Copied

When you choose to install on a multi-node cluster with High Availability (HA) sizes: smallha, medium, or large, a preflight check is automatically enabled during installation.

This preflight check ensures that the cluster has sufficient resources to support ITRS Analytics running in a degraded state, should one of the nodes become unavailable. By allowing ITRS Analytics to run in a degraded state, you gain the opportunity to restore the cluster capacity lost during downtime, helping ensure that ITRS Analytics can return to normal operation.

• If the cluster nodes have uniform resources (for example, identical CPU and memory configurations), the preflight will enforce a strict validation. The installation will fail if the minimum requirements to run in a degraded state are not met.
• If the nodes have non-uniform resources, the preflight will still run, but instead of blocking the installation, it will issue a warning if the degraded state cannot be safely supported. Note that this preflight check will not appear if you specify custom Helm values overrides during installation.

Deployment scenarios Copied

For a correctly provisioned HA cluster, the preflight check will pass:

For smallha installation: 3 even nodes, no mTLS, no custom parameters:

• The HA preflight check issued a warning, as expected.
• Resources allocated for the HA preflight were higher than those for the regular resource request preflights.
• This confirms that the resource cushion values were accurately applied.
• The warning is due to the uneven node resource allocation, not a failure in capacity.

When all nodes are configured with identical CPU and memory, the HA preflight check runs in strict mode and fails if degraded state tolerances are not met: