What causes pods to be evicted in ITRS Analytics?

Issue Copied

In the ITRS Analytics Ingestion app, some pods may not appear as running under Ingestion > Workloads.

To confirm, open the ITRS Analytics debug shell and list pods:

sudo ./itrs-analytics shell
kubectl get pods -A

You may see pods in Evicted status.

ITRS Analytics runs on Kubernetes. When a node is under resource pressure, Kubernetes may evict pods to reclaim memory, disk space, or inodes and prevent starvation.

Resolution Copied

Review the eviction thresholds and available resources on the nodes where ITRS Analytics is installed.

1. Start the Kubernetes API proxy:

   kubectl proxy
   

2. In another terminal, get the node name and fetch the kubelet eviction configuration. Replace <my_node_name> with your node name:

   kubectl get nodes
   curl -s -X GET http://127.0.0.1:8001/api/v1/nodes/<my_node_name>/proxy/configz | jq .kubeletconfig.evictionHard
   

   The command returns the hard eviction thresholds in JSON format:

   

3. Compare the thresholds with the resources available on your nodes. Disk space limits are easy to overlook and are a common cause of eviction.

Ensure your nodes meet the ITRS Analytics resource and hardware requirements. For more detail on how Kubernetes evicts pods under node pressure, see Node-pressure eviction.