# Kubernetes cluster state by HTTP ## Overview The template to monitor Kubernetes state that work without any external scripts. It works without external scripts and uses the script item to make HTTP requests to the Kubernetes API. Template `Kubernetes cluster state by HTTP` - collects metrics by HTTP agent from kube-state-metrics endpoint and Kubernetes API. Don't forget change macros {$KUBE.API.URL} and {$KUBE.API.TOKEN}. Also, see the Macros section for a list of macros used to set trigger values. *NOTE.* Some metrics may not be collected depending on your Kubernetes version and configuration. ## Requirements Zabbix version: 7.0 and higher. ## Tested versions This template has been tested on: - Kubernetes 1.19.10 ## Configuration > Zabbix should be configured according to the instructions in the [Templates out of the box](https://www.zabbix.com/documentation/7.0/manual/config/templates_out_of_the_box) section. ## Setup Install the [Zabbix Helm Chart](https://git.zabbix.com/projects/ZT/repos/kubernetes-helm/browse?at=refs%2Fheads%2Frelease%2F7.0) in your Kubernetes cluster. Internal service metrics are collected from kube-state-metrics endpoint. Template needs to use Authorization via API token. Set the `{$KUBE.API.URL}` such as `://:`. Get the generated service account token using the command: `kubectl get secret zabbix-service-account -n monitoring -o jsonpath={.data.token} | base64 -d` Then set it to the macro `{$KUBE.API.TOKEN}`. Set `{$KUBE.STATE.ENDPOINT.NAME}` with Kube state metrics endpoint name. See `kubectl -n monitoring get ep`. Default: `zabbix-kube-state-metrics`. *NOTE.* If you wish to monitor Controller Manager and Scheduler components, you might need to set the `--binding-address` option for them to the address where Zabbix proxy can reach them. For example, for clusters created with `kubeadm` it can be set in the following manifest files (changes will be applied immediately): - /etc/kubernetes/manifests/kube-controller-manager.yaml - /etc/kubernetes/manifests/kube-scheduler.yaml Depending on your Kubernetes distribution, you might need to adjust `{$KUBE.CONTROL_PLANE.TAINT}` macro (for example, set it to `node-role.kubernetes.io/master` for OpenShift). *NOTE.* Some metrics may not be collected depending on your Kubernetes version and configuration. Also, see the Macros section for a list of macros used to set trigger values. Set up the macros to filter the metrics of discovered Kubelets by node names: - {$KUBE.LLD.FILTER.KUBELET_NODE.MATCHES} - {$KUBE.LLD.FILTER.KUBELET_NODE.NOT_MATCHES} Set up macros to filter metrics by namespace: - {$KUBE.LLD.FILTER.NAMESPACE.MATCHES} - {$KUBE.LLD.FILTER.NAMESPACE.NOT_MATCHES} Set up macros to filter node metrics by nodename: - {$KUBE.LLD.FILTER.NODE.MATCHES} - {$KUBE.LLD.FILTER.NODE.NOT_MATCHES} **Note**, If you have a large cluster, it is highly recommended to set a filter for discoverable namespaces. You can use the `{$KUBE.KUBELET.FILTER.LABELS}` and `{$KUBE.KUBELET.FILTER.ANNOTATIONS}` macros for advanced filtering of Kubelets by node labels and annotations. Notes about labels and annotations filters: - Macro values should be specified separated by commas and must have the key/value form with support for regular expressions in the value (`key1: value, key2: regexp`). - ECMAScript syntax is used for regular expressions. - Filters are applied if such a label key exists for the entity that is being filtered (it means that if you specify a key in a filter, entities which do not have this key will not be affected by the filter and will still be discovered, and only entities containing that key will be filtered by the value). - You can also use the exclamation point symbol (`!`) to invert the filter (`!key: value`). For example: `kubernetes.io/hostname: kubernetes-node[5-25], !node-role.kubernetes.io/ingress: .*`. As a result, the Kubelets on nodes 5-25 without the "ingress" role will be discovered. See the Kubernetes documentation for details about labels and annotations: - - You can also set up evaluation periods for replica mismatch triggers (Deployments, ReplicaSets, StatefulSets) with the macro `{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD}`, which supports context and regular expressions. For example, you can create the following macros: - Set the evaluation period for the Deployment "nginx-deployment" in the namespace "default" to the 3 last values: `{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD:"deployment:default:nginx-deployment"} = #3` - Set the evaluation period for all Deployments to the 10 last values: `{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD:regex:"deployment:.*:.*"} = #10` or `{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD:regex:"^deployment.*"} = #10` - Set the evaluation period for Deployments, ReplicaSets and StatefulSets in the namespace "default" to 15 minutes: `{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD:regex:".*:default:.*"} = 15m` **Note**, that different context macros with regular expressions matching the same string can be applied in an undefined order and simple context macros (without regular expressions) have higher priority. Read the **Important notes** section in [`Zabbix documentation`](https://www.zabbix.com/documentation/7.0/manual/config/macros/user_macros_context) for details. ### Macros used |Name|Description|Default| |----|-----------|-------| |{$KUBE.API.URL}|

Kubernetes API endpoint URL in the format ://:

|`https://kubernetes.default.svc.cluster.local:443`| |{$KUBE.API.READYZ.ENDPOINT}|

Kubernetes API readyz endpoint /readyz

|`/readyz`| |{$KUBE.API.LIVEZ.ENDPOINT}|

Kubernetes API livez endpoint /livez

|`/livez`| |{$KUBE.API.COMPONENTSTATUSES.ENDPOINT}|

Kubernetes API componentstatuses endpoint /api/v1/componentstatuses

|`/api/v1/componentstatuses`| |{$KUBE.API.TOKEN}|

Service account bearer token.

|| |{$KUBE.HTTP.PROXY}|

Sets the HTTP proxy to `http_proxy` value. If this parameter is empty, then no proxy is used.

|| |{$KUBE.STATE.ENDPOINT.NAME}|

Kubernetes state endpoint name.

|`zabbix-kube-state-metrics`| |{$KUBE.API_SERVER.SCHEME}|

Kubernetes API servers metrics endpoint scheme. Used in ControlPlane LLD.

|`https`| |{$KUBE.API_SERVER.PORT}|

Kubernetes API servers metrics endpoint port. Used in ControlPlane LLD.

|`6443`| |{$KUBE.CONTROL_PLANE.TAINT}|

Taint that applies to control plane nodes. Change if needed. Used in ControlPlane LLD.

|`node-role.kubernetes.io/control-plane`| |{$KUBE.CONTROLLER_MANAGER.SCHEME}|

Kubernetes Controller manager metrics endpoint scheme. Used in ControlPlane LLD.

|`https`| |{$KUBE.CONTROLLER_MANAGER.PORT}|

Kubernetes Controller manager metrics endpoint port. Used in ControlPlane LLD.

|`10257`| |{$KUBE.SCHEDULER.SCHEME}|

Kubernetes Scheduler metrics endpoint scheme. Used in ControlPlane LLD.

|`https`| |{$KUBE.SCHEDULER.PORT}|

Kubernetes Scheduler metrics endpoint port. Used in ControlPlane LLD.

|`10259`| |{$KUBE.KUBELET.SCHEME}|

Kubernetes Kubelet metrics endpoint scheme. Used in Kubelet LLD.

|`https`| |{$KUBE.KUBELET.PORT}|

Kubernetes Kubelet metrics endpoint port. Used in Kubelet LLD.

|`10250`| |{$KUBE.LLD.FILTER.NAMESPACE.MATCHES}|

Filter of discoverable metrics by namespace.

|`.*`| |{$KUBE.LLD.FILTER.NAMESPACE.NOT_MATCHES}|

Filter to exclude discovered metrics by namespace.

|`CHANGE_IF_NEEDED`| |{$KUBE.LLD.FILTER.NODE.MATCHES}|

Filter of discoverable nodes by nodename.

|`.*`| |{$KUBE.LLD.FILTER.NODE.NOT_MATCHES}|

Filter to exclude discovered nodes by nodename.

|`CHANGE_IF_NEEDED`| |{$KUBE.LLD.FILTER.KUBELET_NODE.MATCHES}|

Filter of discoverable Kubelets by nodename.

|`.*`| |{$KUBE.LLD.FILTER.KUBELET_NODE.NOT_MATCHES}|

Filter to exclude discovered Kubelets by nodename.

|`CHANGE_IF_NEEDED`| |{$KUBE.KUBELET.FILTER.ANNOTATIONS}|

Node annotations to filter Kubelets (regex in values are supported). See the template's README.md for details.

|| |{$KUBE.KUBELET.FILTER.LABELS}|

Node labels to filter Kubelets (regex in values are supported). See the template's README.md for details.

|| |{$KUBE.LLD.FILTER.PV.MATCHES}|

Filter of discoverable persistent volumes by name.

|`.*`| |{$KUBE.LLD.FILTER.PV.NOT_MATCHES}|

Filter to exclude discovered persistent volumes by name.

|`CHANGE_IF_NEEDED`| |{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD}|

The evaluation period range which is used for calculation of expressions in trigger prototypes (time period or value range). Can be used with context.

|`#5`| ### Items |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Get state metrics|

Collecting Kubernetes metrics from kube-state-metrics.

|Script|kube.state.metrics| |Kubernetes: Control plane LLD|

Generation of data for Control plane discovery rules.

|Script|kube.control_plane.lld

**Preprocessing**

  • Discard unchanged with heartbeat: `3h`

| |Kubernetes: Node LLD|

Generation of data for Kubelet discovery rules.

|Script|kube.node.lld

**Preprocessing**

  • Discard unchanged with heartbeat: `3h`

| |Kubernetes: Get component statuses||HTTP agent|kube.componentstatuses

**Preprocessing**

  • Check for not supported value

    ⛔️Custom on fail: Discard value

| |Kubernetes: Get readyz||HTTP agent|kube.readyz

**Preprocessing**

  • JavaScript: `The text is too long. Please see the template.`

| |Kubernetes: Get livez||HTTP agent|kube.livez

**Preprocessing**

  • JavaScript: `The text is too long. Please see the template.`

| |Kubernetes: Namespace count|

The number of namespaces.

|Dependent item|kube.namespace.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_namespace_created)`

    ⛔️Custom on fail: Discard value

| |Kubernetes: CronJob count|

Number of cronjobs.

|Dependent item|kube.cronjob.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_cronjob_created)`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Job count|

Number of jobs (generated by cronjob + job).

|Dependent item|kube.job.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_job_created)`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Endpoint count|

Number of endpoints.

|Dependent item|kube.endpoint.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_endpoint_created)`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Deployment count|

The number of deployments.

|Dependent item|kube.deployment.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_deployment_created)`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Service count|

The number of services.

|Dependent item|kube.service.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_service_created)`

    ⛔️Custom on fail: Discard value

| |Kubernetes: StatefulSet count|

The number of statefulsets.

|Dependent item|kube.statefulset.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_statefulset_created)`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node count|

The number of nodes.

|Dependent item|kube.node.count

**Preprocessing**

  • Prometheus pattern: `COUNT(kube_node_created)`

    ⛔️Custom on fail: Discard value

| ### LLD rule API servers discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |API servers discovery||Dependent item|kube.api_servers.discovery| ### LLD rule Controller manager nodes discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Controller manager nodes discovery||Dependent item|kube.controller_manager.discovery| ### LLD rule Scheduler servers nodes discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Scheduler servers nodes discovery||Dependent item|kube.scheduler.discovery| ### LLD rule Kubelet discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubelet discovery||Dependent item|kube.kubelet.discovery| ### LLD rule Daemonset discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Daemonset discovery||Dependent item|kube.daemonset.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_daemonset_status_number_ready`

| ### Item prototypes for Daemonset discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] Daemonset [{#NAME}]: Ready|

The number of nodes that should be running the daemon pod and have one or more running and ready.

|Dependent item|kube.daemonset.ready[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Daemonset [{#NAME}]: Scheduled|

The number of nodes running at least one daemon pod and are supposed to.

|Dependent item|kube.daemonset.scheduled[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Daemonset [{#NAME}]: Desired|

The number of nodes that should be running the daemon pod.

|Dependent item|kube.daemonset.desired[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Daemonset [{#NAME}]: Misscheduled|

The number of nodes running a daemon pod but are not supposed to.

|Dependent item|kube.daemonset.misscheduled[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Daemonset [{#NAME}]: Updated number scheduled|

The total number of nodes that are running updated daemon pod.

|Dependent item|kube.daemonset.updated[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### LLD rule PVC discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |PVC discovery||Dependent item|kube.pvc.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_persistentvolumeclaim_info`

  • JavaScript: `The text is too long. Please see the template.`

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for PVC discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] PVC [{#NAME}] Status phase|

The current status phase of the persistent volume claim.

|Dependent item|kube.pvc.status_phase[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

  • JavaScript: `The text is too long. Please see the template.`

| |Kubernetes: Namespace [{#NAMESPACE}] PVC [{#NAME}] Requested storage|

The capacity of storage requested by the persistent volume claim.

|Dependent item|kube.pvc.requested.storage[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] PVC status phase: Bound, sum|

The total amount of persistent volume claims in the Bound phase.

|Dependent item|kube.pvc.status_phase.bound.sum[{#NAMESPACE}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] PVC status phase: Lost, sum|

The total amount of persistent volume claims in the Lost phase.

|Dependent item|kube.pvc.status_phase.lost.sum[{#NAMESPACE}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] PVC status phase: Pending, sum|

The total amount of persistent volume claims in the Pending phase.

|Dependent item|kube.pvc.status_phase.pending.sum[{#NAMESPACE}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### Trigger prototypes for PVC discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: NS [{#NAMESPACE}] PVC [{#NAME}]: PVC is pending||`count(/Kubernetes cluster state by HTTP/kube.pvc.status_phase[{#NAMESPACE}/{#NAME}],2m,,5)>=2`|Warning|| ### LLD rule PV discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |PV discovery||Dependent item|kube.pv.discovery

**Preprocessing**

  • Prometheus to JSON: `The text is too long. Please see the template.`

  • JavaScript: `The text is too long. Please see the template.`

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for PV discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: PV [{#NAME}] Status phase|

The current status phase of the persistent volume.

|Dependent item|kube.pv.status_phase[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

  • JavaScript: `The text is too long. Please see the template.`

| |Kubernetes: PV [{#NAME}] Capacity bytes|

A capacity of the persistent volume in bytes.

|Dependent item|kube.pv.capacity.bytes[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: PV status phase: Pending, sum|

The total amount of persistent volumes in the Pending phase.

|Dependent item|kube.pv.status_phase.pending.sum[{#SINGLETON}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: PV status phase: Available, sum|

The total amount of persistent volumes in the Available phase.

|Dependent item|kube.pv.status_phase.available.sum[{#SINGLETON}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: PV status phase: Bound, sum|

The total amount of persistent volumes in the Bound phase.

|Dependent item|kube.pv.status_phase.bound.sum[{#SINGLETON}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: PV status phase: Released, sum|

The total amount of persistent volumes in the Released phase.

|Dependent item|kube.pv.status_phase.released.sum[{#SINGLETON}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: PV status phase: Failed, sum|

The total amount of persistent volumes in the Failed phase.

|Dependent item|kube.pv.status_phase.failed.sum[{#SINGLETON}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### Trigger prototypes for PV discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: PV [{#NAME}]: PV has failed||`count(/Kubernetes cluster state by HTTP/kube.pv.status_phase[{#NAME}],2m,,3)>=2`|Warning|| ### LLD rule Deployment discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Deployment discovery||Dependent item|kube.deployment.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_deployment_spec_paused`

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for Deployment discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Paused|

Whether the deployment is paused and will not be processed by the deployment controller.

|Dependent item|kube.deployment.spec_paused[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Replicas desired|

Number of desired pods for a deployment.

|Dependent item|kube.deployment.replicas_desired[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Rollingupdate max unavailable|

Maximum number of unavailable replicas during a rolling update of a deployment.

|Dependent item|kube.deployment.rollingupdate.max_unavailable[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Replicas|

The number of replicas per deployment.

|Dependent item|kube.deployment.replicas[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Replicas available|

The number of available replicas per deployment.

|Dependent item|kube.deployment.replicas_available[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Replicas unavailable|

The number of unavailable replicas per deployment.

|Dependent item|kube.deployment.replicas_unavailable[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Replicas updated|

The number of updated replicas per deployment.

|Dependent item|kube.deployment.replicas_updated[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Replicas mismatched|

The number of available replicas that mismatch the desired number of replicas.

|Dependent item|kube.deployment.replicas_mismatched[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus to JSON: `The text is too long. Please see the template.`

  • JavaScript: `The text is too long. Please see the template.`

| ### Trigger prototypes for Deployment discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: Namespace [{#NAMESPACE}] Deployment [{#NAME}]: Deployment replicas mismatch|

Deployment has not matched the expected number of replicas during the specified trigger evaluation period.

|`min(/Kubernetes cluster state by HTTP/kube.deployment.replicas_mismatched[{#NAMESPACE}/{#NAME}],{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD:"deployment:{#NAMESPACE}:{#NAME}"})>0 and last(/Kubernetes cluster state by HTTP/kube.deployment.replicas_desired[{#NAMESPACE}/{#NAME}])>=0 and last(/Kubernetes cluster state by HTTP/kube.deployment.replicas_available[{#NAMESPACE}/{#NAME}])>=0`|Warning|| ### LLD rule Endpoint discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Endpoint discovery||Dependent item|kube.endpoint.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_endpoint_created`

| ### Item prototypes for Endpoint discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] Endpoint [{#NAME}]: Address available|

Number of addresses available in endpoint.

|Dependent item|kube.endpoint.address_available[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Endpoint [{#NAME}]: Address not ready|

Number of addresses not ready in endpoint.

|Dependent item|kube.endpoint.address_not_ready[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Endpoint [{#NAME}]: Age|

Endpoint age (number of seconds since creation).

|Dependent item|kube.endpoint.age[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

  • JavaScript: `return (Math.floor(Date.now()/1000)-Number(value));`

| ### LLD rule Node discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Node discovery||Dependent item|kube.node.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_node_info`

| ### Item prototypes for Node discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Node [{#NAME}]: CPU allocatable|

The CPU resources of a node that are available for scheduling.

|Dependent item|kube.node.cpu_allocatable[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node [{#NAME}]: Memory allocatable|

The Memory resources of a node that are available for scheduling.

|Dependent item|kube.node.memory_allocatable[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node [{#NAME}]: Pods allocatable|

The Pods resources of a node that are available for scheduling.

|Dependent item|kube.node.pods_allocatable[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node [{#NAME}]: Ephemeral storage allocatable|

The allocatable ephemeral-storage of a node that is available for scheduling.

|Dependent item|kube.node.ephemeral_storage_allocatable[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node [{#NAME}]: CPU capacity|

The capacity for CPU resources of a node.

|Dependent item|kube.node.cpu_capacity[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node [{#NAME}]: Memory capacity|

The capacity for Memory resources of a node.

|Dependent item|kube.node.memory_capacity[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node [{#NAME}]: Ephemeral storage capacity|

The ephemeral-storage capacity of a node.

|Dependent item|kube.node.ephemeral_storage_capacity[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Node [{#NAME}]: Pods capacity|

The capacity for Pods resources of a node.

|Dependent item|kube.node.pods_capacity[{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### LLD rule Pod discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Pod discovery||Dependent item|kube.pod.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_pod_start_time`

| ### Item prototypes for Pod discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}] Phase: Pending|

Pod is in pending state.

|Dependent item|kube.pod.phase.pending[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `VALUE(kube_pod_status_phase{pod="{#NAME}", phase="Pending"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}] Phase: Succeeded|

Pod is in succeeded state.

|Dependent item|kube.pod.phase.succeeded[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}] Phase: Failed|

Pod is in failed state.

|Dependent item|kube.pod.phase.failed[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `VALUE(kube_pod_status_phase{pod="{#NAME}", phase="Failed"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}] Phase: Unknown|

Pod is in unknown state.

|Dependent item|kube.pod.phase.unknown[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `VALUE(kube_pod_status_phase{pod="{#NAME}", phase="Unknown"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}] Phase: Running|

Pod is in unknown state.

|Dependent item|kube.pod.phase.running[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `VALUE(kube_pod_status_phase{pod="{#NAME}", phase="Running"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers terminated|

Describes whether the container is currently in terminated state.

|Dependent item|kube.pod.containers_terminated[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `SUM(kube_pod_container_status_terminated{pod="{#NAME}"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers waiting|

Describes whether the container is currently in waiting state.

|Dependent item|kube.pod.containers_waiting[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `SUM(kube_pod_container_status_waiting{pod="{#NAME}"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers ready|

Describes whether the containers readiness check succeeded.

|Dependent item|kube.pod.containers_ready[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `SUM(kube_pod_container_status_ready{pod="{#NAME}"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers restarts|

The number of container restarts.

|Dependent item|kube.pod.containers_restarts[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `SUM(kube_pod_container_status_restarts_total{pod="{#NAME}"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers running|

Describes whether the container is currently in running state.

|Dependent item|kube.pod.containers_running[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `SUM(kube_pod_container_status_running{pod="{#NAME}"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Ready|

Describes whether the pod is ready to serve requests.

|Dependent item|kube.pod.ready[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Scheduled|

Describes the status of the scheduling process for the pod.

|Dependent item|kube.pod.scheduled[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Unschedulable|

Describes the unschedulable status for the pod.

|Dependent item|kube.pod.unschedulable[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `VALUE(kube_pod_status_unschedulable{pod="{#NAME}"})`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers CPU limits|

The limit on CPU cores to be used by a container.

|Dependent item|kube.pod.containers.limits.cpu[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers memory limits|

The limit on memory to be used by a container.

|Dependent item|kube.pod.containers.limits.memory[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers CPU requests|

The number of requested cpu cores by a container.

|Dependent item|kube.pod.containers.requests.cpu[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Containers memory requests|

The number of requested memory bytes by a container.

|Dependent item|kube.pod.containers.requests.memory[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### Trigger prototypes for Pod discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Pod is not healthy||`min(/Kubernetes cluster state by HTTP/kube.pod.phase.failed[{#NAMESPACE}/{#NAME}],10m)>0 or min(/Kubernetes cluster state by HTTP/kube.pod.phase.pending[{#NAMESPACE}/{#NAME}],10m)>0 or min(/Kubernetes cluster state by HTTP/kube.pod.phase.unknown[{#NAMESPACE}/{#NAME}],10m)>0`|High|| |Kubernetes: Namespace [{#NAMESPACE}] Pod [{#NAME}]: Pod is crash looping|

Containers of the pod keep restarting. This most likely indicates that the pod is in the CrashLoopBackOff state.

|`(last(/Kubernetes cluster state by HTTP/kube.pod.containers_restarts[{#NAMESPACE}/{#NAME}])-min(/Kubernetes cluster state by HTTP/kube.pod.containers_restarts[{#NAMESPACE}/{#NAME}],15m))>1`|Warning|| ### LLD rule ReplicaSet discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |ReplicaSet discovery||Dependent item|kube.replicaset.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_replicaset_status_replicas`

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for ReplicaSet discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] ReplicaSet [{#NAME}]: Replicas|

The number of replicas per ReplicaSet.

|Dependent item|kube.replicaset.replicas[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] ReplicaSet [{#NAME}]: Desired replicas|

Number of desired pods for a ReplicaSet.

|Dependent item|kube.replicaset.replicas_desired[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] ReplicaSet [{#NAME}]: Fully labeled replicas|

The number of fully labeled replicas per ReplicaSet.

|Dependent item|kube.replicaset.fully_labeled_replicas[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] ReplicaSet [{#NAME}]: Ready|

The number of ready replicas per ReplicaSet.

|Dependent item|kube.replicaset.ready[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] ReplicaSet [{#NAME}]: Replicas mismatched|

The number of ready replicas that mismatch the desired number of replicas.

|Dependent item|kube.replicaset.replicas_mismatched[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus to JSON: `The text is too long. Please see the template.`

  • JavaScript: `The text is too long. Please see the template.`

| ### Trigger prototypes for ReplicaSet discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: Namespace [{#NAMESPACE}] RS [{#NAME}]: ReplicaSet mismatch|

ReplicaSet has not matched the expected number of replicas during the specified trigger evaluation period.

|`min(/Kubernetes cluster state by HTTP/kube.replicaset.replicas_mismatched[{#NAMESPACE}/{#NAME}],{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD:"replicaset:{#NAMESPACE}:{#NAME}"})>0 and last(/Kubernetes cluster state by HTTP/kube.replicaset.replicas_desired[{#NAMESPACE}/{#NAME}])>=0 and last(/Kubernetes cluster state by HTTP/kube.replicaset.ready[{#NAMESPACE}/{#NAME}])>=0`|Warning|| ### LLD rule StatefulSet discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |StatefulSet discovery||Dependent item|kube.statefulset.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_statefulset_status_replicas`

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for StatefulSet discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: Replicas|

The number of replicas per StatefulSet.

|Dependent item|kube.statefulset.replicas[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: Desired replicas|

Number of desired pods for a StatefulSet.

|Dependent item|kube.statefulset.replicas_desired[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: Current replicas|

The number of current replicas per StatefulSet.

|Dependent item|kube.statefulset.replicas_current[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: Ready replicas|

The number of ready replicas per StatefulSet.

|Dependent item|kube.statefulset.replicas_ready[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: Updated replicas|

The number of updated replicas per StatefulSet.

|Dependent item|kube.statefulset.replicas_updated[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: Replicas mismatched|

The number of ready replicas that mismatch the number of replicas.

|Dependent item|kube.statefulset.replicas_mismatched[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus to JSON: `The text is too long. Please see the template.`

  • JavaScript: `The text is too long. Please see the template.`

| ### Trigger prototypes for StatefulSet discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: StatefulSet is down||`(last(/Kubernetes cluster state by HTTP/kube.statefulset.replicas_ready[{#NAMESPACE}/{#NAME}]) / last(/Kubernetes cluster state by HTTP/kube.statefulset.replicas_current[{#NAMESPACE}/{#NAME}]))<>1`|High|| |Kubernetes: Namespace [{#NAMESPACE}] StatefulSet [{#NAME}]: StatefulSet replicas mismatch|

StatefulSet has not matched the number of replicas during the specified trigger evaluation period.

|`min(/Kubernetes cluster state by HTTP/kube.statefulset.replicas_mismatched[{#NAMESPACE}/{#NAME}],{$KUBE.REPLICA.MISMATCH.EVAL_PERIOD:"statefulset:{#NAMESPACE}:{#NAME}"})>0 and last(/Kubernetes cluster state by HTTP/kube.statefulset.replicas[{#NAMESPACE}/{#NAME}])>=0 and last(/Kubernetes cluster state by HTTP/kube.statefulset.replicas_ready[{#NAMESPACE}/{#NAME}])>=0`|Warning|| ### LLD rule PodDisruptionBudget discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |PodDisruptionBudget discovery||Dependent item|kube.pdb.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_poddisruptionbudget_created`

| ### Item prototypes for PodDisruptionBudget discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] PodDisruptionBudget [{#NAME}]: Pods healthy|

Current number of healthy pods.

|Dependent item|kube.pdb.pods_healthy[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] PodDisruptionBudget [{#NAME}]: Pods desired|

Minimum desired number of healthy pods.

|Dependent item|kube.pdb.pods_desired[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] PodDisruptionBudget [{#NAME}]: Disruptions allowed|

Number of pod disruptions that are allowed.

|Dependent item|kube.pdb.disruptions_allowed[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] PodDisruptionBudget [{#NAME}]: Pods total|

Total number of pods counted by this disruption budget.

|Dependent item|kube.pdb.pods_total[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### LLD rule CronJob discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |CronJob discovery||Dependent item|kube.cronjob.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_cronjob_created`

| ### Item prototypes for CronJob discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Suspend|

Suspend flag tells the controller to suspend subsequent executions.

|Dependent item|kube.cronjob.spec_suspend[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

  • Discard unchanged with heartbeat: `1h`

| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Active|

Active holds pointers to currently running jobs.

|Dependent item|kube.cronjob.status_active[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Last schedule|

LastScheduleTime keeps information of when was the last time the job was successfully scheduled.

|Dependent item|kube.cronjob.last_schedule_time[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

  • JavaScript: `return new Date(value * 1000).toString().slice(0,19);`

| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Next schedule|

Next time the cronjob should be scheduled. The time after lastScheduleTime, or after the cron job's creation time if it's never been scheduled. Use this to determine if the job is delayed.

|Dependent item|kube.cronjob.next_schedule_time[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

  • JavaScript: `return new Date(value * 1000).toString().slice(0,19);`

| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Failed|

The number of pods which reached Phase Failed and the reason for failure.

|Dependent item|kube.cronjob.status_failed[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Succeeded|

The number of pods which reached Phase Succeeded.

|Dependent item|kube.cronjob.status_succeeded[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Completion succeeded|

Number of job has completed its execution.

|Dependent item|kube.cronjob.completion.succeeded[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] CronJob [{#NAME}]: Completion failed|

Number of job has failed its execution.

|Dependent item|kube.cronjob.completion.failed[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### LLD rule Job discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Job discovery||Dependent item|kube.job.discovery

**Preprocessing**

  • Prometheus to JSON: `kube_job_owner{owner_is_controller!="true"}`

| ### Item prototypes for Job discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Namespace [{#NAMESPACE}] Job [{#NAME}]: Failed|

The number of pods which reached Phase Failed and the reason for failure.

|Dependent item|kube.job.status_failed[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Job [{#NAME}]: Succeeded|

The number of pods which reached Phase Succeeded.

|Dependent item|kube.job.status_succeeded[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Job [{#NAME}]: Completion succeeded|

Number of job has completed its execution.

|Dependent item|kube.job.completion.succeeded[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| |Kubernetes: Namespace [{#NAMESPACE}] Job [{#NAME}]: Completion failed|

Number of job has failed its execution.

|Dependent item|kube.job.completion.failed[{#NAMESPACE}/{#NAME}]

**Preprocessing**

  • Prometheus pattern: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### LLD rule Component statuses discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Component statuses discovery||Dependent item|kube.componentstatuses.discovery

**Preprocessing**

  • JSON Path: `$.items`

    ⛔️Custom on fail: Discard value

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for Component statuses discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Component [{#NAME}]: Healthy|

Cluster component healthy.

|Dependent item|kube.componentstatuses.healthy[{#NAME}]

**Preprocessing**

  • JSON Path: `The text is too long. Please see the template.`

    ⛔️Custom on fail: Discard value

| ### Trigger prototypes for Component statuses discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: Component [{#NAME}] is unhealthy||`count(/Kubernetes cluster state by HTTP/kube.componentstatuses.healthy[{#NAME}],#3,,"True")<2 and length(last(/Kubernetes cluster state by HTTP/kube.componentstatuses.healthy[{#NAME}]))>0`|Warning|| ### LLD rule Readyz discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Readyz discovery||Dependent item|kube.readyz.discovery

**Preprocessing**

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for Readyz discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Readyz [{#NAME}]: Healthcheck|

Result of readyz healthcheck for component.

|Dependent item|kube.readyz.healthcheck[{#NAME}]

**Preprocessing**

  • JSON Path: `$.[?(@.name == "{#NAME}")].value.first()`

    ⛔️Custom on fail: Discard value

| ### Trigger prototypes for Readyz discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: Readyz [{#NAME}] is unhealthy||`count(/Kubernetes cluster state by HTTP/kube.readyz.healthcheck[{#NAME}],#3,,"ok")<2 and length(last(/Kubernetes cluster state by HTTP/kube.readyz.healthcheck[{#NAME}]))>0`|Warning|| ### LLD rule Livez discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Livez discovery||Dependent item|kube.livez.discovery

**Preprocessing**

  • Discard unchanged with heartbeat: `3h`

| ### Item prototypes for Livez discovery |Name|Description|Type|Key and additional info| |----|-----------|----|-----------------------| |Kubernetes: Livez [{#NAME}]: Healthcheck|

Result of livez healthcheck for component.

|Dependent item|kube.livez.healthcheck[{#NAME}]

**Preprocessing**

  • JSON Path: `$.[?(@.name == "{#NAME}")].value.first()`

    ⛔️Custom on fail: Discard value

| ### Trigger prototypes for Livez discovery |Name|Description|Expression|Severity|Dependencies and additional info| |----|-----------|----------|--------|--------------------------------| |Kubernetes: Livez [{#NAME}] is unhealthy||`count(/Kubernetes cluster state by HTTP/kube.livez.healthcheck[{#NAME}],#3,,"ok")<2 and length(last(/Kubernetes cluster state by HTTP/kube.livez.healthcheck[{#NAME}]))>0`|Warning|| ## Feedback Please report any issues with the template at [`https://support.zabbix.com`](https://support.zabbix.com) You can also provide feedback, discuss the template, or ask for help at [`ZABBIX forums`](https://www.zabbix.com/forum/zabbix-suggestions-and-feedback)