zabbix/templates/app/consul_http/consul_cluster/README.md

HashiCorp Consul Cluster by HTTP

Overview

The template to monitor HashiCorp Consul by Zabbix that works without any external scripts.
Most of the metrics are collected in one go, thanks to Zabbix bulk data collection.

Template HashiCorp Consul Cluster by HTTP — collects metrics by HTTP agent from API endpoints.
More information about metrics you can find in official documentation.

Requirements

Zabbix version: 7.0 and higher.

Tested versions

This template has been tested on:

• HashiCorp Consul 1.10.0

Configuration

Zabbix should be configured according to the instructions in the Templates out of the box section.

Setup

Template need to use Authorization via API token.

Don't forget to change macros {$CONSUL.CLUSTER.URL}, {$CONSUL.TOKEN}.
Also, see the Macros section for a list of macros used to set trigger values.

This template support Consul namespaces. You can set macro {$CONSUL.NAMESPACE}, if you are interested in only one service namespace. Do not specify this macro to get all of services.
In case of Open Source version leave this macro empty.

NOTE. Some metrics may not be collected depending on your HashiCorp Consul instance version and configuration.
NOTE. You maybe are interested in Envoy Proxy by HTTP template.

Macros used

Name	Description	Default
{$CONSUL.CLUSTER.URL}	Consul cluster URL.	http://localhost:8500
{$CONSUL.TOKEN}	Consul auth token.	<PUT YOUR AUTH TOKEN>
{$CONSUL.NAMESPACE}	Consul service namespace. Enterprise only, in case of Open Source version leave this macro empty. Do not specify this macro to get all of services.
{$CONSUL.API.SCHEME}	Consul API scheme. Using in node LLD.	http
{$CONSUL.API.PORT}	Consul API port. Using in node LLD.	8500
{$CONSUL.LLD.FILTER.NODE_NAME.MATCHES}	Filter of discoverable discovered nodes.	.*
{$CONSUL.LLD.FILTER.NODE_NAME.NOT_MATCHES}	Filter to exclude discovered nodes.	CHANGE IF NEEDED
{$CONSUL.LLD.FILTER.SERVICE_NAME.MATCHES}	Filter of discoverable discovered services.	.*
{$CONSUL.LLD.FILTER.SERVICE_NAME.NOT_MATCHES}	Filter to exclude discovered services.	CHANGE IF NEEDED
{$CONSUL.SERVICE_NODES.CRITICAL.MAX.AVG}	Maximum number of service nodes in status 'critical' for trigger expression. Can be used with context.	0

Items

Name	Description	Type	Key and additional info
Consul cluster: Cluster leader	Current leader address.	HTTP agent	consul.get_leader Preprocessing Check for not supported value ⛔️Custom on fail: Discard value Trim: " Discard unchanged with heartbeat: 1h
Consul cluster: Nodes: peers	The number of Raft peers for the datacenter in which the agent is running.	HTTP agent	consul.get_peers Preprocessing Check for not supported value ⛔️Custom on fail: Discard value JSON Path: $.length() Discard unchanged with heartbeat: 3h
Consul cluster: Get nodes	Catalog of nodes registered in a given datacenter.	HTTP agent	consul.get_nodes Preprocessing Check for not supported value ⛔️Custom on fail: Discard value
Consul cluster: Get nodes Serf health status	Get Serf Health Status for all agents in cluster.	HTTP agent	consul.get_cluster_serf Preprocessing Check for not supported value ⛔️Custom on fail: Discard value
Consul: Nodes: total	Number of nodes on current dc.	Dependent item	consul.nodes_total Preprocessing JSON Path: $.length() Discard unchanged with heartbeat: 3h
Consul: Nodes: passing	Number of agents on current dc with serf health status 'passing'.	Dependent item	consul.nodes_passing Preprocessing JSON Path: $[?(@.Status == "passing")].length() Discard unchanged with heartbeat: 3h
Consul: Nodes: critical	Number of agents on current dc with serf health status 'critical'.	Dependent item	consul.nodes_critical Preprocessing JSON Path: $[?(@.Status == "critical")].length() Discard unchanged with heartbeat: 3h
Consul: Nodes: warning	Number of agents on current dc with serf health status 'warning'.	Dependent item	consul.nodes_warning Preprocessing JSON Path: $[?(@.Status == "warning")].length() Discard unchanged with heartbeat: 3h
Consul cluster: Get services	Catalog of services registered in a given datacenter.	HTTP agent	consul.get_catalog_services Preprocessing Check for not supported value ⛔️Custom on fail: Discard value
Consul: Services: total	Number of services on current dc.	Dependent item	consul.services_total Preprocessing JavaScript: The text is too long. Please see the template. Discard unchanged with heartbeat: 3h

Triggers

Name	Description	Expression	Severity	Dependencies and additional info
Consul cluster: Leader has been changed	Consul cluster version has changed. Acknowledge to close the problem manually.	last(/HashiCorp Consul Cluster by HTTP/consul.get_leader,#1)<>last(/HashiCorp Consul Cluster by HTTP/consul.get_leader,#2) and length(last(/HashiCorp Consul Cluster by HTTP/consul.get_leader))>0	Info	Manual close: Yes
Consul: One or more nodes in cluster in 'critical' state	One or more agents on current dc with serf health status 'critical'.	last(/HashiCorp Consul Cluster by HTTP/consul.nodes_critical)>0	Average
Consul: One or more nodes in cluster in 'warning' state	One or more agents on current dc with serf health status 'warning'.	last(/HashiCorp Consul Cluster by HTTP/consul.nodes_warning)>0	Warning

LLD rule Consul cluster nodes discovery

Name	Description	Type	Key and additional info
Consul cluster nodes discovery		Dependent item	consul.lld_nodes Preprocessing JavaScript: The text is too long. Please see the template. Discard unchanged with heartbeat: 3h

Item prototypes for Consul cluster nodes discovery

Name	Description	Type	Key and additional info
Consul: Node ["{#NODE_NAME}"]: Serf Health	Node Serf Health Status.	Dependent item	consul.serf.health["{#NODE_NAME}"] Preprocessing JSON Path: The text is too long. Please see the template. JavaScript: The text is too long. Please see the template. Discard unchanged with heartbeat: 3h

LLD rule Consul cluster services discovery

Name	Description	Type	Key and additional info
Consul cluster services discovery		Dependent item	consul.lld_services Preprocessing JavaScript: The text is too long. Please see the template. Discard unchanged with heartbeat: 3h

Item prototypes for Consul cluster services discovery

Name	Description	Type	Key and additional info
Consul: Service ["{#SERVICE_NAME}"]: Nodes passing	The number of nodes with service status passing from those registered.	Dependent item	consul.service.nodes_passing["{#SERVICE_NAME}"] Preprocessing JSON Path: The text is too long. Please see the template. Discard unchanged with heartbeat: 3h
Consul: Service ["{#SERVICE_NAME}"]: Nodes warning	The number of nodes with service status warning from those registered.	Dependent item	consul.service.nodes_warning["{#SERVICE_NAME}"] Preprocessing JSON Path: The text is too long. Please see the template. Discard unchanged with heartbeat: 3h
Consul: Service ["{#SERVICE_NAME}"]: Nodes critical	The number of nodes with service status critical from those registered.	Dependent item	consul.service.nodes_critical["{#SERVICE_NAME}"] Preprocessing JSON Path: The text is too long. Please see the template. Discard unchanged with heartbeat: 3h
Consul cluster: ["{#SERVICE_NAME}"]: Get raw service state	Retrieve service instances providing the service indicated on the path.	HTTP agent	consul.get_service_stats["{#SERVICE_NAME}"] Preprocessing Check for not supported value ⛔️Custom on fail: Discard value

Trigger prototypes for Consul cluster services discovery

Name	Description	Expression	Severity	Dependencies and additional info
Consul: Service ["{#SERVICE_NAME}"]: Too many nodes with service status 'critical'	One or more nodes with service status 'critical'.	last(/HashiCorp Consul Cluster by HTTP/consul.service.nodes_critical["{#SERVICE_NAME}"])>{$CONSUL.CLUSTER.SERVICE_NODES.CRITICAL.MAX.AVG:"{#SERVICE_NAME}"}	Average

Feedback

Please report any issues with the template at https://support.zabbix.com

You can also provide feedback, discuss the template, or ask for help at ZABBIX forums