Category: prometheus

Delete promethes metrics – free disk space

My prometheus is set to hold 10years data but currently my disk is 95% full. So i want to delete some metrics that i never use.

  • First we need to enable admin web api
      containers:
      - args:
        - --config.file=/etc/prometheus/prometheus.yml
        - --storage.tsdb.path=/prometheus
        - --storage.tsdb.retention.time=3650d
        - --web.enable-admin-api
  • Delete node_load15 metrics
curl -X POST http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]=node_load15

curl -X POST -g 'http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]={kubernetes_pod_name!=""}'


curl -X POST http://localhost:9090/api/v1/admin/tsdb/clean_tombstones

Promethus inbuilt basic authentication and TLS

github : https://github.com/prometheus/prometheus/pull/8316

from v2.24.0 basic authentication and TLS is inbuilt into prometheus.

webconfig.yml

tls_server_config:
  cert_file: /etc/prometheus/prometheus.cert
  key_file: /etc/prometheus/prometheus.key

basic_auth_users:
  admin: $2y$12$/B1Z0Ohq/g9z/BlD30mi/uRDNdBRs/VrtAZrJDtY73Ttjc8RYHJ2O
  • Start prometheus with webconfig file
./prometheus --web.config.file=webconfig.yml
  • Prometheus will be accessible on https and with basic auth (admin/admin)
  • Password should be bcrypt encrypted – https://bcrypt-generator.com

More : https://github.com/roidelapluie/prometheus/blob/5b4f46a348ae3bc143629f25f0f997f39f30c2c2/docs/configuration/https.md