Unverified Commit 77c60674 authored by PICHOU Kyâne's avatar PICHOU Kyâne
Browse files

Migrate Mattermost metrics from metrics-bot to custom exporter

parent acf460db
......@@ -7,15 +7,24 @@ Pour une documentation générale (administration, fonctionnalités...), voir [l
Ce dossier contient une adaptation minimaliste du [Dockerfile officiel](https://github.com/mattermost/mattermost-docker) de Mattermost.
L'idée de garder une copie du Dockerfile sur ce dépôt est motivée par trois choses :
* Aucune image n'est disponible **officiellement** sur le Docker Hub, même s'il en existe
* En cas de problèmes de sécurité (CVE), on pourra directement agir dessus
* On peut changer les arguments du Dockerfile, comme le type d'instance (`team`) et l'UID de l'utilisateur (à retrouver sur le LDAP).
- Aucune image n'est disponible **officiellement** sur le Docker Hub, même s'il en existe
- En cas de problèmes de sécurité (CVE), on pourra directement agir dessus
- On peut changer les arguments du Dockerfile, comme le type d'instance (`team`) et l'UID de l'utilisateur (à retrouver sur le LDAP).
Aussi, on n'utilise pas le système de sauvegarde `WAL-e`, ce qui nous permet d'utiliser une image `postgres` de base plutôt que de rajouter la couche proposée par l'équipe Mattermost.
Enfin, le Docker Compose est adapté à notre configuration.
Nous ne pouvons pas versionner le fichier de configuration car il est très souvent modifié directement depuis la Console Administrateur : le versionner ici aurait pour effet d'annuler les modifications.
Nous ne pouvons pas versionner le fichier de configuration car il est très souvent modifié directement depuis la Console Administrateur : le versionner ici aurait pour effet d'annuler les modifications.
### Configuration
Avant de démarrer l'instance Mattermost, il est important d'ajouter des variables d'environnement de configuration pour la base de donnée dans le fichier `secrets/mattermost-db.secrets`.
D'autres variables de configuration sont à ajouter au fichier `secrets/mattermost-exporter.secrets` pour le bon fonctionnement de [l'exporter Prometheus](prometheus-exporter/README.md).
Enfin il faut créer un fichier `.env` (dans le même dossier que le Docker Compose) qui devra contenir une variable `METRICS_AUTH`. Cette vairbale correspond à la chaîne d'identification htpasswd utilisée pour authentifier sur l'endpoint des métriques, par exemple `METRICS_AUTH="mattermost:$apr1$bXnknJ0S$GsC.ozNJc/dAkh9uH7Qlg."`
### Procédure de mise à jour
......@@ -27,6 +36,7 @@ Ce n'est pas le plus pratique, mais ni la CI ni Docker ne permet de reprendre un
Il peut arriver que la version de PostgreSQL ne soit plus supportée par Mattermost.
Sans en arriver là, il est bon de régulièrement mettre à jour PostgreSQL :
> While upgrading will always contain some level of risk, PostgreSQL minor releases fix only frequently-encountered bugs, security issues, and data corruption problems to reduce the risk associated with upgrading. For minor releases, the community considers not upgrading to be riskier than upgrading. https://www.postgresql.org/support/versioning/
Les mise à jours mineures (changement du Y de la version X.Y) peuvent se faire sans intervention humaine. On veillera à bien regarder les logs.
......
version : "3.7"
version: "3.7"
networks:
proxy:
external: true
......@@ -50,3 +50,20 @@ services:
networks:
- mattermost
restart: unless-stopped
mattermost-exporter:
image: registry.picasoft.net/pica-mattermost-exporter:0.1.0
build: ./prometheus-exporter
container_name: mattermost-exporter
volumes:
- /etc/localtime:/etc/localtime:ro
env_file: ./secrets/mattermost-exporter.secrets
labels:
traefik.http.routers.mattermost-metrics.entrypoints: websecure
traefik.http.routers.mattermost-metrics.rule: "Host(`team.picasoft.net`) && PathPrefix(`/metrics`)"
traefik.http.routers.mattermost-metrics.service: mattermost-metrics
traefik.http.routers.mattermost-metrics.middlewares: "mattermost-metrics-auth@docker"
traefik.http.middlewares.mattermost-metrics-auth.basicauth.users: "${METRICS_AUTH}"
traefik.http.services.mattermost-metrics.loadbalancer.server.port: 8000
traefik.enable: true
restart: unless-stopped
FROM python:3.9-alpine3.13 as builder
COPY exporter.py /exporter.py
COPY requirements.txt /requirements.txt
RUN apk add --no-cache --virtual .build-deps \
gcc \
python3-dev \
musl-dev \
postgresql-dev \
&& apk add --no-cache libpq \
&& pip install --no-cache-dir -r /requirements.txt \
&& apk del .build-deps
CMD ["/exporter.py"]
# Mattermost Prometheus exporter
Afin d'exporter des métriques de Mattermost pour la métrologie, on utilise cette image Docker qui se charge simplement d'éxécuter un script qui va collecter des informations en base et les exposer sur un endpoint HTTP.
Ce script lit les variables d'environnement suivantes pour sa configuration :
- `EXPORTER_DB_HOST`: hostname du PostgreSQL de Mattermost (par défaut `mattermost-db`)
- `EXPORTER_DB_PORT`: port du PostgreSQL de Mattermost (par défaut `5432`)
- `EXPORTER_DB_NAME`: nom de la base de donnée (par défaut `mattermost`)
- `EXPORTER_DB_USER`: utilisateur pour se connecter à la base (par défaut `mattermost`)
- `EXPORTER_DB_PASSWORD`: mot de passe pour se connecter à la base
- `EXPORTER_COLLECT_INTERVAL`: nombre de secondes entre 2 actualisations des métriques (par défaut `60`)
- `INSTANCE_NAME`: nom de l'instance à ajouter en tag aux métriques (par exemple `team.picasoft.net`)
Pour des raisons de sécurité, il est préférable d'utiliser un compte ayant des droits limités sur la base de données (lecture seule) pour l'exporter. Il est possible de créer ce compte avec les lignes suivantes :
```sql
CREATE USER "mattermost-exporter" WITH PASSWORD 'strongpassword';
GRANT USAGE ON SCHEMA public TO "mattermost-exporter";
GRANT SELECT ON ALL TABLES IN SCHEMA public TO "mattermost-exporter";
```
#!/usr/bin/env python3
"""Prometheus exporter for Mattermost"""
# Imports
import os
import sys
import time
from datetime import datetime
import signal
from prometheus_client import start_http_server, Gauge, REGISTRY, PROCESS_COLLECTOR, PLATFORM_COLLECTOR
import psycopg2
# Mattermost channels types
MM_CHANNEL_PUBLIC = "O"
MM_CHANNEL_PRIVATE = "P"
MM_CHANNEL_DIRECT = "D"
MM_CHANNEL_GROUP = "G"
def db_connect():
"""
Connect to Mattermost database.
:returns: psycopg2 connector
"""
# Get database credentials
host = os.getenv("EXPORTER_DB_HOST", "mattermost-db")
port = int(os.getenv("EXPORTER_DB_PORT", "5432"))
dbname = os.getenv("EXPORTER_DB_NAME", "mattermost")
user = os.getenv("EXPORTER_DB_USER", "mattermost")
password = os.getenv("EXPORTER_DB_PASSWORD")
if password is None:
print("EXPORTER_DB_PASSWORD must be set.")
return None
# Connect to an existing database
conn = psycopg2.connect(host=host, port=port, dbname=dbname, user=user, password=password)
return conn
def get_users_count(db_conn):
"""
Get the number of active and deleted users
:param db_conn: Mattermost database connector
:returns: Dict
{
"active": 160,
"deleted": 12
}
"""
data = {}
# Create database cursor
db_cursor = db_conn.cursor()
# Query for active users
db_cursor.execute(
"SELECT COUNT(DISTINCT u.id) FROM users AS u LEFT JOIN Bots AS b ON u.id = b.userid WHERE u.deleteat = 0 AND b.userid IS NULL;"
)
data['active'] = db_cursor.fetchone()[0]
# Query for delete users
db_cursor.execute(
"SELECT COUNT(DISTINCT u.id) FROM users AS u LEFT JOIN Bots AS b ON u.id = b.userid WHERE u.deleteat != 0 AND b.userid IS NULL;"
)
data['deleted'] = db_cursor.fetchone()[0]
# Close cursor
db_cursor.close()
return data
def get_bots_count(db_conn):
"""
Get the number of active and deleted bots
:param db_conn: Mattermost database connector
:returns: Dict
{
"active": 160,
"deleted": 12
}
"""
data = {}
# Create database cursor
db_cursor = db_conn.cursor()
# Query for active bots
db_cursor.execute("SELECT COUNT(userid) FROM Bots WHERE deleteat = 0;")
data['active'] = db_cursor.fetchone()[0]
# Query for delete bots
db_cursor.execute("SELECT COUNT(userid) FROM Bots WHERE deleteat != 0;")
data['deleted'] = db_cursor.fetchone()[0]
# Close cursor
db_cursor.close()
return data
def get_posts_count(db_conn):
"""
Get the number of posts
:param db_conn: Mattermost database connector
:returns: Count of posts
"""
# Create database cursor
db_cursor = db_conn.cursor()
# Query for user count
db_cursor.execute("SELECT COUNT(id) FROM Posts;")
posts_count = db_cursor.fetchone()[0]
# Close cursor
db_cursor.close()
return posts_count
def get_teams_count(db_conn):
"""
Get the number of teams for each type
:param db_conn: Mattermost database connector
:returns: Dict of teams count by type
{
"public": 12,
"private": 4,
"deleted": 1
}
"""
data = {}
# Create database cursor
db_cursor = db_conn.cursor()
# Query for public teams
db_cursor.execute("SELECT COUNT(id) FROM Teams WHERE allowopeninvite = true AND deleteat = 0;")
data["public"] = db_cursor.fetchone()[0]
# Query for private teams
db_cursor.execute("SELECT COUNT(id) FROM Teams WHERE allowopeninvite = false AND deleteat = 0;")
data["private"] = db_cursor.fetchone()[0]
# Query for deleted teams
db_cursor.execute("SELECT COUNT(id) FROM Teams WHERE deleteat != 0;")
data["deleted"] = db_cursor.fetchone()[0]
# Close cursor
db_cursor.close()
return data
def get_channels_count(db_conn):
"""
Get the number of channels for each type
:param db_conn: Mattermost database connector
:returns: Dict of channels count by type
{
"public": 12,
"private": 4,
"direct": 8,
"group": 1
}
"""
data = {}
# Create database cursor
db_cursor = db_conn.cursor()
# Query for public channels count
db_cursor.execute("SELECT COUNT(id) FROM Channels WHERE type = '"+MM_CHANNEL_PUBLIC+"';")
data["public"] = db_cursor.fetchone()[0]
# Query for private channels count
db_cursor.execute("SELECT COUNT(id) FROM Channels WHERE type = '"+MM_CHANNEL_PRIVATE+"';")
data["private"] = db_cursor.fetchone()[0]
# Query for direct message channel count
db_cursor.execute("SELECT COUNT(id) FROM Channels WHERE type = '"+MM_CHANNEL_DIRECT+"';")
data["direct"] = db_cursor.fetchone()[0]
# Query for group message channels count
db_cursor.execute("SELECT COUNT(id) FROM Channels WHERE type = '"+MM_CHANNEL_GROUP+"';")
data["group"] = db_cursor.fetchone()[0]
# Close cursor
db_cursor.close()
return data
def get_daily_posts_count(db_conn):
"""
Get the number of posts on the current day
:param db_conn: Mattermost database connector
:returns: Count of posts for today
"""
# Create database cursor
db_cursor = db_conn.cursor()
# Get today start and end timestamps
today_start = int(datetime.now().replace(hour=0, minute=0, second=0, microsecond=0).timestamp() * 1000)
today_end = int(datetime.now().replace(hour=23, minute=59, second=59, microsecond=999999).timestamp() * 1000)
# Query for daily posts
db_cursor.execute(
"SELECT Count(Posts.Id) AS Value FROM Posts WHERE Posts.CreateAt <= " + str(today_end) +
" AND Posts.CreateAt >= " + str(today_start) + " GROUP BY DATE(TO_TIMESTAMP(Posts.CreateAt / 1000));"
)
daily_posts_count = db_cursor.fetchone()[0]
# Close cursor
db_cursor.close()
return daily_posts_count
def get_daily_users_count(db_conn):
"""
Get the number of users who wrote a message on the current day
:param db_conn: Mattermost database connector
:returns: Count of users for today
"""
# Create database cursor
db_cursor = db_conn.cursor()
# Get today start and end timestamps
today_start = int(datetime.now().replace(hour=0, minute=0, second=0, microsecond=0).timestamp() * 1000)
today_end = int(datetime.now().replace(hour=23, minute=59, second=59, microsecond=999999).timestamp() * 1000)
# Query for daily users
db_cursor.execute(
"SELECT COUNT(DISTINCT Posts.UserId) FROM Posts WHERE Posts.CreateAt <= " +
str(today_end) + " AND Posts.CreateAt >= " + str(today_start) + ";"
)
daily_users_count = db_cursor.fetchone()[0]
# Close cursor
db_cursor.close()
return daily_users_count
def main():
"""Main function"""
# Number of seconds between 2 metrics collection
collect_interval = int(os.getenv('EXPORTER_COLLECT_INTERVAL', "60"))
# Port for metrics server
exporter_port = 8000
# Get instance name for metrics tag
instance_name = os.getenv('INSTANCE_NAME')
if instance_name is None:
print("INSTANCE_NAME must be set.")
return None
# Connect to Mattermost database
db_conn = db_connect()
if db_conn is None:
print("Cannot connect to Mattermost database.")
sys.exit(-1)
# Define an exit function to close connection
def exit_handler(sig, frame):
print('Terminating...')
# Close database connection
db_conn.close()
sys.exit(0)
# Catch SIGINT and SIGTERM signals
signal.signal(signal.SIGINT, exit_handler)
signal.signal(signal.SIGTERM, exit_handler)
# Remove unwanted Prometheus metrics
[REGISTRY.unregister(c) for c in [
PROCESS_COLLECTOR,
PLATFORM_COLLECTOR,
REGISTRY._names_to_collectors['python_gc_objects_collected_total']
]]
# Start Prometheus exporter server
start_http_server(exporter_port)
# Register metrics
users_gauge = Gauge('mattermost_users_total', 'Number of regular users', ['instance_name', 'state'])
bots_gauge = Gauge('mattermost_bots_total', 'Number of bots users', ['instance_name', 'state'])
channels_gauge = Gauge('mattermost_channels_total', 'Number of channels', ['instance_name', 'type'])
posts_gauge = Gauge('mattermost_posts_total', 'Number of posts', ['instance_name'])
teams_gauge = Gauge('mattermost_teams_total', 'Number of teams', ['instance_name', 'type'])
daily_posts_gauge = Gauge('mattermost_daily_posts_total', 'Number of posts on current day', ['instance_name'])
daily_users_gauge = Gauge(
'mattermost_daily_users_total',
'Number of active users on current day',
['instance_name']
)
# Loop forever
while True:
# Update users
users = get_users_count(db_conn)
for user_state in users:
users_gauge.labels(instance_name=instance_name, state=user_state).set(users[user_state])
# Update bots users
bots = get_bots_count(db_conn)
for bot_state in bots:
bots_gauge.labels(instance_name=instance_name, state=bot_state).set(bots[bot_state])
# Update channels
channels = get_channels_count(db_conn)
for chan_type in channels:
channels_gauge.labels(instance_name=instance_name, type=chan_type).set(channels[chan_type])
# Update posts
posts_count = get_posts_count(db_conn)
posts_gauge.labels(instance_name=instance_name).set(posts_count)
# Update teams
teams = get_teams_count(db_conn)
for team_type in teams:
teams_gauge.labels(instance_name=instance_name, type=team_type).set(teams[team_type])
# Update daily posts
daily_posts_count = get_daily_posts_count(db_conn)
daily_posts_gauge.labels(instance_name=instance_name).set(daily_posts_count)
# Update daily users
daily_users_count = get_daily_users_count(db_conn)
daily_users_gauge.labels(instance_name=instance_name).set(daily_users_count)
# Wait before next metrics collection
time.sleep(collect_interval)
if __name__ == '__main__':
main()
prometheus_client==0.9.0
psycopg2==2.8.6
EXPORTER_DB_USER=mattermost-exporter
EXPORTER_DB_PASSWORD=strongpassword
INSTANCE_NAME="team.picasoft.net"
......@@ -4,14 +4,6 @@
"database": "picasoft"
},
"modules": {
"mattermost": [
{
"url": "https://team.picasoft.net",
"user": "MATTERMOST_USER",
"password": "MATTERMOST_PASSWORD",
"name": "team.picasoft.net"
}
],
"wekan": [
{
"url": "https://kanban.picasoft.net",
......
......@@ -2,16 +2,6 @@
set -e
if [ -z "${MATTERMOST_USER}" ]; then
echo >&2 'Error : missing required ${MATTERMOST_USER} environment variable, exiting.'
exit 1
fi
if [ -z "${MATTERMOST_PASSWORD}" ]; then
echo >&2 'Error : missing required ${MATTERMOST_PASSWORD} environment variable, exiting.'
exit 1
fi
if [ -z "${WEKAN_USER}" ]; then
echo >&2 'Error : missing required ${WEKAN_USER} environment variable, exiting.'
exit 1
......@@ -23,8 +13,6 @@ if [ -z "${WEKAN_PASSWORD}" ]; then
fi
cp /config.json /code/config/config.json
sed -i "s/MATTERMOST_USER/${MATTERMOST_USER}/g" /code/config/config.json
sed -i "s/MATTERMOST_PASSWORD/${MATTERMOST_PASSWORD}/g" /code/config/config.json
sed -i "s/WEKAN_USER/${WEKAN_USER}/g" /code/config/config.json
sed -i "s/WEKAN_PASSWORD/${WEKAN_PASSWORD}/g" /code/config/config.json
......
......@@ -12,3 +12,5 @@ PICA02_TRAEFIK_METRICS_USER=traefik
PICA02_TRAEFIK_METRICS_PASSWORD=superpassword
MONITORING_TRAEFIK_METRICS_USER=traefik
MONITORING_TRAEFIK_METRICS_PASSWORD=superpassword
MATTERMOST_METRICS_USER=mattermost
MATTERMOST_METRICS_PASSWORD=superpassword
......@@ -68,6 +68,15 @@ scrape_configs:
regex: ".*"
target_label: instance
replacement: "paste.picasoft.net"
# Scrape Mattermost metrics
- job_name: mattermost
scheme: "https"
basic_auth:
username: "%{MATTERMOST_METRICS_USER}"
password: "%{MATTERMOST_METRICS_PASSWORD}"
static_configs:
- targets:
- "team.picasoft.net"
# Scrape Etherpad metrics
- job_name: etherpad
scheme: "https"
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment