• Jérome Perrin's avatar
    software/grafana: replace promtail with fluent-bit for log ingestion · 27b400f6
    Jérome Perrin authored
    This comes with new features:
     - apache-style logs are ingested with fields pre-parsed
     - apache-style logs are enriched with geoip fields
     - mariadb slow log have a field for the query fingerprint, like
       pt-query digest
     - systemd journal can be ingested as logs
     - for known log formats, the timestamps of log events are extracted
       from the log itself, which allows ingesting logs with a delay, but
       also comes with new challenge to get the timezone right for zope logs
       which do not contain a timezone field in timestamps and are in the
       instance timezone. For zope logs, we introduce a `timezone` parameter,
       which is the same as the `timezone` parameter from the ERP5 instance.
       Other logs either include a timezone (apache-style logs) or are in
       local machine timzeone (zeo, haproxy errors), which is the same as
       fluent-bit itself, so no conversion is necessary here.
    
    This prepares the integration of fluent-bit in ERP5, to push logs to
    loki like this or wendelin.
    27b400f6