GitLab

Significantly improve the KPI computation and graphing processes with the following changes:

• fluentd and fluentbit unfortunately make no guarantee whatsoever that the data will be received in the order in which in was sent, and the developers have stated that performance is their main priority instead. The KPI calculation process should therefore assume that the new data it receives is out of order, which it now does. It applies adequate corrections to produce correct results instead of erroring out and skipping chunks of data; the resulting large gain in robustness is well worth the minor performance cost.
• Computing KPIs for a particular time interval requires the same type of data for the two previous time periods: a chunk of previous logs is now added to each batch of new data in order to not miss any data points.
• The sparse data storage optimization for the E UTRAN IP Throughput KPI has been reversed as it was not reliable enough. It should instead be possible to use the underlying database's sparse data array compression. The KPI data is now stored as it is returned by xlte, and data format optimizations for graphing purposes are done by the new API endpoint (see below).
• Add a new API endpoint dedicated to fetching KPI data for graphing, which performs several optimizations to lighten the load on the client:
  • As stated above, the data format is optimized to only send the necessary amount of data for plotting, instead of full sparse data which would not be plotted anyway.
  • Additionally, the data will be resampled (by averaging) in regular time intervals to only show at most 1000 points at once on the graph.
  • Finally, KPI data can also be requested in a specific time range, enabling dynamic updates of a graph to show more detail (e.g. when a user zooms in on a particular time range).