• Kirill Smelkov's avatar
    amari.xlog: Emit sync events periodically · 964f954a
    Kirill Smelkov authored
    So that xlog stream becomes self-synchronized and could be used even if
    we start reading it from some intermediate point instead of only from
    the beginning.
    
    We will need this in general - to be able to start reading long log not
    only from its beginning, and also in particular for Wendelin systems
    where logs are uploaded by Fluentd in chunks and some chunks could be
    potentially lost.
    
    Sync events are emitted always unconditionally with default sync
    interval being 10x the longest specified period. We also provide users a
    way to control sync periodicity via explicitly specifying
    "meta.sync/period" query in the logspec.
    
    See !3 (comment 175796) and
    further for related discussion.
    
    This is change of xlog protocol. But it is early days and the only
    direct consumer of xlog is amari.kpi which we adjust accordingly. So it
    should be ok.
    964f954a
kpi_test.py 24.8 KB