1. 18 Oct, 2021 6 commits
    • Ahmed S. Darwish's avatar
      net: sched: Protect Qdisc::bstats with u64_stats · 67c9e627
      Ahmed S. Darwish authored
      The not-per-CPU variant of qdisc tc (traffic control) statistics,
      Qdisc::gnet_stats_basic_packed bstats, is protected with Qdisc::running
      sequence counter.
      
      This sequence counter is used for reliably protecting bstats reads from
      parallel writes. Meanwhile, the seqcount's write section covers a much
      wider area than bstats update: qdisc_run_begin() => qdisc_run_end().
      
      That read/write section asymmetry can lead to needless retries of the
      read section. To prepare for removing the Qdisc::running sequence
      counter altogether, introduce a u64_stats sync point inside bstats
      instead.
      
      Modify _bstats_update() to start/end the bstats u64_stats write
      section.
      
      For bisectability, and finer commits granularity, the bstats read
      section is still protected with a Qdisc::running read/retry loop and
      qdisc_run_begin/end() still starts/ends that seqcount write section.
      Once all call sites are modified to use _bstats_update(), the
      Qdisc::running seqcount will be removed and bstats read/retry loop will
      be modified to utilize the internal u64_stats sync point.
      
      Note, using u64_stats implies no sequence counter protection for 64-bit
      architectures. This can lead to the statistics "packets" vs. "bytes"
      values getting out of sync on rare occasions. The individual values will
      still be valid.
      
      [bigeasy: Minor commit message edits, init all gnet_stats_basic_packed.]
      Signed-off-by: default avatarAhmed S. Darwish <a.darwish@linutronix.de>
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      67c9e627
    • Ahmed S. Darwish's avatar
      u64_stats: Introduce u64_stats_set() · f2efdb17
      Ahmed S. Darwish authored
      Allow to directly set a u64_stats_t value which is used to provide an init
      function which sets it directly to zero intead of memset() the value.
      
      Add u64_stats_set() to the u64_stats API.
      
      [bigeasy: commit message. ]
      Signed-off-by: default avatarAhmed S. Darwish <a.darwish@linutronix.de>
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      f2efdb17
    • Sebastian Andrzej Siewior's avatar
      gen_stats: Move remaining users to gnet_stats_add_queue(). · 10940eb7
      Sebastian Andrzej Siewior authored
      The gnet_stats_queue::qlen member is only used in the SMP-case.
      
      qdisc_qstats_qlen_backlog() needs to add qdisc_qlen() to qstats.qlen to
      have the same value as that provided by qdisc_qlen_sum().
      
      gnet_stats_copy_queue() needs to overwritte the resulting qstats.qlen
      field whith the caller submitted qlen value. It might be differ from the
      submitted value.
      
      Let both functions use gnet_stats_add_queue() and remove unused
      __gnet_stats_copy_queue().
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      10940eb7
    • Sebastian Andrzej Siewior's avatar
      mq, mqprio: Use gnet_stats_add_queue(). · 7361df46
      Sebastian Andrzej Siewior authored
      gnet_stats_add_basic() and gnet_stats_add_queue() add up the statistics
      so they can be used directly for both the per-CPU and global case.
      
      gnet_stats_add_queue() copies either Qdisc's per-CPU
      gnet_stats_queue::qlen or the global member. The global
      gnet_stats_queue::qlen isn't touched in the per-CPU case so there is no
      need to consider it in the global-case.
      
      In the per-CPU case, the sum of global gnet_stats_queue::qlen and
      the per-CPU gnet_stats_queue::qlen was assigned to sch->q.qlen and
      sch->qstats.qlen. Now both fields are copied individually.
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      7361df46
    • Sebastian Andrzej Siewior's avatar
      gen_stats: Add gnet_stats_add_queue(). · 448e163f
      Sebastian Andrzej Siewior authored
      This function will replace __gnet_stats_copy_queue(). It reads all
      arguments and adds them into the passed gnet_stats_queue argument.
      In contrast to __gnet_stats_copy_queue() it also copies the qlen member.
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      448e163f
    • Sebastian Andrzej Siewior's avatar
      gen_stats: Add instead Set the value in __gnet_stats_copy_basic(). · fbf307c8
      Sebastian Andrzej Siewior authored
      __gnet_stats_copy_basic() always assigns the value to the bstats
      argument overwriting the previous value. The later added per-CPU version
      always accumulated the values in the returning gnet_stats_basic_packed
      argument.
      
      Based on review there are five users of that function as of today:
      - est_fetch_counters(), ___gnet_stats_copy_basic()
        memsets() bstats to zero, single invocation.
      
      - mq_dump(), mqprio_dump(), mqprio_dump_class_stats()
        memsets() bstats to zero, multiple invocation but does not use the
        function due to !qdisc_is_percpu_stats().
      
      Add the values in __gnet_stats_copy_basic() instead overwriting. Rename
      the function to gnet_stats_add_basic() to make it more obvious.
      Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      fbf307c8
  2. 16 Oct, 2021 34 commits