1. 15 Oct, 2015 11 commits
    • Tejun Heo's avatar
      cgroup: don't hold css_set_rwsem across css task iteration · ed27b9f7
      Tejun Heo authored
      css_sets are synchronized through css_set_rwsem but the locking scheme
      is kinda bizarre.  The hot paths - fork and exit - have to write lock
      the rwsem making the rw part pointless; furthermore, many readers
      already hold cgroup_mutex.
      
      One of the readers is css task iteration.  It read locks the rwsem
      over the entire duration of iteration.  This leads to silly locking
      behavior.  When cpuset tries to migrate processes of a cgroup to a
      different NUMA node, css_set_rwsem is held across the entire migration
      attempt which can take a long time locking out forking, exiting and
      other cgroup operations.
      
      This patch updates css task iteration so that it locks css_set_rwsem
      only while the iterator is being advanced.  css task iteration
      involves two levels - css_set and task iteration.  As css_sets in use
      are practically immutable, simply pinning the current one is enough
      for resuming iteration afterwards.  Task iteration is tricky as tasks
      may leave their css_set while iteration is in progress.  This is
      solved by keeping track of active iterators and advancing them if
      their next task leaves its css_set.
      
      v2: put_task_struct() in css_task_iter_next() moved outside
          css_set_rwsem.  A later patch will add cgroup operations to
          task_struct free path which may grab the same lock and this avoids
          deadlock possibilities.
      
          css_set_move_task() updated to use list_for_each_entry_safe() when
          walking task_iters and advancing them.  This is necessary as
          advancing an iter may remove it from the list.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      ed27b9f7
    • Tejun Heo's avatar
      cgroup: reorganize css_task_iter functions · ecb9d535
      Tejun Heo authored
      * Rename css_advance_task_iter() to css_task_iter_advance_css_set()
        and make it clear it->task_pos too at the end of the iteration.
      
      * Factor out css_task_iter_advance() from css_task_iter_next().  The
        new function whines if called on a terminated iterator.
      
      Except for the termination check, this is pure reorganization and
      doesn't introduce any behavior changes.  This will help the planned
      locking update for css_task_iter.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      ecb9d535
    • Tejun Heo's avatar
      cgroup: factor out css_set_move_task() · f6d7d049
      Tejun Heo authored
      A task is associated and disassociated with its css_set in three
      places - during migration, after a new task is created and when a task
      exits.  The first is handled by cgroup_task_migrate() and the latter
      two are open-coded.
      
      These are similar operations and spreading them over multiple places
      makes it harder to follow and update.  This patch collects all task
      css_set [dis]association operations into css_set_move_task().
      
      While css_set_move_task() may check whether populated state needs to
      be updated when not strictly necessary, the behavior is essentially
      equivalent before and after this patch.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      f6d7d049
    • Tejun Heo's avatar
      cgroup: keep css_set and task lists in chronological order · 389b9c1b
      Tejun Heo authored
      css task iteration will be updated to not leak cgroup internal locking
      to iterator users.  In preparation, update css_set and task lists to
      be in chronological order.
      
      For tasks, as migration path is already using list_splice_tail_init(),
      only cgroup_enable_task_cg_lists() and cgroup_post_fork() need
      updating.  For css_sets, link_css_set() is the only place which needs
      to be updated.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      389b9c1b
    • Tejun Heo's avatar
      cgroup: make cgroup_destroy_locked() test cgroup_is_populated() · 91486f61
      Tejun Heo authored
      cgroup_destroy_locked() currently tests whether any css_sets are
      associated to reject removal if the cgroup contains tasks.  This works
      because a css_set's refcnt converges with the number of tasks linked
      to it and thus there's no css_set linked to a cgroup if it doesn't
      have any live tasks.
      
      To help tracking resource usage of zombie tasks, putting the ref of
      css_set will be separated from disassociating the task from the
      css_set which means that a cgroup may have css_sets linked to it even
      when it doesn't have any live tasks.
      
      This patch updates cgroup_destroy_locked() so that it tests
      cgroup_is_populated(), which counts the number of populated css_sets,
      instead of whether cgrp->cset_links is empty to determine whether the
      cgroup is populated or not.  This ensures that rmdirs won't be
      incorrectly rejected for cgroups which only contain zombie tasks.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      91486f61
    • Tejun Heo's avatar
      cgroup: make css_sets pin the associated cgroups · 2ceb231b
      Tejun Heo authored
      Currently, css_sets don't pin the associated cgroups.  This is okay as
      a cgroup with css_sets associated are not allowed to be removed;
      however, to help resource tracking for zombie tasks, this is scheduled
      to change such that a cgroup can be removed even when it has css_sets
      associated as long as none of them are populated.
      
      To ensure that a cgroup doesn't go away while css_sets are still
      associated with it, make each associated css_set hold a reference on
      the cgroup if non-root.
      
      v2: Root cgroups are special and shouldn't be ref'd by css_sets.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      2ceb231b
    • Tejun Heo's avatar
      cgroup: relocate cgroup_[try]get/put() · 052c3f3a
      Tejun Heo authored
      Relocate cgroup_get(), cgroup_tryget() and cgroup_put() upwards.  This
      is pure code reorganization to prepare for future changes.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      052c3f3a
    • Tejun Heo's avatar
      cgroup: move check_for_release() invocation · ad2ed2b3
      Tejun Heo authored
      To trigger release agent when the last task leaves the cgroup,
      check_for_release() is called from put_css_set_locked(); however,
      css_set being unlinked is being decoupled from task leaving the cgroup
      and the correct condition to test is cgroup->nr_populated dropping to
      zero which check_for_release() is already updated to test.
      
      This patch moves check_for_release() invocation from
      put_css_set_locked() to cgroup_update_populated().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      ad2ed2b3
    • Tejun Heo's avatar
      cgroup: replace cgroup_has_tasks() with cgroup_is_populated() · 27bd4dbb
      Tejun Heo authored
      Currently, cgroup_has_tasks() tests whether the target cgroup has any
      css_set linked to it.  This works because a css_set's refcnt converges
      with the number of tasks linked to it and thus there's no css_set
      linked to a cgroup if it doesn't have any live tasks.
      
      To help tracking resource usage of zombie tasks, putting the ref of
      css_set will be separated from disassociating the task from the
      css_set which means that a cgroup may have css_sets linked to it even
      when it doesn't have any live tasks.
      
      This patch replaces cgroup_has_tasks() with cgroup_is_populated()
      which tests cgroup->nr_populated instead which locally counts the
      number of populated css_sets.  Unlike cgroup_has_tasks(),
      cgroup_is_populated() is recursive - if any of the descendants is
      populated, the cgroup is populated too.  While this changes the
      meaning of the test, all the existing users are okay with the change.
      
      While at it, replace the open-coded ->populated_cnt test in
      cgroup_events_show() with cgroup_is_populated().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Michal Hocko <mhocko@kernel.org>
      27bd4dbb
    • Tejun Heo's avatar
      cgroup: make cgroup->nr_populated count the number of populated css_sets · 0de0942d
      Tejun Heo authored
      Currently, cgroup->nr_populated counts whether the cgroup has any
      css_sets linked to it and the number of children which has non-zero
      ->nr_populated.  This works because a css_set's refcnt converges with
      the number of tasks linked to it and thus there's no css_set linked to
      a cgroup if it doesn't have any live tasks.
      
      To help tracking resource usage of zombie tasks, putting the ref of
      css_set will be separated from disassociating the task from the
      css_set which means that a cgroup may have css_sets linked to it even
      when it doesn't have any live tasks.
      
      This patch updates cgroup->nr_populated so that for the cgroup itself
      it counts the number of css_sets which have tasks associated with them
      so that empty css_sets don't skew the populated test.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      0de0942d
    • Tejun Heo's avatar
      cgroup: remove an unused parameter from cgroup_task_migrate() · b309e5b7
      Tejun Heo authored
      cgroup_task_migrate() no longer uses @old_cgrp.  Remove it.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      b309e5b7
  2. 25 Sep, 2015 1 commit
    • Tejun Heo's avatar
      cgroup: fix too early usage of static_branch_disable() · a3e72739
      Tejun Heo authored
      49d1dc4b ("cgroup: implement static_key based
      cgroup_subsys_enabled() and cgroup_subsys_on_dfl()") converted cgroup
      enabled test to use static_key; however, cgroup_disable() is called
      before static_key subsystem itself is initialized and thus leads to
      the following warning when "cgroup_disable=" parameter is specified.
      
       WARNING: CPU: 0 PID: 0 at kernel/jump_label.c:99 static_key_slow_dec+0x44/0x60()
       static_key_slow_dec used before call to jump_label_init
       ...
       Call Trace:
        [<ffffffff813b18c2>] dump_stack+0x44/0x62
        [<ffffffff8108dd52>] warn_slowpath_common+0x82/0xc0
        [<ffffffff8108ddec>] warn_slowpath_fmt+0x5c/0x80
        [<ffffffff8119c054>] static_key_slow_dec+0x44/0x60
        [<ffffffff81d826b6>] cgroup_disable+0xaf/0xd6
        [<ffffffff81d5f9de>] unknown_bootoption+0x8c/0x194
        [<ffffffff810b0c03>] parse_args+0x273/0x4a0
        [<ffffffff81d5fd67>] start_kernel+0x205/0x4b8
       ...
      
      Fix it by making cgroup_disable() to record the subsystems to disable
      in cgroup_disable_mask and moving the actual application to
      cgroup_init() which is late enough and where the enabled state is
      first used.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Reported-by: default avatarAndrey Wagin <avagin@gmail.com>
      Link: http://lkml.kernel.org/g/CANaxB-yFuS4SA2znSvcKrO9L_CbHciHYW+o9bN8sZJ8eR9FxYA@mail.gmail.com
      Fixes: 49d1dc4b
      a3e72739
  3. 24 Sep, 2015 2 commits
  4. 22 Sep, 2015 5 commits
    • Tejun Heo's avatar
      cgroup: make cgroup_update_dfl_csses() migrate all target processes atomically · 10265075
      Tejun Heo authored
      cgroup_update_dfl_csses() is responsible for migrating processes when
      controllers are enabled or disabled on the default hierarchy.  As the
      css association changes for all the processes in the affected cgroups,
      this involves migrating multiple processes.
      
      Up until now, it was implemented by migrating process-by-process until
      the source css_sets are empty; however, this means that if a process
      fails to migrate after some succeed before it, the recovery is very
      tricky.  This was considered okay as subsystems weren't allowed to
      reject process migration on the default hierarchy; unfortunately,
      enforcing this policy turned out to be problematic for certain types
      of resources - realtime slices for now.
      
      As such, the default hierarchy is gonna allow restricted failures
      during migration and to support that this patch makes
      cgroup_update_dfl_csses() migrate all target processes atomically
      rather than one-by-one.  The preceding patches made subsystems ready
      for multi-process migration and factored out taskset operations making
      this almost trivial.  All tasks of the target processes are put in the
      same taskset and the migration operations are performed once which
      either fails or succeeds for all.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      10265075
    • Tejun Heo's avatar
      cgroup: separate out taskset operations from cgroup_migrate() · adaae5dc
      Tejun Heo authored
      Currently, cgroup_migreate() implements large part of the migration
      logic inline including building the target taskset and actually
      migrating them.  This patch separates out the following taskset
      operations.
      
       CGROUP_TASKSET_INIT()		: taskset initializer
       cgroup_taskset_add()		: add a task to a taskset
       cgroup_taskset_migrate()	: migrate a taskset to the destination cgroup
      
      This will be used to implement atomic multi-process migration in
      cgroup_update_dfl_csses().  This is pure reorganization which doesn't
      introduce any functional changes.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      adaae5dc
    • Tejun Heo's avatar
      cgroup: reorder cgroup_migrate()'s parameters · 9af2ec45
      Tejun Heo authored
      cgroup_migrate() has the destination cgroup as the first parameter
      while cgroup_task_migrate() has the destination cset as the last.
      Another migration function is scheduled to be added which can make the
      discrepancy further stand out.  Let's reorder cgroup_migrate()'s
      parameters so that the destination cgroup is the last.
      
      This doesn't cause any functional difference.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      9af2ec45
    • Tejun Heo's avatar
      cgroup, memcg, cpuset: implement cgroup_taskset_for_each_leader() · 4530eddb
      Tejun Heo authored
      It wasn't explicitly documented but, when a process is being migrated,
      cpuset and memcg depend on cgroup_taskset_first() returning the
      threadgroup leader; however, this approach is somewhat ghetto and
      would no longer work for the planned multi-process migration.
      
      This patch introduces explicit cgroup_taskset_for_each_leader() which
      iterates over only the threadgroup leaders and replaces
      cgroup_taskset_first() usages for accessing the leader with it.
      
      This prepares both memcg and cpuset for multi-process migration.  This
      patch also updates the documentation for cgroup_taskset_for_each() to
      clarify the iteration rules and removes comments mentioning task
      ordering in tasksets.
      
      v2: A previous patch which added threadgroup leader test was dropped.
          Patch updated accordingly.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      4530eddb
    • Tejun Heo's avatar
      cpuset: migrate memory only for threadgroup leaders · 3df9ca0a
      Tejun Heo authored
      If memory_migrate flag is set, cpuset migrates memory according to the
      destnation css's nodemask.  The current implementation migrates memory
      whenever any thread of a process is migrated making the behavior
      somewhat arbitrary.  Let's tie memory operations to the threadgroup
      leader so that memory is migrated only when the leader is migrated.
      
      While this is a behavior change, given the inherent fuziness, this
      change is not too likely to be noticed and allows us to clearly define
      who owns the memory (always the leader) and helps the planned atomic
      multi-process migration.
      
      Note that we're currently migrating memory in migration path proper
      while holding all the locks.  In the long term, this should be moved
      out to an async work item.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      3df9ca0a
  5. 21 Sep, 2015 1 commit
  6. 18 Sep, 2015 11 commits
    • Tejun Heo's avatar
      cgroup: generalize obtaining the handles of and notifying cgroup files · 6f60eade
      Tejun Heo authored
      cgroup core handles creations and removals of cgroup interface files
      as described by cftypes.  There are cases where the handle for a given
      file instance is necessary, for example, to generate a file modified
      event.  Currently, this is handled by explicitly matching the callback
      method pointer and storing the file handle manually in
      cgroup_add_file().  While this simple approach works for cgroup core
      files, it can't for controller interface files.
      
      This patch generalizes cgroup interface file handle handling.  struct
      cgroup_file is defined and each cftype can optionally tell cgroup core
      to store the file handle by setting ->file_offset.  A file handle
      remains accessible as long as the containing css is accessible.
      
      Both "cgroup.procs" and "cgroup.events" are converted to use the new
      generic mechanism instead of hooking directly into cgroup_add_file().
      Also, cgroup_file_notify() which takes a struct cgroup_file and
      generates a file modified event on it is added and replaces explicit
      kernfs_notify() invocations.
      
      This generalizes cgroup file handle handling and allows controllers to
      generate file modified notifications.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      6f60eade
    • Tejun Heo's avatar
      cgroup: restructure file creation / removal handling · 4df8dc90
      Tejun Heo authored
      The file creation / removal path has always been a bit icky and the
      planned notification update requires css during file creation.
      Restructure as follows.
      
      * cgroup_addrm_files() now takes both @css and @cgrp and is only
        called directly by other file handling functions.
      
      * cgroup_populate/clear_dir() are replaced with
        css_populate/clear_dir() taking @css and @cgrp_override.
        @cgrp_override is used only when files needs to be created on /
        removed from a cgroup which isn't attached to @css which happens
        during subsystem rebinds.  Subsystem loops are moved to the callers.
      
      * cgroup_add_file() now takes both @css and @cgrp.  @css isn't used
        yet but will be used by the planned notification update.
      
      This patch doens't cause any behavior changes.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      4df8dc90
    • Tejun Heo's avatar
      cgroup: cosmetic updates to rebind_subsystems() · 1ada4838
      Tejun Heo authored
      * Use local variables @scgrp and @dcgrp for @src_root->cgrp and
        @dst_root->cgrp respectively.
      
      * Use initializers to set @src_root and @css in the inner bind loop.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      1ada4838
    • Tejun Heo's avatar
      cgroup: make cgroup_addrm_files() clean up after itself on failures · 6732ed85
      Tejun Heo authored
      After a file creation failure, cgroup_addrm_files() it didn't remove
      the files which had already been created.  When cgroup_populate_dir()
      is the caller, this is fine as the caller performs cleanup; however,
      for other callers, this may leave unactivated dangling files behind.
      As kernfs directory removals are recursive, this doesn't lead to
      permanent memory leak but it can, for example, fail future attempts to
      create those files again.
      
      There's no point in keeping around this sort of subtlety and it gets
      in the way of planned updates to file handling.  This patch makes
      cgroup_addrm_files() clean up after itself on failures.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      6732ed85
    • Tejun Heo's avatar
      cgroup: relocate cgroup_populate_dir() · ccdca218
      Tejun Heo authored
      Move it upwards so that it's right below cgroup_clear_dir() and the
      forward declaration is unnecessary.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      ccdca218
    • Tejun Heo's avatar
      cgroup: replace cftype->mode with CFTYPE_WORLD_WRITABLE · 7dbdb199
      Tejun Heo authored
      cftype->mode allows controllers to give arbitrary permissions to
      interface knobs.  Except for "cgroup.event_control", the existing uses
      are spurious.
      
      * Some explicitly specify S_IRUGO | S_IWUSR even though that's the
        default.
      
      * "cpuset.memory_pressure" specifies S_IRUGO while also setting a
        write callback which returns -EACCES.  All it needs to do is simply
        not setting a write callback.
      
      "cgroup.event_control" uses cftype->mode to make the file
      world-writable.  It's a misdesigned interface and we don't want
      controllers to be tweaking interface file permissions in general.
      This patch removes cftype->mode and all its spurious uses and
      implements CFTYPE_WORLD_WRITABLE for "cgroup.event_control" which is
      marked as compatibility-only.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      7dbdb199
    • Tejun Heo's avatar
      cgroup: replace "cgroup.populated" with "cgroup.events" · 4a07c222
      Tejun Heo authored
      memcg already uses "memory.events" for event reporting and other
      controllers may need event reporting too.  Let's standardize on
      "$SUBSYS.events" interface file for reporting events which don't
      happen too frequently and thus can share event notification.
      
      "cgroup.populated" is replaced with "populated" field in
      "cgroup.events" and documentation is updated accordingly.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      4a07c222
    • Tejun Heo's avatar
      cgroup: replace cgroup_on_dfl() tests in controllers with cgroup_subsys_on_dfl() · 9e10a130
      Tejun Heo authored
      cgroup_on_dfl() tests whether the cgroup's root is the default
      hierarchy; however, an individual controller is only interested in
      whether the controller is attached to the default hierarchy and never
      tests a cgroup which doesn't belong to the hierarchy that the
      controller is attached to.
      
      This patch replaces cgroup_on_dfl() tests in controllers with faster
      static_key based cgroup_subsys_on_dfl().  This leaves cgroup core as
      the only user of cgroup_on_dfl() and the function is moved from the
      header file to cgroup.c.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      Cc: Vivek Goyal <vgoyal@redhat.com>
      Cc: Jens Axboe <axboe@kernel.dk>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Michal Hocko <mhocko@kernel.org>
      9e10a130
    • Tejun Heo's avatar
      cgroup: replace cgroup_subsys->disabled tests with cgroup_subsys_enabled() · fc5ed1e9
      Tejun Heo authored
      Replace cgroup_subsys->disabled tests in controllers with
      cgroup_subsys_enabled().  cgroup_subsys_enabled() requires literal
      subsys name as its parameter and thus can't be used for cgroup core
      which iterates through controllers.  For cgroup core, introduce and
      use cgroup_ssid_enabled() which uses slower static_key_enabled() test
      and can be indexed by subsys ID.
      
      This leaves cgroup_subsys->disabled unused.  Removed.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Michal Hocko <mhocko@kernel.org>
      fc5ed1e9
    • Tejun Heo's avatar
      cgroup: implement static_key based cgroup_subsys_enabled() and cgroup_subsys_on_dfl() · 49d1dc4b
      Tejun Heo authored
      Whether a subsys is enabled and attached to the default hierarchy
      seldom changes and may be tested in the hot paths.  This patch
      implements static_key based cgroup_subsys_enabled() and
      cgroup_subsys_on_dfl() tests.
      
      The following patches will update the users and remove duplicate
      mechanisms.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarZefan Li <lizefan@huawei.com>
      49d1dc4b
    • Tejun Heo's avatar
      jump_label: make static_key_enabled() work on static_key_true/false types too · fa128fd7
      Tejun Heo authored
      static_key_enabled() can be used on struct static_key but not on its
      wrapper types static_key_true and static_key_false.  The function is
      useful for debugging and management of static keys.  Update it so that
      it can be used for the wrapper types too.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      fa128fd7
  7. 16 Sep, 2015 4 commits
    • Tejun Heo's avatar
      cgroup: simplify threadgroup locking · 3014dde7
      Tejun Heo authored
      Note: This commit was originally committed as b5ba75b5 but got
            reverted by f9f9e7b7 due to the performance regression from
            the percpu_rwsem write down/up operations added to cgroup task
            migration path.  percpu_rwsem changes which alleviate the
            performance issue are pending for v4.4-rc1 merge window.
            Re-apply.
      
      Now that threadgroup locking is made global, code paths around it can
      be simplified.
      
      * lock-verify-unlock-retry dancing removed from __cgroup_procs_write().
      
      * Race protection against de_thread() removed from
        cgroup_update_dfl_csses().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Link: http://lkml.kernel.org/g/55F8097A.7000206@de.ibm.com
      3014dde7
    • Tejun Heo's avatar
      sched, cgroup: replace signal_struct->group_rwsem with a global percpu_rwsem · 1ed13287
      Tejun Heo authored
      Note: This commit was originally committed as d59cfc09 but got
            reverted by 0c986253 due to the performance regression from
            the percpu_rwsem write down/up operations added to cgroup task
            migration path.  percpu_rwsem changes which alleviate the
            performance issue are pending for v4.4-rc1 merge window.
            Re-apply.
      
      The cgroup side of threadgroup locking uses signal_struct->group_rwsem
      to synchronize against threadgroup changes.  This per-process rwsem
      adds small overhead to thread creation, exit and exec paths, forces
      cgroup code paths to do lock-verify-unlock-retry dance in a couple
      places and makes it impossible to atomically perform operations across
      multiple processes.
      
      This patch replaces signal_struct->group_rwsem with a global
      percpu_rwsem cgroup_threadgroup_rwsem which is cheaper on the reader
      side and contained in cgroups proper.  This patch converts one-to-one.
      
      This does make writer side heavier and lower the granularity; however,
      cgroup process migration is a fairly cold path, we do want to optimize
      thread operations over it and cgroup migration operations don't take
      enough time for the lower granularity to matter.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Link: http://lkml.kernel.org/g/55F8097A.7000206@de.ibm.com
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      1ed13287
    • Tejun Heo's avatar
      Revert "sched, cgroup: replace signal_struct->group_rwsem with a global percpu_rwsem" · 0c986253
      Tejun Heo authored
      This reverts commit d59cfc09.
      
      d59cfc09 ("sched, cgroup: replace signal_struct->group_rwsem with
      a global percpu_rwsem") and b5ba75b5 ("cgroup: simplify
      threadgroup locking") changed how cgroup synchronizes against task
      fork and exits so that it uses global percpu_rwsem instead of
      per-process rwsem; unfortunately, the write [un]lock paths of
      percpu_rwsem always involve synchronize_rcu_expedited() which turned
      out to be too expensive.
      
      Improvements for percpu_rwsem are scheduled to be merged in the coming
      v4.4-rc1 merge window which alleviates this issue.  For now, revert
      the two commits to restore per-process rwsem.  They will be re-applied
      for the v4.4-rc1 merge window.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Link: http://lkml.kernel.org/g/55F8097A.7000206@de.ibm.comReported-by: default avatarChristian Borntraeger <borntraeger@de.ibm.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: stable@vger.kernel.org # v4.2+
      0c986253
    • Tejun Heo's avatar
      Revert "cgroup: simplify threadgroup locking" · f9f9e7b7
      Tejun Heo authored
      This reverts commit b5ba75b5.
      
      d59cfc09 ("sched, cgroup: replace signal_struct->group_rwsem with
      a global percpu_rwsem") and b5ba75b5 ("cgroup: simplify
      threadgroup locking") changed how cgroup synchronizes against task
      fork and exits so that it uses global percpu_rwsem instead of
      per-process rwsem; unfortunately, the write [un]lock paths of
      percpu_rwsem always involve synchronize_rcu_expedited() which turned
      out to be too expensive.
      
      Improvements for percpu_rwsem are scheduled to be merged in the coming
      v4.4-rc1 merge window which alleviates this issue.  For now, revert
      the two commits to restore per-process rwsem.  They will be re-applied
      for the v4.4-rc1 merge window.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Link: http://lkml.kernel.org/g/55F8097A.7000206@de.ibm.comReported-by: default avatarChristian Borntraeger <borntraeger@de.ibm.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: stable@vger.kernel.org # v4.2+
      f9f9e7b7
  8. 12 Sep, 2015 5 commits
    • Linus Torvalds's avatar
      Linux 4.3-rc1 · 6ff33f39
      Linus Torvalds authored
      6ff33f39
    • Linus Torvalds's avatar
      Merge tag 'cris-for-4.3' of git://git.kernel.org/pub/scm/linux/kernel/git/jesper/cris · 6917b51d
      Linus Torvalds authored
      Pull CRIS updates from Jesper Nilsson:
       "Mostly removal of old cruft of which we can use a generic version, or
        fixes for code not commonly run in the cris port, but also additions
        to enable some good debug"
      
      * tag 'cris-for-4.3' of git://git.kernel.org/pub/scm/linux/kernel/git/jesper/cris: (25 commits)
        CRISv10: delete unused lib/dmacopy.c
        CRISv10: delete unused lib/old_checksum.c
        CRIS: fix switch_mm() lockdep splat
        CRISv32: enable LOCKDEP_SUPPORT
        CRIS: add STACKTRACE_SUPPORT
        CRISv32: annotate irq enable in idle loop
        CRISv32: add support for irqflags tracing
        CRIS: UAPI: use generic types.h
        CRIS: UAPI: use generic shmbuf.h
        CRIS: UAPI: use generic msgbuf.h
        CRIS: UAPI: use generic socket.h
        CRIS: UAPI: use generic sembuf.h
        CRIS: UAPI: use generic sockios.h
        CRIS: UAPI: use generic auxvec.h
        CRIS: UAPI: use generic headers via Kbuild
        CRIS: UAPI: fix elf.h export
        CRIS: don't make asm/elf.h depend on asm/user.h
        CRIS: UAPI: fix ptrace.h
        CRISv32: Squash compile warnings for axisflashmap
        CRISv32: Add GPIO driver to the default configs
        ...
      6917b51d
    • Linus Torvalds's avatar
      blk: rq_data_dir() should not return a boolean · 10fbd36e
      Linus Torvalds authored
      rq_data_dir() returns either READ or WRITE (0 == READ, 1 == WRITE), not
      a boolean value.
      
      Now, admittedly the "!= 0" doesn't really change the value (0 stays as
      zero, 1 stays as one), but it's not only redundant, it confuses gcc, and
      causes gcc to warn about the construct
      
          switch (rq_data_dir(req)) {
              case READ:
                  ...
              case WRITE:
                  ...
      
      that we have in a few drivers.
      
      Now, the gcc warning is silly and stupid (it seems to warn not about the
      switch value having a different type from the case statements, but about
      _any_ boolean switch value), but in this case the code itself is silly
      and stupid too, so let's just change it, and get rid of warnings like
      this:
      
        drivers/block/hd.c: In function ‘hd_request’:
        drivers/block/hd.c:630:11: warning: switch condition has boolean value [-Wswitch-bool]
           switch (rq_data_dir(req)) {
      
      The odd '!= 0' came in when "cmd_flags" got turned into a "u64" in
      commit 5953316d ("block: make rq->cmd_flags be 64-bit") and is
      presumably because the old code (that just did a logical 'and' with 1)
      would then end up making the type of rq_data_dir() be u64 too.
      
      But if we want to retain the old regular integer type, let's just cast
      the result to 'int' rather than use that rather odd '!= 0'.
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      10fbd36e
    • Linus Torvalds's avatar
      Merge branch 'writeback-plugging' · e1df8b0a
      Linus Torvalds authored
      Fix up the writeback plugging introduced in commit d353d758
      ("writeback: plug writeback at a high level") that then caused problems
      due to the unplug happening with a spinlock held.
      
      * writeback-plugging:
        writeback: plug writeback in wb_writeback() and writeback_inodes_wb()
        Revert "writeback: plug writeback at a high level"
      e1df8b0a
    • Linus Torvalds's avatar
      writeback: plug writeback in wb_writeback() and writeback_inodes_wb() · 505a666e
      Linus Torvalds authored
      We had to revert the pluggin in writeback_sb_inodes() because the
      wb->list_lock is held, but we could easily plug at a higher level before
      taking that lock, and unplug after releasing it.  This does that.
      
      Chris will run performance numbers, just to verify that this approach is
      comparable to the alternative (we could just drop and re-take the lock
      around the blk_finish_plug() rather than these two commits.
      
      I'd have preferred waiting for actual performance numbers before picking
      one approach over the other, but I don't want to release rc1 with the
      known "sleeping function called from invalid context" issue, so I'll
      pick this cleanup version for now.  But if the numbers show that we
      really want to plug just at the writeback_sb_inodes() level, and we
      should just play ugly games with the spinlock, we'll switch to that.
      
      Cc: Chris Mason <clm@fb.com>
      Cc: Josef Bacik <jbacik@fb.com>
      Cc: Dave Chinner <david@fromorbit.com>
      Cc: Neil Brown <neilb@suse.de>
      Cc: Jan Kara <jack@suse.cz>
      Cc: Christoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      505a666e