- 19 Sep, 2016 40 commits
-
-
Yang Sheng authored
Write out all extents when clear inode. Otherwise we may lose data while umount. Signed-off-by:
Yang Sheng <yang.sheng@intel.com> Signed-off-by:
Jinshan Xiong <jinshan.xiong@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5584 Reviewed-on: http://review.whamcloud.com/12103Reviewed-by:
Bobi Jam <bobijam@gmail.com> Reviewed-by:
Oleg Drokin <oleg.drokin@intel.com> Signed-off-by:
James Simmons <jsimmons@infradead.org> Signed-off-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-
Vitaly Fertman authored
The lock callback timeout is calculated as an average per namespace. This does not reflect individual client behavior. Instead, we should calculate it on a per-export basis. This is the client side changes for upstream client. Signed-off-by:
Vitaly Fertman <vitaly_fertman@xyratex.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4942Reviewed-by:
Andriy Skulysh <Andriy_Skulysh@xyratex.com> Reviewed-by:
Alexey Lyashkov <Alexey_Lyashkov@xyratex.com> Xyratex-bug-id: MRP-417 Reviewed-on: http://review.whamcloud.com/9336Reviewed-by:
James Simmons <uja.ornl@gmail.com> Signed-off-by:
-
Fan Yong authored
Move some inline code out of lmv core into lustre_lmv.h. This is to prepare for use outside of the lmv layer in the future of these functions. Change from passing in struct lmv_stripe_md to just int for lmv_is_known_hash_type. Signed-off-by:
Fan Yong <fan.yong@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5519 Reviewed-on: http://review.whamcloud.com/11845Reviewed-by:
Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by:
Lai Siyao <lai.siyao@intel.com> Reviewed-by:
-
Henri Doreau authored
Added jobid fields to Changelog records (and extended records). The CLF_JOBID flags allows to check if the field is present or not (old format) when reading an entry. Jobids are expressed as 32 chars long, zero-terminated strings. Updated test_205 in sanity.sh. Signed-off-by:
Henri Doreau <henri.doreau@cea.fr> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1996 Reviewed-on: http://review.whamcloud.com/4060Reviewed-by:
Andreas Dilger <andreas.dilger@intel.com> Reviewed-by:
Aurelien Degremont <aurelien.degremont@cea.fr> Signed-off-by:
-
Li Xi authored
The pool related codes have some inconsistency about the length of pool name. Creating and setting a pool name of length 16 to a directory will succeed. However, creating a file under that directory will fail. This patch disables any pool name which is longer or equal to 16. And it changes LOV_MAXPOOLNAME from 16 to 15 which might cause some invalid LLOG records of OST pools with 16 byte names. It is not a problem since invalid LLOG records are just ignored. And OST pools with 16 byte names won't work well anyway on the old versions. There will be problem of inconsistency if part of the servers have this patch and part of the servers don't. But it would be safe to assume that this is not a normal configuration. Signed-off-by:
Li Xi <lixi@ddn.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5054 Reviewed-on: http://review.whamcloud.com/10306Reviewed-by:
-
Stephen Champion authored
With the lockless __generic_file_aio_write introduced in LU-1669, ll_direct_IO_26 is no longer protected by the inode i_isem. This renders obsoltete checks that all transient pages have been handled before and after entry, and requires atomic access to their counter. Signed-off-by:
Stephen Champion <schamp@sgi.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5700 Reviewed-on: http://review.whamcloud.com/12179Reviewed-by:
-
Dmitry Eremin authored
The member lmv_obd->server_timeout and function lmv_set_timeouts() are not used. Signed-off-by:
Dmitry Eremin <dmitry.eremin@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-991 Reviewed-on: http://review.whamcloud.com/11880Reviewed-by:
John L. Hammond <john.hammond@intel.com> Reviewed-by:
-
Stephen Champion authored
When the number of an object in use or circulation is tied to memory size of the system, very large memory systems can overflow 32 bit counters. This patch addresses overflow on page counters in the osc LRU and obd accounting. Signed-off-by:
-
Dmitry Eremin authored
tgts_size is used as unsigned. Signed-off-by:
-
Dmitry Eremin authored
ltd_idx is used as unsigned. Signed-off-by:
-
John L. Hammond authored
In ll_iget() if ll_update_inode() fails then do not call make_bad_inode() on the inode since it may still be in use. Signed-off-by:
-
John L. Hammond authored
Rename LUSTRE_STRIPE_MAXBYTES to LUSTRE_EXT3_STRIPE_MAXBYTES and correct the comment describing its use. Signed-off-by:
Bob Glossman <bob.glossman@intel.com> Signed-off-by:
-
John L. Hammond authored
Move several definition only used in lustre/llite/ to lustre/llite/llite_internal.h. Remove lustre/include/{,linux/}lustre_lite.h and fixup the missing includes in other headers that this exposes. Signed-off-by: -
Patrick Farrell authored
Currently, it's difficult to match brw RPCs to objects and extents from client logs. This patch adds a D_RPCTRACE debug message giving the necessary information. Signed-off-by:
Patrick Farrell <paf@cray.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5531 Reviewed-on: http://review.whamcloud.com/11548Reviewed-by:
Alexey Lyashkov <alexey.lyashkov@seagate.com> Reviewed-by:
Ann Koehler <amk@cray.com> Reviewed-by:
Ryan Haasken <haasken@cray.com> Reviewed-by:
-
John L. Hammond authored
Remove some unused declarations from lustre_lib.h and move some others to more natural headers. Signed-off-by:
-
John L. Hammond authored
We are already touching dentry in CDEBUG macros so it will crash long before these checks. Since this is the case no need to do an additional check. Signed-off-by:
-
John L. Hammond authored
Style cleanup to make the code readable. Signed-off-by:
-
John L. Hammond authored
Change int mode to umode_t. Signed-off-by:
-
John L. Hammond authored
Remove the unused mode parameter from ll_create_it(). Signed-off-by:
-
John L. Hammond authored
Remove the effectively unused lookup_flags parameter from ll_lookup_it(). Signed-off-by:
-
Prakash Surya authored
Testing multi-threaded single shard file write performance has shown the inode mutex to be a limiting factor when using the generic_file_write_iter function. To work around this bottle neck, this change replaces the locked version of that call with the lock less version, specifically, __generic_file_write_iter. In order to maintain posix consistency, Lustre must now employ it's own locking mechanism in the higher layers. Currently writes are protected using the lli_write_mutex in the ll_inode_info structure. To protect against simultaneous write and truncate operations, since we no longer take the inode mutex during writes, we must down the lli_trunc_sem semaphore. Unfortunately, this change by itself does not garner any performance benefits. Using FIO on a single machine with 32 GB of RAM, write performance tests were ran with and without this change applied; the results are below: +---------+-----------+---------+--------+--------+ | fio v2.0.13 | Write Bandwidth (KB/s) | +---------+-----------+---------+--------+--------+ | # Tasks | GB / Task | Test 1 | Test 2 | Test 3 | +---------+-----------+---------+--------+--------+ | 1 | 64 | 452446 | 454623 | 457653 | | 2 | 32 | 850318 | 565373 | 602498 | | 4 | 16 | 1058900 | 463546 | 529107 | | 8 | 8 | 1026300 | 468190 | 576451 | | 16 | 4 | 1065500 | 503160 | 462902 | | 32 | 2 | 1068600 | 462228 | 466963 | | 64 | 1 | 991830 | 556618 | 557863 | +---------+-----------+---------+--------+--------+ * Test 1: Lustre client running 04ec54f. File per process write workload. This test was used as a baseline for what we _could_ achieve in the single shared file tests if the bottle necks were removed. * Test 2: Lustre client running 04ec54f. Single shared file workload, each task writing to a unique region. * Test 3: Lustre client running 04ec54f + this patch. Single shared file workload, each task writing to a unique region. In order to garner any real performance benefits out of a single shared file workload, the lli_write_mutex needs to be broken up into a range lock. That would allow write operations to unique regions of a file to be executed concurrently. This work is left to be done in a follow up patch. Signed-off-by:Prakash Surya <surya1@llnl.gov> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1669 Reviewed-on: http://review.whamcloud.com/6672Reviewed-by:
-
Prakash Surya authored
Testing has shown the ll_inode_inode's lli_write_mutex to be a limiting factor with single shared file write performance, when using many writing threads on a single machine. Even if each thread is writing to a unique portion of the file, the lli_write_mutex will prevent no more than a single thread to ever write to the file simultaneously. This change attempts to remove this bottle neck, by replacing this mutex with a range lock. This should allow multiple threads to write to a single file simultaneously iff the threads are writing to unique regions of the file. Performance testing shows this change to garner a significant performance boost to write bandwidth. Using FIO on a single machine with 32 GB of RAM, write performance tests were run with and without this change applied; the results are below: +---------+-----------+---------+--------+--------+--------+ | fio v2.0.13 | Write Bandwidth (KB/s) | +---------+-----------+---------+--------+--------+--------+ | # Tasks | GB / Task | Test 1 | Test 2 | Test 3 | Test 4 | +---------+-----------+---------+--------+--------+--------+ | 1 | 64 | 452446 | 454623 | 457653 | 463737 | | 2 | 32 | 850318 | 565373 | 602498 | 733027 | | 4 | 16 | 1058900 | 463546 | 529107 | 976284 | | 8 | 8 | 1026300 | 468190 | 576451 | 963404 | | 16 | 4 | 1065500 | 503160 | 462902 | 830065 | | 32 | 2 | 1068600 | 462228 | 466963 | 749733 | | 64 | 1 | 991830 | 556618 | 557863 | 710912 | +---------+-----------+---------+--------+--------+--------+ * Test 1: Lustre client running 04ec54f. File per process write workload. This test was used as a baseline for what we _could_ achieve in the single shared file tests if the bottle necks were removed. * Test 2: Lustre client running 04ec54f. Single shared file workload, each task writing to a unique region. * Test 3: Lustre client running 04ec54f + I0023132b. Single shared file workload, each task writing to a unique region. * Test 4: Lustre client running 04ec54f + this patch. Single shared file workload, each task writing to a unique region. Direct IO does not use the page cache like normal IO, so concurrent direct IO reads of the same pages are not safe. As a result, direct IO reads must take the range lock in ll_file_io_generic, otherwise they will attempt to work on the same pages and hit assertions like: (osc_request.c:1219:osc_brw_prep_request()) ASSERTION( i == 0 || pg->off > pg_prev->off ) failed: i 3 p_c 10 pg ffffea00017a5208 [pri 0 ind 2771] off 16384 prev_pg ffffea00017a51d0 [pri 0 ind 2256] off 16384 Signed-off-by:Bobi Jam <bobijam@hotmail.com> Reviewed-by:
Alexander Boyko <alexander.boyko@seagate.com> Reviewed-by:
Hiroya Nozaki <nozaki.hiroya@jp.fujitsu.com> Reviewed-by:
-
James Simmons authored
Earlier a bunch of interval handling got removed since it wasn't used by the upstream client. Now some of it is needed again for the client code so this patch restores what is needed. Signed-off-by:
-
Vitaly Fertman authored
While clients will resend client->server RPCs, servers would not resend server->client RPCs such as LDLM callbacks (blocking or completion callbacks/ASTs). This could result in clients being evicted from the server if blocking callbacks were dropped by the network (a failed router or lossy network) and the client did not cancel the requested lock in time. In order to fix this problem, this patch adds the ability to resend LDLM callbacks from the server and give the client a chance to respond within the timeout period before it is evicted: - resend BL AST within lock callback timeout period; - still do not resend CANCEL_ON_BLOCK; - regular resend for CP AST without BL AST embedded; - prolong lock callback timeout on resend; some fixes: - recovery-small test_10 to actually evict the client with dropped BL AST; - ETIMEDOUT to be returned if send limit is expired; Signed-off-by:
Johann Lombardi <johann.lombardi@intel.com> Reviewed-by:
-
Vitaly Fertman authored
otherwise, waiting lock may get granted as no BLOCKED_GRANTED flag is returned Signed-off-by:
-
Lai Siyao authored
Statahead thread should wait for inflight stat RPCs to finish in case statahead RPC callback may access data allocated in statahead thread context. ll_sa_entry_fini() should keep old entry if stat RPC is not finished yet. Simplify sai refcounting: * newly allocated sai will hold one refcount, and it will put it after starting statahead thread. * statahead thread holds one refcount. * agl thread holds one refcount. * stat process calls do_statahead_enter() which will try to get sai, and if it's valid, it will revalidate from statahead cache, and put refcount after use. Signed-off-by:
-
Dmitry Eremin authored
Comparison of unsigned value against 0 is always true. Signed-off-by:
-
John L. Hammond authored
Remove RCU2HANDLE macro from lustre_handles.h. Signed-off-by:
-
Henri Doreau authored
Removed the D_CHANGELOG pseudo-debug flag that wasn't actually defined as a usable one. Report the D_CHANGELOG messages as D_HSM ones instead since this is the primary user of these messages. Signed-off-by:
James Nunez <james.a.nunez@intel.com> Signed-off-by:
-
Li Xi authored
Add newly opened llog handle at the tail of handle list to increase lookup speed, especially for cancel operation, because the canceled log will be removed from the list, and lookup is from the beginning. Signed-off-by:
Ian Costello <costello.ian@gmail.com> Reviewed-by:
Mike Pershin <mike.pershin@intel.com> Reviewed-by:
-
Andreas Dilger authored
Move the LUSTRE_VERSION_CODE checks to trigger on x.y.53 instead of x.y.50, so that it is into the development cycle that they are hit instead of right at the start. In many cases, the #warning has been removed (to prevent build errors) and instead the code is just disabled outright. The dead code can be seen easily and removed in the future with less interruption to the development process. Signed-off-by:
Emoly Liu <emoly.liu@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4217 Reviewed-on: http://review.whamcloud.com/8630Reviewed-by:
-
John L. Hammond authored
In lmv_revalidate_slaves() ensure that the request returned by md_intent_lock() is properly released on all paths. Signed-off-by:
wang di <di.wang@intel.com> Reviewed-by:
-
Wang Shilong authored
Previously, when running 'modprobe lustre', it hit the following error message which is becaue of network initialisation failure: modprobe: ERROR: could not insert 'lustre': Input/output error However, error code is there, just let it return to caller, after this patch, error message will be something like: modprobe: ERROR: could not insert 'lustre': Network is down Signed-off-by:
Wang Shilong <wshilong@ddn.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5455 Reviewed-on: http://review.whamcloud.com/11337Reviewed-by:
-
Dmitry Eremin authored
Make lu_buf->lb_len unsigned. Signed-off-by:
-
Jinshan Xiong authored
Otherwise, the coh_page_bufsize keeps increasing when the file's layout keeps changing in lov_init_raid0(). Signed-off-by:
frank zago <fzago@cray.com> Reviewed-by:
-
Dmitry Eremin authored
Cleanup in general headers. * use size_t in cfs_size_round*() * make unsigned index and len in lustre_cfg_*() * make iteration variable the same type as comparing value * make unsigned pages counters Signed-off-by:
-
Bobi Jam authored
'commit 77605e41 ("staging/lustre/clio: add pages into writeback cache in batches")' adds a page to a list aggregate issuing them to writeback cache; A page add is referenced in llite/vvp io scope, while writeback cache commit de-refers it under osc sub io scope, and enabling -lu_ref will detect this scope mismatch. Signed-off-by:
Bobi Jam <bobijam.xu@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4503 Reviewed-on: http://review.whamcloud.com/8970Reviewed-by:
-
John L. Hammond authored
In lmv_intent_lock() if we will return an error then first release any locks referenced by the intent. Signed-off-by:
-
Niu Yawei authored
In brw_interpret(), we forgot page offset when calculating write offset, that leads to wrong kms for sync write. Signed-off-by:
Niu Yawei <yawei.niu@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5463 Reviewed-on: http://review.whamcloud.com/11374Reviewed-by:
Li Dongyang <dongyang.li@anu.edu.au> Reviewed-by:
-
John L. Hammond authored
In the handler for LL_IOC_LMV_GETSTRIPE convert stripe FIDs from little to CPU endian when unpacking lmv_user_md. In lmv_unpack_md_v1() fix a double conversion of the stripe count. Signed-off-by:
Jian Yu <jian.yu@intel.com> Reviewed-by:
-
