- 21 Aug, 2016 40 commits
-
-
Jinshan Xiong authored
A few changes are made in this patch for unstable pages tracking: 1. Remove kernel NFS unstable pages tracking because it killed performance 2. Track unstable pages as part of LRU cache. Otherwise Lustre can use much more memory than max_cached_mb 3. Remove obd_unstable_pages tracking to avoid using global atomic counter 4. Make unstable pages track optional. Tracking unstable pages is turned off by default, and can be controlled by llite.*.unstable_stats. Signed-off-by:
Jinshan Xiong <jinshan.xiong@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4841 Reviewed-on: http://review.whamcloud.com/10003Reviewed-by:
Andreas Dilger <andreas.dilger@intel.com> Reviewed-by:
Lai Siyao <lai.siyao@intel.com> Reviewed-by:
Oleg Drokin <oleg.drokin@intel.com> Signed-off-by:
James Simmons <jsimmons@infradead.org> Signed-off-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-
Jinshan Xiong authored
A page in clio used to occupy 584 bytes, which will use size-1024 slab cache. This patch reduces the per-page overhead to 512 bytes so it can use size-512 instead. Signed-off-by:
Bobi Jam <bobijam@gmail.com> Signed-off-by:
-
John L. Hammond authored
Rename each member of struct mdt_body, adding the prefix mbo_. Signed-off-by:
John L. Hammond <john.hammond@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2675 Reviewed-on: http://review.whamcloud.com/10202Reviewed-by:
Bob Glossman <bob.glossman@intel.com> Reviewed-by:
Mike Pershin <mike.pershin@intel.com> Signed-off-by:
-
Hongchao Zhang authored
the LOV xattr of directory could be either lov_user_md_v1 (size is 32) or lov_user_md_v3 (size is 48), then the actual size of the LOV xattr should be return. Signed-off-by:
Hongchao Zhang <hongchao.zhang@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5100 Reviewed-on: http://review.whamcloud.com/10453Reviewed-by:
James Simmons <uja.ornl@gmail.com> Reviewed-by:
Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by:
jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr> Reviewed-by:
-
Niu Yawei authored
Simple variable rename. Signed-off-by:
Niu Yawei <yawei.niu@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4345 Reviewed-on: http://review.whamcloud.com/10223Reviewed-by:
-
wang di authored
Create striped directory with correct mode, which should be handling same as mkdir. Signed-off-by:
wang di <di.wang@intel.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4929 Reviewed-on: http://review.whamcloud.com/10028Reviewed-by:
-
wang di authored
With DNE every object can have two locks in different namespaces: lookup lock in space of MDT storing direntry and update/open lock in space of MDT storing inode. In lmv_find_cbdata/lmv_lock_lock, it should try the MDT that the FID maps to first, since this can be easily found, and only try others if that fails. In the error handler of lmv_add_targets, it should check whether ld_tgt_count is being increased before ld_tgt_count is being -1. Signed-off-by:
Fan Yong <fan.yong@intel.com> Signed-off-by:
-
Fan Yong authored
When the layout LFSCK repairs orphan OST-object, if the parent MDT-object was lost, then it will re-create the MDT-object and regenerate the LOV EA and fill the target LOV EA slot with the orphan information, and fill other slots with zero (LOV hole); if related LOV EA slot is invalid or hole, then it will refill the target LOV EA slot; if the target slot exceeds current LOV EA tail, then extend the LOV EA, and fill the gaps as zero. Some of the LOV EA holes may cannot be re-filled finally because of lost some OST-objects. And even if they can be re-filled, but there are still some possible race accessings from client before the re-filling. If the client access the LOV EA with hole(s), it may cause some strange behaviour, such as trigger LBUG()/LASSERT() on the client. So we will make the client to be aware of the LOV EA is incomplete. We introduce a new LOV EA pattern flag LOV_PATTERN_F_HOLE for that: any time when the LFSCK repairs the LOV EA with hole(s), the LOV EA will be marked as LOV_PATTERN_F_HOLE; when all the holes in the LOV EA are refilled, the LOV_PATTERN_F_HOLE will be dropped. For a new client, it recongizes the pattern flag LOV_PATTERN_F_HOLE, then it can permit/forbid some opertions on the file with LOV holes: 1) Normal read/write the file with LOV EA hole is permitted, but the application will get EIO error when read data from the dummy slot or write data to the dummy slot. 2) The users can dump the recovered data via some common read tools, such as "dd conv=sync,noerror". 3) Append data to the file which has LOV EA hole will get EIO failure. 4) Other operations will skip the LOV EA hole(s), and will not get failures, such as {s,g}etattr, {s,g}getxattr, stat, chown/chgrp, chmod, touch, unlink, and so on. For an old client, since it will not recognize the new pattern flag LOV_PATTERN_F_HOLE. So the LOV EA with hole will be dicarded with failure, but it will not cause the client to be crashed. Signed-off-by: -
wang di authored
In ll_lookup_it_finish, we need use the real parent(stripe) FID to validate the parent UPDATE lock. Signed-off-by:
-
wang di authored
Normally we know the value of op_mea1 when ll_readdir is called. In the case of '.' or '..' op_mea1 is unknown so for that case fetch the real parents FID. Signed-off-by:
Li Xi <lixi@ddn.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4603 Reviewed-on: http://review.whamcloud.com/9191Reviewed-by:
Li Xi <pkuelelixi@gmail.com> Reviewed-by:
-
John L. Hammond authored
Move the definition of OBD_OCD_VERSION() and similar macros from lustre_idl.h to lustre_ver.h. These macros are primarily used in comparisons to LUSTRE_VERSION_CODE which is defined in lustre_ver.h and so should be defined there as well. Move a few definitions (related to FIDs, quota and striping) from lustre_idl.h to lustre_user.h. Signed-off-by:
-
Jinshan Xiong authored
To simplify the code, change the metrics from bytes to pages. Signed-off-by:
-
Jinshan Xiong authored
If the page is already dirtied in ll_write_end() and kernel tries to call balance_dirty_pages() to write back dirty pages in the same thread, this is deadlock case if the page is already held by clio. This can also fix the issue of LU-4873. Signed-off-by:
-
Christopher J. Morrone authored
It is not permitted in C++ to have a static declaration inside of an anonymous union. The g++ compiler will complaine with an error like this: error: struct ost_id::<anonymous union>::ostid invalid; an anonymous union can only have non-static data members [-fpermissive] This patch changes the code to use an unnamed struct in place of "struct ostid" inside of the anonymous union. That name declaration was completely unnecessary anyway, since it was not used anywhere else. Signed-off-by:
Christopher J. Morrone <morrone2@llnl.gov> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4987 Reviewed-on: http://review.whamcloud.com/10176Reviewed-by:
Robert Read <robert.read@intel.com> Reviewed-by:
-
Gregoire Pichon authored
Root squash exhibits inconsistent behaviour on a client when enabled. If a file is not cached on the client, then root will get a permission denied error when accessing the file. When the file has recently been accessed by a regular user and is still in cache, root will be able to access the file without error because the permission check is only done by the client that isn't aware of root squash. While the only real security benefit from root squash is to deny clients access to files owned by root itself, it also makes sense to treat file access on the client in a consistent manner regardless of whether the file is in cache or not. This patch adds root squash settings to llite so that client is able to apply root squashing when it is relevant. Configuration of MDT root squash settings will automatically be applied to llite config log as well. Update cfs_str2num_check() routine by removing any modification of the specified string parameter. Since string can come from ls_str field of a lstr structure, this avoids inconsistent ls_len field. Signed-off-by:
Gregoire Pichon <gregoire.pichon@bull.net> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1778 Reviewed-on: http://review.whamcloud.com/5700Reviewed-by:
-
John L. Hammond authored
In ll_prep_md_op_data() validate names according to the same formula used in mdd_name_check(). Add mdc_pack_name() to validate the name actually packed in the request. Signed-off-by:
-
wang di authored
Separate master stripe with master object, so 1. stripeEA only exists on master object. 2. sub-stripe object will be inserted into master object as sub-directory, and it can get the master object by "..". By this, it will remove those specilities for stripe0 in LMV and LOD. And also simplify LFSCK, i.e. consistency check would be easier. When then master object becomes an orphan, we should mark all of its sub-stripes as dead object as well, otherwise client might still be able to create files under these stripes. A few fixes for striped directory layout lock: 1. stripe 0 should be locked as EX, same as other stripes. 2. Acquire the layout for directory, when it is being unliked. Signed-off-by:
-
Chris Horn authored
When determining whether an early reply can be sent the server will calculate the new deadline based on an offset from the request arrival time. However, when actually setting the new deadline the server offsets the current time. This can result in deadlines being extended more than at_max seconds past the request arrival time. Instead, the server should offset the arrival time when updating its request timeout. When a client receives an early reply it doesn't know the server side arrival time so we use the original sent time as an approximation. Signed-off-by:
Chris Horn <hornc@cray.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4578 Reviewed-on: http://review.whamcloud.com/9100Reviewed-by:
Christopher J. Morrone <chris.morrone.llnl@gmail.com> Reviewed-by:
-
wang di authored
Add error handler during inode inialization, so inode will become bad inode if something bad happens during inode prepare phase, otherwise the striped directory will not get its layout and being mis-regarded as normal directory. Signed-off-by:
-
John L. Hammond authored
Move ioctl definitions and related functions from lustre_dlm.h, lustre_lib.h, obd.h, to lustre_ioctl.h. Replace the definitions of retired ioctls with comment. Signed-off-by:
-
Vitaly Fertman authored
Move checks for FAILED, DESTROYED flags under ldlm spinlock, destroy flock atomically with the check it is not destroyed yet. Do not put the granted flock into the resource if this is UNLOCK, TEST, or DEADLOCK'ed flock. Later a regression for this patch was reported under LU-7626. The refcount nonzero (1) after lock cleanup errors was reported. The reason is that the case LCK_NL was not handled for obdecho. Patch 17791 resolved this issue which has been combined into this upstream patch. Signed-off-by:
Vitaly Fertman <vitaly_fertman@xyratex.com> Signed-off-by:
Andriy Skulysh <andriy.skulysh@seagate.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2177Reviewed-by:
Alexey Lyashkov <alexey_lyashkov@xyratex.com> Reviewed-by:
Andriy Skulysh <andriy_skulysh@xyratex.com> Reviewed-by:
Mirza Arshad Mirza Hussain <arshad.hussain@seagate.com> Reviewed-by:
Alexey Leonidovich Lyashkov <alexey.lyashkov@seagate.com> Reviewed-on: http://review.whamcloud.com/17791Reviewed-by:
-
Fan Yong authored
Since all the LOV_MAGIC_* definitions have the same postfix values break that value out into its own definition. With this we can check whether the magic's postfix match the LOV_MAGIC_MAGIC or not: if yes, then it is quite possible that the clients has encountered an newer LOV magic. This extra information can let us handle those cases more gracefully. Signed-off-by:
-
Fan Yong authored
Unify the flow control interfaces for MDC RPC and FLD RPC. We allow to adjust the maximum inflight RPCs count via /sys interface. Signed-off-by:
-
John L. Hammond authored
Rename request to preq. Signed-off-by:
Nathaniel Clark <nathaniel.l.clark@intel.com> Reviewed-by:
-
John L. Hammond authored
In lmv_getattr_name() don't return a freed request in the error path. Signed-off-by:
-
Jinshan Xiong authored
Calculate the end of current stripe correctly when the stripe size is not power 2. Signed-off-by:
-
Jinshan Xiong authored
For a partial page write, it will have to issue a READ RPC firstly to get a full uptodate page. If another page is already locked by this thread it can easily cause deadlock. Signed-off-by:
-
Nathaniel Clark authored
Ensure there aren't invalid pointers hanging around after ptlrpc_req_finished is called. Signed-off-by:
-
wang di authored
If remote object is being found in a migrating directory, it should continue to lookup the object in remote MDT, instead of return. Signed-off-by:
-
John L. Hammond authored
By the time that a struct lmv_user_md reaches lmv_placement_policy() it has already been converted to little endian. Therefore use the appropriate macros around accesses to this this field. This issue was found by rewriting the definition of struct lmv_user_md to use the __leXX typedefs and running sparse. Signed-off-by:
Dmitry Eremin <dmitry.eremin@intel.com> Reviewed-by:
Swapnil Pimpale <spimpale@ddn.com> Reviewed-by:
-
John L. Hammond authored
Several inline functions return a structure that was passed in. Their is no need for this so just make these function void. Signed-off-by:
-
John L. Hammond authored
Change the return type of several predicate functions from int to bool. Signed-off-by:
-
John L. Hammond authored
Add a const qualifier wherever possible to the pointer parameters of the inline helper functions in lustre_idl.h and lustre_fid.h. Signed-off-by:
-
John L. Hammond authored
Remove the const for the __u64 parameters for inline functions in lustre_idl.h. Signed-off-by:
-
John L. Hammond authored
In mdc_ioc_swap_layouts() cancel *any* unused locks with LAYOUT or XATTR IBITS set on the two files. (This matches the locks acquired in mdt_swap_layouts(). Previously only locks that conflicted with a CR LAYOUT lock were cancelled.) Signed-off-by:
Faccini Bruno <bruno.faccini@intel.com> Reviewed-by:
-
wang di authored
1. Clear the client dentry cache before migrating file/directory to the remote MDT. 2. Do not return stripe information to client, if it did not get the layout lock. Signed-off-by:
-
Jinshan Xiong authored
Sometimes the rq_commit_cb of BRW RPC can be called twice if that RPC has already committed at reply time. This will cause inaccuracy of unstable pages accounting and then assertion. Signed-off-by:
Prakash Surya <surya1@llnl.gov> Reviewed-by:
-
Fan Yong authored
This enables the client to inject an error by altering the parent FID in order to test if the server side file system checker behaves properly. Signed-off-by:
-
Ryan Haasken authored
Since the log file name contains the current time in seconds, dumping the logs more than once per second causes EEXIST errors to be emitted. Add a static variable to libcfs_debug_dumplog_internal that records the time of the last Lustre log dump. If the current time in seconds is equal to the last time, do not dump logs again. Note that this is not thread-safe. However, in the rare case that two threads try to access last_dump_time simultaneously, the worst thing that could happen is that one of the threads will get an EEXIST error when trying to write the log file. This is no worse than the current situation, and it is not likely to happen. Signed-off-by:
Ryan Haasken <haasken@cray.com> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4129 Reviewed-on: http://review.whamcloud.com/8964Reviewed-by:
-
Dmitry Eremin authored
'plock.cookie' might be used uninitialized in this function. sscanf format specification '%d' expects type 'int *' for 'd', but parameter 3 has a different type '__u32*' Signed-off-by:
-
