I'm writing a networked filesystem which reads data in 2MB blocks from
remote database. However read requests from the kernel come in much
smaller sizes - for example 4K-128K.

Since it would be very slow to refetch the 2MB block for e.g. every
consecutive 4K reads, a cache for fetched data is needed. A custom cache
would do, however since the kernel already implements pagecache to cache
file data, it is logical to use it directly.

FUSE protocol provides primitives for pagecache control. We already have
support for e.g. invalidating a data region for inode (InodeNotify), but
there is more there. In particular it is possible to store and
retrieve data into/from the kernel cache with "store notify" and
"retrieve notify" messages:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n68
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=v4.19-rc6-177-gcec4de302c5f#n756

https://git.kernel.org/linus/a1d75f2582
https://git.kernel.org/linus/2d45ba381a

This patch adds support for "store notify". Adding support for "retrieve
notify" might be added later since a) it is not needed right now, and b)
supporting it is a bit more work since the kernel sends a separate reply
with data and infrastructure has to be added to glue it back to original
"retrieve notify" server-originated request.

For user-visible API I decided not to duplicate FUSE-protocol naming
1-1 and to be more explicit in names - emphasizing it is about cache
control - it is e.g. InodeNotifyStoreCache instead of
InodeNotifyStore, and for retrieving it (hopefully) should be just
InodeRetrieveCache instead of InodeNotifyRetrieveCache and a
separate callback.

Thanks beforehand,
Kirill

Current code was requiring that both

	in.Major >= maj		_and_
	in.Minor >= min

this way e.g. kernel's 7.27 would satisfy requirement of e.g. 7.25, but
8.1 won't do. Fix it.

out.Attr is already of type fuse.Attr .

Improve test to eliminate backing FS semantics

On a Linux system with go-fuse program running as root, the mount is
performed by calling /bin/fusermount, and the unmount is performed with
syscall.Unmount()

This creates a problem on systems (CentOS 6) with a static-but-edited-by-mount
/etc/mtab file.

 - fusermount adds a line to mtab when the go-fuse program starts
 - syscall.Unmount() doesn't edit the file on program exit
 - subsequent invocations of the program fail to mount with:

    "Mount fail: fusermount exited with code 256"

Deleting the now-inaccurate mtab entry clears things up.

There's probably a workaround by adding "-n" option so that mount doesn't
edit mtab in the first place, but it's not obvious where to insert that when
starting with the hello.go example.

This ensure VFS locking continues working, ensuring backward compatibility.

according to golint, comments of a type must start with the type name
- case sensitive.

This makes the package available on all platforms

Co-authored-by: Paul Warren <paul.warren@emc.com>
Co-authored-by: Maria Shaldibina <mshaldibina@pivotal.io>
Co-authored-by: Han-Wen Nienhuys <hanwen@google.com>

When set, all Lookup calls will be forwarded from the kernel, even if
the current tree already has a child with the given name.

This allows the tree to be reconfigured driven by kernel events.

Fixes #211.

All but the newest MacOS versions (xnu-4570.1.46 / High Sierra)
lack utimensat() and UTIME_OMIT, see:
https://github.com/apple/darwin-xnu/blame/0a798f6738bc1db01281fc08ae024145e84df927/bsd/sys/stat.h#L537

The UTIME_OMIT value is intepreted literally by MacOS, resulting
in all files getting a 1970 timestamp
( https://github.com/rfjakob/gocryptfs/issues/229 ).

Emulate UTIME_OMIT by filling in the missing values
using an extra GetAttr call.

To not duplicate the logic in pathfs and nodefs, an
internal "utimens" helper package is created.

This patch fixes the failing utimens tests.

The File returned for NewReadOnlyFile only blocks writes not reads and
writes.

Tests loopbackFileSystem.Utimens() and loopbackfile.Utimens()
at 1-second precision.

The exising TestUtimesNano() test only works on Linux
because it relies on syscall.UtimesNano(), which is not
available on Darwin. The new tests call the Utimens()
functions directly, bypassing FUSE, and work on all
platforms.

Because Darwin does not have syscall.UtimesNano(),
getting the Utimens() implementation right is hard.

The tests currently fail on Darwin, underlining the
need for them ( https://github.com/rfjakob/gocryptfs/issues/229 ):

$ go test ./fuse/nodefs
[...]
--- FAIL: TestLoopbackFileUtimens (0.00s)
	helpers.go:51: mtime has changed: 1525384186 -> 1073
	helpers.go:70: atime has changed: 1525291058 -> 1073
[...]

$ go test ./fuse/pathfs
--- FAIL: TestLoopbackFileSystemUtimens (0.00s)
	helpers.go:51: mtime has changed: 1525384379 -> 1073
	helpers.go:70: atime has changed: 1525291058 -> 1073
[...]

Set the operationHandlers[op].DecodeIn function and
add xattr name parsing.

SETXATTR is special because it is the only opcode that
takes a file name (the xattr name) and a binary blob
(the xattr value). This was not supported by the
file name parsing code:
* setting FileNames = 1 would lump the xattr name
  and value together and truncates the last byte
* setting FileNames = 2 truncated the last xattr
  value byte.

This was solved by adding a special-case to parse(),
which seemed less ugly than adding a special-case
InputDebug(), or leaving the truncation as-is.

Before:
  2018/05/01 16:47:39 Dispatch 6: SETXATTR, NodeId: 3.  12 bytes
After:
  2018/05/01 16:48:36 Dispatch 6: SETXATTR, NodeId: 3. data: {sz 3 f0} names: [user.foo] 12 bytes

The change only affects debug output as doSetXAttr() does its
own bytes.SplitN(). The parsed filename *could* also be used in
doSetXAttr(), but the code seems clearer as-is.

We used to hand out a new generation number even
for already-known handles. This does not seem to
cause problems on Linux, but osxfuse throws errors
to userspace with

        osxfuse: vnode changed generation

showing up in the kernel log.

Introduce a per-handle generation number that stays
constant until the handle is forgotten.

Tested on Linux and MacOS via gocryptfs. Add test for handle map
behavior.

Fixes https://github.com/hanwen/go-fuse/issues/204
See also https://github.com/hanwen/go-fuse/pull/205

./cache_test.go:98: Skip call has possible formatting directive %d
FAIL	github.com/hanwen/go-fuse/fuse/test [build failed]

On mips64le, syscall.Getdents() and struct syscall.Dirent do
not fit together, causing our parseDirents() implementation to
return garbage ( see https://github.com/rfjakob/gocryptfs/issues/200
and https://github.com/golang/go/issues/23624 ).

Switch to unix.Getdents which does not have this problem -
the next Go release with the syscall package fixes is too
far away, and will take time to trickle into distros.

This issue has been reported by Debian maintainer
Felix Lechner.

Thanks to alexanderharm for suggesting this fix.

Markdown-ify README

The function was written with only a single return at
the end, which was elegant, but harder to follow.

Add early returns to handle common cases and add a few
comments.

A functional change is that we now catch when we get
OK with fi == nil and return EINVAL and log a message.
This should not happen and is a bug in the filesystem
implementation.

Passes ./all.bash and the gocryptfs test suite.

Consulting the output size is incorrect as it does not take into account the
cases where an xattr with an empty value exists.

Moreover, treat listxattr exactly the same as getxattr as the two both
can be used for two different operations.

Add a test for an xattr with an empty value
Signed-off-by: Shayan Pooya <shayan@arista.com>

This helps settle how much performance improvement splice()ing files
brings. The answer is that end-to-end read performance is improved by
about 10% for reading large files.

This saves a couple of bytes, and avoids a copy on serializing the
response header.

Add and document a hack for _OP_GETXATTR in serializeHeader.

Trivial patch that fixes a blank line in the debug output.

Before:

  2017/10/03 20:25:56 Dispatch 602: READDIRPLUS, NodeId: 1. data: {Fh 2 off 0 sz 4096  L 0 DIRECTORY,NONBLOCK,0x8000}
  2017/10/03 20:25:56 Serialize 602: READDIRPLUS code: OK value:  832 bytes data

  2017/10/03 20:25:56 Dispatch 603: GETXATTR, NodeId: 7. data: {sz 0} names: [system.posix_acl_access] 24 bytes
  2017/10/03 20:25:56 Serialize 603: GETXATTR code: 61=no data available value:

After:

  2017/10/03 21:02:46 Dispatch 22: READDIRPLUS, NodeId: 1. data: {Fh 2 off 0 sz 4096  L 0 RDONLY,0x8000}
  2017/10/03 21:02:46 Serialize 22: READDIRPLUS code: OK value:  648 bytes data
  2017/10/03 21:02:46 Dispatch 23: READDIRPLUS, NodeId: 1. data: {Fh 2 off 4 sz 4096  L 0 RDONLY,0x8000}
  2017/10/03 21:02:46 Serialize 23: READDIRPLUS code: OK value:

With golang 1.9, calling WaitMount is necessary to avoid hitting the
deadlock mentioned in #165 in tests where a single process acts as
both the fuse daemon and the client.
Signed-off-by: Shayan Pooya <shayan@arista.com>

Reduces Sizeof(nodefs.Inode{}) from 112 to 96 bytes.

The "generation" field in "Inode" is not used anywhere.
"portableHandleMap" has it's own generation field, which
is actually used.

The "check" field, embedded through the "handled" struct,
is checked for zero, but never set.

This is part of an effort to reduce the memory usage of go-fuse
and gocryptfs. Details can be seen at
https://github.com/rfjakob/gocryptfs/issues/132 .

When an app in a FUSE mount calls getdents(2), go-fuse receives
READDIR[PLUS] and calls the filesystem's OpenDir function that
returns []DirEntry.

The data returned from getdents(2) contains an inode number for
each directory entry, "d_ino". Until now, struct DirEntry had no
corresponding field and the value passed to the kernel was always
FUSE_UNKNOWN_INO = 0xffffffff

This broke apps that actually look at the d_ino field, like
"find -inum".

This commit adds the "Ino" filed to struct DirEntry. If the field
is not set by the filesystem, it is set to FUSE_UNKNOWN_INO,
as before. Otherwise it is left alone and passed to the kernel.

loopbackFileSystem's OpenDir function is extended to set the inode
number. A test verifies that the returned inode number is sane.

Fixes https://github.com/hanwen/go-fuse/issues/175