All but the newest MacOS versions (xnu-4570.1.46 / High Sierra)
lack utimensat() and UTIME_OMIT, see:
https://github.com/apple/darwin-xnu/blame/0a798f6738bc1db01281fc08ae024145e84df927/bsd/sys/stat.h#L537

The UTIME_OMIT value is intepreted literally by MacOS, resulting
in all files getting a 1970 timestamp
( https://github.com/rfjakob/gocryptfs/issues/229 ).

Emulate UTIME_OMIT by filling in the missing values
using an extra GetAttr call.

To not duplicate the logic in pathfs and nodefs, an
internal "utimens" helper package is created.

This patch fixes the failing utimens tests.

The File returned for NewReadOnlyFile only blocks writes not reads and
writes.

Tests loopbackFileSystem.Utimens() and loopbackfile.Utimens()
at 1-second precision.

The exising TestUtimesNano() test only works on Linux
because it relies on syscall.UtimesNano(), which is not
available on Darwin. The new tests call the Utimens()
functions directly, bypassing FUSE, and work on all
platforms.

Because Darwin does not have syscall.UtimesNano(),
getting the Utimens() implementation right is hard.

The tests currently fail on Darwin, underlining the
need for them ( https://github.com/rfjakob/gocryptfs/issues/229 ):

$ go test ./fuse/nodefs
[...]
--- FAIL: TestLoopbackFileUtimens (0.00s)
	helpers.go:51: mtime has changed: 1525384186 -> 1073
	helpers.go:70: atime has changed: 1525291058 -> 1073
[...]

$ go test ./fuse/pathfs
--- FAIL: TestLoopbackFileSystemUtimens (0.00s)
	helpers.go:51: mtime has changed: 1525384379 -> 1073
	helpers.go:70: atime has changed: 1525291058 -> 1073
[...]

Set the operationHandlers[op].DecodeIn function and
add xattr name parsing.

SETXATTR is special because it is the only opcode that
takes a file name (the xattr name) and a binary blob
(the xattr value). This was not supported by the
file name parsing code:
* setting FileNames = 1 would lump the xattr name
  and value together and truncates the last byte
* setting FileNames = 2 truncated the last xattr
  value byte.

This was solved by adding a special-case to parse(),
which seemed less ugly than adding a special-case
InputDebug(), or leaving the truncation as-is.

Before:
  2018/05/01 16:47:39 Dispatch 6: SETXATTR, NodeId: 3.  12 bytes
After:
  2018/05/01 16:48:36 Dispatch 6: SETXATTR, NodeId: 3. data: {sz 3 f0} names: [user.foo] 12 bytes

The change only affects debug output as doSetXAttr() does its
own bytes.SplitN(). The parsed filename *could* also be used in
doSetXAttr(), but the code seems clearer as-is.

We used to hand out a new generation number even
for already-known handles. This does not seem to
cause problems on Linux, but osxfuse throws errors
to userspace with

        osxfuse: vnode changed generation

showing up in the kernel log.

Introduce a per-handle generation number that stays
constant until the handle is forgotten.

Tested on Linux and MacOS via gocryptfs. Add test for handle map
behavior.

Fixes https://github.com/hanwen/go-fuse/issues/204
See also https://github.com/hanwen/go-fuse/pull/205

./cache_test.go:98: Skip call has possible formatting directive %d
FAIL	github.com/hanwen/go-fuse/fuse/test [build failed]

On mips64le, syscall.Getdents() and struct syscall.Dirent do
not fit together, causing our parseDirents() implementation to
return garbage ( see https://github.com/rfjakob/gocryptfs/issues/200
and https://github.com/golang/go/issues/23624 ).

Switch to unix.Getdents which does not have this problem -
the next Go release with the syscall package fixes is too
far away, and will take time to trickle into distros.

This issue has been reported by Debian maintainer
Felix Lechner.

Thanks to alexanderharm for suggesting this fix.

Markdown-ify README

The function was written with only a single return at
the end, which was elegant, but harder to follow.

Add early returns to handle common cases and add a few
comments.

A functional change is that we now catch when we get
OK with fi == nil and return EINVAL and log a message.
This should not happen and is a bug in the filesystem
implementation.

Passes ./all.bash and the gocryptfs test suite.

Consulting the output size is incorrect as it does not take into account the
cases where an xattr with an empty value exists.

Moreover, treat listxattr exactly the same as getxattr as the two both
can be used for two different operations.

Add a test for an xattr with an empty value
Signed-off-by: Shayan Pooya <shayan@arista.com>

This helps settle how much performance improvement splice()ing files
brings. The answer is that end-to-end read performance is improved by
about 10% for reading large files.

This saves a couple of bytes, and avoids a copy on serializing the
response header.

Add and document a hack for _OP_GETXATTR in serializeHeader.

Trivial patch that fixes a blank line in the debug output.

Before:

  2017/10/03 20:25:56 Dispatch 602: READDIRPLUS, NodeId: 1. data: {Fh 2 off 0 sz 4096  L 0 DIRECTORY,NONBLOCK,0x8000}
  2017/10/03 20:25:56 Serialize 602: READDIRPLUS code: OK value:  832 bytes data

  2017/10/03 20:25:56 Dispatch 603: GETXATTR, NodeId: 7. data: {sz 0} names: [system.posix_acl_access] 24 bytes
  2017/10/03 20:25:56 Serialize 603: GETXATTR code: 61=no data available value:

After:

  2017/10/03 21:02:46 Dispatch 22: READDIRPLUS, NodeId: 1. data: {Fh 2 off 0 sz 4096  L 0 RDONLY,0x8000}
  2017/10/03 21:02:46 Serialize 22: READDIRPLUS code: OK value:  648 bytes data
  2017/10/03 21:02:46 Dispatch 23: READDIRPLUS, NodeId: 1. data: {Fh 2 off 4 sz 4096  L 0 RDONLY,0x8000}
  2017/10/03 21:02:46 Serialize 23: READDIRPLUS code: OK value:

With golang 1.9, calling WaitMount is necessary to avoid hitting the
deadlock mentioned in #165 in tests where a single process acts as
both the fuse daemon and the client.
Signed-off-by: Shayan Pooya <shayan@arista.com>

Reduces Sizeof(nodefs.Inode{}) from 112 to 96 bytes.

The "generation" field in "Inode" is not used anywhere.
"portableHandleMap" has it's own generation field, which
is actually used.

The "check" field, embedded through the "handled" struct,
is checked for zero, but never set.

This is part of an effort to reduce the memory usage of go-fuse
and gocryptfs. Details can be seen at
https://github.com/rfjakob/gocryptfs/issues/132 .

When an app in a FUSE mount calls getdents(2), go-fuse receives
READDIR[PLUS] and calls the filesystem's OpenDir function that
returns []DirEntry.

The data returned from getdents(2) contains an inode number for
each directory entry, "d_ino". Until now, struct DirEntry had no
corresponding field and the value passed to the kernel was always
FUSE_UNKNOWN_INO = 0xffffffff

This broke apps that actually look at the d_ino field, like
"find -inum".

This commit adds the "Ino" filed to struct DirEntry. If the field
is not set by the filesystem, it is set to FUSE_UNKNOWN_INO,
as before. Otherwise it is left alone and passed to the kernel.

loopbackFileSystem's OpenDir function is extended to set the inode
number. A test verifies that the returned inode number is sane.

Fixes https://github.com/hanwen/go-fuse/issues/175

When a process forks and execs a new process, the child will not be
interested in the parent's file descriptor to /dev/fuse.
Signed-off-by: Shayan Pooya <shayan@arista.com>

TestUnionFsFdLeak looks at global state of the program, and is flaky:
the splice pool keeps open file descriptors, and its number depends on
how many read threads are scheduled in parallel.

Some versions of FUSE need this; fixes pathfs owner tests.

Inode.Parent() has to take the treeLock before looking
at the parents map.

Fixes https://github.com/hanwen/go-fuse/issues/166

The locking code is not hooked in the fuse/ server code, so this
just exercises the kernel's default locking implementation.

The parent tracking moved into nodefs in
commit 0107672a, so there is no need for
taking that lock anymore.

Go 1.9 uses epoll() for more efficient file I/O. File I/O causes a
call to epoll, and the runtime makes this call take up a GOMAXPROCS
slot.

The FUSE kernel module also supports poll: polling on a file residing
in a FUSE file system causes the kernel to sends a POLL request to the
userspace process.  If the process responds with ENOSYS, the kernel
will stop forwarding poll requests to the FUSE process.

In a test for Go FUSE file systems, it is normal to serve the
filesystem out of the same process that opens files in the file
system. If this happens in Go 1.9, the epoll call can take the only
GOMAXPROCS slot left, leaving the process unable to respond to the
FUSE POLL opcode, deadlocking the process.

This change add support for a magic file "/ .go-fuse-epoll-hack" with
node ID uint64(-1), and on starting up the file system, the library
calls poll() on this file, triggering the POLL opcode before the Go
runtime had a chance to do so.

There are two problem scenarios left:

* File system tests that start I/O before calling WaitMount() still
  risk deadlocking themselves.

* The Linux kernel keeps track of feature support in fuse_conn, which notes

         * The following bitfields are only for optimization purposes
         * and hence races in setting them will not cause malfunction

  if our forced ENOSYS gets lost due to a race condition in the
  kernel, this can still trigger.

Fixes golang/go#21014 and #165

This avoids involving the go 1.9 poller in I/O, which causes hangs, as
the poller does not understand the fcntl(O_NONBLOCK) tweak that we
executed on the pipe's file descriptors.

Fixes #164

The root user can issue the umount syscall and does not need the
fusermount binary. Therefore, it is not necessary to fork and exec a new
process just to run the umount binary.

Golang has a global lock held for forking (See exec_unix.go in the golang
source tree):
"Acquire the fork lock so that no other threads create new fds that are
not yet close-on-exec before we fork."

So it would be best if processes would refrain from forking
unnecessarily.
Signed-off-by: Shayan Pooya <shayan@arista.com>

This fixes 'du'.

Fixes #159.

Fixes #160.

This first race was easy to find and fix in the tests themselves.

It exposes another race in nodefs.FileSystemConnector where a node's
mount (and therefore, treeLock) is is not the same as the grand-parent
node's is.

Updates #138.