Skip to content

G
go-fuse
Commit c0f14a4a authored by Kirill Smelkov's avatar Kirill Smelkov
Browse files
Options

Merge remote-tracking branch 'origin/master' into t 

* origin/master:
  fs: remove Inode.Parents
  Update top-level documentation (README and doc.go)
  fs: split zipfs into a example file and a test file
  fs: example for handleless read/write
  fs: rename Options.DefaultPermissions to NullPermissions
  fs: add an example for file handles with direct I/O
  fuse/test: make StatFs tests nonverbose
  internal/testutil: switch off debug if test is called TestNonVerbose
  fs: return OK for unimplemented Flush
  fs: add doc capturing design decisions
  fs: fix doc nits
  fs: add more package docs.
parents 1e564b6a 4dd68784
Show whitespace changes
Inline Side-by-side
Showing with 832 additions and 278 deletions
README.md
View file @ c0f14a4a
......@@ -3,26 +3,30 @@
[![Build Status](https://travis-ci.org/hanwen/go-fuse.svg?branch=master)](https://travis-ci.org/hanwen/go-fuse)
[![GoDoc](https://godoc.org/github.com/hanwen/go-fuse?status.svg)](https://godoc.org/github.com/hanwen/go-fuse)
native bindings for the FUSE kernel module.
Go native bindings for the FUSE kernel module.
## Highlights
You should import and use
[github.com/hanwen/go-fuse/fs](https://godoc.org/github.com/hanwen/go-fuse/fs)
library. It follows the wire protocol closely, but provides
convenient abstractions for building both node and path based file
systems
* High speed: as fast as libfuse using the gc compiler for single
threaded loads.
Older, deprecated APIs are available at
[github.com/hanwen/go-fuse/fuse/pathfs](https://godoc.org/github.com/hanwen/go-fuse/fuse/pathfs)
and
[github.com/hanwen/go-fuse/fuse/pathfs](https://godoc.org/github.com/hanwen/go-fuse/fuse/nodefs).
* Supports in-process mounting of different FileSystems onto
subdirectories of the FUSE mount.
## Comparison with other FUSE libraries
* Supports 3 interfaces for writing filesystems:
- `PathFileSystem`: define filesystems in terms path names.
- `NodeFileSystem`: define filesystems in terms of inodes.
- `RawFileSystem`: define filesystems in terms of FUSE's raw
wire protocol.
The FUSE library gained a new, cleaned-up API during a rewrite
completed in 2019. Find extensive documentation
[here](https://godoc.org/github.com/hanwen/go-fuse/).
* Both NodeFileSystem and PathFileSystem support manipulation of true
hardlinks.
Further highlights of this library is
* Includes two fleshed out examples, zipfs and unionfs.
* Comprehensive and up to date protocol support (up to 7.12.28).
* Performance that is competitive with libfuse.
## Examples
......@@ -54,50 +58,6 @@ subdirectories of the FUSE mount.
fusermount -u /tmp/mountpoint
```
* `unionfs/unionfs.go`: implements a union mount using 1 R/W branch, and
multiple R/O branches.
```shell
mkdir -p /tmp/mountpoint /tmp/writable
example/unionfs/unionfs /tmp/mountpoint /tmp/writable /usr &
ls /tmp/mountpoint
ls -l /tmp/mountpoint/bin/vi
rm /tmp/mountpoint/bin/vi
ls -l /tmp/mountpoint/bin/vi
cat /tmp/writable/DELETION/*
```
* `union/autounionfs.go`: creates UnionFs mounts automatically based on
existence of READONLY symlinks.
Tested on:
- x86 32bits (Fedora 14).
- x86 64bits (Ubuntu Lucid).
## Benchmarks
We use threaded stats over a read-only filesystem for benchmarking.
Automated code is under benchmark/ directory. A simple C version of
the same FS gives a FUSE baseline
Data points (Go-FUSE version May 2012), 1000 files, high level
interface, all kernel caching turned off, median stat time:
platform libfuse Go-FUSE difference (%)
Lenovo T60/Fedora16 (1cpu) 349us 355us 2% slower
Lenovo T400/Lucid (1cpu) 138us 140us 5% slower
Dell T3500/Lucid (1cpu) 72us 76us 5% slower
On T60, for each file we have
- Client side latency is 360us
- 106us of this is server side latency (4.5x lookup 23us, 1x getattr 4us)
- 16.5us is due to latency measurements.
- 3us is due to garbage collection.
## macOS Support
go-fuse works somewhat on OSX. Known limitations:
......@@ -134,12 +94,6 @@ Grep source code for TODO. Major topics:
* Missing support for `CUSE`, `BMAP`, `IOCTL`
* In the path API, renames are racy; See also:
http://sourceforge.net/mailarchive/message.php?msg_id=27550667
Don't use the path API if you care about correctness.
## License
Like Go, this library is distributed under the new BSD license. See
......
doc.go 0 → 100644
View file @ c0f14a4a
// Copyright 2019 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This is a repository containing Go bindings for writing FUSE file
// systems.
//
// Go to https://godoc.org/github.com/hanwen/go-fuse/fs for the
// in-depth documentation for this library.
//
// Older, deprecated APIs are available at
// https://godoc.org/github.com/hanwen/go-fuse/fuse/pathfs and
// https://godoc.org/github.com/hanwen/go-fuse/fuse/nodefs.
package lib
example/multizip/main.go
View file @ c0f14a4a
......@@ -34,7 +34,6 @@ func main() {
opts := fs.Options{
EntryTimeout: &sec,
AttrTimeout: &sec,
DefaultPermissions: true,
}
opts.Debug = *debug
server, err := fs.Mount(flag.Arg(0), root, &opts)
......
example/zipfs/main.go
View file @ c0f14a4a
......@@ -59,7 +59,6 @@ func main() {
opts := &fs.Options{
AttrTimeout: ttl,
EntryTimeout: ttl,
DefaultPermissions: true,
}
opts.Debug = *debug
server, err := fs.Mount(flag.Arg(0), root, opts)
......
fs/README.md 0 → 100644
View file @ c0f14a4a
Objective
=========
A high-performance FUSE API that minimizes pitfalls with writing
correct filesystems.
Decisions
=========
* Nodes contain references to their children. This is useful
because most filesystems will need to construct tree-like
structures.
* Nodes contain references to their parents. As a result, we can
derive the path for each Inode, and there is no need for a
separate PathFS.
* Nodes can be "persistent", meaning their lifetime is not under
control of the kernel. This is useful for constructing FS trees
in advance, rather than driven by LOOKUP.
* The NodeID for FS tree node must be defined on creation and are
immutable. By contrast, reusing NodeIds (eg. rsc/bazil FUSE, as
well as old go-fuse/fuse/nodefs) needs extra synchronization to
avoid races with notify and FORGET, and makes handling the inode
Generation more complicated.
* The mode of an Inode is defined on creation. Files cannot change
type during their lifetime. This also prevents the common error
of forgetting to return the filetype in Lookup/GetAttr.
* The NodeID (used for communicating with kernel) is equal to
Attr.Ino (value shown in Stat and Lstat return values.).
* No global treelock, to ensure scalability.
* Support for hard links. libfuse doesn't support this in the
high-level API. Extra care for race conditions is needed when
looking up the same file through different paths.
* do not issue Notify{Entry,Delete} as part of
AddChild/RmChild/MvChild: because NodeIDs are unique and
immutable, there is no confusion about which nodes are
invalidated, and the notification doesn't have to happen under
lock.
* Directory reading uses the DirStream. Semantics for rewinding
directory reads, and adding files after opening (but before
reading) are handled automatically. No support for directory
seeks.
* Method names are based on syscall names. Where there is no
syscall (eg. "open directory"), we bias towards writing
everything together (Opendir)
To do/To decide
=========
* Symlink []byte vs string.
fs/api.go
View file @ c0f14a4a
......@@ -4,10 +4,57 @@
// Package fs provides infrastructure to build tree-organized filesystems.
//
// A tree-organized filesystem is similar to UNIX or Plan 9 filesystem: it
// consists of nodes with each node being either a file or a directory. Files
// are located at tree leafs. A directory node can have other nodes as its
// children and refer to each child by name unique through the directory.
// Structure of a file system implementation
//
// To create a file system, you should first define types for the
// nodes of the file system tree.
//
// struct myNode {
// fs.Inode
// }
//
// // Node types must be InodeEmbedders
// var _ = (fs.InodeEmbedder)((*myNode)(nil))
//
// // Node types should implement some file system operations, eg. Lookup
// var _ = (fs.NodeLookuper)((*myNode)(nil))
//
// func (n *myNode) Lookup(ctx context.Context, name string, ... ) (*Inode, syscall.Errno) {
// ops := myNode{}
// return n.NewInode(ctx, &ops, fs.StableAttr{Mode: syscall.S_IFDIR}), 0
// }
//
// The method names are inspired on the system call names, so we have
// Listxattr rather than ListXAttr.
//
// the file system is mounted by calling mount on the root of the tree,
//
// server, err := fs.Mount("/tmp/mnt", &myNode{}, &fs.Options{})
// ..
// // start serving the file system
// server.Wait()
//
// Error handling
//
// All error reporting must use the syscall.Errno type. This is an
// integer with predefined error codes, where the value 0 (`OK`)
// should be used to indicate success.
//
// File system concepts
//
// The FUSE API is very similar to Linux' internal VFS API for
// defining file systems in the kernel. It is therefore useful to
// understand some terminology.
//
// File content: the raw bytes that we store inside regular files.
//
// Path: a /-separated string path that describes location of a node
// in the file system tree. For example
//
// dir1/file
//
// describes path root → dir1 → file.
//
// There can be several paths leading from tree root to a particular node,
// known as hard-linking, for example
//
......@@ -17,59 +64,106 @@
// \ /
// file
//
// A /-separated string path describes location of a node in the tree. For example
// Inode: ("index node") points to the file content, and stores
// metadata (size, timestamps) about a file or directory. Each
// inode has a type (directory, symlink, regular file, etc.) and
// an identity (a 64-bit number, unique to the file
// system). Directories can have children.
//
// dir1/file
// The inode in the kernel is represented in Go-FUSE as the Inode
// type.
//
// describes path root → dir1 → file.
// While common OS APIs are phrased in terms of paths (strings), the
// precise semantics of a file system are better described in terms of
// Inodes. This allows us to specify what happens in corner cases,
// such as writing data to deleted files.
//
// Each node is associated with integer ID uniquely identifying the node
// throughout filesystem. The tree-level structure of any filesystem is
// expressed through index-nodes (also known as "inode", see Inode) which
// describe parent/child relation in between nodes and node-ID association.
// File descriptor: a handle returned to opening a file. File
// descriptors always refer to a single inode.
//
// The filesystem nodes are struct that embed the Inode type, so they
// comply with the InodeEmbedder interface. They should be
// initialized by calling NewInode or NewPersistentInode before being
// manipulated further, eg.
// Dirent: a dirent maps (parent inode number, name string) tuple to
// child inode, thus representing a parent/child relation (or the
// absense thereof). Dirents do not have an equivalent type inside
// Go-FUSE, but the result of Lookup operation essentially is a
// dirent, which the kernel puts in a cache.
//
// type myNode struct {
// Inode
// }
//
// func (n *myNode) Lookup(ctx context.Context, name string, ... ) (*Inode, syscall.Errno) {
// child := myNode{}
// return n.NewInode(ctx, &myNode{}, StableAttr{Mode: syscall.S_IFDIR}), 0
// }
// Kernel caching
//
// The kernel caches several pieces of information from the FUSE process:
//
// 1. File contents: enabled with the fuse.FOPEN_KEEP_CACHE return flag
// in Open, manipulated with ReadCache and WriteCache, and invalidated
// with Inode.NotifyContent
//
// 2. File Attributes (size, mtime, etc.): controlled with the
// attribute timeout fields in fuse.AttrOut and fuse.EntryOut, which
// get be populated from Getattr and Lookup
//
// On mounting, the root InodeEmbedder is associated with root of the
// tree.
// 3. Directory entries (parent/child relations in the FS tree):
// controlled with the timeout fields in fuse.EntryOut, and
// invalidated with Inode.NotifyEntry and Inode.NotifyDelete.
//
// The kernel can evict inode data to free up memory. It does so by
// issuing FORGET calls. When a node has no children, and no kernel
// references, it is removed from the file system trees.
// Without Directory Entry timeouts, every operation on file "a/b/c"
// must first do lookups for "a", "a/b" and "a/b/c", which is
// expensive because of context switches between the kernel and the
// FUSE process.
//
// File system trees can also be constructed in advance. This is done
// by instantiating "persistent" inodes from the OnAdder
// implementation. Persistent inodes remain in memory even if the
// kernel has forgotten them. See zip_test.go for an example of how
// to do this.
// Unsuccessful entry lookups can also be cached by setting an entry
// timeout when Lookup returns ENOENT.
//
// File systems whose tree structures are on backing storage typically
// discover the file system tree on-demand, and if the kernel is tight
// on memory, parts of the tree are forgotten again. These file
// systems should implement Lookuper instead. The loopback file
// system created by `NewLoopbackRoot` provides a straightforward
// example.
// The libfuse C library specifies 1 second timeouts for both
// attribute and directory entries, but no timeout for negative
// entries. by default. This can be achieve in go-fuse by setting
// options on mount, eg.
//
// All error reporting must use the syscall.Errno type. The value 0
// (`OK`) should be used to indicate success. The method names are
// inspired on the system call names, so we have Listxattr rather than
// ListXAttr.
// sec := time.Second
// opts := fs.Options{
// EntryTimeout: &sec,
// AttrTimeout: &sec,
// }
//
// Locking
//
// Locks for networked filesystems are supported through the suite of
// Getlk, Setlk and Setlkw methods. They alllow locks on regions of
// regular files.
//
// Parallelism
//
// The VFS layer in the kernel is optimized to be highly parallel, and
// this parallelism also affects FUSE file systems: many FUSE
// operations can run in parallel, and this invites race
// conditions. It is strongly recommended to test your FUSE file
// system issuing file operations in parallel, and using the race
// detector to weed out data races.
//
// Dynamically discovered file systems
//
// File system data usually cannot fit all in RAM, so the kernel must
// discover the file system dynamically: as you are entering and list
// directory contents, the kernel asks the FUSE server about the files
// and directories you are busy reading/writing, and forgets parts of
// your file system when it is low on memory.
//
// The two important operations for dynamic file systems are:
// 1. Lookup, part of the NodeLookuper interface for discovering
// individual children of directories, and 2. Readdir, part of the
// NodeReaddirer interface for listing the contents of a directory.
//
// Static in-memory file systems
//
// For small, read-only file systems, getting the locking mechanics of
// Lookup correct is tedious, so Go-FUSE provides a feature to
// simplify building such file systems.
//
// Instead of discovering the FS tree on the fly, you can construct
// the entire tree from an OnAdd method. Then, that in-memory tree
// structure becomes the source of truth. This means you Go-FUSE must
// remember Inodes even if the kernel is no longer interested in
// them. This is done by instantiating "persistent" inodes from the
// OnAdd method of the root node. See the ZipFS example for a
// runnable example of how to do this.
package fs
import (
......@@ -87,8 +181,7 @@ import (
//
// In general, if an InodeEmbedder does not implement specific
// filesystem methods, the filesystem will react as if it is a
// read-only filesystem with a predefined tree structure. See
// zipfs_test.go for an example. A example is in zip_test.go
// read-only filesystem with a predefined tree structure.
type InodeEmbedder interface {
// populateInode and inode are used internally to link Inode
// to a Node.
......@@ -124,9 +217,12 @@ type NodeAccesser interface {
Access(ctx context.Context, mask uint32) syscall.Errno
}
// GetAttr reads attributes for an Inode. The library will
// ensure that Mode and Ino are set correctly. For regular
// files, Size should be set so it can be read correctly.
// GetAttr reads attributes for an Inode. The library will ensure that
// Mode and Ino are set correctly. For files that are not opened with
// FOPEN_DIRECTIO, Size should be set so it can be read correctly. If
// returning zeroed permissions, the default behavior is to change the
// mode of 0755 (directory) or 0644 (files). This can be switched off
// with the Options.NullPermissions setting.
type NodeGetattrer interface {
Getattr(ctx context.Context, f FileHandle, out *fuse.AttrOut) syscall.Errno
}
......@@ -203,10 +299,11 @@ type NodeFsyncer interface {
Fsync(ctx context.Context, f FileHandle, flags uint32) syscall.Errno
}
// Flush is called for close() call on a file descriptor. In
// case of duplicated descriptor, it may be called more than
// once for a file. The default implementation forwards to the
// FileHandle.
// Flush is called for the close(2) call on a file descriptor. In case
// of a descriptor that was duplicated using dup(2), it may be called
// more than once for the same FileHandle. The default implementation
// forwards to the FileHandle, or if the handle does not support
// FileFlusher, returns OK.
type NodeFlusher interface {
Flush(ctx context.Context, f FileHandle) syscall.Errno
}
......@@ -278,16 +375,31 @@ type DirStream interface {
Close()
}
// Lookup should find a direct child of the node by child
// name. If the entry does not exist, it should return ENOENT
// and optionally set a NegativeTimeout in `out`. If it does
// exist, it should return attribute data in `out` and return
// the Inode for the child. A new inode can be created using
// `Inode.NewInode`. The new Inode will be added to the FS
// tree automatically if the return status is OK.
// Lookup should find a direct child of a directory by the child's name. If
// the entry does not exist, it should return ENOENT and optionally
// set a NegativeTimeout in `out`. If it does exist, it should return
// attribute data in `out` and return the Inode for the child. A new
// inode can be created using `Inode.NewInode`. The new Inode will be
// added to the FS tree automatically if the return status is OK.
//
// If a directory does not implement NodeLookuper, the library looks
// for an existing child with the given name.
//
// The input to a Lookup is {parent directory, name string}.
//
// Lookup, if successful, must return an *Inode. Once the Inode is
// returned to the kernel, the kernel can issue further operations,
// such as Open or Getxattr on that node.
//
// A successful Lookup also returns an EntryOut. Among others, this
// contains file attributes (mode, size, mtime, etc.).
//
// FUSE supports other operations that modify the namespace. For
// example, the Symlink, Create, Mknod, Link methods all create new
// children in directories. Hence, they also return *Inode and must
// populate their fuse.EntryOut arguments.
//
// If not defined, we look for an existing child with the given name,
// or returns ENOENT.
type NodeLookuper interface {
Lookup(ctx context.Context, name string, out *fuse.EntryOut) (*Inode, syscall.Errno)
}
......@@ -302,8 +414,15 @@ type NodeOpendirer interface {
// ReadDir opens a stream of directory entries.
//
// The default ReadDir returns the list of currently known children
// from the tree
// Readdir essentiallly returns a list of strings, and it is allowed
// for Readdir to return different results from Lookup. For example,
// you can return nothing for Readdir ("ls my-fuse-mount" is empty),
// while still implementing Lookup ("ls my-fuse-mount/a-specific-file"
// shows a single file).
//
// If a directory does not implement NodeReaddirer, a list of
// currently known children from the tree is returned. This means that
// static in-memory file systems need not implement NodeReaddirer.
type NodeReaddirer interface {
Readdir(ctx context.Context) (DirStream, syscall.Errno)
}
......@@ -468,9 +587,11 @@ type Options struct {
// functionality of the root node.
OnAdd func(ctx context.Context)
// DefaultPermissions sets null file permissions to 755 (dirs)
// or 644 (other files.)
DefaultPermissions bool
// NullPermissions if set, leaves null file permissions
// alone. Otherwise, they are set to 755 (dirs) or 644 (other
// files.), which is necessary for doing a chdir into the FUSE
// directories.
NullPermissions bool
// If nonzero, replace default (zero) UID with the given UID
UID uint32
......
fs/bridge.go
View file @ c0f14a4a
......@@ -147,7 +147,7 @@ func (b *rawBridge) setEntryOutTimeout(out *fuse.EntryOut) {
}
func (b *rawBridge) setAttr(out *fuse.Attr) {
if b.options.DefaultPermissions && out.Mode&07777 == 0 {
if !b.options.NullPermissions && out.Mode&07777 == 0 {
out.Mode |= 0644
if out.Mode&syscall.S_IFDIR != 0 {
out.Mode |= 0111
......@@ -743,8 +743,7 @@ func (b *rawBridge) Flush(cancel <-chan struct{}, input *fuse.FlushIn) fuse.Stat
if fl, ok := f.file.(FileFlusher); ok {
return errnoToStatus(fl.Flush(&fuse.Context{Caller: input.Caller, Cancel: cancel}))
}
// XXX should return OK to reflect r/o filesystem?
return fuse.ENOTSUP
return 0
}
func (b *rawBridge) Fsync(cancel <-chan struct{}, input *fuse.FsyncIn) fuse.Status {
......
fs/directio_example_test.go 0 → 100644
View file @ c0f14a4a
// Copyright 2019 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs_test
import (
"context"
"fmt"
"log"
"syscall"
"time"
"github.com/hanwen/go-fuse/fs"
"github.com/hanwen/go-fuse/fuse"
)
// bytesFileHandle is a file handle that carries separate content for
// each Open call
type bytesFileHandle struct {
content []byte
}
// bytesFileHandle allows reads
var _ = (fs.FileReader)((*bytesFileHandle)(nil))
func (fh *bytesFileHandle) Read(ctx context.Context, dest []byte, off int64) (fuse.ReadResult, syscall.Errno) {
end := off + int64(len(dest))
if end > int64(len(fh.content)) {
end = int64(len(fh.content))
}
// We could copy to the `dest` buffer, but since we have a
// []byte already, return that.
return fuse.ReadResultData(fh.content[off:end]), 0
}
// timeFile is a file that contains the wall clock time as ASCII.
type timeFile struct {
fs.Inode
}
// timeFile implements Open
var _ = (fs.NodeOpener)((*timeFile)(nil))
func (f *timeFile) Open(ctx context.Context, openFlags uint32) (fh fs.FileHandle, fuseFlags uint32, errno syscall.Errno) {
// disallow writes
if fuseFlags&(syscall.O_RDWR|syscall.O_WRONLY) != 0 {
return nil, 0, syscall.EROFS
}
// capture open time
now := time.Now().Format(time.StampNano) + "\n"
fh = &bytesFileHandle{
content: []byte(now),
}
// Return FOPEN_DIRECT_IO so content is not cached.
return fh, fuse.FOPEN_DIRECT_IO, 0
}
// ExampleDirectIO shows how to create a file whose contents change on
// every read.
func Example_directIO() {
mntDir := "/tmp/x"
root := &fs.Inode{}
// Mount the file system
server, err := fs.Mount(mntDir, root, &fs.Options{
MountOptions: fuse.MountOptions{Debug: false},
// Setup the clock file.
OnAdd: func(ctx context.Context) {
ch := root.NewPersistentInode(
ctx,
&timeFile{},
fs.StableAttr{Mode: syscall.S_IFREG})
root.AddChild("clock", ch, true)
},
})
if err != nil {
log.Fatal(err)
}
fmt.Printf("cat %s/clock to see the time\n", mntDir)
fmt.Printf("Unmount by calling 'fusermount -u %s'\n", mntDir)
// Serve the file system, until unmounted by calling fusermount -u
server.Wait()
}
fs/dynamic_example_test.go 0 → 100644
View file @ c0f14a4a
// Copyright 2019 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs_test
import (
"context"
"log"
"os"
"strconv"
"syscall"
"github.com/hanwen/go-fuse/fs"
"github.com/hanwen/go-fuse/fuse"
)
// numberNode is a filesystem node representing an integer. Prime
// numbers are regular files, while composite numbers are directories
// containing all smaller numbers, eg.
//
// $ ls -F /tmp/x/6
// 2 3 4/ 5
//
// the file system nodes are deduplicated using inode numbers. The
// number 2 appears in many directories, but it is actually the represented
// by the same numberNode{} object, with inode number 2.
//
// $ ls -i1 /tmp/x/2 /tmp/x/8/6/4/2
// 2 /tmp/x/2
// 2 /tmp/x/8/6/4/2
//
type numberNode struct {
// Must embed an Inode for the struct to work as a node.
fs.Inode
// num is the integer represented in this file/directory
num int
}
// isPrime returns whether n is prime
func isPrime(n int) bool {
for i := 2; i*i <= n; i++ {
if n%i == 0 {
return false
}
}
return true
}
func numberToMode(n int) uint32 {
// prime numbers are files
if isPrime(n) {
return fuse.S_IFREG
}
// composite numbers are directories
return fuse.S_IFDIR
}
// Ensure we are implementing the NodeReaddirer interface
var _ = (fs.NodeReaddirer)((*numberNode)(nil))
// Readdir is part of the NodeReaddirer interface
func (n *numberNode) Readdir(ctx context.Context) (fs.DirStream, syscall.Errno) {
r := make([]fuse.DirEntry, 0, n.num)
for i := 2; i < n.num; i++ {
d := fuse.DirEntry{
Name: strconv.Itoa(i),
Ino: uint64(i),
Mode: numberToMode(i),
}
r = append(r, d)
}
return fs.NewListDirStream(r), 0
}
// Ensure we are implementing the NodeLookuper interface
var _ = (fs.NodeLookuper)((*numberNode)(nil))
// Lookup is part of the NodeLookuper interface
func (n *numberNode) Lookup(ctx context.Context, name string, out *fuse.EntryOut) (*fs.Inode, syscall.Errno) {
i, err := strconv.Atoi(name)
if err != nil {
return nil, syscall.ENOENT
}
if i >= n.num || i <= 1 {
return nil, syscall.ENOENT
}
stable := fs.StableAttr{
Mode: numberToMode(i),
// The child inode is identified by its Inode number.
// If multiple concurrent lookups try to find the same
// inode, they are deduplicated on this key.
Ino: uint64(i),
}
operations := &numberNode{num: i}
// The NewInode call wraps the `operations` object into an Inode.
child := n.NewInode(ctx, operations, stable)
// In case of concurrent lookup requests, it can happen that operations !=
// child.Operations().
return child, 0
}
// ExampleDynamic is a whimsical example of a dynamically discovered
// file system.
func Example_dynamic() {
// This is where we'll mount the FS
mntDir := "/tmp/x"
os.Mkdir(mntDir, 0755)
root := &numberNode{num: 10}
server, err := fs.Mount(mntDir, root, &fs.Options{
MountOptions: fuse.MountOptions{
// Set to true to see how the file system works.
Debug: true,
},
})
if err != nil {
log.Panic(err)
}
log.Printf("Mounted on %s", mntDir)
log.Printf("Unmount by calling 'fusermount -u %s'", mntDir)
// Wait until unmount before exiting
server.Wait()
}
fs/example_test.go
View file @ c0f14a4a
......@@ -14,37 +14,30 @@ import (
"github.com/hanwen/go-fuse/fuse"
)
// mountLoopback mounts dir under the given mountpoint
func mountLoopback(dir, mntPoint string) (*fuse.Server, error) {
root, err := fs.NewLoopbackRoot(dir)
// ExampleMount shows how to create a loopback file system, and
// mounting it onto a directory
func Example_mount() {
mntDir, _ := ioutil.TempDir("", "")
home := os.Getenv("HOME")
// Make $HOME available on a mount dir under /tmp/ . Caution:
// write operations are also mirrored.
root, err := fs.NewLoopbackRoot(home)
if err != nil {
return nil, err
log.Fatal(err)
}
// Make the root available under mntDir
return fs.Mount(mntPoint, root, &fs.Options{
// Mount the file system
server, err := fs.Mount(mntDir, root, &fs.Options{
MountOptions: fuse.MountOptions{Debug: true},
})
}
// An example of creating a loopback file system, and mounting it onto
// a directory
func Example_mountLoopback() {
mntDir, _ := ioutil.TempDir("", "")
home := os.Getenv("HOME")
// Make $HOME available on a mount dir under /tmp/ . Caution:
// write operations are also mirrored.
server, err := mountLoopback(mntDir, home)
if err != nil {
log.Panic(err)
log.Fatal(err)
}
fmt.Printf("Mounted %s as loopback on %s\n", home, mntDir)
fmt.Printf("\n\nCAUTION:\nwrite operations on %s will also affect $HOME (%s)\n\n", mntDir, home)
fmt.Printf("Unmount by calling 'fusermount -u %s'\n", mntDir)
// Wait until the directory is unmounted
// Serve the file system, until unmounted by calling fusermount -u
server.Wait()
}
fs/inmemory_example_test.go
View file @ c0f14a4a
......@@ -80,11 +80,6 @@ func Example() {
root := &inMemoryFS{}
server, err := fs.Mount(mntDir, root, &fs.Options{
MountOptions: fuse.MountOptions{Debug: true},
// This adds read permissions to the files and
// directories, which is necessary for doing a chdir
// into the mount.
DefaultPermissions: true,
})
if err != nil {
log.Panic(err)
......
fs/inode.go
View file @ c0f14a4a
......@@ -316,6 +316,15 @@ func (iparent *Inode) setEntry(name string, ichild *Inode) {
// NewPersistentInode returns an Inode whose lifetime is not in
// control of the kernel.
//
// When the kernel is short on memory, it will forget cached file
// system information (directory entries and inode metadata). This is
// announced with FORGET messages. There are no guarantees if or when
// this happens. When it happens, these are handled transparently by
// go-fuse: all Inodes created with NewInode are released
// automatically. NewPersistentInode creates inodes that go-fuse keeps
// in memory, even if the kernel is not interested in them. This is
// convenient for building static trees up-front.
func (n *Inode) NewPersistentInode(ctx context.Context, node InodeEmbedder, id StableAttr) *Inode {
return n.newInode(ctx, node, id, true)
}
......@@ -474,18 +483,6 @@ func (n *Inode) Children() map[string]*Inode {
return r
}
// Parents returns the parents of this Inode, along with the name
// with which they're are a child
func (n *Inode) Parents() map[string]*Inode {
n.mu.Lock()
defer n.mu.Unlock()
r := make(map[string]*Inode, len(n.parents))
for k := range n.parents {
r[k.name] = k.parent
}
return r
}
// Parents returns a parent of this Inode, or nil if this Inode is
// deleted or is the root
func (n *Inode) Parent() (string, *Inode) {
......
fs/readonly_test.go
View file @ c0f14a4a
......@@ -29,7 +29,6 @@ func TestDefaultPermissions(t *testing.T) {
root := &Inode{}
mntDir, _, clean := testMount(t, root, &Options{
DefaultPermissions: true,
OnAdd: func(ctx context.Context) {
dir := root.NewPersistentInode(ctx, &Inode{}, StableAttr{Mode: syscall.S_IFDIR})
file := root.NewPersistentInode(ctx, &Inode{}, StableAttr{Mode: syscall.S_IFREG})
......
fs/readwrite_handleless_example_test.go 0 → 100644
View file @ c0f14a4a
// Copyright 2019 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs_test
import (
"context"
"fmt"
"log"
"sync"
"syscall"
"time"
"github.com/hanwen/go-fuse/fs"
"github.com/hanwen/go-fuse/fuse"
)
// bytesNode is a file that can be read and written
type bytesNode struct {
fs.Inode
// When file systems are mutable, all access must use
// synchronization.
mu sync.Mutex
content []byte
mtime time.Time
}
// Implement GetAttr to provide size and mtime
var _ = (fs.NodeGetattrer)((*bytesNode)(nil))
func (bn *bytesNode) Getattr(ctx context.Context, fh fs.FileHandle, out *fuse.AttrOut) syscall.Errno {
bn.mu.Lock()
defer bn.mu.Unlock()
bn.getattr(out)
return 0
}
func (bn *bytesNode) getattr(out *fuse.AttrOut) {
out.Size = uint64(len(bn.content))
out.SetTimes(nil, &bn.mtime, nil)
}
func (bn *bytesNode) resize(sz uint64) {
if sz > uint64(cap(bn.content)) {
n := make([]byte, sz)
copy(n, bn.content)
bn.content = n
} else {
bn.content = bn.content[:sz]
}
bn.mtime = time.Now()
}
// Implement Setattr to support truncation
var _ = (fs.NodeSetattrer)((*bytesNode)(nil))
func (bn *bytesNode) Setattr(ctx context.Context, fh fs.FileHandle, in *fuse.SetAttrIn, out *fuse.AttrOut) syscall.Errno {
bn.mu.Lock()
defer bn.mu.Unlock()
if sz, ok := in.GetSize(); ok {
bn.resize(sz)
}
bn.getattr(out)
return 0
}
// Implement handleless read.
var _ = (fs.NodeReader)((*bytesNode)(nil))
func (bn *bytesNode) Read(ctx context.Context, fh fs.FileHandle, dest []byte, off int64) (fuse.ReadResult, syscall.Errno) {
bn.mu.Lock()
defer bn.mu.Unlock()
end := off + int64(len(dest))
if end > int64(len(bn.content)) {
end = int64(len(bn.content))
}
// We could copy to the `dest` buffer, but since we have a
// []byte already, return that.
return fuse.ReadResultData(bn.content[off:end]), 0
}
// Implement handleless write.
var _ = (fs.NodeWriter)((*bytesNode)(nil))
func (bn *bytesNode) Write(ctx context.Context, fh fs.FileHandle, buf []byte, off int64) (uint32, syscall.Errno) {
bn.mu.Lock()
defer bn.mu.Unlock()
sz := int64(len(buf))
if off+sz > int64(len(bn.content)) {
bn.resize(uint64(off + sz))
}
copy(bn.content[off:], buf)
bn.mtime = time.Now()
return uint32(sz), 0
}
// Implement (handleless) Open
var _ = (fs.NodeOpener)((*bytesNode)(nil))
func (f *bytesNode) Open(ctx context.Context, openFlags uint32) (fh fs.FileHandle, fuseFlags uint32, errno syscall.Errno) {
return nil, 0, 0
}
// ExampleHandleLess shows how to create a file that can be read or written
// by implementing Read/Write directly on the nodes.
func Example_handleLess() {
mntDir := "/tmp/x"
root := &fs.Inode{}
// Mount the file system
server, err := fs.Mount(mntDir, root, &fs.Options{
MountOptions: fuse.MountOptions{Debug: false},
// Setup the file.
OnAdd: func(ctx context.Context) {
ch := root.NewPersistentInode(
ctx,
&bytesNode{},
fs.StableAttr{
Mode: syscall.S_IFREG,
// Make debug output readable.
Ino: 2,
})
root.AddChild("bytes", ch, true)
},
})
if err != nil {
log.Fatal(err)
}
fmt.Printf(`Try:
cd %s
ls -l bytes
echo hello > bytes
ls -l bytes
cat bytes
cd -
`, mntDir)
fmt.Printf("Unmount by calling 'fusermount -u %s'\n", mntDir)
// Serve the file system, until unmounted by calling fusermount -u
server.Wait()
}
fs/zip_test.go
View file @ c0f14a4a
......@@ -11,13 +11,10 @@ import (
"io/ioutil"
"path/filepath"
"reflect"
"strings"
"sync"
"syscall"
"testing"
"github.com/hanwen/go-fuse/fs"
"github.com/hanwen/go-fuse/fuse"
)
var testData = map[string]string{
......@@ -134,91 +131,3 @@ func TestZipFSOnAdd(t *testing.T) {
t.Errorf("got %q, want %q", got, want)
}
}
// zipFile is a file read from a zip archive.
type zipFile struct {
fs.Inode
file *zip.File
mu sync.Mutex
data []byte
}
var _ = (fs.NodeOpener)((*zipFile)(nil))
var _ = (fs.NodeGetattrer)((*zipFile)(nil))
// Getattr sets the minimum, which is the size. A more full-featured
// FS would also set timestamps and permissions.
func (zf *zipFile) Getattr(ctx context.Context, f fs.FileHandle, out *fuse.AttrOut) syscall.Errno {
out.Size = zf.file.UncompressedSize64
return 0
}
// Open lazily unpacks zip data
func (zf *zipFile) Open(ctx context.Context, flags uint32) (fs.FileHandle, uint32, syscall.Errno) {
zf.mu.Lock()
defer zf.mu.Unlock()
if zf.data == nil {
rc, err := zf.file.Open()
if err != nil {
return nil, 0, syscall.EIO
}
content, err := ioutil.ReadAll(rc)
if err != nil {
return nil, 0, syscall.EIO
}
zf.data = content
}
// We don't return a filehandle since we don't really need
// one. The file content is immutable, so hint the kernel to
// cache the data.
return nil, fuse.FOPEN_KEEP_CACHE, fs.OK
}
// Read simply returns the data that was already unpacked in the Open call
func (zf *zipFile) Read(ctx context.Context, f fs.FileHandle, dest []byte, off int64) (fuse.ReadResult, syscall.Errno) {
end := int(off) + len(dest)
if end > len(zf.data) {
end = len(zf.data)
}
return fuse.ReadResultData(zf.data[off:end]), fs.OK
}
// zipRoot is the root of the Zip filesystem. Its only functionality
// is populating the filesystem.
type zipRoot struct {
fs.Inode
zr *zip.Reader
}
var _ = (fs.NodeOnAdder)((*zipRoot)(nil))
func (zr *zipRoot) OnAdd(ctx context.Context) {
// OnAdd is called once we are attached to an Inode. We can
// then construct a tree. We construct the entire tree, and
// we don't want parts of the tree to disappear when the
// kernel is short on memory, so we use persistent inodes.
for _, f := range zr.zr.File {
dir, base := filepath.Split(f.Name)
p := &zr.Inode
for _, component := range strings.Split(dir, "/") {
if len(component) == 0 {
continue
}
ch := p.GetChild(component)
if ch == nil {
ch = p.NewPersistentInode(ctx, &fs.Inode{},
fs.StableAttr{Mode: fuse.S_IFDIR})
p.AddChild(component, ch, true)
}
p = ch
}
ch := p.NewPersistentInode(ctx, &zipFile{file: f}, fs.StableAttr{})
p.AddChild(base, ch, true)
}
}
fs/zipfs_example_test.go 0 → 100644
View file @ c0f14a4a
// Copyright 2019 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs_test
import (
"archive/zip"
"context"
"flag"
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
"strings"
"sync"
"syscall"
"github.com/hanwen/go-fuse/fs"
"github.com/hanwen/go-fuse/fuse"
)
// zipFile is a file read from a zip archive.
type zipFile struct {
fs.Inode
file *zip.File
mu sync.Mutex
data []byte
}
// We decompress the file on demand in Open
var _ = (fs.NodeOpener)((*zipFile)(nil))
// Getattr sets the minimum, which is the size. A more full-featured
// FS would also set timestamps and permissions.
var _ = (fs.NodeGetattrer)((*zipFile)(nil))
func (zf *zipFile) Getattr(ctx context.Context, f fs.FileHandle, out *fuse.AttrOut) syscall.Errno {
out.Size = zf.file.UncompressedSize64
return 0
}
// Open lazily unpacks zip data
func (zf *zipFile) Open(ctx context.Context, flags uint32) (fs.FileHandle, uint32, syscall.Errno) {
zf.mu.Lock()
defer zf.mu.Unlock()
if zf.data == nil {
rc, err := zf.file.Open()
if err != nil {
return nil, 0, syscall.EIO
}
content, err := ioutil.ReadAll(rc)
if err != nil {
return nil, 0, syscall.EIO
}
zf.data = content
}
// We don't return a filehandle since we don't really need
// one. The file content is immutable, so hint the kernel to
// cache the data.
return nil, fuse.FOPEN_KEEP_CACHE, fs.OK
}
// Read simply returns the data that was already unpacked in the Open call
func (zf *zipFile) Read(ctx context.Context, f fs.FileHandle, dest []byte, off int64) (fuse.ReadResult, syscall.Errno) {
end := int(off) + len(dest)
if end > len(zf.data) {
end = len(zf.data)
}
return fuse.ReadResultData(zf.data[off:end]), fs.OK
}
// zipRoot is the root of the Zip filesystem. Its only functionality
// is populating the filesystem.
type zipRoot struct {
fs.Inode
zr *zip.Reader
}
// The root populates the tree in its OnAdd method
var _ = (fs.NodeOnAdder)((*zipRoot)(nil))
func (zr *zipRoot) OnAdd(ctx context.Context) {
// OnAdd is called once we are attached to an Inode. We can
// then construct a tree. We construct the entire tree, and
// we don't want parts of the tree to disappear when the
// kernel is short on memory, so we use persistent inodes.
for _, f := range zr.zr.File {
dir, base := filepath.Split(f.Name)
p := &zr.Inode
for _, component := range strings.Split(dir, "/") {
if len(component) == 0 {
continue
}
ch := p.GetChild(component)
if ch == nil {
ch = p.NewPersistentInode(ctx, &fs.Inode{},
fs.StableAttr{Mode: fuse.S_IFDIR})
p.AddChild(component, ch, true)
}
p = ch
}
ch := p.NewPersistentInode(ctx, &zipFile{file: f}, fs.StableAttr{})
p.AddChild(base, ch, true)
}
}
// ExampleZipFS shows an in-memory, static file system
func Example_zipFS() {
flag.Parse()
if len(flag.Args()) != 1 {
log.Fatal("usage: zipmount ZIP-FILE")
}
zfile, err := zip.OpenReader(flag.Arg(0))
if err != nil {
log.Fatal(err)
}
root := &zipRoot{zr: &zfile.Reader}
mnt := "/tmp/x"
os.Mkdir(mnt, 0755)
server, err := fs.Mount(mnt, root, nil)
if err != nil {
log.Fatal(err)
}
fmt.Println("zip file mounted")
fmt.Printf("to unmount: fusermount -u %s\n", mnt)
server.Wait()
}
fuse/test/loopback_test.go
View file @ c0f14a4a
......@@ -853,7 +853,7 @@ func TestIoctl(t *testing.T) {
// This test is racy. If an external process consumes space while this
// runs, we may see spurious differences between the two statfs() calls.
func TestStatFs(t *testing.T) {
func TestNonVerboseStatFs(t *testing.T) {
tc := NewTestCase(t)
defer tc.Cleanup()
......@@ -875,7 +875,7 @@ func TestStatFs(t *testing.T) {
}
}
func TestFStatFs(t *testing.T) {
func TestNonVerboseFStatFs(t *testing.T) {
tc := NewTestCase(t)
defer tc.Cleanup()
......
internal/testutil/verbose.go
View file @ c0f14a4a
......@@ -4,10 +4,20 @@
package testutil
import "flag"
import (
"bytes"
"flag"
"runtime"
)
// VerboseTest returns true if the testing framework is run with -v.
func VerboseTest() bool {
var buf [2048]byte
n := runtime.Stack(buf[:], false)
if bytes.Index(buf[:n], []byte("TestNonVerbose")) != -1 {
return false
}
flag := flag.Lookup("test.v")
return flag != nil && flag.Value.String() == "true"
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7