• Chao Yu's avatar
    f2fs: support data compression · 4c8ff709
    Chao Yu authored
    This patch tries to support compression in f2fs.
    
    - New term named cluster is defined as basic unit of compression, file can
    be divided into multiple clusters logically. One cluster includes 4 << n
    (n >= 0) logical pages, compression size is also cluster size, each of
    cluster can be compressed or not.
    
    - In cluster metadata layout, one special flag is used to indicate cluster
    is compressed one or normal one, for compressed cluster, following metadata
    maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
    data including compress header and compressed data.
    
    - In order to eliminate write amplification during overwrite, F2FS only
    support compression on write-once file, data can be compressed only when
    all logical blocks in file are valid and cluster compress ratio is lower
    than specified threshold.
    
    - To enable compression on regular inode, there are three ways:
    * chattr +c file
    * chattr +c dir; touch dir/file
    * mount w/ -o compress_extension=ext; touch file.ext
    
    Compress metadata layout:
                                 [Dnode Structure]
                 +-----------------------------------------------+
                 | cluster 1 | cluster 2 | ......... | cluster N |
                 +-----------------------------------------------+
                 .           .                       .           .
           .                       .                .                      .
      .         Compressed Cluster       .        .        Normal Cluster            .
    +----------+---------+---------+---------+  +---------+---------+---------+---------+
    |compr flag| block 1 | block 2 | block 3 |  | block 1 | block 2 | block 3 | block 4 |
    +----------+---------+---------+---------+  +---------+---------+---------+---------+
               .                             .
             .                                           .
           .                                                           .
          +-------------+-------------+----------+----------------------------+
          | data length | data chksum | reserved |      compressed data       |
          +-------------+-------------+----------+----------------------------+
    
    Changelog:
    
    20190326:
    - fix error handling of read_end_io().
    - remove unneeded comments in f2fs_encrypt_one_page().
    
    20190327:
    - fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
    - don't jump into loop directly to avoid uninitialized variables.
    - add TODO tag in error path of f2fs_write_cache_pages().
    
    20190328:
    - fix wrong merge condition in f2fs_read_multi_pages().
    - check compressed file in f2fs_post_read_required().
    
    20190401
    - allow overwrite on non-compressed cluster.
    - check cluster meta before writing compressed data.
    
    20190402
    - don't preallocate blocks for compressed file.
    
    - add lz4 compress algorithm
    - process multiple post read works in one workqueue
      Now f2fs supports processing post read work in multiple workqueue,
      it shows low performance due to schedule overhead of multiple
      workqueue executing orderly.
    
    20190921
    - compress: support buffered overwrite
    C: compress cluster flag
    V: valid block address
    N: NEW_ADDR
    
    One cluster contain 4 blocks
    
     before overwrite   after overwrite
    
    - VVVV		->	CVNN
    - CVNN		->	VVVV
    
    - CVNN		->	CVNN
    - CVNN		->	CVVV
    
    - CVVV		->	CVNN
    - CVVV		->	CVVV
    
    20191029
    - add kconfig F2FS_FS_COMPRESSION to isolate compression related
    codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
    note that: will remove lzo backend if Jaegeuk agreed that too.
    - update codes according to Eric's comments.
    
    20191101
    - apply fixes from Jaegeuk
    
    20191113
    - apply fixes from Jaegeuk
    - split workqueue for fsverity
    
    20191216
    - apply fixes from Jaegeuk
    
    20200117
    - fix to avoid NULL pointer dereference
    
    [Jaegeuk Kim]
    - add tracepoint for f2fs_{,de}compress_pages()
    - fix many bugs and add some compression stats
    - fix overwrite/mmap bugs
    - address 32bit build error, reported by Geert.
    - bug fixes when handling errors and i_compressed_blocks
    
    Reported-by: <noreply@ellerman.id.au>
    Signed-off-by: default avatarChao Yu <yuchao0@huawei.com>
    Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
    4c8ff709
data.c 90.7 KB