• Kirill Smelkov's avatar
    bigfile: Basic benchmarks · bb9d8bf1
    Kirill Smelkov authored
        - for virtual memory subsytem
        - for ZBigFiles
    
    They are not currently great, e.g. for virtmem we have in-kernel
    overhead of page clearing - in perf profiles, for bigfile_mmap compared
    to file_read kernel's clear_page_c raises significantly.
    
    That is the worker for clearing page memory and we currently cannot
    avoid that - any memory obtained from kernel (MAP_ANONYMOUS, mmap(file)
    with hole, etc...) comes pre-initialized to zeros to userspace.
    
    This can be seen in the benchmarks as well: file_readbig differs from
    file_read in only that the latter uses 1 small buffer and the first
    allocates large memory (cleared by kernel + python does the memset).
    
        bigfile/tests/bench_virtmem.py@125::bench_file_mmap_adler32     0.47  (0.86 0.49 0.47)
        bigfile/tests/bench_virtmem.py@126::bench_file_read_adler32     0.69  (1.11 0.71 0.69)
        bigfile/tests/bench_virtmem.py@127::bench_file_readbig_adler32  1.41  (1.70 1.42 1.41)
        bigfile/tests/bench_virtmem.py@128::bench_bigfile_mmap_adler32  1.42  (1.45 1.42 1.51)
    
        bigfile/tests/bench_virtmem.py@130::bench_file_mmap_md5         1.52  (1.91 1.54 1.52)
        bigfile/tests/bench_virtmem.py@131::bench_file_read_md5         1.73  (2.10 1.75 1.73)
        bigfile/tests/bench_virtmem.py@132::bench_file_readbig_md5      2.44  (2.73 2.46 2.44)
        bigfile/tests/bench_virtmem.py@133::bench_bigfile_mmap_md5      2.40  (2.48 2.40 2.53)
    
    There is MAP_UNINITIALIZED which works only for non-mmu targets and only
    if explicitly allowed when configuring kernel (off by default).
    
    There were patches to disable that pages zeroing, as it gives
    significant speedup for people's workloads, e.g. [1,2] but all of them
    did not got merged for security reasons.
    
    [1] http://marc.info/?t=132691315900001&r=1&w=2
    [2] http://thread.gmane.org/gmane.linux.kernel/548926
    
    ~~~~
    
    For ZBigFile - it is the storage who is dominating in profiles.
    bb9d8bf1
bench_1filezodb.py 1.67 KB