- 03 Apr, 2015 10 commits
-
-
Kirill Smelkov authored
Exposes BigFile - this way users can define BigFile backend in Python. Also exposed are BigFile handles, and VMA objects which are results of mmaping.
-
Kirill Smelkov authored
Does similar things to what kernel does - users can mmap file parts into address space and access them read/write. The manager will be getting invoked by hardware/OS kernel for cases when there is no page loaded for read, or when a previousle read-only page is being written to. Additionally to features provided in kernel, it support to be used to store back changes in transactional way (see fileh_dirty_writeout()) and potentially use huge pages for mappings (though this is currently TODO)
-
Kirill Smelkov authored
Users can inherit from BigFile and provide custom ->loadblk() and ->storeblk() to load/store file blocks from a database or some other storage. The system then could use such files to memory map them into user address space (see next patch).
-
Kirill Smelkov authored
This thing allows to get aliasable RAM from OS kernel and to manage it. Currently we get memory from a tmpfs mount, and hugetlbfs should also work, but is TODO because hugetlbfs in the kernel needs to be improved. We need aliasing because we'll need to be able to memory map the same page into several places in address space, e.g. for taking two slices overlapping slice of the same array at different times. Comes with test programs that show we aliasing does not work for anonymous memory.
-
Kirill Smelkov authored
This will be the core of virtual memory subsystem. For now we just define a structure to describe pages of memory and add utility to allocate address space from OS.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
For BigFiles we'll needs to maintain `{} offset-in-file -> void *` mapping. A hash or a binary tree could be used there, but since we know files are most of the time accessed sequentially and locally in pages-batches, we can also organize the mapping in batches of keys. Specifically offset bits are so divided into parts, that every part addresses 1 entry in a table of hardware-page in size. To get to the actual value, the system lookups first table by first part of offset, then from first table and next part from address - second table, etc. To clients this looks like a dictionary with get/set/del & clear methods, but lookups are O(1) time always, and in contrast to hashes values are stored with locality (= adjacent lookups almost always access the same tables).
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Like taking an exact integer log2, upcasting pointers for C-style inheritance done in a Plan9 way, and wrappers to functions which should never fail.
-
Kirill Smelkov authored
Modelled by ones used in Linux kernel.
-