BigFile: Basic tests
So far BigFile was not unit-tested and because of recent BigFile patches
and fixes Romain suggested to write tests for it.
We test:
- working with BigFile via its public interface:
* GET/PUT both in plain and with ranges variants,
*.getData()/.getSize(), and
* recently-introduced ._appendData()
- that BigFile correctly handles situations where .data is either
None or str or BTreeData and that migration automatically happens
to BTreeData on append.
~~~~
Unlike common case, BigFile more directly works on REQUEST and RESPONSE
(instead of plain object publishing), so to test it we need not only call
methods and compare return values but first prepare proper
request/response, set them up and analyze response headers and content
after method invocation happened.
For preparing request/response Zope provides utility
Testing.makerequest() and its 2 variations but for our case they all
turned out to be not flexible enough - e.g. Testing.makerequest()
hardcodes request stdin=sys.stdin
https://github.com/zopefoundation/Zope/blob/master/src/Testing/makerequest.py#L56
(and we need to provide it to e.g. upload via PUT), makerequest from
Products.CMFCore.tests.test_CookieCrumbler hardcodes request environment
http://svn.zope.org/Products.CMFCore/branches/2.2/Products/CMFCore/tests/test_CookieCrumbler.py?revision=126491&view=markup
(and we need it for convenient way to set request headers), etc, so first we
introduce our own makerequest-alike that
1. always redirects stdout to stringio,
2. stdin content can be specified and is processed,
3. returns actual request object (not wrapped portal).
on top of that we introduce two convenience helpers GET and PUT to prepare
same-named request and then a function to generally invoke a request on object
and check results - i.e. given object and request, find appropriate method,
call it appropriately, verify return value, http status code, response body and
check asserted headers. All that in one line - to keep signal-to-noise ratio high.
~~~~
There are still some things to fix and improve:
- Zope translates 308 http status code (which BigFile PUT with range
query returns) to 500 because that code is experimental:
https://github.com/zopefoundation/Zope/blob/master/src/ZPublisher/HTTPResponse.py#L226
https://github.com/zopefoundation/Zope/blob/master/src/ZPublisher/HTTPResponse.py#L64
- It is not clear (to me) what PUT range query should return for
empty file. In HTTP/1.1 ranges are specified as both start and end
inclusive so currently for empty-file case BigFile code returns "0--1"
(= "0" - "-1") but that is not valid according to HTTP/1.1 spec
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.1
and again, judging from spec, it is not clear how to represent
range "empty".
For now I've left "0--1" checked as correct, but left a note in
tests this is dubiously so.
- Support for 'If-Range' and multiple ranges in 'Range' headers is
not tested.
- There are no scalability tests, i.e. "let's write a lot of data
into BigFile and see how underlying BTreeData behaves"
So all it is is basic tests so we know general BigFile logic and
interface work.
Test is done as a "live test" under erp5_big_file bt5 as per Sebastien
suggestion.
Helped-by: 
Sebastien Robin <seb@nexedi.com>
Showing
This diff is collapsed.
Please register or sign in to comment