Initial version of utility able to check caching

configuration of websites with help of Varnish.

This script use wget to crawl the website and inspect all Status code,
then headers returns by server.

This script parse also varnish logs to know if client's query goes to the backend
or not.

This utility is still under developement, it's released under beta version
for testing purpose only.


git-svn-id: https://svn.erp5.org/repos/public/erp5/trunk/utils/erp5.utils.web_checker/@37439 20353a03-c40f-0410-a6d1-a30d3c3de9de

erp5/util/CHANGES.webchecker.txt

+2010-08-03 Nicolas Delaby
+-------------------------
+First release
\ No newline at end of file

erp5/util/README.webchecker.txt

+Utility able to call wget and varnishlog to extract Headers and return all failures
+according expected caching policy.
+
+This utility is configurable through a dictionnary like
+   configuration = {'url': 'http://www.example.com',
+                   'working_directory': '/home/me/tmp/crawled_content',
+                   'varnishlog_binary_path': 'varnishlog',
+                   'header_list': {'Last-Modified': True,
+                                   'Cache-Control': ('max-age=300', 'max-age=3600',),
+                                   'Vary' : ('Accept-Language, Cookie', 'Accept-Language,Cookie'),
+                                   'Expires': True,
+                                  },
+                   'email_address': 'me@example.com',
+                   'smtp_host': 'localhost',
+                   'debug_level': 'debug',
+                  }
+
+url : website to check
+working_directory : fetched data will be downloaded
+varnishlog_binary_path :  path to varnishlog
+header_list : Key == Header id.
+              value: if equals to True, it means that header needs to be present in RESPONSE
+                     if it is a tuple, the Header value must sastify at least one of the proposed values
+email_address : email address to send result
+smtp_host : smtp host to use
+debug_level : log level of this utility (debug =>very verbose,
+                                         info=>normal,
+                                         warning=>nothing)
+
+This utility requires wget => 1.12
+And a callable varnishlog.
+The utility must be run on same server where varnish is running.
+
+web_checker reads varnishlogs to detect if a Query goes to the backend.