Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
A
apachedex
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Arnaud Fontaine
apachedex
Commits
e03a7ae0
Commit
e03a7ae0
authored
Apr 08, 2013
by
Vincent Pelletier
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Add support for bzip2 and xz compression.
parent
a1d86d67
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
38 additions
and
8 deletions
+38
-8
README
README
+12
-2
TODO
TODO
+3
-0
apachedex/__init__.py
apachedex/__init__.py
+23
-6
No files found.
README
View file @
e03a7ae0
...
...
@@ -53,7 +53,7 @@ Requirements
Dependencies
------------
As such, apachedex has no dependencies outside of standard python 2.7
As such, apachedex has no
strict
dependencies outside of standard python 2.7
installation.
But generated output needs a few javascript files which come from other
projects:
...
...
@@ -72,6 +72,10 @@ If you are running from repository, you need to fetch them first::
python setup.py deps
Also, apachedex can make use of backports.lzma
(http://pypi.python.org/pypi/backports.lzma/) if it's installed to support xz
file compression.
Input
-----
...
...
@@ -82,7 +86,13 @@ Mandatory fields are (in any order) `%t`, `%r` (for request's URL), `%>s`,
`%{Referer}i`, `%D`. Just tell apachedex the value from your apache log
configuration (see `--logformat` argument documentation).
Input files may be provided gzip'ed.
Input files may be provided uncompressed or compressed in:
- bzip
- gzip2
- xz (if module backports.lzma is installed)
Input filename "-" is understood as stdin.
...
...
TODO
View file @
e03a7ae0
- use some templating system instead of hardcoded html strings
- allow user to specify min & max dates
- autodetect more compression formats (as many as python has built-in support
for)
- implement --js & --js-embed even when pkg_resource is available
apachedex/__init__.py
View file @
e03a7ae0
...
...
@@ -33,6 +33,7 @@ from functools import partial
from
operator
import
itemgetter
from
urllib
import
splittype
,
splithost
,
unquote
import
argparse
import
bz2
import
codecs
import
gzip
import
httplib
...
...
@@ -56,6 +57,18 @@ else:
def
getResource
(
name
):
return
pkg_resources
.
resource_string
(
__name__
,
name
)
FILE_OPENER_LIST
=
[
(
gzip
.
open
,
IOError
),
(
bz2
.
BZ2File
,
IOError
),
]
try
:
from
backports
import
lzma
except
ImportError
:
pass
else
:
FILE_OPENER_LIST
.
append
((
lzma
.
open
,
lzma
.
_lzma
.
LZMAError
))
MONTH_VALUE_DICT
=
dict
((
y
,
x
)
for
(
x
,
y
)
in
enumerate
((
'Jan'
,
'Feb'
,
'Mar'
,
'Apr'
,
'May'
,
'Jun'
,
'Jul'
,
'Aug'
,
'Sep'
,
'Oct'
,
'Nov'
,
'Dec'
),
1
))
...
...
@@ -1066,13 +1079,17 @@ def main():
if
filename
==
'-'
:
logfile
=
sys
.
stdin
else
:
logfile
=
gzip
.
open
(
filename
)
try
:
logfile
.
readline
()
except
IOError
:
logfile
=
open
(
filename
)
for
opener
,
exc
in
FILE_OPENER_LIST
:
logfile
=
opener
(
filename
)
try
:
logfile
.
readline
()
except
exc
:
continue
else
:
logfile
.
seek
(
0
)
break
else
:
logfile
.
seek
(
0
)
logfile
=
open
(
filename
)
lineno
=
0
for
lineno
,
line
in
enumerate
(
logfile
,
1
):
if
lineno
%
5000
==
0
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment