FilenameInputPlugin for multi directory run in multi threads.
-
new configuration(compulsory): multi_dir example: multi_dir: ["sample/sample_","../example/example_"]
-
new configuration(not necessary, default value is []): multi_tag example: multi_tag: ["tag1","tag2"] if the length of multi_tag is less than length of multi_dir, it will add blank string automatically.
-
Since we can set the directory in the multi_dir, the path_prefix is deprecated.
-
new configuration(not necessary, default value is 0) order_by_modified_time example: order_by_modified_time: 1 if the value is 1, sort the file in each directory in modified time ascend order. if the value is not 1 neither 0, sort the file in descend order. if the value is the default value 0. sort the file in alphabetical order.
-
new configuration(not necessary, default value is 0) order_by_creation_time example: order_by_creation_time: 1 Be careful: the unix systems do not recorder the creation time of the files only the windows do that.
-
new configuration(not necessary, default value: 1024102410) chunk_size example: chunk_size: 100000
This plugin will load the files in different directories. each directory will be dealt as a task and run in a thread. we should run the tasks in multi threads to speed up the upload. The file order is kept in each directory.
In this plugin, it will create the pages for each file so that the none-bin parser is deprecated now.
For the output Plugin. I rewrite the Ruby to java. The configuration is the same as the ruby plugin.