Comments.

bt5/erp5_wendelin/PathTemplateItem/portal_callables/DataIngestionLine_writeFluentdIngestionToDataStream.py

+"""
+  This script is used during fluentd ingestion.
+  It will write data sent from fluentd by unpacking it first and then appending
+  as a string to respective "Data Stream".
+"""
+
 out_stream["Data Stream"].appendData(''.join([str(c[1]) for c in context.unpack(data_chunk)]))

bt5/erp5_wendelin/PathTemplateItem/portal_callables/DataIngestionLine_writeIngestionToDataStream.py

+"""
+  This script is used during fluentd ingestion.
+  It will append data sent from fluentd to Wendelin 'as it is' to respective "Data Stream".
+  By default data will be encoded in MsgPack format.
+"""
+
 out_stream["Data Stream"].appendData(data_chunk)

[Klaus Wölfel]

@Tyagov First thanks for adding comments to scripts, this is very helpful. For exactenss, I propose to rewrite as follows:

  This script is used during fluentd ingestion.
  It assumes data comes in msgpack encoded in the following format: mspack(timestamp, data).
  It will first unpack the msgpack, then remove the first item of the tuple (timestamp)
  and append str(data) to "Data Stream".

  Note that what is saved to Data Stream might be different from what fluentd was reading initially,
  depending on fluentd plugin configuration. For example fluentd might convert json to msgpack,
  then what is saved in Data Stream might be str(python_dict) and not json.

[Klaus Wölfel]

@Tyagov this script is not specific to fluentd and it is completely independent of msgpack. Therfore I propose to rephrase as follows:

This script is a general ingestion script which can be used with fluentd or with other http based ingestion tools.
It will append data sent to Wendelin 'as it is' to respective "Data Stream".

Note that by default fluentd data is encoded in msgpack format, this script will not unpack it.