Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
slapos slapos
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Labels
    • Labels
  • Merge requests 122
    • Merge requests 122
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Environments
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • nexedi
  • slaposslapos
  • Merge requests
  • !1672

Merged
Created Oct 24, 2024 by Kirill Smelkov@kirrOwner

software/ors-amarisoft: Fix fluentbit ingestion not to barf on large lines

  • Overview 5
  • Commits 1
  • Changes 2

Fluentbit Tail input documentation[1] says that by default maximum buffer size is 32K which turned out to be too small in practice because we hit a situation where enb.xlog started to have lines with ~ 34K and so fluentbit ingestion stopped to work with the following error in fluentbit log:

[2024/10/23 20:30:23] [error] [input:tail:tail.0] file=/srv/slapgrid/slappart19/srv/monitor/public/enb.xlog requires a larger buffer size, lines are too long. Skipping file.

-> Fix that by increasing max buffer size to 1M which seems to be high enough at least for now.

Maybe it will make sense to configure this as unlimited, but I'm not sure if going as unlimited is universally a good idea.

[1] https://docs.fluentbit.io/manual/pipeline/inputs/tail

/cc @lu.xu, @jhuge, @tomo, @paul.graydon

Assignee
Assign to
Reviewer
Request review from
None
Milestone
None
Assign milestone
Time tracking
Source branch: y/fluent-1M
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7