Fluentd become slowly after pressure testing

I try to sent massive log to test fluentd will handle the request perfectly.

Config is no problem and work well, but it’s weird that after the pressure testing and make sure the server resource is released, fluentd performance become very slow and is unusual.

After that, I modify config from flush_mode immediately to interval, and seems the situation become well as I expected.

I wondering there is anything I misunderstanding about the setting to cause ? Shouldn’t I use the immediately?

workers 4 @type forward port 24224

<filter **>
@type parser
key_name message
remove_key_name_field false
reserve_data true

@type grok
grok_failure_key grokfailure

pattern %{TIMESTAMP_ISO8601:timestamp}|%{SPACE}%{LOGLEVEL}|%{SPACE}%{DATA:SERVICE}|%{SPACE}%{INT:CODE1}|%{SPACE}%{DATA:ACTION}|[%{NOTSPACE:session_id}]%{SPACE}#[%{DATA:LOG_TYPE}][%{DATA:LOG_TITLE}:%{SPACE}%{GREEDYDATA:KV_DATA}]
time_key timestamp
time_format “%F %T.%L”
timezone Asia/Taipei


pattern %{TIMESTAMP_ISO8601:timestamp}
time_key timestamp
time_format “%F %T.%L”
timezone Asia/Taipei

<filter **>
@type key_value_parser
remove_key true
key KV_DATA
keys_delimiter /,/
kv_delimiter_char ‘=’
filtered_keys_regex /^[A-Z]/

<filter **>
@type dict_map
default_value “其他”
key_name WalletCoopSysSeq
destination_key_name WalletCoopSysSeqDisplayName
dictionary_path /etc/td-agent/walletstatuscode_dict_map.json

<filter **>
@type dict_map
default_value “其他”
key_name TxStatus
destination_key_name TxStatusDisplayName
dictionary_path /etc/td-agent/txstatus_dict_map.json

<filter **>
@type value_to_key_with_dynamic_value
new_key TxStatusDisplayName

<filter **>
@type code_to_name
wallet_code WalletCoopSysSeq
pay_code CorpCoopSysSeq
display_key CorpCoopSysSeqDisplay
condition_file_path /etc/td-agent/plugin/code_to_name.json

<match **>
@type copy

@type elasticsearch
host 10.99.1.119
port 9200
time_key timestamp
logstash_format true
logstash_prefix mpb-${tag}
include_timestamp
utc_index false

flush_mode immediate
flush_thread_count 2
retry_forever true
retry_max_interval 3