Windiws server fluentd stops working

I’ve started a fluentd service on Windows Server using td-agent.
The config is watching a folder where my logs are written out, however these files are locked until fully written out causing fluentd to error and stop watching this folder. (bear in mind if I start fluend whilst a locked file exist it keeps retrying to read the file until it becomes unlocked although it still prints errors. I’d like this behaviour.)
The error is of the sort: Unexpected error raised. Stopping the timer.
Is there anything I can do to get around this? I’d like it to retry with a delay ideally.

The error is of the sort: Unexpected error raised. Stopping the timer.

Please copy & paste your exact error log including former & later lines.

BTW it may be the same issue with WindowsFile: Retrieve correct Win32 error codes by ashie · Pull Request #3337 · fluent/fluentd · GitHub
I saw same error too.
Please try fluentd v1.12.3 or td-agent 4.1.1.

Hello @ashie ,
Thank you for the follow up.
So the versions seem fine to me:

C:\Windows\system32>fluentd --version
fluentd 1.12.3

C:\Windows\system32>td-agent --version
td-agent 4.1.1 fluentd 1.12.3 (b6e76c09d60ee866de6470f2bdd2551e7c8591d3)

The actual error is this:

2021-05-06 13:32:00 +0100 [info]: adding source type=“tail”
2021-05-06 13:32:01 +0100 [info]: #0 starting fluentd worker pid=10288 ppid=6548 worker=0
2021-05-06 13:32:01 +0100 [info]: #0 fluentd worker is now running worker=0
2021-05-06 13:33:01 +0100 [error]: #0 Unexpected error raised. Stopping the timer. title=:in_tail_refresh_watchers error_class=Fluent::Win32Error error=“Fluent::Win32Error: code: 32, The process cannot access the file because it is being used by another process. - D:/Users/SharedPC/Documents/alteryx logs/Alteryx_Log_1620304364_1.log”
2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin/file_wrapper.rb:120:in initialize' 2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin/file_wrapper.rb:31:in new’
2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin/file_wrapper.rb:31:in stat' 2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin/in_tail.rb:319:in block in expand_paths’
2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin/in_tail.rb:315:in each' 2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin/in_tail.rb:315:in expand_paths’
2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin/in_tail.rb:350:in refresh_watchers' 2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin_helper/timer.rb:80:in on_timer’
2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/ run_once' 2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/ run’
2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin_helper/event_loop.rb:93:in block in start' 2021-05-06 13:34:05 +0100 [error]: #0 C:/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.12.3/lib/fluent/plugin_helper/thread.rb:78:in block in thread_create’
2021-05-06 13:34:05 +0100 [error]: #0 Timer detached. title=:in_tail_refresh_watchers

AFAIK there is no way to avoid this error without modifying the code when the file is locked by another process, Fluentd should recover such case as you said though.
To tell the truth I wanted to fix it at v1.12.3 but not yet done. I’ll try to fix it in the next release…

Thank you @ashie,
Looking forward to the fix. Please do share a PR or update me once its ready. Really eager to get it at least running

As a side note, if you start fluend whilst there is a locked file it does go into a retry loop. It only breaks when a new file apears in the folder that is locked.