Recently a customer reported several incidents with a scheduled data transfer task, causing the business to complain about the lack of fresh data. Sadly, the scheduled task failed randomly. We were missing crucial insights to fix the scheduled task and/or inform the business of an incident. Therefore, the first step was to monitor the scheduled task status using Google Cloud Monitoring. The monitoring implementation details are shared in this blog.

Scheduled Task Monitoring

The next steps are needed to monitor your scheduled tasks:

1. Enable Task Scheduler logging
2. Forward Windows Event Logs to Cloud Logging
3. Monitor failed tasks with Cloud Monitoring

Use the reference implementation on GitHub to kick-start your setup.

Scheduled Task Logging

“The Task Scheduler service allows you to perform automated tasks. With this service, you can schedule any program to run at a convenient time for you or when a specific event occurs. The Task Scheduler monitors the time or event criteria that you choose and then executes the task when those criteria are met.” – About the Task Scheduler.

Task Scheduler execution details are logged to the Windows Event Log by enabling all tasks history. These logs indicate that a task and its associated actions started and completed e.g. “Task Scheduler successfully completed task ‘YourTask’ , instance ‘{72efc060-52b3-4a0a-a656-c7527c912082}’ , action ‘C:\Program Files\PowerShell\7\pwsh.exe’ with return code 0.”. You enabled these logs using the following command:

wevtutil set-log Microsoft-Windows-TaskScheduler/Operational /enabled:true

Forwarding Windows Event Logs

The Windows Event Log is forwarded to Cloud Logging by the Ops Agent. More specifically, the Task Scheduler Event Log is forwarded using the following configuration:

logging:
  receivers:
    windows_event_log:
      type: windows_event_log
      channels: [System, Application, Security, "Microsoft-Windows-TaskScheduler/Operational"]
      receiver_version: 2

Monitoring Failed Tasks

A Log-based alert notifies the operations team of a failing task: a task completing with an error exit code. Since the exit code is reported at the action level, the alert condition becomes:

-- Only "Action completed logs"
jsonPayload.Channel="Microsoft-Windows-TaskScheduler/Operational"
jsonPayload.EventID="201"

-- Only "My Task Name" and failure exit code.
jsonPayload.StringInserts="\\My Task Name"
jsonPayload.Message !~ "return code 0\.$"

Conclusion

Don’t wait for the business to report an incident and mess up your day. Take back control by monitoring your scheduled tasks using failed task alerts, owning the issue and patching the root cause.

Image by Enrique from Pixabay