ArcPy Python Logging Handler

Update 01Nov2022 - After Pat's comment, I decided to create a discrete repo, ArcPy-Logger, to make it easier to use.  In the process, I also updated the Gist based on a few tweaks I made, and added brief documentation as well.

Especially with long-running or even scheduled tasks, logging is an essential tool for tracking progress and checking results after a scheduled task. I have recently been working on a Python package to be used with ArcGIS Pro in scripts run as scheduled tasks, with ArcGIS Pro as Python Geoprocessing tools, and performing analysis and processing tasks in Spark. In all three places, I need a single way to be able to track messages, warnings, and errors. Logging is the perfect utility for messaging, but ArcGIS Pro, arcpy specifically, does not easily tie into this framework.

When running scripts as scheduled tasks, the logging.FileHandler provides the ability to easily write out results to a file to check the results later. Similarly, when running scripts from the command line logging.StreamHandler facilitates easily reporting results to the console. With configured with a logging.Logger object, all of this is easily accomplished using the debug(), info(), warning(), error() and critical() methods. ArcGIS Pro, however, does not provide an easy tie-in to this framework.

Finally, and most difficult, if running a task in ArcGIS Pro, I need progress reported to the user interface. The package included with ArcGIS, arcpy, includes three methods providing this capability, AddMessage, AddWarning and AddError.

It took a little while, but finally, through digging into the native Python Logging module to understand how it works, I was able to put together a custom Logging Handler to tie into the arcpy methods. While at it, I also wrapped up getting a logger configured to write to ArcPy, the console, and optionally a file all into one function.

If you've ever tried to pull all this together, hopefully, this makes your life a lot easier!