ArcPy Python Logging Handler

Especially with long running or even scheduled tasks, logging is an essential tool for tracking progress and checking results after a scheduled task. Recently I have been working on a Python package to be used with ArcGIS Pro in scripts run as scheduled tasks, with ArcGIS Pro as Python Geoprocessing tools and also to perform analysis and processing tasks in Spark. In all three places, I need a single way to be able to track messages, warnings and errors. Logging is the perfect utility for messaging, but ArcGIS Pro, arcpy specifically, does not easily tie into this framework.

When running scripts as scheduled tasks, the logging.FileHandler provides the ability to easily write out results to a file to check the results later. Similarly, when running scripts from the command line, logging.StreamHandler facilitates easily reporting results to the console. With configured with a logging.Logger object, all of this is easily accomplished using the debug(), info(), warning(), error() and critical() methods. ArcGIS Pro, however, does not provide an easy tie-in to this framework.

Finally, and most difficult, if running a task in ArcGIS Pro, I need progress reported to the user interface. The package included with ArcGIS, arcpy, includes three methods providing this capability, AddMessage, AddWarning and AddError.

It took a little while, but finally, through digging into the native Python Logging module to understand how it works, I was able to put together a custom Logging Handler to tie into the arcpy methods. While at it, I also wrapped up getting a logger configured to write to ArcPy, the console and optionally a file all into one function.

If you've ever tried to pull all this together, hopefully this makes you life a lot easier!