Lessons Learned and Ideas for Python Toolbox Coding

Note: A colleague asked for a ready to run PyScripter template implementing these suggestions. This template can now be found on GitHub at: https://github.com/knu2xs/python-toolbox-template. Instructions for using PyScripter to create ArcGIS Python Toolboxes can be found in a previous blog posting.

Recently I dove into and started working with a new feature introduced with ArcGIS 10.1, Python Toolboxes. I really like how everything for a toolbox is contained in a single python script. After a week of working on a tool off and on, I came up with a few modifications to the default template making life a little easier. These are not being presented as best practices. They are nothing more than ideas you can examine and implement if you like them.

Using the template provided in the help, I ran across a few shortcomings easily overcome with a little code organization. First, defining all the functions requires a lot of repetitive code. Second, it can be cumbersome to try and test the code logic if it is all part of the tool in the execute method. Third, if the parameters are defined in the function as suggested by the template, it kind of defeats the purpose of creating a tool class. It makes it extremely difficult to extend the class and create another tool.

Following along with this discussion may be easier if you first take a look at the help documentation discussing exactly what an ArcGIS Python Toolbox is, and also grab the ArcGIS Python Toolbox template since this is the starting point for the discussion. Also, if you are planning on using PyScripter for working with these toolboxes, it may be beneficial to set up PyScripter according to the instructions provided in a previous blog posting.

Global Function for Parameter Creation

def getParameterInfo(self):
param0 = arcpy.Parameter(
    displayName='Database Name',
    name='dbName',
    datatype='GPString',
    parameterType='Required',
    direction='Input")

param1 = arcpy.Parameter(
    displayName = 'Instance',
    name = 'instance',
    datatype = 'GPString',
    parameterType = 'Required',
    direction = 'Input')
param1.value = 'myComputer'

param2 = arcpy.Parameter(
    displayName = 'Database Management System',
    name = 'dbms',
    datatype = 'GPString',
    parameterType = 'Required',
    direction = 'Input')
param2.value = 'PostgreSQL'

param3 = arcpy.Parameter(
    displayName = 'Superuser Name', 
    name = 'suName',
    datatype = 'GPString',
    parameterType = 'Required'
    direction = 'Input')
param3.value = 'owner'

param4 = arcpy.Parameter(
    displayName = 'Superuser Password',
    name = 'suPswd',
    datatype = 'GPEncryptedString',
    parameterType = 'Required',
    direction = 'Input')
param4.value = 'Monk3ys'

param5 = arcpy.Parameter(
    displayName = 'SDE User Password', 
    name = 'sdePswd', 
    datatype = 'GPEncryptedString',
    parameterType = 'Required',
    direction = 'Input')
param5.value = 'Ch1mps'

param6 = arcpy.Parameter(
    displayName = 'Data Owner Name',
    name = 'ownerName',
    datatype = 'GPString',
    parameterType = 'Required',
    direction = 'Input')
param6.value = 'dataOwner'

param7 = arcpy.Parameter(
    displayName = 'Data Owner Password',
    name = 'ownerPswd',
    datatype = 'GPEncryptedString',
    parameterType = 'Required',
    direction = 'Input')
param7.value = 'B0nobos'

param8 = arcpy.Parameter(
    displayName = 'Authorization File',
    name = 'authFile',
    datatype = 'DEFile',
    parameterType = 'Required',
    direction = 'Input')
param.value = \
    r'C:\Program Files\ESRI\License10.1\sysgen\keycodes'

# parameter list
params = [parm0, param1, param2, param3, param4, param5,
    param6, param7, param8]

# return parameter list to ArcGIS
return params</pre>

If the above looks painful...trust me, it is. Notice how much repetition there is. A global method can be created above the toolbox declaration streamlining the parameter creation process and optional default value assignment.
def parameter(displayName, name, datatype, defaultValue=None,
    parameterType=None, direction=None):
    '''
    The parameter implementation makes it a little difficult to 
    quickly create parameters with defaults. This method
    prepopulates some of these values to make life easier while
    also allowing setting a default vallue.
    '''
    # create parameter with a few default properties
    param = arcpy.Parameter(
        displayName = displayName,
        name = name,
        datatype = datatype,
        parameterType = 'Required',
        direction = 'Input')

    # set new parameter to a default value
    param.value = defaultValue

    # return complete parameter object
    return param
The parameterType, direction and defaultValue parameters are optional so they can be set if needed or desired. Using this function, the same parameter declaration becomes much more concise.
def getParameterInfo(self):

    # create parameters
    param0 = parameter('Database Name', 'dbName', 'GPString')
    param1 = parameter('Instance', 'instance', 'GPString',
        'myComputer')
    param2 = parameter('Database Management System', 'dbms',
        'GPString', 'PostgreSQL')
    param3 = parameter('Superuser Name', 'suName','GPString')
    param4 = parameter('Superuser Password', 'suPswd', 
        'GPEncryptedString', 'Monk3ys')
    param5 = parameter('SDE User Password', 'sdePswd',
        'GPEncryptedString', 'Ch1mps')
    param6 = parameter('Data Owner Name', 'ownerName', 'GPString',
        'dataOwner')
    param7 = parameter('Data Owner Password', 'ownerPswd', 
        'GPEncryptedString', 'B0nobos')
    param8 = parameter('Authorization File', 'authFile', 'DEFile',
        r'C:\Program Files\ESRI\License10.1\sysgen\keycodes')

    # parameter list
    params = [parm0, param1, param2, param3, param4, param5,
        param6, param7, param8]

    # return parameter list to ArcGIS
    return params
The parameter function takes care of the first step, consolidating parameter creation, but there still are a couple of other things to clean up the script.
Separate the Business Logic
The template provided by the help documentation suggests to put all the business logic, the actual part of your script doing the work, in the execute method of the tool. This creates problems for two reasons. First, it makes tracking down errors during debugging difficult since it is hard to separate whether it is a problem with the business logic or something with the tool properties. Second, thorough trial and error I discovered the execute method does not support subclassing. This means you cannot create another tool very similar to the first and call the execute method from the superclass tool.
To avoid these problems, I created a class at the top of the script with all the business logic and tested all this business logic ahead of time. Then, once ready to build the toolbox, there is no doubt about the business logic. It works. Rather than show a huge code sample, you can see this implemented in the SDE Workspace Creation Toolbox. At the top of the script, all the business logic is contained in the sdeWorkspace class created toward the top.
Parameters as Tool Class Attributes
In ArcGIS Python toolboxes, tools are created as classes. However, if strictly following the template provided in the help documentation, establishing tool parameters is done in a function of the tool class. Since tool parameters are created as a list in a method and not as object attributes in the __init__ method, the tool parameters are not accessible as object attributes. Each individual tool class cannot be extended to create new tools. It kind of defeats the purpose of object oriented programming.
This can be overcome by creating a new list as an attribute of the tool object in the init  method. In this way when subclassing the first tool class, parameters can be substituted by replacing items in the list using Python list methods. Borrowing from the larger example referenced above, the SDE Workspace Creation Toolbox demonstrates this.
def parameter(displayName, name, datatype, defaultValue=None,
    parameterType=None, direction=None):
    '''
    The parameter implementation makes it a little difficult to
    quickly create parameters with defaults. This method
    prepopulates some of these values to make life easier while
    also allowing setting a default vallue.
    '''
    # create parameter with a few default properties
    param = arcpy.Parameter(
        displayName = displayName,
        name = name,
        datatype = datatype,
        parameterType = 'Required',
        direction = 'Input')

    # set new parameter to a default value
    param.value = defaultValue

    # return complete parameter object
    return param

class Toolbox(object):
    def __init__(self):
        # ArcGIS required properties
        self.label='SDE Tools'
        self.alias='sdeTools'

        # List of tool classes associated with this toolbox
        self.tools=[CreateSdeTool]

class CreateSdeTool(object):

    def __init__(self):

        # ArcGIS tool properties
        self.label = 'Create SDE Workspace'
        self.canRunInBackground = False

        # list of parameters for tool
        self.parameters=[
            parameter('Database Name', 'dbName', 'GPString'),
            parameter('Instance', 'instance', 'GPString',
                globalInstance),
            parameter('Database Management System', 'dbms',
                'GPString', globalDbms),
            parameter('Superuser Name', 'suName','GPString'),
            parameter('Superuser Password', 'suPswd',
                'GPEncryptedString', globalSuPswd),
            parameter('SDE User Password', 'sdePswd',
                'GPEncryptedString', globalSdePswd),
            parameter('Data Owner Name', 'ownerName', 'GPString',
                globalOwnerName),
            parameter('Data Owner Password', 'ownerPswd',
                'GPEncryptedString', globalOwnerPswd),
            parameter('Authorization File', 'authFile', 'DEFile',
                globalAuthFile)]

    def getParameterInfo(self):
        # send parameters to ArcGIS
        return self.parameters
Now when another tool needs to be created extending the first tool's functionality, items in the parameter list can be replaced by directly referencing the item in the list. The business logic is captured in the class created outside of the tool and only referenced in the execute method. Hence, another tool, one importing an xml workspace file into a new SDE geodatabase, takes very little coding to create since it borrows most of the functionality from the original tool.
class FileToSdeTool(CreateSdeTool):

    def __init__(self):

        # Call superclass constructor
        CreateSdeTool.__init__(self)

        # Replace ArcGIS tool properties
        self.label = 'File to SDE Workspace'
        self.canRunInBackground = False

        # Replace first tool parameter to become file geodatabase
        self.parameters[0] = parameter('File Geodatabase',
            'fileGdb', 'DEWorkspace')

        self.parameters[0].filter.list = ["Local Database"]

    def execute(self, parameters, messages):

        # create instance of sdeWorkspace object
        sde = sdeWorkspace()

        # set attributes
        sde.instance = parameters[1].valueAsText
        sde.dbms = parameters[2].valueAsText
        sde.suName = parameters[3].valueAsText
        sde.suPswd = parameters[4].value
        sde.sdePswd = parameters[5].value
        sde.ownerName = parameters[6].valueAsText
        sde.ownerPswd = parameters[7].value
        sde.authFile = parameters[8].valueAsText

        # call fileToSde method
        sde.fileToSde(parameters[0].valueAsText)

        return
Most of the lines, assigning the parameter values to object attributes, are something I could further condense if I further cleaned up my business logic.
It took a while to discover some of the interesting behaviors of the way Python toobloxes are implemented with ArcGIS. Using a function for parameter creation, separating the business logic into another class and making the parameters part of the tool attributes greatly streamlined the process of working with these tools. Now Python Toolboxes likely are going to become the standard way I create geoprocessing toolboxes from now on.