Installing Python Packages on an ArcGIS Python Installation
It is little secret I use a ton of Python to get tasks in ArcGIS done. If serious about getting volumes of data processed with ArcGIS, Python is far and above the way to get things done quickly. If you start venturing far into this world, you will discover a whole raft of great modules created by a healthy community of Python developers available through what is referred to as the Python Package Manager or as it typically known by the command line command, pip.
The idea is simple enough. Python maintains a library of modules anybody can automatically download and use. All you have to do is open up a command line prompt and type pip install <package-name>. It all sounds so easy. The is one little problem. Python packages can be built with and frequently do include C++ code. This C++ code needs to be compiled. Linux and Mac have this decently covered. Windows however, is a gigantic pain to get this working on. This is the Windows challenge. It does not end here, though.
When ArcGIS for Desktop (ArcMap and ArcCatalog) are installed, Python 2.7 32-bit is installed in C:\Python27\ArcGIS10.3. When Background Geoprocessing is installed, Python 2.7 64-bit is installed in C:\Python27\ArcGISx6410.3. If you are like me and also install ArcGIS Pro, you also get Python 3.4 64-bit installed in C:\Python34. Hence, if you have all of ArcGIS for Desktop installed, you have three versions of Python installed and three places where you can theoretically access pip.
Thus, our challenge is two-fold. First, we need a way to compile C++ so we can install Python packages. Second, even if we can compile and install packages, we need to figure out a way to access pip for each Python installation.
Addressing compiling, you can download and install the Windows SDK from Microsoft. This will provide the compiler and other resources needed to compile the packages. When installing, the thing you really need is the C++ compiler. Why this is not better documented someplace, I have no clue. It is a gigantic pain to figure out.
Next up is the challenge of finding and using pip for each of our three installations. Two have it already installed. One does not.
The two installations with pip and the respective locations are:
- Python 2.7 32-bit
C:\Python27\ArcGIS10.3\Scripts - Python 3.4 64-bit
C:\Python34\Scripts
To get pip on the Python 2.7 64-bit installation, download the Python pip install script get-pip.py. Save it somewhere easy to access. If like me, I just saved it to my desktop.
Next, open up the command prompt and navigate to the directory where the Python executable is located.
> cd C:\Python27\ArcGISx6410.3
Finally, directly access the Python 2.7 64-bit executable to run get-pip.py and install pip on this version.
> ./python.exe C:\Users\<userlogin>\Desktop\get-pip.py
Now, we have pip on all three installations of Python. They can all be accessed from their respective Scripts directories.
- Python 2.7 32-bit
C:\Python27\ArcGIS10.3\Scripts - Python 2.7 64-bit
C:\Python27\ArcGISx6410.3\Scripts - Python 3.4 64-bit
C:\Python34\Scripts
To install packages found in the Python Package Repository, we can now find something useful such as ArcREST, navigate to each of the installations' Scripts directory and install the package. Hence, for ArcREST:
> cd C:\Python27\ArcGIS10.3\Scripts
> ./pip.exe install arcrest
> cd C:\Python27\ArcGISx6410.3\Scripts
> ./pip.exe install arcrest
> cd C:\Python34\Scripts
> ./pip.exe install arcrest
Redundant? Yes, a little. But now at least you can get packages installed consistently across all three Python installations on your machine!