Calling IDL programs from Python scripts
This page presents an example of how to call an IDL program from a Python script.
Image Analysis Pipelines: I work mostly with 3D and 4D medical images (MRI, PET). More and more, I find myself using a hodgepodge of image analysis programs integrated into a single data processing pipeline. This lets me select the best program for the task. In the field of medical image analysis in general and brain imaging in particular, the convergence of most analysis programs to a few widely used file formats has made this approach much easier, and the urge to use several programs from different sources is irresistible.
IDL programs: I have written a lot of IDL programs as part of the Spamalize package. Some programs are large GUI-oriented programs that require interaction, but many are smaller programs that focus on a single task. These programs are robust, predictable, and have been fairly well tested, so I want to use them in my pipelines. (Also, since I wrote them, I can usually remember their names or at least their existence, which makes writing the script that much easier!) By reusing my existing IDL programs, I can take advantage of the powerful IDL image processing libraries in the context of a data pipeline. For this scheme to work, you will need to have a full IDL (or Matlab, etc.) licence.
Python scripting language: The challenge is how to fire up the IDL engine, call the desired program, feed it parameters, and retrieve the results from a non-IDL upper level script. The following shows my solution to this problem. The basic approach would also work for similar environments such as Matlab, etc. I use Python for writing scripts because, after trying out some competitors over the years (Tcl, Perl) which I found lacking, I have so far not run into anything I could not solve with Python. Since I am a fairly lazy programmer that is really saying something!
The basic approach is to pass the filenames(s) for the input and output data, but not the data itself. There are more sophisticated ways to link programs and pass data between IDL and external callers, but I wanted an approach that would be maintenance free, i.e. I wouldn't have to recompile anything with new releases of IDL, or maintain multiple versions for different OS's.
The approach can be summarized as:
The following example shows how each of the pieces interacts, using an IDL program that rotates a 3D image. The context is that the main Python program needs to coregister a PET image to an MRI image, but the PET image has a different orientation than the MRI. The rotation is performed as part of the pre-processing to get the images into better inital alignment.
For simplicity, this code is unix/linux specific, as the directory names use the unix convention. A few additional lines of code would be needed to detect the OS and change the format to e.g. MS-DOS format.
The source code is denoted by Courier font. Commentary is sprinkled throughout in the standard font. I have tried to indicate Python indentation conventions with spaces, but this might not show up on all browsers, so you will have to use a little common sense to interpret the Python code snippets.
-----
Start the main Python program, handle parameters, etc, make sure to include needed libraries:
import image_manip
import utils_tro
... initial processing steps go here... then we get to the meat:
# Rotate the image so it has same orientation as MNI template: #
fn_rot = image_manip.rot(fn_orig, rot=[2])
Top-level pipeline program checks to see if it worked:
if not utils_tro.is_file(fn_rot):
print 'Error- failed to rotate file:'+prog_str
print ' '+fn_orig
sys.exit(1)
Carry on, pass name of rotated image file to the next step in the pipeline...
-----
And that's it! So, once you write the IDL program and Python wrappers, it is child's play to incorporate it into a data processing pipeline.
Following are the important bits from the Python wrapper for image rotation (image_manip.rot).
def rot(fn, rot=[0])
# make command to rotate the file (rot_str tells direction of rotation): #
The command is contained in a string array. Each element of the array must contain a complete single-line command. Multiple sequential commands are possible by including multiple elements in the string array. Each is executed in order.
com = ['SPAM_ROT_IMG_FILE, \"'+fn+'\", '+rot_str+'\n']
# Stipulate the string to use when searching the IDL program messages
# for the actual written filename(s) from the IDL output text:
msg = ['Wrote rotated image to ANALYZE file: (spam_rot_img_file)']
The above string must be known ahead of time by inspecting the IDL program source. It helps if you can alter the code so it prints out a unique message for this operation. The message must always be the same for a particular IDL program, i.e. it cannot contain a timestamp or the filename. The way I have it set up, the filename must be printed on the output line immediately following the above informational line.
The IDL-calling program (idl_command) is in the "image_manip.py" library. It returns the filename of the resulting new rotated image file if all goes well.
# Call the Python program that calls the idl program: #
fn_actual = idl_command(com, msg, check_dir=dir_out, verbose=1, clean=0)
Python rotation wrapper then checks to see if it worked by checking for the existence of the new file:
if not utils_tro.is_file(fn_actual):
print 'Error- failed to rotate file: '+prog_str
print ' '+fn
return 0
return fn_actual
If succesful, the new rotated filename is passed to the top-level Python pipeline.
A single Python program is used as the gate to IDL. This program, called "image_manip.idl_command", creates a text file that contains a series of commands that set needed IDL variables, defines the IDL path, and finally calls the desired IDL image processing program. Then, "idl_command" calls IDL with the text file as the startup file. The startup file can only contain single-line commands (no loops or if-statements) so minimal processing is possible here. All complex processing must occur either in the IDL processing program, or in one of the Python scripts.
def idl_command(commands, fn_markers)
### Create text file with IDL commands: ###
timestamp = utils_tro.timestamp()
The IDL program "image_manip.spam_dir_define" is a Python program that returns a string containing the full path of the IDL source-code directory. In this example, the directory is hardcoded in "image_manip.spam_dir_define" as "/apps/spamalize/" Depending on the complexity of the IDL program, most or all of this setup stage may be unnecessary.
dir_com = spam_dir_define()+'temp/'
A unique name for the text-file contains a timestamp:
fn_com1 = dir_com+'spam_com_'+timestamp
fn_com2 = fn_com1 + '.pro'
comfile = open(fn_com2, 'w', 0)
# set up IDL paths, Spamalize variables: #
comfile.write('spam_dir = \"/apps/spamalize/install/\"\n')
comfile.write('!PATH = !PATH+ \":\" +spam_dir\n')
comfile.write('!QUIET = 1\n')
The IDL program "idl_init_generic" defines variables (particularly the directory paths) for the local environment and for the various IDL programs which mostly like to work within the Spamalize environment:
comfile.write('idl_init_generic, /SPAMALIZE'+'\n')
# write each desired command to the text-file: #
for com in commands:
comfile.write(com)
comfile.write('EXIT\n')
comfile.close()
After writing the text file containing the desired IDL commands, IDL itself is invoked with the text-file as the startup file:
### Start up IDL with the command file as the startup file: ###
com = '/apps/linux/bin/idl '+fn_com1
The IDL process is spawned using the Python command "os.popen", which lets textual output from the IDL program be captured and stored in a string array, "data". A seperate string array, "err", is maintained in case the process generates an error. The contents of both string arrays can be examined by the caller (i.e. this program, "idl_command").
# spawn process:
child = os.popen(com)
data = child.read()
err = child.close()
if err:
raise RuntimeError, '%s failed with error code %d' % (com, err)
return 0
# Search string array "data" for name(s) of created filenames: #
fn_actual = 1
cntr=0
for msg in fn_markers:
m = string.find(data, msg) # location of message announcing filename
if (m < 0):
print 'Error- could not find message in IDL program output:'+prog_str
print ' '+msg
print data
return 0
# find beginning, end characters of string: #
b = string.find(data, '/', m)
e = string.find(data, '\n', b)
fn = data[b:e]
# if "data" contains multiple elements, get them all: #
if (cntr == 0):
fn_actual = [fn]
else:
fn_actual = fn_actual + [fn]
cntr = cntr+1
# delete text file: #
if clean: os.remove(fn_com2)
return fn_actual
Finally, we get to the IDL program which actually does the image rotation. It can read many formats, which is what makes it nice to use in a pipeline. It usually writes only one format (ANALYZE-7.5).
PRO spam_rot_img_file, fn, axis_flag, VERBOSE=VERBOSE
;* Read the input image data: *
img_old = EZ_READ_IMG(fn)
... Rotate the 3D image volume...
;* write the new rotated volume to the output filename: *
fn_out = fn_dir+fn_nam+'_rot.img'
EZ_WRITE_ANLZ, volume_new, FILENAME=fn_out, FN_NEW=fn_new
The following line is important! This is the exact text (including spaces) that is searched for by the calling Python program (idl_command). The line after that contains the filename, and only the filename, of the output data. This new filename is passed up the chain to the top-level Python pipeline.
print, 'Wrote rotated image to ANALYZE file: '+prog_str
print, ' '+fn_new
The only modification I had to make to the IDL code (spam_rot_img_file) was to make sure the message-string was unique, and the output filename was printed on its own single line.
If you have other ideas for how this could work, or would like more detailed code examples, please contact me (Terry Oakes) at: troakes - at - wisc.edu