13.1 Dynamic Execution and the exec Statement

With Python's exec statement, it is possible to execute code that you read, generate, or otherwise obtain during the running of a program. The exec statement dynamically executes a statement or a suite of statements. exec is a simple keyword statement with the following syntax:

exec code[ in globals[,locals]]

code can be a string, an open file-like object, or a code object. globals and locals are dictionaries. If both are present, they are the global and local namespaces, respectively, in which code executes. If only globals is present, exec uses globals in the role of both namespaces. If neither globals nor locals is present, code executes in the current scope. Running exec in current scope is not good programming practice, since it can bind, rebind, or unbind any name. To keep things under control, you should use exec only with specific, explicit dictionaries.

13.1.1 Avoiding exec

More generally, use exec only when it's really indispensable. Most often, it is better avoided in favor of more specific mechanisms. For example, a frequently asked question is, "How do I set a variable whose name I just read or constructed?" Strictly speaking, exec lets you do this. For example, if the name of the variable you want to set is in variable varname, you might use:

exec varname+'=23'

Don't do this. An exec statement like this in current scope causes you to lose control of your namespace, leading to bugs that are extremely hard to track and more generally making your program unfathomably difficult to understand. An improvement is to keep the "variables" you need to set, not as variables, but as entries in a dictionary, say mydict. You can then use the following variation:

exec varname+'=23' in mydict

While this is not as terrible as the previous example, it is still a bad idea. The best approach is to keep such "variables" as dictionary entries and not use exec at all to set them. You can just use:

mydict[varname] = 23

With this approach, your program is clearer, more direct, more elegant, and faster. While there are valid uses of exec, they are extremely rare and they should always use explicit dictionaries.

13.1.2 Restricting Execution

If the global namespace is a dictionary without key '_ _builtins_ _', exec implicitly adds that key, referring to module _ _builtin_ _ (or to the dictionary thereof), as covered in Chapter 8. If the global namespace dictionary has a key '_ _builtins_ _' and the value doesn't refer to the real module _ _builtin_ _, code's execution is restricted, as covered in the upcoming section Section 13.2.

13.1.3 Expressions

exec can execute an expression because any expression is also a valid statement (called an expression statement). However, Python ignores the value returned by an expression statement in this case. To evaluate an expression and obtain the expression's value, see built-in function eval, covered in Chapter 8.

13.1.4 Compile and Code Objects

To obtain a code object to use with exec, you normally call built-in function compile with the last argument set to 'exec' (as covered in Chapter 8). I recommend using compile on statements held in a string and then using exec on the resulting code object, rather than giving exec the string to compile and execute. This separation lets you check for syntax errors separately from evaluation-time errors. You can often arrange things so the string is compiled once and the code object is executed repeatedly, speeding things up. eval can also benefit from such separation.

A code object has a read-only attribute co_names, the tuple of the names used in the code. Knowing what names the code is about to access may sometimes help you optimize the preparation of the dictionary you pass to exec or eval as the namespace. Since you need to provide values only for those names, you may save work by not preparing other entries.

For example, your application may dynamically accept code from the user with the convention that variable names starting with data_ refer to files residing in subdirectory data that user-written code doesn't need to read explicitly. User-written code may in turn compute and leave results in global variables with names starting with result_, which your application will write back as files in subdirectory data. Thanks to this convention, you may later move the data elsewhere (e.g., to BLOBs in a database), and user-written code won't be affected. Here's how you might implement these conventions efficiently:

def exec_with_data(user_code_string):
    user_code = compile(user_code_string, '<user code>', 'exec')
    datadict = {  }
    for name in user_code.co_names:
        if name.startswith('data_'):
            datafile = open('data/%s' % name[5:], 'rb')
            datadict[name] = datafile.read(  )
            datafile.close(  )
    exec user_code in datadict
    for name in datadict:
         if name.startswith('result_'):
            datafile = open('data/%s' % name[7:], 'wb')
            datafile.write(datadict[name])
            datafile.close(  )