8.2.1 Performance Measures



First and foremost, the outputs of a model-its computed performance measures-should be scrutinized. They should be understandable to both agency decision makers and the public. They should reflect to the extent possible the agency's stated objectives, relate to measurable quantitites in the actual system, be statistically stable under one set of operating policies, and if the model is to be used to compare alternatives, be dependent on the operating policy selected.

Given a meaningful and complete set of performance measures, one can begin to discuss returns on investment of a model-selected policy in an urban system, thereby bringing it slightly closer to its industrial counterparts. [DAEL 761. For instance, in an urban services context one could begin to say: "Policy X costing $ Y has brought the same improvement in operation as N additional personnel, costing $Z, would bring"; thus, a fraction (Y/Z) of the "more patrol personnel" investment brings about an identical improvement in service. [Recall our patrol allocation study above ("Why Spend Time and Money Making Decisions?").] Such statements, however, may not be easy to make in systems with multiple and conflicting objectives; in such cases the model should provide the decision maker with estimated values of each of the competing performance measures, some of which are perhaps aggregate-reflecting overall efficiency-while others are distributive-reflecting equity of service among neighborhoods. Then the decision maker has to weigh factors subjectively (and objectively) to select an appropriate operating policy, perhaps using methods of multiattribute decision analysis [KEEN 76].