Distributed Systems Integrity Control Issues

A system of controls for distributed (i.e., decentralized, dispersed, and cooperative) systems will need to be developed that addresses:

•  Multisystem configuration management

•  Establishing and maintaining connectivity

•  Prevention of exploitation of connectivity

•  Multilevel, multisite information transfers

•  Contingency planning, backup, and recovery

Distributed systems are depicted in the three-dimensional continuum (Exhibit 5) represented by the simplest decentralized case in one bottom corner (centralized remote processing) and the most complicated cooperative case (fully interoperable system of systems) in the opposite top corner. Decentralized systems represent a stepwise departure from centralized processing and isolated system(s) controls.

Exhibit 5.  Decentralized Processing Complexities

For any two related systems, there generally exists some data common to the two systems. The larger the amount of common data and the more dynamic the data are, the more vulnerable the decentralized system is to integrity loss. Configuration management of the changes to common data, applications, and hardware can reduce the vulnerability to integrity loss. In addition, the processes for updating common data, applications, and hardware require controls to ensure that the approved changes and only the approved changes are received and installed.

Analysis from multiple systems may produce erroneous or tainted results caused by the inability to synchronize the data. If any correlation of time-based transactions from different platforms is required, these systems require either a synchronous time source or manual synchronization and periodic verification.

In implementations of a decentralized system where two identical (or equivalent) software applications and/or hardware platforms exist, users must periodically switch processing roles as part of planning, training, and disaster preparedness. The following suggestions are provided as guidelines for establishing a baseline set of controls that ensure high integrity and minimal risk accountability for managing distributed systems.

All common data, hardware, software, and each component system should be identified formally in a Distributed System Configuration Management (CM) Plan. Distributed System CM Plans must document system-level policies, standards and procedures, responsibilities, and requirements. For distributed systems where the nodes are not located at one site or where the components are not covered in a single CM Plan, management will need to appoint a Configuration Control Authority for all distributed system-level changes. Management must ensure that sufficient resources and personnel are provided for the Configuration Control Authority to manage distributed system-level changes. Additionally,

1.  Site-level CM Plans should be hierarchically subordinate to distributed system-level CM Plans.

2.  All changes at the site level need to be reviewed by a site Configuration Control Authority for potential impact at the distributed system level.

3.  The Distributed System CM Plan should describe the distribution controls and audit checks that are used to ensure that the common data and applications are the same version across the decentralized system.

For distributed systems where the managers of components do not report to (are not managed by) the same organization, the Configuration Control Authority needs to enter into a more formal agreement with each of the managers. A memorandum of agreement should be generated that establishes policies, standards and procedures, roles, responsibilities, and requirements for the total system. At a minimum a memorandum of agreement must identify, document, and provide a detailed description of the information to be provided from each component and the recipient of that information. It must also provide a description of each level of sensitivity or criticality for each data item, delineating the levels of sensitivity or criticality at which the data will be used, and the process for moving each data item to each operation level.

All memoranda of agreement should include a description by component and interface, of all security countermeasures required of each component. This description should focus on:

1.  Security countermeasures to ensure confidentiality, integrity, and availability during the transfer of data and applications software.

2.  Access control countermeasures to ensure that the transfer process is not used to gain unauthorized access to each component.

3.  Countermeasures to ensure that the transferred data and applications are received only by the intended receiver (for data and applications requiring a high level of confidentiality).

4.  A description of the overall distributed system security policy.

It is essential to include a detailed description of the transfer process between each component, identifying:

1.  A description of any physical and media controls to be used.

2.  Electronic transfers (bulletin board systems, communications software not integrated with the decentralized component) must include a description of the software used.

3.  The software communications protocol and standards used.

4.  Encryption methods and devices used.

5.  The security features and limitations of the communications application used.

6.  All hardware requirements, hardware settings, and protocols used.

7.  Assignment of all decentralized system-level responsibilities and authorities, including network management, performance monitoring and tuning, training, training plan development and management, resource configuration management, software and data configuration management, system access control and audit management.

8.  A description of all required components or site-level security roles and responsibilities, including resource, software, and data configuration management; access control; site security management; security awareness training and training management; as well as verification and validation of security relevant issues and audit control management.

9.  An identification and needs assessment of the user community, including the levels of sensitivity or functional criticality of the information expected to be created, maintained, accessed, shared, or disseminated in or by the decentralized system.

10.  A description of the information required in each component’s audit trail and how the audit trail tasks will be divided among the components.

11.  Any results of risk assessments and how controls mitigate perceived risks.