Internet Draft Network Working Group Internet Draft Expiration Date: April 2001 Yangguang Xu Daniel O. Awduche Eve Varma Yong Xue Ramesh Nagarjan UUNet/WorldCom Lucent Curtis Brownmiller Dimitri Papadimitriou WorldCom Rudy Hoebeke Alcatel John Eaves TyCom Osama Aboul-Magd Michael Mayer Hirokazu Ishimatsu Nortel Japan Telecom Raj Jain Nayna Nov. 2000 Generalized MPLS Control Plane Architecture for Automatic Switched Transport Network draft-xu-mpls-ipo-gmpls-arch-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." Y. Xu et al. Page [1] Internet Draft GMPLS Architecture for ASTN Nov. 2000 The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract Many solutions have been proposed to enable automatically switched transport networks (ASTN). This document describes a control plane architecture that can be applied to different packet and circuit switching technologies (including fiber, waveband, wavelength, PDH, and SONET/SDH). The control plane technology is based on IP/MPLS control plane protocols. As such, this document is based on the concepts introduced in [MPLambdas] and [GMPLS-SIG] from an architectural perspective. It also describes how this control plane architecture could facilitate control plane integration of networks across technical, administrative, and business domains. This memo includes generic procedures, key concepts, and technical considerations for the generalized MPLS control plane architecture. It is intended to accentuate understanding of the application domains, create architectural alignment, and serve as guide for protocol engineering. Table of Contents 1 Specification ...................................... 3 2 Acronyms .......................................... 4 3 Introduction to Generalized MPLS Control Plan ...... 4 4 Architecture Overview .............................. 6 4.1 System Components .................................. 6 4.2 Network Overview ................................... 8 4.3 Control Plane Transport Network .................... 9 5 Detailed Architectural Considerations .............. 11 5.1 NE Level Resource Discovery ........................ 11 5.1.1 NE Level Resource Table ............................ 11 5.1.2 Resource Discovery Stages .......................... 12 5.2 State Information Dissemination .................... 13 5.2.1 Logical Link/Bundle ................................ 13 5.2.2 Link State Information ............................ 14 5.2.3 Link State Dissemination Mechanisms ................ 15 5.2.4 Flooding Control ................................... 15 5.3 Path Selection ..................................... 16 5.3.1 Routing Constraints in ASTN ........................ 17 Y. Xu et al. Page [2] Internet Draft GMPLS Architecture for ASTN Nov. 2000 5.3.2 Path Selection for Traffic Engineering ............. 20 5.3.3 Policy Consideration in Path Selection ............. 20 5.3.4 Result of Path Selection ........................... 20 5.4 Path Control ....................................... 21 5.4.1 MPLS Control Plane Concepts for ASTN ............... 21 5.4.1.1 Upstream and Downstream ............................ 21 5.4.1.2 Generalized Label in ASTN .......................... 22 5.4.1.3 Label Space ....................................... 23 5.4.1.4 Label Binding ...................................... 23 5.4.1.5 Label Distribution ................................. 24 5.4.1.6 Circuit LSP Tunnel ................................. 25 5.4.1.7 Circuit LSP Concatenation ......................... 25 5.4.2 LSP Information Maintenance ........................ 25 5.4.3 LSP Restoration .................................... 25 5.4.3.1 LSP Restoration Stages ............................. 25 5.4.3.2 Link Level Restoration ............................. 26 5.4.3.3 Path Level Restoration ............................. 26 5.4.4 Circuit LSP Verification .......................... 27 5.4.5 Control Plane Failure Handling ...................... 27 6 Inter-domain Control Plane Integration ............. 28 6.1 Technology Domains and Hierarchy ................... 28 6.2 Service Interface between Network Domains .......... 29 6.3 Control Plane Design for Integration ............... 30 6.3.1 NE Level Resource Discovery ........................ 30 6.3.2 State Information Dissemination .................... 31 6.3.2.1 Potential Forwarding Adjacency ..................... 31 6.3.3 Path Selection ..................................... 32 6.3.4 Path Control ....................................... 33 6.3.4.1 Service Invocation ................................. 33 6.3.4.2 LSPs of Different Technology Domains ............... 33 6.3.4.3 LSPs over Multiple ASes ............................ 33 6.3.4.4 LSP Restoration .................................... 34 7 Security Considerations ............................ 34 8 Intellectual Property .............................. 34 9 Acknowledgment ..................................... 34 10 Authors' Addresses ................................. 34 11 References ......................................... 36 1. Specification The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Y. Xu et al. Page [3] Internet Draft GMPLS Architecture for ASTN Nov. 2000 2. Acronyms ASTN: Automatic Switched Transport Network NE: Network Element ENE: Edge Network Element SNE: Source Network Element DNE: Destination Network Element SLA: Service Level Agreement BER: Bit Error Ratio LSP: Label Switched Path PDM: Packet Division Multiplexing TDM: Time Division Multiplexing SDM: Space Division Multiplexing LO: Low Order HO: High Order 3. Introduction to Generalized MPLS Control Plane A network can be functionally divided into a data or transport plane, a control plane, and a management plane. The control plane consists of network control components and network element (NE) control components. The network control plane components perform network level coordination functions including: state information management (acquisition, representation, dissemination), decision making (e.g., path selection), and action invocation (e.g., signaling). The network element control components control the local operation of specific network elements. Traditional circuit switched transport networks have virtually no automatic network control, which resulted in very slow service provisioning. Automatically switched transport networks, such as optical transport networks, are required and feasible in the emerging data centric networks which are growing very rapidly. In particular, automatic switching of available capacity is needed to quickly and efficiently provision bandwidth wherever and whenever needed to address the increasing demands for network bandwidth. Even though differences exist in the technology and operation of different transport networks, especially within the data or transport planes, all switched transport networks share very similar characteristics in their control planes. These control plane commonalties are also shared by connection-oriented data networks, such as MPLS, as noted in [Mplamdas]. It is now widely recognized that it is desirable to establish generalized control architecture that is applicable to different technologies because it facilitates network inter-working. One of the major applications of MPLS is the set-up of explicitly routed LSPs [MPLS-ARCH] for Traffic Engineering (TE) purposes. Given that a label switched path (LSP) is an instance of a network connection, it is evident that the LSP concept can Y. Xu et al. Page [4] Internet Draft GMPLS Architecture for ASTN Nov. 2000 be extended to support the set-up of any type of network connection via an MPLS-based control plane as noted in a number of recent IETF drafts (see e.g., [Mplambdas]). This applicability of the control protocols to multiple network domains makes MPLS a good candidate for a generalized control plane protocol. For this reason, there is significant interest throughout the industry to define a generalized MPLS-based traffic engineering control plane for ASTN. However, in order to generalize and apply the MPLS traffic engineering control plane concepts to ASTNs, it is important to recognize the unique features of ASTNs which demand specific adaptations to the MPLS control protocols. Specically, ASTNs have many unique features that should be taken into consideration in the design of pertinent control plane technologies and architectures. Examples of these unique features include: -- Network level features -- ASTN serves as the backbone for user traffic. The traffic load and topology changes are not as dynamic as those for many packet swtiched data networks. -- ASTN provides services with different types of SLAs (BER, availability, maximum downtime and etc.) than those that typically prevail for data connections. -- ASTN has additional link state parameters than the link state metrics used for data network topologies. -- Circuit paths are setup with many technology dependent constraints. The path selection process for circuit connections is also impacted by technology-specific considerations. -- Data LSPs are generally considered to be uni-directional while circuit connections can be uni-directional or bi-directional. -- Network element level features -- Circuit switched NEs may have hundreds or even thousands of physical ports. Many of them terminate on the same neighbor and share the same attributes. -- Ports in circuit switched NE are not typically assigned IP addresses. -- Each physical port includes many technology dependent attributes. The characteristics of ASTNs also have an impact on the data network control plane. In particular, it changes the data network's basic assumption that the transport network consists of fixed pipes. In the new paradigm, the transport network can be conceived as a large circuit switch with a dynamically configurable back plane. Network devices that are connected to a ASTN can potentially establish direct connectivity with each other. Effort should be expended to enable data networks exploit these new capabilities of ASTNs to achieve true holistic end-to- end network traffic engineering. Y. Xu et al. Page [5] Internet Draft GMPLS Architecture for ASTN Nov. 2000 As more applications enabled by ASTN are discovered, they will affect existing network characteristics and generate new requirements. This demands a control plane architecture that is flexible and easily extensible. 4. Architecture Overview A well-designed control plane architecture should provide service providers with enhanced network control capabilities and relieve operators from unnecessary manual operations. At the same time, it should facilitate network interoperation and integration between different networks with different transport technologies such as circuit switched networks and data networks. -- It should be applicable to all circuit switched networks; e.g., fiber (Optical Transport Section - OTS), waveband (Optical Multiplex Section - OMS), wavelength (Optical Channel - OCh), SONET/SDH, and PDH. In order to achieve this goal, it is essential to isolate technology dependent aspects from technology independent aspects and to address them separately. -- It should be sufficiently flexible to accommodate different network scenarios (service provider business models). This goal may be achieved by partitioning the control plane into a number of distinct components. This, for example, allows vendors and service providers to decide the logical placement of these components, and also allows the service provider to decide the security and policy aspects of these components. Within the context of ASTNs, the circuit switched network control plane deals primarily with connection services. In a larger sense, this functionality is a subset of the network management functions; the complement being aspects of fault management, configuration management, accounting management, performance management, security management, and policy management functions. MPLS control plane components could be applied to ASTN to achieve automatic connection control. It should also be noted that for many circuit connection services, the MPLS based control plane is just one alternative, as capabilities will also be available to provide the same functionality in a less automated fashion. 4.1 System Components Corresponding to the MPLS traffic engineering control plane model defined in [MPLS-TE], the generalized MPLS control plane can be divided into several modules, namely, element level resource discovery, state information dissemination, path selection and path control modules. These functional components work together to Y. Xu et al. Page [6] Internet Draft GMPLS Architecture for ASTN Nov. 2000 complement each other. Their structure and the relationships between them establish the overall architecture. This approach is intended to avoid focus upon certain functional components of the architecture, to the inadvertent exclusion of others, which could result in unnecessary dependencies and poor network level solutions. The basic modules are described below. -- Network element level resource discovery Resource discovery is defined as the transaction that establishes, verifies, updates and maintains the NE adjacencies and their port- pairs association for their transport (data) plane. During resource discovery, control plane network should have built up its own adjacencies and is in operation mode already. The resource discovery module generates a complete NE level resource map, which including attributes, neighbor identifiers, and pertinent real-time operational status. The control procedure of this component should be very generic. At the same time, some of its sub-components may have to be technology and perhaps even product specific. Beyond physical resource discovery, this component may also include service/policy negotiation between adjacent NEs. -- State information dissemination State information dissemination encompasses the manner in which NE level resource information is disseminated throughout the network to form network level resource database. State information distribution involves a number of steps. First, the NE level physical resource map is acquired and summarized into logical links (section 5.2.1) based on link attributes. They are then distributed throughout the network by either piggybacking onto the control plane transport network (section 4.3) IGP, or through some other means such as application layer protocols. Key issues that relate to state information dissemination include the specific types of state information that needs to be distributed and the mechanisms for representing and disseminating this information. -- Path selection Transport network routing procedures typically utilize explicit routing, where path (connection) selection can be done either by an operator, through the use of simulation and planning tools, by a NMS or by the network itself. Hop by hop (link by link) routing is also possible and may have its own benefits under some circumstances. In a switched transport network, a path (connection) is normally requested with certain constraints, which mainly come from the customers' requirements, operators' traffic engineering Y. Xu et al. Page [7] Internet Draft GMPLS Architecture for ASTN Nov. 2000 policies, and network characteristics. Path selection for a connection request should utilize constrained-based routing algorithms that compute paths in the presence of multiple constraints and objectives: -- Conform to constraints such as physical diversity, etc. -- Achieve traffic engineering objectives in the transport network -- Adhere to operator policies on routing such as preferred routes -- Conform to network specific constraints -- Path (Connection) control Path control deals with path operations such as path creation, deletion and modification (auto-rerouting and protection switching). This is an area that many MPLS control plane concepts are applicable. Details of these functional components are elaborated in Section 5. 4.2 Network Overview A typical connection operation within a single autonomous system (AS) involves the following types of roles played by various entities: service agent, edge NE (ENE), and intermediate NE (INE). -- The service agent represents the system that originates an end-to- end circuit connection request. This role could be played by a client NE, NMS, or network scheduling tool. It may or may not be within the scope of ASTN control plane. Depending on which network application scenario is supported, and which services are provided by the service provider, the request from the service agent to the source NE may contain the whole explicit path (connection), or a subset of the explicit route, or only indicate path (connection) end points. The service agent may also include constraints, objective criteria, and policies as part of the request. -- ENE has two functional roles: source NE (SNE) role and destination NE (DNE) role. SNE initiates the connection set-up operation through the associated transport network upon receipt of an authenticated request from the service agent. DNE is the termination point of the connection set-up operation over the transport network. SNE and ENE are irrelevant to transport traffic flow direction. That is, they do not imply the direction of traffic flow. Furthermore, a particular node may perform the role of SNE for one connection set-up operation, and perform the role of DNE for another connection set-up operation. Y. Xu et al. Page [8] Internet Draft GMPLS Architecture for ASTN Nov. 2000 A SNE typically supports all the functional components mentioned above. In addition to having the capability to play the role of INE (see below) and DNE, a SNE may also support the capability to perform path selection computations. Furthermore, a SNE should maintain an inventory of path information for all circuit LSPs that were originated from itself. -- Intermediate NE (INE) exists along the path between an SNE and a DNE. The INE accepts signaling requests from adjacent NEs and executes them by rearranging its switch matrix and performing other local operations to facilitate establishment of the connection. In executing connection setup requests, an INE selects specific physical resource according to its local knowledge and according to pre-defined policies. INE maintains LSP ID as a reference to support other types of requests. It doesn't need to maintain full path (connection) level information. However, it does need to maintain local physical resource information for all LSPs that pass through it. Such local information need not exist on other Nes, except in the form of resource availability and usage information which the INE may disseminate to other nodes. The roles of ENE and INE are associated with each LSP through the switched transport network. However, each NE in the network could play any or all of these roles. Specifically, the role a NE plays depends upon context, especially its role in the establishment of each LSP traversing it. An LSP traversing multiple autonomous systems ASes may be divided into multiple AS-sections, such that each AS-section corresponds to one AS. Each section may be considered to have its own AS ENE and AS INE. Corresponding interfaces and responsibilities within the context of multiple ASes require further study. 4.3 Control Plane Transport Network and Requirements The underlying control plane infrastructure consists of a set of distributed controllers that communicate and coordinate with each other to facilitate connection set-up. A data communication network is needed to enable control messages to be exchanged between controllers via the distributed control plane. There is no separate transport network for the control plane in traditional IP networks, where control messages are exchanged through the same transport medium as the user data traffic. For circuit based networks, the ability to have an independent network for control message transportation is a fundamental requirement because a typical transport NE may not have the capability to terminate data traffic through the bearer channels. Y. Xu et al. Page [9] Internet Draft GMPLS Architecture for ASTN Nov. 2000 A control network may be defined as the transport infrastructure that is used to convey control traffic. The control network can be either in-band or out-of-band relative to the user traffic. Thus, the MPLS-based control plane must not make assumptions about its physical transport medium for control traffic. An implication of this is that the control network of a circuit switched network may have a different physical topology than its transport plane network. Another implication is that the health of the control plane and transport plane should be separately maintained. In the event of control plane failure (for example, communications channel or control entity failure), new circuit LSP (connection) operations shall not be accepted, but existing connections shall not be dropped. Control network failure should still allow dissemination of failure events to a management system for operations, maintenance, and management purposes. This implies a need for separate notifications and status codes for the control plane and transport plane. In general, because of the importance of the control plane to the overall operation of the network, the control plane transport infrastructure should be highly survivable. Therefore, additional procedures need to be defined for control plane failure recovery. Inter-working of the control networks is the first step towards control plane inter-operation. To maintain a certain level of interoperability, it is desirable to have an IP based control network. Control networks from different administrative domains or different types of transport networks can be interconnected in various ways to facilitate control interaction between different domains, or types of networks. The control network is an integral but independent part of the overall control plane. It is critical to the scalability, reliability, and performance of the overall control plane. To maintain the integrity of control message delivery, the control network has several important requirements, some of which are listed below: -- Control plane message transport must be secure. This requirement stems from the fact that the information exchanged over the control plane is service-provider specific and security is of utmost importance. Thus, service providers may not want to expose internal network information outside their network boundaries even to their own customers. In some cases, control traffic may need to have its own dedicated physical medium or be carried on its own virtual private network. -- Control message transport reliability has to be guaranteed in almost all situations, even during what might be considered catastrophic failure scenarios of the service transport networks. Since the control plane also supports connection restoration functions, failures in the service transport network that also Y. Xu et al. Page [10] Internet Draft GMPLS Architecture for ASTN Nov. 2000 affect the control plane may result in the inability to restore traffic or a degradation in the service restoration time. -- The transport plane connection service performance largely depends on its message transport. Time sensitive operations, such as protection switching, may need certain QoS guarantees. Furthermore, message transport should be guaranteed or recovered quickly in event of control network failure. -- The control network needs to be extensible and scalable in order to make the control plane scalable as the requirements imposed upon it increases. 5. Detailed Architectural Considerations This section covers what happens within an AS of a single technical domain (section 6.1). Network interactions are covered in section 6. 5.1 Network Element Level Resource Discovery 5.1.1 Element Level Resource Table For circuit switched NEs, physical resources are usually dictated by interface ports and their characteristics. Each NE maintains its own resource table indexed by local port ID. The content of the NE level resource table is as below. Physical and logical attributes may span multiple columns. Figure 1 shows a NE level resource table depicting some of the attributes that can be associated with interface ports. +----------------------------------------------------------------------+ | Local Port | Neighbor | Neighbor | Physical | Logical | Oper. | | ID | NE addr. | Port ID | attrs | attrs | State | +----------------------------------------------------------------------+ Figure 1: NE Level Resource Table Physical attributes are technology and vendor specific. Examples are: -- Signal format (SONET, Ethernet, etc.) -- Transmission bit rate -- Optics Type (SR, LR, etc.) -- Multiplexing structure -- Wavelength -- Directionality (Transmit, Receive, Bi-directional) Several physical attributes can be abstracted into one logical attribute. Meanwhile, logical attributes may not be assigned to a particular physical port directly, instead, they may describe characteristics of a physical port pool. Logical attribute examples are: -- VPN ID Y. Xu et al. Page [11] Internet Draft GMPLS Architecture for ASTN Nov. 2000 -- SRLG (Shared risk link group)[OPT-BUND] -- Protection type Operational state indicates the real-time state of a physical port. It includes: -- Unequipped -- In-service -- Standby -- Failed -- Mis-configured -- Unknown 5.1.2 Resource Discovery Stages Resource discovery can be achieved through either manual provisioning or automated procedures. The procedures are generic while the specific mechanisms and control information can be technology dependent. Typically, there are several steps involved in resource discovery: -- Self resource awareness/discovery The results of self resource awareness/discovery is to populate the local ID, physical attributes, logical constraints parameters in the element resource table. -- Neighbor discovery and port association This step discovers the adjacencies in the transport plane and their port association and populates the neighbor NE address and port ID fields in the resource table. Because the control plane network may be different from the transport plane network in ASTN, NEs that are not adjacent in the control plane network may be adjacent in the transport network. In order to unambiguously identify the transport plane neighbors and their port associations, it is essential to have in-band events (along the transport plane) coordinated with other control messages (along the control plane). An example of in-band events for NE capable of electrical signal processing in the transport plane is a byte stream containing the local NE address and port ID. Both [LMP] and [ND-SONET/SDH] adopt this approach. Bi-directional links may use in-band signaling between both ends. Uni-directional links may employ messages through the control network to coordinate with in-band signals to achieve bidirectional control coordination. [AND] For a pure OXC without O-E-O capability, an analog signal (power on/off) could principally be used as the in-band event. Control messages over the control network can then be used to co-ordinate Y. Xu et al. Page [12] Internet Draft GMPLS Architecture for ASTN Nov. 2000 and associate additional significance with the in-band event to identify neighbor adjacency and port associations. Details of this type of scenario are for further study. -- Resource verification and monitoring After neighbor resource discovery, neighbors should detect their operation state and verify their configurations such as physical attributes in order to ensure compatibility. Such verification can be done through control message over the control plane network without using in-band signals. In case of any mismatch, the port should be marked as mis-configured, and notification should be issued to operators. Resource monitoring is a continual process. Neighbor discovery and port association procedures are repeated periodically. Resource monitoring procedures update resource state and report changes to relevant control entities. -- Service negotiation Service negotiation essentially covers all aspects related to service related to rules/policy negotiation between neighbors. A typical example is the negotiation of rules for label binding and assignment. In this case, neighbors build up a label binding and look-up table according to the physical connectivity map. They may exchange information on technology specific details such as multiplexing schemes. They may also negotiate the semantics for complex labels, and decide upon label assignment order. 5.2 State Information Dissemination In link state routing, ASTN links can be represented either as point-to- point or non-broadcast multiple access links. 5.2.1 Logical Link/Bundle Typically, core circuit switches contain thousands of physical ports. Therefore, scalability requires minimizing global information and keeping information and decision making locally as much as possible. For this reason, link aggregation plays a key role in ASTN topology representation. Parallel link connections can be summarized into an aggregate logical link, which hides the link connection details not necessary for network level control functions, such as path selection. The summarized information is then distributed to the rest of the network. Y. Xu et al. Page [13] Internet Draft GMPLS Architecture for ASTN Nov. 2000 A logical link or bundle is defined as a collection of link connections that share some common logical or physical attributes that are significant for path selection purposes. Link connections within a bundle are equivalent for routing purposes. However, the notion of equivalence is not static. It is configurable according to the service providers' requirements. The granularity of bundling abstraction can range from physical tributary to node and even to AS. The granularity of bundling can also be used to balance between network operational performance optimality and control plane scalability. The more details that are depicted in the topology database, the more accurate the path selection could be. However, too much information could result in an implementation that is not scalable. The trade off needs to be carefully considered and engineered. Bundling details are technology specific. Below are some examples of bundling for different technologies: -- SONET/SDH 4 OC48s with the same multiplexing structure and going to the same neighbor. -- Wavelength 100 protected wavelengths with wavelength convertibility and going to the same neighbor 5.2.2 Link State Information Link state information includes both static information and dynamic information. This information may be technology dependent. -- Static information is usually required for circuit operation. It includes neighbor connectivity, logical link attributes (which could be the same as individual attributes), total bandwidth and etc. -- Some dynamic information are required for circuit LSP operation, while others are mainly used for traffic engineering purposes. Dynamic information includes information that changes constantly, such as bandwidth availability, current bandwidth fragmentation information, etc. The total number of attributes in a ASTN can be huge. The decision as to what should be distributed and what could be negotiated during connection setup requires a compromise between control plane scalability and connection operation performance. The more the number of attributes that are conveyed in link state information, the more accurate the path selection decisions. However, it may result in reduced control plane Y. Xu et al. Page [14] Internet Draft GMPLS Architecture for ASTN Nov. 2000 scalability. In general, scalability and simplicity should be favored above abundance of information and call setup speed in case of irreconcilable conflicts. Stability is another crucial consideration. The more the amount of information conveyed and the more frequent their rate of change, the more the network will tend towards less stable operating regimes. The objective should be to distribute just enough link state information to support connection set-up operations. More information could be distributed provided it does not affect control plane scalability and stability. Furthermore, information reduction mechanisms should be utilized whenever possible. The basic concept is to only flood what is needed to accomplish specific control goals. For example, static information and dynamic information could be flooded independently. To build up the initial network topology database, all pertinent static and dynamic information needs to first be disseminated. Subsequently, as long as static information remain unchanged, only the dynamically changing information needs to be flooded. 5.2.3 Link State Dissemination Mechanisms Link state information can be piggybacked through IGP or EGP of the control plane network according to network application scenarios. For example, mechanisms introduced by [OSFP-OMPLS]/[ISIS-OMPLS] can be used to disseminate ASTN link state information within a single area. Alternately, optical link state and general ASTN advertisement can be built as a separate application on top of any control network. This solution can tailor existing IGP/EGP's behaviors to ASTN and meanwhile permits flexibility in choice of the underlying control plane transport network. 5.2.4 Flooding Control Topology updates should happen only when a significant topology change occurs or upon expiration of a periodic timer. -- Static information change requires immediate link state information flooding. -- Flooding of dynamic information change is triggered by thresholds as defined below. -- Periodic topology refreshing is needed to ensure that network topology information in all NEs is consistent and correct. Some dynamic attributes are important for connection operation. For Y. Xu et al. Page [15] Internet Draft GMPLS Architecture for ASTN Nov. 2000 example, to SONET/SDH network, bandwidth continuity is critical for connection operation. Because of frequent connection operations, a SONET/SDH connection may become fragmented. An OC48 pipe with total 24 free STS-1s left doesn't mean that it can accommodate an OC-12c connection request. So bandwidth continuity change should also be distributed in order to avoid rejection of connection requests. Other dynamic attributes are important for traffic engineering in ASTN. For example, the bandwidth consumption state of a logical link may be used to adjust link cost and balance network traffic. The frequent change of link states requires frequent flooding in order to keep the topology database updated. However, there is a tradeoff between accuracy of the topology and the scalability of the control plane as note earlier. Operators can control link state flooding by configuring thresholds and frequencies according to a particular network situation. Generally, there are two types of thresholds: absolute threshold and relative threshold. -- Absolute threshold defines change relative to total resource. -- Relative threshold describes changes relative to previous available resource. Relative thresholding is preferable to absolute thresholding. Using bandwidth availability as an example, it is desirable in general to advertise less frequently when a link is lightly loaded and increase the frequency as the link loading increases. This is automatically accomplished by defining a threshold for the relative change in available bandwidth. There could be thresholds for key attributes of a logical link. A significant change of any relative attribute could trigger flooding. Details of how to set thresholds to control flooding are outside the scope of this document. 5.3 Path Selection Path selection could be done either off-line or on-line. The path selection algorithms may also be executed in real-time or non-real time depending upon their computational complexity, implementation, and specific network context. Also, off-line computation is typically a centralized operation whereas on-line computation is typically distributed. -- Off-line computation may be facilitated by simulation and/or network planning tools. Off-line computation can help provide guidance to subsequent real-time computations. Y. Xu et al. Page [16] Internet Draft GMPLS Architecture for ASTN Nov. 2000 -- On-line computation may be done whenever a connection request is received. Off-line and on-line path selection may be used together to make network operation more efficient. Operators could use on-line computation to handle a subset of path selection decisions and use off- line computation for complicated traffic engineering and policy related issues such as demand planning, service scheduling, cost modeling and global optimization. In case of on-line computation, there are two choices: explicit routing or hop-by-hop routing. -- In ASTN, explicit routing is preferred as it provides superior traffic engineering capabilities. Explicit routing may in fact be mandatory if multiple constraints are considered in the path selection decision. -- Hop by hop routing can make more accurate next hop choice because each node can consult its local resource information that may be unavailable to other nodes. However, hop-by-hop routing requires path calculation at each node and needs loop-prevention mechanisms. Hop-by-hop routing is also more difficult when constraints other than additive link metrics are taken into account. 5.3.1 Routing Constraints in ASTN There are potentially many routing constraints of interest in ASTNs. Some of them are technology specific, such as optical constrains addressed by [OLCP]. In the following, we highlight some of the major constraints of interest, but the list is not meant to be exhaustive. Also, the discussion is in general equally applicable to all types of circuit networks. However, we highlight any differences where applicable. One of the major services provided by a transport network is restoration. Restoration introduces several routing constraints. A major one is that of physically diverse routing. Restoration is often provided by either pre- computing or finding alternate routes for connections in real-time. These alternate routes need to be physically diverse from the primary route in at least the failed link for efficiency purposes or completely diverse. The simplest form of diversity is node disjointness of the primary and alternate routes. More complete notions of diversity can be addressed by logical attributes such as shared risk link groups (SRLG). These logical attributes are abstracted by operators from several physical attributes such as fiber trench ID and destructive areas. Because methods of automatically identifying physical separation do not exist yet these physical attributes require operators' configuration. Y. Xu et al. Page [17] Internet Draft GMPLS Architecture for ASTN Nov. 2000 In order to address this constraint, path selection algorithms are needed that can find a diverse pair of routes in the specified fiber graph. Algorithms for node-disjoint diverse routes have been developed [DISJ-PATH]. Another restoration constraint is related to shared capacity mesh restoration architectures [ON-REST]. In these architectures, restoration capacity is shared among multiple demands. In order to guarantee that restoration capacity is available on a chosen route, one needs information on the free capacity on that route, demands whose alternate routes share that route or some links on it and their failure sets. Consider two demands that do not share any link on their primary routes but share parts of an alternate route. In this case, the capacity on the alternate route can be shared between the two demands since the primary routes (and hence demands) can hardly at the same time under single-failure scenarios. Optimal routing in these cases requires considerable network level information and good heuristics that limited information about the network and routing of demands. Some heuristic approaches are proposed in [ON-REST]. However, further study is needed to develop a complete solution. Other constraints of interest include node/LSP inclusion/exclusion, propagation delay, wavelength convertibility and connection bandwidth among others. Node exclusion involves specifying a set of nodes that should be avoided for routing. These are typically specified on a connection basis. An interesting application of node exclusion has been proposed by Japan Telecom [DIST_AREA]. Certain office sites of this carrier are in earthquake prone areas and to the extent possible it is desired to route traffic of premium customers so as to avoid these sites. A possible application of node inclusion is to route traffic through a site that has some special equipment such as for connection monitoring. Since transport connections are physical connections involving fixed time slots or wavelengths, there are no queuing delays like in packet networks. However, propagation delay may be a factor for large global networks and especially networks with satellite links. Consider traffic from New York to London on a global network. It may be undesirable to route this traffic the long way around the world (over the Pacific Ocean) and rather choose the short route over the Atlantic Ocean. This can be realized by requiring the propagation delay of the chosen route to be less than a certain value. Wavelength convertibility is a problem encountered in wavelength/waveband networks. It refers to the ability to cross- connect two different wavelengths. The wavelengths may be completely disparate or just slightly different due to differences in the ITU grid wavelengths supported by a network vendor. Wavelength convertibility is typically accomplished via use of electrical transponders. Owing to the cost of OEO conversions, operators may Y. Xu et al. Page [18] Internet Draft GMPLS Architecture for ASTN Nov. 2000 choose to selectively deploy transponders. In this case, end to end connections have to employ the same wavelength through the network. This constraint introduces considerable challenges in path selection. In order to determine if a connection can be supported on a given route, path selection needs to determine if the same wavelength, say red, is available on each leg of the route. This requires that the path selection be aware of all the available wavelengths on each leg of the route. This information may be unavailable if link bundling includes all wavelengths on a link into the same bundle. When routing is carried out in a distributed manner at source NE of the connection, this implies that considerable local information about the status of network links needs to be distributed across the network. An alternate less efficient approach might be to guess availability of wavelengths on a computed route and determine feasibility of the route via local probing of link resources along the computed route. However, this approach is likely to result in considerable connection crank-backs and wasted network processing resources. When wavelength convertibility is available at each intermediate NE, however, no information about availability of particular wavelengths on a link is needed. In this case, the total number of available wavelengths is sufficient for routing purposes. The above discussion applies equally to TDM circuit networks by considering time slots rather than particular color wavelengths. Time slot interchange capabilities are analogous to wavelength convertibility. Bandwidth availability is an important consideration in routing. This is simplified in a wavelength optical network since all requests are for end to end wavelength paths. However, in a TDM transport network such as SONET/SDH, connection requests can be variable bandwidth ranging from VT1.5 to as high as OC-768 with current technologies. Routing needs to ensure that sufficient network capacity is available on the chosen route to support the connection. Further challenges are introduced by different concatenation schemes in SONET/SDH networks. An example would be a contiguous concatenated STS-3c signal, which require three adjacent time slots be allocated. This implies that a time slot map of the link be distributed to all nodes in the network to facilitate routing decisions. Alternately, in the logical link representation, one would need N different logical links to represent all possible STS-N signals. This could be expensive in terms of control network bandwidth. A preferable approach is to advertise just the largest contiguous block of time slots available on a logical link instead of the entire time slot map. This is sufficient to determine supportability of a connection on a link. Note that the detailed information on local resource availability is only used for routing decisions. Actual assignment of local resources to a connection is performed by individual NEs. Y. Xu et al. Page [19] Internet Draft GMPLS Architecture for ASTN Nov. 2000 5.3.2 Path Selection for Traffic Engineering TBD 5.3.3 Policy Consideration in Path Selection Operators' policy aspects of routing should also be considered. While distributed, dynamic path selection has significant advantages such as scalability, it is often advantageous to provide the network operator some degree of control over path selection. Such control can be provided in multiple ways. One possibility is to allow the operator to provide a set of preferred routes between a source destination pair. Automatic path selection may then choose between these preferred routes. Alternately, static link weights could be employed. The automatic path selection may then choose routes, when feasible, that minimizes the static link weight. But the preferred routes should typically be used only as a guide as in the extreme case they reduce to fixed pre-determined routes and one loses the flexibility to correct any mismatch between network capacity and traffic patterns. 5.3.4. Result of Path Selection The result of explicit path selection is a linked list of explicit routed hops. In order to provide just enough information for each NE to choose the right physical resource for circuit connection operation, it should be in logical link level of details. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ER-Hop (Mandatory) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Logical Link ID (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2: Path Selection Result Explicit hop can be either node ID, AS ID or existing LSP ID as defined in current MPLS signaling protocols. The logical link (bundle) is defined in 5.2.1. It's flexible in usage. For example, a logical link ID can be a VPN ID. If a low order SONET LSP needs to tunnel through a existing high order SONET VPN, it can use this VPN ID to restrict which resource to use. This concept is similar to LSP tunnel, except, instead within a LSP, it's within a pre-established circuit VPN. Y. Xu et al. Page [20] Internet Draft GMPLS Architecture for ASTN Nov. 2000 5.4 Path Control 5.4.1 MPLS Control Plane Concepts for ASTN In data networks, MPLS covers both the control plane (label binding, label distribution and etc.) and the transport plane (packet forwarding). Label binding associates FECs to labels; label distribution distributes label-binding information. Together with mapping between FEC and next hop, the label switching forwarding table is constructed to forward packets. In circuit switched networks, there is no packet forwarding. Labels are only used to set up LSPs by making cross connects. After LSPs are set up, there is no dynamic label switching at all. So only the MPLS control plane components are truly applicable to circuit switched networks. There are several ways to apply MPLS control plane concepts in circuit switched networks. Given that circuit paths are service- driven, one possible understanding is to treat the service request as a FEC. It contains next hop (or destination address for hop by hop routing) and incoming physical interface (e.g., if negotiation is not required at connection creation time) or path constraints ( e.g., if negotiation is required). Because labels are used for traffic forwarding, they are analogous to cross connect ID. Label binding is then conceptually equivalent to choosing cross-connection across the switch matrix. An alternate interpretation can be constructed as follows: In circuit switched networks, setting up a LSP means making cross connects along the desirable path. Labels are used to determine cross connect configurations. Assuming a non-blocking switch fabric, determining a cross-connect means determining external ingress and egress physical interfaces. So labels are used to indicate specific physical interfaces. Label binding entails choosing the correct ingress or egress interface according to path selection decision and local policies. In both of the above interpretations, making cross connects is equivalent to writing to a label switching forwarding table. After the path is set up, user traffic simply follows the dedicated physical path (cross connect). The concepts below elaborate on the second interpretation. 5.4.1.1. Upstream and Downstream Upstream and downstream in circuit switched network are based on path create request direction. They are irrelevant to user traffic direction. The NE that receives the request first is treated as an Y. Xu et al. Page [21] Internet Draft GMPLS Architecture for ASTN Nov. 2000 upstream NE relative to the NE that receives the request later. 5.4.1.2. Generalized Label in ASTN Because the label is used to identify the physical resource in circuit switches, first we should understand how physical resource is represented in circuit switches. Typically, a specific resource in circuit switches can be described as a concatenation of port ID and channel/sub-channel ID within the port (assuming non-blocking switch fabric). -- Port identifiers are product dependent. Different vendors have different naming and addressing structures for their products. For example, a port ID may be prefixed with bay ID, shelf ID and slot ID. However, neighbors don't need to understand the semantic of each other's port ID. The port association information in the NE resource table can serve as a look-up table. In the example below, when A asks B for a connection, A will find B's port ID through its resource table and use it when communicating with B. +-----+ +-----+ ------|\ | | /|-------- | \ | | / | | \ | | / | | \ | x y| / | | \|------------------|/ | | | | | | | | | | | | | | A | | B | +-----+ +-----+ Figure 3: Label Illustration -- Channel/Sub-channel ID is technology dependent. Generally, there are different multiplexing structures for different technologies. Channel/Sub-channel ID indicates the channel/sub-channel within a port. For example, within an OC-48 port with STS-1 granularity, each STS-1 may have a channel/sub-channel ID. For two physical ports connected by a physical link, these two ports should share common physical attributes including multiplexing structure, which are technology specific and clearly defined by standards. Y. Xu et al. Page [22] Internet Draft GMPLS Architecture for ASTN Nov. 2000 So the label format can be defined as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | G-label (port ID section) | | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | G-label (channel/sub-channel ID section) | | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4: G-Label Format For G-label channel/sub-channel section, technology specific content can be either encoded in certain fixed format as suggested by [GMPLS- SIG] for each technology or mapped through a separate label association procedure at resource discovery time with technology specific rules. Furthermore, for complex labels, such as different concatenations in SONET/SDH, extra rules and information may need to be provided. Again these rules are technology specific. They can be either hard-coded as suggested by [GMPLS-SIG] or negotiated between neighbors at resource discovery time. 5.4.1.3. Label Space For out-of-band control signaling, only "per-platform label space" is applicable to circuit switches. In this case, both port ID and channel/sub-channel ID sections are required. In case of in-band control signaling, the port section is not necessary, labels may have "per-interface label space". 5.4.1.4. Label Binding Label binding include both FEC to label binding and label to next hop binding. Label to next hop binding in ASTN is hard-wired. FEC to label binding in ASTN is selecting a specific physical resource according to path selection result and local rules. It is a service- driven and happens at the same time when label distribution happens. The procedure for choosing a specific physical resource requires local intelligence. A path create request message only contains logical link level information. When a node receives the request, it typically has many physical resources to choose. In order to smooth path creation Y. Xu et al. Page [23] Internet Draft GMPLS Architecture for ASTN Nov. 2000 operation, there could be local rules for choosing a specific resource. For example, in bi-directional path create request, in order to minimize the chance of neighbors choosing the same interface, adjacent nodes may form a master-slave relationship and decide different orders of selecting interfaces. 5.4.1.5. Label Distribution Label distribution in circuit switched network is equivalent to signaling mechanism for circuit path operations. It's always ordered and on-demand. MPLS signaling mechanisms can be extended as signaling protocol in circuit switched networks. Path operations consist of a set of basic transactions between adjacent NEs. Each transaction may involve multiple signaling messages. The basic transactions are: -- Create: This transaction creates a link of an end-to-end path. Although there is only one create transaction for the end-to-end path, the resource allocation is made on a link-by-link basis. MPLS signaling protocols always involve label request and response. They can be mapped into path create request and response directly. In general, most of circuit path constraints should be considered at path selection time rather than negotiated at path creation time. However, some constrains, for example, the label set [GMPLS-SIG], could be considered at the path creation time by using label request and response messages. These constrains are normally expensive to be considered at path selection time because that either control plane scalability may be hurt or path selection algorithms could become too complicated. In the explicit routing case, a create request should contain information that is essential for path creation, e.g., ER hop list, current ER hop pointer. If the incoming label can be decided, it should be given at the request time so that NEs can minimize operation delay by allowing circuit switches to perform cross connect and wait for response from other NEs in parallel. In the example of figure 3, NE A could issue the cross connect request to the local system controller and send label the request downstream to B simultaneously. Create response return LSP ID. -- Delete: This transaction de-allocates a specific link of a LSP identified by its LSP ID. Y. Xu et al. Page [24] Internet Draft GMPLS Architecture for ASTN Nov. 2000 This request contains the LSP ID to be torn down. After a NE receives this request, it releases local cross connect according to the LSP ID and meanwhile forwards the request downstream. Release and withdraw messages in MPLS signaling can be mapped to delete request and delete response. -- Query: This transaction queries the status of a specific link of an existing LSP identified by a LSP ID. Current MPLS signaling needs to be extended to support this function. 5.4.1.6. Circuit LSP Tunnel New LSPs can tunnel through existing LSPs of higher granularity. After a circuit LSP is created, it can be treated as a FA in the topology database. A new circuit LSP can tunnel through it by indicating the LSP ID in its ER hop list. For example, if a low order (LO) SONET LSP needs to tunnel through an existing high order (HO) SONET LSP, the LO LSP create request includes the HO SONET LSP ID in its ER hop list. Its label only needs to indicate LO resource within the existing HO LSP. 5.4.1.7. Circuit LSP Concatenation A new LSP can also be concatenated through several existing LSPs of the same type. These LSPs must share the same characteristics. Their LSP IDs are also included in path create request ER hop list. 5.4.2 LSP Information Maintenance LSPs can be uniquely identified by LSP IDs. In circuit switched networks, A LSP ID can be defined as a source NE address + a source NE ingress label. An LSP has two sets of information: ER hop list and local resource usage information. ER hop list should be maintained at the ENE. Both ENEs and INEs should maintain the LSP ID and associated local physical resource usage information. 5.4.3 Circuit LSP Restoration 5.4.3.1 Restoration Stages Circuit LSP restoration is very important for transport network and quite different from packet LSP in terms of both requirements and Y. Xu et al. Page [25] Internet Draft GMPLS Architecture for ASTN Nov. 2000 mechanisms. Circuit LSP restoration typically includes following four stages. -- Failure Detection -- Failure Notification -- Traffic Restoration -- Post-Restoration Cleanup Mechanisms of these four stages are technology dependent. Some technologies provide special infrastructure to speed up path restoration. For example, SONET/SDH detects LOS and LOF very quickly and uses in-band overhead bytes for failure notification and co- ordinate actions. They also adopt special topology (ring) and algorithms (BLSR, UPSR and etc.) for fast restoration. In order to speed up overall restoration time, each stage has to be designed carefully and utilize existing infrastructure as effectively as possible. For example, failure notification should avoid message- based indication if byte/bit stream based indication is available. In case of message based notification, direct notification to end points is more desirable than hop-by-hop processing. For traffic restoration, handshaking and negotiation should be avoided as much as possible. Circuit LSPs can be either link protected or path protected. 5.4.3.2 Link level protection. In link level protection, restorations happen around the point of failure, e.g., either the failed NE or link. In SONET/SDH, BLSR/SPRing is a typical link level protection. This type of restoration requires accurate fault localization, i.e., pinpointing the precise location of the fault along the LSP so that rerouting can be performed around it. In optical networks, fault localization typically involves OEO conversion at every network element so that digital monitoring of the optical channel (OCh) overhead can be carried out. However, installing such digital monitoring equipment to determine performance degradation might be very expensive. Alternatively, controlling the expense by sharing the monitoring equipment over many optical channels leads to an unacceptably long detection time for degraded signal condition (fault) owing to correlation that have to be performed across multiple monitoring nodes. 5.4.3.3 Path level protection Path level protection is a more popular choice for mesh network. In this case path restoration is done from the endpoints. Failure notification need to be propagated to end points where the restoration is triggered. Y. Xu et al. Page [26] Internet Draft GMPLS Architecture for ASTN Nov. 2000 There are several types of path protections. -- Dedicated protection This type of protection is the fastest and also the least cost effective. It requires a dedicated physically diverse protection path for each working path. Physically diversity could have different meanings depending on the degree of threat to be protected against. It can be node disjoint, link disjoint and span disjoint. SRLG could be assigned to physical link by operators to meet different requirements. For example, an operator can assign physical links passing same disaster zone the same SRLG ID to guarantee services even in case of catastrophic disasters such as earthquakes, and floods. -- Shared protection Shared protection is fast and more cost effective than dedicated protection. In this case, multiple paths could share the same reserved protection sections (not necessarily paths). However, this requires more information to be distributed in the network and complicated algorithms. A scalable, effective and efficient solution needs to be designed. -- Auto-rerouting Auto-rerouting is a non-preplanned protection. No dedicated resource is reserved for working paths. A protection path is created after detecting the failure of a working LSP. The auto-rerouting doesn't need to wait for network topology convergence. A new path can be created by excluding failed nodes in case that they can be identified on-the-fly. Alternatively, when failures cannot be precisely located on-the-fly, a new path can be created simply by excluding all nodes along the failed path. 5.4.4 Circuit LSP Verification Serving as transport backbone, circuit LSPs are normally required to be strictly verified before delivering to customers. For ASTN, some verification procedures can be automated. After a circuit LSP is created, OAM&P signals or simulated user traffic can be inserted and tested to verify against path connectivity, constraints and some aspects of SLA. 5.4.5 Control Plane Failure Handling Failures may happen at both the control plane and the transport plane. Many current MPLS mechanisms have the assumption that control plane failure indicates transport plane failure because control Y. Xu et al. Page [27] Internet Draft GMPLS Architecture for ASTN Nov. 2000 traffic and data traffic are mixed together in the same physical path in IP network. So MPLS signaling mechanisms require existing LSPs to be torn down in case of control plane failure. This is undesirable for circuit switched network, where control plane and transport plane may be separated and circuit LSPs are usually backbones for user traffic. One way to handle control plane failures is to keep current circuit LSPs in operation but abort requests that are being processed. Details for performing graceful control plane failure handling and recovery need to be furthered studied. For example, notification messages should be extended for both types of failures. Also control plane recovery mechanism should be introduced. In case that a NE controller recovered, it may need to consult with its neighbors to synchronize link state information and current LSPs status. With control plane taking over many responsibilities from the management plane, control plane failure also has significant impact on management plane. For example, the billing system may rely on NE control entity for resource usage information. Details of these implications need to be further studied. 6. Inter-domain Control Plane Integration One of major purposes of a generalized MPLS control plane is to facilitate network inter-operation. Domains are defined to facilitate network partitioning for connection management purposes. Typically, networks can be divided into technology and business domains. Consequently, there are two types of interfaces between domains: -- Business related interfaces are between administrative domains. -- Technology related interfaces are between different technology domains. The following sections focuses on this topic. 6.1 Technology Domains and Hierarchy Circuit networks can be divided into logical network layers according to networks' switching granularity even though they may be within a single administrative domain. These layers form typical client/server relations, with lower layer network provides transport service to upper layer networks. Figure 6 shows examples of this hierarchy (not exact signal mapping and encapsulation). For example, Optical networks (OCh granularity) provide transport service to SONET/SDH HO networks, which in turn provide transport service to SONET/SDH LO Y. Xu et al. Page [28] Internet Draft GMPLS Architecture for ASTN Nov. 2000 networks. An IP network can use different circuit switched transport networks as its direct server network. Edge/Access routers use DS-1 or DS-3 networks as their server network, while core routers may request STS or optical channel services directly. PDM (packet/cell) ____________________________________________________________ | | | TDM LO (DS-1, VT1.5 etc.) | | | ______________________________V_______ | | | | | TDM HO (DS-3, STS-1 etc.) | | | __________________________________V______V____ | | | WDM (OCh/OMS) | | ______________________________________V_____________V___ | SDM (OTS) | __________________________________________V_________________ Figure 6: Network Hierarchy 6.2 Service Interfaces Between Network Domains There is a clear business related service interface between networks owned by different service providers. This interface is identified as public service interface [CARR-REQ]. This type of interface includes service requests from both the same type and different types of networks. A carrier will not relinquish control of its resources outside of its administrative boundaries. No outside entity will be allowed to reconfigure the network directly. They can only take requests through the service interface. For different types of networks owned by the same service provider and provisioned within a single AS, there are still logical service interfaces between any two of them. These interfaces are identified as private service interfaces [CARR-REQ]. For example, if a service provider owns both an IP network and a switched optical core transport network, traffic that flows in the IP network shouldn't affect the optical network directly. Optical LSPs are created by different reasons and set up at different time from packet LSPs. For example, it's not practical to set up an OC48 pipe for a 64Kps IP stream. LSPs Y. Xu et al. Page [29] Internet Draft GMPLS Architecture for ASTN Nov. 2000 in transport network are normally created according to the traffic pattern and aggregated traffic requirements of the IP network. Then they serve as tunnels for IP traffic. Functions at the private service interface forms a base set for network interaction. This base set includes four functional components covered in section 5 and additional service triggering function, which is normally the traffic engineering function in the client network that analyzes client network traffic and request service from server ASTN accordingly. At the public service interface, functions include the base set and stricter security and policy controls. 6.3 Control Plane Design for Integration Each technology has its own unique features, requirements, and evolution path. It is preferable for networks of different technologies to have independent control entities. While it is, in principle, feasible to integrate them, it is impractical and unnecessary for efficient network operations. However, different control entities don't require different control plane protocols. On the contrary, unified control plane protocols with technology specific extensions could facilitate network inter- operation and network application migration. It should be noticed that different control entities don't need to be physically separated. In reality, there may be no clear physical boundary between networks. For example, a network can be made of hybrid IP/SONET/DWDM NEs. In this case, service interfaces are within the NEs. The four functional modules discussed in section 5 for a single domain are revisited below in a multiple technology domain environment to examine how they can support interactions of network from different technology domains, what are the extensions, and how security and policy control should be apply. 6.3.1 Resource Discovery Resource discovery happens between neighbors. A mechanism designed for a technology domain can be applied to any pair of NEs interconnected through interfaces of the same technology. However, because resource discovery means certain information disclosure between two business domains, it's under service providers' security and policy control. In certain network scenario, a service Y. Xu et al. Page [30] Internet Draft GMPLS Architecture for ASTN Nov. 2000 provider who owns the transport network may not be willing to disclose any internal addressing scheme to its client. So a client NE may not have the neighbor NE address and port ID in its NE level resource table. 6.3.2 State Information Dissemination Topology information may be proprietary to service providers. So state information dissemination should be under careful control. There are different levels of information dissemination controls between networks of different technology domains. Routing mechanisms introduced in [IPO-FRAM] could be extended to networks of other technologies. Normally, a single type of information dissemination control can not meet all needs of a service provider. ASTN is configurable to provide service provider controls for different applications. -- Total Dissemination without Control Topology dissemination is done through control plane network IGPs or high layer applications. -- Controlled Dissemination In this case, partial ASTN topology information is disseminated to designated targets. Policy based EGPs can be extended to apply here. For one type of circuit VPN services, a customer may lease specific resources (internal bandwidth, not only access points) from a service provider and want to manage these resource by itself. The customer may require the service provider to only leak its VPN topology to the customer. -- No Information Dissemination This is the most widely used practice. For different types of networks that belonging to different service providers, there is normally no link state information dissemination between them at all. For another type of circuit VPN services, customers lease access points instead of specific internal resource of a backbone ASTN. Then, from the customer's view, the network appears as a non-blocking backplane, independent of any internal network structure. 6.3.2.1 Potential Forwarding Adjacency NEs of different technology domains don't need to know each other's network topology. They should be transparent to each other. What a NE should care is the topology of its own technology domain. Y. Xu et al. Page [31] Internet Draft GMPLS Architecture for ASTN Nov. 2000 Enabled by ASTN, there are three types of routes in a client topology database: (1) fixed routes, (2) Forwarding Adjacencies (FA) and (3) Potential Forwarding Adjacencies (PFA). -- Fixed routes are conventional in current IGP and EGP. -- FA is introduced by [LSP-HIER]. It's the route that were dynamically created and can be torn down if necessary. -- PFAs are routes that can be connected through a low layer network, but are not connected yet. Enabled by ASTN, NEs that directly connected to a ASTN become FFAs. Link state dissemination (for PFAs and potential routes that are reachable through PFAs) can be done either through existing network connections or the control plane network of the ASTN that provides transport services. In the later case, link state dissemination is under the control of server ASTN client management. The server ASTN should maintain client information such as where a client is attached, which user group a client belongs to, and what's a client's service contract/policies. Serving as a EGP proxy with configurable filters, the server ASTN determines whose information to distribute to which client. With these three types of route information in the topology database, the service agent (section 4.2) at the client network could decide when and where to set up or tear down FAs for more efficient network resource utilization according to client network data traffic pattern and requirements. It then requests the service from ASTN that provide the transport service accordingly. One of PFA's applications is enabling the virtual terabit router. It's especially useful for hybrid NEs with both IP switch fabric and circuit switch fabric. When interconnected together, they can aggregate traffic and choose appropriate circuit shortcuts by dynamically adjusting their circuit connections through circuit switch fabric in order to avoid unnecessary packet switching and minimize user traffic port consumption for NE interconnection. 6.3.3 Path Selection Path selection can be done by the party having the access to network level resource information. As path selection is very technology dependent, it's suggested that it be done within domains (both business domain and technical domain). However, service providers have the flexibility to do whatever they want. Y. Xu et al. Page [32] Internet Draft GMPLS Architecture for ASTN Nov. 2000 6.3.4 Path Control 6.3.4.1 Service Invocation In packet networks, LSPs are created either because of topology changes (control-driven) or data traffic flow (data-driven). In ASTN, circuit LSPs are service-driven (could be view as one type of control-driven). They are created either because of customer requests or client network traffic engineering decisions. In the signaling perspective, even though intra-domain and inter- domain signaling may use the same protocol, their message content are quite different. In the network view, service invocation happens between a service agent and a SNE. Service requests include create, delete, modify and query. Details are being studied by OIF and ODSI. 6.3.4.2 LSPs of Different Technology Domains LSPs in different technology domains are treated as different entities. They are set up at different time and triggered by different reasons. Coarse granularity LSPs can serve as tunnels for fine granularity LSPs. 6.3.4.3 LSPs over Multiple ASes In data networks, label stacking is only used for packet forwarding over multiple sub-networks. There is no such equivalence in circuit switched network. Nevertheless, ER hops can be stacked for LSP creation through multiple ASes. In case of explicit routing, an SNE may only include AS IDs in its ER hop list. When a path create request enters a transmit AS, its AS SNE needs to compute a ER hop list for the AS and push it up to the original ER hop stack. Before the request leaving the AS, its AS DNE needs to pop up the domain ER hop list. When path operation messages passing across AS boundaries, they usually should include some authentication information. Each AS needs to validate these information in order to guarantee network operation integrity. Y. Xu et al. Page [33] Internet Draft GMPLS Architecture for ASTN Nov. 2000 6.3.4.4 LSP Restoration -- A single network failure may affect LSPs of different technical domains. This topic involves multi-layer restoration. For details, please refer to [MUL-SURV]. -- For a LSP over multiple ASes (business domains), restoration could be done either end-to-end or within each AS depending on SLAs of the user to service provider and service provider to service provider. Details and implications to signaling protocols need further study. 7. Security Considerations Security is critical where different business domains interact. When service invocations happen between business domains, encryption and authentication mechanisms are required at the service interfaces. Within a single business domain, strict security needs further study. However, the control plane network should be isolated from user traffic to avoid sensitive information being leaked out. Many details need further study. 8. Intellectual Property The IETF has been notified of intellectual property rights claimed in regard to some or all of the specification contained in this document. For more information consult the online list of claimed rights. 9. Acknowledgment The authors would like to thank Patrice Lamy, Wyley Robinson, George Newsome, John Ellson, Harvey Epstein, Zhi-wei Lin, Siva Sankaranarayanan, Dave Chen, and Girish Pahlya for their helpful suggestions and comments on this work. 10. Authors' Addresses Yangguang Xu 21-2A41, 1600 Osgood Street Lucent Technologies, Inc. North Andover, MA 01845 Email: xuyg@lucent.com Y. Xu et al. Page [34] Internet Draft GMPLS Architecture for ASTN Nov. 2000 Eve Varma 3C 509, 101 Crawfords Corner Rd Lucent Technologies, Inc. Holmdel, NJ 07733-3030 Email: evarma@lucent.com Ramesh Nagarajan 3M-335, 101 Crawfords Corner Rd. Lucent Technologies, Inc. Holmdel, NJ 007733 Email: rameshn@lucent.com Rudy Hoebeke Alcatel Francis Wellesplein 1, 2018 Antwerp, Belgium Email: rudy.hoebeke@alcatel.be Dimitri Papadimitriou Alcatel F. Wellesplein 3, B-2018 Antwerpen, Belgium Email: dimitri.papadimitriou@alcatel.be Hirokazu Ishimatsu Japan Telecom Co. Ltd. 2-9-1 Hatchobori, Chuo-ku, Tokyo, 104-0032 Japan E-mail: hirokazu@japan-telecom.co.jp Osama S. Aboul-Magd Nortel Networks P.O. Box 3511, Station "C" Ottawa, Ontario, Canada K1Y - 4H7 Email: osama@nortelnetworks.com Michael Mayer Nortel Networks P.O. Box 402 Ogdensburg NY 13669 Email: mgm@nortelnetworks.com Daniel O. awduche.com UUNET/Worldcom 22001 Loudoun County Parkway Ashburn, VA 20147 Email: awduche@uu.net Y. Xu et al. Page [35] Internet Draft GMPLS Architecture for ASTN Nov. 2000 Yong Xue Global Network Architecture UUNET/WorldCom Ashburn, Virginia Email: yxue@uu.net Curtis Brownmiller WorldCom Richardson, TX 75082 Email: curtis.brownmiller@wcom.com John Eaves TyCom 1C-240, 250 Industrial Way West Eatontown, NJ 07724 Email: jeaves@tycomltd.com Raj Jain Nayna Networks, Inc. 157 Topaz St. Milpitas, CA 95035 Email: raj@nayna.com Y. Xu et al. Page [36] Internet Draft GMPLS Architecture for ASTN Nov. 2000 11. References [MPLambdaS] D. Awduche, etc. al., "Multi-Protocol Lambda Switching: Combining MPLS Traffic Engineering Control with Optical Crossconnects", draft- awduche-mpls-te-optical-02.txt, Work in Progress, July 2000. [MPLS-ARCH] E. Rosen, et. al. "Multiprotocol Label Switching Architecture", draft-ietf-mpls-arch-07.txt, Work in Progress, July 2000. [MPLS-TE] D. O. Awduche et al., "Requirements for Traffic Engineering over MPLS", RFC 2702, September 1999. [GMPLS-SIG] P. Ashwood-Smith, et al., "Generalized MPLS - Signaling Functional Description", draft-ietf-MPLS-generalized-mpls-signaling-00.txt, Work in Progress, Oct. 2000. [OPT-BUND] B. Rajagopalan et al., "Link Bundling in Optical Networks", draft- rs-optical-bundling-01.txt, Work in Progress, September, 2000. [AND] H. Koay, Y. Xu, "Automatic Neighbor Discovery for Optical Networks", draft-koay-mpls-and-optical-00.txt, Work in Progress, Sept,2000 [LMP] J. Lang et al., " Link Management Protocol", draft-ietf-mpls-lmp- 00.txt, Work in Progress, September, 2000. [ND-OIF] G. Bernstein et al., "Methods for Neighbor Discovery in SONET & SDH", OIF2000-159, August, 2000. [OLCP] A. Chiu et al., "Unique Features and Requirements for The Optical Layer Control Plane", draft-chiu-strand-unique-OLCP-00.txt, Work in Progress, July 2000. [LSP-HIER] K. Kompella et al., "LSP Hierarchy with MPLS TE", draft-kompella- lsp-hierarchy-00.txt, Work in Progress, June 2000. [IPO-FRAM] B. Rajagopalan, et al., "IP over Optical Networks: A Framework", work in progress, July, 2000 [DIST-AREA] Y. Hayata, et al., "Carrier needs regarding survivability and maintenance of switched optical networks", draft-hayata-ipo- carrier-needs-00.txt, Work in Progress, June, 2000. [OSPF-OMPLS]K. Kompella, et al., "OSPF Extensions in Support of MPL(ambda)S", draft-kompella-ospf-ompls-extensions-00.txt, Work in Progress, June, 2000. [ISIS-OMPLS]K. Kompella, et al., "IS-IS Extensions in Support of MPL(ambda)S", draft-kompella-isis-ompls-extensions-00.txt, Work in Progress, June, 2000. Y. Xu et al. Page [37] Internet Draft GMPLS Architecture for ASTN Nov. 2000 [MUL-SRUV] J. Meijen, et al., "Multi-layer Survivability", White Paper, http://www.lucent-optical.com/resources/whitepapers/wp008.pdf, Oct, 1999. [CARR-REQ] Y. Xue, M. Lazar, "Carrier Optical Services Framework and Associated Requirement for UNI", OIF2000-155, Work in Progress, Oct, 2000. [ON-REST] B. Doshi et al., "Optical Network Design and Restoration," Bell Labs Technical Journal, Vol. 4, No. 1, Jan-Mar 1999. [DISJ-PATH] J. W. Suurballe, "Disjoint Paths in a Network," Networks, Vol. 4, 1974, pp. 125¡145. Y. Xu et al. Page [38]