Internet Draft IETF Draft Changcheng Huang Multi-Protocol Label Switching Vishal Sharma Expires: January 2001 Srinivas Makam Ken Owens Tellabs Operations, Inc. July 2000 A Path Protection/Restoration Mechanism for MPLS Networks <draft-chang-mpls-path-protection-01.txt> Status of this memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Abstract It is expected that MPLS-based recovery could become a viable option for obtaining faster restoration than layer 3 rerouting. To deliver reliable service, however, multi-protocol label switching (MPLS)[1], [2] requires a set of procedures to provide protection of the traffic carried on the label switched paths (LSPs). This imposes certain requirements on the path recovery process [3], and requires procedures for the configuration of working and protection paths, for the communication of fault information to appropriate switching elements, and for the activation of appropriate switchover actions. This document specifies a mechanism for path protection switching and restoration in MPLS networks. Table of Contents Page 1. Introduction 2 Chang et al [Page 1] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 2.0 Purpose and Motivation 3 3.0 Key Features of the Proposed Mechanism 4 4.0 Core MPLS Path Protection Components 7 4.1 Reverse Notification Tree (RNT) 9 4.2 Protection Domain 11 4.3 Multiple Faults 12 4.4 Timers and Thresholds 13 5.0 Configuration 14 5.1 Establishing a Recovery/Protection Path 14 5.2 Creating an RNT 15 5.3 Engineering a Protection Domain 17 5.4 Configuring Timers 18 6.0 Fault Detection 19 7.0 Fault Notification 21 8.0 Switch Over 22 9.0 Switchback or Restoration 22 10.0 Protocol Specific Extensions 22 11.0 Security Considerations 22 12.0 Acknowledgements 22 13.0 Intellectual Property Considerations 22 14.0 AuthorsÆ Addresses 22 15.0 References 23 1.0 Introduction With the migration of real-time and high-priority traffic to IP networks, and with the need for IP networks to increasingly carry mission-critical business data, network survivability has become critical for future IP networks. Current routing algorithms, despite being robust and survivable, can take a substantial amount of time, to recover from a failure, on the order of several seconds to minutes, which can cause serious disruption of service in the interim. This is unacceptable for many applications that require a highly reliable service, and has motivated network providers to give serious consideration to the issue of network survivability. Path-oriented technologies, such as MPLS, can be used to support advanced survivability requirements and enhance the reliability of IP networks. Different from legacy IP networks, MPLS networks establish label switched paths (LSPs), where packets with the same label follow the same path. This potentially allows MPLS networks to pre-establish protection LSPs for working LSPs, and achieve better protection switching times than those in legacy IP networks. With this objective in mind, the present contribution describes a mechanism to protect paths (or path segments) in MPLS networks. Before discussing the specifics of this contribution, we first outline the major components of a path protection solution, and point out those that are within the scope of this document. A complete solution for path protection requires the following components: (i) A method for selecting the working and protection paths. (ii) A method for signaling the setup of the working and o practical cases of interest: (a) when only the end node of a path is responsible for detecting faults, and (b) when all the nodes along the path are responsible for detecting faults. The main focus of this draft is the specification of an efficient fault notification mechanism that takes LSP merging into account (irrespective of whether they are physically or virtually merged). Switchover and switchback mechanisms also are also within the scope of the draft, but component (vi) is outside the scope of the draft, so the draft does not specify the details of the mechanisms used to detect that a fault has been repaired. 2.0 Motivation and Purpose The framework document [3] lays out the various options for MPLS- based restoration/recovery. However, candidate approaches corresponding to various viable recovery options are still needed. Our work on proposing a path protection mechanism for MPLS networks is motivated by the belief that path protection (in conjunction with local repair) will be needed for truly reliable network operation. The purpose of this contribution is to propose a path protection mechanism that is: (i) fast (compared to Layer 3, with the goal of being comparable to SONET), (ii) scalable, (iii)bandwidth efficient, Chang et al. Expires January 2001 [Page 3] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 (iv)allows for path merging (i.e., is merging compatible), and (v) works at the MPLS layer (that is, only uses knowledge that is commonly available to MPLS routing and signaling modules). 3.0 Key Features of the Proposed Mechanism This contribution describes an MPLS-based path recovery mechanism that can facilitate fast protection switching. The mechanism currently supports 1:1 protection [3]. Bypass tunneling is for further study. First, because tunnel setup itself is not adequately defined yet, and second, because even assuming a tunnel could be setup, in the presence of tunnels (or tunneled segments) the mechanism still requires the ability to setup bi-directional tunnels, which is not defined yet. The mechanism has several timers to enable it to inter-work with protection mechanisms at other layers. Some of the key features of the protection mechanism are: -- Special tree structure to efficiently distribute fault and/or recovery information. Existing published proposals for MPLS recovery have not addressed the issue of fault notification in detail. Specifically, none of these proposals has discussed how to perform fault notification for the label merging case. In this draft, we propose a new fault notification structure called the reverse notification tree (RNT), which makes fault notification efficient and scalable (we provide details of the RNT in subsequent sections). -- Lightweight notification mechanism. Reliable transport mechanisms, such as TCP, are typically state- oriented and therefore difficult to scale. It is also very difficult to support point-to-multipoint communications based on reliable transport mechanisms. In our scheme, therefore, we use a stateless notification mechanism to achieve scalability. The notification is based on the transmission of UDP encapsulated IP packets that are sent periodically until the nodes responsible for switchover learn of the fault. Since no acknowledgements or handshaking between adjacent nodes is needed, the mechanism works only with timers and does not require the maintenance of state. --Minimize delays of a recovery cycle. An objective of the mechanism proposed in this draft is to minimize the duration of the recovery cycle. Thus a stateless transport mechanism together with high priority for control traffic minimizes notification delay. Likewise, a simple label merging approach to handle the traffic arriving on the working and protection paths Chang et al. Expires January 2001 [Page 4] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 eliminates the need for synchronization (or handshaking) between the LSRs at the two ends of a recovery path. . -- Work at the MPLS layer (that is, use information available to the MPLS signaling and routing modules at the nodes) The mechanism is designed to operate using only MPLS constructs and the knowledge available to the MPLS modules at the nodes. Therefore, the mechanism assumes, by default, that the working and protection paths merge at a path merge LSR (PML) within the domain under consideration. However, since the mechanism does not depend on the path selection method, it also works in settings where a PML does not exist, and a path selection algorithm (outside the scope of this I-D) determines that the working and protection paths must terminate at different egress LSRs. Note, however, that for the path selection mechanism to be able to make this determination, it may need knowledge beyond that which is commonly available to the MPLS modules at a node. This is because determining whether a working path can be protected by another path with a different egress LSR requires Layer 3 knowledge to ascertain whether the LSR terminating the recovery path is acceptable. In the remainder of this document, we will focus on the PML case, with the understanding that the non- PML case is also supported. -- Ability to permit recovery mechanisms at different layers to coexist. In the evolving IP network infrastructure, recovery will increasingly be possible at different layers, and interworking of recovery mechanisms at different layers will be needed to ensure smooth network operation in the presence of faults. For example, optical layer or SONET layer recovery mechanisms could be used to recover optical paths or SONET channels. However, these mechanisms may initially be limited to ring topologies and may not provide the right level of granularity at which recovery might be desired, making MPLS-based protection desirable. While MPLS-based protection can be faster than IP layer rerouting it may be more costly to use (it may need to reserve extra bandwidth for example). In certain cases it may become too complicated for MPLS protection to be effective (for example, when there are multiple faults, or faults on both the working and the protection paths), making it necessary for IP layer rerouting to take over. In this draft, we assume that MPLS layer protection is used first, and, if necessary, recovery may be handed to IP layer rerouting following that. In addition to the key features outlined above, some other characteristics of the mechanism are: -- A liveness message to detect faults. Although fault detection is outside the scope of this draft, we will allow the existence of a generic æælivenessÆÆ message that can complement any fault detection mechanism. This liveness message may, Chang et al. Expires January 2001 [Page 5] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 for example, be provided as part of an user/control plane OAM function, or by an existing Hello message (as the RSVP ôHelloö message) with an appropriately set timer value. We do not define specific liveness mechanisms in this draft, deferring instead to work on OAM in MPLS, which is where we expect such a liveness message to be defined. Our assumption is that faults fall into different classes, and that different faults may be detected and corrected by different layers. Some faults (for example, the loss of signal or transmitter faults) may be detected and corrected by lower layer mechanisms (such as SONET), while others (for example, failure of the reverse link) may be detected (but may not be corrected) by lower layers and may be communicated to the MPLS layer. Still other faults (such as node failures or faults on the reverse link) may not be detected by lower layers, and will have to be detected and corrected at the MPLS layer. Therefore, we adopt the liveness message as a complementary fault detection mechanism. We note that in this draft we confine our discussion of protection to a single MPLS domain, and do not consider protection/recovery across multiple MPLS domains or across multiple administrative boundaries. We note, however, that protection mechanisms in different domains may be concatenated, and that (at least initially) these mechanisms may work autonomously, across the (possibly) multiple points of attachment between two adjacent domains. However, coordination of protection mechanisms across multiple domains or across multiple transport technologies is currently out of the scope of this document. 4.0 Core MPLS Path Protection Components This document assumes the terminology given in[1], [2], [3] , and introduces some additional terms. For the convenience of the reader, we repeat here some of the terminology from earlier documents. Working Path The protected path that carries traffic before the occurrence of a fault. The working path is the part of the LSP between the PSL and the PML (if any) or, in the absence of a PML, between the PSL and an egress LSR. A working path is denoted by the sequence of LSRs through which it passes. For example, in Fig. 1, the working path that starts at LSR 1 and terminates at LSR 7 is denoted by (1-2-3-4- 6-7). Recovery Path The path by which traffic is restored after the occurrence of a fault. In other words, the path along which traffic is directed by the recovery mechanism. The recovery path can either be an equivalent recovery path and ensure no reduction in quality of service or be a limited recovery path and thereby not guarantee the same quality of service (or some other criteria of performance) as the working path. A recovery path is also denoted by the sequence of Chang et al. Expires January 2001 [Page 6] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 LSRs through which it passes. Again, in Fig. 1, the recovery path that starts at LSR 1 and terminates at LSR 7 is denoted by (1-5-7). Path Switch LSR (PSL) An LSR that is the transmitter of both the working path traffic and its corresponding recovery path traffic. The PSL is responsible for switching of the traffic between the working path and the recovery path. The PSL is the origin of the recovery traffic, but may or may not be the origin of the working traffic (that is the working path may be transiting the PSL). Path Merge LSR (PML) An LSR that receives both working path traffic and its corresponding recovery path traffic, and either merges their traffic into a single outgoing path, or, it is itself the destination, passes the traffic on to the higher layer protocols. The PML is the destination of the recovery path but may or may not be the destination of the working path. Intermediate LSR An LSR on a working or recovery path that is neither a PSL nor a PML for that path. FIS (Fault Indication Signal) A signal that indicates that a fault along a path has occurred. It is relayed by each intermediate LSR to its upstream or downstream neighbor, until it reaches an LSR that is set up to perform MPLS recovery. FRS (Fault Recovery Signal) A signal that indicates that a fault along a path has been repaired. Like the FIS, it is relayed by each intermediate LSR to its upstream or downstream neighbor, until it reaches an LSR that performs a switchback to the path for which the FIS was received. Liveness Message A generic name for any message exchanged periodically between two adjacent LSRs that serves as a link probing mechanism. It provides an integrity check of the forward and backward directions of the link between the two LSRs as well as a check of neighbor liveness. Path Continuity Test A test that verifies the integrity and continuity of a path or a path segment. The details of such a test are beyond the scope of this draft. (This could be accomplished, for example, by sending a control message along the same links and nodes as those traversed by the data traffic.) 4.1 Reverse Notification Tree Since LSPs are unidirectional entities and recovery requires the notification of faults to the LSR(s) responsible for switchover to Chang et al. Expires January 2001 [Page 7] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 the recovery path, a mechanism must be provided for the fault indication and the fault repair notification to travel from the point of occurrence of the fault back to the PSL(s). The situation is complicated in the following two cases: (i) Physically merged LSPs: With label mergingmultiple working paths may converge to form a multipoint-to-point tree, with the PSLs as the leaves. In this case, therefore, the fault indication and - repair notification should be able to travel along a reverse path of the working path to all the PSLs affected by the fault. For example, in Fig. 1, for a fault along link 34 the affected PSLs are 1 and 9, where as for a fault along link 23, the only affected PSL is 1. (ii) Virtually merged LSPs: When several LSPs originating at different LSRs share a common segment beyond some node, and share a common identifier (such as the SESSION ID in RSVP-TE), we call such LSPs virtually merged. In this case also, savings in notification can be realized by sending a single notification towards the affected PSLs along segments shared by the LSPs emanating from these PSLs, and allowing the notification to branch out at the merge node(s). For example, in Fig. 1, for a failure along link 67 a single notification could be sent for working paths 1-2-3-4-6-7 and 8-9-3-4-6-7 along their common segment 7-6-4-3. The notification would branch out at node 3, which is the node where the LSP from node 1 to node 7 and the LSP from node 8 to node 7 merge. In both the cases above, an appropriate notification path can be provided by the reverse notification tree (RNT which is a point-to- multipoint tree that is an exact mirror image of the converged working paths, along which the FIS and the FRS travel. (see Fig. 1). There are several advantages to using an RNT: -- The RNT can be established in association with the working path(s), simply by making each LSR along a working path remember its upstream neighbor (or the collection of upstream neighbors whose working paths converge at the LSR and exit as one). Thus, no multicast routing is required. We elaborate more on the RNT in Section 3. -- Only one RNT is required for all the working paths that merge (either physically or virtually) to form the multipoint-to-point forward path. The RNT is rooted at an appropriately chosen LSR along the common segment of the merged working LSPs and is terminated at the PSLs. All intermediate LSRs on the converged working paths share the same RNT. Therefore, the RNT enables a reduction in the signaling overhead associated with recovery. Unlike schemes that treat each LSP independently, and require signaling between a PSL and the PML individually for each LSP, the RNT allows for only one (or a small number of) signaling messages on the shared segments of the LSPs. -- The RNT can be implemented either at Layer 3 or at Layer 2. In either case, the delay along the RNT needs to be carefully controlled. This may be ensured by giving the highest priority to Chang et al. Expires January 2001 [Page 8] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 the fault and repair notification packets, which travel along the RNT. PML +----+ L[11,13] +----+ +----+ | 11 |------+ +======| 14 |=========================| 15 | | | | || | | P[14,15] | | +----+ | || +----+ +----+ | || | : +----+ ||P[13,14] | | | 13 |======+ | : PSL | |-------+ | | +----+<-..-: | | : | | | | | L[12,13]| : |L[13,5] | : +----+ | +----+ L[5,15] | | | 12 |------+ | |-----------------------------------+ : | | +===| 5 |<-.-..-..-..-..-..-..-..-..-..-..-..-+ +----+ || | |======================================+ P[1,5] || +----+ P[5,7] || +============+ || || || || || +----+ +----+ L[2,3] L[4,6] +----+ L[6,7] +----+ | 1 |----| 2 |--------+ +-------| 6 |----------| 7 | | |<.-.| |<-..-+ | | +-..-<| |<-..-..-..| | +----+ +----+ N32 : |I23 | : +----+ +----+ PSL | | | | PML || : | | : || | | | | || : | L[3,4] | : || +----+ I34 +----+ || | 3 |--------| 4 | P[10,7]|| | |<-..-..-| | || +----+ N43 +----+ || I93 | | || | : || | |N39 || | : || +----+ +----+ | | +----+ || | 8 |-----| 9 |----+ : | 10 |=============+ | | | |<-..-.+ P[9,10] | | +----+ +----+==========================+----+ PSL Legend: --- = Working path === = Protection path -..- = Reverse Notification Tree ---- = Working path L[x,y] = Working path link between nodes x and y. P[x,y] = Protection path link between nodes x and y. Lxy = Label used by the LSP traversing link L[x,y] from x to y. Nxy = Label used for RNT traffic sent from node x to node y. Ixy = Interface between nodes x and y. Chang et al. Expires January 2001 [Page 9] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 Figure 1: Illustration of MPLS protection configuration 4.2 Protection Domain A protection domain is defined as the set of LSRs over which a working path and its corresponding recovery path are routed. Thus, a protection domain is bounded by the LSRs that provide the switching and (if needed) the merging functions for MPLS protection, namely, the PSL and the PML (if present), respectively. In other words, a protection domain in bounded by the PSL at one end, and by the LSRs that form the end of the working or protection path at the other. In general, if the working and protection paths do not merge within the MPLS domain, the LSRs at the end of the working and protection paths may be egress LSRs. The PSL and the PML (alternatively, the end points of the working and protection paths within the MPLS domain under consideration) are identified during the setting up of an LSP, either via an offline algorithm or by an algorithm that runs at the head-end of an LSP to decide the specific nodes that the LSP must pass through. (Note that segments of the LSP between the PSL and the PML may be loosely routed, as long as the PSL and PML are known). Recovery should ideally be performed between the source and destination (end-to-end), but in some cases segment recovery may be desired (for example, when certain segments are more unreliable than others) or may be the only option (due to the topology of the network, see Fig. 1). For example, in Fig. 1, the working path 8-9-3-4-6-7, can only have protection on the segment 9- 3-4-6-7. Note that when multiple LSPs merge into a single LSP or when multiple LSPs that share a common identifier follow the same path beyond some node, the working paths corresponding to these LSPs also converge. As explained in Section 2.4, an RNT can be used in this case for propagating the failure and repair notification back to the concerned PSL(s). We can therefore have a situation where different protection domains share a common RNT. A protection domain is denoted by specifying the working path and the recovery path. For example, in Fig. 1, the protection domain bounded by LSR 1 and LSR 7, is denoted by (1-2-3-4-6-7, 1-5-7). 4.2.1 Relationship between protection domains with different RNTs When protection domains have different RNTs, two cases may arise, depending on whether or not any portions of the two domains overlap, that is, have nodes or links in common. If the protection domains do not overlap, the protection domains are independent (note that by virtue of the RNTs in the two domains being different, neither the working paths nor the RNTs in the two domains can overlap). In other words, failures in one domain do not interact with failures in the other domain. For example, the protection domain defined by (9-3-4- 6-7, 9-10-7) is completely independent of the domain defined by (11- 13-5-15, 11-13-14-15). As a result, as long as faults occur in independent domains, the network shown in Fig. 1 can tolerate Chang et al. Expires January 2001 [Page 10] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 multiple -faults (for example, simultaneous failures on the working path in each domain). If protection domains with disjoint RNTs overlap, it implies that the protection path of one intersects the working path of the other. Therefore, although failures on the working paths of the two domains do not affect oneanother, failures on the protection path of one may affect the working path of the other and visa versa. For example, the protection domain defined by (1-2-3-4-6-7, 1-5-7) is not independent of the domain defined by (11-13-5-15, 11-13-14-15) since LSR 5 lies on the protection path in the former domain and on the working path in the latter domain. 4.2.2 Relationship between protection domains with the same RNT When protection domains have the same RNT, different failures along the working paths may affect both paths differently. As shown in Fig. 1, for example, working paths 1-2-3-4-5-7 and 9-3-4-6-7 share the same RNT. As a result, for a failure on some segments of the working path, both domains will be affected, resulting in a protection switch in both (for example, the segment 3-4-6-7 in Fig. 1). Likewise, for failures on other segments of the working path, only one domain may be affected (for example, failure on segment 2-3 affects only the first working path 1-2-3-4-6-7, where as failure on the segment 9-3 affects only the second working path 9-3-4-6-7). 4.3 Multiple Faults We note that transferring the working traffic to the recovery path is enough to take care of multiple faults on the working path. However, if multiple faults happen such that there is at least one failure on both the working and recovery paths, MPLS layer recovery may no longer suffice. In this case, the network will either have to allow for Layer 3 rerouting or have the PSL inform the administrator via an alarm, thus enabling the manual reconfiguration of a different working and backup path. (An OAM functionality could greatly simplify such communication.) Note that for a PSL to be able to generate an alarm, it must also have a mechanism for detecting faults on the recovery path, such as a RNT for the recovery path (to allow for the fault notification on the recovery path to be propagated to the PSL). 4.4 Timers and Thresholds For its proper operation, the protection mechanism described in this contribution uses the following timers and thresholds: Timer or Sym Function Threshold bol Chang et al. Expires January 2001 [Page 11] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 Inter FIS t1 Interval at which successive FIS packets are packet timer transmitted by a LSR to its upstream neighbor. Max. FIS t2 Max. time for which FIS packets are duration timer transmitted by an LSR to its upstream peer. Inter FRS t1Æ Interval at which successive FRS packets are packet timer sent by a LSR to its upstream neighbor. Max. FRS t2Æ Max. time for which the FRS packets are sent duration timer by an LSR to its upstream neighbor. Liveness msg. t4 Interval at which successive liveness sender timer messages are sent by an LSR to peer LSRs that have a working path (and RNT) through this LSR. Liveness msg. t4' A timer set to count down the interval at the receiver end of which a liveness message should be timeout timer received. Hold-off Timer t5 Interval between the detection of a failure [3] at an LSR, and the generation of the first FIS message, to allow time for lower layer protection to take effect. Wait-to-Restore T6 Interval between the detection of a Timer [3] recovery/failure at an LSR, and the generation of the first FRS message, to allow time for the stability of restoration. Lost liveness K No. of liveness messages that can be lost message before an LSR will declare a fault and threshold generate the first FIS. Table 1: Illustrating the various timers. 5.0 Configuration Chang et al. Expires January 2001 [Page 12] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 In the following sections, we describe the operation of the path protection mechanism, and explain the various steps involved with reference to Fig. 1. Protection configuration consists of two aspects: establishing the protection path and creating the reverse notification tree. 5.1 Establishing a Recovery/Protection Path The establishment of the recovery path requires the identification of the working path, and hence the protection domain. In most cases, the working path and its corresponding recovery path would be specified during LSP setup, either via a path selection algorithm (running at a centralized location or at an ingress LSR) or via administrative configuration. Observe that the specification of the path, does not, strictly speaking, require the entire path to be explicitly specified. Rather, it requires only that the PSL and PML (or in the absence of a PML, the path egress points out of the MPLS domain) be specified, with the segments between them being - loosely routed, if required. In other words, the path would be established between the two nodes at the boundaries of the protection domain via (possibly loose) explicit (or source) routing using LDP [4], [5] /RSVP [6], [7]signaling (alternatively, via constraint-based routing, or using manual configuration). Ingress Ingress Egress Egress Egress Egress Label of Interface Label of Interface Label of Interface RNT of RNT RNT of RNT RNT of RNT N43 I34 N32 I23 N39 I93 Table 2. An example inverse cross-connect table for LSR 3 using MPLS (Layer 2) RNT Egress Egress Next Hop Egress Ingress Label of Interface IP Interface Label of Working of Address of RNT working Path Working of RNT path Path L34 I34 I9 I93 L93 I2 I23 L23 Table 3. An example inverse cross-connect table for LSR 3 using a hop-by-hop (Layer 3) RNT The roles of the various core protection/recovery components are: Chang et al. Expires January 2001 [Page 13] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 PSL: The PSL initiates the working LSP and the recovery LSP. It is also responsible for storing information about which LSPs (or portions thereof) have protection enabled, and for maintaining a binding between outgoing labels specifying the working path and the protection/recovery path. The latter enables the switchover to the recovery path upon the receipt of a protection switch trigger. The PSL also maintains the timers, t4, t4Æ, t5, t6, and the threshold K. PML: The PML participates in the setting up of a recovery path as a merging LSR. . Therefore, it learns during signaling (or configuration) about which working and protection paths are merged to the same outgoing LSP. The PML also maintains timers t1, t1',t2, t2Æ, t4, t4', t5, t6, and the threshold K. Intermediate LSR: An intermediate LSR participates in the setup of the recovery path, either as a normal LSR or as a merging LSR. It also maintains timers t1, t1', t2, t2Æ, t4, t4Æ, t5, t6, and the threshold K. 5.2 Creating the RNT The RNT is used for propagating the FIS and the FRS, and can be created by a simple extension to the LSP setup process. During the establishment of the working path, the signaling message carries with it the identity (address) of the upstream node that sent it (for example, via the path attribute in RSVP). Each LSR along the path simply remembers the identity of its immediately prior upstream neighbor on each incoming link. Through the neighbor discovery mechanism of the routing protocol, each LSR finds the interface connecting it to the upstream LSRs. (It is assumed in this draft that there is a bi-directional connection between two neighboring LSRs, such as a bi-directional SONET link, a bi-directional lower layer network link (e.g., an ATM VP), or a pair of bi-directional tunnels over an IP subnetwork.) The node then creates an ææinverseÆÆ cross-connect table that for each protected outgoing LSP maintains a list of the incoming LSPs that merge into that outgoing LSP, together with the identity of the upstream node and incoming interface that each incoming LSP comes through. Upon receiving an FIS, an LSR extracts the labels contained in it (which are the labels of the protected LSPs that use the outgoing link that the FIS was received on) and checks whether the current LSR is the PSL for that LSP. If it is it terminates the FIS. Otherwise, it consults its inverse cross-connect table to determine the identity of the upstream nodes that the protected LSPs come from, and creates and transmits an FIS to each of them. Therefore, based on whether the RNT is implemented at Layer 3 or Layer 2, two cases arise: If the RNT is implemented by a point-to-multipoint LSP, then the working path can be bound to the ingress label and interface of the RNT LSP at a LSR. The ingress label and interface can then be used as an index into the "inverse" cross-connect table to find the egress labels and interfaces of the RNT LSP as shown in Table 2. Chang et al. Expires January 2001 [Page 14] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 Upon receiving an FIS, an LSR extracts the labels and checks whether it is the PSL for that LSP. If it is, it terminates the FIS. Otherwise, it consults its inverse cross-connect table to determine the outgoing labels and interfaces, performs a label swap and forwards the FIS to the appropriate upstream node(s). For example, consider Figure 1, and assume that a Layer 2 point-to-multipoint RNT, rooted at LSR 7 and extending to LSRs 1 and 9, is bound to the multipoint-to-point forward paths starting at LSRs 1 and 8 and terminating at LSR 7. Now in case of a fault on link L[4,6], LSR 3 receives an FIS on the RNT in a labeled packet with label N43. It uses this label as an index into its inverse cross-connect table, and learns that there are two previous nodes (namely those reachable via interfaces I23 and I93 respectively) that the FIS needs to be forwarded to. It encapsulates the received FIS into a labeled packets with labels N32 and N39, and dispatches them along interfaces I23 and I93 respectively. If the RNT is implemented by a hop-by-hop Layer 3 mechanism, using, for example, UDP packets (with a specific port number to identify notification message type), then the egress label and interface of the working path can be used as an index into the inverse cross- connect table to obtain the IP addresses of the previous hop(s) and the associated outgoing interface(s), as illustrated in Table 3. On each hop, the FIS carried in the UDP packet carries the label and interface of the working path for that hop. Thus, if the receiving node is not a PSL, the label and interface in the FIS can be extracted and can be used to access the inverse cross-connect table. The label and interface used by the working LSP on the hop(s) to the upstream node(s) are then inserted into FIS packet(s), and the FIS packet(s) transmitted to the appropriate upstream node(s) along the interface specified the inverse cross-connect table. Therefore, in the hop-by-hop mechanism the FIS packets are not forwarded by a node to its previous hops using its default layer 3 forwarding table. Rather, these packets are forwarded via the outgoing interface extracted from the nodeÆs inverse cross-connect table. As in the example above, in case of a fault on link L[4,6], LSR 3 receives an FIS from LSR 4 that contains the outgoing label L34 and the outgoing interface I34 of the LSP affected by the fault. LSR3 uses these to index its inverse cross-connect table (see Table 3), and learns, as before, that there are two previous nodes (those reachable via interfaces I23 and I93, respectively) that must receive an FIS. It then creates two FIS packets as follows. The first, for transmission along interface I23, contains the label L23 used by LSR 2 to transmit data to LSR 3 along the working LSP. The second, for transmission along interface I93, contains the label L93 used by LSR 9 to transmit data to LSR 3 along the working LSP. The roles of the various core protection/recovery components are: PSL: The PSL must be able to correlate the RNT with the working and recovery paths. To this end, it maintains a table with a list of working LSPs protected by an RNT, and the identity of the recovery LSPs that each working path is to be switched to in the event of a Chang et al. Expires January 2001 [Page 15] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 failure on the working path. It need not maintain an inverse cross- connect table (for those LSPs and working paths for which it is the PSL). PML: The PML, in general, has to remember all of its upstream neighbors and associate them with the appropriate working paths and RNTs. If the PML is also the root of the RNT, it has to associate each of its upstream nodes with a working path and RNT, but it need not maintain an inverse cross-connect table (for those LSPs and working paths for which it is a PML). Intermediate LSR: An intermediate LSR has to only remember all of its upstream neighbors and associate them with the appropriate working paths and RNTs, and has to maintain an "inverse" cross- connect table. 5.3 Engineering a Protection Domain For 1:1 protection, the bandwidth (if any) reserved for a protection/recovery path should be the same as the bandwidth reserved for its corresponding working path. This guarantees the same bandwidth for the protected traffic after protection switching. If the LSRs on the protection path support excess mode [3], the bandwidth reserved on the protection path for protecting high priority traffic can be used by other lower priority traffic streams. That is, lower priority traffic that has a segment in common with the recovery path, use the bandwidth of the recovery path, as long as the recovery path is not called into use. When the recovery path is pressed into service, the low priority traffic will be discarded to allow for the actual working traffic to take its place. Also, if delay, jitter or other QoS parameters are to be satisfied, the protection path in 1:1 protection should be chosen such that these requirements are satisfied. Since the volume of signaling traffic (e.g., FIS/FRS messages, or liveness messages) is small, in general bandwidth need not be reserved for the signaling traffic provided that there exist other mechanisms that can ensure that the delay requirements of signaling messages are met (by using, for example, the highest priority for signaling messages). For bypass tunneling protection, multiple working LSPs may share the same protection bandwidth by tunneling protection LSPs over a common path. This requires that the working paths of these LSPs be disjoint, except at the PSL and PML, so that they can be assumed to not all fail at the same time. In this case, the bandwidth reserved on the tunnel will be the maximum of all individual paths. Otherwise, a bypass tunnel could be created to carry all the backup paths, with the bandwidth reserved for the tunnel being the maximum bandwidth required over all failure scenarios on the working LSPs. 3.4 Configuring Timers Chang et al. Expires January 2001 [Page 16] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 The purpose of timers t1/t1' is to control the tradeoff between notification delay of the FIS/FRS and the resources consumed when sending the FIS/FRS. If t1/t1' is large, it may take a relatively long time for the node that initiated the FIS/FRS transmission to send the second the FIS/FRS if the first FIS/FRS message is lost, thereby increasing notification delay. On the other hand, if t1/t1' is small, the repetitive sending of FIS/FRS messages may waste bandwidth and processing power because the first message may already have reached the PSL(s). It is assumed that after t2/t2' it is not necessary to do protection at MPLS layer, either because it is no longer useful or because by that time an upper layer protection mechanism will have been triggered. The timers t4/t4' are used to control the frequency of liveness messages sent between neighboring LSRs, so their purpose is the same as those of timers t1/t1Æ. While frequent exchanges of liveness messages can unnecessarily consume network resources, too few exchanges may delay the discovery of faults. To accommodate delay jitter, t4' may be set at a slightly different value from t4. The timers t5/t6 are used to allow lower layer protection to take effect before initiating MPLS layer recovery mechanisms (for example, an automatic protection switching between fibers that comprise a link between two LSRs). Following the detection of a fault/fault repair S/FRS packet, respectively. This allows for the lower layer protection to take effect and for the LSR to learn this through one of several ways: via an indication from a lower layer, or by the resumption of the reception of a liveness message, or by the lack of LF, LD, PF or PD conditions (see definitions in [3]). The threshold K helps to minimize false alarms due to the occasional loss of a liveness message, which may occur, for example, either due to a temporary impairment in a link or a peer LSR or due to a buffer overflow. 6.0 Fault Detection Each LSR must be able to detect certain types of faults, such as PF, PD, LF, and LD [3] and propagate an FIS message towards the PSL. Here we consider unidirectional link faults, bi-directional (or complete) link faults, and node faults. Essentially, the node upstream of the fault must be able to detect/learn about the fault. This motivates the need for a "liveness" message, which allows a node upstream of the fault to detect the fault either directly or implicitly. Also, the fault detection mechanism must provide the trigger for generating the FIS. Broadly, the types of mechanisms that could be triggers for the FIS are: i) Lower layer mechanisms Chang et al. Expires January 2001 [Page 17] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 ii) MPLS-based detection mechanisms, which may be used to detect link faults, via a liveness message for example. iii) User-plane OAM mechanisms, such as a path continuity test, which may be used to detect other faults, such as mis- connections (arising from incorrect entries in the label forwarding table, for example). 6.1 Unidirectional Link Fault A uni-directional link fault implies that only one direction of a bi-directional link has experienced a fault. 6.1.2 Downlink Fault A fault on a link in the downstream direction will be detected by the node downstream of the faulty link, either via the PF or PD condition being detected at the MPLS layer, or via LF or LD signals being propagated to the MPLS layer by the lower layer or via the absence of liveness messages. The downstream node will then periodically transmit FIS messages to its upstream neighbor (via the uplink), which will propagate these further upstream (using its inverse cross-connect table) until they eventually reach the appropriate PSLs, which will perform the protection switch. Therefore, in Fig. 1, if link L34 has a fault, LSR 4 will detect the fault via one of the means described above, and start transmitting an FIS packet once every t1 time units back to LSR 3 over link L43. The traffic in the queues of LSR 4 will continue to be serviced. LSR 3 in turn will propagate the FIS over the RNT back to LSR 2 and LSR 9. The actual protection switch will be performed by LSRs 9 and 1, t3 time units after the receipt of the first FIS. LSR 4 will stop transmitting FIS messages t2 time units after the transmission of the first FIS message. 6.1.3 Uplink Fault A fault on a link in the upstream direction will be detected by a node upstream of the faulty link, either via a LF or LD being detected at the lower layer and propagated to the MPLS layer (if there was traffic on this reverse link), or via the PD or PF condition being detected at the MPLS layer, or via absence of liveness messages. The upstream node will then periodically send out FIS messages to the node upstream of it, which in turn will propagate these further until eventually the PSL(s) learns of the failure and performs the protection switch. Therefore, in Fig. 1, if link L43 experiences a fault, LSR 3 will detect the fault, and transmit an FIS to nodes 2 and 9. Node 2, in turn, will transmit an FIS to node 1, and nodes 1 and 9 will perform the actual protection switch 6.2 Bi-directional link fault or Node Fault Chang et al. Expires January 2001 [Page 18] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 When both directions of the link have a fault (as in the case of a fiber cut), nodes at both ends of the link will detect the fault either due to the LF or PF signal or due to the absence of liveness messages. Both will transmit FIS messages to their upstream nodes. However, it is only the node upstream of the failed link whose FIS messages will propagate further upstream, eventually reaching the appropriate PSLs, which will perform the protection switch to the recovery path. The case of a node fault is similar, with the node upstream of the failed node detecting the failure (due to loss of liveness messages, for example) and propagating that information via the FIS message. For example in Fig. 1, when both directions of the link between nodes 3 and 4 experience a fault (or when node 4has a fault), LSR 3 will detect this failure via the non-reception of the liveness message, and transmit FIS messages to nodes 2 and 9 as before. When nodes 1 and 9 receive the FIS message they will perform the protection switch after waiting for an interval of t3 time units. The roles of the various core protection components in failure detection are the same. The PSL, PML, and intermediate LSR must all be able to detect PF and PD conditions and/or be able to interpret and respond to the LF and LD indications received from the lower layers. 7.0 Fault Notification The rapid notification of a fault is effected by the propagation of the FIS message along the RNT. Due to the timers built into the FIS/FRS propagation mechanism, the transportation of FIS/FRS messages does not require a reliable mechanism like TCP. Any LSR may generate an FIS, but a PSL is the only LSR that may terminate it. For instance, in Fig. 1 if link L23 fails, LSR 3 will detect it and transmit a FIS to LSR 2 (after waiting for time T2), its upstream neighbor along link L23. The FIS will contain the incoming labels (at node 3) of those LSPs on link L23 that have protection enabled. Upon receiving the FIS message, LSR 2 will consult its inverse-cross connect table and generate an FIS message for LSR 1, which on receiving the first FIS packet will wait for time t3 before performing a protection switch. The node which initiates the FIS will continue to send FIS messages at an interval of t1 until timer t2 expires. After t2 expires it is assumed that either upper layer protection will be triggered or enough number of FIS messages will have been sent to reach the desired reliability in conveying fault information to the PSL(s). The roles of the various core protection switching components are: PSL: The PSL does not generate a FIS message, but must be able to detect FIS packets. Chang et al. Expires January 2001 [Page 19] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 PML: The PML must be able to generate the FIS packets in response to detecting failure, and should transmit them over the RNT. The PML begins FIS transmission after continuously detecting a fault for T2 time units, and does so every t1 time units for a maximum of t2 time units. Intermediate LSR: An intermediate LSR must be able to generate/forward FIS packets, either as a result of continuously detecting a fault for T2 time units or in response to a received FIS packet. It must transmit these to all its affected upstream neighbors as per its inverse cross-connect table. Again, it does so every t1 time units for a maximum of t2 time units. 8.0 Switch Over The switch over is the actual switching of the working traffic from the working path to the recovery path. This is performed by a PSL, t3 time units after the reception of the first FIS packet. For example, in Fig. 1, consider protection domain (1-2-3-4-6-7, 1- 5-7). When link L34 fails, the PSL LSR 1 on learning of the failure will perform a protection switch of the protected traffic from the working path 1-2-3-4-6-7 to the backup path 1-5-7. Notice that LSR 7 acts as a protection merge LSR, merging traffic from the working and backup paths. Since buffered packets from LSR 4 may continue to arrive at LSR 7 even after the protection switch (the dampening timer t43at the PSL tends to mitigate this), a short-term misordering of packets may happen at LSR 7, until the buffers on the working path drain out. The role of the core protection components is as follows: PSL: Performs the protection switch upon receipt of the FIS message, but after waiting for time t3 following the first FIS message. PML: The PML automatically merges protection traffic with working traffic. For a short period of time this may cause misordering of packets, since packets buffered at LSRs downstream of the fault may continue to arrive at the PML along the working path. Intermediate LSR: The intermediate LSR has no special action. 9.0 Switch Back Switch back or restoration is the transfer of working traffic from the recovery path to the working path, once the working path is repaired. This may be because the recovery path may be a limited recovery path [3], or because the working path is deemed to be preferred [3] in some respect. Restoration may be automatic or it may be performed by manual intervention (or not performed at all). In the revertive mode, restoration is performed upon the receipt of the FRS message. A path continuity test may be performed to ensure the integrity of the entire path before switching. I n the non- revertive mode it may be performed by operator intervention. Chang et al. Expires January 2001 [Page 20] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 The role of the core protection components is similar here to what it is for protection switching. The PML does not need to do anything, unless it was the node that detected the failure, in which case it transmits a FRS upstream T8 time units after continuously detecting recover signal from lower layer or after detecting liveness messages from its peers. The intermediate LSR generates the FRS message if it was the node that detected the recovery or generates a FRS to relay the restoration status received from a downstream node. The PSL performs the restoration switch t3Æ seconds after receiving the first FIS message. 10.0 Protocol Specific Extensions The signaling protocol specific extensions needed to implement the mechanism outlined in this draft are discussed in separate documents [8] 11.0 Security Considerations The MPLS protection that is specified herein does not raise any security issues that are not already present in the MPLS architecture. 12.0 Intellectual Property Considerations In accordance with the intellectual property rights procedures of the IETF standards process, to the extent that Tellabs has patents, pending applications and/or other intellectual property rights that are essential to implementation of any subject matter submitted by Tellabs that is included in a standard, Tellabs is prepared to grant, on the basis of reciprocity (grantback), a license on such subject matter under terms and conditions that are reasonable and non-discriminatory. 13.0 Acknowledgements We would like to thank our colleague Ben Mack-Crane, and members of the MPLS WG list, in particular Dave Allan, Bora Aykol, Neil Harrisson, and J. Noel Chiappa, for suggestions, feedback, and corrections to the first version of this draft. 14.0 AuthorsÆ Addresses Changcheng Huang Vishal Sharma Tellabs Operations, Inc. Tellabs Research Center 4951 Indiana Avenue One Kendall Square Lisle, IL 60532 Bldg. 100, Ste. 121 Phone: 630-512-7754 Cambridge, MA 02139-1562 Changcheng.Huang@tellabs.com Phone: 617-577-8760 Vishal.Sharma@tellabs.com Chang et al. Expires January 2001 [Page 21] IETF Draft A Path Protection Mechanism for MPLS Networks July 2000 Srinivas Makam Ken Owens Tellabs Operations, Inc. Tellabs Operations, Inc. 4951 Indiana Avenue 1106 Fourth Street Lisle, IL 60532 St. Louis, MO 63126 Phone: 630-512-7217 Phone: 314-918-1579 Srinivas.Makam@tellabs.com Ken.Owens@tellabs.com 15.0 References [1] Rosen, E., Viswanathan, A., and Callon, R., "Multiprotocol Label Switching Architecture", Work in Progress, Internet Draft, August 1999. [2] Callon, R., Doolan, P., Feldman, N., Fredette, A., Swallow, G., Viswanathan, A., "A Framework for Multiprotocol Label Switching", Work in Progress, Internet Draft , September 1999. [3] Makam, V., Sharma, V., Huang, C., Owens, K., Mack-Crane, B., et al, "A Framework for MPLS-based Recovery," Work in Progress, Internet Draft <draft-makam-mpls-recovery-frmwrk-00.txt>, February 2000. [4] Andersson, L., Doolan, P., Feldman, N., Fredette, A., Thomas, B., "LDP Specification", Work in Progress, Internet Draft , September 1999. [5] Jamoussi, B. "Constraint-Based LSP Setup using LDP", Work in Progress, Internet Draft <draft-ietf-mpls-cr-ldp-03.txt>, September 1999. [6] Braden, R., Zhang, L., Berson, S., Herzog, S., "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification", RFC 2205, September 1997. [7] Awduche, D. et al "Extensions to RSVP for LSP Tunnels", Work in Progress, Internet Draft <draft-ietf-mpls-rsvp-lsp-tunnel-04.txt, September 1999. [8] Huang, C., Sharma, V., Makam. V, and Owens, K., "Extensions to RSVP-TE for MPLS Path Protection," Internet Draft, , July, 2000. Chang et al. Expires January 2001 [Page 22]