Internet Draft Internet Draft James Luciani Expiration Date: Sept., 10, 2000 Tollbridge Technologies Bala Rajagopalan Tellium, Inc. Daniel Awduche UUNET (MCI Worldcom) Brad Cain Bilel Jamoussi Nortel Networks IP over Optical Networks - A Framework draft-ip-optical-framework-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 except that the right to produce derivative works is not granted. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract The Internet transport infrastructure is moving towards a model of high-speed routers interconnected by optical core networks. A consensus is emerging in the industry on utilizing IP-centric protocols for the optical control plane [9, 10], as well as for dynamic bandwidth provisioning for IP services. Also, there has recently been significant activity in defining models for IP transport over optical networks, specifically, the routing and signaling aspects [2,6,7]. This draft attempts to define a framework for IP over Optical networks, considering both the IP control plane for optical networks as well as IP transport over optical networks (together referred to as "IP over optical networks"). Within this framework, we develop a common set of terms and concepts which allows to discuss these IP over optical technologies in a consistent fashion. Additionally, this draft surveys some architectural frameworks that might be useful and appropriate for the deployment of IP over hybrid optical networks that contain IP routers, WDM multiplexers, and optical cross-connects (OXCs). This document is complementary to the framework of Multiprotocol Lambda Switching proposed in [2]. Internet-Draft draft-ip-optical-framework-00.txt 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119. 3. Introduction Optical network technologies have evolved rapidly in terms of functions, capabilities, and densities. The increasing importance of optical transport networks is evidenced by the copious amount of attention focused on IP over optical networks and related photonic and electronic interworking issues by all the major network service providers, telecommunications equipment vendors, and standards organizations all over the world. One important factor driving the trend towards multi-wavelength optical networking is the desire to capitalize upon the opportunities and challenges presented by the exponential growth of the Internet and the resulting intense demand for broadband services. It has been realized that optical networks must be survivable, flexible, and controllable. There is, therefore, an ongoing trend to introduce intelligence in the control plane of optical transport systems to make them more versatile [2]. An essential attribute of intelligent optical transport networks is the capability to instantiate and route optical channels in real-time or near real-time, and to provide capabilities that enhance network survivability. Furthermore, there is a need for multi-vendor optical network interoperability, when an optical transport network may consist of interconnected vendor-specific optical sub-networks [9]. Many advocates of intelligent optical transport systems contend that `optical routing' will eventually become cheaper than `electrical routing,' and that all optical networking will eliminate the electronic bottlenecks imposed by processing in the electrical domain. These are clearly very important strategic factors motivating the current intense activity to evolve and commercialize optical transport systems. The real benefit of intelligent optical networks, however, will accrue from the potential to construct more scalable networks, and to expedite the provisioning process while enabling a plethora of protection and restoration capabilities in operational contexts. The optical network must also be versatile because some service providers and network contexts may provide generic optical layer services that may not be specific to any digital clients above the optical transport network. In the context of an automatically switched optical network, it would be necessary to have a control layer that can handle such generic optical services. Internet-Draft draft-ip-optical-framework-00.txt This memo is an attempt to bind the problem space at hand to a framework and to provide a common language with which to speak about the IP-based control and IP transport over optical networks. The following sections describe a set of candidate models for this, along with a discussion of their relative advantages and disadvantages. It is certainly not the intent of this draft to promote any particular model over the others. However, prior lessons learned from layering IP over other media will tend to highlight particular aspects of the models which may make one approach more appropriate than another in certain circumstances. In the following sections, we consider the IP control plane issues in optical networks and describe three generic models for IP transport over optical networks. These models are: (1) the "Overlay" model, (2) the "Integrated" model, and (3) the "Peer" model. The reader can see that the terminology used to describe these models have some similarity to the terminology previously used to describe IP over ATM models. These transport models differ from each other in a number of ways. One important manner in which the models differ is in the way that routing and signaling protocols are run over the IP and optical subnetwork layers. Some of these considerations were alluded to in [2] and include the following aspects: o whether there is a single monolithic instance of routing and signaling protocols that span the IP and optical domains; o whether there are separate instances of routing and signaling protocols for the IP domain and the optical transport network. o If there are separate instances of routing protocols for each domain then the following additional consideration apply: whether routing information is leaked from one routing protocol instance into the other; o In the case of a single monolithic instance of the routing protocol for both the IP and optical domains, whether both domains actively participate in the exchange of routing information or whether one layer is passive with respect to the mutual exchange of routing information. Another manner in which the models differ is with respect to the way in which label switching protocols might run over them or in conjunction with them. Clearly, a single monolithic label switching protocol would be very interesting architecturally and administratively because of its potential simplicity, conceptual integrity, and ease of management; especially from the perspective of network operations control. But the semantics of "label switching" and the establishment and maintenance of "LSPs" in an optical network may be different in optical networks as compared to traditional MPLS networks [9]. There are also some challenging protocol issues to be addressed, however. For example, how would IP QoS requirements be mapped onto the QoS capabilities of the optical transport network (that is, in the contexts where such QoS mappings are actually relevant). Internet-Draft draft-ip-optical-framework-00.txt As for IP-based control plane for optical networks, the most challenging issues are related to meeting the strict reliability and time constraints in the optical domain. While some mappings are straightforward in adopting IP-based protocols for optical network control, others are not [9]. Furthermore, proprietary mechanisms will be used in optical sub-networks for certain functions like restoration. The interworking of these sub-networks when putting together a large core network is an issue. 3.1 Terminology This subsection introduces some terminology for describing common concepts in IP over optical networks. These terms serve as a blueprint which allow common issues in the IP over optical networks to be described consistently and coherently. WDM --- Wavelength Division Multiplexing (WDM) is a technology that allows multiple optical signals operating at different wavelengths to be multiplexed onto a single fiber so that they can be transported in parallel through the fiber. In general, each optical wavelength may carry digital client payloads at a different data rate (e.g., OC-3c, OC-12c, OC- 48c, OC-192c, etc.) and in a different format (SONET, Ethernet, ATM, etc.) For example, there are many commercial WDM networks in existence today that support a mix of SONET signals operating at OC-48c (approximately 2.5 Gbps) and OC-192 (approximately 10 Gbps) over a single optical fiber. An optical system with WDM capability can achieve parallel transmission of multiple wavelengths gracefully while maintaining high system performance and reliability. In the near future, commercial WDM systems are expected to concurrently carry more than 160 wavelengths at data rates of OC-192c and above, for a total of 1.6 Tbps and more. The term WDM will be used throughout this document to refer to both WDM and DWDM (Dense WDM). The exception is when it is necessary to differentiate between WDM and DWDM, in which case, the distinction will be made explicit. In general, it is worth noting that WDM links are affected by the following factors, which may introduce impairments into the optical signal path: 1. The number of wavelengths on a single fiber. 2. The serial bit rate per wavelength. 3. The type of fiber. 4. The amplification mechanism. 5. The number of nodes through which the signal passes before it reaches the egress node or before regeneration. All these factors and others not mentioned here constitute domain specific features of optical transport networks. As noted in [2], these features should be taken into account in developing standards based solutions for IP over WDM systems. Internet-Draft draft-ip-optical-framework-00.txt Non-Broadcast Multi-Access Network (NBMA) ----------------------------------------- A subnetwork can be non-broadcast either because it technically does not support broadcasting or because broadcasting is not feasible for one reason or another (e.g., an extended Ethernet which is too large) [7]. IP Routed hop ------------- Within the context of this memo, an IP routed hop is any device that is IP aware and is able to forward IP packets from an input port to an output after possibly performing some operations on the packets. An IP routed hop device will participate in IP routing protocols such as OSPF, or IS-IS, or derivatives thereof. An IP routed hop device is thus distinguished from a pure switch which does not participate in an IP routing protocol. Switch routers, routing switches, and label switched routers are all potentially capable of acting as both pure switches and IP routed hop elements. Trust domain ------------ A trust domain is a network under a single technical administration in which most nodes in the network are considered to be secure or trusted in some fashion. An example of a trust domain is a campus or corporate network in which all routing protocol packets are considered to be authentic, without the need for additional security schemes to prevent unauthorized access to the network infrastructure. Generally, the "single" administrative control rule may be relaxed in practice if a set of administrative entities agree to trust one another to form an enlarged heterogeneous trust domain. However, to simplify the discussions in this memo, it will be assumed, without loss of generality, that the term trust domain applies to a single administrative entity. Optical Channel Trail --------------------- An optical channel trail is a point-to-point optical connection between two access points in an optical transport network. Wavelength continuity property ------------------------------ An optical channel trail is said to satisfy the wavelength continuity property if it consists of just one wavelength end-to-end. Wavelength continuity is required in optical networks with no wavelength conversion feature. Internet-Draft draft-ip-optical-framework-00.txt Wavelength path vs. lightpath ----------------------------- In an optical network without the wavelength conversion feature, we define a wavelength path (WLP) as an optical path between an ingress and egress node of the WDM network that uses exactly the same wavelength from ingress node to egress node. That is, a WLP uses a specific wavelength between each node from source to destination. Thus, a wavelength path satisfies the wavelength continuity property. On the other hand, in an optical network with wavelength conversion, we define a lightpath (LP) as an optical path from an ingress node to an ingress that may or may not consist of the same WL at each hop. That is, an LP may use a specific wavelength Lambda(k) between some node Ni and Ni+1 but may use another wavelength Lambda(l) between Ni+1 and Ni+2 such that Lambda(k) != Lambda(l) and Ni != Ni+1!= Ni+2. Hence, as used in this document, a lightpath is a generalization of the notion of wavelength path. Shortcut/cut-through path ------------------------- A shortcut (used synonymously throughout this document with the term cut-through) is defined as an LP or optical channel trail which causes packets between its endpoints to bypass a number of normally routed hops within the trust domain. To be more precise, a shortcut makes its end-points appear to be adjacent to each other with respect to routed hops, even though the short-cut LP itself may traverse a number of intermediate nodes. Flow ---- For the purpose of this document, the term flow will be used to represent the smallest separable stream of data, as seen by an endpoint (source or destination node). It is to be noted that the term flow is heavily overloaded in the networking literature. Within the context of this document, it may be convenient to consider a wavelength as a flow under certain circumstances. However, if there is a method to partition the bandwidth of the wavelength, then each partition may be considered a flow, for example by quantizing time into some nicely manageable intervals, it may be feasible to consider each quanta of time within a given wavelength as a flow. Traffic Trunk ------------- A abstraction of traffic flow that follows the same path between two access points which allows some characteristics and attributes of the traffic to be parameterized. Internet-Draft draft-ip-optical-framework-00.txt Opaque vs. transparent optical networks --------------------------------------- A transparent optical network is an optical network in which each transit node along a path passes optical transmission without transducing the optical signal into an electrical signal and back to an optical signal. More generally, all non-terminus nodes in a transparent optical network will pass optical signals without performing retiming and reshaping and thus such nodes are unaware of many characteristics of the carried signals. One could, for example, carry analog signals side by side with digital signals (potentially of varying bit rate) on different wavelengths over such a network. Note that repowering of signals at transit nodes is potentially permitted in transparent optical networks. This is a result of the availability of all optical amplification devices such as Erbium Doped Fiber Amplifiers (EDFAs). In opaque optical networks, by comparison, transit nodes will perform manipulation of the optical signals which they are carrying. An example of such manipulation would be 3R regeneration (reshaping, retiming, regeneration/amplification). Opaque optical networks are, by far, the most common type of network deployed today. However, there is intense interest in the development and researching of practical all optical networks. 4. The Network Model The network model considered in this draft consists of of IP routers attached to an optical core network, and connected to their peers over dynamically established switched optical paths. The optical core itself is assumed to be incapable of processing individual IP packets. The interaction between the IP routers and the optical core is over a well-defined signaling and routing interface. The optical network is assumed to consist of multiple Optical Cross- Connects (OXCs) interconnected by optical links in a general topology (refered to as an "optical mesh network"). This network may be multi- vendor, where individual vendor OXCs constitute "clouds" or "sub- networks". Each sub-network itself is assumed to be mesh-connected. Each OXC is assumed to be capable of switching a data stream from a given input port to a given output port. This switching function is controlled by appropriately configuring a cross-connect table. Conceptually, the cross-connect table consists of entries of the form , indicating that data stream entering input port i will be switched to output port j. An "optical path" from an ingress port in an OXC to an egress port in a remote OXC is established by setting up suitable cross-connects in the ingress, the egress and a set of intermediate OXCs such that a continuous physical path exists from the ingress to the egress port. Optical paths are assumed to be bi-directional, i.e., the return path from the egress port to the ingress port follows the same path as the forward path. The switching within the OXC may be accomplished in the electrical domain, or the OXC may be all-optical. Internet-Draft draft-ip-optical-framework-00.txt Furthermore, multiple data streams output from an OXC may be multiplexed onto an optical link using WDM technology. The WDM functionality may exist outside of the OXC, and be transparent to the OXC. Or, this function may be built into the OXC. In the latter case, the cross-connect table (conceptually) consists of pairs of the form, <{input port i, Lambda(j)}, {output port k, Lambda(l)}>. This indicates that the data stream received on wavelength Lambda(j) over input port i is switched to output port k on Lambda(l). Automated establishment of optical paths involves setting up the cross-connect table entries in the appropriate OXCs in a coordinated manner such that the desired physical path is realized. In general, it can be expected that topologically adjacent OXCs in an optical mesh network will be connected via multiple, parallel (bi- directional) optical links. It is assumed that one or more control channels exist between neighboring OXCs for signaling purposes. Under this network model, a switched optical path must be established between a pair of IP routers before they can communicate. This path might traverse multiple optical sub-networks and be subject to different provisioning and restoration procedures in each sub-network. The IP-based control plane issue is that of designing standard signaling and routing protocols for coherent end-to-end provisioning and restoration of optical paths across multiple optical sub-networks. Similarly, IP transport over such an optical network involves determining IP reachability and seamless establishment of paths from one IP endpoint to another over an optical core. 5. IP-based Control Plane Issues The control plane issues can be classified into signaling for provisioning, signaling for restoration and routing issues. The difference between the two signaling procedures is that restoration signaling is bound by strict time constraints, whereas the time constraint for provisioning is more relaxed. Some of the signaling issues are described in [9, 10]. To summarize, the following are some of the issues that must be addressed when considering the development of IP-based signaling for optical networks: o What is the signaling interface between the IP routers and the optical network? o How to automatically discover local topology information between adjacent OXCs so that end-to-end path establishment and restoration are automated? o How to establish bi-directional paths without race conditions? o How to ensure fault-tolerance at the data plane when the control plane may be affected by failures? o Whether soft-state-based signaling protocols are suitable in the optical network environment, o How to ensure signaling for restoration can meet the strict time bounds? Internet-Draft draft-ip-optical-framework-00.txt o Whether separate signaling protocols must be developed for restoration signaling, o How to allow proprietary signaling protocols within optical sub-networks for local restoration (and perhaps for provisioning) while developing a uniform standard end-to-end protocol? o How does MPLS-based restoration at the IP level work with optical level fast restoration? o How to accommodate both OXCs with wavelength conversion capability and those without in the optical network? o Can out-of-band and in-band signaling procedures co-exist? The routing issues deal with similar problems of interworking: o How to ensure that end-to-end reachability information is propagated across the optical network? o How to accommodate proprietary optimizations within optical sub-networks for provisioning and restoration of paths? o Whether dynamic and pre-computed routing information can be used together, and if so, what is the interaction between them? o How to deal with dense adjacencies between OXCs? o What QoS and service-related parameters need to be defined? o How to ensure fault-tolerant operation at the protocol level when the hardware supports fault tolerance? o How to address the scalability issue? We do not get into the details of these issues, but defer them for discussion in future drafts. We just note that a clear set of requirements for IP-based control plane protocols (signaling and routing) need to be defined before addressing the specific issues. There is also some overlap between the issues mentioned above and the issues in IP transport over Optical networks discussed next. 6. IP transport over Optical Networks 6.1 The overlay model Under the overlay model, IP is more or less independent of the optical subnetwork from a routing perspective. The overlay model is a client-server model, in which the IP domain is a client of the optical domain. In this scenario, the optical network provides point-to-point optical links for the transport of IP packets through the optical domain. This model is conceptually similar to the classical IP over ATM or MPOA models, but applied to a optical subnetwork directly. Internet-Draft draft-ip-optical-framework-00.txt Under the overlay model, the IP/MPLS routing, topology distribution, and signaling protocols are independent of the routing, topology distribution, and signaling protocols at the optical layer. MPLS may nonetheless provide a mechanism to cut through or bypass routed hops. In the overlay model, lambda routing, topology distribution, and signaling protocols would have to be defined for the optical domain. In certain circumstances, it may also be feasible to statically configure the optical channels that provide connectivity in the overlay model through network management. Static configuration is, however, unlikely to scale in very large networks. A protocol like GSMP might also be employed here, in conjunction other control plane protocols to establish lightpaths and optical channel trails in the overlay model. Note that in this model, a cut-through lambda may not necessarily affect the L3 routing information; i.e., a shortcut may or may not add an entry to the L3 routing table. 6.2 The integrated/augmented model In the integrated model, the IP/MPLS layers act as peers of the optical transport network, such that a single routing protocol instance runs over both the IP/MPLS and optical domains. Presumably a common IGP such as OSPF or IS-IS, with appropriate extensions, will be used to distribute topology information (see e.g.,[5] and [6]). In the case of OSPF, opaque LSAs will be used to advertise topology state information [5]. In the case of IS-IS, extended TLVs will have to be defined to propagate topology state information [6]. One tacit assumption here is that a common addressing scheme will also be used for the optical and IP networks. A common address space can be trivially realized by using IP addresses in both IP and optical domains. Thus, the optical network elements become IP addressable entities as noted in [2]. In the augmented model, there are actually separate routing instances in the IP and optical domains, but information from one routing instance is leaked into the other routing instance. For example, IP addresses could be assigned to optical network elements and carried within the optical routing protocols to allow reachability information to be shared with the IP domain, and to support some degree of automated resource discovery. 6.3 The peer model A peer model is somewhat similar to an integrated model in that IP reachability information might be passed around within the optical routing protocol but the actual flow would be terminated at the edge of the optical network and will only be re-established upon reaching a non-peer capable node either at the edge of the optical domain or at the edge of a domain which implements both peer and overlay models. The overlay, integrated, and peer models can also be described using the terminology of "termination capable" (TC) and "terminology incapable" (TI) interfaces, in conjunction with the generalized notion of LSP nesting described in [6]. Internet-Draft draft-ip-optical-framework-00.txt 7. Some starting assumptions WDM and TDM in the same network element --------------------------------------- A practical assumption would be that if SONET (or some other TDM mechanism that is capable partitioning the bandwidth of a wavelength) is used, then TDM leveraged as an additional method to differentiate between "flows." In such cases, wavelengths and time intervals (sub- channels) within a wavelengths become analogous to labels (as noted in [2]) which can be used to make switching decisions. This would be somewhat akin to using VPI (e.g., wavelength) and VCI (e.g., TDM sub- channel) in ATM networks. More generally, this will be akin to label stacking and to LSP nesting within the context of Multiprotocol Lambda Switching [2]. Wavelength converters --------------------- Some form of wavelength conversion may exist at some switching elements, however, this may not be case in some pure optical switching elements. A switching element is essentially anything more sophisticated than a simple repeater, that is capable of switching and converting a wavelength Lambda(k) from an input port to a wavelength Lambda(l) on an output port. In this display, it is not necessarily the case that Lambda(k) = Lambda(l), nor is it necessarily the case that the data carried on Lambda(k) is switched through the device without being examined or modified. It is not necessary to have a wavelength converter at every switching element. A number of studies have attempted to address the issue of the value of wavelength conversion in an optical network. Such studies typically use the blocking probability (the probability that a lightpath cannot be established because the requisite wavelengths are not available) as a metric to adjudicate the effectiveness of wavelength conversion. The IP over optical architecture must take into account hybrid networks with some OXCs capable of wavelength conversion and others incapable of this. Service provider peering points ------------------------------- There are proposed inter-network interconnect models which allow certain types of peering relationships to occur at the optical layer. This is consistent with the need to support optical layer services independent of higher layers payloads. In the context of IP over optical networks, peering relationships between different trust domains will eventually have to occur at the IP layer, on IP routing elements, even though non-IP paths may exist between the peering routers. Optical Network as an NBMA Network ---------------------------------- A mesh based optical network is very much like an NBMA network in terms of its subnetwork characteristics. Internet-Draft draft-ip-optical-framework-00.txt Rate of LP setups ----------------- Dynamic establishment of optical channel trails and lightpaths is quite desirable in IP over optical networks, especially when such instantiations are driven by a stable traffic engineering control system, or in response to authenticated and authorized requests from client. However, there are many proposals suggesting the use of dynamic, data-driven shortcut-LP setups in IP over optical networks. The arguments put forth in such proposals are quite reminiscent of similar discussions regarding ATM deployment in the core of IP networks. Deployment of highly dynamic data driven shortcuts within core networks has not been widely adopted by carriers and ISPs for a number of reasons: possible CPU overhead in core network elements, complexity of proposed solutions, stability concerns, and lack of true economic drivers for this type of service. This draft assumes that this paradigm will not change and that highly dynamic, data-driven shortcut LP setups are for future investigation. Instead, the optical channel trails and LPs that are expected to be widely used at the initial phases in the evolution of IP over optical networks will include the following: o Dynamic connections for control plane traffic and default path routed data traffic, o Establishment and re-arrangement of arbitrary virtual topologies over rings and other physical layer topologies. o Use of stable traffic engineering control systems to engineer LP connections to enhance network performance, either for explicit demand based QoS reasons or for load balancing). Other issues surrounding dynamic connection setup within the core center around resource usage at the edge of the optical domain. One potential issue pertains to the number of flows that can be processed by an ingress or egress network element either because of aggregate bandwidth limitations or because of a limitation on the number of flows (e.g., LPs) that can be processed concurrently. Another possible short term reason for dynamic shortcut LP setup would be to quickly pre-provisioned paths based on some criteria (TOD, CEO wants a high BW reliable connection, etc.). In this scenario, a set of paths is pre-provisioned, but not actually instantiated until the customer initiates an authenticated and authorized setup requests which is consistent with existing agreements between the provider and the customer. In a sense, the provider may have already agreed to supply this service, but will only instantiate it by setting up a lightpath when the customer submits an explicit request. Internet-Draft draft-ip-optical-framework-00.txt Distributed vs. centralized cut through path calculation -------------------------------------------------------- One model of shortcut path calculation is to have a centralized mechanism (perhaps simply a suitably equipped high end PC) which has complete knowledge of the physical topology, the available wavelengths, and where applicable relevant time domain information. In this centralized model, the centralized resource acts a server or bandwidth broker. A corresponding client will reside on each network element that can source or sink an LP. The source client would query the server in order to set up an LP from the source to the destination. The server would then check to see if such an LP can be established based on prevailing conditions. Furthermore, depending on the specifics of the model, the server may either setup the LP on behalf of the client or provide the necessary information to the client or to some other entity to allow the LP to instantiated (e.g., the information may consist of a set of WLPs to be used at each hop within the trust domain). Another option may be for the server to indicate that it is not possible to setup an LP to the egress point but that it might be possible to bypass a certain number of nodes and terminate the shortcut at a routed hop which is closer to the destination than the source node is. The second possibility is a distributed model in which all nodes maintain a synchronized topology database, and advertise topology state information to maintain and refresh the database. A constraint- based routing entity on each node may then use the information in the topology database and other relevant details to compute appropriate paths through the optical domain. Once a path is computed, a signaling protocol such as RSVP can be used to instantiate the LP. 8. What services need to be there? Who needs QoS when bandwidth is free? ------------------------------------- Even though bandwidth may become abundant and relatively cheap in the future, it does prelude the need for traffic engineering capabilities to ensure that the bandwidth is actually used effectively to deliver high quality services. Automated traffic engineering capabilities can also drastically reduce the amount of manual overhead required for network operations control. LP VPNs ------- There has been some talk of the use of LPs as methods to create VPN links across carriers. In the case of a ring, one could in fact imagine a given WL not only serving to isolate one routing domain from another but also to act as a multidrop for the pt-mpt case (assuming that this is desirable). Internet-Draft draft-ip-optical-framework-00.txt Traffic engineered paths for fault tolerance and load balancing --------------------------------------------------------------- Constraint based LP routing and non-equal cost multipath: In the mesh optically switched configurations, multiple feasible paths would exist between ingress and egress nodes at the boundaries of the optical cloud. In this case, for the purposes of traffic engineering, it might be valuable to have capability to setup LPs which satisfy certain requirements subject to certain constraints. There is nothing new in this idea, except that the connection oriented infrastructure is built from optical rather than traditional L2 devices. Nonetheless, this potential should be noted and should be considered when defining appropriate reference models. What about Multi-Link Trunking ------------------------------ Multi-Link Trunking, also known as bundling (see [2]) is a concept that allows multiple channels to be aggregated to form a single "trunk." Thus for N links, the effective bandwidth for a multilink trunk may be as large as N*BW where BW is the bandwidth of a single channel. In the IP world there is concern for ordered packet delivery, so simply forwarding each packet to a different channel in a round robin fashion (for example) is not a reasonable solution. What needs to be done is some pre-classification prior to transmission of each packet to ensure that packets belonging to the same "micro" flow are delivered in order by sending them through the same channel consistently. For example, a simple but popular way to accomplish this is to apply a deterministic hash function to certain fields in the IP packet headers such that the output of the hash function can eventually be associated with certain virtual interfaces which are correspondingly mapped to specific channels within the trunk on which the packets will be sent. When a channel of a trunk fails (e.g., a laser fails) and that failure is detected then one of two things could happen: 1) the packets of the micro flows associated with the failed channel are dropped while the rest of the flow goes on unhindered, 2) the mapping function is dynamically modified so that that the impacted micro flows are redirected down another link. Thus multi-link trunking provides some higher layer mechanism to enhance network survivability through fault tolerance. What about an IMUX ------------------ For the purposes of this draft the definition of an IMUX is somewhat similar in concept to multi-link trunking in that for a given flow, data within that flow will travel in parallel over multiple "channels." In the IMUX case, however, a given flow is cut up into segments/chunks of equal sized data (bits, bytes, words, or whatever) without regard to any higher layer notion of flows and addressing, and the segments are distributed across the links of a trunk bundle. Thus there is an implied re-assembly mechanism on the other side. Again, this segmentation and re-assembly (SAR) function is without regard to packet boundaries from the perspective of the IMUX (of course the higher layers will need to make sense of the re-assembled flow at the egress). Furthermore, within an IMUX, links may remain unused and may be used in case of failure as backup to the failed links. Internet-Draft draft-ip-optical-framework-00.txt This function is one possible way to build aggregate bandwidth trunks from slower channels. The current practical reality is that the above mentioned type of IMUX can only be done over relatively short distances at high data rates, although this situation is likely to improve in the future as technology evolves. Wavelength and TDM signaling ---------------------------- It will be assumed that there exists some default communication mechanism between routers prior to using any of the routing and signaling mechanisms. If a ring topology exists then this default mechanism would most likely be pre-configured (e.g., a default communication WL between routers or perhaps routers and ADMs). If a switched infrastructure exists then it is likely that some dynamic routing protocol would exist or at the very least some NM interface would need to exist in order to statically connect each router with its appropriate peer within the trust domain. 9. Some potential protocols for signaling 9.1 How might GSMP play here? What is GSMP? ------------- The GSMP allows a "controller" to control a "switch". The system of controller plus switch typically functions as a network node in a TCP/IP, ATM or Frame Relay network. Broadly speaking, the controller runs "control protocols" that provide the required network layer services such as IP routing protocols, LDP, PNNI signaling and routing, etc. The term "control plane" refers to the set of protocols and mechanisms used to support a node of one network type, e.g. an ATM control plane. A controller may support multiple control planes. If the physical network supports multiple control planes then the term "logical network" is used to refer to a network as seen by one control plane. The switch is a general label switch with appropriate label switched ports. The controller is responsible for installing and deleting connection state in the switch. How might GSMP fit in? --------------------------- In a general mesh network where the OXCs do not participate in topology distribution protocols and where a router has full knowledge of LP, L2, and L3 topology GSMP might be used to communicate a binding between, for example, Lambda(i) coming in on port j of an OXC and going out on Lambda(k) going out port m. In this scenario, an ingress (or egress) router would signal each OXC between ingress and egress nodes along the LP. It would signal using GSMP to ask the OXC to set up its cross connect management table appropriately. Internet-Draft draft-ip-optical-framework-00.txt 9.2 How might MPLS play here? What is MPLS? ------------- MultiProtocol Label Switching (MPLS) is a switching method in which the hop by hop switching decision is based on a label. The ordered set of labels defines a Label Switched Path (LSP). Each LSP has associated with it a set of criteria which describe the traffic that traverses the LSP. The set of criteria groups the traffic into a Forwarding Equivalence Class (FEC). Typically, IP version 4 traffic is the traffic of interest, and this IP traffic is grouped into FECs. These FECs can be associated with LSPs. In the most basic configuration, a FEC groups IP traffic based solely on the destination address, and associates them with what is commonly known as a next hop: a tuple that defines the lower layer address of the next IP router in the routed path, and the `interface' over which the traffic must be sent. Interface is, not surprisingly, an overloaded term. In the simplest configuration, the interface corresponds to a physical port, or PHY. Interface may also refer to a logical entity. For this discussion it will refer to a physical entity. In more complex, and potentially more useful configurations, a FEC can be used to group traffic into more flexible classes. For instance, instead of grouping traffic from all sources into a single FEC (default IP routing behavior), traffic from different sources and/or with different Diffserv code-points can be grouped into different FECs. The setup of LSPs is accomplished using one of several candidate signaling protocols. Two such protocols are the extensions to the ReSource ReserVation Protocol (RSVP), and the Label Distribution Protocol (LDP) along with its constraint-based routing extension CR-LDP. A device that can classify layer 3 traffic into a FEC for the purpose of associating this FEC with an LSP is known as a Label Edge Router (LER). A device that bases its switching decision solely on the label is known as a Label Switch Router (LSR). Traffic enters and exits the MPLS domain through LERs. Traffic traverses an MPLS domain through LSRs. How might MPLS fit in? ---------------------- So what relevance does this have for IP over optical networks? It is possible to model wavelengths, and potentially TDM channels within a wavelength as "labels" (see [2] for an enumeration of relevant commonalities). Furthermore, the wavelength `labels' need not change at every hop. For instance, an ADM can serve as an LSR just as well as an optical switch. An IP router with WDM capability can serve as an LER. WDM LSPs can be established solely using RSVP or LDP, or in conjunction with some other signaling protocol. If separate signaling protocols are used to establish parts of the LSP, then some extensions to LDP may be needed. If RSVP or LDP is used solely for label provisioning, then IP router functionality must be associated with each potential label switched hop. Also, each physical interface involved in an LSP must also be associated with a IP addressable interface. The use of WDM wavelengths and channels is not unlike the use of labels in conventional label switched technologies as noted in [2]. Internet-Draft draft-ip-optical-framework-00.txt For both label switching and WDM switches (OXCs), once the label has been provisioned by the protocol, the IP router no longer participates in forwarding of the traffic in the FEC. Instead, the native capabilities of the device are used to switch traffic to the eventual egress LER. WDM label switching is compatible with label merging. Label merging is a technique that can be used to merge paths from several related sources into a common path. 9.3 How might the NHRP or MPOA play here? What are NHRP and MPOA? ----------------------- The Next Hop Resolution Protocol (NHRP) was developed by the IETF "Internetworking over NBMA" (ION) working group for shortcut/router- bypass forwarding of unicast Network Layer (effectively this means IP) flows across a single Non-Broadcast Multi-Access (NBMA) subnetwork such as ATM and Frame Relay and perhaps may be applicable to WDM. In this context, the term subnetwork refers to the underlying media/data-link layer on which IP runs and does not refer to the practice of grouping ranges of addresses into subnets. It is further implied that a given subnetwork has a single data link layer over which datalink signaling may be performed. So for example, a shortcut would not be setup across multiple ATM clouds with distinct signaling domains. NHRP functions are performed by two types of logical entities: Next Hop Server (NHS) and Next Hop Client (NHC). NHS is typically implemented in routers, while NHC can be implemented in routers and NBMA-attached hosts. As defined in RFC 2332 NHRP uses a request/ response process to determine the NBMA address of the egress device for the NBMA cloud be it a host (the actual destination specified in the request) or an edge router for the NBMA. The ingress device does this by sending the NHRP requests along the routed path toward an ultimate destination. If the destination is connected to the NBMA subnetwork, then the NBMA next hop is the destination station itself. Otherwise, the NBMA next hop is the egress router from the NBMA subnetwork that is "nearest" to the destination station along the routed path. Further, since each router along the routed path must examine an NHRP request in order to determine which router to send the request to on its way toward its ultimate destination, each router may save state regarding information contained in the NHRP packet. Thus NHRP is also an ideal signaling protocol in addition to being an address resolution protocol (the former being pertinent here whereas the latter may not be). MPOA incorporates NHRP with LAN Emulation (LANE) to provide an efficient means for unicast packet transfer between emulated LANs. Although initially designed to accommodate any network layer protocol, IP support has been its primary application. There are two types of logical entities: MPOA Server (MPS), which is implemented in NBMA- attached routers and calls upon the functions of a co-located NHS, and MPOA Client (MPC), which is typically implemented in NBMA devices with LAN Emulation Client (LEC) functions which would be implemented in WDM ADMs or WDM switches. When an MPC device detects a sustained packet flow to an internetwork destination to be forwarded through a router, it initiates a Target Resolution process to the router. This Internet-Draft draft-ip-optical-framework-00.txt Target Resolution utilizes NHRP Resolution and has the same purpose of determining the NBMA address of the egress device and in some circumstances allows the transit MPSs to maintain state about the flow. The resolution message is forwarded along the routed path until it reaches the egress router for the destination. The egress device's reply is propagated back to the initial MPC device. In the case of a WDM subnetworks, once an LP has been setup, data packets to/through the egress device would then diverted over this LP shortcut. This reduces the processing at intermediate routers for the remainder of the flow and establishes a more direct path for the flow. How might NHRP fit in? ---------------------- In short, NHRP may be applied as a signaling mechanism for what is referred to as optical flow switching. To request a shortcut, the existing packet format for the NHRP Resolution Request would be used with a new extension in the form of a modified Forward Transit NHS Extension is included. The extension would include enough information at each hop (including the source and destination) to uniquely identify which wavelength(and potentially a `time interval'/ quanta within some cyclic measure of time such as an epoc in sonet) to use when bypassing each routed/forwarded hop and which port that the request was received on. When the egress NHS receives the Resolution request, and assuming there are sufficient resources at each bypassed hop (resources include both the willingness to forward another WL as well as the existence of an unused wavelength), the egress NHS will send a resolution reply back to the sender. For simplicity, it will be assumed that a given port is bi-directional. The architecture does not require this per se but can work with unidirectional links (only less elegantly). When the egress NHS sends the reply, it sends the reply back toward the receiver through the port on which the NHS received the resolution request (as stated previously, this is mostly a logical construct and does not preclude the existence of unidirectional links). However before the egress NHS does this, it programs its forwarding engine to drop the data which it receives on the appropriate wavelength (and potentially in a more granular the quanta) from the upstream NHS. Upon receiving an NHRP reply from a downstream neighbor, the upstream NHS programs its forwarding engine to send data for the shortcut on the wavelength it dedicated during the resolution request process and also programs its forwarding engine to receive the data from its upstream neighbor on the wavelength which the upstream neighbor said it would use and then the current NHS propagates the NHRP resolution reply back upstream on the port from which it received the resolution request. This process continues until the source of the resolution request is reached. In this way the shortcut is setup from ingress to egress. If wavelength conversion is not done on a hop by hop basis than the problem is difficult to do in a fully distributed manner. There is still merit in using signaling however. In this case, the ingress device would need to query some server which is administering the wavelength allocation process to ask permission and to ask for a wavelength to be allocated from ingress to egress. If the request Internet-Draft draft-ip-optical-framework-00.txt is granted then the same process would be carried out as previously described except that a given wavelength would be required to be allocated at each hop. Other ways of doing this are possible but this is by far the most simple. Note that an option here would be (whether or not wavelength conversion exists) to allow a transit NHS to terminate the shortcut if the downstream transit NHS has insufficient resources to sync the bypass. Note in this case, no bandwidth is gained by using shortcuts since all data would then need to go over the default link between the current NHS and the downstream NHS, however, it is possible that some delay and jitter performance might be gained in this context. 9.4 How might RSVP play here? What is RSVP? ------------- RSVP [RFC 2205] is a unicast and multicast signaling protocol, designed to install and maintain reservation state information at each routing engine along a path of a stream of data. The state handled by RSVP is defined by services specified by various groups (e.g., the Integrated Services WG, DiffServ (in the future), MPLS, etc.). RSVP has the following characteristics: o RSVP makes resource reservations for both unicast and multicast applications, adapting dynamically to changing group membership as well as to changing routes. o RSVP is simplex, i.e., it makes reservations for unidirectional data flows. There are extensions to RSVP that allow it to also handle traffic aggregates. o RSVP is receiver-oriented, i.e., the receiver of a data flow generally initiates and maintains the resource reservation used for that flow. o RSVP maintains "soft" state in routing engines, providing graceful support for dynamic membership changes and automatic adaptation to routing changes. o RSVP is not a routing protocol but depends upon existing and future routing protocols. o RSVP can transport and maintain traffic control and policy control parameters that are opaque to RSVP. o RSVP provides several reservation models or "styles" (defined below) to fit a variety of application needs. o RSVP provides transparent operation through routers that do not support it. Internet-Draft draft-ip-optical-framework-00.txt How might RSVP fit into IP over Optical Networks? ------------------------------------------------- References [5], [6] and [10] discuss how RSVP can be extended to serve as a signaling protocol for the establishment of optical channels and lightpaths. Furthermore, in the previous sections, if the terms NHS is substituted with`RSVP capable router', `NHRP Resolution Request' with `Path message', and `NHRP Resolution Reply' with `ReSV message,' then RSVP can be used to address the problem described in the NHRP example above with more richly defined semantics for QoS. 10. Security Considerations TBD. 11. References 1. Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 2. D. Awduche, Y. Rekhter, J. Drake, R. Coltun, "Multi-Protocol Lambda Switching: Combining MPLS Traffic Engineering Control With Optical Crossconnects," Internet Draft, Work in Progress, November 1999. 3. D. Awduche, J. Malcolm, J. Agogbua, M. O'Dell, "Requirements for Traffic Engineering over MPLS," RFC-2702, September, 1999. 4. D. Awduche, "MPLS and Traffic Engineering in IP Networks," IEEE Communications Magazine, December 1999. 5. D. Awduche, D. Awduche, L. Berger, D. Gan, T. Li, G. Swallow, and V. Srinivasan "Extensions to RSVP for LSP Tunnels," Internet Draft, Work in Progress, September 1999. 6. K. Kompella et al, "Extensions to IS-IS/OSPF and RSVP in support of MPL(ambda)S," Internet Draft, Work in Progress, February 2000. 7. D. Basak, D. Awduche, J. Drake, and Y. Rekhter, "Multi-protocol Lambda Switching: Issues in Combining MPLS Traffic Engineering Control With Optical Crossconnects," Internet Draft, Work in Progress, February 2000. 8. J. Luciani et al, "NBMA Next Hop Resolution Protocol (NHRP)," RFC-2332, April 1998. 9. B. Rajagopalan, D. Saha, B. Tang, and K. Bala, "Signaling Framework for Automated Establishment and Restoration of Paths in Optical Mesh Networks," draft-rstb-optical-signaling-framework-00.txt, March, 2000. 10. D. Saha, B. Rajagopalan and B. Tang, "RSVP Extensions for Signaling Optical Paths", draft-saha-rsvp-optical-signaling-00.txt, March, 2000. Internet-Draft draft-ip-optical-framework-00.txt 12. Acknowledgments We would like to thank Zouheir Mansourati and Ian Duncan of Nortel Networks for their comments on this draft. 13. Author's Addresses James V. Luciani TollBridge Technologies P,O. Box 1010 Concord, MA 01742 Email: james_luciani@mindspring.com Bala Rajagopalan Tellium, Inc. 2 Crescent Place P.O. Box 901 Oceanport, NJ 07757-0901 Email: braja@tellium.com Daniel O. Awduche UUNET (MCI Worldcom) Loudoun County Parkway Ashburn, VA 20247 Phone: 703-886-5277 Email: awduche@uu.net Brad Cain, Bilel Jamoussi Nortel Networks 600 Tech Park Billerica, MA 01821 Phone: 978-288-4734 Email: bcain,jamoussi@nortelnetworks.com ******** This draft expires on Sept., 10, 2000 ***********