Internet Draft


Internet Draft                                  James Luciani
Expiration Date: Sept., 10, 2000                 Tollbridge Technologies
                                                Bala Rajagopalan 
                                                 Tellium, Inc.
                                                Daniel Awduche
                                                 UUNET (MCI Worldcom)
                                                Brad Cain
                                                Bilel Jamoussi
                                                 Nortel Networks


                IP over Optical Networks - A Framework
                
                   draft-ip-optical-framework-00.txt
        
Status of this Memo

   This document is an Internet-Draft and is in full conformance 
   with all provisions of Section 10 of RFC2026 except that the 
   right to produce derivative works is not granted. 
   
   Internet-Drafts are working documents of the Internet 
   Engineering Task Force (IETF), its areas, and its working 
   groups. Note that other groups may also distribute working 
   documents as Internet-Drafts. 
   
   Internet-Drafts are draft documents valid for a maximum of 
   six months and may be updated, replaced, or obsoleted by 
   other documents at any time. It is inappropriate to use 
   Internet- Drafts as reference material or to cite them other 
   than as "work in progress." 
   
   The list of current Internet-Drafts can be accessed at 
   http://www.ietf.org/ietf/1id-abstracts.txt 
   The list of Internet-Draft Shadow Directories can be accessed 
   at http://www.ietf.org/shadow.html.
   
   
1. Abstract
   
   The Internet transport infrastructure is moving towards a model of 
   high-speed routers interconnected by optical core networks. A 
   consensus is emerging in the industry on utilizing IP-centric 
   protocols for the optical control plane [9, 10], as well as for 
   dynamic bandwidth provisioning for IP services. Also, there has 
   recently been significant activity in defining models for IP 
   transport over optical networks, specifically, the routing and 
   signaling aspects [2,6,7]. This draft attempts to define a framework 
   for IP over Optical networks, considering both the IP control plane 
   for optical networks as well as IP transport over optical networks 
   (together referred to as "IP over optical networks"). Within this 
   framework, we develop a common set of terms and concepts which allows 
   to discuss these IP over optical technologies in a consistent fashion.  
   Additionally, this draft surveys some architectural frameworks that 
   might be useful and appropriate for the deployment of IP over hybrid 
   optical networks that contain IP routers, WDM multiplexers, and 
   optical cross-connects (OXCs). This document is complementary to the 
   framework of Multiprotocol Lambda Switching proposed in [2].


   Internet-Draft	               draft-ip-optical-framework-00.txt



   
2. Conventions used in this document
   
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this 
   document are to be interpreted as described in RFC-2119.
   
   
3. Introduction
   
   Optical network technologies have evolved rapidly in terms of 
   functions, capabilities, and densities. The increasing importance of 
   optical transport networks is evidenced by the copious amount of 
   attention focused on IP over optical networks and related photonic 
   and electronic interworking issues by all the major network service 
   providers, telecommunications equipment vendors, and standards 
   organizations all over the world.
   
   One important factor driving the trend towards multi-wavelength 
   optical networking is the desire to capitalize upon the 
   opportunities and challenges presented by the exponential growth of 
   the Internet and the resulting  intense demand for broadband services. 
   
   It has been realized that optical networks must be survivable, 
   flexible, and controllable. There is, therefore, an ongoing trend to 
   introduce intelligence in the control plane of optical transport 
   systems to make them more versatile [2]. An essential attribute of 
   intelligent optical transport networks is the capability to 
   instantiate and route optical channels in real-time or near real-time,
   and to provide capabilities that enhance network survivability. 
   Furthermore, there is a need for multi-vendor optical network 
   interoperability, when an optical transport network may consist of 
   interconnected vendor-specific optical sub-networks [9].
   
   Many advocates of intelligent optical transport systems contend that 
   `optical routing' will eventually become cheaper than `electrical 
   routing,' and that all optical networking will eliminate the 
   electronic bottlenecks imposed by processing in the electrical domain.  
   These are clearly very important strategic factors motivating the 
   current intense activity to evolve and commercialize optical transport 
   systems. The real benefit of intelligent optical networks, however, 
   will accrue from the potential to construct more scalable networks, 
   and to expedite the provisioning process while enabling a plethora 
   of protection and restoration capabilities in operational contexts. 
   
   The optical network must also be versatile because some service 
   providers and network contexts may provide generic optical layer 
   services that may not be specific to any digital clients above the 
   optical transport network.  In the context of an automatically 
   switched optical network, it would be necessary to have a control 
   layer that can handle such generic optical services.  
   





   Internet-Draft	               draft-ip-optical-framework-00.txt



   This memo is an attempt to bind the problem space at hand to a 
   framework and to provide a common language with which to speak about 
   the IP-based control and IP transport over optical networks. The 
   following sections describe a set of candidate models for this, 
   along with a discussion of their relative advantages and 
   disadvantages.  It is certainly not the intent of this draft to 
   promote any particular model over the others. However, prior lessons 
   learned from layering IP over other media will tend to highlight 
   particular aspects of the models which may make one approach more 
   appropriate than another in certain circumstances.
   
   In the following sections, we consider the IP control plane issues in 
   optical networks and describe three generic models for IP transport 
   over optical networks. These models are: (1) the "Overlay" model, 
   (2) the "Integrated" model, and (3) the "Peer" model. The reader can 
   see that the terminology used to describe these models have some 
   similarity to the terminology previously used to describe IP over 
   ATM models. 

   These transport models differ from each other in a number of ways.  
   One important manner in which the models differ is in the way that 
   routing and signaling protocols are run over the IP and optical 
   subnetwork layers. Some of these considerations were alluded to 
   in [2] and include the following aspects:

   o whether there is a single monolithic instance of routing 
     and signaling protocols that span the IP and optical domains;

   o whether there are separate instances of routing and signaling 
     protocols for the IP domain and the optical transport network.

   o If there are separate instances of routing protocols for 
     each domain then the following additional consideration 
     apply:  whether routing information is leaked from one 
     routing protocol instance into the other; 

   o In the case of a single monolithic instance of the 
     routing protocol for both the IP and optical domains, 
     whether both domains actively participate in the 
     exchange of routing information or whether one layer is 
     passive with respect to the mutual exchange of routing 
     information. 
   
   Another manner in which the models differ is with respect to the way 
   in which label switching protocols might run over them or in 
   conjunction with them.  Clearly, a single monolithic label switching 
   protocol would be very interesting architecturally and 
   administratively because of its potential simplicity, conceptual 
   integrity, and ease of management; especially from the perspective 
   of network operations control.  But the semantics of "label switching"
   and the establishment and maintenance of "LSPs" in an optical network 
   may be different in optical networks as compared to traditional MPLS 
   networks [9]. There are also some challenging protocol issues to be 
   addressed, however.  For example, how would IP QoS requirements be 
   mapped onto the QoS capabilities of the optical transport network 
   (that is, in the contexts where such QoS mappings are actually 
   relevant).


   Internet-Draft	               draft-ip-optical-framework-00.txt


   
   As for IP-based control plane for optical networks, the most 
   challenging issues are related to meeting the strict reliability and 
   time constraints in the optical domain. While some mappings are 
   straightforward in adopting IP-based protocols for optical network 
   control, others are not [9]. Furthermore, proprietary mechanisms 
   will be used in optical sub-networks for certain functions like 
   restoration. The interworking of these sub-networks when putting 
   together a large core network is an issue.
   
   
3.1 Terminology
   
   This subsection introduces some terminology for describing common 
   concepts in IP over optical networks. These terms serve as a 
   blueprint which allow common issues in the IP over optical networks 
   to be described consistently and coherently.
   
   WDM
   ---
   
   Wavelength Division Multiplexing (WDM) is a technology that allows 
   multiple optical signals operating at different wavelengths to be 
   multiplexed onto a single fiber so that they can be transported in 
   parallel through the fiber. In general, each optical wavelength may 
   carry digital client payloads at a different data rate (e.g., OC-3c, 
   OC-12c, OC- 48c, OC-192c, etc.) and in a different format (SONET, 
   Ethernet, ATM, etc.)   For example, there are many commercial WDM 
   networks in existence today that support a mix of SONET signals 
   operating at OC-48c (approximately 2.5 Gbps) and OC-192 
   (approximately 10 Gbps) over a single optical fiber. An optical 
   system with WDM capability can achieve parallel transmission of 
   multiple wavelengths gracefully while maintaining high system 
   performance and reliability. In the near future, commercial WDM  
   systems are expected to concurrently carry  more than 160 wavelengths 
   at data rates of OC-192c and above, for a total of 1.6 Tbps and more. 
   The term WDM will be used throughout this document to refer to both 
   WDM and DWDM (Dense WDM).  The exception is when it is necessary to 
   differentiate between WDM and DWDM, in which case, the distinction 
   will be made explicit.
   
   In general, it is worth noting that WDM links are affected by the 
   following factors, which may introduce impairments into the optical 
   signal path:

   1. The number of wavelengths on a single fiber.
   2. The serial bit rate per wavelength. 
   3. The type of fiber.
   4. The amplification mechanism.
   5. The number of nodes through which the signal passes before 
      it reaches the egress node or before regeneration.

   All these factors and others not mentioned here constitute domain 
   specific features of optical transport networks. As noted in [2], 
   these features should be taken into account in developing standards 
   based solutions for IP over WDM systems. 
   


   Internet-Draft	               draft-ip-optical-framework-00.txt



   Non-Broadcast Multi-Access Network (NBMA)
   ----------------------------------------- 

   A subnetwork can be non-broadcast either because it technically does 
   not support broadcasting or because broadcasting is not feasible for 
   one reason or another (e.g., an extended Ethernet which is too large) 
   [7].
   
   
   IP Routed hop
   -------------
   
   Within the context of this memo, an IP routed hop is any device that 
   is IP aware and is able to forward IP packets from an input port to 
   an output after possibly performing some operations on the packets. 
   An IP routed hop device will participate in IP routing protocols 
   such as OSPF, or IS-IS, or derivatives thereof. An IP routed hop 
   device is thus distinguished from a pure switch which does not 
   participate in an IP routing protocol.  Switch routers, routing 
   switches, and label switched routers are all potentially capable of 
   acting as both pure switches and IP routed hop elements.
   
   
   Trust domain
   ------------ 

   A trust domain is a network under a single technical administration 
   in which most nodes in the network are considered to be secure or 
   trusted in some fashion.  An example of a trust domain is a campus 
   or corporate network in which all routing protocol packets are 
   considered to be authentic, without the need for additional 
   security schemes to prevent unauthorized access to the network 
   infrastructure.  Generally, the "single" administrative control 
   rule may be relaxed in practice if a set of administrative 
   entities agree to trust one another to form an enlarged heterogeneous 
   trust domain. However, to simplify the discussions in this memo, it 
   will be assumed, without loss of generality, that the term trust 
   domain applies to a single administrative entity.  

   Optical Channel Trail
   ---------------------
   
   An optical channel trail is a point-to-point optical connection 
   between two access points in an optical transport network.
   
   Wavelength continuity property 
   ------------------------------   

   An optical channel trail is said to satisfy the wavelength continuity 
   property if it consists of just one wavelength end-to-end. Wavelength 
   continuity is required in optical  networks with no wavelength 
   conversion feature.
   




   Internet-Draft	               draft-ip-optical-framework-00.txt



   Wavelength path vs. lightpath
   ----------------------------- 

   In an optical network without the wavelength conversion feature, we 
   define a wavelength path (WLP) as an optical path between an ingress 
   and egress node of the WDM network that uses exactly the same 
   wavelength from ingress node to egress node.  That is, a WLP uses a 
   specific wavelength between each node from source to destination. 
   Thus, a wavelength path satisfies the wavelength continuity property. 
   On the other hand, in an optical network with wavelength conversion, 
   we define a lightpath (LP) as an optical path from an ingress node to 
   an ingress that may or may not consist of the same WL at each hop. 
   That is, an LP may use a specific wavelength Lambda(k) between some 
   node Ni and Ni+1 but may use another wavelength Lambda(l) between 
   Ni+1 and Ni+2 such that Lambda(k) != Lambda(l) and Ni != Ni+1!= Ni+2. 
   Hence, as used in this document, a lightpath is a generalization of 
   the notion of wavelength path.
   
   
   Shortcut/cut-through path
   ------------------------- 

   A shortcut (used synonymously throughout this document with the term 
   cut-through) is defined as an LP or optical channel trail which causes 
   packets between its endpoints to bypass a number of normally routed 
   hops within the trust domain. To be more precise, a shortcut makes its 
   end-points appear to be adjacent to each other with respect to routed 
   hops, even though the short-cut LP itself may traverse a number of 
   intermediate nodes. 
   
   Flow
   ---- 

   For the purpose of this document, the term flow will be used to 
   represent the smallest separable stream of data, as seen by an 
   endpoint (source or destination node).  It is to be noted that the 
   term flow is heavily overloaded in the networking literature. Within 
   the context of this document, it may be convenient to consider a 
   wavelength as a flow under certain circumstances. However, if 
   there is a method to partition the bandwidth of the wavelength, 
   then each partition may be considered a flow, for example by  
   quantizing time into some nicely manageable intervals, it may be 
   feasible to consider each quanta of time within a given wavelength as 
   a flow. 
   
   Traffic Trunk
   ------------- 

   A abstraction of traffic flow that follows the same path between two 
   access points which allows some characteristics and attributes of the 
   traffic to be parameterized.
   




   Internet-Draft	               draft-ip-optical-framework-00.txt



   Opaque vs. transparent optical networks
   ---------------------------------------
   
   A transparent optical network is an optical network in which each 
   transit node along a path passes optical transmission without 
   transducing the optical signal into an electrical signal and back 
   to an optical signal.  More generally, all non-terminus nodes in a 
   transparent optical network will pass optical signals without 
   performing retiming and reshaping and thus such nodes are unaware of 
   many characteristics of the carried signals.  One could, for example, 
   carry analog signals side by side with digital signals (potentially of 
   varying bit rate) on different wavelengths over such a network. 
   
   Note that repowering of signals at transit nodes is potentially 
   permitted in transparent optical networks.  This is a result of 
   the availability of all optical amplification devices such as Erbium 
   Doped Fiber Amplifiers (EDFAs).
   
   In opaque optical networks, by comparison, transit nodes will perform 
   manipulation of the optical signals which they are carrying.  An 
   example of such manipulation would be 3R regeneration (reshaping, 
   retiming, regeneration/amplification).  
   
   Opaque optical networks are, by far, the most common type of network 
   deployed today.  However, there is intense interest in the development 
   and researching of practical all optical networks.
   
4. The Network Model
   
   The network model considered in this draft consists of of IP routers 
   attached to an optical core network, and connected to their peers 
   over dynamically established switched optical paths. The optical core 
   itself is assumed to be incapable of processing individual IP packets. 
   The interaction between the IP routers and the optical core is over a 
   well-defined signaling and routing interface.
   
   The optical network is assumed to consist of multiple Optical Cross-
   Connects (OXCs) interconnected by optical links in a general topology 
   (refered to as an "optical mesh network"). This network may be multi-
   vendor, where individual vendor OXCs constitute "clouds" or "sub-
   networks". Each sub-network itself is assumed to be mesh-connected.
   
   Each OXC is assumed to be capable of switching a data stream from a 
   given input port to a given output port. This switching function is 
   controlled by appropriately configuring a cross-connect table. 
   Conceptually, the cross-connect table consists of entries of the form 
   , indicating that data stream entering 
   input port i will be switched to output port j.  An "optical path" 
   from an ingress port in an OXC to an egress port in a remote OXC 
   is established by setting up suitable cross-connects in the ingress, 
   the egress and a set of intermediate OXCs such that a continuous 
   physical path exists from the ingress to the egress port. Optical 
   paths are assumed to be bi-directional, i.e., the return path from 
   the egress port to the ingress port follows the same path as the 
   forward path. The switching within the OXC may be accomplished in 
   the electrical domain, or the OXC may be all-optical. 


   Internet-Draft	               draft-ip-optical-framework-00.txt



   Furthermore, multiple data streams output from an OXC may be 
   multiplexed onto an optical link using WDM technology. The WDM 
   functionality may exist outside of the OXC, and be transparent to 
   the OXC. Or, this function may be built into the OXC. In the latter 
   case, the cross-connect table (conceptually) consists of pairs of 
   the form, <{input port i, Lambda(j)}, {output port k, Lambda(l)}>. 
   This indicates that the data stream received on wavelength Lambda(j) 
   over input port i is switched to output port k on Lambda(l). 
   Automated establishment of optical paths involves setting up the 
   cross-connect table entries in the appropriate OXCs in a coordinated 
   manner such that the desired physical path is realized.
   
   In general, it can be expected that topologically adjacent OXCs in an 
   optical mesh network will be connected via multiple, parallel (bi-
   directional) optical links. It is assumed that one or more control 
   channels exist between neighboring OXCs for signaling purposes.
   
   Under this network model, a switched optical path must be established 
   between a pair of IP routers before they can communicate. This path 
   might traverse multiple optical sub-networks and be subject to 
   different provisioning and restoration procedures in each sub-network.
   The IP-based control plane issue is that of designing standard 
   signaling and routing protocols for coherent end-to-end provisioning 
   and restoration of optical paths across multiple optical sub-networks. 
   Similarly, IP transport over such an optical network involves 
   determining IP reachability and seamless establishment of paths from 
   one IP endpoint to another over an optical core.
   
5. IP-based Control Plane Issues
   
   The control plane issues can be classified into signaling for 
   provisioning, signaling for restoration and routing issues. The 
   difference between the two signaling procedures is that restoration 
   signaling is bound by strict time constraints, whereas the time 
   constraint for provisioning is more relaxed. Some of the signaling 
   issues are described in [9, 10]. To summarize, the following are 
   some of the issues that must be addressed when considering the 
   development of IP-based signaling for optical networks:
   
   o What is the signaling interface between the IP routers and the 
     optical network?

   o How to automatically discover local topology information between 
     adjacent OXCs so that end-to-end path establishment and restoration 
     are automated?

   o How to establish bi-directional paths without race conditions?

   o How to ensure fault-tolerance at the data plane when the control 
     plane may be affected by failures?

   o Whether soft-state-based signaling protocols are suitable in the 
     optical network environment, 

   o How to ensure signaling for restoration can meet the strict time 
     bounds?



   Internet-Draft	               draft-ip-optical-framework-00.txt



   o Whether separate signaling protocols must be developed for 
     restoration signaling,

   o How to allow proprietary signaling protocols within optical 
     sub-networks for local restoration (and perhaps for provisioning) 
     while developing a uniform standard end-to-end protocol?

   o How does MPLS-based restoration at the IP level work with optical 
     level fast restoration?

   o How to accommodate both OXCs with wavelength conversion capability 
     and those without in the optical network?

   o Can out-of-band and in-band signaling procedures co-exist?
   
   The routing issues deal with similar problems of interworking:
   
   o How to ensure  that end-to-end reachability information is 
     propagated across the optical network?  

   o How to accommodate proprietary optimizations within optical 
     sub-networks for provisioning and restoration of paths?

   o Whether dynamic and pre-computed routing information can be used 
     together, and if so, what is the interaction between them?

   o How to deal with dense adjacencies between OXCs?

   o What QoS and service-related parameters need to be defined?

   o How to ensure fault-tolerant operation at the protocol level when 
     the hardware supports fault tolerance?

   o How to address the scalability issue?  

   We do not get into the details of these issues, but defer them for 
   discussion in future drafts. We just note that a clear set of 
   requirements for IP-based control plane protocols (signaling and 
   routing) need to be defined before addressing the specific issues. 
   There is also some overlap between the issues mentioned above and 
   the issues in IP transport over Optical networks discussed next.
    
6. IP transport over Optical Networks

6.1 The overlay model
   
   Under the overlay model, IP is more or less independent of the 
   optical subnetwork from a routing perspective.  The overlay model 
   is a client-server model, in which the IP domain is a client of the 
   optical domain. In this scenario, the optical network provides 
   point-to-point optical links for the transport of IP packets through 
   the optical domain. This model is conceptually similar to the 
   classical IP over ATM or MPOA models, but applied to a optical 
   subnetwork directly. 
   




   Internet-Draft	               draft-ip-optical-framework-00.txt



   Under the overlay model, the IP/MPLS routing, topology distribution, 
   and signaling protocols are independent of the routing, topology 
   distribution, and signaling protocols at the optical layer.  MPLS 
   may nonetheless provide a mechanism to cut through or bypass routed 
   hops.  In the overlay model, lambda routing, topology distribution, 
   and signaling protocols would have to be defined for the optical 
   domain. In certain circumstances, it may also be feasible to 
   statically configure the optical channels that provide connectivity 
   in the overlay model through network management. Static configuration 
   is, however, unlikely to scale in very large networks. A protocol 
   like GSMP might also be employed here, in conjunction other control 
   plane protocols to establish lightpaths and optical channel trails in 
   the overlay model. Note that in this model, a cut-through lambda may 
   not necessarily affect the L3 routing information; i.e., a shortcut 
   may or may not add an entry to the L3 routing table.  
   
6.2 The integrated/augmented model
   
   In the integrated model, the IP/MPLS layers act as peers of the 
   optical transport network, such that a single routing protocol 
   instance runs over both the IP/MPLS and optical domains. Presumably 
   a common IGP such as OSPF or IS-IS, with appropriate extensions, will 
   be used to distribute topology information (see e.g.,[5] and [6]). 
   In the case of OSPF, opaque LSAs will be used to advertise topology 
   state information [5]. In the case of IS-IS, extended TLVs will have 
   to be defined to propagate topology state information [6]. One tacit 
   assumption here is that a common addressing scheme will also be used 
   for the optical and IP networks. A common address space can be 
   trivially realized by using IP addresses in both IP and optical 
   domains. Thus, the optical network elements become IP addressable 
   entities as noted in [2].
   
   In the augmented model, there are actually separate routing instances 
   in the IP and optical domains, but information from one routing 
   instance is leaked into the other routing instance. For example, IP 
   addresses could be assigned to optical network elements and carried 
   within the optical routing protocols to allow reachability 
   information to be shared with the IP domain, and to support some 
   degree of automated resource discovery.
   
6.3 The peer model
   
   A peer model is somewhat similar to an integrated model in that IP 
   reachability information might be passed around within the optical 
   routing protocol but the actual flow would be terminated at the edge 
   of the optical network and will only be re-established upon reaching 
   a non-peer capable node either at the edge of the optical domain or 
   at the edge of a domain which implements both peer and overlay models. 
   
   The overlay, integrated, and peer models can also be described using 
   the terminology of "termination capable" (TC) and "terminology 
   incapable" (TI) interfaces, in conjunction with the generalized 
   notion of LSP nesting described in [6].  
   




   Internet-Draft	               draft-ip-optical-framework-00.txt



7. Some starting assumptions
   
   
   WDM and TDM in the same network element
   ---------------------------------------
   
   A practical assumption would be that if SONET (or some other TDM 
   mechanism that is capable partitioning the bandwidth of a wavelength) 
   is used, then TDM leveraged as an additional method to differentiate 
   between "flows."  In such cases, wavelengths and time intervals (sub-
   channels) within a wavelengths become analogous to labels (as noted 
   in [2]) which can be used to make switching decisions. This would be 
   somewhat akin to using VPI (e.g., wavelength) and VCI (e.g., TDM sub-
   channel) in ATM networks. More generally, this will be akin to label 
   stacking and to LSP nesting within the context of Multiprotocol 
   Lambda Switching [2]. 
   
   Wavelength converters
   --------------------- 

   Some form of wavelength conversion may exist at some switching 
   elements, however, this may not be case in some pure optical 
   switching elements.  A switching element is essentially anything more
   sophisticated than a simple repeater, that is capable of switching 
   and converting a wavelength Lambda(k) from an input port to a 
   wavelength  Lambda(l) on an output port.  In this display, it is not 
   necessarily the case that Lambda(k) = Lambda(l), nor is it 
   necessarily the case that the data carried on Lambda(k) is switched 
   through the device without being examined or modified. 
   
   It is not necessary to have a wavelength converter at every switching 
   element.  A number of studies have attempted to address the issue of 
   the value of wavelength conversion in an optical network. Such studies 
   typically use the blocking probability (the probability that a 
   lightpath cannot be established because the requisite wavelengths 
   are not available) as a metric to adjudicate the effectiveness of 
   wavelength conversion.  The IP over optical architecture must take 
   into account hybrid networks with some OXCs capable of wavelength 
   conversion and others incapable of this.
   
   Service provider peering points
   ------------------------------- 

   There are proposed inter-network interconnect models which allow 
   certain types of peering relationships to occur at the optical 
   layer. This is consistent with the need to support optical layer 
   services independent of higher layers payloads. In the context of IP
   over optical networks, peering relationships between different trust 
   domains will eventually have to occur at the IP layer, on IP routing 
   elements, even though non-IP paths may exist between the peering 
   routers. 
   
   Optical Network as an NBMA Network
   ----------------------------------   

   A mesh based optical network is very much like an NBMA network in 
   terms of its subnetwork characteristics.  


   Internet-Draft	               draft-ip-optical-framework-00.txt



   
   Rate of LP setups
   -----------------
   
   Dynamic establishment of optical channel trails and lightpaths is 
   quite desirable in IP over optical networks, especially when such 
   instantiations are driven by a stable traffic engineering control 
   system, or in response to authenticated and authorized requests from 
   client.
   
   However, there are many proposals suggesting the use of dynamic, 
   data-driven shortcut-LP setups in IP over optical networks. The 
   arguments put forth in such proposals are quite reminiscent of 
   similar discussions regarding ATM deployment in the core of IP 
   networks.  Deployment of highly dynamic data driven shortcuts 
   within core networks has not been widely adopted by carriers 
   and ISPs for a number of reasons: possible CPU overhead in core 
   network elements, complexity of proposed solutions, stability 
   concerns, and lack of true economic drivers for this type of 
   service.  This draft assumes that this paradigm will not change 
   and that highly dynamic, data-driven shortcut LP setups are for 
   future investigation.  
   
   Instead, the optical channel trails and LPs that are expected to be 
   widely used at the initial phases in the evolution of IP over optical
   networks will include the following: 
   
   o Dynamic connections for control plane traffic and default path 
     routed data traffic, 

   o Establishment and re-arrangement of arbitrary virtual topologies 
     over rings and other physical layer topologies. 

   o Use of stable traffic engineering control systems to engineer LP 
     connections to enhance network performance, either for explicit 
     demand based QoS reasons or for load balancing).   
   
   Other issues surrounding dynamic connection setup within the core 
   center around  resource usage at the edge of the optical domain. 
   One potential issue pertains to the number of flows that can be 
   processed by an ingress or egress network element either because of 
   aggregate bandwidth limitations or because of a limitation on the 
   number of flows (e.g., LPs) that can be processed concurrently. 
   
   Another possible short term reason for dynamic shortcut LP setup 
   would be to quickly pre-provisioned paths based on some criteria 
   (TOD, CEO wants a high BW reliable connection, etc.).  In this 
   scenario, a set of paths is pre-provisioned, but not actually 
   instantiated until the customer initiates an authenticated and 
   authorized setup requests which is consistent with existing 
   agreements between the provider and the customer.   In a sense, the 
   provider may have already agreed to supply this service, but will 
   only instantiate it by setting up a lightpath when the customer 
   submits an explicit request. 
   
   


   Internet-Draft	               draft-ip-optical-framework-00.txt



   Distributed vs. centralized cut through path calculation
   --------------------------------------------------------   

   One model of shortcut path calculation is to have a centralized 
   mechanism (perhaps simply a suitably equipped high end PC) which has 
   complete knowledge of the physical topology, the available 
   wavelengths, and where applicable relevant time domain information.  
   In this centralized model, the centralized resource acts a server or 
   bandwidth broker. A corresponding client will reside on each network 
   element that can source or sink an LP.  The source client would query 
   the server in order to set up an LP from the source to the destination.  
   The server would then check to see if such an LP can be established 
   based on prevailing conditions. Furthermore,  depending on the 
   specifics of the model, the server may either setup the LP on behalf 
   of the client or provide the necessary information to the client or 
   to some other entity to allow the  LP to instantiated (e.g., the 
   information may consist of a set of WLPs to be used at each hop 
   within the trust domain). Another option may be for the server to 
   indicate that it is not possible to setup an LP to the egress point 
   but that it might be possible to bypass a certain number of nodes and 
   terminate the shortcut at a routed hop which is closer to the 
   destination than the source node is.
   
   The second possibility is a distributed model in which all nodes 
   maintain a synchronized topology database, and advertise topology 
   state information to maintain and refresh the database. A constraint-
   based routing entity on each node may then use the information in the 
   topology database and other relevant details to compute appropriate 
   paths through the optical domain. Once a path is computed, a signaling 
   protocol such as RSVP can be used to instantiate the LP. 
   
8. What services need to be there?
   
   Who needs QoS when bandwidth is free?
   -------------------------------------
   
   Even though bandwidth may become abundant and relatively cheap in 
   the future, it does prelude the need for traffic engineering 
   capabilities to ensure that the bandwidth is actually used 
   effectively to deliver high quality services. Automated traffic 
   engineering capabilities can also drastically reduce the amount of 
   manual overhead required for network operations control.  
   
   LP VPNs
   -------

   There has been some talk of the use of LPs as methods to create VPN 
   links across carriers.  In the case of a ring, one could in fact 
   imagine a given WL not only serving to isolate one routing domain 
   from another but also to act as a multidrop for the pt-mpt case 
   (assuming that this is desirable).
   


   Internet-Draft	               draft-ip-optical-framework-00.txt



   Traffic engineered paths for fault tolerance and load balancing
   ---------------------------------------------------------------
   
   Constraint based LP routing and non-equal cost multipath: In the mesh 
   optically switched configurations, multiple feasible paths  would 
   exist between ingress and egress nodes at the boundaries of the 
   optical cloud.  In this case, for the purposes of traffic engineering,
   it might be valuable to have capability to setup LPs which satisfy 
   certain requirements subject to certain constraints.  There is nothing 
   new in this idea, except that the connection oriented infrastructure is 
   built from optical rather than traditional L2 devices.  Nonetheless, 
   this potential should be noted and should be considered when defining 
   appropriate reference models.
   
   What about Multi-Link Trunking
   ------------------------------
   
   Multi-Link Trunking, also known as bundling (see [2]) is a concept that 
   allows multiple channels to be aggregated to form a single "trunk."  
   Thus for N links, the effective bandwidth for a multilink trunk may be 
   as large as N*BW where BW is the bandwidth of a single channel.  In the 
   IP world there is concern for ordered packet delivery, so simply 
   forwarding each packet to a different channel in a round robin fashion 
   (for example) is not a reasonable solution. What needs to be done is 
   some pre-classification prior to transmission of each packet to ensure 
   that packets belonging to the same "micro" flow are delivered in order 
   by sending them through the same channel consistently.  For example, a 
   simple but popular way to accomplish this is to apply a deterministic 
   hash function to certain fields in the IP packet headers such that the 
   output of the hash function can eventually  be associated with certain 
   virtual interfaces which are correspondingly mapped to specific 
   channels within the trunk on which the packets will be sent. When a 
   channel of a trunk fails (e.g., a laser fails) and that failure is 
   detected then one of two things could happen: 1) the packets of the 
   micro flows associated with the failed channel are dropped while the 
   rest of the flow goes on unhindered, 2) the mapping function is 
   dynamically modified so that that the impacted micro flows are 
   redirected down another link.  Thus multi-link trunking provides some 
   higher layer mechanism to enhance network survivability through fault 
   tolerance.  
   
   What about an IMUX
   ------------------
   
   For the purposes of this draft the definition of an IMUX is somewhat 
   similar in concept to multi-link trunking in that for a given flow, 
   data within that flow will travel in parallel over multiple 
   "channels."  In the IMUX case, however, a given flow is cut up into 
   segments/chunks of equal sized data (bits, bytes, words, or whatever) 
   without regard to any higher layer notion of flows and addressing, and 
   the segments are distributed across the links of a trunk bundle.  Thus 
   there is an implied re-assembly mechanism on the other side.  Again, 
   this segmentation and re-assembly (SAR) function is without regard to 
   packet boundaries from the perspective of the IMUX (of course the 
   higher layers will need to make sense of the re-assembled flow at the 
   egress).  Furthermore, within an IMUX, links may remain unused and may 
   be used in case of failure as backup to the failed links.  


   Internet-Draft	               draft-ip-optical-framework-00.txt



   This function is one possible way to build aggregate bandwidth trunks 
   from slower channels. The current practical reality is that the 
   above mentioned type of IMUX can only be done over relatively short 
   distances at high data rates, although this situation is likely to 
   improve in the future as technology evolves.
   
   Wavelength and TDM signaling
   ----------------------------
   
   It will be assumed that there exists some default communication 
   mechanism between routers prior to using any of the routing and 
   signaling mechanisms.  If a ring topology exists then this 
   default mechanism would most likely be pre-configured (e.g., a 
   default communication WL between routers or perhaps routers and 
   ADMs).  If a switched infrastructure exists then it is likely 
   that some dynamic routing protocol would exist or at the very 
   least some NM interface would need to exist in order to statically 
   connect each router with its appropriate peer within the trust 
   domain.
   
9. Some potential protocols for signaling
   
9.1 How might GSMP play here?
   
   What is GSMP?
   -------------
   
   The GSMP allows a "controller" to control a "switch". The system of 
   controller plus switch typically functions as a network node in a 
   TCP/IP, ATM or Frame Relay network. Broadly speaking, the controller 
   runs "control protocols" that provide the required network layer 
   services such as IP routing protocols, LDP, PNNI signaling and 
   routing, etc. The term "control plane" refers to the set of protocols 
   and mechanisms used to support a node of one network type, e.g. an 
   ATM control plane. A controller may support multiple control planes. 
   If the physical network supports multiple control planes then the 
   term "logical network" is used to refer to a network as seen by one 
   control plane.
   
   The switch is a general label switch with appropriate label switched 
   ports. The controller is responsible for installing and deleting 
   connection state in the switch.
   
   How might GSMP fit in?  
   ---------------------------
   
   In a general mesh network where the OXCs do not participate in 
   topology distribution protocols and where a router has full 
   knowledge of LP, L2, and L3 topology GSMP might be used to 
   communicate a binding between, for example, Lambda(i) coming in on 
   port j of an OXC and going out on Lambda(k) going out port m.  In 
   this scenario, an ingress (or egress) router would signal each OXC 
   between ingress and egress nodes along the LP.  It would signal using 
   GSMP to ask the OXC to set up its cross connect management table 
   appropriately.  
   


   Internet-Draft	               draft-ip-optical-framework-00.txt



9.2 How might MPLS play here?
   
   What is MPLS?
   -------------
   
   MultiProtocol Label Switching (MPLS) is a switching method in which 
   the hop by hop switching decision is based on a label.  The ordered 
   set of labels defines a Label Switched Path (LSP).  Each LSP has 
   associated with it a set of criteria which describe the traffic 
   that traverses the LSP.   The set of criteria groups the traffic 
   into a Forwarding Equivalence Class (FEC).  Typically, IP version 4 
   traffic is the traffic of interest, and this IP traffic is grouped 
   into FECs.  These FECs can be associated with LSPs.  In the most 
   basic configuration, a FEC groups IP traffic based solely on the 
   destination address, and associates them with what is commonly known 
   as a next hop: a tuple that defines the lower layer address of the 
   next IP router in the routed path, and the `interface' over which the 
   traffic must be sent. Interface is, not surprisingly, an overloaded 
   term.  In the simplest configuration, the interface corresponds to a 
   physical port, or PHY.  Interface may also refer to a logical entity.  
   For this discussion it will refer to a physical entity.  In more 
   complex, and potentially more useful configurations, a FEC can be 
   used to group traffic into more flexible classes.  For instance, 
   instead of grouping traffic from all sources into a single FEC
   (default IP routing behavior),  traffic from different sources 
   and/or with different Diffserv code-points can be grouped into 
   different FECs.  The setup of LSPs is accomplished using one of  
   several candidate signaling protocols.  Two such protocols are the 
   extensions to the ReSource ReserVation Protocol (RSVP), and the Label 
   Distribution Protocol (LDP) along with its constraint-based routing 
   extension CR-LDP.   
   
   A device that can classify layer 3 traffic into a FEC for the purpose 
   of associating this FEC with an LSP is known as a Label Edge Router
   (LER).  A device that bases its switching decision solely on the 
   label is known as a Label Switch Router (LSR).  Traffic enters and 
   exits the MPLS domain through LERs.  Traffic traverses an MPLS domain 
   through LSRs. 

   How might MPLS fit in?  
   ----------------------
   
   So what relevance does this have for  IP over optical networks?  It 
   is possible to model wavelengths, and  potentially TDM channels within 
   a wavelength as "labels" (see [2] for an enumeration of relevant 
   commonalities).  Furthermore, the wavelength `labels' need not change 
   at every hop.  For instance, an ADM can serve as an LSR just as well 
   as an optical switch.  An IP router with WDM capability can serve as 
   an LER.  WDM LSPs can be established solely using RSVP or LDP, or in 
   conjunction with some other signaling protocol. If separate signaling 
   protocols are used to establish parts of the LSP, then some extensions 
   to LDP may be needed. If RSVP or LDP is used solely for label 
   provisioning, then IP router functionality must be associated with 
   each potential label switched hop.  Also, each physical interface 
   involved in an LSP must also be associated with a IP addressable 
   interface. The use of  WDM wavelengths and channels is not unlike 
   the use of labels in conventional label switched technologies as noted 
   in [2].  


   Internet-Draft	               draft-ip-optical-framework-00.txt



   For both  label switching and WDM switches (OXCs), once the 
   label has been provisioned by the protocol, the IP router no longer 
   participates in forwarding of the traffic in the FEC. Instead, the 
   native capabilities of the device are used to switch traffic to the 
   eventual egress LER.  WDM label switching is compatible with label 
   merging.  Label merging is a technique that can be used to merge paths 
   from several related sources into a common path.
   
9.3 How might the NHRP or MPOA play here?
   
   What are NHRP and MPOA? 
   -----------------------
   
   The Next Hop Resolution Protocol (NHRP) was developed by the IETF 
   "Internetworking over NBMA" (ION) working group for shortcut/router-
   bypass forwarding of unicast Network Layer (effectively this means 
   IP) flows across a single Non-Broadcast Multi-Access (NBMA) 
   subnetwork such as ATM and Frame Relay and perhaps may be applicable 
   to WDM.  In this context, the term subnetwork refers to the underlying 
   media/data-link layer on which IP runs and does not refer to the 
   practice of grouping ranges of addresses into subnets.  It is further 
   implied that a given subnetwork has a single data link layer over 
   which datalink signaling may be performed.  So for example, a 
   shortcut would not be setup across multiple ATM clouds with distinct 
   signaling domains.
   
   NHRP functions are performed by two types of logical entities: Next 
   Hop Server (NHS) and Next Hop Client (NHC).  NHS is typically 
   implemented in routers, while NHC can be implemented in routers and 
   NBMA-attached hosts. As defined in RFC 2332 NHRP uses a request/
   response process to determine the NBMA address of the egress device 
   for the NBMA cloud be it a host (the actual destination specified in 
   the request) or an edge router for the NBMA.  The ingress device does 
   this by sending the NHRP requests along the routed path toward an 
   ultimate destination. If the destination is connected to the NBMA 
   subnetwork, then the NBMA next hop is the destination station itself.  
   Otherwise, the NBMA next hop is the egress router from the NBMA 
   subnetwork that is "nearest" to the destination station along the 
   routed path.  Further, since each router along the routed path must 
   examine an NHRP request in order to determine which router to send 
   the request to on its way toward its ultimate destination, each router 
   may save state regarding information contained in the NHRP packet.  
   Thus NHRP is also an ideal signaling protocol in addition to being 
   an address resolution protocol (the former being pertinent here 
   whereas the latter may not be).
   
   MPOA incorporates NHRP with LAN Emulation (LANE) to provide an 
   efficient means for unicast packet transfer between emulated LANs.  
   Although initially designed to accommodate any network layer protocol, 
   IP support has been its primary application.  There are two types of 
   logical entities:  MPOA Server (MPS), which is implemented in NBMA-
   attached routers and calls upon the functions of a co-located NHS, 
   and MPOA Client (MPC), which is typically implemented in NBMA devices 
   with LAN Emulation Client (LEC) functions which would be implemented 
   in WDM ADMs or WDM switches.  When an MPC device detects a sustained 
   packet flow to an internetwork destination to be forwarded through a 
   router, it initiates a Target Resolution process to the router. This 


   Internet-Draft	               draft-ip-optical-framework-00.txt



   Target Resolution utilizes NHRP Resolution and has the same purpose 
   of determining the NBMA address of the egress device and in some 
   circumstances allows the transit MPSs to maintain state about the 
   flow.  The resolution message is forwarded along the routed path 
   until it reaches the egress router for the destination. The egress 
   device's reply is propagated back to the initial MPC device. 
   
   In the case of a WDM subnetworks, once an LP has been setup, data 
   packets to/through the egress device would then diverted over this 
   LP shortcut. This reduces the processing at intermediate routers 
   for the remainder of the flow and establishes a more direct path 
   for the flow.  
   
   How might NHRP fit in? 
   ---------------------- 

   In short, NHRP may be applied as a signaling mechanism for what is 
   referred to as optical flow switching. To request a shortcut, the 
   existing packet format for the NHRP Resolution Request would be 
   used with a new extension in the form of a modified Forward Transit 
   NHS Extension is included.  The extension would include enough 
   information at each hop (including the source and destination) to 
   uniquely identify which wavelength(and potentially a `time interval'/
   quanta within some cyclic measure of time such as an epoc in sonet) 
   to use when bypassing each routed/forwarded hop and which port that 
   the request was received on. When the egress NHS receives the 
   Resolution request, and assuming there are sufficient resources at 
   each bypassed hop (resources include both the willingness to forward 
   another WL as well as the existence of an unused wavelength), the 
   egress NHS will send a resolution reply back to the sender.  For 
   simplicity, it will be assumed that a given port is bi-directional.  
   The architecture does not require this per se but can work with 
   unidirectional links (only less elegantly).  When the egress NHS 
   sends the reply, it sends the reply back toward the receiver through 
   the port on which the NHS received the resolution request (as stated 
   previously, this is mostly a logical construct and does not preclude 
   the existence of unidirectional links).  However before the egress 
   NHS does this, it programs its forwarding engine to drop the data 
   which it receives on the appropriate wavelength (and potentially in 
   a more granular the quanta) from the upstream NHS.  Upon receiving an 
   NHRP reply from a downstream neighbor, the upstream NHS programs 
   its forwarding engine to send data for the shortcut on the 
   wavelength it dedicated during the resolution request process and 
   also programs its forwarding engine to receive the data from its 
   upstream neighbor on the wavelength which the upstream neighbor said 
   it would use and then the current NHS propagates the NHRP resolution 
   reply back upstream on the port from which it received the resolution 
   request. This process continues until the source of the resolution 
   request is reached.  In this way the shortcut is setup from ingress 
   to egress.  
   
   If wavelength conversion is not done on a hop by hop basis than the 
   problem is difficult to do in a fully distributed manner.  There is 
   still merit in using signaling however.  In this case, the ingress 
   device would need to query some server which is administering the 
   wavelength allocation process to ask permission and to ask for a 
   wavelength to be allocated from ingress to egress.  If the request 


   Internet-Draft	               draft-ip-optical-framework-00.txt



   is granted then the same process would be carried out as previously 
   described except that a given wavelength would be required to be 
   allocated at each hop.  Other ways of doing this are possible but 
   this is by far the most simple.  

   Note that an option here would be (whether or not wavelength 
   conversion  exists) to allow a transit NHS to terminate the shortcut 
   if the downstream transit NHS has insufficient resources to sync the 
   bypass.  Note in this case, no bandwidth is gained by using shortcuts 
   since all data would then need to go over the default link between 
   the current NHS and the downstream NHS, however, it is possible that 
   some delay and jitter performance might be gained in this context.
   
   
9.4 How might RSVP play here? 
   
   What is RSVP? 
   -------------
   
   RSVP [RFC 2205] is a unicast and multicast signaling protocol, 
   designed to install and maintain reservation state information at 
   each routing engine along a path of a stream of data.  The state 
   handled by RSVP is defined by services specified by various groups 
   (e.g., the Integrated Services WG, DiffServ (in the future), MPLS, 
   etc.). RSVP has the following characteristics:
   
   o RSVP makes resource reservations for both unicast and multicast 
     applications, adapting dynamically to changing group membership 
     as well as to changing routes.

   o RSVP is simplex, i.e., it makes reservations for unidirectional 
     data flows. There are extensions to RSVP that allow it to also 
     handle traffic aggregates.

   o RSVP is receiver-oriented, i.e., the receiver of a data flow 
     generally initiates and maintains the resource reservation used 
     for that flow.

   o RSVP maintains "soft" state in routing engines, providing graceful 
     support for dynamic membership changes and automatic adaptation to 
     routing changes. 

   o RSVP is not a routing protocol but depends upon existing and future 
     routing protocols. 

   o RSVP can transport and maintain traffic control and policy control 
     parameters that are opaque to RSVP. 

   o RSVP provides several reservation models or "styles" (defined below)
     to fit a variety of application needs. 

   o RSVP provides transparent operation through routers that do not 
     support it. 
   




   Internet-Draft	               draft-ip-optical-framework-00.txt



   How might RSVP fit into IP over Optical Networks?
   -------------------------------------------------

   References [5], [6] and [10] discuss how RSVP can be extended to 
   serve as a signaling protocol for the establishment of optical 
   channels and lightpaths. Furthermore, in the previous sections, 
   if the terms NHS is substituted with`RSVP capable router', 
   `NHRP Resolution Request' with `Path message', and `NHRP 
   Resolution Reply' with `ReSV message,' then RSVP can be used 
   to address the problem described in the NHRP example above with more 
   richly defined semantics for QoS.  
   
   
10. Security Considerations

   TBD.
   
   
11. References

   1. Bradner, S., "Key words for use in RFCs to Indicate Requirement 
      Levels", BCP 14, RFC 2119, March 1997. 
   
   2. D. Awduche, Y. Rekhter, J. Drake, R. Coltun, "Multi-Protocol 
      Lambda Switching: Combining MPLS Traffic Engineering Control 
      With Optical Crossconnects," Internet Draft, Work in Progress, 
      November 1999.
   
   3. D. Awduche, J. Malcolm, J. Agogbua, M. O'Dell, "Requirements 
      for Traffic Engineering over MPLS," RFC-2702, September, 1999.
   
   4. D. Awduche, "MPLS and Traffic Engineering in IP Networks," IEEE 
      Communications Magazine, December 1999.
   
   5. D. Awduche, D. Awduche, L. Berger, D. Gan, T. Li, G. Swallow, 
      and V. Srinivasan "Extensions to RSVP for LSP Tunnels," Internet 
      Draft, Work in Progress, September 1999.
   
   6. K. Kompella et al, "Extensions to IS-IS/OSPF and RSVP in support 
      of MPL(ambda)S," Internet Draft, Work in Progress, February 2000.
   
   7. D. Basak, D. Awduche, J. Drake, and Y. Rekhter, "Multi-protocol 
      Lambda Switching: Issues in Combining MPLS Traffic Engineering 
      Control With Optical Crossconnects," Internet Draft, Work in 
      Progress, February 2000.
   
   8. J. Luciani et al, "NBMA Next Hop Resolution Protocol (NHRP)," 
      RFC-2332, April 1998.
   
   9. B. Rajagopalan, D. Saha, B. Tang, and K. Bala, "Signaling Framework 
      for Automated Establishment and Restoration of Paths in Optical 
      Mesh Networks," draft-rstb-optical-signaling-framework-00.txt, 
      March, 2000.
   
  10. D. Saha, B. Rajagopalan and B. Tang, "RSVP Extensions for Signaling 
      Optical Paths", draft-saha-rsvp-optical-signaling-00.txt, March, 
      2000.


   Internet-Draft	               draft-ip-optical-framework-00.txt



   
   
12. Acknowledgments
   
   We would like to thank Zouheir Mansourati and Ian Duncan of Nortel 
   Networks for their comments on this draft.
   
   
13. Author's Addresses
   
   
      James V. Luciani
      TollBridge Technologies
      P,O. Box 1010
      Concord, MA 01742
      Email: james_luciani@mindspring.com

      Bala Rajagopalan
      Tellium, Inc. 
      2 Crescent Place 
      P.O. Box 901 
      Oceanport, NJ 07757-0901 
      Email: braja@tellium.com
   
      Daniel O. Awduche
      UUNET (MCI Worldcom)
      Loudoun County Parkway
      Ashburn, VA 20247
      Phone: 703-886-5277
      Email: awduche@uu.net
   
      Brad Cain, Bilel Jamoussi
      Nortel Networks
      600 Tech Park
      Billerica, MA 01821
      Phone: 978-288-4734
      Email: bcain,jamoussi@nortelnetworks.com
   
      
        ******** This draft expires on Sept., 10, 2000 ***********