Internet Draft Network Working Group Luc Ceuppens (Chromisys) Internet Draft Dan Blumenthal (Chromisys) Expiration Date: September 2000 John Drake (Chromisys) Jacek Chrostowski (Cisco Systems) W.L. Edwards (iLambda Networks) Performance Monitoring in Photonic Networks in support of MPL(ambda)S draft-ceuppens-mpls-optical-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract Realizing the important role that photonic switches can play in data-centric networks, work has been going on within the IETF to combine the control plane of MPLS (more specifically traffic engineering) with the point-and-click provisioning capabilities of photonic switches [1]. This document outlines a proposal to expand this initiative to include DWDM, OADM and ATM systems. It also proposes to expand the work beyond simple establishment of optical paths and include optical performance monitoring and management. The combined path routing and performance information that will be carried and shared between these network elements will allow the elements or element management system (EMS) to adequately assess the "health" of an optical path (which can be a wavelength or fiber strand). The routers and/or ATM switches at the edges of the photonic network will then use this information to dynamically manage the millions of wavelengths available in the photonic layer. Ceuppens/Blumenthal et al. [Page 1] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 2. Introduction Over the last two years, DWDM has proven to be a cost-effective means of increasing the bandwidth of installed fiber plant. While the technology originally only served to increase the size of the fiber spans, it is quickly becoming the foundation for networks that will offer customers a new class of high-bandwidth and broadband capabilities. Sales of DWDM systems will reach $6 billion in North America by the end of 2000. This roughly translates into tens of thousands of wavelengths deployed within optical networks, either as point-to- point connections or in ring topologies. In addition, several millions of wavelengths are projected to be deployed in enterprise, metropolitan, regional, and long haul networks by 2007 in the United States alone. These wavelengths will require routing, add/drop, and protection functions, which can only be achieved through the implementation of network-wide management and monitoring capabilities. Current- generation DWDM networks are monitored, managed and protected within the digital domain, using SONET and its associated support systems. However, to leverage the full potential of wavelength-based networking, the provisioning, switching, management and monitoring functions have to move from the digital to the optical domain. Efficiently managing (i.e., adding, dropping, routing, protecting, and restoring) the growing number of traffic-bearing wavelengths can only be achieved through a new breed of optical networking element. This network element is the photonic switch*. Photonic switching is the next logical step in a long history of switching technology that started with manual "plug board" operators, evolved to mechanical crossbar and finally digital switching. Photonic switching will enable transparent photonic networks. Photonic networks will greatly simplify the architecture of both the network and the network nodes by establishing end-to-end optical paths across the network. An end-to-end photonic path behaves as a transparent$ "clear channel", so that there is virtually nothing in the path to limit the throughput of the fibers. * Photonic switches are often referred to as optical cross-connects (OXC). However, today's OXCs are based on electrical rather than photonic switching fabrics, and therefore do not demonstrate the optical transparency required to grow photonic networks in the future. We use the term photonic switch to distinguish these classes. $ Transparency implies that signals with any type of modulation schemes (analog or digital), any bit rate, and any type of format can be superimposed and transmitted without interfering with one another, and without their information being modified within the network. Opaque networks do not have this property. Ceuppens/Blumenthal et al. [Page 2] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 A transparent channel essentially behaves like an ideal communications with almost no noise and very large bandwidth. Secondly, as the nodes in a photonic network have essentially no data processing to do, they can be made extremely simple and hence very cheap. Finally, optical node simplicity also means simplicity of control and management. Without any doubt, the next revolution in the telecommunications industry will occur within the optical domain. Now that the basic components are available to build photonic networks, the most important innovations will come from adding intelligence that enables the interworking of all the network elements (Routers, ATM switches, DWDM transmission systems and photonic switches). This new photonic internetwork will make it possible to provision high bandwidth in seconds, turning the new optical technology into a revenue spinner for the service provider rather than just a way of saving money. However, the intelligent open photonic network can only be built if the currently vertically layered network migrates to a horizontal model where all network elements work as peers to dynamically establish optical paths through the network. The IETF has already addressed the interworking of routers and optical switches through the MPL(amda)S initiative [1]. We propose to expand this initiative to include DWDM systems and ATM systems. We also propose to expand the work beyond simple establishment of optical paths and include optical performance monitoring and management. The combined information that will be carried and shared between these network elements will allow the elements or element management system (EMS) to adequately assess the "health" of an optical path (which can be a wavelength or fiber strand). The routers and/or ATM switches at the edges of the photonic network will then use this information to dynamically manage the millions of wavelengths available in the photonic layer. As a summary, the following functions need to be covered 1. Dynamic Bandwidth Provisioning 2. Optical Performance Monitoring 3. Signaling for 1 and 2. The remainder of this document discusses these functions into more detail. Ceuppens/Blumenthal et al. [Page 3] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 3. Dynamic bandwidth provisioning As indicated above, the photonic network uses photonic switches and optical transmission equipment to provide point-to-point connections to attached internetworking devices. These connections will typically take the shape of dedicated wavelengths, but can also be SONET leased line services or gigabit Ethernet connections. While the photonic network will typically provide these bandwidth services to IP routers, the model should be extended to include ATM switches. While the idea of bandwidth-on-demand is certainly not new, existing networks do not support instantaneous service provisioning. Current provisioning of bandwidth is painstakingly static. Activation of large pipes of bandwidth takes anything from weeks to months. The imminent introduction of photonic switches in the transport networks opens new perspectives. Combining the bandwidth provisioning capabilities of photonic switches with the traffic engineering capabilities of MPLS [2], will allow routers and ATM switches to request bandwidth where and when they need it. To make this work, however, requires more than simply advertising the availability of routes by the photonic switches to the routers and/or switches. They will also need to provide information about the characteristics and performance of the paths. Adequately assessing the status and health of an optical path through the photonic network requires a detailed cooperation between the photonic switches and the transmission systems providing the basic transport capabilities in the long-haul network. 4. Performance Monitoring Service providers to date have limited the role of DWDM in the network to creating "virtual fiber", i.e., the straightforward increase in capacity of the fiber plant, even if this meant a dramatic increase in complexity since each virtual fiber required the deployment of its own SONET equipment. The reason behind this restricted role is the worry about network management, alarm monitoring and protection capabilities of DWDM systems and the photonic layer in general. Current performance monitoring in optical networks requires termination of a channel (wavelength) at an OEO (optical-electrical- optical conversion) point to detect bits related to BER of the payload or frame (e.g., SONET LTE monitoring). For example, one form of error checking can be carried out at the SONET level by monitoring the B1 and J0 overhead bytes of the SONET stream. However, while these bits indicate if errors have been received, they do not supply channel-performance data. This makes it very difficult to assess the actual cause of the degraded performance. Ceuppens/Blumenthal et al. [Page 4] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 The premise of photonic networks requires the availability of tools to measure and control the smallest granular component of such networks --the wavelength channel. These functions include the monitoring of amplifiers and switches at add/drop sites, the deployment and commissioning of DWDM routes, as well as the restoration and protection of networks. This must be accomplished with speed and accuracy over an extended period of time. Fast and accurate determination of the various performance measures of a wavelength channel implies that measurements have to be done while leaving it in optical format. In the remainder of this document we will refer to this as "optical performance monitoring" (OPM). One possible way of achieving this is by tapping a portion of the optical power from the main channel using a low loss tap of about 1%. In this scenario, the most basic form of OPM will utilize a power-averaging receiver to detect loss of signal (LOS) at the optical power tap point. Existing DWDM systems use OTDRs (Optical time-domain reflectometers) to measure the parameters of the optical links. As photonic networks mature, it will be desirable to generate a more detailed picture of the channel "health" in a manner that can be communicated to the EMS and other network control entities, as well as between other network elements. By monitoring various OPM parameters, one can attempt to estimate the BER, detect gradual or sudden performance degradation, and report these to local or global NMS entities, and to attached internetworking devices. Also, fiber spans are typically characterized, or calibrated, during the provisioning process on DWDM systems, as fiber manufacturer, fiber type etc. all have a bearing on how the various DWDM spectrums should be populated. It would be useful to have the calibrated data for each fiber span available as part of the overall information on the photonic layer. All the available information can then be correlated across the network to make decisions on fault isolation and take appropriate actions such as rerouting the connection or adaptively downgrading or upgrading the bit-rate of a channel. When deploying an optical network it is common practice to document the baseline for all operating parameters, such as signal power, bit-error rate, OSNR, etc., prior to network turn-on. During normal operation, network elements equipped with OPM capabilities can report any degradation events of the optical channel to the network operations center (NOC) and to the other network elements. The element management system (EMS) can document the degradation of the photonic layer in time by saving optical performance monitoring data in an archival database. As channels are added, removed or rerouted, the NOC can continuously monitor and analyze the status as channels are dynamically managed. With the advent of an open photonic network, we can envision a trend of leasing channels or wavelengths that span multiple networks. This will require optical interconnects between various networks. Ceuppens/Blumenthal et al. [Page 5] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 Invariably, as channels are handed off between carriers, problems can occur which require monitoring to resolve conflicts. Most of these issues occur at network boundaries. In addition, if service providers offer various levels of QoS, both networks will have a way of negotiating the end-to-end QoS of the channels and both service providers will need a way to ensure that the other party lives up to the service contract. Here again, independent monitoring is needed to ensure quality and continuity of service. The issue of effective OPM sensitivity will impact how pervasive each technique is used in a network due to cost and complexity. Certain techniques may require an optical amplifier at the tap point resulting in OPM module sensitivity equivalent to that of the final path termination point. Other issues that need to be addressed include definition of OPM at the section, line and path levels. Since monitoring can be in principal performed at any point within the network, traditional use of LTE points does not carry over. Another problem related to transparency lies in determining the threshold values for the various parameters at which alarms must be declared. Very often these values depend on the bit rate on the channel and should ideally be set depending on the bit rate. However, in a truly transparent network, one may have to set alarms to correspond to the highest possible bit rate that can be present on a channel. In addition, since a signal is not terminated at an intermediate node, if a wavelength fails, all nodes along the path downstream of the failed wavelength could trigger an alarm. This can lead to a large number of alarms for a single failure, and makes it somewhat more complicated to determine the cause of the alarm (alarm correlation). We see as potential candidates, the following OPM functions: 1. Dispersion (chromatic and polarization mode): The distortion or spreading of bits due to variations in propagation velocity of different wavelengths and polarization modes in the fiber and other network elements. 2. Optical signal-to-noise ratio (OSNR): The ratio of optical power in a primary data channel to the power in optical background noise accumulated during transmission and switching. This ratio is usually specified within some optical bandwidth of a receiver filter. The OSNR of a channel at the destination receiver will set the limit of the final detected SNR. 3. Bit-rate The data rate of the channel in a transparent system will be necessary to make decisions on other performance metrics. 4. Q-Factor A measure of the signal-to-noise ratio (SNR) assuming Gaussian noise statistics. Ceuppens/Blumenthal et al. [Page 6] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 5. Wavelength registration The determination of which wavelengths are present on a given fiber. 6. Wavelength selective component drift The drift of a laser, filter, mux or other wavelength selective component relative to the ITU grid. 7. Optical cross talk Two types of cross talk are of interest, in-band and out-of-band. In-band cross talk is seen as at the same wavelength as the primary channel and appears as cross talk in the electronic domain. Out-of-band cross talk appears as a different wavelength in the presence of the primary wavelength and appears as cross talk in the optical domain. 8. Optical power transients Changes in the optical powers that are not due to normal bit transitions. May be due to optical amplifier gain transients or other transient non-linearity in the system. 9. Bit-error-rate In a SONET environment BER can be directly measured on the channel using means to look at bits within the data stream. However, in a purely photonic network there will typically not be access to the data streams carried over the channel. However, by interpreting the other optical parameters, the system should be able to estimate the BER with relatively good accuracy, as well as guarantee bit error rate performance to the users of the channel. 10.Jitter Random fluctuations in the location of rising and falling edges of bits relative to a local or recovered clock reference. As line speeds continue to increase, jitter will become a critical performance parameter. 11.Insertion loss Indicates the input to output loss of a network element. When examining excessive power loss along the path of a channel the ability to measure insertion loss of individual network elements is very useful, specifically when compared against an archival database. 12.Optical power level In addition to verifying the service level provided by the network to the user, performance monitoring is also necessary to ensure that the users of the network comply with the requirements that were negotiated between them and the network operator. For example, one function may be to monitor the wavelength and power levels of Ceuppens/Blumenthal et al. [Page 7] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 signals being input to the network to ensure that they meet the requirements imposed by the network. To make any Performance Measurement metrics meaningful, major effort should be on conducting serious testing to draw correlation between the proposed Optical measurement metrics with the quality of the signals (electrical). 5. Signaling The vast majority of existing communications networks uses framing and data formatting overhead as the means to communicate between network elements and management systems. It is clear however, that truly transparent and open photonic networks can only be built with transparent signaling support. Arguments in favor of transparency include, but are certainly not limited to: - Framing and formatting makes the network opaque and as such inhibits the creation of bit rate and protocol transparent networks. As overhead information is processed in the digital domain, it requires an optical-to-electrical and electrical-to- optical conversion at every point in the network where traffic is inserted or dropped and at each point where management and monitoring is required. This imposes severe limitations and is probably the single biggest inhibitor of growth in current optical networks. That is why "digital wrappers" are not a viable solution. In fact, an all-optical network using digital wrappers is a contradiction in terms! - Attached internetworking equipment and customer equipment may not support the framing overheads. - In today's optical network (I.e., SONET) the service and infrastructure layer are inseparable. As a result, "optical- network-ignorant" protocols such as (ten) gigabit Ethernet, fiber channel or ESCON cannot be transported without being translated to the infrastructure layer. Hence the need for adaptations such as "gigabit Ethernet over SONET", "packet over SONET" etc. In contrast, separate control plane techniques# supports flexible control and management of multi-vendor networks and will pave the way for truly open and transparent photonic networks. # There may be instances where some "embedded" wavelength routing information is required. One such instance is in existing networks where DWDM junctions are "hard-wired" and the end-to-end path may consist of different wavelengths. Ceuppens/Blumenthal et al. [Page 8] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 As already mentioned, the approach proposed by the MPLS/TE [1] task force of IETF propose to combine recent advances in MPLS traffic engineering control plane with emerging photonic switching technology to provide a framework for real-time provisioning of optical channels and allow the use of uniform semantics for network management operations control in hybrid networks consisting of photonic switches and label switching routers (LSRs). While the proposed approach is particularly advantageous for data-centric optical internetworking systems, it can easily be expanded to include basic transmission services. Similarly, it can be expanded beyond simple bandwidth provisioning to include optical performance monitoring. It is worth mentioning that while the signaling is used to communicate all monitoring results, the monitoring itself is done on the actual data channel, or some range of bandwidth around the channel. Therefore, all network elements must be guaranteed to pass this bandwidth in order for monitoring to happen at any point in the network. Several signaling flows have to be supported: - between the internetworking equipment and the photonic cross- connect - between the photonic cross-connect and the DWDM transmission systems - between the DWDM systems and optical amplifiers - between the DWDM systems and optical add/drop multiplexers - between the internetworking devices and optical add/drop multiplexers or DWDM transmission systems (if this connection does not run through a PXC) We propose that the connection signaling be limited to exchanges between the internetworking device and the transmission network element it is directly connected to. This transmission element (e.g., a photonic cross-connect) then interfaces with the DWDM systems (if present) and so forth. This allows for the photonic switches to discover the transmission network topology and characteristics prior to attached devices asking for connections. It also caters for the continued support of any proprietary signaling that may already exist between DWDM and/or other transmission systems (whether in-band or out-of-band). All that is required is support of the standard external signaling interface. We also propose these signaling flows be supported on a dedicated wavelength, configured throughout the network. Whether this wavelength is part of the standard ITU grid or not, is beyond the scope of this document. We recommend however, that the signaling wavelength be a standard ITU channel, considering that the combination of existing C-band (1530- to 1560-nm) and the emerging S- (upper 1400-nm region) and L- (1570- to 1625-nm) transmission bands will leave little room for suitable non-ITU wavelengths. Ceuppens/Blumenthal et al. [Page 9] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 Since dedicating an entire wavelength might not always be viable, we should envision the possibility of using this wavelength also for data traffic and envisage a way of sending the non-time-critical traffic in between the management traffic. The signaling protocol can easily be based on existing protocols. A slightly modified OSPF can be used for optical network topology discovery and distribution, as well as for route computation and path selection. Topology advertisement includes not only the nodes and the links to the nodes, but also characteristics of the links. The actual signaling protocol can be RSVP as extended for MPLS/TE. Finally, path management includes monitoring the path for failures, knowledge of failure restoration policies, and path teardown. 6. Summary This document outlined a proposal to expand Multi-Protocol Lambda Switching in two areas: - Include network elements such as DWDM, OADM and ATM switches to create a versatile transparent and open photonic network. - Expand the work beyond basic connection establishment and include performance-monitoring capabilities Work in this area should be closely coordinated with activities in the T1 committee, ITU and OIF to ensure a consistent industry-wide solution. 8. Security Considerations This document raises no new security issues. 9. References [1] Awduche, D., Y. Rekhter, J. Drake and R. Coltun, "Multi-Protocol Lambda Switching: Combining MPLS Traffic Engineering Control With Optical Crossconnects", work in progress, November 1999. [2] Awduche, D., J. Malcolm, J. Agogbua, M. O'Dell and J. McManus, "Requirements for Traffic Engineering Over MPLS", RFC 2702, September 1999. 10.Acknowlegdements We would like to thank Curtis Brownmiller (MCI WorldCom) for his review and comments. Ceuppens/Blumenthal et al. [Page 10] Internet Draft draft-ceuppens-mpls-optical-00.txt March 2000 11.Author's Addresses Luc Ceuppens W.L. Edwards Chromisys iLambda Networks 1012 Stewart Drive Aspen, CO Sunnyvale, CA 94086 970.948.7104 Email: lceuppens@Chromisys.com Email: texas@ilambda.com Dan Blumenthal Jacek Chrostowski Chromisys Cisco Systems 421 Pine Avenue 365 March Rd Goleta, CA 93117 Canata, Ontario K2K2C9 Email: danb@Chromisys.com Email: jchrosto@cisco.com John Drake Chromisys 1012 Stewart Drive Sunnyvale, CA 94086 Email: jdrake@Chromisys.com Ceuppens/Blumenthal et al. [Page 11]