Internet Draft MPLS Working Group Muckai Girish Internet Draft SBC Technology Resources, Inc. Rayadurgam Ravikanth Document: draft-vaananen-mpls-svc- Nokia Research Center diff-framework-00.txt Expiration Date: September 2000 Pasi Vaananen Nokia Telecommunications March 2000 A Framework for Service Differentiation in MPLS Networks Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract It has been recognized that the success of the MPLS depends on the ability to better support the multiservice traffic integration with some levels of service guarantees, which are not feasible to implement with the current destination prefix only based packet forwarding paradigms. The efficient support for these services throughout the network is expected to be possible using label based forwarding paradigm in the network. Through the use of either RSVP based or LDP/CR-LDP based signaling, MPLS can also provide certain QoS guarantees using the LSPs. The goal of this document is to define a framework for service differentiation in MPLS networks. We discuss a set of services that have been identified so far for IP, and describe the traffic management mechanisms in various network elements that are needed for enabling the implementation of these more advanced services in MPLS networks. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 1 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 This document describes the mechanisms and their applications with the intent to approach the level of the traffic management capabilities that are currently available in hybrid router/ATM or frame relay networks using the MPLS. This document concentrates on the issues from the public network operators point of view, although most of the discussion applies as well in the local network environments. Concepts and mechanisms described in this document are based on the previous work done in various working groups of IETF and other standardization bodies. Applicable concepts and terminology from previous work has been used as much as possible. This document concentrates on the MPLS specific issues, number of related mechanisms and concepts are only briefly presented for the sake of completeness, and the other related work is referred, where applicable. 2. Introduction The ability of IP networks to support service level differentiation and traffic engineering is becoming very important. This area has been addressed in various working groups of IETF (e.g. INTSERV, RSVP, ISSLL, RAP, DIFFSERV, IPPM, QOSR, TEWG), IRTF (E2E), ATM Forum (TM), Frame Relay Forum, ITU-T, and various other organizations and user consortiums. We build on the ideas and previous work done in these working groups, and try to construct a coherent set of capabilities around the label based packet forwarding technology discussed in the MPLS working group of IETF, as described in the MPLS framework document [Callon99] and the MPLS architecture document [Rosen99a]. The starting point is to identify a set of possible services that are implementable using current IP standards which will also cater to the various needs of customers and service providers in IP networking. We then move on to the focus of this draft which is to describe the set of traffic management functions and elements both in the control plane and the data plane that are needed for service differentiation. The TM functions done by the various network elements such as hosts, CPE devices, edge routers and core routers are then presented. Finally, the TM functions that are mandatory and optional for providing the services are listed for each of the network elements. The main purpose of this draft is to explicitly specify how the various technologies fit together in creating a multi-service IP network. In a sense this draft describes, in generality, the kind of functional blocks, in addition to the standard protocols, that may need to be present in MPLS capable nodes. In presenting a consolidated view of how service differentiation may be achieved, this document points to how varying technologies developed in possibly different groups in IETF are tied together. Further, such a M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 2 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 view also strives to identify work items which may have implications in various IETF groups. It should be emphasized, however, that this draft only identifies rather generic functional blocks that would be needed, and it is fully recognized that actual implementations may vary depending on the functions that the node aims to support. In the light of this, the objective of this paper is simply informational. The document tries to take an evolutionary rather than revolutionary approach. We feel that the deployment of the technologies presented can be started on a small scale, and without changes to the host communication and application protocols, while this framework attempts to be flexible enough to be able to accommodate such changes when the technology matures and the incremental deployment is determined to be feasible and necessary. We hope that MPLS will evolve towards supporting the capabilities outlined in this document, but do realize that much more detailed discussions, research and specification work needs to be done before the complete set of "wishes" can be accomplished. 3. Service Differentiation The advanced services requiring the use of the traffic management mechanisms can be broadly divided into two categories on the basis of (i) the level of assurance on service guarantees that can be achieved and (ii) the granularity of guarantees (simple to complex) that is provided. This division is made here to support the discussion of the related traffic management issues. MPLS will be used to provide the services that are being defined in the IETF, such as those based on Differentiated Services and Integrated Services. MPLS label switched paths can be used to construct aggregate paths, with the result that less state needs to be maintained. This document primarily deals with the components of traffic management that will be necessary to support the various services, and the issues discussed here are generic enough to apply to the various service categories that MPLS will need to support. The basic service categories differ from each other on basis of the end-to-end assurance with respect to the certain performance metrics, such as packet loss, delay, and delay variation these services expect from the network. The implementation of the more advanced service categories than pure best-effort affects the implementation of both data and control plane functionality in intermediate nodes. Some of the affected datapath functions are congestion control, queuing, packet classification, policing, scheduling, shaping and service mapping to interfaces. The associated control plane functions that are affected M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 3 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 include signaling protocols, accounting, policy control and routing protocols. As a starting point we describe a set of services that can be supported in IP networks, using already defined mechanisms such as IntServ, DiffServ, congestion control etc. In later sections we will define the set of traffic management mechanisms that will be necessary/useful to support this service differentiation. 3.1 Differentiated and Integrated Services 3.1.1 Differentiated services Packet forwarding and queuing treatments for differentiated services are specified in the IETF DiffServ working group. The DiffServ working group does not standardize services, instead, it standardizes a small number of packet treatment mechanisms or PHBs (Per Hop Behaviors). End-to-end services can be constructed from these PHBs with appropriate traffic conditioning actions. The differentiated services architecture is documented in [RFC2475], while the differentiated services framework is documented in [Bernet99a]. The expedited forwarding (EF) PHB is proposed in [RFC2598], whereas the assured forwarding (AF) PHB group is presented in [RFC2597]. 3.1.1.1 Differentiated services in MPLS environments Generally no per LSP state need to be maintained in the network elements and the goal is to support a small, fixed number of service categories. It is expected that the state maintenance for the scoped services (defined later in this section) can be done in behavioral aggregate basis, although the parameters have to be specified on per-LSP basis, as well as admission control needs to be done for individual LSPs. Measurement based admission control may help in achievement of better utilization of the resources (subject to verification by simulation). Per stream attributes distributed using the label distribution mechanisms can include the differentiated service categories associated with the LSP. The mappings from Differentiated Service classes to MPLS paths are specified in [Francois99]. The support of differentiated services in MPLS environments requires signaling support for the association of the desired category with the label, or alternatively each packet needs to carry the information of the desired service category. MPLS allows the allocation of the bandwidth for the differential services in conjunction with other services in a controlled manner. This allows the network operator to allocate the available bandwidth between the differentiated service category and other categories, on a per LSP basis, providing good basic mechanisms required by the efficient network traffic engineering. 3.1.2 Integrated Services M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 4 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Several scalability issues related to RSVP signaling have been identified that suggest that the support of the Integrated services in the backbone networks is not feasible at reasonable cost. Therefore, there are efforts ongoing in IETF for mapping these services to simpler mechanisms, i.e. DiffServ and/or MPLS on the backbone level (see [Bernet99b]). The model that is seen as enabling the provisioning of these services on the backbone level are based on the running full RSVP/IntServ in the stub networks and mapping the packets to simpler mechanisms in the border nodes of the public IP network. Services are specified in IntServ working group, while associated signaling mechanisms are specified in the RSVP working group. DiffServ and ISSLL working group are currently working on the service mappings, with most of the data plane work completed, biggest open issue is currently how to do admission control for the DiffServ capable backbone network. 3.1.3 Scoped (and guaranteed) services These services provide hard guarantees that are explicitly specified for different granularities, and specific topological scopes, such as from network boundary to network boundary or end-to-end. Services are further specified using different, service specific set of parameters, such as bandwidth and/or delay, determined by the requested service class. The scoped / guaranteed services may be based on the contractual guarantees or user-network signaling, such as RSVP. Signaling protocol to disseminate associated service parameter information is required inside network. In IETF, guaranteed services have been specified by INTSERV and MPLS working groups. Integrated service framework is described in [RFC1633]. There are currently two services that have been defined by INTSERV; controlled load [RFC2211] and guaranteed service [RFC2212]. These services should be supported in MPLS environments. Service parameter mappings to different link layers specified in the ISSLL working groups should be applicable to MPLS, when augmented with the label encapsulation procedures specified in the MPLS WG. 3.2 A Framework for Services We specify a framework for services here and all of the proposed services can be specified with appropriate attributes of this framework. The framework is described by two components: the traffic contract and the service objectives. The traffic contract describes the packet arrival patterns that the customer has contractually agreed to adhere to which is enforced by the service provider. The service objectives can be described by a combination of throughput, loss, delay and jitter parameters. Note that all parameters are not M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 5 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 necessarily specified for all the services that are described in the ensuing sections. 3.3 Service Scoping The level of assurance that can be provided using different mechanisms depends on the scope of the network region where the service is provided. Smallest useful service scope is likely a single subnet, and the largest scope is the whole Internet. Clearly, as the scope is extended, the provisioning of the service and achievable service level guarantees gets more complicated. Some examples of the different service scopes are: - Internet - Autonomous System - Routing domain - Within subnetwork - Anywhere --> destination network - Source Network --> destination network - Boundary router --> boundary router - Source host --> destination host The above list provides only some possibilities, and the scoping is clearly not subject to standardization, and will be performed as a network / service engineering task by the network operators. There have been proposals for very high assurance levels, that generally require communication of some attributes describing the traffic characteristics, and also definition of some quite limited scope (such as between two specific operator boundary router interfaces). The attributes required by the network for provisioning such services can be statically allocated (i.e. configured in NE:s) or dynamically requested and allocated using some form of signaling mechanism, such as RSVP and/or MPLS signaling. It should be noted that even in situations where the exact traffic characteristics and scope are known only at very coarse level (such as maximum profile of traffic in behavioral aggregate and inside ISP network), operators may want to provision certain service quality level for traffic on certain behavioral aggregate, as agreed to in service level agreements. This practice is analogous to SLAs in which some ISPs currently offer some delay bounds and loss ratios for best effort traffic. However the purpose of service differentiation is to provide the ability to independently specify different parameters for different service categories. Such guarantees need to be addressed using network engineering practices, together with monitoring and traffic engineering. Below, we describe some examples of IP services that can be constructed using available IP technologies. 3.4 Examples of IP services M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 6 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 3.4.1 Best effort service Best effort service is the prominent service used in the current Internet. The characteristics of the service can be deduced from the name, as there is generally no guarantees for any of the associated performance metrics. Best effort service is an unscoped service, since it is not specified with respect to any particular destination. 3.4.2 Enhanced best effort service (EBE) This service is similar to the current best effort service, but with higher service quality perceived by the end-user, regardless of the applications. Enhanced best effort service can be realized without using any specific signaling protocols inside the network. It differs from "plain old best effort" because of the use of more advanced congestion control mechanisms/traffic engineering. The purpose is to provide more controlled and more fair behavior during congestion periods. 3.4.2.1 EBE with RED One of the proposed enhancements required for the better end-to-end operation over "traditional" best effort service include passive congestion control mechanisms based on packet drop policies, such as random early detection (RED) [Floyd93], [RFC2309], as well as fair and enhanced queuing schemes. The use of RED does not require the maintenance of per-flow state in the network elements, and still can provide better goodput for TCP-like, adaptive applications as well as somewhat improved fairness. 3.4.2.2 EBE with ECN In addition to passive congestion control mechanisms, active congestion control mechanisms, such as Explicit Congestion Notification [RFC2481]. ECN is based on explicit congestion feedback from network elements to transport protocol (TCP) in hosts. Extensions for supporting ECN in MPLS networks are specified in [Ramakr99]. 3.4.2.3 EBE with Traffic Engineering The use of label based forwarding paradigm (MPLS) adds capabilities for network operator traffic engineering, such as better ways to control the path selection for the traffic. This can significantly improve the performance seen by the best effort traffic. 3.4.3 Bounded loss service (BLS) This service guarantees the delivery with a certain bounded loss for traffic that conforms to a specified rate. While bounded loss simply M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 7 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 refers to the service objective (as defined above), this really represents three different services when the traffic contract enforcement is also considered. - Three color marker based service [RFC 2697, RFC 2698]: Here the customerÆs traffic is marked with one of three colors (using a dual token bucket scheme) and the packets marked with each color are guaranteed a certain maximum loss probability. Practically this could mean that packets conforming to committed rate specification have better delivery guarantee than those that exceed it. - Frame relay service: Here the packets are once again given loss probability bounds depending on whether the average (based on some sliding window or jumping window scheme) arrival rate conforms to agreed upon thresholds. - Customer marking based service: Here the customer might mark packets with different colors (representing different loss guarantees) and as long as the operator can verify (and enforce) the conformance of the markings to the traffic contract, these loss rates are guaranteed. 3.4.4 Virtual leased line service Virtual leased line service is specified to provide service resembling the service experienced by the leased, point-to-point line. VLL service does not have explicit delay or jitter guarantees, but both are expected to be "low". To work as desired, VLL service must be scoped, and strictly policed (all non-compliant traffic is discarded) on the edges of the network. The only service parameter associated with the virtual leased line service is peak bandwidth. An example of supporting virtual leased line service is described in appendix A of [RFC2598] with the EF PHB and appropriate traffic conditioning actions. 3.4.5 Guaranteed service Guaranteed service provides deterministic guarantees for both bandwidth, and upper bound on per packet delay. Guaranteed service is best suited for applications that require absolute delay bounds, and cannot adapt to network conditions. Guaranteed service is defined in [RFC2212]. 3.4.6 Controlled load service Controlled load service provides the service of the network that closely approximates the service user would receive from unloaded network. Controlled load service guarantees that bandwidth is allocated to the user, but does not guarantee the maximum delay bound experienced by packets. Controlled load service is defined in [RFC2211]. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 8 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 3.5 Mapping user traffic to service categories From the end-user/application perspective, the traffic can be classified on basis of the various performance parameters, including: - Loss sensitivity (loss tolerant = adaptive, loss sensitive = not adaptive) - Delay sensitivity - Desired assurance of the service level In addition to the above, the level of service guarantees that can be expected from the network is dependent on the scope of the traffic (i.e. are endpoints of the connection known beforehand, and resources provisioned in advance for the stated scope using either signaling or provisioning). The end-to-end service should be dependent on the combination of the above factors, and the mapping to underlying mechanisms should be based on the user expectations. Note that the loss and delay sensitivity are not binary parameters, various levels of service with respect to these parameters are expected by different applications. Figure 3.2.4.-1 Example classification of the user traffic In addition to the scope of the service, one important aspect that further characterizes the type of service desired by the application is whether application is able to measure and adjust to network capabilities (i.e. adaptive applications), or if it requires some minimum level of performance with respect to loss, delay, or other metrics (i.e. non-adaptive applications). Adaptive applications use feedback loop to achieve the adaptation between traffic source and sink, while non-adaptive applications do not use feedback loop. Typical example of adaptive traffic is TCP traffic, while UDP traffic (e.g. voice over IP mapped on UDP/RTP) is typically non- adaptive. Using the information provided by the users and applications, the end-user traffic can be classified as depicted in the figure 3.2.4.- 1. The required traffic management mechanisms for the support of the fulfillment of the stated expectations are described in sections 8 and 9. Figure 3.2.4.-2 Example mapping from the classes to IntServ/DiffServ categories Figure 3.2.4.-2 depicts the example of how the traffic can be classified to different service categories. Note that the other mappings are also possible, and the actual mapping will be controlled by the network operator service class configuration and/or router that performs the packet classification process. However, the above picture is useful in understanding the congestion M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 9 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 control and policing actions that need to be applied on each traffic class. Since the DiffServ working group is not defining the services, it is important that the implementation is flexible with respect to configuration of the policing actions, control mechanisms associated with different classes and mapping and scheduling mechanisms. Official work to specify standard mappings have not been started in IETF, but there is expectation that most (if not all) services can be supported by the DiffServ categories specified so far, when augmented by the appropriate signaling and/or traffic conditioning mechanisms. 3.5.1 Summary of service scoping Table 3.2.4. presents the summary of the scoping requirements for the different service types, in the order of the level of guarantee expected. Generally, the more guarantees or assurances of the service characteristics are specified, more tight scoping is required for achieving the service level objectives. Note that the scoping here generally means that service provisioning needs to know both the origination and end points (or set of those) to be able to provision service. Service: Scoped: Unscoped: Best Effort - X - DE Control traffic - X - PR 6,7 [Precedence (X) - PR ? X - PR ?] *** [Assured forwarding (X) - AF1,2 X - AF 3,4] Controlled Load X - AF1 - Virtual leased line X - EF (/ AF1?) - Guaranteed X - EF - where, - = Not applicable, X=default, (X) = optional ***) The precedence classes have been originally defined to be unscoped, but overloading of codepoints by AF service may change this for certain classes. All scoped services require the communication of the associated service parameters (see service definitions above) across the network elements in the path, as well as policing in the edges. Unscoped services require either no policy enforcement or policy enforcement on the basis of service level agreements (SLA) on border network elements. 3.6 Service level agreements Service level agreements are the contracts between users and network operators (or between network operators) that define the service(s) provided, pricing models and tariffs, service class parameters, service scoping, penalties, etc. Service level agreements ultimately M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 10 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 completely define the service characteristics and finally yield to requirements for the operator and customer equipment related to service provisioning and enforcement. Since the service level agreements are operator specific, and not subject to standardization (at least not in the IETF), it is important that the network elements and supporting management and provisioning systems provide a rich set of highly configurable mechanisms to allow operators to build the desired service. Mechanisms that are involved in the construction of service are related to: - Service provisioning, management and monitoring - Service contract enforcement - Service monitoring - Accounting and billing All of the above high level requirements result in the need for specific mechanisms and protocols that need to be supported in network elements. These mechanisms are discussed in more detail in chapter 5 for data plane mechanisms, and chapter 4 for control plane mechanisms. Service level agreements and traffic conditioning agreements for the DiffServ networks are discussed in DiffServ framework document, [Bernet99a]. 3.7 Virtual private networks Virtual private networks are services designed to provide traffic isolation (in the sense of the overlapping / private address spaces and/or isolation of the traffic to own virtual connections) and/or service level guarantees for the traffic controlled by the VPN customer. VPNs can use any of the connection paradigms provided by the IP network or alternatively by virtual connection type network architectures, such as the mechanisms provided by MPLS. VPNs generally do not impose additional requirements for the traffic management functions, except those that are already described in this document, but can be considered specific type of application of these mechanisms. The mechanisms described in this document allow (but are not limited to) the construction of the following VPN types: Tunneled best effort, unscoped: this type of VPNs are "traditional" VPNs in the sense that the network service provider need not to be aware of the VPN at all. The disadvantage is that no service level guarantees can be provisioned for the VPN. Tunneled differentiated, unscoped: this type of VPN can also be constructed without specific knowledge by the service provider about VPN use per se. VPN customers can use any of the differentiated service categories which are covered by the SLA/TCA between the user and the provider. Marking can be performed by customer by marking the DSCP field of the VPN packets, or alternatively by the service provider by applying suitable traffic filter in the border nodes to M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 11 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 allow differentiated treatment. In the latter case, the filter parameters need to be covered by the SLA. Tunneled differentiated, scoped: this type of VPN is otherwise same as tunneled differentiated unscoped, except that the VPN service endpoints are part of the SLA in addition to the parameters specified above. Virtual circuit tunneled: these VPNs are always scoped, as the tunnel endpoints need to be specified in every case. Tunnels can be associated with any of the supported traffic class using a desired set of parameters. Traffic can be associated with the tunnel either by customer (requires tunneling to CPE equipment) or by operator by applying traffic filter. Tunnel topologies supported can include point-to-point and multipoint to point. Virtual circuit tunneled VPN is conceptually similar to the traditional Frame Relay or ATM VPN service, with the exception that the control protocols and the traffic classes are specified in IETF. Virtual Private Routed Networks (VPRNs): this is a specific case of the VPN mainly applicable to large VPN applications. The difference between this and other approaches is that the tunnels and the associated topology is constrained and built automatically by the network operator, using a specific VPN identifier as an additional attribute for scoping and distributing topology information of the VPRN service. Tunnels are used to isolate traffic from different VPRNs, while the connectivity information is distributed typically as an attribute to routing information distribution. VPRN service effectively builds routing tables in intermediate nodes for each VPRN, and uses the tunnel identifies to determine the context (i.e. to select the appropriate forwarding table) of the packets belonging to different VPRNs. The virtual circuit tunneled and VPRN type VPN:s require the use of the separate logical connections for the traffic isolation and/or guarantees. Other types typically encapsulate VPN packets to some form of the traditional IP tunnels (e.g. IPIP, IPSEC or LSTP) to achieve the traffic isolation. VPNs in general are discussed in [Ferguson98], and various methods for constructing MPLS based VPNs in particular are discussed in [CIMI99], [RFC2430] and [Gleason99]. 3.8 MPLS to customer An Operator can run MPLS towards the customer premises, but there are some important considerations that need to be taken in the account in such environments. Since the customer is a non-trusted entity from the operatorÆs standpoint, and the MPLS allows the establishment of the switched paths towards the destination, there is no possible way for the M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 12 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 operator to control what enters into the LSP that the subscriberÆs traffic enters into. This opens the possibility of denial of service attacks, and other kinds of malicious uses that could possibly be prevented by the ingress filtering on the operatorÆs ingress node. When the traffic enters the LSP, it is impossible to determine where the traffic originated from after it is merged with the other traffic, assuming that the bogus source addresses are used. The only way to prevent this would be to terminate the LSPs originated from customer premises on the operatorÆs border node, but in such case there is no reason to run MPLS to the customer for this type of traffic at all. Additionally, as the customer does not have the information of the operatorÆs traffic aggregation policies and access to the routing information, customer will not be able to perform traffic aggregation. This would, in practice, mean that the MPLS sessions between operator and subscriber would have to be based on individual flows, and operator would be responsible for appropriate aggregation. An environment, where the use of the MPLS to customer premises makes sense is when the MPLS is used to create VPNs for the customer. The customer could then assign the traffic that is destined on the LSP thatÆs part of the VPN to appropriate VPN. Even in these environments, it would make sense to use ordinary routing for other traffic. This assumes that the VPN LSP endpoint(s) trusts the sending entity to some extent, as the traffic would be carried quite transparently through the operatorÆs network. In any case, all traffic that is entering onto operatorÆs network that is destined to the public network should be validated for the source address before encapsulating to any label switched path. For the above reasons, MPLS to the customerÆs premises does not appear to be very useful. 4. Control Plane Mechanisms for Service Differentiation Functions 4.1 Introduction This chapter describes the mechanisms required in the various parts of the network to control the data plane traffic management functions described in the next chapter. These mechanisms include policy and signaling aspects required to set up, and to maintain the LSPs. Note that the location of these mechanisms in the networks is not discussed in this chapter, a discussion of the location of mechanisms in different network environments is given in Section 7. 4.2 Triggers M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 13 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Triggers are events that cause the changes in the LSP configuration. These changes may be LSP establishment, reconfiguration, deletion or attribute modification. The triggers either require going through full or partial LSP establishment process depending on the type of the trigger T riggers can broadly be classified as falling under one of the two following categories. - configuration event - signaling event Either of these two events can in turn arise due to a variety of causes such as - Topology change: Topology changes are events that are associated with the changes in network topology, and may potentially result in the requirement for large number of reconfiguration of a large number of LSPs. Topology changes are brought to the attention of the label distribution subsystem by the routing protocols and the monitoring of the status of the established LSPs. - Traffic pattern change: Traffic pattern change is an event triggered by the user activity that is observed by the network element resulting in the change of the traffic characteristics received over interface. Examples include the appearance of the new traffic flow, or timeout of the existing flow. These changes may affect how the LSPs are set up or attributes of the LSPs. Traffic pattern related changes should be attempted to be kept as local. - Explicit request for establishment/teardown/modification of LSP The scope of the change initiated by trigger can be either local (i.e. inside of the network element), regional (i.e. affects the configuration of the peer MPLS nodes) or global (i.e. affects all network elements that compose the LSP). The frequency of the regional and global changes should be minimized. As the finer granularity of control of the LSP attributes is required (e.g. explicit reservations), this becomes increasingly hard to achieve. Properties associated with different kinds of triggers are discussed in sections 5.1.1 to 5.1.3. 4.2.1 Configuration events A configuration event can affect either policy or LSP configuration parameters. Policy changes affect the admission or classification policy being used in the node. LSP parameter change affects the attributes associated with the statically configured label switched path. The policy related changes can either force the re-evaluation of the current classifications or be taken into use gradually, as new paths are used. Although the immediate re-evaluation would be desirable, it may have negative effects on the performance and the handling of the current traffic. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 14 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Parameter changes may require the communication of the change to peer LSRs that compose the LSP (signaling initiated by the ærootÆ node of the LSP), or be configured onto each LSR along the path individually (management initiated change). In either case, these changes should be taken into effect immediately. 4.2.2 Signaling events Signaling event is an externally received trigger that explicitly affects the way LSPs are set, and depending on the signaling event type, may result either in setting-up, tearing down or modifying attributes of the LSPs using an available signaling protocol such as RSVP or LDP or CR-LDP (see [Andersson99], [Awduche99] and [Jamoussi99]). 4.3 Policy and admission control Policies and admission control form a set of processes that directly or indirectly control the set-up of the label switched paths through the network element. 4.3.1 Routing policy For the new LSP requests, routing policies applied on the Internetwork are the first controlling policy that controls the potential routes the LSP can take through the network. Routing policies are thus not directly involved in the topological control of the LSP establishments, but they control the information in the routing information base that the MPLS signaling protocols use to determine available routes. It should be noted that the current routing protocols use the topology and metric information to select the "best" route, and do not generally know anything about the path characteristics or services supported on the path available in the routing database. 4.3.2 Classification policy Classification is based on two categories of information, specifically information in the headers of the received packets and the control and policy information provided by the configuration (management plane), routing and signaling protocols. IPv4 Header information useable in the classification process: - Destination address - Source address - IP protocol field - TOS (DSCP field) TCP/UDP header information useable in the classification process: M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 15 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 - Source port number - Destination port number Additional header fields may be parts of the classification, if desired. The classifier makes a decision on the basis of the preconfigured classification policy information, which specifies the kind of treatment the packets belonging to a flow would like to receive. Note that the classification policy alone does not guarantee that the desired behavior will be achieved, this is further refined by the admission policy, admission control and policing functions. For IP packets, the classification process can be generally accomplished by applying the filter template of the form {DA prefix, SA prefix, PRO, TOS, SPN, DPN} to each individual packet. Any of the fields can be a wildcard, so for example all traffic destined to web server would be specified using filter {*,*,6,*,*,80}. In some cases, there may be several different filters that may match the same packet, and the results of the match for the most specific filter should be used in such cases. In addition to packet header information, local information may be added to the classification process. One example of such local information type is the interface the packet was received from. The classification policy determines how the individual flow should be treated, including attributes such as the reservation type and granularity, differentiated service class, etc. On the basis of the filtering result, the packet may be associated with the LSP, or flow identifier. 4.3.3 Admission policy Admission policy is the process to determine if the new request for the LSP set-up or attribute modification with some set of reservations is administratively acceptable. This is administratively configured, and is associated with the given granularity entity, such as individual user, user community, or peer AS. The type and the granularity of the information that will be taken into account by the admission policy depends on the interface type, local policy and trigger type (e.g. signaling versus configuration event). When reservation requests of coarse granularity are considered (e.g. individual LSP set-up on public network interface supporting a large corporation), the admission policies are typically applied against the parameters associated with the aggregate set of all reservations currently associated with the community, reservation parameters and M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 16 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 the administratively configured maximum resources allocated to that community. 4.3.4 Admission control Admission control is the process that is used to determine the resource availability to support a new request or the modification of attributes associated with existing label switched paths. Admission control is invoked as a final step, after it has been determined that the route to the destination is available, and the permission to process the request is granted by admission policy. Admission control gets more complex when the granularity of the reservations increases, being not invoked at all for best-effort traffic, and being most complex for guaranteed service. 4.4 Mapping On the basis of the flow identification performed by the classifier, the mapping process maps the packets to the appropriate label switched path. This process is configured taking into account the traffic class, attributes associated with the flow and the topology information. The mapping function is responsible for achieving the aggregation. Depending on the traffic class, two styles of mappings can exist; direct and indirect mapping. 4.4.1 Direct mapping Direct mapping can be used when the reservation does not have explicit guarantees, like bandwidth associated with it. Traffic classes suitable for direct mapping are best effort and differentiated services without bandwidth allocations. In direct mapping, the association is done directly from the packet classifier outcome to the desired LSP. 4.4.2 Indirect mapping Indirect mapping needs to be used when the reservation does have explicit guarantees, like bandwidth associated with it, and the aggregation of these is desired. The need for indirect mapping arises from the requirement to maintain per reservation state so that the individual reservation and its associated resources can be removed from the aggregate LSP. The reservation state deletion shall commence immediately after the end of reservation is detected, either through timeout, determined by observing transport header bits, or as result of signalling event. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 17 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 The associated parameter changes in the LSP configuration may be made more infrequently, especially when the frequency of the individual reservation establishments and deletions associated with given aggregated LSP is high and the reservations are relatively homogenous. This reduces the signaling load between the MPLS nodes the along the LSP. 4.5 Routing and Path selection MPLS path establishment can be triggered in a couple of different ways. In the case of best effort LSPs, simple topology information may suffice. On the other hand, in the case of QoS enabled LSPs, MPLS allows for the establishment of explicitly routed LSPs. This leads to the need for efficient mechanisms for path selection. For the MPLS network elements to be able to automatically locate paths with the sufficient resources available, routing protocols that are able to take into account additional path attributes instead of just topological connectivity and preconfigured metrics of the available paths is needed. An important step in this direction is that of Constraint Based Routing (CBR) [RFC2702]. Through the use of resource classes, certain constraints on the desired path may be put. These resource classes, which will be designated by the operator, will be a key tool in traffic engineering. After resource classes are used to constrain the routes that can be used, a generic QoS routing mechanism will be needed to select the desired path. The draft framework for QoS routing work effort have been developed in QOSR working group of the IETF [RFC2386]. However, the routing protocols with suitable metrics to be used in the environments with fine-granularity service guarantees inside or between the domains need to be developed. 4.6 Accounting and Authentication Accounting, billing and authentication issues will have to be adequately handled. These are essential to the operator for the purposes of being able to accurately bill and provide the contracted services to the customer. 5. Data Plane Mechanisms for Service Differentiation Functions This chapter describes the data plane mechanisms required in the various parts of the network to provide support for the transport of the traffic with the service parameters through the network. These mechanisms include all the mechanisms that are involved in per packet decisions that are performed in the intermediate network nodes. The parameters controlling these mechanisms are determined by the control plane mechanisms described in the previous chapter. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 18 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Note that the location of these mechanisms in the networks is not discussed in this chapter, discussion of the location of mechanisms in different network environments is given in chapter 7. 5.1 Label forwarding paradigm In the best-effort label based forwarding, MPLS nodes use the simple exact match lookup to determine the egress link where the packet should be sent. When the services that require the support for service level differentiation are implemented, MPLS node uses the same exact match label lookup and possibly additional link layer specific fields such as the EXP field (for shim header encapsulations) or the CLP bit (ATM) or DE (FR), to determine not only where the packet should be destined, but also the additional state information associated with the label, related to queuing and scheduling of the packet. 5.2 Classification 5.2.1 What is classification and where it should be done The purpose of the classification process is to determine the queuing / scheduling treatment that the packets should get as they traverse through the network. The result of the classification determine the following attributes: - service class the packet should be carried on, - for differentiated services the drop priority and / or delay priority for the packet - for guaranteed services the parameters determining the desired service guarantees Packets may be classified as belonging to different service categories in the various places of the end-to-end path traversed. Likely places where the packet classification may occur are: - Customer premises access router - Host - OperatorÆs domain ingress router When the hosts performs the classification, it may base the classification decisions either on the protocol used (part of the host protocol stack), or the attributes communicated from the application. Guaranteed service parameters will likely be based on the parameters communicated by applications. When classification is performed by the routers (either CPE router or operatorÆs border router), the classification decisions have to be based on the protocol information carried on the packet. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 19 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Initial deployment is likely to be based on the classification on the routers, as there is no support for performing classifications in the host protocol stacks. When the classification is performed in routers, modifications to host protocols and applications are not required. Additionally, it is easier to set up administrative classification policies when the classification is performed in routers. The stand-alone and integrated equipment for performing the classification for controlling the traffic are currently available, but there are neither any standard ways to manage these, nor standard ways on how the classification results are used to control the data stream. One common characteristic of current solutions is that they are usually decoupled from the other network equipment. Depending on the place where the classification is performed, the procedures performed on subsequent nodes will vary. 5.2.2 Packet Classification Packet classification performs the mapping of the individual packets onto desired LSPs. Packet classification process essentially assigns each arriving non-labelled packet onto suitable label switched path, which has to be available before the packet classifier can perform its function. Prior to the packet classification, the LSP has to have been set up using either the flow classification process or other mechanisms, such as setting up the LSP on the basis of information provided by management, topology (e.g. routing protocol) or signalling protocol. As discussed in the next subsection , flow classification process may help packet classification by producing the keys to increase the packet classifier performance. 5.2.3 Flow Classification The purpose of the flow classification process is to reduce the processing load associated with making the decision of which label to associate with the arriving packets. If the full classification can be performed for each packet without performance penalty and the suitable label exists, the flow classification is not required. Flow classification is the process of associating individual traffic flows to labels and is needed when an already established mapping (via a configuration or a signaling event, for instance) of the packet to an LSP is not available. This process needs the consideration of the classification policy to be able to associate a label with the flow. Depending on the aggregation environment, the label may be associated with single flow, or if flow aggregation is supported and a suitable label already exists, flows may be aggregated to that label. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 20 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Flow classification needs to be performed at least once for each new flow. Flow classification is typically performed on the edge MPLS nodes, where the packets from non-MPLS network domain enter onto MPLS network domain. This process can also produce a simple key, such as the entry in the hash table to be subsequently used by the packet classifier for the faster determination of the label that needs to be associated with individual packets. As more fine-grained control becomes necessary, flow classification becomes mandatory, because the accomplishment of fine-grained guarantees involve the setting up the new LSP or modifying the parameters of the existing LSP. In some cases, if it is determined that the suitable label for carrying the flow does not exist, a new LSP needs to be set up or the attributes of the existing LSP needs to be changed. The applications that are allowed to do this should be subject to careful consideration, as it is preferable to have the LSPs set up beforehand, otherwise the signaling protocol modifications done on a per flow basis consume too much resources and become the performance/scalability bottleneck. However, this is useful for some applications whose characteristics are known beforehand to require relatively long lasting flow with service level requirements, such as e.g. videoconferencing. Classification mechanisms that require edge routers to maintain per- flow state information are susceptible to denial of service attacks by malicious users. One can create the attack based on sending the packets with various destination address / port combinations in rapid sequence, causing the per flow state to be established for each packet. This can lead to exceeding of the per-flow state maintenance and flow establishment handling capacities of the routers performing the classification. There is no easy cure against such an attack, except administratively limiting the amount of the per-flow state that is associated with the interface. Together with the source address validation, this at least can provide information of where the attack originated from. For more information of flow measurements and classification, see [Claffy95], [RFC2063], [RFC1954], [Cisco97]. 5.2.4 Classification results for differentiated services For differentiated services, classification determines the differentiated service attributes, such as drop precedence bit values and delay precedence bit values. In cases where these attributes differ from those carried in the received IP packet header, the received header bits may be overwritten or depending on the implementation of the DiffServ support in MPLS, left alone. If the differentiated services attributes are allocated on per LSP basis, then the attributes are associated with the label switched M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 21 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 path, and the result of the classification process should be the label to that path and possibly other link layer specific fields such as the EXP field or the CLP bit or the DE bit. 5.2.5 Classification results for Integrated services For the integrated services, the label for the LSP that has the associated reservation attributes may be the result of the classification process. Alternatively, in fine-grained flow based systems, the flow identifier which can be used to determine its individual traffic characteristics may be the result of the classification process. In this case, these are mapped to aggregated LSP by mapping function following the classification function. 5.2.6 Problems with non end-system classifications There are some known problems in performing the classifications in intermediate network elements, which are discussed below. Whether these present a problem, and if so, the extent of the problem depends on the environment the classification function is performed, and needs to be addressed on a case-by-case basis. 5.2.6.1 Classification in presence of IPSEC When the transport protocol headers are encrypted, as described in IPSEC document "IP Encapsulating Security Payload (ESP)" [RFC1827], the transport layer (UDP/TCP) header information, such as port numbers cannot be used as parameters for determining onto which flow the packet belongs to. This implies that classification has to be performed before the encryption is applied, in the customer device (typically host, router or firewall) that performs the encryption process. Also, as per flow information is not available in the public network, it is possible to run MPLS all way to subscriber and use the label to identify IPSEC encrypted flow encapsulated onto one label. This way, it would be possible for the operator to enforce the requested parameters on per encrypted flow basis. It is also possible to achieve this using the RSVP signaling to the user, using the IPSEC extensions specified in [RFC2207], which basically uses SPI instead of the destination port number to identify the flow. 5.2.6.2 Classification in presence of dynamic address assignment The increasing use of the dynamic assignment of the IP addresses make it hard to determine the end-system the packets originated M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 22 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 from. Dynamic address assignments are common in the environments that employ DHCP, or NATs. If the end-system address is an important part of the classification policy, then the means to communicate the address - physical system mappings to classifier needs to be arranged. One possible way to achieve this in DHCP environments might be to have DHCP/DNS mapping in use, and resolve IP addresses on basis of DNS bindings. In environments, where the classification is based more on the protocol information carried in the packets, dynamic address assignment is not problem. This is due to the fact that the dynamically assigned addresses are expected to be same for the duration of the session, and the flow classifier can still use these addresses for identifying individual sessions. 5.2.6.3 Classification in presence of dynamic port numbers Some applications assign the port numbers they use dynamically, and it is very difficult or even impossible to make the correct classification on basis of such assignments. For such environments, it appears that the easiest way to achieve the correct classification is to let host determine the desired classification. 5.2.7 Classification state maintenance Classification state maintenance process is related to the deletion of the per flow state and associated LSP bindings that are not required anymore. Examples that lead to the removal of classification state are flow time-out, ending of the individual flow recognized by other means (e.g. TCP FIN) or signaling event to signify the end of reservation request. Classification state maintenance activities ensure that the unused flow state information is deleted at appropriate intervals to free up the resources in network elements. Classification state maintenance activity shall be mostly local to the MPLS node. Only when the reservations are made on individual flow basis, this affects the LSP bindings between peer MPLS nodes. If the reservation type for the flow was guaranteed reservation, and the flow was aggregated on the LSP with other guaranteed flows, state maintenance activity triggers the modification of the reservation attributes of the LSP the flow was mapped onto, but does not result in teardown of the LSP. 5.3 Policing and Marking In environments where packet classification is performed by the end- user's router or userÆs computer, it is important for the network operator to be able to enforce the traffic contract to disallow the users to exceed their contractual limits for the advanced services. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 23 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 This is performed using traffic policing, which monitors the user's traffic. The policing function can, depending on the service used, either drop packets, or move the packets to lower drop priority or different traffic class. An alternative to policing is to allow users send whatever they want, and meter the usage of different services and bill the user based on what enters the public network. However, one likely alternative is to use a combination of these mechanisms, so that the user can send up to some maximum value specified by the traffic contract per class / service, and get billed on the basis of combination of basic fee and usage. In cases where the classification is performed by the operator, the traffic contract can be enforced as part of the classification process. Policing actions can be taken at several granularity levels. Policing can be made for individual flows, when the per-flow reservations are in effect. Operator likely wants to police on basis of aggregated traffic contract on customers interface, and on MPLS network boundaries policing can be based on the individual LSP parameters. An edge LSR can either police at the LSP level or at the service contract level. For instance, several customers who all have their own SLAs can be mapped onto a given DiffServ class, several of which could in turn be mapped onto an LSP. In such a case, the policing may be performed at the customer SLA level (including per individual class specified in the SLA) as opposed to LSP level policing. 5.4 Aggregation, Merging and Deaggregation 5.4.1 Aggregation Aggregation means that multiple flows that are treated similarly in the network are associated onto the same label. Depending on the supported service type, the effort to support aggregation ranges from straightforward to very complicated. General guidelines for aggregation to meet the scalability requirements suggest that all flows that can be aggregated onto same label should be aggregated. Aggregation is the process that is performed at the first place the packet classification is performed, and involves the association of the different packets that belong to same forwarding equivalence class to the same label. Aggregation conserves label space, as the labels do not have to be associated with the individual traffic flows. 5.4.2 Merging Merging is also a form of traffic aggregation, but is performed to label switched paths, instead of the individual packets. In merge capable node, packets coming from multiple ingress LSPs and M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 24 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 belonging to the same forwarding equivalence class are sent out on the single label switched path. The merging process helps to conserve the label space, and also reduces the amount of the connection state that needs to be maintained in the intermediate network elements. 5.4.3 Aggregation and merging of traffic with service guarantees Aggregation of the traffic with service guarantees itself is not a problem, the problem is to come up with the associated service parameters for the aggregated path, in such a way that the minimum amount of the resources are reserved, and the guarantees of individual reservations are maintained through the aggregated path. Aggregation of traffic with just bandwidth guarantees is relatively straightforward; the attributes of the resulting aggregated label switched paths can be computed on the basis of the guarantees given for the individual paths or flows that are aggregated. The computation of the aggregate path parameters can be based on simply a sum of the attributes of flows or paths that the aggregate is composed of, or can take into account additional factors like oversubscription factor. When explicit guarantees for both delay and bandwidth are given, aggregation becomes much harder, especially if the delay requirements are tight. Several aggregation strategies for traffic both with and without delay guarantees are considered in [Schwantag97], and [RFC2430]. 5.4.4 Deaggregation Deaggregation is the opposite of aggregation and merging, in the sense that it terminates the label switched path and performs layer three lookup for the individual packets to determine their next destination. Deaggregation can associate the packets either with new label switched path, or to the interface to non-MPLS network. Note that the service class related information associated with the labeled packets is not lost in the deaggregation, because the attributes of the LSP the packet arrived on are available at the deaggregation point. If the LSPs are constructed through the MPLS domain, from a set of domain ingress interfaces to a single domain egress interface, and packets not associated with this egress interface are not merged or aggregated to same LSP, deaggregation process is not needed. In such cases, if the interface is to a non-MPLS domain, the MPLS header is simply removed. 5.4.5 Merging using multi-level path M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 25 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Multilevel paths can be constructed using multiple labels on stack, or alternatively partitioning the label space to represent different levels (like VPI/VCI in ATM networks). The operations associated with label stacks are described in the MPLS framework document [Callon99] and label stack encoding proposal is described in [Rosen99b]. The routing and scheduling decisions of the packets encapsulated on the multilevel label switched path are performed on the basis of the top level label. Termination of the multilevel LSP is performed at the deaggregation point, where the top level label is removed (referred as label pop in [Callon99]). The second level label is then available for use as the basis for routing and scheduling mechanisms. Multilevel paths are useful when the several paths with similar, but different service guarantees are aggregated onto same path. At the deaggregation point, the path characteristics of the individual aggregated paths that the higher level path is composed of can be determined on the basis of second level label. Figure 5.4.5. Multilevel path example Consider the simple MPLS network composed of four nodes A-D depicted on Figure 5.4.5. There are two traffic sources with reservations entering node A, from non-MPLS domains. These two sources are aggregated and leave node A on LSPx. At node B, the additional LSP (LSPy) that is destined towards same node is merged to same LSP, and the combination leaves node B as LSPz. Original labels are pushed to the label stack, and traffic leaves node B with top level label LSPz. At node C, no traffic is either merged or removed from the LSPz, LSP label just gets replaced and traffic leaves the node C with new label LSP zÆ. The traffic arrives at node D, which deaggregates the traffic to itÆs constituents LSPÆs, denoted as LSP yÆ and LSP XÆ. Now consider that all of the traffic entering and leaving the network has reservations. The capacity of LSPx is thus function of RES1_in and RES2_in. The capacity of aggregated LSPz is function of LSPx and LSPy, which at least LSPx is aggregate. As the node C does not modify the aggregate in any way, it does not need to know the parameters of the individual components the aggregate LSPz is composed of. Node D, which acts as deaggregation point for LSPyÆ and LSPxÆ needs to know the traffic attributes of both original LSPy and LSPx, but it does not need to know anything about the parameters of RES1_in and RES2_in. Compared to the model where each path requires the individual LSP through the network, the use of aggregation and multilevel paths can save significant amount of state information and signaling overhead M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 26 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 in the network The use of the multilevel labels enables the de- aggregation point still distinguish between different sources received in the aggregated LSP and to treat the traffic according to their original reservations. For this to be possible, there needs to be signalling mechanism between the aggregation point and deaggregation point to communicate the traffic attributes of the second level labels that are deaggregated. Note that this does not mean that the deaggregation point does need to know attributes of all individual LSPs, that are aggregated, deaggregated LSP may still be aggregate on other level. Also, if there are large number of aggregated flows on a single LSP, and there is deaggregation point that needs to split the traffic to number of the aggregated egress LSPs, the deaggregation point only needs to know which of the second level flows should be associated with which egress aggregate LSP, and the total aggregate value of each egress aggregated LSP. Significant benefits can be achieved at the backbone level, by aggregating all the traffic with reservations with similar characteristics onto same LSP. The backbone nodes need only know the reservation parameters of the aggregated traffic, not the parameters of individual second level LSPs that compose the aggregate. Signaling protocol needs to be run between the sending and receiving domain to be able sort out the individuals in the receiving end, but the backbone does not need to be participating in this signaling other than carrying the signaling messages. The attributes of the aggregated LSP can either be modified on basis of changes of the constituents of aggregate, but up to single message per change is required to achieve this. Additionally, if this results in rapid changes to aggregate attributes, this can be dampened e.g. by having the threshold of the minimum change to aggregate attributes that needs to happen before the aggregate parameters are signaled to be changed. 5.5 Queuing and congestion management 5.5.1 Queue management Queue management mechanisms manage the available queue space, and also determine the appropriate handling of the arriving packet, on the basis of the label switched path the packet is received on and the status of the desired queue. Queue management is closely related to congestion control, as congestion can be loosely defined as a condition where the queuing point on the network element has exceeded or is about to exceed its allocated queue space, forcing the packets be dropped instead of queued for resource. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 27 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Packet handling decisions include which queue packet should be queued on, and also whether the packet should be admitted to that queue, moved to lower priority queue or dropped. Note that the moving of the individual packets between the different queues is not necessarily a good course of action, unless all packets of same flow are put to same queue. This is because the moving of the individual packets of the flow to lower priority queue is likely cause the packet re-ordering. Since queuing mechanisms vary on the basis of the supported services and are local onto network element, they need not be subject to standardization efforts. 5.5.2 Queuing principles Various queuing principles can be used for achieving the support of the required traffic classes. Complexity increases as multiple queues with complicated relations are to be implemented. These mechanisms are extensively studied in queuing literature and are not discussed in detail here. 5.5.3 Congestion control 5.5.3.1 Passive congestion control schemes Passive congestion control schemes are based on dropping of the packets when they arrive at the congestion point. Passive schemes rely on the end-to-end protocols to find out that the packet loss has occurred and retransmit the dropped traffic with reduced traffic. Most of the Internet at a moment relies exclusively on the use of the passive congestion control schemes. TCP congestion control algorithms have been designed to act exclusively on the basis of packet loss information. Over time, numerous algorithms for the more intelligent drop policies have been developed, examples include RED [Floyd93] and W- RED. These algorithms attempt to increase fairness of the usage of congested resource, to provide preferential treatment (typically more likely to get accepted onto queue) for some portion of the flows or to increase the end-to-end throughput in congestion conditions. In fact, it is suggested that the implementation of the Assured Forwarding PHB in Differentiated Services be implemented using RED. Such passive congestion control schemes are quite likely going to be commonplace in the routers of the future. 5.5.3.2 Active congestion control schemes While passive congestion control algorithms do certainly work, one of their characteristics is that they waste network resources, as the traffic first is transmitted onto the congestion point, where it M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 28 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 is dropped, and then retransmitted later. Dropped packets thus introduce extra overhead in the network portion before the congestion point. To avoid these disadvantages, there have been proposals to make the congestion control more active. The goal of the active congestion control approaches is to reduce or eliminate the packet loss due to the congestion, or to push the drop point towards the point originating the traffic. By active congestion control, we mean that the network more directly informs the traffic sources of the congestion situations, and more importantly even before the congestion actually occurs [RFC2481]. Such active congestion control schemes are still being discussed and have not been standardized yet. 5.5.4 Packet scheduling Scheduling algorithms determine the order in which traffic waiting in the queues are scheduled for transmission. Scheduling decisions are based on the queue specific information e.g. queue priority, weight, state, etc. The need of complex scheduling mechanisms depends on the capabilities provided in the network element, such as shaping, multiple service class queues, and complex queuing policy. In FIFO based queuing systems scheduling is trivial (transmit when you have the opportunity). However, the emerging services such as Expedited Forwarding and Assured Forwarding are will necessitate more complex scheduling algorithms. The CR-LDP specification [Jamoussi99] specifies a set of traffic parameter and QoS attributes that can be signaled for a given LSP. These parameters will likely necessitate more complex scheduling and queuing mechanisms as well. 5.6 Traffic shaping Traffic shaping is the process of modifying the traffic characteristics to conform to desired traffic profile. Shaping can be used in various parts of the network to make sure that the resulting traffic conforms to the traffic contract, and thus has a better chance not to get discarded by the policing or congestion control mechanisms in the network. Traffic characteristics tend to get modified by the network, as multiple traffic streams interact, and traffic goes through buffer and scheduling algorithms. The process of shaping inside the network to make traffic to better conform to its original profile is called reshaping. Examples of the possible shaping points are end-station, MPLS edge node, or MPLS core node. Shaping can be associated with any granularity, which has defined traffic characteristics, from M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 29 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 application flow to aggregated label switched path. Shaping may be achieved as part of scheduling functionality. 5.7 Load sharing Load sharing can be implemented with MPLS routers using the path selection based on the load on the available links, and splitting the aggregated streams that are associated with different LSPs to different available links. Load sharing is especially important because of emergence of the Dense Wavelength Division Multiplexing (DWDM) systems, because these essentially divide the same fiber to up to tens of different channels going to the same destination node. Efficient load sharing allows the tight integration of the routed traffic and the transmission capabilities. Some of the issues related onto integration of optical networks and Internet are discussed in [Touch97]. MPLS based load sharing has advantage over the conventional router based load sharing, because it can take in the account also where the packets originated from, unlike the typical conventional routers. Without the knowledge where the traffic came from, it is not possible in the receiving node to easily guarantee that the packets are sent in the same order as they were sent in the previous node. Packet reordering causes performance degradation problems with TCP and some other transport protocols. 6. Label Switched Path Topologies Services are implemented by assigning attributes to label switched paths. The path is composed of point-to-point segments between adjacent MPLS nodes. In complex topologies (excluding point-to-point) each individual segment may have different values for its attributes, depending on the location of the segment along the path and the topology of entire path. This is also true when the flows with resource allocations are aggregated to stream that is associated with the same LSP. Properties of the different LSP topologies and related traffic management issues are discussed in the following sections. 6.1 Point-to-point Point-to-point LSP is the simplest of the label switched path topologies, and this is the basic building block of all LSPs. In this document, point-to-point LSPs that have their own labels and attributes, and both the label and its associated attributes have local significance between the MPLS network elements. These local LSPs are called segments in this document. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 30 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 In the simplest case, where the end-to-end LSP with the attributes is built by concatenating a set of these segments, all segments have the same attributes, while the label has only the local significance between neighbor MPLS nodes. More complex topologies can be constructed by concatenating the segments and using traffic merge (mpt-pt) and copy operations (pt-mpt) in the network elements to achieve the desired topological LSP constructs. 6.2 Point-to-multipoint Point to multipoint topologies can be constructed using the packet copy function at the ingress point-to-point LSP segment on the MPLS network element. The incoming packets are duplicated for each outgoing label switched path. Point to multipoint topologies are important for supporting of the multicast packet delivery. 6.3 Multipoint-to-point Point to multipoint topologies are important for scalability reasons. Multipoint to point topologies can be constructed using the packet merge function at the MPLS network element. The incoming packets from multiple ingress label switched paths are merged onto same outgoing label switched path. In addition to aggregating the traffic destined onto single destination, in the presence of traffic with explicit guarantees, aggregation of the traffic parameters to get the attributes for each of the LSP segment composing the multipoint to point tree is required for supporting aggregation of the traffic with explicit guarantees. Note that this can yield for the different segment to get different attributes as the traffic is merged onto the shared multipoint-to-point tree. 6.4 Multipoint-to-multipoint Multipoint to multipoint topologies cannot be directly constructed using the same labels, but these can be constructed using desired combination of point-to-point, multipoint-to-point and point-to- multipoint LSPs. Exact decomposition to simpler topologies depends on the desired connectivity in multipoint to multipoint topology. Traffic management requirements of such simpler topologies can be treated as for the simpler topologies used. For example, full mesh connectivity between a set of endpoints can be achieved using multipoint-to-point LSPs, with each endpoint acting as a receiver of separate multipoint to-point tree. Another way to achieve this is by merging using multi-level label stacks which was presented in Section 5.4.5. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 31 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 7. Network Functional Partitioning For the purposes of this document, we divide the network elements into four categories, hosts, CPE routers, operator border MPLS nodes (LERs) and core MPLS nodes (LSRs). Note that this is just a simple model to facilitate the discussion in this document and there is no reason that the roles of these network elements cannot be combined. Edge MPLS nodes (LERs) are the nodes that connect the MPLS aware network domain to non-MPLS aware domain. An example of such an element would be a border router connecting the users attached with Ethernet to the MPLS aware core network domain. Both CPE routers and domain border nodes are discussed as MPLS edge nodes, as their characteristics can be quite same, depending on the protocols and extent of the MPLS reaches to. Domain border MPLS nodes are the special cases of the edge MPLS node that connect the two MPLS aware domains together. Core MPLS nodes are the MPLS nodes in the core of the network, that are connected only to the other MPLS nodes; to the edge MPLS nodes and / or to other core MPLS nodes. 7.1 Network models Figure 7.1-1. Public MPLS network domain interface Figure 7.1.-1 depicts the interface between the MPLS network operator and operatorÆs subscriber network. Subscriber is connected on the MPLS border node, and depending of the environment can support different service categories and run different protocols towards the subscriberÆs domain. The partitioning of functionality of CPE router and operator border router in different situations is discussed in section 9.2.2. 7.2 Network element categories This chapter defines the roles of the different MPLS nodes in the network, and identifies some basic functionality that these nodes need to perform to be able to support the traffic management. For the purposes of this discussion, functionality is divided between hosts, edge MPLS nodes and core MPLS nodes. The basic assumption is that instead of using the label information just to make a forwarding decision, MPLS nodes capable of supporting differentiated services will use label and associated information also as a part of the scheduling decision. 7.2.1 MPLS edge nodes M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 32 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 In this context we include both CPE router and operatorÆs MPLS domain in discussion as edge nodes, as the traffic management functionality is somehow divided between these two nodes, and the mechanisms described in sections 5 and 6 of this document apply to both. An MPLS domain edge node contains interfaces to non-MPLS networks, as well as to MPLS network domain. There are different scenarios that determine how the functionality between the public operatorÆs MPLS border node and the CPE node needs to be divided. Figure 7.2.1. Implementation framework for MPLS edge node TM functionality The functionality and the implementation framework of the MPLS domain edge node is depicted in Figure 7.2.1. As a summary of the functionality that needs to be performed at the ingress point of the MPLS domain, the following list applies: Mandatory functions for operator border router: - Admission policy - Admission control - Direct mapping - Indirect mapping - Policing - Shaping - Aggregation - Deaggregation - Queue management - Scheduling - Label distribution - Constraint based routing Mandatory functions in either CPE equipment or operatorÆs border router: - Classification policy - Packet classification - Classification state maintenance Remaining functions, that are optional, may be performed in hosts, CPE router, operator MPLS border router, or not implemented at all: - Flow classification - Flow policing - Merging - Congestion marking M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 33 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 An MPLS network ingress point, as viewed from the MPLS domain side has to classify the traffic according to the desired service categories and allocate the traffic to the LSPs. This association between the packets at the domain ingress point and the label switched path with path attributes determines how the packet will be treated in all subsequent network elements in the LSP associated with the label. In addition, ingress MPLS node has to enforce the traffic contract between the subscriber and the public MPLS domain operator and participate on the label distribution process. More detailed descriptions of the above listed functions are given in sections 5 and 6 of this document. Note that from the direction of the operatorÆs MPLS domain towards the customer domain, the following functions are not mandatory: - Flow classifier - Packet classifier - Classification policy - Indirect mapping - Direct mapping - Flow policing The partitioning of the edge functionality is dependent on the services offered to the customer, and who is responsible for performing the traffic classification. 7.2.2 MPLS core nodes Figure 7.2.2 Implementation framework for MPLS core node TM functionality MPLS core nodes are high capacity switching elements, that contain only MPLS interfaces. Core nodes need to forward packets at high speed and differentiate the queuing treatment on basis of the label they are received with. These nodes also participate in routing and label distribution protocols, and have to support admission control for the traffic that has reservation requests. The important thing to note is that the associated state information for the treatment of the arriving packets can be determined on basis of label, there is no need for the knowledge or reapplication of the admission policies or traffic filtering. The following is a list of the traffic management functions typically performed by core node: Mandatory functions: M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 34 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 - Admission policy - Admission control - Queue management - Congestion control - Scheduling - Label distribution - Constraint based routing - Shaping Optional functions: - Aggregation - Deaggregation - Merging - Policing - Congestion marking 7.2.3 Hosts Hosts are initially likely to be just as they are at the moment, i.e. not supporting anything more than the best effort application. In the future, hosts may participate in DiffServ packet classification or support signaling mechanism, such as RSVP to request explicit service guarantees. It is also possible that at some point, hosts participate on the label distribution protocol. All of the above functions for the hosts, except the best effort communication capabilities shall remain optional. Hosts may desire to participate on MPLS domain by running the LDP/RSVP protocol to request and terminate the paths through the network, possibly with some attributes associated with the requested paths. The additional advantage of the host participation may be that, high-performance hosts may use the flow labeled LSPs to cache the state information inside the host protocol stack to increase performance by speeding up or bypassing some of the multilayer protocol stack processing. Because the hosts have limited information of the overall network topology and the aggregation strategies used by the network, hosts should only participate by originating and terminating the LSPs with the fine granularity. Aggregation and deaggregation functions should thus be left to the network. Host that actively participates in the MPLS have to support the following mechanisms depending on the services used: - LDP/RSVP processing (Mandatory) M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 35 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 - Classification policy (Mandatory) - Packet Classification (Mandatory) - Classification state maintenance (Mandatory) Hosts that actively participate on the MPLS may additionally support some of the following mechanisms: - Traffic shaping (Optional) - Active congestion control (Optional) - Scheduling (Optional) - Flow Classification (Optional) In addition, hosts may choose to participate in the Intserv environment that is also MPLS capable, and use RSVP to carry labels with the reservations. Note that there are important security considerations that generally make it infeasible for the untrusted hosts to directly participate on the operatorÆs MPLS domain in any way, discussed in more detail in section 9.2.2.4. However, for the operator owned "trusted" servers, such as web hosting facilities, etc. host participation may have some performance advantages. 8. Implementation of Services Using Specified TM Functions In this section we indicate the TM functions that must be or optionally implemented by the edge routers, CPE devices and core routers for supporting the services that were described earlier. In the tables, we use the notation that Y = mandatory, N = not needed, O = optional (but in most cases it is recommended for efficient network operation & control). 8.1 Edge Routers TM Function EBE/ EBE/ EBE/ RED ECN TE BLS VLL GS CL Congestion marking N Y O N N N N Admission policy N N N Y Y Y Y Admission control N N N Y Y Y Y Direct mapping Y Y Y Y Y Y Y Indirect mapping N N N Y Y O Y Policing/Marking O O O Y(1) Y(2) Y(3) Y Shaping O O O O O O O Aggregation Y Y Y Y O O O Queue mgmt Y O Y N N N N Scheduling Y Y Y Y Y Y Y Label distribution Y Y Y Y Y Y Y Constr. Based rout O O Y Y Y Y Y Classific policy N N N Y(4) Y Y Y Packet classific N N N Y(5) Y Y Y M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 36 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Classification state maintenance N N N Y(6) Y Y Y Flow class N N N N Y Y Y Flow policing N N N N Y Y Y Merging O O O O O(7) O O (1) This is different for the various cases of bounded loss service (2) Policing is strictly enforced. Non-conforming packets are dropped (3) Policing is strictly enforced. Non-conforming packets are dropped (4) The classification policy is applicable if it is done in the edge router. If the CPE router does the classification, then the edge router does not have to do this (5) The classification is either done in the CPE or at the edge router (6) Whichever device (CPE or border router) does the packet classification, only maintains classification state (7) Not recommended under most conditions 8.2 Core Routers TM Function EBE/ EBE/ EBE/ RED ECN TE BLS VLL GS CL Admission policy N N N Y Y Y Y Admission control N N N Y Y Y Y Policing/Marking N N N N N N N Shaping N N N N N N N Aggregation N N N N N N N Deaggegation N N N N N N N Merging O O O O O O O Queue management Y Y Y Y N N N Scheduling Y Y Y Y Y Y Y Label distribution Y Y Y Y Y Y Y Constr. based rout O O Y Y Y Y Y Congestion marking N Y O N N N N 9. Security Considerations As the support for the different levels of services, together with the different pricing structures comes in the effect, the mechanisms to monitor the service usage, enforce the service contract between parties, authorization and billing will become important. It is essential to develop the associated protocols in a such way, that the different forms of service abuse, such as different forms of theft of service are not easily possible. Since this document is not protocol specification, the specifics of the implementation alternatives are not discussed here. 10. References M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 37 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 [Andersson99] "LDP Specification", L. Andersson, P. Doolan, N. Feldman, A. Fredette, B. Thomas, work in progress, draft-ietf-mpls- ldp-spec-06.txt, October 1999 [Awduche99] "Extensions to RSVP for LSP tunnels", D. O. Awduche, L. Berger, D. Gan, T. Li, G. Swallow, V. Srinivasan, work in progress, draft-ietf-mpls-rsvp-tunnel-04.txt, September 1999 [Bernet99a] "A Framework for Differentiated Services", Y.Bernet, J. Binder, S. Blake, M. Carlson, S. Keshav, E. Davies, B. Ohlman, D. Verma, Z. Wang, W. Weiss, work in progress, draft-ietf-diffserv- framework-02.txt, February, 1999 [Bernet99b] "A Framework for Integrated Services Operation over DiffServ Networks", Y. Bernet et al., work in progress, draft-ietf- issll-diffserv-rsvp-03.txt, September, 1999 [Callon99] "A Framework for Multiprotocol Label Switching", R. Callon,P. Doolan, N. Feldman, A. Fredette, G. Swallow, and A. Viswanathan, work in progress, draft-ietf-mpls-framework-05.txt, September, 1999 [Claffy95] "A parameterizable methodology for Internet traffic flow profiling", K.C. Claffy, H-W. Braun, G. C. Polyzos, IEEE Journal on Selected Areas in Communications, vol. 13, no. 8, pp. 1481-1494, October 1995. [CIMI99] "IP VPNs and MPLS: Twin Keys to 21st Century Public IP Success", CIMI Corp., 1999 [Cisco97] "Netflow", White Paper, Cisco Systems, 1997 [Ferguson98] "What is a VPN ?", Paul Ferguson, Geoff Huston, March 1998 [Floyd93] "Random Early Detection gateways for Congestion Avoidance", S. Floyd, V. Jacobsen, IEEE/ACM Transactions on Networking, volume 1 number 4, August 1993, Pages 397-413 [Francois99] "MPLS support of Differentiated Services", F. Le Faucheur, work in progress, <draft-ietf-mpls-diff-ext-03.txt>, February 2000. [RFC2764] "A Framework for IP Based Virtual Private Networks", Bryan Gleeson, Arthur Lin, Juha Heinanen, Grenville Armitage, Andrew Malis, RFC-2764, February 2000 [Jamoussi99] "Constraint-Based LSP Setup using LDP", B. Jamoussi, editor, Work in progress, draft-ietf-mpls-cr-ldp-03.txt, September 1999. M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 38 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 [RFC1633] "Integrated Services in the Internet Architecture: an Overview", R. Braden, D. Clarck, S. Shenker, RFC-1633, June 1994 [RFC1827] "IP Encapsulating Security Payload (ESP)", R. Atkinson, RFC-1827, August 1995 [RFC1954] "Transmission of Flow Labelled IPv4 on ATM Data Links", P. Newman, W. L. Edwards, R. Hinden, E. Hoffman, F. Ching Liaw, T. Lyon, G. Minshall, RFC-1954, May 1996 [RFC2063] "Traffic Flow Measurement: Architecture", N. Brownlee, C. Mills, G. Ruth, RFC-2063, January 1997 [RFC2205] "Resource Reservation Protocol (RSVP) - Version 1 Functional Specification", R. Braden, L. Zhang, S. Berson, S. Herzog, S. Jamin, RFC-2205, September 1997 [RFC2208] "Resource ReSerVation Protocol (RSVP) Version 1 Applicability Statement: Some Guidelines on Deployment", A. Mankin, F. Baker, B. Braden, S. Bradner, M. O`Dell, A. Romanow, A. Weinrib, L. Zhang, RFC-2208, September 1997 [RFC2211] "Specification of the Controlled-Load Network Element Service", J. Wroclawski, RFC-2211, September 1997 [RFC2212] "Specification of the Guaranteed-Quality of Service", S. Shenker, C. Partridge, R. Guerin, RFC-2212, September 1997 [RFC2309] "Recommendations on Queue Management and Congestion Avoidance in the Internet", B. Braden, D. Clarck, J. Crowcroft, B. Davie, S. Deering, D. Esterin, S. Floyd, V. Jacobson, G. Minshall, G. Partridge, L. Pettersson, K. Ramakrishnan, S. Schenker, J. Wroclawski and L. Zang, RFC-2309, April 1998 [RFC2389] "A Framework for QoS-based Routing in the Internet", E. Crawley, R. Nair,B. Rajagopalan , H. Sandick, RFC-2386, August, 1998 [RFC2430] "A Provider Architecture for Differentiated Services and Traffic Engineering (PASTE)", T. Li, Y. Rekhter, RFC-2430, October 1998 [RFC2474] "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", K. Nichols, S. Blake, F. Baker, D. Black, RFC-2474, December 1998 [RFC2475] "An Architecture for Differentiated Services", S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W. Weiss, RFC-2475, December 1998 [RFC2481] "A Proposal to add Explicit Congestion Notification (ECN) to IP", K. K. Ramakrishnan, S. Floyd, RFC-2481, January 1999 M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 39 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 [RFC2597] "Assured Forwarding PHB Group", J. Heinanen, F. Baker, W. Weiss, J. Wroclawski, RFC-2597, June 1999 [RFC 2697] "A Single Rate Three Color Marker", J. Heinanen, R. Guerin, RFC 2697, September, 1999. [RFC 2698] "A Two Rate Three Color Marker", J. Heinanen, R. Guerin, RFC 2698, September, 1999. [RFC2598] "An Expedited Forwarding PHB", V. Jacobson, K. Nichols, K. Poduri, RFC-2598, June 1999 [RFC2702] "Requirements for Traffic Engineering Over MPLS", Daniel O. Awduche, Joe Malcolm , Johnson Agogbua, Mike O'Dell, Jim McManus, RFC-2702, September 1999. [Ramakrish99] "A Proposal to Incorporate ECN in MPLS", K. K. Ramakrishnan, Sally Floyd, Bruce Davie, work in progress, draft- ietf-mpls-ecn-00.txt , June 1999 [Rosen99a] "A proposed architecture for MPLS", E. Rosen, A. Viswanathan and R. Callon, work in progress, draft-ietf-mpls-arch- 06.txt, August 1999 [Rosen99b] "MPLS Label Stack Encodings", E.C. Rosen,Y. Rekhter, D. Tappan, D. Farinacci, G. Fedorkow, T. Li, A. Conta, work in progress, draft-ietf-mpls-label-encaps-07.txt, September 1999 [Schwantag97] "An Analysis of the Applicability of RSVP", Ursula Schwantag, Diploma Thesis, Universitat Karlsruhe, July 15, 1997 [Smith97] "Research Challenges for the Next Generation Internet", J.E. Smith, F. W. Weingarten, Computing Research Association, May 12-14, 1997 [Touch97] "Bridging the Gap Between Optical Networks and the Internet: Summary of a Mini-Workshop", DRAFT, Oct. 1-2, 1997, Arlington, VA, Joe Touch, Ken Young, Joe Berthold 11. Acknowledgments We would like to thank Daniel Awduche, Bilel Jamoussi and Shankar Rao for their useful comments. 12. Author's Addresses Muckai Girish SBC Technology Resources, Inc. 4698 Willow Road Pleasanton, CA 94588 USA M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 40 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Phone: (925) 598-1263 Email: mgirish@tri.sbc.com Rayadurgam Ravikanth Nokia Research Center 3 Burlington Woods Drive, Suite 260 Burlington, MA 01803 USA Phone: (781) 238-4905 Email: rayadurgam.ravikanth@nokia.com Pasi Vaananen Nokia Telecommunications 3 Burlington Woods Drive, Suite 250 Burlington, MA 01803 USA Phone: (781) 238-4981 Email: pasi.vaananen@nokia.com M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 41 Internet Draft A Framework for Service Differentiation in MPLS Networks March 2000 Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implmentation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into M. Girish et al. draft-vaananen-mpls-svc-diff-framework-00.txt 42