Internet Draft Network Working Group Eric C. Rosen Internet Draft Cisco Systems, Inc. Expiration Date: January 2001 July 2000 OSPF as the PE/CE Protocol in BGP/MPLS VPNs draft-rosen-vpns-ospf-bgp-mpls-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract [VPN] describes a method of providing a VPN service. That method allows a variety of different protocols to be used as the routing protocol between the Customer Edge (CE) router and the Provider Edge (PE) router. However, it does not fully specify the procedures which must be implemented within the Provider's network when OSPF is used as the PE/CE routing protocol. This document provides that specification. Rosen [Page 1] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 Table of Contents 1 Specification of Requirements .......................... 2 2 Introduction ........................................... 2 3 Requirements ........................................... 3 4 BGP/OSPF Interaction Procedures for PE routers ......... 5 4.1 Overview ............................................... 5 4.2 Details ................................................ 7 4.2.1 General ................................................ 7 4.2.2 Handling LSAs from the CE .............................. 8 4.2.3 Automatic creation of sham links ....................... 10 4.2.4 VPN-IP routes received via BGP ......................... 12 5 Acknowledgments ........................................ 13 6 Authors' Address ....................................... 14 7 Bibliography ........................................... 14 1. Specification of Requirements The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 2. Introduction [VPN] describes a method by which a Service Provider (SP) can use its IP backbone to provide a VPN service to customers. In that method, a customer's edge devices (CE devices) are connected to the provider's edge routers (PE routers). If the CE device is a router, then the PE router may become a routing peer of the CE router (in some routing protocol), and may as a result learn the routes which lead to the CE's site and which need to be distributed to other PE routers that attach to the same VPN. The PE routers which attach to a common VPN use BGP to distribute the VPN's routes to each other. A CE router can then learn the routes to other sites in the VPN by peering with its attached PE router in a routing protocol. CE routers at different sites do not, however, peer with each other. It can be expected that many VPNs will use OSPF as their internal routing protocol. This does not necessarily mean that the PE routers Rosen [Page 2] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 need to use OSPF to peer with the CE routers. Each site in a VPN can use OSPF as its intra-site routing protocol, while using, e.g., BGP or RIP to distribute routes to a PE router. However, it is certainly convenient, when OSPF is being used intra-site, to use it on the PE- CE link as well, and [VPN] explicitly allows this. Like anything else, the use of OSPF on the PE-CE link has advantages and disadvantages. The disadvantage to using OSPF on the PE-CE link is that it gets the PE router involved in a VPN site's IGP, however peripherally. The advantages though are: - The administrators of the CE router need not have any expertise in any routing algorithm other than OSPF. - The CE routers do not need to have support for any routing algorithms other than OSPF. - If a customer is transitioning his network from a traditional OSPF backbone to the VPN service described in [VPN], the use of OSPF on the PE-CE link eases the transitional issues. It seems likely that some SPs and their customers will resolve these trade-offs in favor of the use of OSPF on the PE-CE link. Thus we need to specify the procedures which must be implemented by a PE router in order to make this possible. (No special procedures are needed in the CE router though; CE routers just run whatever OSPF implementations they may have.) 3. Requirements Consider a set of VPN sites which are thought of as a common "OSPF domain". These will almost certainly be a set of sites which together constitute an "intranet", and each of which runs OSPF as its intra-site routing protocol. Per [VPN], the VPN routes are distributed among the PE routers by BGP. If the PE uses OSPF to distribute routes to the CE router, the standard procedures governing BGP/OSPF interactions [OSPF] would cause routes from one site to be delivered to another as AS-external routes (in type 5 LSAs). This is undesirable; it would be much better to deliver such routes in type 3 LSAs (as inter-area routes), so that they can be distinguished from any "real" AS-external routes that may be circulating in the VPN. (That is, so that they can be distinguished by OSPF from routes which really do not come from within the VPN.) Hence it is necessary for the PE routers to implement a modified version of the BGP/OSPF interaction procedures. Rosen [Page 3] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 In fact, we would like to have a very general set of procedures which allows a customer to easily replace a legacy private OSPF backbone with the VPN service. We would like this procedure to meet the following set of requirements: - The procedures should not make assumptions about the OSPF topology. In particular, it should not be assumed that customer sites are OSPF stub sites or NSSA sites. Nor should it be assumed that a customer site contains only one OSPF area, or that it has no area 0 routers. - If VPN sites A and B are in the same OSPF domain, then routes from one should be presented to the other as OSPF intra-network routes. In general, this can be done by presenting such routes in type 3 LSAs. Note that this allows two VPN sites to be connected via an "OSPF backdoor link". That is, one can have an OSPF link between the two sites which is used only when the VPN backbone is unavailable. (This would not be possible with the ordinary BGP/OSPF interaction procedures. The ordinary procedures would present routes via the VPN backbone as AS-external routes, and these could never be preferred to intra-network routes.) This may be very useful during a period of transition from a legacy OSPF backbone to a VPN backbone. - It should be possible to make use of an "OSPF backdoor link" between two sites, even if the two sites are in the same OSPF area, and neither of the routers attached to the inter-site backdoor link is an area 0 router. This can also be very useful during a transition period, and eliminates any need to reconfigure the sites' routers to be ABRs. Assuming that it is desired to have the route via the VPN backbone be preferred to the backdoor route, the VPN backbone itself must be presented to the CE routers at each site as a link between the two PE routers to which the CE routers are respectively attached. - CE routers, connected to PE routers of the VPN service, may themselves function as OSPF backbone (area 0) routers. An OSPF backbone may even consist of several "segments" which are interconnected themselves only via the VPN service. In such a scenario, full intercommunication between sites connected to different segments of the OSPF backbone should still be possible. Rosen [Page 4] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 - The transition from the legacy private OSPF backbone to the VPN service must be simple and straightforward. The transition is likely to be phased, such that customer sites are migrated one by one from the legacy private OSPF backbone to the VPN service. During the transition, any given site might be connected to the VPN service, to the legacy OSPF backbone, or to both. Complete connectivity among all such sites must be maintained. Since the VPN service is to replace the legacy backbone, it must be possible, by suitable adjustment of the OSPF metrics, to make OSPF prefer routes which traverse the SP's VPN backbone to alternative routes which do not. - The OSPF metric assigned to a given route should be carried transparently over the VPN backbone. Routes from sites which are not in the same OSPF domain will appear as AS-external routes. We presuppose familiarity with the contents of [OSPF], including the OSPF LSA types, and will refer without further exegesis to type 1, 2, 3, etc., LSAs. Familiarity with [VPN] is also presupposed. 4. BGP/OSPF Interaction Procedures for PE routers 4.1. Overview [VPN] defines the notion of a Per-Site Routing and Forwarding Table, or VRF. A PE router must be capable of running multiple instances of OSPF, where each instance is associated with a particular VRF. (Whether these instances are realized as separate processes, or merely as separate contexts of a common process, is an implementation matter.) BGP is used to distribute routes among the set of PE routers that attach to a single OSPF domain. Per [VPN], these routes are distributed as VPN-IP routes. Import/export to/from particular VRFs is governed via Route Targets. To meet the above requirements, a PE which imports a particular route into a particular VRF needs to know whether that route comes from the same OSPF domain and the same OSPF area as the CEs to which it is attached. Our procedure is to encode this information as BGP Extended Communities attributes [EXT], and have BGP distribute it along with the VPN-IP route. The OSPF metric of a route is also carried as a BGP attribute of the route. If two PEs attach to different VPN sites that are in the same OSPF area (as indicated by the OSPF area number), the PE/CE links to those Rosen [Page 5] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 site SHOULD be treated as links within that area. In addition, each PE SHOULD flood, into that area, a type 1 LSA advertising a link to the other PE. If this procedure is followed, two VPN sites in the same OSPF area will see the VPN backbone as a link within that area, a link between two PE routers. We refer to this link as a "sham link". Section 4.2.3 specifies the procedures by which a PE determines when to "create" and advertise a sham link. Every PE attached to a particular OSPF network MUST be an OSPF area 0 router. This allows it to distribute inter-area routes to the CE via Type 3 LSAs. The CE router might or might not be an area 0 router, and the PE/CE link might or might not be an area 0 link. If the OSPF network contains area 0 routers (other than the PE routers), at least one PE router must have an area 0 link to a non-PE area 0 router in that OSPF network. (The non-PE area 0 router functions as a CE router.) This ensures that inter-area routes and AS-external routes can be leaked between the PE routers and the non- PE OSPF backbone. When a type 3 LSA is sent over an area 0 link from a PE router to a CE router, a special bit in the LSA Options field is set. This is used to ensure that if any CE router sends this type 3 LSA to a PE router, the PE router will not further redistribute it. Two sites which are not in the same OSPF area will see the VPN backbone as being an integral part of the OSPF backbone. However, if there are area 0 routers which are NOT PE routers, then the VPN backbone actually functions as a sort of higher level backbone, providing a third level of hierarchy above area 0. This allows a legacy OSPF backbone to become disconnected during a transition period, as long as the various segments all attach to the VPN backbone. When a PE router needs to distribute to a CE router a route which comes from a site outside the latter's OSPF domain, the PE router presents itself as an ASBR, issues a type 4 LSA for itself if necessary, and distributes the route in a type 5 LSA. OSPF route tagging is used to ensure that a type 5 LSA generated by a PE router will be ignored by any other PE router that may receive it. A special OSPF route tag, which we will call the VPN Route Tag, will be used for this purpose. Rosen [Page 6] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 4.2. Details 4.2.1. General If a PE and a CE are communicating via OSPF, the PE MUST create, and MUST flood to the CE, a type 1 LSA advertising its link to the CE. The PE MUST have an OSPF router id which is valid (i.e., unique) within the OSPF domain. The PE MUST also be configured to know which OSPF area the link is in. A PE-CE link may be in any area, including area 0; this is a matter of the OSPF configuration. Whether or not the PE-CE link is an area 0 link, the PE itself MUST be an OSPF area 0 router. That is, the link state topology from a site will not be passed along by the PE. The PE MUST support at least one OSPF instance for each OSPF domain to which it attaches. Each instance of OSPF MUST be associated with a single VRF. If n CEs associated with that VRF are running OSPF on their respective PE/CE links, then those n CEs are OSPF adjacencies of the PE in the corresponding instance of OSPF. Generally, though not necessarily, if the PE attaches to several CEs in the same OSPF domain, it will associate the interfaces to those PEs with a single VRF. If the OSPF domain has any area 0 routers (other than the PE routers), then at least one of those MUST be a CE router, and MUST have an area 0 link to at least one PE router. This adjacency MAY be via an OSPF virtual link. For each OSPF domain, there MUST be a globally unique identifier which can be encoded in a BGP Extended Communities attribute. Routes from a particular OSPF domain will, when distributed in BGP as VPN- IPv4 routes, carry this attribute. The OSPF Domain will be encoded as the VPN of Origin Extended Community [VPN, EXT] attribute. (It is shown in [VPN] how to assign globally unique values to this attribute.) Each VRF that is associated with an OSPF instance must be configured with a value of this attribute. All VRFs that correspond to the same OSPF domain MUST be configured with the same value of this attribute. If a particular VRF in a PE is associated with an instance of OSPF, then it will be configured with a special OSPF route tag value, which we call the VPN Route Tag. This route tag will be included in any Type 5 LSAs that the PE originates and sends to any of the attached CEs. Its value is arbitrary, but must be distinct from any OSPF Route Tag being used within the OSPF domain. Its value MUST therefore be configurable. Rosen [Page 7] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 If a PE router needs to use OSPF to distribute to a CE router a route which comes from a site outside the CE router's OSPF domain, the PE router SHOULD present itself to the CE router as an Autonomous System Border Router (ASBR), and SHOULD report such routes as AS-external routes. That is, these PE routers originate Type 5 LSAs reporting the extra-domain routes as AS-external routes. Each such Type 5 LSA MUST contain an OSPF route tag whose value is that of the VPN Route Tag. This tag identifies the route as having come from a PE router. The VPN Route Tag MUST be used to ensure that a Type 5 LSA originated by a PE router is not redistributed through the OSPF area to another PE router. When a PE/CE link is an area 0 link, the high-order bit of the LSA Options field (previously unused) is used to distinguish type 3 LSAs which report routes across the VPN backbone from other VPN sites. We refer to this as the DN bit. The DN bit is set in a type 3 LSA which is sent from a PE router to a CE router across an area 0 link. When a PE router receives, from a CE router, a type 3 LSA with the DN bit set, the route is ignored. Without this mechanism, the type 3 LSA could be sent by a PE to a CE, flooded through a number of OSPF routers, and then sent to another PE; this could cause a loop. This mechanism prevents such loops. 4.2.2. Handling LSAs from the CE This section specifies the way in which a PE router handles the OSPF LSAs it receives from a CE router. If a route is received in a Type 5 LSA, and if it has an OSPF route tag value equal to the VPN Route Tag, then that route is ignored. If a route is received in a Type 3 LSA, and if it has the DN bit set in its Options Field, then the route is ignored. Otherwise, there is normal OSPF processing, for the relevant instance of OSPF in the PE. Next, the PE must examine the corresponding VRF. For every address prefix which appears there, the PE must create a VPN-IPv4 route in BGP. These routes must have the following attributes: - The VPN of Origin Extended Community attribute must uniquely and globally identify the OSPF domain, as explained above. Rosen [Page 8] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 - OSPF Route Type Extended Community Attribute. This is encoded as follows: * Type: 0x8000 * Area Number: 4 bytes, encoding a 32-bit area number. For AS- external routes, the value is 0. A non-zero value identifies the route as being internal to the OSPF domain, and as being within the identified area. Area numbers are relative to a particular OSPF domain. * OSPF Route Type: 1 byte, encoded as follows: ** 1 or 2 for intra-area routes (depending on whether the route came from a type 1 or a type 2 LSA -- however this difference is not significant to the procedures specified herein) ** 3 for summary routes ** 5 for external routes (area number must be 0) ** 7 for NSSA routes. * Options: 1 byte. Currently this is only used if the route type is 5 or 7. Setting the least significant bit in the field indicates that the route carries a type 2 metric. - OSPF Router ID Extended Community. This attribute specifies the router-id corresponding to the router that is identified in the BGP Next Hop attribute. * Type: 0x8001 * Router ID: 4 Byte * Unused: 2 Byte. Must be zero. Note that this router-id is an address in the address space of the OSPF network (i.e., the address space of the VPN), NOT in the address space of the Service Provider network. That is, it must be unique only within an OSPF domain. - MED. By default, this should be set to the value of the OSPF distance associated with the route, plus 1. The intention of all this is the following. OSPF Routes from one site are converted to BGP, distributed across the VPN backbone, and possibly converted back to OSPF routes before being Rosen [Page 9] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 distributed into another site. With these attributes, BGP carries enough information about the route to enable the route to be converted back into OSPF "transparently", just as if BGP had not been involved. Routes which a PE receives in type 4 LSAs MUST be ignored. The attributes specified above are in addition to any other attributes which routes must carry in accord with the [VPN]. The Site of Origin attribute, which is usually required by [VPN], is OPTIONAL for routes which a PE learns from a CE via OSPF. Use of the Site of Origin attribute would, in the case of a multiply homed site (i.e., a site attached to several PE routers), prevent an intra-site route from being reinjected into a site from the VPN backbone. Such a reinjection would not harm the routing, because the route via the VPN backbone would be advertised in a type 3 LSA, and hence would appear to be an inter-area route; the real intra-area route would be preferred. But unnecessary overhead would be introduced. On the other hand, if the Site of Origin attribute is not used, a partitioned site will find itself automatically repaired, since traffic from one partition to the other will automatically travel via the VPN backbone. Therefore the use of a Site of Origin attribute is optional, so that a trade-off can be made between the cost of the increased overhead and the value of automatic partition repair. 4.2.3. Automatic creation of sham links Suppose that the PE is attached to a CE which is in OSPF domain D, and that the PE/CE link is in OSPF area n. Suppose the PE receives a labeled VPN-IPv4 route via BGP, whose Route Target attribute allows the route to be installed in the PE's VRF for OSPF domain D. Suppose that all of the following conditions hold: - either the route has no Site of Origin attribute, or the route's Site of Origin attribute is not the same as the Site of Origin associated with the CE, - the route has an OSPF route type whose area number is the same as the area number of the PE/CE link (i.e., area n), Rosen [Page 10] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 - the route has an OSPF Router ID attribute. Then the PE will create a type 1 LSA, describing a link between itself (i.e., its own OSPF router id for its instance of OSPF in OSPF domain D) and the router id which is the value of that attribute. This link (call it a "sham link") is considered to be in area n. It is incorporated in the PE's link state database for the area, and flooded to the CE. (Of course, if this LSA already exists, a new one is not created.) Sham links may be created for any area, including area 0. If it is known that there are no backdoor links in a particular, it may be desirable to disable the creation of sham links for that area, especially if that area is area 0. If, due to BGP withdrawals, there are no longer any received BGP routes for which the above conditions hold, the sham link is removed, and the necessary OSPF actions taken. Similarly, the sham link must be removed if a PE finds that the other endpoint is unreachable, according to the routing algorithm of the VPN backbone network. The OSPF metric associated with a sham link must be configurable (and there must be a configurable default). Even though LSAs are not flooded on the link, as long as there is an OSPF "backdoor connection" between the VPN sites, LSAs from one site will flow to the other. Whether traffic between the sites flows via the backdoor link or via the VPN backbone depends on the settings of the OSPF link metrics. The metrics can be set so that the backdoor link is not used unless connectivity via the VPN backbone fails, for example. PEs SHOULD NOT flood LSAs on a sham link. When the Site of Origin attribute is used, if two PEs are attached to the same VPN site (as identified by the Site of Origin attribute), a sham link is NOT created between them. Note that a BGP route which is determined to be an intra-area route is never redistributed into OSPF. Instead, the sham link is created and advertised, and normal OSPF procedures will find the intra-area route. This assumes that if an area spans two or more sites, there is at least one site which exports at least one route to each other site, and which imports at least one route from each other site. This will cause the creation of sufficient bidirectional sham links to provide full OSPF intra-area connectivity. In a two-site OSPF area, as long as the backdoor link is up, OSPF Rosen [Page 11] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 will cause the VRF to be populated with intra-area routes, and the corresponding BGP routes will not be installed in the VRF. Note that the route via a sham link is an intra-area route, and may be preferred by OSPF to the route that uses the backdoor link. If the backdoor link fails, OSPF will remove these intra-area routes. Inter-site routes from BGP will then be installed in the VRF, and reported to the CE in type 3 LSAs. (If the preferred intra-area route were using the sham link rather than the backdoor link, failure of the backdoor link may not cause any change in the route taken the packets, even though that route may cease to be an intra-area route and become instead an inter-area route.) If a particular LSA has been issued by the PE router in order to distribute a particular BGP route, and the BGP route is later withdrawn, the corresponding LSA MUST be flushed. 4.2.4. VPN-IP routes received via BGP This section describes how the PE router handles VPN-IP routes received via BGP. If a received BGP VPN-IP route is not installed in the VRF, nothing is reported to the CE. A received route will not be installed into the VRF if some other route is preferable. (Note that a route which is not installed in the VRF may still cause the PE to create an OSPF link to another PE as specified in the previous section.) Note that according to the usual OSPF route preference rules, intra- area routes, as computed by the OSPF, will be installed in the VRF in preference to any other routes received over BGP. This means that the CE will simply not hear about inter-area or external routes to address prefixes for which there is an intra-area route. In the following, we specify what is reported, in OSPF LSAs, by the PE to the CE, assuming that the PE is not configured to do any further summarization or filtering of the routing information before reporting it to the CE. If ONE of the following conditions holds for a given route that is received via BGP: - the route does not have a VPN of Origin attribute (identifying the OSPF domain), or Rosen [Page 12] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 - the route's VPN of Origin attribute has a value which is not the same as that associated with the VRF, or - the route has no OSPF Route Type attribute then the route MUST be distributed to the CE in a type 5 LSA with a type 2 metric. By default, the MED, if present, is converted to a type 2 metric. If the MED is not present, a default type 2 metric value is used. Otherwise, if the route has an OSPF route type of external route, it MUST be advertised to the CE in a type 5 LSA. By default, the MED, if present, is converted to a type 1 or type 2 metric, as determined by the external route property of the VPN-IPv4 route. If no MED is present, a default type 2 metric value is used. Whenever a type 5 LSA is originated by a PE router and sent to a CE router, the VPN Route Tag MUST be placed in the LSA. Otherwise, if the route has an OSPF route type of "summary route", the route should be treated as if it had been received in an OSPF type 3 LSA. This means that the PE will report the route in a type 3 LSA to the CE. The DN bit is set in this LSA. (Note that this case is possible even if the VPN-IP route carries an area number identical to that of the CE router. This means that if an area is "partitioned" such that the two pieces are connected only via the VPN backbone, it appears to be two areas, with inter-area routes between them.) Whenever the PE sends the CE a type 5 LSA, it MUST also create a type 4 LSA for itself, presenting itself as an ASBR. a Note that this way of handling AS-external routes makes every PE appear to be an ASBR attached to all the AS-external routes. In a multihomed site, this can result in a number of type 5 LSAs containing the same information. 5. Acknowledgments Significant contributions to this work have been made by Derek Yeung and Yakov Rekhter. Thanks to Ross Callon for his comments. Rosen [Page 13] Internet Draft draft-rosen-vpns-ospf-bgp-mpls-00.txt July 2000 6. Authors' Address Eric C. Rosen Cisco Systems, Inc. 250 Apollo Drive Chelmsford, MA, 01824 E-mail: erosen@cisco.com 7. Bibliography [EXT] "BGP Extended Communities Attribute", draft-ramachandra-bgp- ext-communities-04.txt, Ramachandra, S. and Tappan, D. May, 2000. [OSPF] "OSPF Version 2", RFC 2328, Moy, J., April 1998. [VPN] "BGP/MPLS VPNs", draft-rosen-rfc2547bis-01.txt, Rosen, E., et. al., May 2000. Rosen [Page 14]