Internet Draft INTERNET-DRAFT Matt Crawford Fermilab <draft-ietf-ipngwg-esd-analysis-00.txt> Allison Mankin ISI Thomas Narten IBM John W. Stewart, III ISI Lixia Zhang UCLA March 1997 IPng Analysis of the GSE Proposal <draft-ietf-ipngwg-esd-analysis-00.txt> Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow Directories on ds.internic.net (US East Coast), nic.nordu.net (Europe), ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim). Distribution of this memo is unlimited. This Internet Draft expires September, 1997. Abstract On February 27-28 1997, the IPng Working Group held an interim meeting in Palo Alto, California to consider adopting Mike O'Dell's ``GSE - An Alternate Addressing Architecture for IPv6'' proposal [GSE]. In GSE, 16-byte IPv6 addresses are split into three portions: a globally unique End System Designator (ESD), a Site Topology Partition (STP) and a Routing Goop (RG) portion. The STP corresponds (roughly) to a site's subnet portion of an IPv4 address, whereas the draft-ietf-ipngwg-esd-analysis-00.txt [Page 1] INTERNET-DRAFT March 1997 RG identifies the attachment point to the public Internet. Routers use the RG+STP portions of addresses to route packets to the link to which the destination is directly attached; the ESD is used to deliver the packet across the last hop link. An important idea in GSE is that nodes within a Site would not need to know the RG portion of their addresses. Border routers residing between a Site and its Internet connect point would dynamically replace the RG part of source addresses of all outgoing IP datagrams, and the RG part of destination addresses on incoming traffic. This document provides a detailed analysis of the GSE plan. Much of the analysis presented here is an expansion of official meeting minutes, though it also includes issues uncovered by the authors in the process of fully fleshing out the analysis. In summary, the consensus of the attendees of the PAL1 meeting was that having routers rewrite the Routing Goop portion of addresses should not be adopted, though other parts of the GSE plan should (e.g., having globally unique ESDs). After completing the first draft of this document, the authors still strongly concur with this outcome. As a first draft, this document should not be considered to represent the views of the IPng Working Group. Instead, it should be viewed as the rough consensus of the PAL1 attendees and the strong consensus of the five authors. It is hoped that this first draft of the document will be the catalyst for discussions to refine the written analysis, and especially the conclusions, so that after some number of iterations it will represent the consensus of the working group. draft-ietf-ipngwg-esd-analysis-00.txt [Page 2] INTERNET-DRAFT March 1997 Contents Status of this Memo.......................................... 1 1. Introduction............................................. 4 2. Addressing and Routing in IPv4........................... 5 2.1. The Need for Aggregation............................ 6 2.2. The Pre-CIDR Internet............................... 7 2.3. CIDR and Provider-Based Addressing.................. 8 2.4. Multihoming and Aggregation......................... 11 3. GSE Background........................................... 12 3.1. Motivation For GSE.................................. 12 3.2. GSE Address Format.................................. 13 3.3. Routing Stuff (RG and STP).......................... 14 3.4. End-System Designator............................... 15 3.5. Address Rewriting by Border Routers................. 16 3.6. Renumbering and Rehoming Mid-Level ISPs............. 17 3.7. Support for Multihomed Sites........................ 18 3.8. Explicit Non-Goals for GSE.......................... 19 4. Analysis of GSE's Advantages and Disadvantages........... 19 4.1. End System Designator............................... 19 4.1.1. IP Addresses in the IPv4 Internet.............. 19 4.1.2. Overloading Addresses: Network Layer Issues.... 20 4.1.3. Overloading Addresses: Transport Layer Issues.. 22 4.1.4. Benefits of Globally Unique ESDs............... 23 4.1.5. ESD: Network Layer Issues...................... 24 4.1.6. ESD: Transport Layer Issues.................... 25 4.1.7. ESD: Application Layer Issues.................. 32 4.1.8. When ESDs are Not Unique....................... 34 4.1.9. DNS PTR Queries................................ 36 4.1.10. Reverse Mapping of ESDs....................... 38 4.1.11. Reverse Mapping of Complete GSE Addresses..... 39 4.1.12. The ICMP ``Who Are You'' Message.............. 40 4.2. Renumbering and Domain Name System (DNS) Issues..... 41 4.2.1. How Frequently Can We Renumber?................ 41 4.2.2. Efficient DNS support for Site Renumbering..... 42 4.2.3. Synthesizing AAAA Records...................... 43 4.2.4. Two-Faced DNS.................................. 43 4.2.5. Bootstrapping Issues........................... 44 4.2.6. DNS PTR RRs Not Needed......................... 45 4.2.7. Renumbering and Reverse DNS Lookups............ 45 4.3. Address Rewriting Routers........................... 46 4.3.1. Load Balancing................................. 46 4.3.2. End-To-End Argument: Don't Hide RG from Hosts.. 47 4.4. Multi-homing........................................ 47 draft-ietf-ipngwg-esd-analysis-00.txt [Page 3] INTERNET-DRAFT March 1997 5. Recommendations.......................................... 49 6. Security Considerations.................................. 50 7. Acknowledgments.......................................... 50 8. References............................................... 51 9. Authors' Addresses....................................... 52 1. Introduction In October of 1996, Mike O'Dell published an Internet-Draft (dubbed ``8+8'' that proposed significant changes to the IPv6 addressing architecture. The 8+8 proposal was the topic of considerable discussion at the December, 1996 IETF meeting in San Jose. Because the proposal offered both potential benefits (e.g., enhanced routing scalability) and risks (e.g., changes to the basic IPv6 architecture), the IPng Working Group held an interim meeting on February 27-28, 1997 to consider adopting the 8+8 proposal. The meeting, at which over 45 persons attended, was held at Sun Microsystems' PAL1 facility in Palo Alto, CA. Shortly before the interim meeting, an updated version of the Internet-Draft was produced, in which the name of the proposal was changed from ``8+8'' to ``GSE,'' for the three separate components of the address: Global, Site and End-System Designator. This last version of the GSE proposal was published as an Informational RFC [GSE] for historical purposes. The stated purpose of the meeting was to evaluate the GSE proposal and make a firm decision to either: 1) Definitely adopt GSE for IPv6, 2) Adopt GSE contingent upon certain other documents being successfully completed by the April, 1997 IETF, or 3) Definitely don't adopt it. The well-attended meeting generated high caliber, focused technical discussions on the issues involved, with participation by almost all of the attendees. By the middle of the second day there was unanimous agreement by the attendees that the GSE proposal as written presented too many risks and should not be adopted as the basis for IPv6. However, the attendees also concluded that some of the issues discussed in the GSE proposal were equally applicable to the current draft-ietf-ipngwg-esd-analysis-00.txt [Page 4] INTERNET-DRAFT March 1997 IPv6 provider-based addressing plan and had enough benefit to warrant making changes to some existing IPv6 documents. These changes include: 1) Making changes to the IPv6 provider-based addressing document, to facilitate increased aggregation. 2) Creating hard boundaries in IPv6 addresses to clearly distinguish between the portions used to identify hosts, for routing within a Site, and for routing within the Public Internet. 3) Designating the low-order 8 bytes of IPv6 addresses to be a globally unique End System Designator (ESD). This change has potential benefits to future transport protocols (e.g., TCPng). 4) Make a clear distinction between the ``locator'' part of an address and the ``identifier'' part of the address. The former is used to route a packet to its end point, the latter is used to identify an end point, independent of the path used to deliver the packet. Although this is a potentially revolutionary change to IPv6 addressing model, existing transport protocols such as TCP and UDP will not take advantage of the split. Future transport protocols (e.g., TCPng), however, may. 5) Making changes to the way AAAA records are stored within the DNS, so that renumbering a Site (e.g., when a Site changes ISPs) requires few changes to the DNS database in order to effectively change all of a Site's address AAAA RRs. The remainder of this document attempts to capture the debate and discussion that led to the above changes. 2. Addressing and Routing in IPv4 Before dealing with details of GSE, we present some background about how routing and addressing works in ``classical IP'' (i.e., IPv4). We present this background because the GSE proposal proposes a fairly major change to the base model. In order to properly evaluate the benefits of GSE, one must understand what problems in IPv4 it alleges to improve or fix. The structure and semantics of a network layer protocol's addresses are absolutely core to that protocol. Addressing substantially impacts the way packets are routed, the ability of a protocol to scale and the kinds of functionality higher layer protocols can provide. Indeed, addressing is intertwined with both routing and draft-ietf-ipngwg-esd-analysis-00.txt [Page 5] INTERNET-DRAFT March 1997 transport layer issues; a change in any one of these can impact another. Issues of administration and operation (e.g., address allocation and required renumbering), while not part of the pure exercise of engineering a network layer protocol, turn out to be critical to the viability of that protocol in a global and commercial network. The interaction between addressing, routing and especially aggregation, is particularly relevant to this document, so some time will be spent describing it. Addresses in IPv4 serve two purposes: 1) Unique identification of an interface. That is, the IP address by itself identifies which interface a packet should be delivered to. 2) Location information of that interface. Routers extract location information from packets in order to route them towards their ultimate destination. That is, addresses identify ``where'' the intended recipient is located within the Internet topology. What is important to note is that these identification and location requirements have been met through the use of the same value, namely the IP address. As will be noted repeatedly in this document, the ``over-loading'' of IPv4 addresses with multiple semantics has some undesirable implications. For example, the embedding of IPv4 addresses within transport protocol addresses that identify the end point of a connection involves those transport protocols with routing. This entanglement is inconsistent with a strictly layered model in which routing would be a completely independent function of the network layer and not directly impact the transport layer. In addition to architectural uncleanliness, combining the locator and identifier has the practical impact of complicating the support for mobility. In a mobile environment, the location of an end-station may change even though its identity stays the same; transport connections should be able to survive such changes. In IPv4, however, one cannot change the locator without also changing the identifier. Consequently, conventional wisdom for some time has been that having separate values for location and identification could be of significant benefit. The GSE proposal attempts to make such a separation. 2.1. The Need for Aggregation IPv4 has seen a number of different addressing schemes. Since the original specification, the two major additions have been subnetting and classless routing. The purpose of adding subnetting was to allow draft-ietf-ipngwg-esd-analysis-00.txt [Page 6] INTERNET-DRAFT March 1997 a collection of tens or hundreds of networks located at one site to be viewed from afar as being just one IP network (i.e., to aggregate all of the individual networks into one bigger network). A practical benefit of subnetting was that all of a site's hosts, even if scattered among tens or hundreds of LANs, could be reached via a single routing table entry in routers located far from the site. In contrast, prior to subnetting, a site with ten LANs might advertise ten separate routing table entries to the routing subsystem of the Internet. The benefits of aggregation should be clear. The amount of work involved in computing forwarding tables from routing tables is dependent in part on the number of network routes (i.e., destinations) to which best paths are computed. If each site has 10 internal networks, and each of those individual networks is advertised to the global routing subsystem as individual routing entries, the complexity of computing forwarding tables can easily be an order of magnitude greater than if each site advertised just a single route that covered all of the addresses used within the site. 2.2. The Pre-CIDR Internet In the early days of the Internet, the Internet's topology and its addressing were treated as orthogonal. Specifically, when a site wanted to connect to the Internet, it approached a centralized address allocation authority to obtain an address and then approached a provider about procuring connectivity. This procedure for address allocation resulted in a system where the addresses used by customers of a certain provider bore little relation to the addresses used by other customers of that provider. In other words, though the topology of the Internet was mostly hierarchical, the addressing was not (in a global sense), and little aggregation of routes took place. An example of such a topology and addressing scheme shown in Figure 1. draft-ietf-ipngwg-esd-analysis-00.txt [Page 7] INTERNET-DRAFT March 1997 +----------------+ | |------- Customer1 (192.2.2.0) | |------- Customer2 (128.128.0.0) | Provider A |------- Customer3 (18.0.0.0) | |------- Customer4 (193.3.3.0) | |------- Customer5 (194.4.4.0) +----------------+ | | | | +----------------+ | Provider B | +----------------+ Figure 1 Figure 1 shows Provider A having 5 customers, each with their own independently obtained network addresses. Providers A and B connect to each other. In order for Provider B to be able to send traffic to Customers1-5, Provider A must announce each of the 5 networks to Provider B. That is, the routers within Provider B must have explicit routing entries for each of Provider A's customers, 5 separate routes in Figure 1. Experience has shown that this approach scales very poorly. In the Default-Free Zone (DFZ) of the Public Internet, where routers must maintain routing tables for all reachable destinations, the cost of computing forwarding tables quickly becomes unacceptable large. A large part of the cost is related to the seemingly redundant computations that must be made for each individual network, even though the reality is that many reside at the same end site. 2.3. CIDR and Provider-Based Addressing Classless Inter-Domain Routing (CIDR) and its associated provider- assigned address allocation policy were introduced (in part) to help reduce the size of and cost of computing forwarding tables. In CIDR, sites that want to connect to the Internet approach a provider to procure both connectivity and a network address; providers have large blocks of address space and assign pieces of them out to customers such that customers of the same provider have addresses with some number of leading bits in common. Note that CIDR started the use of the term ``prefix'' to refer to a Classless network. The combination of CIDR and provider-based addressing results in the ability for a provider to address many hundreds of sites while introducing just draft-ietf-ipngwg-esd-analysis-00.txt [Page 8] INTERNET-DRAFT March 1997 *one* network address into the DFZ global routing system, i.e., aggregating all of its customers addresses under one prefix. An example of such a topology and addressing scheme is shown in Figure 2. +----------------+ | |------- Customer1 (204.1.0.0/19) | |------- Customer2 (204.1.32.0/23) | Provider A |------- Customer3 (204.1.34.0/24) | |------- Customer4 (204.1.35.0/24) | |------- Customer5 (204.1.36.0/23) +----------------+ | | | | +----------------+ | Provider B | +----------------+ Figure 2 In Figure 2, Provider A has been assigned the classless block, or ``aggregate,'' 204.1.0.0/16 (i.e., a network prefix with 16 bits for the network part and 16 bits for local use). Provider A has 5 customers, each of which has been assigned a (longer) prefix subordinate to the aggregate. Note that unlike the pre-CIDR days of ``classful addressing'' the amount of address space assigned to a customer no longer needs to limited to the hard byte-boundary of the ``classful days'' In order for Provider B to be able to forward traffic to Customers1-5, Provider A need only announce a single prefix, 204.1.0.0/16, because that prefix covers all of its customers. The benefit for Provider B is that its routers need only a single routing table entry to reach all of Provider A's customers. Note the difference between the cases described in Figures 1 and 2. The important difference in the two Figures is that the latter example uses fewer routes to reach the same number of destinations. CIDR was a critical step for the Internet: in the early 1990s the overhead of computing and constructing forwarding tables in the DFZ required to support the Classful Internet was almost more than the commercially-available hardware and software of the day could handle. The introduction of BGP4's classless routing and provider-based address allocation policies resulted in an immediate relief. Having said that, however, there are some weaknesses of the system. First, the internet addressing model shifted from one of ``address owning'' to ``address lending.'' In pre-CIDR days sites acquired addresses draft-ietf-ipngwg-esd-analysis-00.txt [Page 9] INTERNET-DRAFT March 1997 from a central authority independent of who their network provider was, and a site could assume it ``owned'' the address it was given. Owning addresses meant that once one had been given a set of network addresses, one could always use them and assume that no matter where a site connected to the Internet, the prefix for that network could be injected into the public routing system, and in particular, into the DFZ. With CIDR, however, it is simply no longer possible for each individual site to have its own private network prefix be injected into the DFZ; there would simply be too many of them. Consequently, if a site decides to change providers, then it needs to number itself out of space given to it by the new provider and give its old address back to the old provider. To understand this, consider if, from Figure 2, Customer3 changes its provider from Provider A to Provider C, but does not renumber. The picture would be as follows: +----------------+ | |------- Customer1 (204.1.0.0/19) | |------- Customer2 (204.1.32.0/23) | Provider A | +------| |------- Customer4 (204.1.35.0/24) | | |------- Customer5 (204.1.36.0/23) | +----------------+ | | +----------------+ | | Provider B | | +----------------+ | | | | +----------------+ +------| Provider C |------- Customer3 (204.1.34.0/24) +----------------+ Figure 3 In Figure 3, each of Provider A, B and C are directly connected to the other providers. In order for Provider B to reach Customers 1, 2, 4 and 5, Provider A still only announces the 204.1.0.0/16 aggregate. However, in order for Provider B to reach Customer 3, Provider C must also announce the prefix 204.1.34.0/24. Prefix 204.1.34.0/24 is called a ``more-specific'' of 204.1.0.0/16; another term used is that Customer3 and Provider C have ``punched a hole in'' Provider A's block. The result of this is that from Provider B's view, the address space underneath 204.1.0.0/16 is no longer cleanly aggregated into a single prefix and instead the aggregation has been broken because the addressing is inconsistent with the topology; in order to maintain reachability to Customer3, Provider B must carry two prefixes where it used to have to carry only one prefix. draft-ietf-ipngwg-esd-analysis-00.txt [Page 10] INTERNET-DRAFT March 1997 The example in Figure 3 explains why sites must renumber if existing levels of aggregation are to be maintained. While it is certainly clear that one or two ``exceptions'' to the ideal case can be tolerated, the reality in today's Internet is that there are thousands of providers, many with thousands of individual customers. Some renumbering of sites would seem essential to maintain sufficient aggregation. The empirical cost of renumbering a site in order to maintain aggregation has been the subject of much discussion. The practical reality, however, is that forcing all sites to renumber is difficult given the size and wealth of companies that now depend on the Internet for running their business. Thus, although the technical community came to consensus that address lending was necessary in order for the Internet to continue to operate and grow, the reality has been that some of CIDR's benefits have been lost because sites refuse to renumber. One unfortunate characteristic of CIDR at an architectural level is that the pieces of the infrastructure which benefit from the aggregation (i.e., the providers which constitute the default-free part of the routing infrastructure) are not the pieces that incur the cost to achieve the aggregation. The logical corollary of this statement is that the pieces of the infrastructure which do incur cost to achieve aggregation (e.g., sites which renumber when they change providers) don't directly see the benefit. The word ``directly'' is used here because one could claim that the continued operation of the Internet is a benefit, though it is an indirect benefit and requires selflessness on the part of the site in order to recognize it. 2.4. Multihoming and Aggregation As sites become more dependent on the Internet, they have begun to install additional connections to the Internet to improve robustness and performance. Such sites are called ``multi-homed.'' Unfortunately, when a site connects to the Internet at multiple places, the impact can be much like a site that switches providers but refuses to renumber. In the pre-CIDR days, multi-homed sites were typically known by only one network address. When that site's providers would announce the site's network into the global routing system, a ``shortest path'' type of routing would occur so that pieces of the Internet closest to the first provider would use the first provider and other pieces would use the second provider. This allowed sites to deal with the load on their multiple connections with the routing system itself. draft-ietf-ipngwg-esd-analysis-00.txt [Page 11] INTERNET-DRAFT March 1997 With CIDR, if a multi-homed site is known by a single prefix taken from one of its providers, then that prefix is aggregable by the provider which assigned the address but *not* aggregable by the other providers. One way to prevent entropy from taking over under CIDR is to have multi-homed sites use address space from all of its providers. Though this in itself is not so difficult, it changes the way load-sharing is handled, complicating the engineering by requiring much more foresight and by introducing complexities like DNS and its caching system. So, like those sites which refuse to renumber, many multi- homed sites today are known by a single prefix, thus reducing the efficiency of the global routing system. To be clear, there certainly are ways with CIDR for sites to be multi-homed without having a negative impact on the routing infrastructure, and there are some sites that do this today. However, operational experience to date has shown an unwillingness on the part of most sites to do the work necessary to multi-home in a way that is CIDR-friendly. Sites have more experience doing load-sharing under the pre-CIDR type of multi-homing than post-CIDR, so this is one reason for the reluctance. Another reason, however, is that in its documentation, CIDR presents several options related to multi-homing, but it does not choose one option and fully flesh out related details like load-sharing. While the analysis of GSE will end up showing that it actually does little to improve the support for multi-homing, it should be given great credit for giving the topic significant attention as a distinguished service from the beginning, rather than as an after-thought. 3. GSE Background This section provides background information about GSE with the intent of making this document stand-alone with respect to the GSE ``specification.'' Additional details on GSE can be found in [GSE]. First the motivations behind GSE will be discussed, then the important technical details will be described and finally some explicit non-goals will be listed. 3.1. Motivation For GSE The primary motivation for GSE is the fact that the chief IPv6 global unicast address structure, provider-based, is fundamentally the same as IPv4 with CIDR and provider-based aggregation. Many people are not satisfied with the scaling factors achieved with CIDR and provider- based aggregation and think that better solutions can, and in fact draft-ietf-ipngwg-esd-analysis-00.txt [Page 12] INTERNET-DRAFT March 1997 must, be found. The GSE draft asserts that IPv4 with CIDR has not achieved the aggressive aggregation required for the route computation functions of the default-free zone of the Internet to scale for IPv4, and that the larger addresses of IPv6 simply exacerbate the problem. More importantly, a key aspect of provider-based aggregation is its requirement that end sites be renumbered in response to topological changes (e.g., when an end site switches ISPs). The GSE proposal asserts that acceptable aggregation can continue only if renumbering is forced, but the future viability of forced renumbering is unclear given the increasing dependence on the Internet by litigious commercial organizations. This fact is particularly relevant in cases where end-users are asked to renumber because an upstream provider has changed its transit provider (i.e., the end site is forced to renumber by forces outside of its control). Finally, GSE deals significantly with Sites that have multiple Internet connections. In some addressing schemes (e.g., CIDR), this ``multi-homing'' can create exceptions to the aggregation and result in poor scaling. That is, the public routing infrastructure needs to carry multiple distinct routes for the multi-homed site, one for each independent path. GSE recognizes the ``special work done by the global Internet infrastructure on behalf of multi-homed sites,'' [GSE] and proposes a way for multi-homed Sites to gain some benefit without impacting global scaling. This includes a specific mechanism that providers could use to support multi-homed Sites, presumably at a cost that the Site would consider when deciding whether or not to become multi-homed. 3.2. GSE Address Format The key departure of GSE from classical IP addressing (both v4 and v6) is that rather than over-loading addresses with both locator and identifier purposes, it splits the address into two elements: the high-order 8 bytes for routing (called ``Routing Stuff'' throughout the rest of this document) and the low-order eight bytes for unique identification of an end point. The structure of GSE addresses is: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | Routing Goop | STP| End System Designator | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ 6+ bytes ~2 bytes 8 bytes Figure 4 draft-ietf-ipngwg-esd-analysis-00.txt [Page 13] INTERNET-DRAFT March 1997 3.3. Routing Stuff (RG and STP) The Routing Goop (RG) describes the place in the Internet topology where a Site connects, so it is used to route datagrams to the Site. RG is structured as follows: 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | xxx | 13 Bits of LSID | Upper 16 bits of Goop | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3 4 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Bottom 18 bits of Routing Goop | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5 The RG describes the location of a Site's connection by identifying smaller and smaller regions of topology until finally the single link is identified. Before interpreting the bits in the RG, it is important to understand that routing with GSE depends on decomposing the Internet's topology into a specific graph. At the highest level, the topology is broken into Large Structures (LSs). An LS is basically a region that can aggregate significant amounts of topology. Examples of potential LSs are large providers and exchange points. Within an LS the topology is further divided into another graph of structures, with each LS dividing itself however it sees fit. This division of the topology into smaller and smaller structures can recurse for a number of levels, where the trade-off is ``between the flat-routing complexity within a region and minimizing total depth of the substructure.'' [ESD] Having described the decomposition process, we can now examine the bits in the RG. After the 3-bit prefix identifying the address as GSE, the next 13 bits identify the LS. By limiting the field to 13 bits, a ceiling is defined on the complexity of the top-most routing level. In the next 34 bits, a series of subordinate structure(s) are identified until finally the leaf subordinate structure is identified, at which point the remaining bits identify the individual link within that leaf structure. The remaining 14 bits are used for routing structure within a Site, similar to subnetting with IPv4, though these bits are *not* part of the Routing Goop. The distinction between Routing Stuff and Routing Goop is that RG controls routing in transit networks, while Routing Stuff includes the RG plus the Site Topology Partition (STP). The STP is used for routing structure draft-ietf-ipngwg-esd-analysis-00.txt [Page 14] INTERNET-DRAFT March 1997 within a Site. The GSE proposal formalizes the ideas of Sites and of public versus private topology. In the first case, a Site is a set of hosts, routers and media which have one or more connections to the Internet. A Site can have an arbitrarily complicated topology, but all of that complexity is hidden from everyone outside of the Site. A Site only carries packets which originated from, or are destined to, that Site; in other words, a Site cannot be a transit network. A Site is private topology, while the transit networks form the public topology. [Editorial note: An attempt was made to capitalize ``Site'' when assuming the GSE model and lower-case ``site'' when referring to the less formal idea of a site in IPv4.] A datagram is routed through public topology using just the RG, but within the destination Site routing is based on the Site Topology Partition (STP) field. 3.4. End-System Designator The End-System Designator (ESD) is an unstructured 8-byte field that uniquely identifies that end-system from all others. The leading contender for the role of a 64-bit globally unique ESD is the recently defined ``EUI-64'' identifier [EUI64]. These identifiers consist of a 24-bit ``company_id'' concatenated with a 40-bit ``extension.'' (Company_id is just a new name for the Organizationally Unique Identifier (OUI) that forms the first half of an 802 MAC address.) Manufacturers are expected to assign locally unique values to the extension field, guaranteeing global uniqueness for the complete 64-bit identifier. A range of the EUI-64 space is reserved to cover pre-existing 48-bit MAC addresses, and a defined mapping insures that an ESD derived from a MAC address will not duplicate the ESD of a device that has a built-in EUI-64. The mapping of MAC addresses into EUI-64 identifiers is as follows: a 48-bit MAC address xx-xx-xx-yy-yy-yy is mapped into the 64-bit EUI-64 identifier xx-xx-xx-FF-FE-yy-yy-yy. The existence of the reserved range and defined mapping of 48-bit MAC addresses makes the EUI-64 a seemingly ideal ESD candidate. ESDs derived from existing IEEE MAC addresses should be compatible with future network media that use EUI-64 as station identifiers (e.g., FireWire, Futurebus+, SCI). In some cases, interfaces may not have access to an appropriate MAC address or EUI-64 identifier. A globally unique ESD must then be obtained through some alternate mechanism. Any organization draft-ietf-ipngwg-esd-analysis-00.txt [Page 15] INTERNET-DRAFT March 1997 possessing a valid company_id could sell identifiers out of its allocation, though there may be a requirement that EUI-64's be sold only in the form of an electronically-readable part. The IANA, to choose one example, could generate identifiers using its company_id (00005E hex). Another scheme for an IETF-specific ESD space would be to use one or both of the two lowest-order bits of the first octet as a flag. In a company_id, those two bits are reserved for use as the Global/Local and Individual/Group bits, so if either is set, lack of conflict with an EUI-64 would seem to be assured. The most important feature of the ESD is its global uniqueness. End- points of communication would only care about the ESD; as examples, TCP peers could be identified by the ESD alone, checksums would exclude the RG (the sender doesn't know its RG, so can't include it in the checksum), and on receipt of a datagram only the ESD would be used in testing whether a packet is intended for local delivery. 3.5. Address Rewriting by Border Routers Another fundamental aspect of GSE is that Site border routers rewrite addresses of the packets they forward across the Site/Internet boundary. Within a Site, nodes need not know the RG associated with their addresses. They simply use a designated ``Site-Local RG'' value for internal addresses. When a packet is forwarded to the Public Internet, the border router inserts the appropriate RG into the packet's source address. Likewise, when a packet from the Public Internet is forwarded into a Site, the border router replaces the RG part of the destination address with the designated Site-Local RG. Having border routers rewrite addresses obviates the need for end Sites to renumber --- GSE's approach isn't so much to ease renumbering as to simply make it completely transparent to end Sites. To achieve transparency, the RG(s) by which a Site is known would *not* be known to hosts or routers within the core of that Site. (Note: RG can be plural in the previous sentence because multi-homed Sites are known by multiple RGs.) Instead, the RG(s) for the Site would be known only by the exit router(s), either through static configuration or through a dynamic protocol with the upstream provider. Because end-hosts don't know their RG(s), they don't know their entire 16-byte address(es), so they can't specify the full address in the source fields of packets they originate. Consequently, when datagram leaves a Site, the egress border router fills in the high-order portion of the source address with the appropriate RG. The point of keeping the RG hidden from nodes within the core of a draft-ietf-ipngwg-esd-analysis-00.txt [Page 16] INTERNET-DRAFT March 1997 Site is to ensure the changeability of this value without impacting the Site itself. It is expected that the RG will need to change relatively frequently (e.g., several times a year) in order to support scalable aggregation as the topology of the Public Internet changes. A change to a Site's RG would only require a change at the Site's egress point (or points, in the case of a multi-homed Site); and it's well possible that this change would be accomplished through a dynamic protocol with the upstream provider. Hiding a Site's RG from its internal nodes does not, however, mean that changes to RG have no impact on end Sites. Since the full 16- byte address of a node isn't a stable value (the RG portion can change), a stored address may contain invalid RG and be unusable if it isn't ``refreshed'' through some other means. For intra-Site communication, however, it is expected that only the Site-Local RG would be used (and stored) which would always work for intra-Site communication regardless of changes to the Site's external RG. In addition to rewriting source addresses upon leaving a Site, destination addresses are rewritten upon entering a Site. To understand the motivation behind this, consider a Site with three connections. Because each of those connections has its own RG, each destination within the Site would be known by three different 16-byte addresses. As a result, intra-Site routers would have to carry a routing table three times larger than expected. Instead, GSE proposes replacing the RG in inbound packets with the special ``Site-local RG'' value to reduce intra-Site routing tables to the minimum necessary. To be clear, when a node initiates a flow to a node in another Site, the initiating node knows the full 16-byte address for the destination through some mechanism like a DNS query. So this initiating node places the full 16-byte address in the destination address field of the datagram, and that field stays in tact through the first Site and through all of the Public Topology; it is only when the datagram arrives at the destination Site that the RG portion of the destination address is rewritten with the distinguished ``Site-Local RG'' value. When the destination host needs to send return traffic, that host will also know the full 16-byte address for the destination because it appeared in the source address field of the arriving packet. 3.6. Renumbering and Rehoming Mid-Level ISPs One of the most difficult-to-solve components of the renumbering problem is that of renumbering mid-level service providers. Specifically, if SmallISP1 changes its transit provider from BigISP1 draft-ietf-ipngwg-esd-analysis-00.txt [Page 17] INTERNET-DRAFT March 1997 to BigISP2, then all of SmallISP1's customers would have to renumber into address space covered by an aggregate of BigISP2 (if the overall size of routing tables is to stay the same). GSE deals with this problem by handling the RG in DNS with indirection. Specifically, a Site's DNS server specifies the RG portion of its addresses by referencing the *name* of its immediate provider, which is a resolvable DNS name (this obviously implies a new Resource Record type). That provider may define some of the low-order bits of the RG and then reference its immediate provider. This chain of reference allows mid-level service providers to change transit providers, and the customers of that mid-level will simply ``inherit'' the change in RG. 3.7. Support for Multihomed Sites GSE defines a specific mechanism for providers to use to support multi-homed customers that gives those customers more reliability than singly-homed Sites, but without a negative impact on the scaling of global routing. This mechanism is not specific to GSE and could be applied to any multi-homing scenario where a site is known by multiple prefixes (including provider-based addressing). Assume the following topology: Provider1 Provider2 +------+ +------+ | | | | | PBR1 | | PBR2 | +----x-+ +-x----+ | | RG1 | | RG2 | | +--x-----------x--+ | SBR1 SBR2 | | | +-----------------+ Site Figure 6 PBR1 is Provider1's border router while PBR2 is Provider2's border router. SBR1 is the Site's border router that connects to Provider1 while SBR2 is the Site's border router that connects to Provider2. Imagine, for example, that the line between Provider1 and the Site goes down. Any already existing flows that use a destination address including RG1 would stop working. In addition, any DNS queries that return addresses including RG1 would not be viable addresses. If PBR1 and PBR2 knew about each other, however, then in this case PBR1 could draft-ietf-ipngwg-esd-analysis-00.txt [Page 18] INTERNET-DRAFT March 1997 tunnel packets destined for RG1-prefixed addresses to PBR2, thus keeping the communication working. 3.8. Explicit Non-Goals for GSE It is worth noting explicitly that GSE does not attempt to address the following issues: 1) Survival of TCP connections through renumbering events. If a Site is renumbered, TCP connections using a previous address will continue to work only as long as the previous address still works (i.e., while it is still "valid" using RFC 1971 terminology). No attempt is made to have existing connections switch to the new address. 2) It is not known how mobility can be made to work under GSE. 3) It is not known how multicast can be made to work under GSE. 4) It is not known whether the performance cost of having routers rewrite portions of the source and destination address in packet headers is acceptable. That GSE doesn't address the above does not mean they cannot be solved. Rather the issues haven't been studied in sufficient depth. 4. Analysis of GSE's Advantages and Disadvantages 4.1. End System Designator 4.1.1. IP Addresses in the IPv4 Internet As described earlier, in the IPv4 Public Internet, IP addresses contain two pieces of information: a unique identifier and a locator. A key aspect of the embedded location information is that it must be aggregable, so that a single routing table entry can cover many destination addresses. In practice, this means that sites that are topologically close to each other must share a common prefix, as exemplified in provider-based addressing [RFC 2073] and CIDR [RFC1817]. Without sufficient aggregation, routing in the Public Internet can not scale [RFC2008]. Note that embedding location information within an address has the side-effect of helping ensure that all addresses are globally unique. draft-ietf-ipngwg-esd-analysis-00.txt [Page 19] INTERNET-DRAFT March 1997 If interfaces on two different nodes are assigned the same unicast address, the routing subsystem will (generally) deliver packets to only one of those nodes. The other node will quickly realize that something is wrong (since communication using the duplicate address fails) and take corrective action (e.g., obtain a proper address). Embedding location information within an address also provides some, though not much, protection from forged addresses. Although it is trivial to forge a source address in today's Internet, the routing subsystem will in most cases forward any return traffic sent to that address to its proper destination --- not to an arbitrary node masquerading as someone else. To masquerade as someone else requires subverting the routing subsystem, placing the intruder somewhere on the normal routing path between the masqueraded host and its peer, etc. Allowing the Routing Stuff and ESD portions of an address to be changed independent of each other potentially increases the ease with which packets intended for a particular ESD can be misrouted or hijacked elsewhere. As discussed in later sections, additional checks must be made to reduce the threat of hijacking. 4.1.2. Overloading Addresses: Network Layer Issues Embedding location information within an address has some important consequences. At the network layer, a node compares the destination address of received packets against the addresses of its attached interfaces. Only if the addresses of received packets match are packets handed up to higher layer protocols. The entire address (including the Routing Stuff part) must match. Otherwise, the packet is assumed to be intended for some other node and forwarded on (if received by a router) or silently discarded (if received by a host). This has subtle but significant implications: 1) If a receiving host has multiple interfaces, it has multiple IP addresses. When a packet addressed to a multi-homed host is received on an interface other than the one to which a packet is addressed, the host may reject (i.e., silently discard) the packet, if it implements the ``Strong ES Model'' defined in [RFC1122]. 2) In IPv6 (and recent IPv4 stacks), an interface may have more than one unicast IP address assigned to it. Indeed, one way to renumber an end site is to phase out an address (i.e., "deprecate" it using RFC 1971 termininology) while simultaneously phasing in a new one. Once the deprecated address becomes invalid, packets sent to the invalid address will no longer be accepted by the node, even though the packet may have intuitively reached its intended recipient. Thus, even if a draft-ietf-ipngwg-esd-analysis-00.txt [Page 20] INTERNET-DRAFT March 1997 packet sent to an invalid address is somehow delivered to the intended recipient (e.g., via tunneling), the receiver would reject the packet because the address it was sent to no longer belongs to any of the node's interfaces. Consequently, any communication using the invalid address will fail (e.g., new and existing TCP connections). Anyone wishing to communicate with the node must learn and switch to the new address. 3) Because an address also indicates ``where'' the destination resides within the Internet, a mobile node that moves from one part of the Internet to another must obtain a new address that reflects its new location. Moreover, the routing subsystem will continue to forward packets sent to the mobile node's previous address to the node's previous point of attachment where they are likely be discarded. That is, even if a mobile node is willing to continue accepting packets addressed to one its previous addresses, it is unlikely that they will be received (in the absence of something like Mobile IP [RFC2002]). 4) A multi-homed host has multiple interfaces, each with its own address(es). If one of its interfaces fails, packets could, in theory, be delivered to one of the host's other interfaces. In practice, however, the routing subsystem has no way of knowing that the interface to which a packet is addressed has failed and what alternate interface addresses the packet could be delivered to. Consequently, packets sent to a multi-homed host won't be delivered to the intended recipient, even though the node is reachable (through an alternate address). Note that the above problems fall into two general categories: 1) Today's routing subsystem is unable to automatically deliver a packet to a host's ``alternate'' addresses (if the host is multi-homed) or a new address (if the host moves), should there be a problem delivering a packet to the destination address listed in the packet. It is possible to imagine, however, future routing advances addressing this problem (e.g., Mobile IP). 2) Even if a packet is delivered to its intended destination, the packet may still be rejected because the packet's destination address does not match any of the addresses assigned to destination's interfaces. This problem does not appear to be insurmountable and could be rectified (for example) by having a host remember its previous addresses. draft-ietf-ipngwg-esd-analysis-00.txt [Page 21] INTERNET-DRAFT March 1997 4.1.3. Overloading Addresses: Transport Layer Issues Although the problems discussed previously appear to have viable solutions, additional complications occur at the transport level. Transport protocols such as TCP and UDP use embedded IP addresses to identify the end points of a transport connection. Specifically, the communicating end points of a transport connection are uniquely identified by the sender's source IP address and source port number together with the recipient's destination IP address and port number. Once a connection has been established, the IP addresses can not change. In particular, if a mobile host moves to a new location and obtains a new address, packets intended for a TCP connection created prior to the move cannot use the new address. TCP will treat any packets sent to the new address as belonging to a different TCP connection. It is possible to imagine changes to TCP that might allow connections to change the addresses they are using mid-connection without breaking the connection. However, some subtle issues arise: 1) Packets intended for a pre-existing connection must be demultiplexed to that connection as part of any negotiation to change the addresses that identify that transport end point. However, because the demultiplexing operation uses the transport addresses of the pre-existing TCP connection (which is based on the previous address), TCP packets sent to a new address won't be delivered to the desired transport end point (which still uses the previous address). Consequently, packets would need to be sent to the previous address. However, by the time a mobile node has moved and knows its new address, packets sent to the previous address may no longer be delivered (i.e., they may not be forwarded to the mobile host's new location). 2) When a mobile host moves, it could inform its TCP peers that it has a new address. However, such a message could not be delivered to the remote TCP connection if it was sent using its new address for its source address. Just as above, such packets would not be demultiplexed to the correct TCP connection. On the other hand, there are difficulties if it attempts to send packets using its previous address from its new location. Because of the danger of spoofing attacks, routers are now encouraged to actively look for, and discard traffic from, a source address that does not match known addresses for that region of the Internet [CERT]. Consequently, such packets cannot be expected to be delivered. Although the previous discussion used mobile nodes as an example, the same problem arises in other contexts. For example, if a site is draft-ietf-ipngwg-esd-analysis-00.txt [Page 22] INTERNET-DRAFT March 1997 being renumbered in IPv6, it may have two addresses, a previous (i.e., deprecated) one being phased out and a new (i.e., preferred) one being phased in. At the transport level, the problem of switching addresses is similar in many respects to the mobility problem. 4.1.4. Benefits of Globally Unique ESDs An alternate approach is to break an address into two distinct portions: 1) An End System Designator (ESD) that uniquely identifies an end point of communication (independent of the interface through which that was reached). Such an identifier should be globally unique so that a node that receives a packet can definitively determine whether the packet is intended for it by comparing only the ESD portion of the address. 2) A ``locator'' or Routing Stuff that is used by the routing subsystem to deliver a packet to the appropriate end system identified by the ESD. Having a clear separation between the Routing Stuff and the ESD portion of an address gives protocols some additional flexibility. At the network layer, for example, recipients can examine just the ESD portion of the destination addresses when determining whether a packet is intended for them. This means that if a packet is delivered to the correct destination node, the node will accept the packet, regardless of how the packet got there, i.e., without regard to the Routing Stuff of the address, which interface it arrived on, etc. Excluding the Routing Stuff of an address when making address comparisons also makes it possible to change the Routing Stuff of an address to reflect a mobile node's new location, or an alternate interface on a multi-homed host. Such packets would then be delivered and accepted by the target host. The idea of using addresses that cleanly separate the Routing Stuff from an ESD is not new [references XXX]. However, there are several different flavors. In its pure form, a sender would only need to know the ESD of an end point in order to send packets to it. When presented with a datagram to send, network software would be responsible for finding the Routing Stuff associated with the ESD so that the packet can be delivered. A key question, then, is who is responsible for finding the Routing Stuff associated with a given ESD? There are a number of possibilities: 1) The network layer could be responsible for doing the mapping. The advantage of such a system is that an ESD could be stored draft-ietf-ipngwg-esd-analysis-00.txt [Page 23] INTERNET-DRAFT March 1997 essentially forever (e.g., in configuration files), but whenever it is actually used, network layer software could automatically perform the mapping to determine the appropriate Routing Stuff for the destination. Likewise, should an existing mapping become invalid, network layer software could dynamically determine the updated quantity. Unfortunately, building such a mapping mechanism that is scalable is a hard problem. 2) The transport layer could be responsible for doing the mapping. It could perform the mapping when a connection is first opened, periodically refreshing the binding for long-running connections. Implementing such a scheme would change the existing transport layer protocols TCP and UDP significantly. 3) Higher-layer software (e.g., the application itself) could be responsible for performing the mapping. This potentially increases the burden on application programmers significantly, especially if long-running connections are required to survive renumbering and/or deal with mobile nodes. It should be noted that the GSE proposal [GSE] does not embrace the general model. Indeed, it proposes the last. The network layer (and indeed the transport layer) is always presented both the Routing Stuff (RG + STP) and the ESD together in one IPv6 address. It is not the network (or transport) layer's job to determine the Routing Stuff given only the ESD. When an application has data to send, it queries the DNS to obtain the IPv6 AAAA record for a destination. The returned AAAA record contains both the Routing Stuff and the ESD of the specified destination. While such an approach eliminates the need for the lower layers to be able to map ESDs into corresponding Routing Stuff, it also means that when presented with an address containing an incorrect (i.e., no longer valid) Routing Stuff, the network will be unable to deliver the packet to its correct destination. It is up to applications themselves to deal with such failures. Note that addresses containing invalid Routing Stuff will result any time cached addresses are used after the Routing Stuff of the address becomes invalid. This may happen if addresses are stored in configuration files, or with long-running communication. 4.1.5. ESD: Network Layer Issues Along with the flexibility offered by separating the ESD from the Routing Stuff come additional considerations that must be considered at the network layer: 1) If a receiver observes that recent packets are arriving with a different Routing Stuff in the source address than before, it draft-ietf-ipngwg-esd-analysis-00.txt [Page 24] INTERNET-DRAFT March 1997 may want to send return traffic using the new Routing Stuff. However, such information should not be accepted without appropriate authentication of the new Routing Stuff, otherwise it would be trivial to hijack existing transport connections. Always using the most recently received Routing Stuff of an address to send return traffic without appropriate authentication leads to a vulnerability that is equivalent in potential danger to ``reversing and using an unauthenticated received source route.'' Note also that in the GSE proposal, since a sender does not know their own RG, it is not possible for the sender to compute an Authentication Header via IPSEC that covers the RG portion of an address. Thus, a recipient of new RG would need to authenticate the received information via some alternate (undefined) mechanism. Finally, receipt of packets from different Routing Stuff than before does not necessarily indicate a permanent change. In the GSE proposal, for example, when a Site is multi-homed, some of its packets may exit via one egress router and other packets via a different egress router. Even packets originated from the same source may exit through multiple egress routers. Consequently, a node may receive traffic from the same sender in which the Routing Stuff part changes on every packet. 2) In general, whenever an address is embedded within a packet (including within data), one must consider whether all the bits in the address should be used in computations, or whether just the ESD portion should be used. Examples where such decisions would need to be made include, but are not limited to, Neighbor Discovery packets containing Neighbor Solicitations and Responses [RFC 1970], IPSEC packets being demultiplexed to their appropriate Security Association, IP deciding whether to accept an IP datagram (before reaching the transport level), the reassembly of fragments, transport layer demultiplexing of received packets to end points, etc. 4.1.6. ESD: Transport Layer Issues Previous sections have made clear that the embedding of full IPv6 addresses (i.e., Routing Stuff) within transport connection endpoint identifiers poses problems for mobility and site renumbering. This section discusses an alternate approach, in which transport endpoint identifiers use ESDs rather than full addresses (with embedded Routing Stuff). draft-ietf-ipngwg-esd-analysis-00.txt [Page 25] INTERNET-DRAFT March 1997 In the following discussion, it should be kept in mind that the IPng Recommendation [RFC 1752] states that a transition to IPv6 cannot also require deployment of a ``TCPng'', that is, make changes to TCP that decrease its overall robustness. In addition, although we focus on TCP, UDP-based protocols also depend on the Routing Stuff in similar ways, e.g., starting with the UDP checksum of the peers' addresses. Indeed, we believe that TCP is the ``easy'' case to deal with, for two reasons. First, TCP is a stateful protocol in which both ends of the connection can negotiate with each other. Some UDP- based protocols are stateless, and remember nothing from one packet to the next. Consequently, changing UDP-based protocols may require the introduction of ``session'' features, perhaps as part of a common ``library'', for use by applications whose transport protocol is relatively stateless. Second, changes to UDP-based protocols in practice mean changing individual applications themselves. 4.1.6.1. Dumultiplexing Packets to Transport Endpoints Connections in GSE are identified by the ESDs rather than full IPv6 addresses (with embedded Routing Stuff). That is: unique TCP connection: srcaddr dstaddr srcport destport unique GSE TCP connection: srcESD dstESD srcport dstport Consequently, when demultiplexing incoming packets, TCP would ignore the Routing Stuff portions of addresses when delivering packets to their proper end point. Although there are potential benefits to this approach (discussed below), demultiplexing of ESDs is in fact a requirement with GSE. If a site is multihomed, the packets it sends may exit different egress border routers during the lifetime of a connection. Because each border router will place its own RG into the source addresses of such packets, the receiving TCP must ignore (at least) the RG portion of addresses when demultiplexing received packets. The alternative would be to make TCP less robust with respect to changes in routing, i.e., if the path changed, packets delivered correctly would be discarded by the receiving TCP rather than processed. 4.1.6.2. Pseudo-Header Checksum Calculations Having routers rewrite the RG portion of addresses means that TCP cannot include the RG in its checksum calculation; the sender does not know its own RG. Consequently, upon receipt of a TCP segment, the receiver has no way of determining whether the RG portion of an address has been corrupted (or modified) in transit (the implications of this are discussed below). draft-ietf-ipngwg-esd-analysis-00.txt [Page 26] INTERNET-DRAFT March 1997 4.1.6.3. RG Selection When Sending Packets When a host has a packet to send, what RG should it use? There are three cases. If the host is performing an ``active open'', it queries the DNS to obtain the destination address, which contains appropriate RG. If the host is responding to an active open from a remote peer, the source address of packets from that peer contain usable RG. The interesting case is when RG changes mid-connection. 4.1.6.4. Mid-Connection RG Changes During a connection, the RG appearing on subsequent packets is susceptible to change through renumbering events, and indeed more frequently, to change through Site-internal routing changes that cause the egress point for off-Site traffic to change. It is even possible that traffic balancing schemes could result in the use of two egress routers, with roughly every other packet exiting through a different egress router. Consequently it may be desirable to switch to the just-received RG, as the old RG may no longer be valid (e.g., a border router has failed). However, simply using the most- recently-received RG makes it trivial to hijack connections. The way TCP packets are demultiplexed under GSE, they will be delivered to the correct endpoint even though TCP may send to its peer at a deprecated RG or one that is less optimal because the peer's Border Router has changed. It would seem highly desireable for TCP connections to be able to survive such events. However, the completion of renumbering events (so that an earlier RG is now invalid) and certain topology changes would require TCP to switch sending to a new RG mid-connection. To explore the whole space, we considered ways of allowing this mid-connection RG change. If TCP connection identifiers are based on ESDs rather than full addresses, traffic from the same ESD would be viewed as coming from the same peer, regardless of its source RG. This makes it trivial for any Internet host to impersonate another, and have such traffic be accepted by TCP. Because this vulnerability is already present in today's Internet (forging full source addresses is trivial), the mere delivery of incoming datagrams with the same ESD but a different RG does not introduce new vulnerability to TCP. In today's Internet, any node can already originate FINs/RSTs from an arbitrary source address and potentially or definitely disrupt the connection. Therefore, changing RG for acceptance, or acceptance of traffic independent of its source RG, does not significantly worsen existing robustness, as far as our analysis has gone. We also considered allowing TCP to reply to each segment using the RG draft-ietf-ipngwg-esd-analysis-00.txt [Page 27] INTERNET-DRAFT March 1997 of the most recently-received segment. Although, this allows TCP to survive some important events (e.g., renumbering), it also makes it trivial to hijack connections, unacceptably weakening robustness compared with today's Internet. A sender simply needs to guess the sequence numbers in use by a given TCP connection [Bellovin 89] and send traffic with a source RG that redirects responses to the intruder's current location. Providing protection from hijacking implies that the RG used to send packets must be bound to a connection end point (e.g., it is part of the connection state). Although it may be reasonable to accept incoming traffic independent of the source RG, the choice of sending RG requires more careful consideration. Indeed, any subsequent change in what RG is used for sending traffic must be properly authenticated using cryptographic means. In the GSE proposal, it is not clear how to authenticate such a change, since the remote peer doesn't even know what RG it is using! Consequently, the only reasonable approach in GSE is to send to the peer at the first RG used by the peer for the entire life of a connection. That is, always continue to use the first RG used. One question that arises is what impact corrupted RG would have on robustness. Because the RG is not covered by any checksums, it would be difficult to detect such corruption. Moreover, once a specific RG is in use, it does not change for the duration of a connection. The interesting case occurs on the passive side of a TCP connection, where a server accepts incoming connections from remote clients. If the initial SYN from the client includes corrupted RG, the server TCP will create a TCP connection (in the SYN-RECEIVED state) and cache the (corrupted) RG with the connection. The second packet of the 3- way handshake, the SYN-ACK packet, would be sent to the wrong RG and consequently not reach the correct destination. Later, when the client retransmits the (unacknowledged) SYN, the server will continue to send the SYN-ACK using the bad RG. Eventually the client times out, and the attempt to open a TCP connection fails. Figure 7 shows the details. draft-ietf-ipngwg-esd-analysis-00.txt [Page 28] INTERNET-DRAFT March 1997 TCP A TCP B 1. CLOSED LISTEN 2. SYN-SENT -->--> SYN-RECEIVED 3. <-- <-- SYN-RECEIVED 4. SYN-SENT --> --> SYN-RECEIVED 5. <-- <-- SYN-RECEIVED ... TCP A times out Figure 7 We next considered relaxing the restriction on switching RGs in an attempt to avoid the previous failure scenario. The situation is complicated by the fact that the RG on received packets may change for legitimate reasons (e.g., a multi-homed site load-shares traffic across multiple Border Routers). The key question is how can one determine which RG is valid and which is not? That is, for each of the RGs a sender attempts to use, how can it determine which RG worked and which did not? Solving this problem is more difficult than first appears, since one must cover the cases of delayed segments, lost segments, simultaneous opens, etc. If a SYN-ACK is retransmitted using different RGs, it is not possible to determine which of those RGs worked correctly. We concluded that the only way TCP could determine that a particular RG was used to deliver segments was if it received an ACK for a specific sequence number in which all transmissions of that sequence number used the same RG. We analyzed multiple cases of RG changing within the time of the opening handshake. One example is diagrammed in Figure 8, and it and two others are summarized in Table 1. We observed that RG flap and large numbers of passive opens may coincide, for instance, when a power failure at a server farm affects both internal routers and servers. draft-ietf-ipngwg-esd-analysis-00.txt [Page 29] INTERNET-DRAFT March 1997 time TCP A time TCP B t0 --> t1 t3 <-- t1 TCP B's SYN,ACK is delayed and crosses with retransmit of TCP A's SYN on which RG has changed from M to N t2 --> t3 t4 --> t3 ESTABLISHED TCP B decides to use DST RG=M for TCP A, because it heard from RG=M and was ACK'd on a send to RG=M Figure 8 SYNFROM SYNACKTO ACKFROM SELECT W W X W ------------------------------------ W X W X W ------------------------------------ W W X X Y ?? Table 1 At best, an RG selection algorithm for TCP would be relatively straightforward but would require new logic in implementations of TCP's opening handshake --- a significant transition issue. We are not certain that a valid algorithm is attainable, however. RG changes would have to be handled in all cases handled by the opening handshake: delayed segments, lost segments, undetected bit errors in RG, simultaneous opens, old segments, and so on. In the end, we concluded that although the corrupted SYN case of Figure 7 was a potential problem, the changes that would need to be made to TCP to robustly deal with such corruption would be significant, if tractable at all. This would result in transition to GSE needing a significant TCPng transition. Our final conclusion is that transport protocol end points must make an early, single choice of the RG to use when sending to a peer and draft-ietf-ipngwg-esd-analysis-00.txt [Page 30] INTERNET-DRAFT March 1997 stick with that choice for the duration of the connection. Specifically: 1) The demultiplexing of arriving packets to their transport end points should use only the ESD, and not the Routing Stuff. 2) If the application chooses an RG for the remote peer (i.e., an active open), use the provided RG for all traffic sent to that peer, even if alternative RGs are received on subsequent incoming datagrams from the same ESD. 3) For all other cases, use the first RG received with a given ESD for all sending. We recommend that a means be found for RGs to be checksummed if the GSE address structure is used. 4.1.6.5. Duplicate ESDs with Differing RGs Another interesting case occurs if two different (client) nodes are using the same ESD, and they attempt to communicate with a common server. In such cases, the RG (or STP) portions of the address may be different. However, since only the ESD is used in demultiplexing packets to their transport end points, traffic from two different hosts may be delivered and processed by one transport endpoint. Given the above rules that bind RG to existing connections, only one RG will be used, and all traffic from the server will be sent to the same client. It would appear that in most cases, only one connection would reach ESTABLISHED state, and the others would time out. One implication of binding RG information to TCP connection state is that we may be opening the door to additional security threats. One denial of service attack, for instance, would be for an intruder to masquerade as another host and ``wedge'' connections in a SYN- RECEIVED state by sending SYN segments containing on invalid RG in the source IP address. Subsequent connection attempts to the wedged host from the legitimate party (if they used the same TCP port numbers) would then not complete, since return traffic would be sent to the wrong place. 4.1.6.6. Summary: ESD and RG Not Strictly Independent We cannot emphasize enough that the use of an ESD independent of an associated RG can be very dangerous. That is, communicating with a peer implies that one is always talking to the same peer for the duration of the communication. But as has been described in previous sections, such assurance can only take place if there are assurances that only properly authenticated RG is used --- RG authenticated by draft-ietf-ipngwg-esd-analysis-00.txt [Page 31] INTERNET-DRAFT March 1997 the peer. Consequently, we conclude that the rules for transport processing when ESDs are present differ from classical IP. Specifically: 1) The demultiplexing of packets to transport connection end points should use ESDs, but should not use the Routing Stuff part of addresses. 2) Once a packet has been delivered to its transport endpoint, that packet should not be processed without first examining the source RG used. Whether (and how) the information in the received packet is used is dependent on the transport protocol itself. A protocol could chose to completely ignore the packet, it could selectively use parts of the packet (e.g., to attempt out-of-band authentication of the RG), or it could process the packet in its entirety. It must not, however, use the received RG to send subsequent return traffic without first authenticating the RG. 4.1.7. ESD: Application Layer Issues In this section we define applications as user processes that must exchange data reliably with remote processes. Such distributed processes need system support to reliably identify remote processes. It is desirable (necessary?) for such end identifications to meet the following requirements: 1) The identifier assigned to each end point should be globally unique. 2) This uniqueness should be easily enforceable because it is difficult, or probably impossible, to provide an absolute guarantee on the uniqueness of these identifiers. Applications relying on this uniqueness must be prepared for duplicate detection; at the same time, one must be prepared to detect maliciously forged identifiers. The last point is becoming increasingly important as the Internet continues to grow exponentially, both in the scale of user population and in the scope of diverse applications that cover all walks of life. In the original design of the IPv4 architecture, globally unique IP addresses are used as the globally unique identifiers for all interfaces reachable on the Internet. Thus, a user process easily obtains a globally unique identifier by attaching a local port number draft-ietf-ipngwg-esd-analysis-00.txt [Page 32] INTERNET-DRAFT March 1997 to the address. One fundamental architectural change suggested by GSE is to split the address into completely separate interface identifiers and locators (routing stuff). In the rest of this section we discuss the pros and cons of each of the two approaches. 4.1.7.1. The Impacts of Address and Identifier Overloading In IPv4 Because global IP addresses are unique, using addresses as identifiers automatically provides the needed uniqueness property of an identifier. Moreover, duplicates can be easily detected. If two more interfaces claim the same unicast address, then due to shortest path routing, packets destined to that IP address are, generally speaking, delivered only to the interface closest to the source. Nodes with duplicate addresses can detect the error by observing the lack of connectivity to certain locations. Furthermore, using addresses as identifiers makes forging difficult. If a malicious node inserts a false source address in its outgoing packets, although the packets are likely to be delivered to the destination host, it is almost impossible for the malicious node to receive any reply data. We can further prevent the packets with faulty source address from being delivered by making all routers perform reverse source address checking, that is, checking each incoming packet to see if it comes from the correct interface as indicated by the routing table, a practice enforced by all IP multicast routing protocols. Such source address checking provides a simple, universal and effective enforcement to correct interface identifications. Even if some routers are compromised and allow packets with false source addresses to pass through, delivering packets with forged source addresses to the destination would require that all routers along the path be compromised. The fundamental disadvantage of overloading addresses with identification information is that changes in addresses lead to changes in identification, which implies that all hosts will be aware of all address changes, an issue GSE is designed to resolve. It is worth noting that keeping TCP connections running across renumbering is a non-goal of GSE design. 4.1.7.2. The Impact of Separating Locators and Identifiers GSE uses the upper 8 bytes of IPv6 addresses as locators and the lower 8 bytes as globally unique identifiers (ESD). The chief advantage of this complete separation of Routing Stuff and ESD is a stable identifier per interface, independent from all renumbering changes in a network with a provider-based addressing architecture. draft-ietf-ipngwg-esd-analysis-00.txt [Page 33] INTERNET-DRAFT March 1997 One intention of the GSE design is to use the ESD alone in establishing and maintaining TCP connection state. The discussions at the interim meeting, however, revealed significant resistance to doing this. The consensus from the meeting was that it is not adequate to use an ESD alone for end point identification due to the ease of hijacking a TCP connection. Incoming packets with a wrong ESD can easily be detected as coming from an incorrect source, however, incoming packets with a correct ESD cannot be easily trusted as being from the correct source. Various approaches to using RG as part of TCP source verification were discussed at the meeting. Using ESDs as end point identifications seems to require two steps of processing. In the first step, the ESD can be used for PCB lookups. In the second step, the entire address (including RG) must be considered because one cannot safely take packets with an arbitrary RG. So the purpose of the first step is to locate the intended end point of an incoming packet. The second step then can make a separate decision as to how to act on the received data (accept, reject or perform out-of-band authentication on the RG). Due to the conflict between applications' desire to use RG information for remote end checking and GSE's desire to hide RG from hosts, however, none of the approaches can satisfy both desires at the same time. Thus the disadvantages of the Routing Stuff and ESD separation comes directly from this separation. We believe that neither the global uniqueness of ESDs nor their correct use is enforceable, thus easy detection of wrong ESDs becomes the key. Unfortunately, short of using IPSEC for every IP packet delivery, using ESD alone loses the advantage of easy forge detection that comes from the address overloading in IPv4 design. 4.1.8. When ESDs are Not Unique In IPv4, communication usually fails quickly when addresses are not unique. There are two cases to consider, depending on whether the two interfaces assigned duplicate addresses are attached to the same or to different links. When two interfaces on the same link use the same address, a node (host or router) sending traffic to the duplicate address will in practice send all packets to one of the nodes. On Ethernets, for example, the sender will use ARP (or Neighbor Discovery in IPv6) to determine the link layer address corresponding to the destination address. When multiple ARP replies for the target IP address are received, the most recently received response replaces whatever is already in the cache. Consequently, the destinations a node using a draft-ietf-ipngwg-esd-analysis-00.txt [Page 34] INTERNET-DRAFT March 1997 duplicate IP address can communicate with depends on what its neighboring nodes have in their ARP caches. In most cases, such communication failures become apparent relatively quickly, since it is unlikely that communication can proceed correctly on both nodes. It is also the case that a number of ARP implementations (e.g., BSD- derived implementations) log warning messages when an ARP request is received from a node using the same address as the machine receiving the ARP request. When two interfaces on different links use the same address, the routing subsystem will generally deliver packets to only one of the nodes because only one of the links has the right ``prefix'' or ``subnet part'' corresponding to the IP address. Consequently, the node using the address on the ``wrong'' link will generally never receive any packets sent to it and will be unable to communicate with anyone. For obvious reasons, this condition is usually detected quickly. An important observation is that, with classical IP, when different nodes mistakenly assign the same IP address to different interfaces, problems become apparent relatively quickly because communication with several (if not all) destinations fails. In contrast, failure scenarios differ when globally unique ESDs are assumed, but two nodes mistakenly select the same one. At first glance it might appear that two nodes using the same ESD cannot communicate. However, this is not necessarily the case. In the GSE proposal, for example, a node queries the DNS to obtain an IPv6 address. The returned address includes the Routing Stuff of an address (the RG+STP portions). Since the sending host transmits packets based on the entire destination IPv6 address, the sender may well forward the packet to a router that delivers the packet to its correct destination (using the information in the Routing Stuff). It is only on receipt of a packet that a node would extract the ESD portion of a datagram's destination address and ask ``is this for me?'' A more problematic case occurs if two nodes using the same ESD communicate with a third party. To the third party, packets received from either machine might appear to be coming from the same machine since they are both using the same ESD. Consequently, at the transport level, if both machines choose the same source and destination port numbers (one of the ports --- a server's well-known port number will likely be the same), packets belonging to two distinct transport connections will be demultiplexed to a single transport end point. draft-ietf-ipngwg-esd-analysis-00.txt [Page 35] INTERNET-DRAFT March 1997 When packets from different sources using the same source ESD are delivered to the same transport end point, a number of possibilities come to mind: 1) The transport end point could accept the packet, without regard to the Routing Stuff of the source address. This may lead to a number of robustness problems, if data from two different sources mistakenly using the same ESD are delivered to the same transport or application end point (which at best will confuse the application). 2) The transport end point could verify that the Routing Stuff of the source address matches one of a set of expected values before processing the packet further. If the Routing Stuff doesn't match any expected value, the packet could be dropped. This would result in a connection from one host operating correctly, while a connection from another host (using the same ESD) would fail. 3) When a packet is received with an unexpected Routing Stuff the receiver could invoke special-purpose code to deal with this case. Possible actions include attempting to verify whether the Routing Stuff is indeed correct (the saved values may have expired) or attempting to verify whether duplicate ESDs are in use (e.g., by inventing a protocol that sends packets using both Routing Stuff and verifies that they go are delivered to the same end point). 4.1.9. DNS PTR Queries IPv4 uses the top-level domain ``IN-ADDR.ARPA'' to hold PTR Resource Records. PTR RRs allow a client to map IP addresses back into the domain name corresponding to that address. IPv4 addresses can be put into the DNS because they have hierarchical structure -- the same hierarchy used to aggregate routes. The ability to map an IP address into its corresponding DNS name is used in several contexts: 1) Network packet tracing utilities (e.g., tcpdump) display the contents of packets. Printing out the DNS names appearing in those packets (rather than dotted IP addresses) requires access to an address-to-name mapping mechanism. 2) Some applications perform ``cheap'' authentication by using the DNS to map a source address of a peer into a DNS name. Then, the client queries the DNS a second time, this time asking for the draft-ietf-ipngwg-esd-analysis-00.txt [Page 36] INTERNET-DRAFT March 1997 address(es) corresponding to the peer's DNS name. Only if one of the addresses returned by the DNS matches the peer address of the TCP connection is the source of the TCP connection accepted as being from the indicated DNS name. It is important to note that although two DNS queries are made during the above operation, it is the second one --- mapping the peer's DNS name back into an IP address --- that provides the authentication property. The first transaction simply obtains the peer's DNS name, but no assumption is made that the returned DNS name is correct. Thus, the first DNS query could be replaced by an alternate mechanism without weakening the (already weak) authentication check described above. One possible alternate mechanism, an ICMP ``Who Are You'' message, is described in Section 4.1.12. 3) Applications that log all incoming network connections (e.g., anonymous FTP servers) may prefer logging recognizable DNS names to addresses. 4) Network administrators examining logs or other trace data containing addresses may wish to determine the DNS name of some addresses. Note that this may occur sometime after those addresses were actually used. Although DNS PTR records have proven useful in several contexts, there is also widespread agreement that, in practice, most of the IP addresses in use today are not properly registered in the PTR hierarchy. Consequently, more often than not, PTR queries fail to return usable information. Thus, the overall utility of PTR records is questionable. It is also worth noting that the primary reason that so few addresses are properly registered in the PTR space is the lack of incentive for doing so. With no key piece of the Internet infrastructure depending on such mappings being in place (or correct), there is little harm in failing to keep it up-to-date. Finally, it might appear at first glance that secure DNS [RFC2065] provides a means for cryptographically signing a PTR record and thereby providing authentication. Things are not so simple, however. The signature on a PTR record indicates that the entity owning an address has given it a DNS name. It does not mean that the owner of the address is authorized to use that specific name. For example, anyone owning an address can set up a PTR record indicating that the address corresponds to the name ``www.ietf.org''. However, the name ``www.ietf.org'' belongs to only one entity, regardless of how many PTR records indicate otherwise. draft-ietf-ipngwg-esd-analysis-00.txt [Page 37] INTERNET-DRAFT March 1997 4.1.10. Reverse Mapping of ESDs If an end point is identified via an ESD rather than by its full address, do we need the ability map the ESD alone into some other meaningful quantity, such as a fully qualified domain name? The benefits of being able to perform such a mapping are analogous to those described in the preceding section. The primary difficulty with constructing such a mapping is that it requires that ESDs have sufficient structure to support the delegating mechanism of a distributed database such as DNS. The sorts of built-in identifiers now found in computing hardware, such as ``EUI-48'' and ``EUI-64'' addresses [IEEE802, IEEE1212], do not have the structure required for this delegation. Hence, stateless autoconfiguration [RFC1971] cannot create addresses with the necessary hierarchical property. Another possibility would be to define ESDs with sufficient structure to permit the construction of a mapping mechanism. However, analysis performed during the IPng deliberations concluded that close to 48- bits of hierarchy were needed to identify all the possible sites 30-40 years from now. That would leave only 2 bytes for host numbering at a site, a number clearly incompatible with stateless autoconfiguration [RFC1971]. There are several arguments against requiring a global ESD-lookup capability. Adding sufficient structure to an 8-byte ESD requires more bits than are compatible with stateless autoconfiguration. In addition, experience with the IN-ADDR.ARPA domain suggests that the required databases will be poorly maintained. Finally, imposing a required hierarchical structure on ESDs would also introduce a new administrative burden and a new or expanded registry system to manage ESD space. While the procedures for assigning ESDs, which need only organizational and not topological significance, would be simpler than the procedures for managing IPv4 addresses (or DNS names), it is hard to imagine such a process being universally well-received or without controversy; it seems a laudable goal to avoid the problem altogether if possible. Finally, there is an argument based on allocation efficiency. One of the design criteria for IPng was support for 10^9 networks and 10^12 hosts ``and preferably much more'' [RFC1726], and estimates as high as 10^15 hosts have been published. Since GSE uses 64 bits to designate the Site and subnet and the same number to designate the end system, the allocation efficiency for the latter assignment process must be much greater. In terms of the H ratio [RFC1715], the RG+STP portion of the address needs to achieve an H ratio of only 0.14, which is as low a value as any of the example numbering schemes draft-ietf-ipngwg-esd-analysis-00.txt [Page 38] INTERNET-DRAFT March 1997 examined in RFC 1715, while the ESD assignment must achieve 0.19 to 0.23, depending on the total host estimate one accepts. These values are in the middle to high range of the schemes examined, indicating that they represent a feasible efficiency -- achievable with careful management. 4.1.11. Reverse Mapping of Complete GSE Addresses Within a Site, one could imagine maintaining a database keyed on unstructured 8-byte ESDs. However, it is a matter of debate whether such a database could be kept up-to-date at reasonable cost, without making unreasonable assumptions as to how large Sites are going to grow, and how frequently ESD registrations will be made or updated. Note that the issue isn't just the physical database itself, but the operational issues involved in keeping it up-to-date. For the rest of this section, however, let us assume such a database can be built. A mechanism supporting a lookup keyed on a flat-space ESD from an arbitrary Site requires having sufficient structure to identify the Site that needs to be queried. In practice, an ESD will almost always be used in conjunction with Routing Stuff (i.e., a full 16-byte address). Since the Routing Stuff is organized hierarchically, it becomes feasible to maintain a DNS tree that maps full GSE addresses into DNS names, in a fashion analogous to what is done with IPv4 PTR records today. It should be noted that a GSE address lookup will work only if the Routing Stuff portion of the address is correctly entered in the DNS tree. Because the RG portion of an address is expected to change over time, this assumption will not be valid indefinitely. As a consequence, a packet trace recorded in the past might not contain enough information to identify the off-Site sources of the packets in the present. This problem can be addressed by requiring that the database of RG delegations be maintained for some period of time after the RG is no longer usable for routing packets. Finally, it should be noted that the problem where an address's RG ``expires'' with the implication that the mapping of ``expired'' addresses into DNS names may no longer hold is not a problem specific to the GSE proposal. With provider-based addressing, the same issue arises when a site renumbers into a new provider prefix and releases the allocation from a previous block. draft-ietf-ipngwg-esd-analysis-00.txt [Page 39] INTERNET-DRAFT March 1997 4.1.12. The ICMP ``Who Are You'' Message Although there is widespread agreement of the utility of being able to determine the DNS name one is communicating with, there is also widespread concern that repeating the experience of the ``in- addr.arpa'' domain is undesirable. Consequently, an old proposal to define an ICMP ``Who Are You?'' message was resurrected [RFC1788]. A client would send such a message to a peer, and that peer would return an ICMP message containing its DNS name. Asking a remote host to supply its own name in no way implies that the returned information is accurate. However, having a remote peer provide a piece of information that a client can use as input to a separate authentication procedure provides a starting point for performing strong authentication. The actual strength of the authentication depends on the authentication procedure invoked, rather than the (untrustable) piece of information provided by a remote peer. Reconsidering the ``cheap'' authentication procedure described in Section 4.1.9, the ICMP ``Who Are You'' replaces the DNS PTR query used to obtain the DNS name of a remote peer. The second DNS query, to map the DNS name back into a set of addresses, would be performed as before. Because the latter DNS query provides the strength of the authentication, the use of an ICMP ``Who Are You'' message does not in any way weaken the strength of the authentication method. Indeed, it can only make it more useful in practice, because virtually all hosts can be expected to implement the ``Who Are You'' message. The ``Who Are You'' message would contain an identifier for matching replies to requests, as well as a nonce value to provide resistance to spoofing. In order to minimize the number of WRU packets on the Internet, the WRU messages should be sent by DNS servers who would then cache the answers. This has the pleasant side-effect of reducing the impact on existing applications (i.e., they would continue to look up addresses using the same API as before). In many cases there is a natural TTL that the target node can provide in its reply: either the remaining lifetime of a DHCP lease or the remaining valid time of a prefix from which the address was derived through stateless autoconfiguration. The ``Who Are You?'' (WRU) message described in [section ref: ``Reverse Mapping of Complete IPv6 Addresses'' under Analysis/ESD/WRU] is robust against renumbering, since it follows the paths of valid routable prefixes. Essentially, it uses the Internet routing system in place of the DNS delegation scheme. It is attractive in the context of GSE-style renumbering, since no host or DNS server needs to be updated after a renumbering event for WRU- draft-ietf-ipngwg-esd-analysis-00.txt [Page 40] INTERNET-DRAFT March 1997 based lookups to work. It has advantages outside the context of GSE as well, including a more decentralized, and hence more scalable, administration and easier upkeep than a DNS reverse-lookup zone. It also has drawbacks: it requires the target node to be up and reachable at the time of the query and to know its fully qualified domain name. It is also not possible to resolve addresses once those addresses become unroutable. In contrast, the DNS PTR mirrors, but is independent of, the routing hierarchy. The DNS can maintain mappings long after the routing subsystem stops delivering packets to certain addresses. The requirement that the target node be up and reachable at the time of the query makes it very uncertain that one would be able to take addresses from a packet log and translate them to correct domain names at a later date. This is a design flaw in the logging system, as it violates the architectural principle, ``Avoid any design that requires addresses to be ... stored on non-volatile storage.'' [RFC1958] A better-designed system would look up domain names promptly from logged addresses. Indeed, one of the authors is pleased to be able to state that his site has been doing that for some years. (Speculative note: Proxy servers to answer WRU queries are possible. If the boundary between the global and site portions of addresses are fixed and/or the boundary between the routing and the end-node portions are fixed, then one could define a well-known anycast address for proxy WRU service per site and/or per subnet. The low- order portion of this address would presumably be created from the IANA's IEEE OUI. The WRU client-side interface would have to be defined to try this address after or before sending a query to the target address itself. Nodes answering to this anycast address could reply to WRU queries using a database maintained by private means. By carrying a /128 route site-wide or in the site's provider, these servers need not even be located within the subnet or site they serve. Co-location of the proxy WRU servers with some DNS servers is a natural choice in some scenarios.) 4.2. Renumbering and Domain Name System (DNS) Issues 4.2.1. How Frequently Can We Renumber? One premise of the GSE proposal [GSE] is that an ISP can renumber the Routing Goop portion of a Site's addresses transparently to the Site (i.e., without coordinating the change with the Site). This would make it possible for backbone providers to aggressively renumber the Routing Goop part of addresses and achieve a high degree of route aggregation. On closer examination, frequent (e.g., daily) draft-ietf-ipngwg-esd-analysis-00.txt [Page 41] INTERNET-DRAFT March 1997 renumbering turns out to be difficult in practice because of a circular dependency between the DNS and routing. Specifically, if a Site's Routing Stuff changes, nodes communicating with the Site need to obtain the new Routing Stuff. In the GSE proposal, one queries the DNS to obtain this information. However, in order to reach a Site's DNS servers, the pointers controlling the downward delegation of authoritative DNS servers (i.e., DNS ``glue records'') must use addresses (with Routing Stuff) that are reachable. That is, in order to find the address for the web server ``www.foo.bar.com'', DNS queries might need to be sent to a root DNS servers, as well as DNS servers for ``bar.com'' and ``foo.bar.com''. Each of these servers must be reachable from the querying client. Consequently, there must be an overlap period during which both the old Routing Stuff and the new Routing Stuff can be used simultaneously. During the overlap period, DNS ``glue records'' would need to be updated to use the new addresses (including Routing Stuff). Only after all relevant DNS servers have been updated and older cached RRs containing the old addresses have timed out can the old address be deleted. An important observation is that the above issue is not specific to GSE: the same requirement exists with today's provider-based addressing architecture. When a site is renumbered (e.g., it switches ISPs and obtains a new set of addresses from its new provider), the DNS must be updated in a similar fashion. 4.2.2. Efficient DNS support for Site Renumbering When a site renumbers to satisfy its ISP, only the site's routing prefix needs to change. That is, the prefix reflects where within the Internet the site resides. Although some sites may also change the numbering of their internal topology when switching providers, this is not a requirement. Rather, it may be a convenient time to also perform any desired internal renumbering since one practical view is that any address renumbering tends to cause disruptions. In the current Internet, when a site is renumbered, the addresses of all the site's internal nodes change. This requires a potentially large update to the RR database for that site. Although Dynamic DNS [DDNS] could potentially be used, the cost is likely to be large due to the large number of individual records that would need to be updated. In addition, when DHCP and DDNS are used together [DHCP/DDNS interaction XXX], it may be the case that individual hosts ``own'' their own A or AAAA records, further complicating the question of who is able to update the contents of DNS RRs. One change that could reduce the cost of updating the DNS when a site is renumbered is to split addresses into two distinct portions: a draft-ietf-ipngwg-esd-analysis-00.txt [Page 42] INTERNET-DRAFT March 1997 Routing Stuff that reflects where a node attaches to the Internet and a ``site internal part'' that is the site-specific part of an address. During a renumbering, only the Routing Stuff would change; the ``site internal part'' would remain fixed. Furthermore, the two parts of the address could be stored in the DNS as separate RRs. That way, renumbering a site would only require that the Routing Stuff RR of a site be updated; the ``site-internal part'' of individual addresses would not change. To obtain the address of a node from the DNS, a DNS query for the name would return two quantities: the ``site internal part'' and the DNS name of the Routing Stuff for the site. An additional DNS query would then obtain the specific RR of the site, and the complete address would be synthesized by concatenating the two pieces of information. Implementing these DNS changes increases the practicality of using Dynamic DNS to update a site's DNS records as it is renumbered. Finally, it may be useful to divide a node's AAAA RR into the three logical parts proposed in [GSE], namely RG, STP and ESD. Whether or not it is useful to have separate RRs for the RG and STP portions of an address is an issue that requires further study. 4.2.3. Synthesizing AAAA Records If AAAA records are comprised of multiple distinct RRs, then one question is who should be responsible for synthesizing the AAAA from its components: the resolver running on the querying client's machine or the queried name server? To minimize the impact on client hosts and make it easier to deploy future changes, it is recommended that the synthesis of AAAA records from its constituent parts be done on name servers rather than in client resolvers. [this section is really weak; what more is needed?] 4.2.4. Two-Faced DNS The GSE proposal [GSE] attempts to hide the RG part of addresses from nodes within a Site. If the nodes do not know their own RG, then they can't store or use them in ways that cause problems should the Site be renumbered and its RG change (i.e., the cached RG become invalid). A Site's DNS servers, however, will need to have more information about the RG its Site uses. Moreover, the responses it returns will depend on who queries the server. A query from a node within the Site should return an address with an RG portion equal to ``Site local,'' whereas a query for the same name from a client located at a Site draft-ietf-ipngwg-esd-analysis-00.txt [Page 43] INTERNET-DRAFT March 1997 elsewhere in the Internet would return the appropriate RG portion. Such context-dependent DNS servers are commonly referred as ``two- faced DNS servers.'' Some issues that must be considered in this context: 1) A DNS server may recursively attempt to resolve a query on behalf of a requesting client. Consequently, a DNS query might be received from a proxy rather from the client that actually seeks the information. Because the proxy may not be located at the same Site as the originating client, a DNS server cannot reliably determine whether a DNS request is coming from the same Site or a remote Site. One solution would be to disallow recursive queries for off-Site requesters, though this raises additional questions. 2) Since cached response are, in general, context sensitive, a name server may be unable to correctly answer a query from its cache, since the information it has is incomplete. That is, it may have loaded the information via a query from a local client, and the information has a Site-local prefix. If a subsequent request comes in from an off-Site requester, the DNS server cannot return a correct response (i.e., one containing the correct RG). [what else needs mentioning?] 4.2.5. Bootstrapping Issues If Routing Stuff information is distributed via the DNS, key DNS servers must always be reachable. In particular, the addresses (including Routing Stuff) of all root DNS servers are, for all practical purposes, well-known and assumed to never change. It is not uncommon for the addresses of root servers to be hard-coded into software distributions. Consequently, the Routing Stuff associated with such addresses must always be usable for reaching root servers. If it becomes necessary or desirable to change the Routing Stuff of an address at which a root DNS server resides, the routing subsystem will likely need to continue carrying ``exceptions'' for those addresses. Because the total number of root DNS servers is relatively small, the routing subsystem is expected to be able to handle this requirement. All other DNS server addresses can be changed, since their addresses are typically learned from an upper-level DNS server that has delegated a part of the name space to them. So long as the delegating server is configured with the new address, the addresses of other servers can change. draft-ietf-ipngwg-esd-analysis-00.txt [Page 44] INTERNET-DRAFT March 1997 4.2.6. DNS PTR RRs Not Needed Both IPv4 and IPv6 include a mechanism for mapping addresses into DNS names (i.e., the PTR RR and IN-ADDR.ARPA and IP6.INT domains respectively). The need for such a mechanism can be decreased significantly through the proposed ICMP ``Who Are You'' message (see Section 4.1.12). In any case, PTR records can be implemented essentially as today. Assuming that the Routing Stuff of addresses remains hierarchical, each ISP would be responsible for delegating its part of the Routing Stuff to its downstream customers. Eventually, the Routing Stuff would reach the ISP to which the end site connects. This ISP would have a pointer to the site's DNS servers which be responsible for the rest of the mapping. 4.2.7. Renumbering and Reverse DNS Lookups It is certain that many sites will from time to time undergo a renumbering event, either through the mechanisms proposed for GSE or using the facilities already specified for IPv6. It would be useful to an outside node corresponding with such a site to be able to distinguish a legitimate renumbering from an attempt to impersonate the site. We claim that the DNS IP6.INT zone, without security extensions [RFC2065], is of no use in making this determination and that even a completely secured IP6.INT zone is of little use compared with the ``forward'' DNS zone. The first half of the claim is almost self-evident. An impersonator can set up an insecure zone at some point in the IP6.INT hierarchy and load it with any desired data. This is the reason that current applications doing minimal access control follow a reverse lookup with a forward lookup. With a secured reverse zone, the problem of verifying an apparent renumbering of a site can still be quite complex in the general case, and will certainly be outside the scope of a transport protocol, if survival of long-running sessions is contemplated. Under provider- based addressing [RFC2073], renumbering is expected to occur due to a change in network topology (e.g., a change in a provider relationship at some point in the address aggregation tree). This alters the global prefixes in use below the point of the change, and correspondingly alters the chain of delegations of the DNS reverse- mapping tree. And, although operational experience with secure DNS is quite limited, it seems likely that there would also be a change in the chain of certifications of the signing key of the leaf zone representing the site. It is then problematic to translate draft-ietf-ipngwg-esd-analysis-00.txt [Page 45] INTERNET-DRAFT March 1997 established trust in the old reverse mapping zone into trust in the new zone. Certainly it's simpler to rely on the forward zone only. The only function of the reverse zone, then, is to suggest an entry point to the forward zone's database. It is this function which we propose to achieve by means of a new ICMP message exchange. 4.3. Address Rewriting Routers One of the most novel pieces of GSE is the rewriting of addresses as datagrams enter and leave Sites. If only a small number of routers know the RG portion of the addresses, then the operational impact of renumbering a Site would be small. In fact, assuming that the critical security issues are dealt with, one could imagine a dynamic protocol that a Site uses with its upstream provider to be told what RG to use, so it might even be possible to renumber a Site transparently. GSE's ability to ensure that the RG portion of a Site's addresses reflect the actual location of that Site within the Public Internet means that very aggressive aggregation (i.e., better route scaling) can be achieved. Both GSE and other route-scaling approaches that use provider-based addressing depend on aggressive aggregation, but while other schemes rely largely on operational policies, GSE attempts to include mechanisms in its core to ensure that aggressive aggregation happens in practice. GSE has an advantage over other provider-based addressing schemes like IPv4's CIDR with respect to the ``fair distribution of work.'' CIDR addresses the scaling of routing in DFZ portions of the Internet, but the cost of carrying out the renumbering to maintain the aggregation falls on the shoulders of subscribers who are far away from the DFZ; in other words, subscribers must do the work of renumbering so that their provider (or possibly even their provider's provider) see better aggregation. With GSE, the majority of the cost required to make the routing scale would be incurred by the parties who reap the benefits. 4.3.1. Load Balancing While not considered a major advantage, with GSE, multi-homed Sites can more easily achieve symmetry with respect to which of their links is used for a given flow. With GSE, if HostA in multi-homed Site1 initiates a flow to HostB in Site2, then when the initial packet leaves Site1 the source address will be rewritten with an RG that identifies the outgoing link used. As a result, when HostB needs to send return traffic, it will use the full 16-byte address from the draft-ietf-ipngwg-esd-analysis-00.txt [Page 46] INTERNET-DRAFT March 1997 arriving packet and this necessarily means that traffic for this flow coming into Site1 will use the same circuit that outgoing traffic for that flow took. 4.3.2. End-To-End Argument: Don't Hide RG from Hosts Despite these significant advantages, however, it was felt that address rewriting by routers should not be pursued as part of the current standardization effort. Although hiding RG knowledge from hosts has advantages in simple scenarios, that lack of knowledge also makes it difficult to solve important problems. For example, a host in a multi-homed site is known by multiple addresses, but without knowing its address the host can play no role in the source address selection; instead, the host relies on the routing infrastructure to magically select the right one, i.e., by selecting the egress router the packet uses to leave the site. In this particular case, the historically difficult-to-solve problem of source address selection is made more difficult by moving it from an intra-host decision to a distributed one. Now a site's internal routers would have to have sufficient knowledge to decide which egress router to forward traffic to, perhaps on a source-by-source (or worse) basis. Another end-to-end problem resulting from address rewriting has to do with how transport connections should deal with the RG portion of the address in incoming packets, particularly when the RG changes. The sections on transport issues deal with the subject in much more detail. Interesting questions arise about address rewriting when dealing with tunnels. It was realized that any node that can terminate a tunnel whose other end point is in a different Site must be able to behave as a Site border router and do address rewriting. This means that the RG may need to be configured in more than just a Site's egress router, thus making renumbering more problematic. Another problem related to both performance and ``architectural cleanliness'' has to do with IPv6's Routing Headers. It may be necessary for addresses other than just the simple source and destination to be rewritten. And again, this rewriting would need to be done by both egress routers and nodes which terminate tunnels that go to other Sites. 4.4. Multi-homing Multi-homing can mean many things. In the context of GSE, multi- homing refers to a Site having more than one connection to the Internet and being known by multiple RGs. In many ways this is close draft-ietf-ipngwg-esd-analysis-00.txt [Page 47] INTERNET-DRAFT March 1997 to multi-homing with IPv6 provider-based addressing. It is hard to make comparisons to IPv4 because multi-homing has traditionally been done in an ad hoc fashion. With GSE, the ability of a Site to control the load-sharing over its multiple links is not clear, partially because there is little operational experience with multi-homed sites known by multiple prefixes (with IPv4 the site is generally only known by a single prefix). The following analysis is relevant to any scheme where an Internet-connected site is known by multiple prefixes. For flows that the multi-homed site initiates, load-sharing is impacted by the source address used because that is the address that the remote site will use for return traffic. If we assume the model of routers rewriting source addresses, then the outgoing link selected determines the load-sharing because that also determines what RG is contained in the source address. If the routers do not rewrite source addresses, then the end-host itself will have to make the source address selection, and the optimal choice may require knowledge of the topology. For flows initiated by someone outside of the multi- homed site, the load-sharing is dependent on the destination address specified, so the DNS has a large impact on load-sharing. There is some amount of operational experience in using DNS to control load on servers (e.g., having a Web server resolve to multiple addresses), though that is load-sharing of a different resource and at a different scope and scale. It is also worth noting that the selection of the optimal outgoing link may well depend on the destination, which has particularly interesting results on the DNS understanding topology (and brings up the question of whether the servers or the resolvers are responsible for knowing the topology). One advantage that GSE has for multi-homed Sites is symmetry. Because the source address is selected based on the outgoing link, and that source address is what determines the return path, flows initiated by the Site will be symmetric with respect to which of the Site's links is used. The multi-homing mechanism described in Section 3.2.4 has some weaknesses and complexities. First, the mechanism only supports healing a failed link and not a router; in other words, referencing Figure 6, from Section 3.2.4, if PBR1 were not up at all, then it could not tunnel the packets anywhere. One could imagine ways of distributing PBR1's knowledge of PBR2 to other routers within Provider1 to add more reliability, though this makes the problem distributed rather than point-to-point and therefore more difficult. Second, in the general case static identification of PBR2 to PBR1, and vice-versa, is not adequate. Imagine, for example, that the link to PBR1 is much faster than the link to PBR2. In this case, it's possible that packets whose destination addresses contain RG1 might draft-ietf-ipngwg-esd-analysis-00.txt [Page 48] INTERNET-DRAFT March 1997 normally transit PBR2 without going directly to the Site. So there seems to be a need for a dynamic protocol between PBR1 and PBR2 to notify when PBR2, for example, should forward RG1-prefaced destinations directly to the Site as opposed to forwarding it towards PBR1. Another note about multi-homing is the potential impact of internal topology changes in the face of address rewriting. Using the previously referenced diagram, if a flow from a host within the Site is leaving via SBR1, but then something happens such that SBR2 becomes the host's closest exit point, then the remote end point of the flow will begin seeing different RG. Reasons such as this are why the repercussions on the transport layer are so important (e.g., whether or not transport peers pay attention to the RG). 5. Recommendations This section should be viewed as ``proto recommendations'' and not final recommendations. It is impossible to have final recommendations until there exists an analysis on which there is consensus. A straw-man set of recommendations, along with some related open questions, is presented below: 1) Make changes to the IPv6 provider-based addressing document to facilitate aggressive aggregation that is also operationally realistic. 2) Create hard boundaries in IPv6 addresses to clearly distinguish between the portions used to identify hosts, for routing within a site, and for routing within the Public Internet. 3) Designate the low-order 8 bytes of IPv6 addresses to be a globally unique End System Designator (ESD). This change has potential benefits to future transport protocols (e.g., TCPng). A point of discussion on this topic is whether, in the short- term, the ESD will be used alone; if it isn't to be used alone, then how important is the global uniqueness? 4) Make a clear distinction between the ``locator'' part of an address and the ``identifier'' part of the address. The former is used to route a packet to its end point, the latter is used to identify an end point, independent of the path used to deliver the packet. Although this is a potentially revolutionary change to IPv6 addressing model, existing transport protocols such as TCP and UDP will not take advantage of the split. Future transport protocols (e.g., TCPng), however, may. draft-ietf-ipngwg-esd-analysis-00.txt [Page 49] INTERNET-DRAFT March 1997 5) Make changes to the way AAAA records are stored within the DNS, so that renumbering a site (e.g., when a site changes ISPs) requires few changes to the DNS database in order to effectively change all of a site's address AAAA RRs. 6) Don't hide a node's full address from that node. In a scheme where all nodes know their full address, address rewriting should not be necessary. 7) Consider multi-homing and its effect on aggregation and route scaling from the beginning. Have a goal of architecting a way to do multi-homing that is both scalable and operationally practical, and consider related issues such as load-sharing. 8) Consider the issue of subnetting. For example, how are point- to-point links numbered? With IPv4, current practice is to number point-to-point links out of ``/30'' subnets. However, do network masks longer than 64 bits make sense with the concept of the low-order 8 bytes being a globally unique ESD? If not, then is it acceptable to either leave point-to-point links un- numbered or to use an entire subnet for each point-to-point link? Will there need to be an exception for IPv6 host routes (i.e., /128s) as a work-around for the bootstrapping issue of addressing root DNS servers? If /128s are allowed, but not masks between /65 and /127, inclusive, then a possible way to number point-to-point links within a backbone is to dedicate a single subnet to them and route them as /128s. 9) Search for ways to minimize the impact that renumbering can have on intra-site communication. Renumbering operations that change only the RG portion of addresses should not impact existing intra-site communication. One possible approach is to encourage the use of site-local addresses for all intra-site communication. 6. Security Considerations TBD 7. Acknowledgments Thanks go to Steve Deering and Bob Hinden (the Chairs of the IPng Working Group) as well as Sun Microsystems (the host for the meeting) for the planning and execution of the interim meeting. Thanks also goes to Mike O'Dell for writing the 8+8 and GSE drafts. By publishing draft-ietf-ipngwg-esd-analysis-00.txt [Page 50] INTERNET-DRAFT March 1997 these documents and speaking on their behalf, Mike was the catalyst for some very valuable discussions that are expected to result in improved IPv6 addressing. Special thanks to the attendees of the meeting who carried on the high caliber discussions which were the source for this document. 8. References [DANVERS] Minutes of the IPNG working Group, April 1995. ftp://ftp.ietf.cnri.reston.va.us/ietf-online-proceedings/ 95apr/area.and.wg.reports/ipng/ipngwg/ ipngwg-minutes- 95apr.txt. [EUI64] 64-Bit Global Identifier Format Tutorial. http://standards.ieee.org/db/oui/tutorials/EUI64.html. Note: ``EUI-64'' is claimed as a trademark by an organization which also forbids reference to itself in association with that term in a standards document which is not their own, unless they have approved that reference. However, since this document is not standards-track, it seems safe to name that organization: the IEEE. [Bellovin 89] ``Security Problems in the TCP/IP Protocol Suite'', Bellovin, Steve, Computer Communications Review, Vol. 19, No. 2, pp32-48, April 1989. [CERT] CERT(sm) Advisory CA-96.21 (ftp://info.cert.org/pub/cert_advisories) [DDNS] ``Dynamic Updates in the Domain Name System (DNS UPDATE)'', Paul Vixie (Editor), draft-ietf-dnsind-dynDNS-11.txt, November, 1996. [GSE] ``GSE - An Alternate Addressing Architecture for IPv6', Mike O'Dell, draft-ietf-ipngwg-gseaddr-00.txt. [IEEE802] IEEE Std 802-1990, Local and Metropolitan Area Networks: IEEE Standard Overview and Architecture. [RFC1122] ``Requirements for Internet hosts - communication layers'', R. Braden, 10/01/1989. [IEEE1212] IEEE Std 1212-1994, Information technology-- Microprocessor systems: Control and Status Registers (CSR) Architecture for microcomputer buses. [RFC1715] The H Ratio for Address Assignment Efficiency. C. Huitema. draft-ietf-ipngwg-esd-analysis-00.txt [Page 51] INTERNET-DRAFT March 1997 [RFC1726] Technical Criteria for Choosing IP:The Next Generation (IPng). F. Kastenholz, C. Partridge. [RFC1752] ``The Recommendation for the IP Next Generation Protocol,'' S. Bradner, A. Mankin, 01/18/1995. [RFC1788] ``ICMP Domain Name Messages'', W. Simpson, 04/14/1995 [RFC1958] Architectural Principles of the Internet. B. Carpenter. [RFC1971] IPv6 Stateless Address Autoconfiguration. S. Thomson, T. Narten. [RFC2002] ``IP Mobility Support'', 10/22/1996, C. Perkins. [RFC2008] ``Implications of Various Address Allocation Policies for Internet Routing'', Y. Rekhter, T. Li. [RFC2065] Domain Name System Security Extensions. D. Eastlake, C. Kaufman. [RFC2073] An IPv6 Provider-Based Unicast Address Format. Y. Rekhter, P. Lothberg, R. Hinden, S. Deering, J. Postel 9. Authors' Addresses Matt Crawford John Stewart Fermilab MS 368 USC/ISI PO Box 500 4350 North Fairfax Drive Batavia, IL 60510 USA Suite 620 Phone: 708-840-3461 Arlington, VA 22203 USA EMail: crawdad@fnal.gov Phone: 703-807-0132 EMail: jstewart@isi.edu Allison Mankin Lixia Zhang USC/ISI UCLA Computer Science Department 4350 North Fairfax Drive 4531G Boelter Hall Suite 620 Los Angeles, CA 90095-1596 USA Arlington, VA 22203 USA Phone: 310-825-2695 EMail: mankin@isi.edu EMail: lixia@cs.ucla.edu Phone: 703-807-0132 Thomas Narten IBM Corporation 3039 Cornwallis Ave. PO Box 12195 - F11/502 Research Triangle Park, NC 27709-2195 draft-ietf-ipngwg-esd-analysis-00.txt [Page 52] INTERNET-DRAFT March 1997 Phone: 919-254-7798 EMail: narten@raleigh.ibm.com draft-ietf-ipngwg-esd-analysis-00.txt [Page 53]