Internet Draft Internet Engineering Task Force Ming Hou Differentiated Services Working Group Hussein T. Mouftah Internet Draft Queen's University,Canada Expires September 1999 June, 1999 Performance Evaluation of Premium Service <draft-hou-diffserv-premium-eval-00.txt> Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. This document is a product of the IETF Differentiated Services Working Group. Comments are solicited and should be directed to the working group mailing list. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Distribution of this memo is unlimited. Abstract The performance evaluation of QoS required services helps in observing the behavior of the Diffserv, and is an important part of the work of the Diffserv Working Group. In this document we evaluate the Premium service in terms of packet loss and jitter under Diffserv. A postscript version of this document is also available. 1. Introduction The Premium service, which was proposed in [7], requires a low jitter and low packet loss during data transmission. It can be implemented by the EF PHB. By Diffserv protocol, the arriving Premium packets are geared to the EF aggregates on the node output interface. The EF queue has a higher priority in being forwarded than other queues, and packets in it are forwarded to exhaust before the packets in other queues. This provides a very short EF queue. Thus the Premium packets will experience a little queuing delay. Packets from other services different from the Premium service, will be dropped with higher priority when there is not enough buffer at the node output interface. Since the Premium packets are forwarded prior to others, the Premium service is expected to have a relatively lower packet loss than other services. This report is organized as follows: Section 2 describes the IP simulator which is used in evaluating the Premium service; Section 3 presents the simulation results for Premium service in terms of jitter and packet loss; Section 4 gives the conclusive remarks. 2. Description of simulator In order to observe the various aspect of Diffserv, we have developed the Queen's University IP Simulator (QUIPS). This real-time discrete-event simulator is written in JAVA language and runs on UNIX platform. QUIPS has been designed in a modular way, which contains three modules: the Graphical User Interface (GUI), the network module and the control module. This modular design fashion makes it easy for further extension. In the simulation model, the network has been decomposed into four components: the link, the node, the sender and the receiver. A sender and a receiver represent the data producer and the data consumer, respectively. The sender sends packets with different service requirement to the receiver, and the receiver computes the end-to-end delay and other QoS parameters after receiving the packet. A link is modeled as a uni-directional link, since the QoS requirements on the two directions are not always the same. The node component represents a router in a computer network. Both the routing and the forwarding engine are implemented in the node. The control module has an event queue, which handles the operation of different network elements. The monitor in the control module gathers statistics during simulation runs. Although only Diffserv is implemented in QUIPS, new functions and IP protocols can be easily added to the simulator without a significant change to the codes because of its modular design. 3. Simulation results and analysis Using QUIPS, both the Assured service and the Premium service can be simulated and evaluated under Diffserv. This study focuses only on the performance of the Premium service while the investigation into the Assured service is left to future work. The EF PHB has the highest priority in being forwarded, thus it can provide a low delay, low loss service. The EF PHB is assigned to the Premium traffic in order to implement the Premium service. There is a number of mechanisms that can be used in implementing the EF PHB. The simplest one is a priority queue where the arrival rate of the queue is strictly less than its departure rate. As jitter comes from the queueing delay along the path, a feature of this implementation is that EF-marked microflows will see very little jitter at their subscribed rate if all DS nodes along the path use this implementation since packets spend little time in queues. Since the Premium Service is a low jitter, low loss service, its performance evaluation is focused on its behavior in terms of jitter and packet loss under Diffserv. Different network parameters may have different effect on the performance of the Premium traffic. First is the output buffer size. When a Premium packet arrives, if there is no enough output buffer for it, it will be either dropped, which affects the packet loss; or be delayed, that is, be waited withing the router until resources available, which increases the amount of jitter. Secondly, the link bandwidth occupitation of the Premium traffic is an important parameter in performance evaluation. As the density of the Premium traffic increases, new Premium packets will arrive at the router before the old ones can be sent. In this case, although the Premium packets have higher priority in getting network resources than the other type of traffic, newly arrived Premium packets will be lost in the competition of resources with the Premium packets which have arrived ealier but have not been sent yet. Thus, the jitter and the packet loss will increase. The packet length also has some effects on the performace measurements. Assuming a lengthy best-effort packet is transmitting while a Premium packet is arriving. The Premium packet shall be delayed although it has a higher priority. This causes the increase of jitter, and, if a number of Premium packets are jammed, the packet loss of Premium packets will increase. All the above parameters play an important role in the behavior of Premium traffic under Diffserv, thus the evaluations in this draft are performed by choosing different values of the output buffer size, the Premium traffic allocation and the packet length. A two-hop DS domain with the link bandwidth of 100 Mbps is modeled, as shown in Figure 1. It is assumed that the SLAs and the TCAs have been established, and remain unchanged during simulation. For the EF-marked packets, or the Premium Service required packets, sources have been set up to produce packets at a constant bit rate with a variation of +/- 10% of the subscribed peak rate. A mixture of other kinds of traffic, the best effort service, is used to fill the link. 4. Jitter Evaluation In evaluating the packets' jitter, the jitter is recorded as the added delay normalized by the time to send a packet at the subscribed peak rate. The simulation parameters are changed and the Premium packets' behavior is observed under Diffserv in three aspects: for different output buffer size of routers, for different packet size, and for different Premium service allocation. During each kind of simulation, only one parameter of the three is changed and the other two remain fixed. 4.1 Evaluation with Different Buffer Size This jitter evaluation of Premium packets is simulated under four different output buffer size: 2, 5, 10 packets and an infinite buffer. The packet length is assumed to be 1500 bytes for all simulation runs, while the Premium traffic is subscribed as 30% of the link bandwidth. The average delay of Premium service packets and the jitter Premium packets experienced are shown in Figures 2 and 3, respectively. >From Figure 2, one can see that after a short transient period at the beginning of a simulation run, the average delay tends to be constant. The transient area should be ignored because of the system unsteadyness at the begining of a simulation run. The average delay of Premium packets in all the cases is very small, with a slight difference among them. The average delay decreases as the number of output buffer increases. However, the results of the infinite buffer size are very close to the results of the 10-packet size buffer. The jitter for all test cases are very close, as shown in Figure 3. A detailed comparison of the jitter at different Premium packet percentile is listed in Table 1. Both the figure and the table imply that the variaration of the output buffer size does not have much effect on the jitter behavior of Premium packets. ==================================================================== |Buffer|Percentile|Jitter(mSec)|Percentile|Jitter|Percentile|Jitter| -------------------------------------------------------------------- | 2 | 50 | 0.015 | 75 | 0.043| 95 |0.0505| -------------------------------------------------------------------- | 5 | 50 | 0.0205 | 75 | 0.036| 95 |0.0505| -------------------------------------------------------------------- | 10 | 50 | 0.0205 | 75 | 0.036| 95 |0.0505| -------------------------------------------------------------------- |infinity| 50 | 0.0205 | 75 | 0.036| 95 |0.0505| ==================================================================== Table 1. Comparison of Jitter with Different Buffer Size 4.2 Evaluation with Different Packet Size Here, the node output buffer is chosen to be infinite; thus there is no packet loss during transmission. The individual source rates of generating the Premium packets are picked as the 30% of the link bandwidth or 30 Mbps. Figure 4 shows the delay each 1500-byte EF packet experienced. One may observe that a few packets have a longer delay than the others because of the congestion at the node output interface. However, from the graph of packet percentile vs jitter (Figure 5), one can see that most of the packets experience low jitter. The absolute value of the jitter for different packet size (1000 bytes, 1500 bytes and 2500 bytes) are compared. From Figure 5, one can see that the jitter value is bigger as the packet size increases. However, the normalized jitters are almost the same, and are all less than 0.7 packet-time (Table 2). A comparison of the average packet delay experienced with different packet sizes is shown in Figure 6. Ignore the transient period, the average delay tends to be constant. One can see that the average delay of the Premium packets is increasing as the packet size increases. Long packet will cause larger delay, because at the same speed it will take longer time to transmit a long packet than to transmit a short packet. New packet will arrive before previous one completely transmitted, thus the new one have to be waited in the queue and increases the end-to-end delay in reault. ========================================================================= |Packet Size|Percentile|Jitter(mSec)|Percentile|Jitter|Percentile|Jitter| ------------------------------------------------------------------------- | 1000 | 50 | 0.18 | 75 | 0.36 | 95 | 0.62 | ------------------------------------------------------------------------- | 1500 | 50 | 0.16 | 75 | 0.35 | 95 | 0.7 | ------------------------------------------------------------------------- | 2500 | 50 | 0.16 | 75 | 0.31 | 95 | 0.61 | ========================================================================= Table 2. Comparison of Normalized Jitter with Different Packet Size 4.3 Evaluation with Different Premium service allocation In this section, the effect of varying the Premium traffic allocation on the jitter behavior is investigated. The Premium traffic to be allocated is chosen as a small percentage of the link bandwidth. The allocation is chosen to be 5%, 10%, 20% and 30% of the link bandwidth. Figure 7 shows the simulation results of the average delay, which support the notion that the density of the Premium traffic will cause the average delay to be increased. With very small percentage of Premium traffic, such as 5\% or 10\% of the link bandwidth, the average delay is small, but with a small change on the allocation, the average delay will change significantly. The small allocation of Premium traffic also has an outstanding jitter performance. As shown in Figure 8, the 5% allocation has better jitter performance than the others. A comparison of the jitter value for each test case is presented in Table 3. As the traffic allocation for the Premium packet increases, more Premium packets will apprear at the output interface of routers, and probability of more than one Premium packet competing for the same network resources will increase. The subsequent Premium packets are put into the waiting queue, when there is a Premium packet being transmitted. ======================================================================== |Allocation|Percentile|Jitter(mSec)|Percentile|Jitter|Percentile|Jitter| ------------------------------------------------------------------------ | 5% | 50 | 0.008 | 75 | 0.01 | 95 | 0.044| ------------------------------------------------------------------------ | 10% | 50 | 0.008 | 75 | 0.01 | 95 | 0.044| ------------------------------------------------------------------------ | 20% | 50 | 0.013 | 75 |0.035 | 95 | 0.052| ------------------------------------------------------------------------ | 30% | 50 | 0.021 | 75 |0.037 | 95 | 0.051| ======================================================================== Table 3. Normalized Jitter with Different Premium Traffic Allocation 5. Packet loss evaluation In this section, the behavior of the Premium packets under Diffserv in terms of packet loss is investigated. Similar to the case of the jitter evaluation, the simulation parameters are varied and the Premium packets' behavior is observed under Diffserv in three aspects: for different output buffer size of routers, for different packet size, and for different Premium service allocation. During each type of simulation, only one parameter of the three is changed and the other two remain fixed. The simulation results show the loss probability of packets for the network topology shown in Figure 1. The output buffer of the nodes is assumed to be of two-packet size, and the network load ranges from zero to three times of the link bandwidth among which is 30% of the Premium traffic. 5.1 Evaluation with Different Packet Size The packet loss for different packet size with an output buffer of two packets is shown in Figure 9. One can see that varying of the packet length does not have much effect on the packet drop. 5.2 Evaluation with Different Buffer Size In Figure 10, a comparison of the Premium packet loss probability and the best effort packet loss probability is provided. Since the Premium packets have the highest priority in forwarding among the traffic streams, they will experience less amount of loss than the best effort packets. One can see that the best effort packets have a very high probability of being lost , while the Premium packets have relatively lower probability of packet loss. However, as the network load increases, the Premium packets will be dropped much more because of the congestion at the nodes output interface. The packet loss behavior of the Premium traffic as the output buffer is increased is tested as shown in Figure 11. Also, the overall packet loss with different buffer size has been investigated, as shown in Figure 12. The buffer sizes are picked as 2, 3, 5 and 10 packets size, respectively. One can see that the packet drop is decreasing as the buffer size increases. 5.3 Evaluation with Different Premium traffic allocation Three Premium traffic allocations, 5%, 10% and 30%, have been adopted for the investigation results reported in this section. The results of the packet loss probability for the Premium packets under different traffic allocation are shown in Figure 13. As expected, the loss probability increases as the traffic allocation increases. Even a small change in the allocation, will have a big effect on the packet loss probability. This is because when the Premium traffic increases, at the output interface of the routers, more Premium packets will compete for the network resource, and the routers have to drop more packets in order to satisfy others. For the same reason, the packet loss probability for the Premium packets together with the best-effort packets will increase as the Premium traffic allocation increases, as shown in Figure 14. Finally, a comparison of the probability of packet loss for the Premium packet with the best-effort packets, for different Premium traffic allocation is provided in Figure 15. 6. Conclusion This draft has briefly described an IP simulator developed by Queen's University computer networks group. The performance of the Premium service has been evaluated by QUIPS under Diffserv. The QoS performance measures are chosen to be the jitter and the packet loss; and the different network parameters, such as buffer size, packet size and percentile allocation of Premium service, are used for different simulation runs. The simulation results show that the Premium service can be provided using Diffserv. Diffserv. 7. References [1] C. Alaettiniglu, A. U. Shanker, etc. Design and Implementation of MaRS: A Routing Testbed, September 1992, http://www.ccs.neu.edu/home/matta [2] C. Alaettiniglu, A. U. Shanker, etc. MaRs - version 1.0 Programmer's Manual, June 1991, http://www.ccs.neu.edu/home/matta [3] D. Black, S. Blake, M. Carlson, E. Davies, Z. Wang, W. Weiss, An Architecture for Differentiated Services, Internet rfc2475, December 1998 [4] Y. Bernet, D. Durham, F. Reichmeyer, Requirements of Diff-serv Boundary Routers, Internet draft, draft-bernet-diffedge-01.txt, November 1998 [5] Y. Bernet, et al., A Framework for Differentiated Services, Internet draft, draft-ietf-diffserv-framework-01.txt, October 1998 [6] V. Jacobson, K. Nichols, K. Poduri, An Expedited Forwarding PHB, Internet draft, draft-ietf-diffserv-phb-ef-01.txt, November 1998 [7] K. Nichols, V. Jacobson, and L. Zhang, A Two-bit Differentiated Services Architecture for the Internet, Internet Draft, draft-nichols-diff-svc-arch-00.txt, November 1997. 8. Authors' Addresses Ming Hou Department of Electrical and Computer Engineering Queen's University Kingston, Ontario Canada K7L 3N6 houm@eleceng.ee.queensu.ca Hussein T. Mouftah Department of Electrical and Computer Engineering Queen's University Kingston, Ontario Canada K7L 3N6 mouftah@eleceng.ee.queensu.ca