Satellite Communications in the Global Internet: Issues, Pitfalls, and Potential

Satellite Communications in the Global Internet: Issues, Pitfalls, and Potential

Yongguang Zhang <ygz@isl.hrl.hac.com>
Dante De Lucia <dante@isl.hrl.hac.com>
Bo Ryu <ryu@isl.hrl.hac.com>
Son K. Dao <son@isl.hrl.hac.com>
Hughes Research Laboratories
USA

Abstract

Recent deployment of commercial products in geosynchronous earth orbit (GEO) satellite communication networks demonstrates the promise of ubiquitous access to the Internet. Delivering this promise to end users requires integrating satellite communications into existing Internet transmission links. This effort is the topic of heated debate over whether common Internet applications can perform satisfactorily over networks with high-latency links, such as those involving GEO satellite transmissions. Through simulations and actual satellite experiments, we contend that most applications can work effectively. Certain TCP-based applications have technical shortcomings that surface in both terrestrial and satellite high-speed networks, but feasible solutions exist. Furthermore, multicast applications and the network as a whole can benefit from the broadcast nature of satellite transmission and its simple network topology, as demonstrated by our experiments using NASA's Advanced Communications Technology Satellite (ACTS).

Contents

• Introduction
  • Satellite networks
  • Applications
• TCP over long delay networks
  • Performance issues in TCP control mechanisms
    • Window size
    • Bandwidth adaptation
    • Selective acknowledgment
    • Slow start
    • Congestion avoidance
    • TCP for transactions
  • Impacts on end-to-end performance
  • Approaches for performance improvement
    • TCP extensions
    • Middleware
    • Application protocol approaches
• Multicast
• Conclusion
• Acknowledgments
• References

Introduction

Satellite networks

Most current Internet backbones and subnets are wired terrestrial networks (e.g., cable, telephone, and fiber optics) with bandwidth ranging from T1 (1.5 megabits per second [Mbps]) to OC-12 (622 Mbps). Currently, researchers are working on the next-generation Internet that will support high-bandwidth applications (> 45 Mbps), and ubiquitous computing with mobile/wireless networks (wireless local area network [LAN] and ATM, wireless metropolitan networks, and satellite networks). Among these mobile/wireless networks, GEO satellite networks offer great potential for multimedia applications with their ability to broadcast and multicast large amounts of data over a very large area, thus achieving global connectivity [12]. Internet distribution via satellites, particularly GEO satellites, has the following merits:

High bandwidth

A Ka-band (20-30 GHz) satellite can deliver throughput at gigabits per second rates.

Inexpensive

A satellite communications system is relatively inexpensive because there are no cable-laying costs, and one satellite covers a very large area.

Untethered communication

Users can enjoy untethered mobile communication anywhere within the footprints of the satellite.

Simple network topology

Compared with the mesh interconnection model of the terrestrial Internet, GEO satellite networks have much simpler delivery paths. The simpler topology often results in more manageable network performance.

Broadcast/multicast

Satellite networks are naturally attractive for broadcast/multicast applications (such as MBone [7]). In contrast, multicast in a mesh interconnection network requires complicated multicast routing. Performance can vary for each multicast group member and is dependent on the route from the source.

On the other hand, satellite communications present one major challenge with respect to the performance of Internet applications -- the communication latency between two earth stations connected by a satellite. For GEO satellite communications systems, the latency is at least 250 milliseconds (sometimes framing, queuing, and on-board switching can add extra delays, making the end-to-end latency as high as 400 milliseconds). This is approximately 10 times higher than a point-to-point fiber optics connection across the continental United States. The latency might not affect bulk data transfer and broadcast-type applications, but it will hurt highly interactive applications that require extensive handshaking between two sites. Unfortunately, one of the major Internet transport protocols, TCP, requires such interaction.

LEO (low earth orbit) and MEO (medium earth orbit) satellites can also provide global and broadband communication capacities. Latency in a LEO system is comparable with terrestrial fiber optics, usually only about twice as large. Because neither a LEO nor a MEO satellite can stay in a fixed position relative to the surface of the earth, a constellation of many satellites is required to provide complete coverage, increasing the complexity of the system relative to GEO systems. Network management is a much harder problem because handoff, tracking, and routing need to be done properly. The advantage of simple topology is lost, and so is single-source broadcast/multicast capability.

Applications

Common Internet applications include Web browsing, file transfer protocol (FTP), remote login (Telnet), video teleconferencing, e-mail, and broadcast. They all use IP (Internet Protocol) as the transmission mechanism, so they can seamlessly run over satellite networks. However, performance varies among applications because their requirements on network bandwidth and responsiveness, their tolerance to communication noise, and their implementation techniques are very different.

Remote control and login

Remote control and login are very delay sensitive. Typically a user expects responsiveness on the order of tens of milliseconds during a remote login session. Remote control may require even faster response, depending on applications. When compared with the often congested and chaotic terrestrial Internet, system response time over GEO satellites is somewhat slower but more stable. If a user can endure a half-second to one-second delay, remote control and remote login applications can run smoothly over satellites.

Information dissemination and broadcast

Satellite networks are better media to deliver bulk data, anywhere and anytime. Some illustrative examples include maps and situation awareness data, stock market and financial numbers, battlefield information, and medical data. Data broadcast, such as webcasting, network news, and TV programs can be very expensive for point-to-point networks, but is ideally suited to broadcast satellites. Therefore, GEO satellites are far more suitable for these applications than is the traditional terrestrial network.

Videoconferencing

Videoconferencing and video distribution applications can usually tolerate a certain amount of packet loss; thus they can be built on top of UDP (such as the MBone video tool vic [9]) Typically the protocol requires no bidirectional synchronous (handshake-style) communication, hence latency is not a prohibitive issue. Compared with the terrestrial Internet, GEO satellites can provide better quality in videoconferencing thanks to more available bandwidth and simpler network topology.

Electronic mail

Traditionally electronic mail is not interactive. It does not require a great deal of bandwidth and can tolerate reasonable delays (often in terms of minutes). It should work fine over any network.

Information retrieval (WWW, FTP)

Many such applications require reliable data transmissions, hence they are usually built on top of robust protocols like TCP. Because of the "acknowledge-and-retransmission" scheme being used, these protocols are often sensitive to communication latency. However, many of the information retrieval applications, especially the online interactive ones, desire the fastest possible response. Their performance will depend on how TCP is being used in the implementation, that is, how much information is being retrieved and whether it can be retrieved as a single transfer or a number of smaller transfers. This effect of TCP is of particular interest to our study and we will examine this issue in more detail later.

Interactive gaming

Certain applications that require instantaneous reaction time, like highly reactive network gaming, do not work over GEO satellites because of physical delay limitations. Other types of interactive gaming, such as chess, would not suffer from the delay.

The following figure summarizes these applications. They vary substantially in demands on bandwidth and responsiveness. The implementation techniques are also very different, as are the impacts of network delays. Some applications require guaranteed delivery; they use TCP and are sensitive to latency. Others can use UDP or other real-time protocols that can tolerate delays. Multicast/broadcast applications use reliable multicast such as SRM, which is less sensitive to delays than TCP and thus works fine over satellite links. It is therefore inappropriate to make simple statements on whether GEO satellites make better Internet access. The purpose of this paper is to show the prospects for Internet distribution over satellites and let users decide by their application priorities.

TCP over long delay networks

On today's Internet, TCP is the protocol used by the vast majority of applications. The performance of TCP over long-delay networks will have a direct impact on the performance of Internet access using GEO satellites. In this section, we will analyze the performance issues in TCP control mechanisms and current approaches to adopt and improve TCP for long-delay networks.

Performance issues in TCP control mechanisms

Window size

TCP flow control starts from the "window size" concept of a TCP connection. It determines how much data can be outstanding (i.e., unacknowledged) in the network. In long-delay networks, there can be many unacknowledged segments. Theoretically, the amount of data that can be in a transit is given by the bandwidth-delay product. In practice, memory and operating system resources limit the window size. In the current TCP standard, the maximum window size in TCP is 64 Kb. (Because of historical implementation issues concerned with signed and unsigned arithmetic, the practical window size is often limited to 32 Kb.)

To maximize bandwidth utilization in a satellite network, TCP needs a much larger window size. For example, on a satellite link with a round-trip delay of 0.8 seconds and bandwidth of 1.54 Mbps, the theoretical optimal window size is 154 Kb, or considerably more than a maximum window size of 32K or 64K.

A new TCP extension, or TCP-LW for "large-window" [6], has been defined to increase the maximum window size from 2¹⁶ to 2³², allowing better utilization of links with large bandwidth delay products. To obtain good TCP performance over satellite links, both sender and receiver use a version of TCP that implements TCP-LW. Applications should also set the size of the send and receiver buffers to be bandwidth times delay.

Bandwidth adaptation

TCP adapts to the available bandwidth of the network by increasing its window size as congestion decreases and reducing the window size as it increases. The speed of the adaptation is proportional to the latency, or the round trip time of the acknowledgment. In a satellite network with longer latency, bandwidth adaptation takes longer and, as a result, TCP congestion control is not as effective. Furthermore, it will take much longer for TCP's linear increase to recover the window size after a packet loss if a TCP "large-window" extension is used.

Selective acknowledgment

The standard TCP acknowledgment scheme is cumulative. If a segment is lost, TCP senders will retransmit all data sent starting from the lost segment without regard to the successful transmission of later segments. TCP considers this lost segment as an indication of congestion and reduce its window size by half. Recently the newly defined standard TCP-SACK (Selective ACKnowledge) allows the receiver to explicitly inform the sender of the loss. Consequently, a sender can retransmit the lost segments immediately rather than waiting for a timeout, reacting to supposed congestion, and multiplicatively decreasing its window. If lost segments are not caused by congestion, or the congestion is transient, throughput in TCP-SACK should be much better. This will be helpful in satellite networks because anything that triggers timeouts and window size reduction will force a lengthy recovery in TCP.

Slow start

When a TCP connection first starts up or is idle for a long time, it needs to quickly determine the available bandwidth on the network. It does so by starting with an initial window size of one segment (usually 512 bytes), then increasing the window size as packets are delivered successfully and acknowledgments arrive, until reaching the network saturation state (indicated by a packet drop). On the one hand, slow start avoids congesting the network before it has a good assessment of the available bandwidth; on the other hand, TCP bandwidth utilization is suboptimal during the procedure. Therefore, the shorter TCP slow start lasts, the better performance it can achieve. The total time of a TCP slow start period is approximately RTT*log₂(B/MSS), where RTT is the round trip time (twice the latency), B the available bandwidth, and MSS the TCP segment size. Although the growth is exponential, for high-bandwidth and long-delay networks (e.g., satellite links and terrestrial gigabit network), this can take a significant amount of time.

Congestion avoidance

Recently, new techniques have been introduced in TCP to avoid congestion before it happens. The first approach, called RED (random early detection) gateways [5], requires each gateway to monitor its own queue length. When imminent congestion is detected the TCP sender is notified. By dropping a packet earlier than it would normally, RED sends an implicit notification of congestion. The sender is effectively notified by the timeout of this packet. The principle behind the RED approach is that a few earlier-than-usual drops may help avoid more packet drops later on. The TCP sender can then reduce its window before serious congestion occurs.

Another approach is to have the TCP sender predict when congestion is about to occur and reduce its transmission window before intermediate routers drop packets (TCP Vegas [3]). TCP can keep track of the minimum round trip time seen during a transfer and use the most recently observed round trip time to compute the data queued in the network. TCP can also keep track of the throughput before and after the congestion window changes to estimate the network congestion level. If estimates indicate that the number of packets queued in the network is rising, it reduces the congestion window. As it observes the number decreasing it increases the congestion window.

Although neither approach has been widely adopted, both hold promise for satellite networks. As we mentioned earlier, TCP congestion control responds to congestion slowly because of latency. If such congestion can be avoided before it happens, it is a big win for high-speed and long-delay networks.

TCP for transactions

Many TCP applications involve only simple communications between the client and the server. The interaction is called a transaction: a client sends a request to a server and the server replies. The Hypertext Transfer protocol (HTTP) for WWW browsing applications is a typical example of TCP with transactional behavior. Under standard TCP, even a small transaction involving a single request segment and a reply must undergo TCP's three-way handshake in preparation for bidirectional data transfer. If the request is bigger than a segment, TCP must also undergo the slow start procedure. It is very inefficient to establish such a TCP connection, send and receive an insignificant amount of data, and then tear it down.

Transaction TCP, or T/TCP [2,10], is an extension to TCP designed to make such behavior more efficient. T/TCP does this by bypassing the three-way handshake and slow start, using the cached state information from previous connections. Although T/TCP is designed mainly for short client-server interaction applications, it can be used to reduce the impact of latency on the beginning of a TCP connection. If slow start can be avoided, significant performance improvement can be achieved in a satellite-based network.

Impacts on end-to-end performance

We have conducted a simple set of simulation experiments to explore the correlation among TCP window size, network bandwidth, latency, and end-to-end performance (response time and throughput). The network simulation tool we used, NS [8], allows arbitrary network configuration and different TCP versions. The figure below describes the network topology of our experiment. We varied the interconnection link bandwidth from 128 Kbps (kilobits per second) to 6.176 Mbps, the latency from 10 ms to 400 ms (which covers local network, LEO, MEO, and GEO satellite links), and the workload (the amount of data sent in one TCP connection) from 1 KB to 100 MB. We also varied the TCP window size from 2 KB to 512 KB.

We recorded the throughput and response time of each TCP connection. Throughput determines the bandwidth utilization of the link from the system manager's point of view, while the response time reflects performance as perceived by the user. A small sample of the results is compiled in the table below:

Bandwidth	6.176 Mbps	1.544 Mbps	0.384 Mbps	0.384 Mbps
Amount Transfer	100M	10K	1M	10K
Measurement	throughput	throughput	response time	response time
Result in figure (click for detail and explanation)

From the above results we can see that latency is a decisive factor in a satellite network only for the response times of small transfers. If an interactive application often opens a TCP connection only to send a small amount of data, it will perform very inefficiently. The next section will suggest better alternatives under this condition.

Approaches for performance improvement

Over the years, people have developed various techniques to mitigate the impact of latency. The first alternative is to adopt a version of TCP that performs better over the satellite and does not hinder performance over terrestrial networks. The second approach relies on Internet gateways at the satellite network boundaries to perform special functions to speed up TCP sessions. The third approach is to develop better implementations of common applications that use TCP more efficiently and more sensitively.

TCP extensions

Some of the TCP problems experienced on GEO satellite links today will arise in future high-speed terrestrial networks because of the similar bandwidth-times-delay property. Problems like large window size, prolonged slow start period, and ineffective bandwidth adaptation affect both networks. They place satellite networks and gigabit terrestrial networks in the same class of extensions for better performance. Some of the techniques discussed earlier, including TCP-LW, TCP-SACK, TCP-Vegas, T/TCP, and RED gateways, can alleviate the problems for both networks. For example, TCP-Vegas can reduce the number of packets lost to congestion, hence reducing long delays after packet drops. In a similar way, TCP-SACK can reduce the need for retransmission of unnecessary data. Other techniques like rate control and selective negative acknowledgments may further improve efficiency. The benefits of applying these extensions/improvements have been demonstrated in MITRE's attempt to develop TCP modifications specifically tailored for space communications [4].

Middleware

Great performance improvements can be made in many cases by working at the Internet infrastructure level without the need to modify TCP. This layer is sometimes referred to as the middleware layer. While enhancements at the transport-layer require changes to the operating system of each end host, enhancements at the middleware layer usually require few or no such modifications.

Split-TCP

The idea of split-TCP (sometimes referred to as indirect TCP), is to break an end-to-end TCP connection into two or three segments. Each segment is itself a complete TCP connection. Data streams are forwarded from one segment to another (buffering if necessary). When split-TCP is applied to a satellite network, the middle segment spans the long-latency satellite link, and the other two segments connect the routers that join the terrestrial Internet and the satellite link to the original endpoints.

Splitting isolates the effects of long latency. If the first and the last TCP segment span a low-latency network, TCP slow start can speed up more quickly and the normal window size (without TCP-LW) will work just fine. The middle segment, however, should implement special features such as T/TCP and use large windows to cope with long latency. This way, TCP performance can be improved with only minor changes to application software.

TCP spoofing

In TCP spoofing, an intermediate gateway (usually at the satellite uplink) prematurely acknowledges a TCP segment without waiting for the actual acknowledgment from the receiver . This gives the sender the illusion of a low-latency network so the TCP slow start phase can progress more rapidly. The intermediate gateway buffers segments in transit. When the actual acknowledgment from the receiver arrives at the gateway, it is suppressed to prevent duplicate acknowledgments from reaching the sender. If the receiver's acknowledgment never arrives and the gateway times out, it retransmits the lost segment from its local buffer.

Like split-TCP, TCP spoofing breaks the concept of end-to-end semantics because the sender may think a segment has arrived at the destination while it is actually still in transit. This is acceptable in many applications, such as WWW browsing through proxy, but it may cause problems if an application is built upon end-to-end semantics.

Application protocol approaches

TCP has its shortcomings when used over long-delay networks. These can be avoided if Internet applications use TCP more effectively, for example, by avoiding small and short transfers, as suggested in the previous section.

Persistent TCP connections

In some client/server-type applications, the client program sends a request to the server and the server replies. If each such request/reply is implemented in a separate TCP session, the overall performance of the application will suffer significantly over a satellite network. However, we can speed up the performance by rewriting the application to use a single TCP session for all the small requests/replies. For example, the current HTTP protocol for WWW applications performs poorly because a Web client fetches each Web object (page of text, icons, images, etc.) in a separate TCP connection. A typical Web page consists of many such small objects. Although the amount of data is moderate, it would take tens of seconds to fetch such a page over a GEO satellite. Recently, a new upcoming standard, HTTP 1.1, alleviates the performance problem by using a persistent connection to combine many small transfers into a single fetch and to pipeline the transfers so that the transmission delays overlap with each other. Using HTTP 1.1, a typical Web page transmission time can be reduced to a few seconds over a GEO satellite, which is comparable to the terrestrial Internet.

Caching

Caching can reduce network utilization and latency as seen by the user. Caches are currently available for protocols in use by the World Wide Web (e.g., HTTP, FTP, and Gopher). It requires minimal cooperation from end users. Caching combined with data broadcast creates a new information retrieval/dissemination paradigm that makes better utilization of network bandwidth. Commonly requested and filtered Web documents are multicast to local caches. Most of the Web requests are satisfied by a nearby cache, with only occasional retrievals from remote servers. This paradigm is better supported by GEO satellite networks because of its broadcast and high bandwidth capability.

Application-specific proxies

Application-specific proxies can use domain knowledge to match network constraints and reduce the effect of latency. For example, a WWW proxy that prefetches Web pages can eliminate round trips across satellite links. An X window proxy that converts synchronous X request/response pairs into asynchronous communications can significantly improve the responsiveness of X window programs (several times faster in many cases) [11].

Multicast

GEO satellites can be a natural media for MBone (the virtual network over Internet for multicast applications [7]) because of their large coverage area and broadcast capability. Currently, to multicast over the terrestrial MBone, the data must travel over numerous links and duplicate itself at numerous intermediate routers (see figure below for a recent map of MBone [1]). This takes up significant bandwidth and raises the possibility of transient congestion from TCP cross traffic at each router along the path.

On the contrary, multicast over satellite can deliver data directly to end-user networks or hosts with minimal cost. The following figure envisions the future MBone with multicast satellite connecting strategic points on the Internet.

To demonstrate the potential role of satellites in Internet multicast, we have set up an experimental MBone link over the NASA ACTS satellite between the NASA Lewis Research Center (LeRC) in Ohio and Hughes Research Laboratories (HRL) in California. The link allows two sites to connect directly instead of going through tens of hops over the terrestrial MBone. (See above figure for location of LeRC and HRL.)

Internet applications that use multicast can gain substantial benefit from a satellite network. First, multicast applications do not use TCP and are not sensitive to long bandwidth delay products in the same way TCP is. Moreover, satellite networks can bypass potentially tens of intermediate nodes, thus reducing the chances of packet drop and large delay jitter attributable to congestion. To illustrate this, we have conducted an MBone videoconferencing experiment between LeRC and HRL. LeRC transmits two identical video streams with video tool vic [9] to HRL simultaneously, one over terrestrial MBone and the other over MBone over ACTS. At HRL, two workstations receive the streams respectively, and record the following quality of service (QoS) parameters at the receiver: packet loss, delay jitter, bandwidth, and frame rate. We set the transmission rate at the source at 128 Kbps and 8 frames per second. (Because of the current MBone bandwidth restriction on the terrestrial Internet, we limited the bandwidth to 128 Kbps, otherwise the performance advantage of MBone over ACTS would be even bigger.) The following table shows the performance comparison for each QoS parameter.

Measurement	bandwidth	frame rate	transient packet loss rate	packet delay variation
Results in figure (click for details)

The first and second charts compare packet loss and delay jitter. Along the terrestrial MBone path, the packet loss rate is well above 70 (first chart) and jitter occasionally hits 200 milliseconds or more (second chart). These results translate into poor performance through low frame rate and low bandwidth. The third chart shows that the video bandwidth received over terrestrial MBone is only one-fourth of that received over ACTS, and the fourth chart shows that the frame rate over terrestrial MBone is only one-sixth. However, we observe no packet loss over ACTS from the first chart, and from the second chart jitter was kept under 100 milliseconds. Consequently, the receiver gets the original bandwidth and frame rate transmitted by the sender. These results imply that the use of GEO satellites as part of MBone can deliver high-quality services to end users in a bandwidth- and cost-effective manner.

Conclusion

Satellite networks promise a new era of global connectivity, but also present new challenges to common Internet applications. In this work, we have shown that many popular Internet applications perform to user expectation over satellite networks, such as videoteleconferencing, MBone multicast, bulk data transfer, background electronic mail, and non-real-time information dissemination. Some other applications, especially highly interactive applications such as Web browsing, do suffer from the inefficiencies of the current TCP standard over high-bandwidth long-latency links. However, performance of these applications can be improved by utilizing many of the techniques discussed in this paper. In the long term, further improvements can be made at the protocol level by extending the current TCP standard, although much work needs to be done on possible extensions to ensure that they do not negatively affect the Internet as a whole.

Acknowledgments

We would like to extend our thanks to NASA Lewis Research Center for its assistance in some of the satellite experiments. We would also like to thank the Hughes Spaceway Group for valuable feedback and technical assistance.

References

1. E. Amir. A map of the MBone: August 5th, 1996. http://www.cs.berkeley.edu/~elan/mbone.html.
2. R. Braden. Extending TCP for transactions-concepts. Internet Request for Comments RFC 1379, November 1992.
3. L. S. Brakmo, S. W. O'Malley, and L. L. Peterson. TCP Vegas: New techniques for congestion detection and avoidance. ACM SIGCOMM 1994, pages 24-35, May 1994.
4. R. C. Durst, G. J. Miller, and E. J. Travis. TCP extensions for space communications. Proceedings of the 2nd ACM Conference on Mobile Computing and Networking, November 1996.
5. S. Floyd and V. Jacobson. Random early detection gateways for congestion avoidance. IEEE/ACM Transactions on Networking, 1(4):397-413, August 1993.
6. V. Jacobson, R. Braden, and D. Borman. TCP extensions for high performance. Internet Request for Comments RFC 1323, May 1992.
7. M. R. Macedonia and D. P. Brutzman. MBone provides audio and video across the Internet. IEEE Computer, 27(4):30-36, April 1994.
8. S. McCanne. ns - lbnl network simulator. http://www-nrg.ee.lbl.gov/ns/.
9. S. McCanne and V. Jacobson. vic: A flexible framework for packet video. ACM Multimedia, November 1995, San Francisco, California, pages 511-522, November 1995.
10. W. R. Stevens. TCP/IP Illustrated, Volume 3. Addison-Wesley Publishing Company, 1996.
11. Y. Zhang and S. K. Dao. HBX: High bandwidth X for satellite internetworking. In Proceedings of the 10th X Technical Conference, X Resource, Issue 17, February 1996.
12. Y. Zhang and S. K. Dao. Integrating Direct Broadcast Satellites with Local Wireless Access, In Proceedings of the First International Workshop on Satellite-Based Information Services (WOSBIS), Rye, New York, November 13, 1996.