Starlink simulation shows low latency without inter-satellite laser links

Handley's simulation shows that, while not as fast as an equivalent ISLL path, long bent-pipe paths would typically have lower latency than terrestrial fiber routes between the same two points.

Mark Handley, a professor at University College London, has made two terrific videos based on runs of his simulation of the first -- 1,584 satellite -- phase of SpaceX's Starlink Internet-service constellation. I discussed the first video, which assumes that the satellites have inter-satellite laser links (ISLLs), in recent post.

While SpaceX plans to deploy ISLLs in the future, their early satellites do not have them since at 27,000 km/hr they are state-of-the-art technology and may also encounter political problems in some nations. Since it could be a year or so before SpaceX begins launching ISLL-equipped satellites, Handley has made a second video that assumes the phase one satellites do not have ISLLs. This post discusses that video.

Satellite footprint (source)

Each satellite has four phased-array antennas that can rapidly switch narrowly focused connections to terrestrial antennas falling within a large "footprint" area. The terrestrial antennas might be Internet-connected ground stations or end-user terminals. If there were no ISLLs, long-distance traffic would have to be relayed by bouncing packets up and down between satellites and the ground.

Many people -- me included -- have assumed that these "bent pipe" hops would significantly increase latency on long-distance paths, but Handley's simulation shows that, while not as fast as an equivalent ISLL path, long bent-pipe paths would typically have lower latency than terrestrial fiber routes between the same two points.

Sample Seattle-New York path

Consider, for example, this six-hop route between Seattle and New York. The bent-pipe route has a round-trip time of 36ms versus 78ms for the current Internet and 38ms for an hypothetical great circle fiber route, which would be impossible because of mountains and other obstructions.

That example was taken from a run in which only six orbital planes had been populated and it assumed ground stations at popular SpaceX locations plus a few others that Handley assumed would be added at strategic points. With only six orbital planes, global coverage is spotty but it is solid around 53 degrees north (and south). As more satellites are added, coverage becomes wider and latency improves. By the time all 1,584 satellites are in operation, there is global coverage and latency is consistently better than today's terrestrial Internet.

User terminals as relays

While adding satellites improves performance, adding additional ground stations has an even greater impact. That suggests the possibility of relaying traffic through idle end-user terminals, which also have phased array antennas. Handley ran a simulation assuming relays every 100 km and found that latency across the US was roughly cut in half and jitter (latency variance) also declined, but the number of route changes increased to about one every five seconds. That sounds like a lot of overhead but Handley feels that it is feasible to handle. It would also require a more expensive user terminal, a little power and the permission of the user so SpaceX might subsidize the terminals or charge less for service.

Handley also considered inter-continental relaying, which would require relay stations on strategically placed ships at sea. (It turns out that no ships would be needed to cross the Northern Pacific, but that would require a relay station in Russia, which might be a political problem). He doesn't mention the possibility, but couldn't cargo and cruise ships act as slowly moving relay stations? (They will certainly want to be terminal-users).

The example shown above is for an east-west link but Handley also looked at long north-south links and found that ground relays actually beat ISLLs in some cases and were always better than fiber, but the best results are achieved by a combination of ISLL and terrestrial links, which we can look forward to once SpaceX and others begin deploying satellites with ISLLs.

Handley concludes by pointing out that since he started making the video, SpaceX had revised their constellation configuration from 24 66-satellite planes to 72 22-satellite planes. It turns out that once the first 1,584-satellite phase is complete, there is pretty much no difference between the new and old configurations, but it does require a few more satellites to be deployed before the trans-Atlantic and Pacific relays will work continuously. Note that SpaceX hopes to complete the first phase by early 2021.

I can't conclude this post without mentioning Handley's charming disclaimer that he has no inside information, but, based on public statements, has made reasonable assumptions about "what they could do if they wanted to, but probably isn't what they will actually do."

Watch the video:



Update 1/1/2020

Handley presented a paper on this research at the 2019 ACM HotNets Conference. You can see a video of his presentation and download a copy of his paper here. (The video of his talk is free, but the paper is behind a paywall).

Update 1/13/2020

There has been further discussion of this topic in the Reddit Starlink Community. Commenters have pointed out that ISLLs are cutting edge technology and the current cost of Mynaric's 10 Gbps terminals is prohibitive, though it will doubtless drop with mass production (and Mynaric has hired an ex-SpaceX executive). While SpaceX has announced plans to launch satellites with ISLLs by the end of 2020, those may just be for testing. Furthermore, they will have many legacy satellites in orbit by then and those will not be replaced for around five years. SpaceX will not have a 100% ISLL constellation until 2026. Perhaps OneWeb made a wise decision in postponing ISLLs and Wall Street arbitrage traders will have to wait a few years for ultra-low latencies.

Inter-satellite laser link update

SpaceX satellite mesh with four
laser terminals on each satellite

Inter-satellite laser links (ISLLs) and electronically steerable flat panel antennas are critical technologies for constellations of low-Earth orbit (LEO) Internet-service satellites. Low-cost antennas are critical for the mass consumer market and ISLLs are required for an effective Internet backbone in space. In an earlier post, we saw that progress is being made on antennas, this one looks at ISLLs.

The figure to the right is taken from a simulation of the first phase of SpaceX's planned broadband Internet service, Starlink. It shows 66 satellites in each of 24 53-degree orbital planes -- a total of 1,584 satellites at an altitude of 550 km. Each satellite has four laser-communication terminals. Two on the front and back and two on the sides. Since the front and back lasers link to satellites in the same orbital plane, they remain at the same place in the sky relative to each other while the side lasers must move to track one another. (To visualize the dynamic nature of the links between the constantly moving satellites, check this clip from the animated simulation).

When Elon Musk introduced his Starlink plan to prospective employees in 2015, he said his goal was to transport "a majority of long-distance Internet traffic" and "about 10 percent of local consumer and business traffic." He pointed out that satellites would have an advantage over terrestrial links since the speed of light is faster in space than through optical fiber and fewer router hops would be needed to reach a distant location.

In addition to mitigating the digital divide by serving rural areas and small organizations, Musk and his competitors at OneWeb, Telesat, Amazon, and Leosat hope to service high-end, high-margin customers like enterprises, governments and maritime, airline and mobile phone companies. ISLLs are necessary for serving those lucrative high-end markets.

Initially, SpaceX proposed five ISLLs for each satellite -- the fifth would have been a link to a satellite in the crossing plane, but last November they cut back to four. The fifth terminal would have been difficult to engineer because while the front, back and side-mounted terminals move slowly relative to each other, this simulation shows that satellites in crossing planes would have been traveling at 7.3 km/second relative to each other. Furthermore, links between satellites in crossing planes would be of short duration. Designing and manufacturing them would have taken time and money.

Furthermore, because of the 53-degree orbit inclination, about half the satellites are moving northeast and half are moving southeast at any time and place. That favors east-west links over north-south links and since most of the lucrative low-latency, long-link traffic is in the northern hemisphere, they could not justify the cost or possible deployment delay. That is not to say they will not deploy them in the future. (Note that the initial five-link constellation was to orbit at an altitude of 1,100, not 550 km. Future plans call for constellations at 1,100 and 335-345 km and there may be ISLLs between all of them).

Tesat laser communication roadmap

But even with 4-links, the terminals are still under development and will be expensive. At least two companies are working on ISLLs, Mynaric and Tesat.

Tesat already markets a laser communication terminal for LEO to ground transmission from CubeSats. Their CubeLCT is 9 x 9.5 x 3.5 cm, has a mass of 360 grams, consumes 8 Watts of power and communicates through the atmosphere to the Earth at 100 Mbps, with a 1 Mbps channel from the ground to LEO. They are developing an ISLL terminal based on that experience and, judging from the diagram shown here, they are pursuing laser communication between the ground, LEO and geostationary satellites.

Mynarc has announced that their ISLL terminal, the MLT-80, will be available in high-volume production this year and both companies are working on faster terminals. A while ago, I suggested that SpaceX would probably develop their own ISLL, but last March, Bulent Altan, a former SpaceX Vice President, joined Mynarc as co-CEO and a few days later Mynaric announced that they had raised $12.5 million from mystery constellation customer. Might the mystery company be SpaceX? Might it be Amazon, which entered the race late and has enough money to pay for terminals or even buy a stake in Mynaric or Tesat? We will know soon because test satellites equipped with Mynaric’s terminals should be launched in late-2019.

The following are selected characteristics of their forthcoming ISSLs:

	Mynaric	Tesat
Link distance	4,500 km	6,000 km
Data rate (full duplex)	10 Gbps	10 Gbps
Target mass	<20 kg	<15 kg
Power consumption	<60 W	80 W

Sources Tesat, Myarnic

The SpaceX simulation shown above was for satellites with 4 ISLLs, but SpaceX launched their first 60 satellites without the ISLLs and, as far as I know, has not said if forthcoming satellites will have them or not. Arthur Sauzay, a French environment and space lawyer has pointed out that SpaceX argued for the allocation of radio frequencies for ISLs in a comment to a recent Whitehouse report on the impact of emerging technologies and their impact on non-federal spectrum demand, but they seem too large, heavy and slow to support a LEO network with long-distance, low-latency links.

OneWeb has decided not to use ISLs in their first constellation and will route traffic through terrestrial gateways. This decision seems to have been at least partially motivated by Russian insistence that satellite traffic passes through gateways within their borders. I imagine China and other nations will impose the same restriction.

Telesat remains committed to ISSLs, but say they will have the flexibility in their network-control system to route traffic coming to a country over satellite or terrestrial links. Erwin Hudson, vice president of Telesat LEO is confident that ISLs will be cheap enough to allow them to compete successfully with terrestrial fiber and 5G, offering fast, 30 ms latency broadband. They also have a $2.8 million contract to study inter-satellite laser links between their constellation and Blackjack, DARPA's 20 LEO satellite constellation and they are collaborating with Google on software, so we might see laser links between Telesat satellites and Google's balloons.

LeoSat is unique in that they are not pursuing the consumer and small organization markets, but are focused exclusively on large, high-end customers. They will provide fast, low latency, encrypted, reliable point-to-point connections to governments at up to 1.2 Gbps with latency under 50 ms and they have over $1 billion in pre-launch customer agreements. ISLLs are mandatory for the markets they are pursuing and since two geostationary satellite operators, Jsat and Hispasat, are investors in LeoSat, they may very well link to them in the future to offer a service similar to the SpaceDataHighway of Airbus and the European Space Agency.

China's Hongyun LEO broadband project is an ISLL unknown. China is doubtless working on laser communication in space, but I have no idea whether or not they will use it in their broadband constellation. Since they say the goal of the project is to serve rural China and they regulate Internet links to the outside world, Hongyun satellites may serve exclusively as "bent pipe" relays between rural locations and China's terrestrial network.

ISLLs will be needed if the Internet backbone in space is to compete with the terrestrial backbone and serve high-value applications. It seems that making cost-effective ISLLs for LEO constellations was harder than Elon Musk and others anticipated, but first production models are now on the horizon and they will improve over time.

For a copy of the PowerPoint presentation I use for teaching this topic click here.

An optimistic update from Telesat

Once the 100 inclined-orbit satellites are in orbit, they may be able to utilize their inter-satellite laser links to achieve the 30 ms latency Goldman spoke of.

Polar (green) and inclined (red) orbits

Emily Jackson interviewed Dan Goldberg, Telesat President and CEO, in a recent episode of the Down to Business podcast. The interview followed the announcement that the Canadian Government would contribute $85 million (all amounts are in Canadian dollars) to support research and development in support of Telesat's planned constellation of low-Earth orbit (LEO) satellites and another $600 million to subsidize Internet connectivity in rural Canada.

Goldberg pointed out that all governments subsidize rural connectivity and said the $600 million grant was expected to generate $600 million in revenue from below-market-rate sales to telephone companies and ISPs. The remaining capacity would be sold to others and he said they anticipated sales to enterprises, governments, ships, and airlines, but did not mention marketing directly to consumers. (Only SpaceX seems to be targeting consumers from the start).

In return for the R&D contribution, Telesat has agreed to support approximately 500 professional jobs in Canada and invest $215 million in R&D. (That R&D includes the first dozen or so test satellites). Telesat has a profitable, established geostationary satellite business and will fund part of the constellation themselves, but they will also need debt and equity financing and Goldberg said this government support would make it easier for them to finance the constellation.

This financial news is important, but Goldberg's optimism about the technology is what caught my attention. They have been working on their LEO project for six years and during that time the cost of launching satellites -- geostationary as well as LEO -- has fallen dramatically and he expects it to continue to do so. He also predicted that the cost of mass-produced satellites will fall dramatically and he is confident that inter-satellite laser links (ISLLs) and electronically-steerable phased-array antennas will be cheap enough to allow them to compete successfully with terrestrial fiber and 5G, offering fast, 30 ms latency broadband. (ISLLs present both technological and political problems).

The only technological concern he expressed was with regard to the problem of radio interference. He did not say anything specific on these technologies but did point out that Telesat has been providing satellite service for 50 years and is the "leading satellite technical consultant" in the world. (Three percent of their revenue is from consulting).

Goldberg summed up his optimism by saying:

Our confidence level in terms of our ability to bring this disruptive capability to the market and provide an extraordinarily high-quality, disruptive broadband service to Canadians and also to everybody else living in the world is extraordinarily high. This is not some high, big-gamble, futuristic new technology. This technology will be disruptive but it is ready for prime time.

Yes, but ...

SpaceX simulation with uncovered areas

Goldberg said they could could achieve global coverage with only 72 satellites and a simulation by Mark Handley predicts that SpaceX will not completely cover the planet with 792 satellites. How do we explain the difference?

SpaceX with 792 satellites would have much more capacity than Telesat with 72 satellites and Telesat does not plan to offer service with only 72 satellites. They plan to start service at the end of 2022 with around 200 satellites in polar orbit. They will add 100 more in inclined orbit in 2023 and perhaps eventually reach 500 satellites. Those 200 polar-orbit satellites will serve the polar regions, fulfilling their promise to provide connectivity in rural Canada. (This is reminiscent of China's Hongyun LEO satellite project which will focus on rural China).

While the 200 polar orbit satellites will provide coverage in rural Canada, they will be partially reliant upon terrestrial ground stations to reach the entire globe and therefore latency will suffer and they already have two far-north ground stations in support of their established, profitable geosynchronous satellite business. Furthermore, in 2016 Telesat filed for a patent on a "Dual LEO Satellite System and Method for Global Coverage" and once the 100 inclined-orbit satellites are in orbit, they may be able to utilize their inter-satellite laser links to achieve the 30 ms latency Goldman spoke of.

Are inter-satellite laser links a bug or a feature of ISP constellations?

I hope each of these companies has someone in charge of thinking about what might go wrong with a single, satellite-based network providing fast, low-cost links anywhere on the global Internet.

Inter-satellite laser links (source)

OneWeb, SpaceX, Telesat and Leosat all aspire to be global Internet service providers using constellations of low-Earth orbit (LEO) satellites. Their success will require still-unproven technological innovation, but there are also political stumbling blocks.

OneWeb has already encountered significant political problems in Russia. Russia launched OneWeb's first test satellites and has a billion dollar contract for 20 additional launches, but Russian security officials are lobbying against OneWeb's offering service on the grounds that it might facilitate spying. When Anatoly Zak, an expert on the Soviet space program, investigated that claim, he concluded that "With the launch of the OneWeb constellation, the Russian rocket industry stands to earn millions, but the Kremlin is terrified at the prospect of unhindered access to the Internet by its citizens."

As far as I know, OneWeb is still planning to offer service in Russia, but they have had to make financial and technical concessions. They agreed to become a minority partner in the company that will market their service in Russia and, significantly, they agreed to drop the inter-satellite laser links (ISLLs) from their constellation and pass all Russian traffic through ground stations in Russia.

Dropping ISLLs, which are still unproven for this application, will simplify the design of their satellites and save development time and cost. It will also reduce satellite complexity, size, and weight and save power, but there will be costs.

OneWeb throughput simulation (source)

They will need more ground stations if they offer global service without ISSLLs. OneWeb founder Greg Wyler says they will have more than 40 such gateways, each capable of “seeing” satellites up to 4,000 kilometers away. A team of MIT researchers ran a simulation of a 720-satellite OnWeb constellation and they estimate that 71 ground stations would be required to reach maximum throughput. (Anatoly Zak estimates that four to six gateways will be in Russia and speculates that hackers may be able to illegally connect to satellites in neighboring countries). Dropping ISLLs will also add latency, especially on long-distance links.

What about the other would-be global LEO projects?

Telesat will retain ISSLs, but accommodate countries on a country-by-country basis. Erwin Hudson, vice president of Telesat LEO, says we "have the flexibility in our network control system to route traffic all kinds of different ways. There are no rules that traffic has to go over the inter-satellite links."

SpaceX is going forward with ISSLs, and they are also aware of the political problems. During a recruiting talk at the opening of their Seattle satellite-design office four years ago, CEO Elon Musk said "I'm hopeful that we can structure agreements with various countries to allow communication but it is a country-by-country basis ... it's not gonna take longer than five years to do that and not all countries will agree at first ... that's fine."

Leosat is also prepared to build gateways to make accommodations. For example, CEO Mark Rigolle says they would be willing to build a gateway in China to accommodate the government; however, that would be inconsistent with their primary marketing focus of providing low-latency, secure, point-to-point links globally.

These companies are all thinking about concessions they need to make in order to operate in countries that want to surveil citizens and control their access to information, but what about guarding against aggression? I hope each of these companies has someone in charge of thinking about what might go wrong with a single, satellite-based network providing fast, low-cost links anywhere on the global Internet.

During the recruiting talk mentioned above, Elon Musk said "[the constellation design] is a really difficult technical problem to solve so that's why we need the smartest engineering talent around the world to solve the problem and, you know, to also make sure we don't create Skynet." At that time, I asked "Would global Internet service providers require unique regulation and, if so, what should it be and who has the power to do it?" and said I was "less worried about Musk creating SkyNet than creating Comcast on Steroids," but I was naive.

Follow these links for background on the four projects: SpaceX, OneWeb, Telesat and Leosat.

Update 1/13/2020

OneWeb hopes to establish three ground stations in China. They have signed a "framework agreement" for one in Sanya, a city in southern China, and hope to have that and two others approved. They are also seeking partners to market and support their service. They plan a total of 45 global ground stations.

Can constellations of Internet routing satellites compete with long-distance terrestrial cables?

The goal will be to have the majority of long-distance traffic go over this network.
Elon Musk

SpaceX orbital path schematic, source

Four companies, SpaceX, OneWeb, Boeing and Leosat are working on constellations of low-Earth orbiting satellites to provide Internet connectivity. While all four may be thinking of competing with long, terrestrial cables, SpaceX CEO Elon Musk said "the goal will be to have the majority of long-distance traffic go over this (satellite) network" at the opening of SpaceX's Seattle office in 2015 (video below) and Leosat is focusing on high-end fast, point-point links.

Can he pull that off?

Their first constellation will consist of 4,425 satellites operating in 83 orbital planes at altitudes ranging from 1,110 to 1,325 km. They plan to launch a prototype satellite before the end of this year and a second one during the early months of 2018. They will start launching operational satellites in 2019 and will complete the first constellation by 2024.

The satellites will use radios to communicate with ground stations, but links between the satellites will be optical.

At an altitude of 1,110 kilometers, the distance to the horizon is 3,923 kilometers. That says each satellite will have a line-of-sight view of all other satellites that are within 7,846 kilometers, forming an immense mesh network. Terrestrial networks are not so richly interconnected and cables must zig-zag around continents and islands if undersea and other obstructions if under ground.

Latency in a super-mesh of long, straight-line links should be much lower than with terrestrial cable. Additionally, Musk says the speed of light in a vacuum is 40-50 percent faster than in a cable, cutting latency further.

Let's look at an example. I traced the route from my home in Los Angeles to the University of Magallanes in Punta Arenas at the southern tip of Chile. As shown here, the terrestrial route was 14 hops and the theoretical satellite link only five hops. (The figure is drawn roughly to scale).

So, we have 5 low-latency links versus 14 higher-latency links. The gap may close somewhat as cable technology improves, but it seems that Musk may be onto something.

Check out the following video of the speech Musk gave at the opening of SpaceX's Seattle office. His comments about the long-distance connections discussed here come at the three-minute mark, but I'd advise you to watch the entire 26-minute speech:

Update 10/30/2017

Elon Musk has set a goal of having the majority of long-distance Internet traffic go over the SpaceX satellite network. The key to that is a richly connected, optical mesh linking their fast-moving satellites.

SpaceX is building a vertically-integrated organization -- rockets, satellites, ground stations, etc. are all being designed and manufactured in-house as opposed to OneWeb, which is working with strategic investors and partners. Based on this, I suspect that SpaceX is designing their own optical mesh network for inter-satellite communication.

I've not heard OneWeb talking about optical links between their satellites but, if they are planning for that, they might be seeking a partner and Mynaric may be a likely candidate. Mynaric says they have solved the problem of steering a narrow laser beam sufficiently accurately to keep it locked on a target only centimeters in diameter on a moving platform hundreds of kilometers away. OneWeb might satisfy their inter-satellite communication requirement by partnering with Mynaric.

Mynaric technology might be a good fit for OneWeb and other airborne platforms.

=====
Update 12/21/2017

As stated above, Elon Musk set a goal of having the majority of long-distance Internet traffic traverse the SpaceX satellite network. He based that goal on an inaccurate assumption about terrestrial networks and said nothing about competition from other LEO satellite networks. Let's look at both factors.

In his talk, Musk claimed that a route from Seattle to South Africa would involve 200 router hops. That is a major overstatement. I just ran a trace from my home to Durbin Technical University in South Africa and it was only 16 hops with an average latency of 350 ms. Musk also asserted that it would take only 2 or 3, perhaps four hops via satellite. My estimate to Chile, shown above, was 5 hops so that claim may have been a little optimistic. Regardless, it is far fewer than 200 hops.

Second -- Musk drew a comparison to terrestrial cables, pointing out that transmission is slower through a cable than in space and cables had to weave around geographic obstacles. That is true, but it seems that SpaceX will have at least one strong, satellite-based long-link competitor, Leosat. Leosat is focusing on the market for low-latency, secure, point-point links.

I can't believe I am contradicting Elon Musk (I am a big fan), but he may not reach his goal in the long-distance Internet service market.