Sunday, March 27, 2022

Five thousand SpaceX Starlink terminals for Ukraine

Starlink terminals at the Lviv IT Cluster
On March 1, I wrote that a small number of SpaceX Starlink terminals had arrived in Ukraine, and they would be an important asset for distribution to selected government and resistance leaders and journalists. I didn't know who would get the terminals or how many there were, but it was a single truckload. A week or so later, we learned that two more shipments of terminals had arrived and fifty of them went to DTEK, a company struggling to repair Ukrainian electrical infrastructure. However, as far as I knew, it was still a small number of terminals.

That changed on March 19 when the Washington Post reported that "A person familiar with Starlink’s effort in Ukraine, speaking on the condition of anonymity to discuss sensitive matters, said there are more than 5,000 terminals in the country." We'd seen pictures of three truckloads before, but this was a whole new level. Enough for armed forces, infrastructure companies like DTEK, NGOs, and government leaders.

I am speculating, but I think those 5,000 terminals may have been purchased in a joint project of SpaceX and Lviv_IT_Cluster, a community of over 180 leading information technology companies, authorities, and educational institutions employing over 25,000 IT experts that appears well on the way to achieving the goal of making Lviv the technological center of Eastern Europe. 

On March 22, Lviv IT Cluster announced they had purchased a large number of Starlink terminals, and the first batch had arrived in Lviv. By March 26, they were distributing terminals to critical infrastructure providers.

Lviv IT Cluster says these terminals were not contributed but were purchased at a "most-favorable" rate and while the number of terminals was not disclosed it is "sufficient to support critical and military infrastructure as well as IT business in Western Ukraine."  The terminals will not be resold, but "will be handed over to strategically important enterprises and institutions such as electric utilities, medical system, rescue services, security system, power grids, transport hubs, government agencies, and much more."

I'm impressed by the speed of delivery of this aid to Ukraine as well as its scope. Terminals were delivered to Lviv, in western Ukraine and distributed to users a few weeks after an email exchange between Lviv IT Cluster CEO Stepan Veselovskyi and Elon Musk. 

Niels Groeneveld has compiled a list of fourteen types of military aid that have been provided to Ukraine by twenty-seven nations. I would add Starlink terminals to the list.

Update 3/28/2022

Aerorozvidka, a specialist air reconnaissance unit within the army that was created by model plane enthusiasts in 2014, has been picking off tanks, command trucks, and vehicles carrying electronic equipment since the invasion began. They rely on Starling for secure communication and "strike at night when Russians sleep ... we use a drone with thermal vision at night, the drone must connect through Starlink to the artillery guy and create target acquisition.” They also have bomb-equipped drones. 

Hundreds of civilian drone fliers using public communication channels have also been mobilized. I bet Putin didn't plan on that.

Identifying targets using drones with night-vision cameras

Russian tanks, stranded after the destruction of Russian supply trucks

Update 4/8/2022

Here is an example of citizens using a Starlink terminal when the terrestrial Internet is unavailable. When the Russians retreated from a town near Kyiv, electricity and mobile communications were down, but residents were able to communicate through this Starlink terminal with its Wi-Fi router.


Update 4/10/2022

On April 5, The US Agency for International Development (USAID) reported that SpaceX donated "roughly $10 million" worth of 3,667 Starlink terminals and associated internet service to Ukraine. 5,000 terminals were delivered in total, with USAID buying "the additional 1,333 terminals." That press release was replaced the following day with a release omitting the details and simply saying 5,000 terminals had been delivered. This article adds more details.
I asked SpaceX and USAID if these were the same 5,000 terminals described in this post but got no answer.

Update 4/20/2022

Nexta_TV reports that there are 10,000 Starlink terminals in Ukraine. If that is the case, the 5,000 terminals described in this post and the 5,000 terminals attributed to USAID & SpaceX were separate shipments.

Update 4/22/2022

Mykhailo Fedorov, Vice Prime Minister and Minister of Digital Transformation of Ukraine has confirmed that there are now 10,000 Starlink terminals in Ukraine. An unspecified number of them have been given to the Army and others are being used to maintain the operation of critical energy and telecommunications facilities, in health care facilities, and even in agriculture. This announcement was also reported here.



Update 6/9/2022

In an all-hands company presentation, Elon Musk said there were now 15,000 Starlink terminals in Ukraine, and as of May 2, Mykhailo Fedorov, Vice Prime Minister and Minister of Digital Transformation of Ukraine said there were about 150.000 daily Starlink users. I wonder how many there are now.



Monday, March 21, 2022

SpaceX to launch satellites for competitor OneWeb

Elon Musk may be the richest person in the world because he is motivated by more than profit.

Covering the US flag on a Russian rocket (image source)
On March 2, Russia covered the US flag on the Roscosmos Space Agency rocket that was scheduled to launch 36 OneWeb broadband satellites on March 5. The Russians made two obviously untenable demands -- that OneWeb guarantee that the satellites would not be used for military purposes and the United Kingdom government remove its investment in the company. OneWeb declined and the satellites were removed from the rocket. 

On March 17, speaking at a USAID forum, Bala Balamurali, OneWeb's Director for Southeast Asia & the Pacific, said the plan to offer service later this year had slipped to early next year due to the Ukraine-Russia conflict. The schedule delay and loss of expensive satellites and payment for this and future launches was a major setback for OneWeb and they began searching for a new launch provider.

The plaques on the "wall of patents" at Tesla were replaced
by a mural (image source)
They found one -- broadband competitor SpaceX. Coming to the rescue of a competitor might seem like bad business, but Elon Musk may be the richest person in the world (with the possible exception of Putin) because he is motivated by more than profit. Musk does not see the success of the Starlink broadband business as an end in itself, but as contributing to his larger goals of extending the scope and scale of consciousness beyond Earth, achieving AI-human symbiosis, and transitioning to sustainable energy.

In 2014, Musk made a similar decision in support of his goal of transitioning the world to sustainable energy when he released Tesla's 249 patents into the public domain, saying "All Our Patent Are Belong To You" (derived from an obscure meme). He open-sourced them.

Update 4/22/2022

OneWeb has also contracted for satellite launches with New Space India Limited (NSI), the commercial arm of the Indian Space Research Organisation. They will launch satellites with both NSI and SpaceX in 2022, so the Russian cancellation might not cause a large delay.

Update 4/272022

OneWeb has announced that they will begin service in India in early 2023. Before the Russian invasion of Ukraine, they planned to begin serving India in mid-2022. The invasion cost their schedule to slip by about six months.

Update 3/27/2023

OneWeb has given up on retrieving satellites left in Russia

CEO Neil Masterson said that the company has largely given up on retrieving 36 satellites currently held by Roscosmos, saying "I spend no time thinking about it. We've completely moved on ... There is value in getting them back, but I can tell you that I'm not getting them back any time soon."

Tuesday, March 08, 2022

SpaceX Starlink in Ukraine -- a week later

The Internet has been down in the city of Mariupol since March 2. If there is a Starlink terminal there, it is still online.

Ten Starlink satellites serving Odessa through ground
stations in Turkey, Lithuania, and Poland

Last week I wrote about the arrival of a truckload of SpaceX Starlink terminals in Ukraine and their potential value to government and resistance leaders. A lot has happened in the ensuing week -- this is an update.

Last week, using Mike Puchol's Starlink tracking service, I found that users in Kyiv would have 100 percent uptime with connections through as many as nine satellites to ground stations in Turkey, Poland, and Lithuania. When the Turkish ground station was unreachable it dropped to as few as five, but uptime remained 100%.

I've since used his service to check connectivity snapshots in three geographically dispersed cities -- Odessa, Lviv, and Kharkiv. I always found between seven and ten satellites in service in Odessa and Lviv and between two and six in Kharkiv. This is not surprising since Lviv is west of Kyiv, hence closer to the Polish and Lithuanian ground stations, Odessa in the south is closer to the one in Turkey and Kharkiv is to the east -- near Russia.

Starlink Speedtest in Kyiv
Puchol's simulation mode shows there should be 100% availability throughout the country at all times and, since there are few terminals, performance should be good. That was confirmed by an OOKLA Speedtest run by Oleg Kutkov in Kyiv. During the third quarter of 2021, OOKLA observed median upload and download speeds of 87.35 and 13.54 Mbps in the United States and as shown here, Kutkov observed much faster download speed and similar upload speed. The Kyiv test showed a "ping" time of 75 ms compared to a median "latency" of 44ms reported by OOKLA. I'm not sure if their reported "latency" is one-way or round trip. If the latter, they are comparable,

There have been several software updates since last week. Most importantly Elon Musk tweeted that SpaceX had enabled roaming in Ukraine so anyone with a terminal can transmit for a while then turn it off and move to another location or, if necessary, can remain online in a moving vehicle. In the same Tweet, Musk announced that they had reduced peak power consumption so the terminal could be powered by a car cigarette lighter. (There must have been some performance hit).

Musk also tweeted that "Some Starlink terminals near conflict areas were being jammed for several hours at a time" and they were bypassing the jamming with a software update. He added that he was curious to see what they tried next -- this sounds like "whack-a-mole."

It's noteworthy that in our "software-defined everything" world, SpaceX can make significant changes to the constellation with a software update. (The downside is illustrated in an over-the-air software hack of ViaSat modems).

Musk warned that Starlink terminals could be targeted and advised users to turn them on only when needed, to place an antenna as far away from people as possible, and to cover it with light camouflage. He also tweeted that some governments (not Ukraine) had asked him to block Russian news sources, but, as a "free-speech absolutist," he refused to do so.

President Zelenskyy Tweeted that the second shipment of terminals was on its way after speaking with Musk. 

In related news, Anonymous has declared cyberwar on Russia and there are Telegram channels for IT professionals in support of Ukraine, one of which is English-language.

Finally, there was also a PR tweet by Ukraine's famed heavyweight champion boxers the Klitschko brothers posing with a couple of Starlink terminals. 

Some Musk critics see his disaster-relief efforts or delivering terminals to Ukraine as publicity stunts of little practical value. Starlink terminals in Ukraine are terrific publicity, but they are also valuable tools for communication by political and resistance leaders if they are unable to safely access the Internet or it is blocked. For example, it has been reported that Mariupol is without electricity and water, and the Internet has been down since March 2. If there is a Starlink terminal there, it is still online.


Update 3/10/2022
Oleg Kutkov reported faster connectivity with a wired connection to the router instead of WiFi.

Update 3/14/2022

Photos of Starlink terminals in various locations have been posted on Telegram's Ukrainian IT Army and Facebook's SpaceX Starlink in Ukraine groups, but little was said about their use and users. 

Fifty of the square-antenna terminals from the second shipment have gone to the DTEK Group. Half of them will be used for support of Ukrainian energy infrastructure and half will go to DTEK businesses. Click here for more on DTEK's war effort

Update 3/17/2022

The Internet in the port city of Mariupol has been down since March 2 and the city is reported to be without electricity, gas, and water. If there are any Starlink terminals in the city, they should have no trouble getting online. I've periodically checked connectivity there and found between two and six satellites in service. (The rare time it got down to two satellites, the connection was through the Lithuanian ground station).

Six satellites in service over Mariupol

Update 3/19/2022

From the Washington Post: "A person familiar with Starlink’s effort in Ukraine, speaking on the condition of anonymity to discuss sensitive matters, said there are more than 5,000 terminals in the country. We'd seen pictures of three truckloads before, but this is a whole new level. Enough for armed forces, infrastructure companies like DTEK, NGOs, and government leaders.

Update 3/26/2022

DTEK's power engineers have received another 170 Starlink satellite terminals from SpaceX. The terminals "will help the company's power companies to maintain stable operation of power grids more efficiently and will help emergency repair crews to restore power as soon as possible." This sounds like part of the 5,000 terminals mentioned above. 

Update 5/5/2022

It's been over two months since I first wrote about Starlink terminals in Ukraine. At that time, a few hundred Starlink terminals had been delivered. Within a month there were 5,000 terminals and that was soon updated to 10,000. Now it's being reported that there are 150,000 daily users and the Starlink app has been downloaded 215,000 times

As predicted, Starlink has become an important government asset. This article shows some of the ways Starlink terminals are being used. The article is in Ukrainian, but the Google and Microsoft Translations are perfectly readable.



Update 10/13/20

I've previously written about the unprecedented role of the Internet in the war and the contribution of Starlink. Some current measures of the magnitude of the contribution of Starlink are:







Tuesday, March 01, 2022

SpaceX Starlink Service in Ukraine Is an Important Government Asset

The terminals will be used by key people who are running the government and resistance.

At 4:04 AM on February 26 Mykhailo Fedorov, Vice Prime Minister and Minister of Digital Transformation of Ukraine requested Starlink service from Elon Musk and at 2:45 PM on the 26th, Elon Musk tweeted "Starlink service is now active in Ukraine. More terminals en route." On February 28 at 12:29 PM Fedorov posted a photo of a truck load of terminals. (Kyiv is 10 hours ahead of California). 

I don't know where the terminals had been stored or how they shipped them, but the delivery time was impressive and twenty-four minutes after the truck arrival was posted, Oleg Kutkov, an engineer, tweeted an OOKLA Speedtest result

I don't know how many terminals are in Ukraine now -- Elon Musk has promised to send more -- or who will get them, but I assume it is a small number -- perhaps 100. They will be used by government and army officials who run the government and coordinate the resistance and journalists and communication specialists who report to Ukraine and the world. 

Until Starlink satellites are equipped with inter-satellite laser links, terminals will require access to ground stations. I checked for connectivity in Kyiv at two separate times using Mike Puchol's Starlink tracking Web service and found ground stations in Turkey, Lithuania, and Poland. 
Kyiv connects thru satellites (blue)
to ground stations (orange).
The first time I checked, there were nine satellites that could reach all three gateways (as shown here), and the second time five could reach two gateways -- the Turkish gateway was out of reach.

Does it matter?

In 2009 and 2011 the United States attempted to smuggle satellite Internet terminals into Cuba. The best-known effort was by Alan Gross, who spent five years in a Cuban prison when he was caught and the second was an attempt to bring equipment in by a faux surfing film production company with dishes disguised as boogie boards. (If you are curious, I covered both attempts in-depth on my blog on the Cuban Internet).

These attempts failed, but if they had succeeded, they would have been drops in the bucket -- of little importance to either the US or Cuba.

But Starlink in Ukraine is a very different case because there was no organized opposition in Cuba. These terminals will be used by key people who are running the government and resistance. They will be able to communicate synchronously or asynchronously from wherever they are in Ukraine with each other and the outside world. 

Concerns have been raised about the possibility of Russians finding and destroying these terminals using aerial direction finding, but that would be difficult. For one thing, Ukraine is a large country -- over 233,000 square miles and the terminals could be switched off when not in use. Furthermore, SpaceX is currently testing roaming in California and Nevada, The Starlink terminals are small and easily moved and SpaceX should enable roaming in Ukraine. As an added precaution, terminals could be set up a short distance from the people using them. 

This is more than a publicity stunt by SpaceX. 

Update 3/2/2020

I've been sporadically checking the state of connectivity in Kyiv today and have observed between five and nine links to ground stations. I've only seen as much as 2 seconds outage once and quality has been always 98 or 99%.



Update 3/3/2202

SpaceX has enabled fixed and mobile roaming in Ukraine and reduced peak power consumption. 













Update 5/9/2202

Survey article on SpaceX Starlink in Ukraine:
  • HOW STARLINK WORKS
  • WHO BROUGHT STARLINK TO UKRAINE?
  • WHO ALREADY USES STARLINK IN UKRAINE?
  • IS EVERYONE ALLOWED TO USE STARLINK IN UKRAINE?
  • HOW TO ORDER A TERMINAL IN UKRAINE AND HOW MUCH IT COSTS
  • HOW WELL STARLINK WORKS IN UKRAINE
  • HOW TO CONNECT TO STARLINK
  • DOES IT MAKE SENSE TO USE STARLINK AT ALL?
  • WHAT WILL HAPPEN TO STARLINK IN UKRAINE AFTER THE WAR?
The article is in Ukrainian, but the Google and Microsoft Translations are perfectly readable.

Update 5/12/2022

By March 26, 2022, 590 SpaceX Starlink satellite terminals had been transferred to Ukrainian medical and healthcare institutions according to Health Minister Viktor Lyashko.

Update 5/25/2022

Mykhailo Fedorov, Vice Prime Minister and Minister of Digital Transformation of Ukraine posted a short video saying 150,000 daily users were using over 10,000 Starlink terminals in support of key infrastructure, healthcare, financial institutions, and energy plants as well as providing mobile backhaul in several towns. Fedorov thanks Elon Musk. Pretty Good Publicity.



Update 6/9/2022

A recent Politico story on Starlink in Ukraine showed how quickly SpaceX was able to get their first terminals into and online in Ukraine. The request for terminals and subsequent spectrum access permission took place over twitter -- no dealing with regulatory bureaucracy.
USAID bought the first 1,300 Starlink terminals and SpaceX donated 3,600. The service is free.
The invasion occurred on a Thursday. On Friday Elon Musk said he wanted to get Starlink up over Ukraine. By Sunday Starlink was active and on Monday the first five hundred terminals were delivered. By Wednesday, 475 terminals were online.