Friday, April 26, 2024

Starlink service is great on (some) cruise ships.

The Seabourn Venture has six Starlink terminals, three on each side of the top deck.

I recently used Starlink on a cruise along the coast of Northwest Africa and I'll summarize my experience below, but first let me explain why I put some in the title of this post. I posted the following request on the Reddit Cruise group: "What has been your experience of Starlink Internet service on Seabourn or other cruise lines? How was latency? Do video chats work smoothly? Games? etc." Here are some of the replies:

  • It was amazing. Streamed video on two devices at once, I was able to log in through my VPN and work remotely across the world, and WiFi calling was seamless. I had no idea cruise Internet could be that great.
  • The Internet was great, even in Antarctica.
  • It was FAST.
  • Zoom was fine. WhatsApp calls crystal clear too.
  • Latency was a bit high for something like live multiplayer gaming, but good enough for video chats or video streaming.
  • Very good mb/s. Variable latency. 
  • It's usually still on the slower side of things.
  • It's better than without Starlink, but still pretty terrible.
  • It's slow hot garbage.
  • It still Sucks.
My experience was positive. I was on the Seabourn Venture for a two-week cruise which went as far south as The Gambia then followed the coast north to the Mediterranean finally stopping in Málaga, Spain.  The ship had six Starlink antennas and two GEO satellite antennas. The GEO satellites were used for ship navigation, weather tracking, etc. The crew on the bridge said their work was not affected by Starlink.
There were two Internet packages, surfing and streaming. The surfing package is free for passengers and the streaming package is free for the crew. Passengers paid about $20 per day for the faster streaming package. As the name implies, streaming sites like YouTube and Netflix were blocked for surfing accounts.
According to the ship's Computer Systems Officer (CSO), streaming users always had Starlink access and surfing passengers used the GEO satellites unless there was excess Starlink capacity. He could not say what the criteria for excess capacity were, but their goal is to always keep the streaming customers happy. (Company policy prohibits crew members from playing games).
My Starlink experience was positive. I tested streaming in my cabin by watching four Netflix videos simultaneously on Android and Apple phones, an iPad, and a laptop. The screens were small and Netflix would have reduced video quality, but it worked smoothly. Video chats using WhatsApp and Zoom also worked well despite latency being consistently over 100 ms unloaded and considerably higher while up and downloading files. 
Latency on the ship was much higher than Elon Musk's stated goal of 20 ms -- what are the sources of that latency? 
First is the WiFi network configuration and your location within the ship. The size of the ship and the provisioned capacity are also factors. Carnival Corporation owns Seabourn and 8 other cruise lines. I was on The Venture, Carnival's smallest ship, which has six Starlink terminals and can accommodate up to 264 passengers with a crew of 120. Carnival's largest ships, the Jubilee and Celebration, each accommodate up to 6.632 passengers with 1,735 crew members. I wonder how many Starlink terminals they have. Demand also varies during the day and between port and sea days.
Since the only ground stations in Africa are in Nigeria, our traffic traversed inter-satellite links for most of the trip. Once on the ground, traffic was tunneled through the Carnival VPN to a public Internet point of presence (PoP) in an English-speaking location like London, Sydney, or Miami. 
When Starlink was first installed, traffic sometimes went through Hong Kong and users got Chinese replies. The PoP location can also affect copyright restrictions. When max.com became unavailable, the CSO explained that they had been switching back and forth between London and Miami for maintenance purposes.
This complexity explains the varied results I reported at the start of this post and no doubt Carnival and other cruise lines monitor and tune their networks constantly. If they are doing well they should advertise performance and customer satisfaction survey results.

Monday, March 11, 2024

Starlink has begun delivering promised latency cuts

In his January 12th  SpaceX update, Elon Musk said the biggest goal for Starlink from a technical standpoint is to get the mean latency below 20 ms. He expanded by saying that given the speed of light, 8 ms is the absolute minimum latency for a satellite at 550 km. He believes they can optimize terrestrial and inter-satellite links, and minimize queueing delays and dropped packets, to recude the the rest of the time to below 10 ms. He predicted that eventually"Starlink will be more responsive than ground Intenet in most cases."

A month later, we saw early results of the latency-reduction effort. On February 12, Oleg Kutkov tested Starlink's Rev 3 and Rev 4 terminals and, as shown here, he found no latency inflation as background upload and download speeds increased simultaneously when using the Rev 3 terminal. He compared the Rev 3 and 4 terminals and found that Rev 4 upload and download speeds were about 50% faster than Rev3. Average ping times were somewhat improved for Rev 4 (88 vs 93 ms), but jitter was significantly lower (9.2 vs 111.9 ms).
You should also check out Dave Taht's take on Oleg's results. He concedes that Starlink has improved dramatically, and outlines steps they could take to further reduce latency.

Last week, SpaceX released news of progress toward the 20 ms latency goal. They have worked to reduce latency throughout the Starlink system. Since the begining of the year, they have deployed and tested 193 different satellite software builds, 75 gateway software builds, 222 Starlink software builds, and 57 WiFi software builds. 

This is the latency view of the interactive map.
For a month preceding March 7, SpaceX collected data every 15 seconds from millions of Starlink routers. In analyzing the data, they defined worst-case latency is the point at which 99% of times are shorter and peak hours as 6-9 PM local time. 

In the United States, they found that median latency was reduced by more than 30%, from 48.5ms to 33ms during peak usage hours, and worst-case peak hour latency had dropped by over 60%, from over 150ms to less than 65ms. Outside the United States median latency was reduced by up to 25% and worst-case latencies by up to 35%. The map shown here is interactive and shows availability and upload and download speeds in addition to latency.

SpaceX says it has “tuned our algorithms to prefer paths with lower latency, no matter how small the difference, and to remove any and all sources of unnecessary and non-physical latency." Dave Taht and his colleagues at  LibreQoS might disagree, but latency will improve over time regardless. 
Latency will improve as SpaceX launches more satellites with more capacity and inter-satellite laser links and the launch rate will increase when Starship becomes available. Adding ground stations will also improve latency.  (Note that the only African light-colored areas in the above latency map are within reach of the only ground stations on the continent).
I hope Musk achieves the 20 ms goal for Starlink. Doing so would not only benefit Starlink customers, it would call the attention of the FCC and terrestrial Internet service providers to the importance of latency as a performance and marketing metric.

Thursday, February 22, 2024

Civilian Tech Mobilization in Ukraine

Rosie the Riveter, US World War II poster (source)
As was the case in the US during World War II, civilian volunteers are making important contributions to the Ukrainian war effort.

On February 8, 2022, the first truck load load of Starlink terminals arrived in Kyiv. A week later they were being used. By April 2022, there were 5,000 terminals in Ukraine, and 42,000 as of April 2023. (At this point, SpaceX and Ukraine have gone silent. Neither ChatGPT4, Gemini, Copilot, Perplexity, nor I could not find a current terminal count).

Whatever the number of terminals in the country, they require support. They were purchased, delivered, and set up. Users were trained and they require real-time access for troubleshooting and assistance. Broken terminals have to be repaired and some terminals have been modified. Civilian tech volunteers are doing much of this work. 

There are several Starlink support centers throughout Ukraine. For obvious reasons, they are secretive about their work, but one large one is Nebogray in Lviv. Neborgray has repaired 5,976 Starlink terminals and converted 516 for portable use mounted on vehicle roofs. In addition to the service centers, there are many individual craftsmen and small services throughout the country.

The work at Nebogry is performed by highly qualified volunteers. For example, Oleg Kutkov. is a senior engineer at Ubiquiti, and he devotes his spare time to Starlink research. He bought what may have been the first Starlink terminal in Ukraine on eBay before the war and does teardowns and research studies like this recent unboxing and review of the Version 4 Starlink terminal on his blog. Oleg is an active participant in the Starlink mailing list and the 15,700-member People's Starlink Facebook group.

The Facebook group was created by The People's Starlink project, which is involved in refurbishing, adapting, repairing, and providing technical support, as well as procuring and upgrading satellite communication terminals from SpaceX's Starlink for the needs of the Armed Forces of Ukraine and other defenders of Ukraine. 

With the help of many contributors, including Oleg, People's Starlink founder Vladimir Stepanets has written a 246-page Starlink Handbook for Military Users, which begins with a message from the author “Greetings defenders of Ukraine!”

Starlink Handbook for Military Users
The handbook is divided into eight, richly illustrated modules:

  • What is Starlink?
  • Starlink Terminals
  • Powering Starlink terminals
  • Expanding and collapsing Starlink terminals
  • Terminal management and settings
  • Safety of using Starlink terminals
  • Diagnostics and problem-solving
  • Starlink in network infrastructures

This is the second (and first public) edition of the handbook and it will continue evolving. It is currently available in Ukrainian, but Stepanets is discussing translations into several other languages and plans to publish it as a book.

Point of invincibility in Bucha, Ukraine
In response to Russian attacks on critical infrastructure, Ukraine has established thousands of Points of Invincibility, tent structures equipped with generators. The government is working to provide a Starlink terminal for each of them in addition to heat, water, lighting, and more.

The IT Army of Ukraine is an international, loosely connected organization of Ukrainian and foreign ethical hackers. They have created an online service that Ukrainian allies can use to generate denial-of-service attacks. Of course, one man's "ethical hacker" is another man's "terrorist," and Ukraine has petitioned The International Criminal Court to investigate Russian cyberattacks as war crimes. The International Committee of the Red Cross has published rules of engagement for civilian hackers involved in conflicts and the IT Army will make a best effort to follow the rules.

It was obvious from the early days of the war that two technologies -- Starlink and drones -- were going to play major roles. Model airplane hobbyists created an air reconnaissance unit within the army when fighting began in 2014 and Starlink enabled surveillance drones to relay target coordinates to artillery units. 

Today, non-technical people like Violetta Oliynyk, an artist and jeweler, are assembling drones in their spare time. She learned drone assembly by taking an online course from Prometheus, a nine-year-old education site with over 400 courses online. (The course was developed for the Victory Drones project). Social Drones UA is another volunteer drone assembly project. They vet then train and support potential assemblers with a how-to video and online support. 

Ukrainians are also assembling battery packs from batteries in discarded vapes, which is reminiscent of Americans saving and turning in excess cooking fat to be used in explosives during World War II.

Civilian volunteers and Ukrainian tech companies have pivoted to military innovation and production. Ukraine was technologically advanced before the war and has been forced to innovate and improvise. If Ukraine survives, the tech sector will thrive when peace comes. 

I've presented a few examples of civilian tech support for the Ukrainian resistance to the Russian invasion. There are many others, and if you are so inclined, the Internet makes it possible for you to contribute to them. Many project Websites have contribution links and you can also consult Reddit's list of vetted Ukrainen charities.

Update 3/8/2024

I believe Oleg is by Volodymyr Zelenskyy's ear.
Oleg Kutkov was among the "people embodying the spirit of Ukraine," chosen by Time Magazine as Person of the Year for 2022. Time wrote:
Ukraine first came back online when Elon Musk activated his low-­altitude Starlink satellite internet, as he would later do in Iran. The net was crucial to Ukrainian forces, who were issued the compact, portable Starlink antennas. But in Kyiv, self-­described “tech and space nerd” Oleg Kutkov reconstructed a Starlink dish from eBay, and after contacting SpaceX support, caught a signal. “I was the first civilian user of Starlink here in Ukraine,” says Kutkov, 34, who began a Face­book group that has grown to 8,700 people. “They read about me in the news, and they were all worrying about connectivity because the internet is really important here to get all the news, to get notifications and so on.”



 


Friday, January 19, 2024

Amazon Project Kuiper vs SpaceX Starlink

Amazon's Project Kuiper is far behind Starlink and is under time pressure, but Amazon has several things going for it.

In 2019, I wrote that Amazon would be a formidable satellite-ISP competitor. I still think so, but I didn’t expect it would be over four years until they launched the first test satellites. In the meantime, SpaceX has put over 5,000 satellites in orbit and has over two million Starlink customers.

Amazon has permission to launch 3,236 satellites. They must manufacture and launch at least half of them by July 2026 and the remainder by July 2029. Can they do it? After many delays, they have finally launched two test satellites, confirming that inter-satellite laser links (ISLLs) worked at 100 Gbps while sending traffic “in both directions from the internet over an AWS fiber-optic connection to our ground gateway station, up to our satellites, and then down to a customer terminal at our test location.” All Starlink satellites launched since September 2021 will have ISLLs, so by the time Kuiper is complete, July 2026, all or nearly all Starlink sats will have them and they will have a much larger constellation.

Amazon has not launched any production satellites and they will have to hurry to meet the 2026 and 2029 deadlines. They have signed contracts for 83 launches over a five-year period, which they say will provide capacity for “the majority” of the constellation. SpaceX was conspicuously not one of the vendors, and a shareholder lawsuit pointed out that Amazon had not considered SpaceX as a provider and nearly 45% of the overall value is for launches and engines from Blue Origin, a rocket company founded by Jeff Bezos. Subsequently, Kuiper signed a 3-launch contract with SpaceX.

(Note that Blue Origin has not yet launched their forthcoming New Glenn rocket, which was initially scheduled to fly in 2020. The New Glenn will have greater capacity than SpaceX’s current Falcon rocket but significantly less than their forthcoming Starship).

Amazon's Project Kuiper is far behind Starlink and is under time pressure, but Amazon has several things going for it:

  • In his first letter to stockholders, Jeff Bezos stressed that Amazon was an infrastructure company and that has been borne out by subsequent investments in facilities and services. Amazon will bundle Kuiper access with data storage and cloud computing services.
  • Kuiper will offer service-level agreements to non-consumer customers.
  • Amazon will be Kuiper’s largest customer. With over 1.5 million employees staffing offices, warehouses, and other facilities, Amazon will use the Kuiper constellation internally as will their fleets of delivery trucks, planes, shipping containers, and perhaps delivery drones someday.
  • Amazon is already in the space business with its satellite ground station service.
  • The US, Taiwanese, and other governments and militaries will see Amazon as a more reliable supplier of critical infrastructure than Starlink given Elon Musk’s political activism and Tesla’s dependence on China.
  • Some potential customers may not approve of Elon Musk’s political involvement.
  • Re-usability gives Starlink a large launch-cost advantage, but if Elon Musk can afford Twitter, Jeff Bezos can afford Kuiper.
  • Amazon's New Glenn rocket is designed to be reusable and eight Chinese private and state-owned entities are developing reusable rockets.
  • Kuiper will be launching state-of-the-art satellites and selling state-of-the-art terminals.
  • Satellite antennas are expensive, and Amazon has experience designing and manufacturing consumer devices like the Echo and Kindle. 
  • Amazon has announced three Kuiper antennas.
  • A user terminal is more than just an antenna and Dave Täht, Chief Science Officer at LibreQoS, has been calling attention to Starlink’s latency problem for years. It now seems Elon Musk is ready to listen. Amazon should talk with Täht.
  • Amazon is already talking with enterprises, governments, schools, hospitals, and mobile operators. They have contracts with Verizon, Vodafone, and NTT and licenses to operate in more than 15 countries, including Brazil, Canada, France, Mexico, and the US.
I could go on but you get the idea -- I think Kuiper will survive despite a rocky start and will eventually offer Starlink healthy competition.

Update Feb 9, 2024

I listed Jeff Bezos’s wealth and the expected use of the Kuiper constellation among the causes of my optimism. A report on the capital expenditures by the three major cloud service companies puts Amazon’s commitment to invest $10 billion over several years in Project Kuiper in context. In 2023 alone “Amazon’s (relative) CAPEX austerity continues, as the company spent a measly $53.7 billion, a decline of 20%" and it has invested around $380 billion since 2000.