If ISPs have insufficient incentive to invest in infrastructure, who will? Google? Telcos? Government (at all levels)? Premises owners?
The issue is infrastructure investment and it is in our collective interest for that investment to be made. Comcast could invest in the infrastructure needed to insure rapid delivery of Netflix and other's traffic and pass that cost on to the paying customers at a fair rate of return on the investment. But, they make more money by refusing to upgrade their infrastructure, thereby slowing delivery of content and making their customers dissatisfied with content providers like Netflix. If there were competition in the ISP market, customers would switch to the ISP that provided the best price/performance, but since there is not competition, Comcast is able to reap monopoly profits. If they happen to have a competitor in a given location, perhaps AT&T, they together reap oligopoly profits.
What is the solution? One hope is for Google Fiber to provide meaningful investment and competition, which might work in the short run, but one has to wonder about the long run -- cities with Google Fiber would still be oligopolies.
Another hope is Verizon and other phone companies competing agressively, but it seems the cable and phone companies have reached a gentleman's non-competition agreement with phone companies focusing on mobile connectivity and cable companies on fixed connectivity.
If the ISPs will not make the necessary investments, government (at all levels) must make wholesale infrastructure investments and apply regulation as we do with roads and utilities. There is also a role for home and building owners investing in the last "100 meter" infrastructure as we do with water, gas and sewers.
-----
Update 6/2/2014
Data on Comcast performance before and after their agreement with Netflix
-----
Update 6/5/2014
Americans consumers are less satisfied with the ISP industry than any other industry, yet are satisfied with the package delivery industry. Both deliver things, but the package delivery industry has private and public competition.
-----
Update 6/7/2014
There is a long of discussion (250 comments) of this post on Slashdot.