Speaking of surveillance -- face recognition

NSA email and phone surveillance has been big news the last couple of weeks, but they caught the Boston bombers with photos and we have no right of privacy in public.

This hi-res photo of Vancouver Canucks hockey fans is a composite with a total of 2.11 gigapixels.

Zooming in, we can clearly recognize people under the Homer Street sign near the back of the crowd.

Here is another example, a picture of President Obama's inauguration:

Zooming in, we see crowd sourcing the identification of the people -- over 1,500 people in the audience have been identified on Facebook:

These photos were made several years ago. A few turns of the Moore's Law crank and what will we (They) be able to do?

It turns out that facial recognition software has a hard time with sun glasses and hats.  Do you want to stay anonymous in a crowd? Here is one possible solution:

-----

Update 7/8.2013

The examples given above are based on facial recognition in public, but the new Microsoft Xbox One will bring a Kinect 2, with an always on camera, into your home and Rolling Stone wonders about the privacy implications. Rolling Stone notes that Microsoft has filed for a patent that would use the Kinect camera to monitor the number of viewers in a room. That sounds good in a pitch to an advertiser, but how about recognizing the people in the room? Who needs Big Brother?

-----

Update 8/21/2013

Here is a 360 degree spherical view of the people at a Steelers football game -- zooming in one can recognize individual faces:

The controls in the image viewer allow one to pan, tilt and zoom and the bubbles are associated with people whose faces have been recognized by crowd sourcing
-----

Update 8/24/2013

The New York Times reports (http://nyti.ms/179TFpy) on a Department of Homeland Security research project on surveillance in large crowds, the Biometric Optimal Surveillance System (BOSS). They say the technology is not ready for prime time, but they are making gains:

The federal government is making progress on developing a surveillance system that would pair computers with video cameras to scan crowds and automatically identify people by their faces, according to newly disclosed documents and interviews with researchers working on the project....

-----

Update 8/25/2013

This evening, CBS Sixty Minutes broadcasted a segment on face recognition.



The video and a transcript are online and the segment tag line, "say goodbye to anonymity," reveals the editorial slant.

Think of all the people you have identified and who have identified you in Facebook or Google Plus photographs. They are a database against which to run a facial recognition algorithm and once they have recognized a face, they can match it to the profile of interests, demographic data, likes and dislikes of that person.

Sixty Minutes emphasized commercial applications -- receiving a discount coupon for your favorite beverage seconds after you walk into a restaurant or seeing personalized ads at the mall.

They also mentioned the database of photographs of criminals the FBI is amassing. By next year they expect to be able to search by face as well as fingerprint and DNA.

The Web site includes a video segment titled "Facebook and the FBI," which was not broadcast.



The FBI representative says all the right things, but these days government credibility is not all that high. As Bruce Schneier points out in this Atlantic Monthly article, the government is relying on industry to do its surveillance.

Does any of this worry you? How long until Google or Facebook start getting warrants to turn over the names of the people walking through your hometown mall?
-----

Update 8/29/2013

Facebook may add profile photos to their facial recognition database. They are only considering it and would allow users to opt out if they implement it. (I imagine they will be clever enough to ignore cartoons or non-face photographs).
-----

Update 11/7/2013

Investigators sifted through surveillance video after the Boston Marathon bombing. They found pictures of the suspected bombers, but were unable to identify them given these images. With higher resolution cameras and improved software, would it have been possible to identify them using an Internet image search tool like Google Image Search? Would it be possible to automate the entire sifting and identification process?


-----

Update 11/7/2013

The New York Police Department deployed helicopters, police boats, scuba divers, bomb-sniffing dogs and hundreds of cameras, as well as hundreds of officers, at the November 3 New York City Marathon.

_____
Update 10/13/2014

Researchers at the Chinese Academy of Sciences Chongqing Institute have developed a facial recognition system which they say "can reach 99.5% accuracy." (I'm not sure how to interpret that statement). It is being used at border crossings.

The Los Angeles Unified School District buys iPad Trojan Horses for 30,000 students

The Los Angeles Unified School District (LAUSD) will spend $30 million over the next two years on iPads for 30,000 students.

The measure passed unanimously in spite of opponents arguing that the money would be better spent on staff or to pilot both Windows and Apple systems and saying facility bond money should not be used for relatively short-lived teaching tools.

But, IMHO, they missed the key point -- the machines are Trojan Horses for textbook giant Pearson.

The LAUSD received 13 proposals and the top three called for different hardware -- Apple, Dell and HP -- but all three included an app to deliver the Pearson Common Core System of Courses along with other third-party educational apps.

This comes at a good time for Pearson. The Common Core curriculum is not yet established, but many states are committed to it, starting next year. The new tablets and the new commitment to the Common Core curriculum will arrive around the same time, and busy faculty (and those hired to train them) will adopt the Pearson material.

I've seen nothing about the financial arrangement between Pearson and LAUSD.  I wonder what part, if any, of the $678 the schools are paying for the tablets will go to Pearson and what revenue, if any, Pearson will receive from adoption of their curriculum.

I do not mean to criticize Pearson's offerings -- I've not even seen them. (One must have a Pearson School account to see samples). This bundling of Pearson's material will give them a significant advantage, nipping competition in the bud.

Consider, for example, the Common Core math curriculum of the Khan Academy. Will a teacher or school be as likely to select that as the Pearson curriculum? I suspect not, and once they commit to one instruction/assessment platform, they will be locked in.

This situation reminds me of the introduction of the IBM PC in 1981. IBM's entering the PC market was a watershed event, legitimatizing the personal computer as a business and professional tool. IBM offered three operating systems, Microsoft PC-DOS, Digital Research CP/M and the UCSD P-System. Prior to the introduction of the PC, CP/M was the dominant operating system for business and professional applications, but it died almost over night because, although you could get it on an IBM PC, it cost more. Digital Research faded away and Microsoft became ... Microsoft.

After twenty years of serving up "pdf textbooks," we are starting to see innovation in Internet-based teaching material and systems -- will the new market be dominated by the same big-three textbook companies?

-----

Update 9/25/2013

The LA Times reports that the iPads were locked down so students could only do school work on them. As one student said when she realized they were for school work only "You can't do nothing with them ... You just carry them around."

Well, students quickly figured out a simple hack to get around the limitation, so they could do the things teenagers do with iPads. School district officials reacted immediately -- they halted home use of the Apple tablets until further notice.

I understand that they do not want the kids tweeting or whatever during class, but they are throwing out the baby with the bathwater. Expecting students to stay within the confines of some educational program developed by Pearson or anyone else will not work. It flies in the face of student culture and expectations and ignores all of the truly educational uses for an iPad or other tablet that are available to faculty and students.

-----
Update 8/27/2014

The Los Angeles Unified School District (LAUSD) has suspended its contract for iPads and Pearson Publishing software. The rollout program has had logistical problems, but, more important, critics (including me) have pointed out that it was a bad idea to invest so heavily in hardware that would become obsolete relatively soon and act as a "trojan horse," establishing Pearson as the publisher of LAUSD courseware. However, this move seems to have been precipitated by possible irregularities in the awarding of the contract.

-----
Update 4/16/2015

I told you so ... From coverage by Los Angeles raido station KPCC:

Los Angeles Unified told Apple Inc. this week that it will not spend another dollar on the Pearson software installed on its iPads and is seeking a multimillion-dollar refund from the technology giant.

Locking the school district into current hardware and a single courseware vendor made no sense in an era of rapidly changing technology and, even more important, pedagogical innovation.


Be sure to follow this link to KPCC's coverage.

The importance of government research

Blumeberg columnist Mark Buchanan has written an article with the catchy title Who Created the IPhone, Apple or the Government?, in which he argues that government sponsored research has played a major role in the invention of the iPhone and much more.

I wrote a recent post about Presidential science initiatives, including several by President Obama, and have written articles on the role of government sponsored research in developing the personal computer and the Internet:

• Before the Altair: the history of personal computing, CACM 1993
• Seeding networks: the federal role, CACM 1996

I also compiled a bibliography of early contributions in:

• The precursors of personal computing, ACM SIGSMALL/PC Notes, 1988

To retrieve these articles click here and scroll down to or search for the titles.

-----

Update 7/1/2013

In a provocative blog post, Lee Vinsel asks what new technologies might come out of the NSA surveillance activity.  Will we get innovative new technologies or are they just using established techniques of big data mining and server farms? Also -- if there are new technologies, will they belong to the US people or to contractors like Booz Hamilton, which hired Edward Snowden?

This leads me to wonder whether the increasing reliance of government at all levels on contractors will diminish socially shared innovation.

University consortium will innovate in on-line education

Provosts from a consortium of 13 research universities (mostly in the Big Ten) have issued a position paper calling for shared effort to take advantage of "new technologies and course redesign" to "improve instructional quality, enhance student learning outcomes, and extend the reach of campus instructional offerings."

They do not say so explicitly, but they seem to be saying they can innovate themselves rather than rely on the MOOC industry, and they propose collaborting on their online offerings.

As we have noted, there are now two major open-source MOOC platforms, which could be used by these or other universities to develop and offer their own material for offering as MOOCs or to supplement other teaching material, and we hope to see them offered as hosted services in the future.

Hopefully other consortia and university systems like mine (California State University) are thinking along similar lines. Let a thousand flowers bloom.

Teaching with Google Glass

An article in the Chronicle of Higher Education quotes faculty members speculating on ways to use Google glass in teaching, for example:

• Watch a teacher do something through his or her own eyes
• Stream the conductor's view of an orchestra to an audience during a concert
• See first-person point of view of a subject in a documentary film
• See lecture notes without standing at the podium

Here is a promotional video of a teacher's first-person field trip to CERN that was streamed back to a physics class:

Peer-peer limitations for Skype (and others)

Matthew Kaufman, Principal Architect for Skype, explains the reasons for their transition from a relatively secure peer-peer architecture to a less secure client-server architecture. His comments take place in the context of a discussion of the NSA data gathering revelations on a popular discussion list.

Kaufman says the difficult, costly transition was driven by two things. The first was bugs in the most common peer operating system, Windows.

Second was the rise of phones, tablets and other portable devices. Desktop PCs are typically connected at high speed for long periods of time, but portable devices are not. When they are connected, it is over slow, expensive links and the protocol consumes a lot of power.

These considerations apply to any application, not just Skype.

My old ACM publications -- 1965-2000

Between the years 1965-2000 I wrote 51 articles for Association for Computing Machinery (ACM) publications. ACM now allows authors to publish links to their articles on their own Web sites, so you can see some of what I was doing and paying attention to in those days by clicking here.

Update 6/28/2013

Temporary access to top Wiley computer science journal articles

The Internet is disrupting the business model of academic publishers who appear to be charging a lot for articles that are written and refereed by volunteers. They do have coordination and overhead costs, but are also trying to improve the way they are perceived and operate.

Through July 31, Wiley Computer Science is granting free access to their top cited articles from 2010 to 2011. That is a small step in the right direction, but not nearly as useful as ACM's offering.

Common Core meets MOOC -- fortuitous?

Nearly every state has adopted the Common Core Standards (CC) in an effort to standardize and improve K-12 language and math education in the US. To some, that might sound like a straight jacket that limits the options for teachers and forces a cookie cutter uniformity on students with diverse skills and interests.

The critics may be right and I would have agreed with them when I was young -- won over by school reformers like Paul Goodman and working to establish "alternative schools" in Los Angeles, but, today, I am ready to give the CC a chance. As a professor at a state university, I have seen the math and writing skills of our average undergraduates -- they are very poor. For example, I have had many students who were unable to convert units of measurement, understand percents and proportions or write a coherent, on-topic paragraph.

The push for the CC coincides with the rise of MOOCs and flipped classes, which might be fortuitous. For example, the Khan Academy has a CC math curriculum. I would be surprised if using that material turned out to be less effective on the average than what we are doing today.

Teaching material is one critical ingredient, another is assessment. There are efforts at developing CC assessment material. For example, the Smarter Balance Assessment Consortium has 21 governing states and is working to develop CC assessment material for the 2014-15 school year. They have posted some sample questions, and they go beyond rote multiple choice. Consider this 11th grade math question:

The student answers by dragging parabola to the proper place on the graph. The reading and writing questions are also complex and challenging.

It is to soon to declare victory for the CC, but it is a serious effort and worth a try. We should consider using CC materials and techniques in our remedial classes, and, if the CC succeeds, it will change the role of the university -- we will offer fewer remedial classes and our students will be better prepared.

Executive order directs sharing of Federal spectrum within ten years

Citizen surveillance has been headline news for several days, but something just happened under the radar that may be more important in the long run. The President issued an executive order with a list of actions to be taken by federal agencies and offices to "accelerate shared access to spectrum."

This did not happen over night -- it was preceded by a couple years of research and politicking -- here are two key milestones:

• In July 2012, the President's Council of Advisors on Science and Technology (PCAST) issued a report calling for the use of smart radios in sharing federal spectrum.
  
• In May 2013, The President appointed Tom Wheeler, who had been an Invited Expert on the PCAST report, to head the FCC.

If you are interested in this topic, you should check the executive order and the post announcing it. We are venturing into new political and technical territory here, and The President has also authorized $100 million to fund research on spectrum sharing.

No doubt AT&T et al will do their best to thwart this effort as they were able to derail the attempt to create competition by way of the 1996 Telecommunication Act. They may win this battle too, but if they don't, we may be using some interesting technology within the next decade.

High and low end approaches to telepresence

Marvin Minsky of MIT published an article on telepresence in Omni in 1980. He credits science fiction visionaries and goes on to describe several practical applications for manipulating things at a distance. Minsky's examples involve point-to-point links between the operator and the remote device.

When the Internet came along, it became possible to make that link across the network. My first encounter with Internet-based telepresence was the remotely operated telescope at Bradford University in the UK. One visited the telescope Web site and filled in a form like the one shown here. Your request was queued, and you were informed by email when your image was ready.

(If you are not an ACM Digital Library member, you can view the above article by clicking here and searching for the article entitled "The Internet is not TV: Web publishing").

Well, the technology (and the science fiction visions) have progressed significantly since that time. Let's look at two current examples -- one high end and one low end.

The high end example is AVA 500 a robot that can wander autonomously around a building and put you in a remote teleconference. Developed by Cisco and iRobot, the AVA 500 will lease for around $2,500 per month. Check it out in this video:

At the low end, we have Romo, a $149 trainable robot that uses an iPhone as a camera/processor and can be controlled from any iOS device. Financing for Romo has been on Kickstarter.

I don't know about you, but I expect that I will have a Romo before I have an AVA 500.

-----

Update 6/20/2013

Mat Lee pointed me to a Kickstarter project that hopes to put a telescope in space. The project will be run by NASA veterans and has reached its $1 million goal with ten days to go. What a difference 18 years makes!

EdX is out and open -- let a thousand flowers bloom

EdX and Stanford merged their course delivery platforms several months ago and announced that they would open source the combined code base in June.  EdX is now out and open.

The description of the code is terse and I can only get a rough feel for the overall architecture. The edX-platform is the main repository, covering both the learning management system and the authoring tool, Studio. XBlock is a component architecture for building courseware and there are modules for machine learning, voting, nested comments, discussion and automated, staff and peer grading.

They have a modular architecture and hope to attract a broad developer community. The current release contains code from Stanford, Berkeley, the University of Queensland in Australia, 10gen (Mongo DB) and the Concord Consortium, an education technology research group.

We now have two new open source teaching platforms, Google's Course Builder and edX.  It is encouraging to note that Google's slogan is "don't be evil" and EdX, a non-profit, has lofty teaching and research goals.

Much of today's Internet-based education is repackaged material from the textbook-and-lecture era -- old wine in a new bottle.  These open source platforms promise innovation -- new wine for our new bottle, the Internet.

EdX has a "simple, flexible" configurator to deploy ready-to-go instances in the Amazon cloud.  That will be cheaper and simpler than running one's own server, but deploying an edX (or Google) server still requires technical expertise.

These early versions will evolve rapidly and I suspect that, when the code is more mature, hosted versions will be made available.  That will lower the cost and hassle barriers further -- and a thousand flowers will bloom.