Speaking of surveillance -- face recognition

NSA email and phone surveillance has been big news the last couple of weeks, but they caught the Boston bombers with photos and we have no right of privacy in public.

This hi-res photo of Vancouver Canucks hockey fans is a composite with a total of 2.11 gigapixels.

Zooming in, we can clearly recognize people under the Homer Street sign near the back of the crowd.

Here is another example, a picture of President Obama's inauguration:

Zooming in, we see crowd sourcing the identification of the people -- over 1,500 people in the audience have been identified on Facebook:

These photos were made several years ago. A few turns of the Moore's Law crank and what will we (They) be able to do?

It turns out that facial recognition software has a hard time with sun glasses and hats.  Do you want to stay anonymous in a crowd? Here is one possible solution:

-----

Update 7/8.2013

The examples given above are based on facial recognition in public, but the new Microsoft Xbox One will bring a Kinect 2, with an always on camera, into your home and Rolling Stone wonders about the privacy implications. Rolling Stone notes that Microsoft has filed for a patent that would use the Kinect camera to monitor the number of viewers in a room. That sounds good in a pitch to an advertiser, but how about recognizing the people in the room? Who needs Big Brother?

-----

Update 8/21/2013

Here is a 360 degree spherical view of the people at a Steelers football game -- zooming in one can recognize individual faces:

The controls in the image viewer allow one to pan, tilt and zoom and the bubbles are associated with people whose faces have been recognized by crowd sourcing
-----

Update 8/24/2013

The New York Times reports (http://nyti.ms/179TFpy) on a Department of Homeland Security research project on surveillance in large crowds, the Biometric Optimal Surveillance System (BOSS). They say the technology is not ready for prime time, but they are making gains:

The federal government is making progress on developing a surveillance system that would pair computers with video cameras to scan crowds and automatically identify people by their faces, according to newly disclosed documents and interviews with researchers working on the project....

-----

Update 8/25/2013

This evening, CBS Sixty Minutes broadcasted a segment on face recognition.



The video and a transcript are online and the segment tag line, "say goodbye to anonymity," reveals the editorial slant.

Think of all the people you have identified and who have identified you in Facebook or Google Plus photographs. They are a database against which to run a facial recognition algorithm and once they have recognized a face, they can match it to the profile of interests, demographic data, likes and dislikes of that person.

Sixty Minutes emphasized commercial applications -- receiving a discount coupon for your favorite beverage seconds after you walk into a restaurant or seeing personalized ads at the mall.

They also mentioned the database of photographs of criminals the FBI is amassing. By next year they expect to be able to search by face as well as fingerprint and DNA.

The Web site includes a video segment titled "Facebook and the FBI," which was not broadcast.



The FBI representative says all the right things, but these days government credibility is not all that high. As Bruce Schneier points out in this Atlantic Monthly article, the government is relying on industry to do its surveillance.

Does any of this worry you? How long until Google or Facebook start getting warrants to turn over the names of the people walking through your hometown mall?
-----

Update 8/29/2013

Facebook may add profile photos to their facial recognition database. They are only considering it and would allow users to opt out if they implement it. (I imagine they will be clever enough to ignore cartoons or non-face photographs).
-----

Update 11/7/2013

Investigators sifted through surveillance video after the Boston Marathon bombing. They found pictures of the suspected bombers, but were unable to identify them given these images. With higher resolution cameras and improved software, would it have been possible to identify them using an Internet image search tool like Google Image Search? Would it be possible to automate the entire sifting and identification process?


-----

Update 11/7/2013

The New York Police Department deployed helicopters, police boats, scuba divers, bomb-sniffing dogs and hundreds of cameras, as well as hundreds of officers, at the November 3 New York City Marathon.

_____
Update 10/13/2014

Researchers at the Chinese Academy of Sciences Chongqing Institute have developed a facial recognition system which they say "can reach 99.5% accuracy." (I'm not sure how to interpret that statement). It is being used at border crossings.