5 Chapter 8 – Facial Recognition
by Steve Covello - Granite State College (USNH); Concord; and NH
“Before integrating any new technologies into American life, we must be absolutely sure that those innovations are imbued with our values.” – U.S. Senator Edward Markey, 2018.
Overview
Do any of today’s most common technologies make you feel as though your privacy or liberty are threatened?
For example, it commonplace today to see people taking photos with their smartphones. It has become so ubiquitous now that an observation was made that the sound people make when watching Tiger Woods strike a golf ball has changed in the past 15+ years because of smartphones.
In 2002, they applauded after his shot. Today, everyone is taking a picture or video with a smartphone in their hands, so they shout instead of applaud.
2002 🆚 2018
How watching Tiger has changed post-smartphone. pic.twitter.com/CGZd0kQpBM
— Jamie Kennedy (@jamierkennedy) August 8, 2018
But portable photographic technology was not always perceived as an accepted presence in society. When the first Kodak portable cameras were sold to the public around 1888, a cry of outrage ensued about the breach of privacy that was being inflicted by “all those Kodakers” taking people’s pictures in public. Over time, however, society became accustomed to it while social norms adjusted to define what is considered acceptable behavior for individuals using camera technology.
The lesson here is that people tend to see certain new technologies, at first encounter, as a threat, but then relax about it once they get used to it.
Now let’s look at another case of an emerging technology that is coming to the forefront of mainstream attention:
Enter: facial recognition.
Several companies (see below) have developed facial recognition software systems that can be licensed to developers for use in law enforcement, customs enforcement, property management/landlords, and passenger checks in airports by the Transportation Safety Administration (TSA).
However, strong objections have been levied by the ACLU, congressional leaders, privacy advocates, and others who feel that facial recognition is not only an invasion of privacy, but a potential weapon to be used to advance discrimination.
Despite these concerns, the presence of facial recognition technology has not unsettled Facebook users who upload 350,000,000 images a day – all processed in Facebook’s DeepFace facial recognition system to conveniently identify their friends in each image. Nor is there much concern from iPhone users about Apple’s FaceID feature for biometric authentication. And then there is Snapchat’s Face Lenses accessory for animating facial images which can acquire several dimensions of a face within a second.
In this chapter, we will examine the affordances of facial recognition systems, where they are used, and then explore some of the ethical issues surrounding their use.
Key Terms
Physiognomy – A pseudoscientific study of a person’s facial features or expression as indicative of character or ethnic origin; the supposed art of judging character from facial characteristics. Here is more on the history of physiognomy dating back several centuries before classical Greek culture. Findings of these efforts have often been associated with proponents of eugenics theories (Valla, J. M., Ceci, S. J., & Williams, W. M., 2011).
Required How facial recognition it works
Below (fig. 1) is a graphical representation of a basic facial recognition system. Each phase of the process is defined below.
Localization: The process of identifying the location of key facial features in an image, or in simpler terms, finding where the face is.
Registration: The affixing of points on the localized face that correspond with key registration points on a standardized matrix (see Fig. 2).
Feature extraction: Once the registration points have been established, a set of features can be extracted from the matrix that corresponds to a model that has been “taught” to the computer through the input of examples. The section of the graphic referring to “other modalities” means that it is possible to combine facial data with data sets from other forms of acquisition, such as audio, posture, gait, or other physiological data.
Classification: A facial recognition system can be trained to classify input data into prescribes sets that correspond to patterns. For example, a system can classify input into a set associated with stress and then flag an image if it matches the preset “stress” pattern.
Regression: A statistical process that attempts to produce a “best fit” trend to a set of data. In this case, it is used to provide a match to a database within a statistical degree of certainty, such as “we are 90% certain that the input image matches against the existing profile identified as [ person “X” ].”
Below is a link to a slideshow with a simplified visual representation of the facial recognition process. It is produced by the Electronic Frontier Foundation (EFF), a non-profit organization that advocates for privacy and legal issues related to emerging technologies.
“Facial Recognition What is it and how does it work?” slideshow.
What should you be focusing on?
Your objectives in this module are:
- Identify the current facial recognition systems in use today.
- Identify the various contexts of deployment, by whom, for what purpose, and under what moral/ethical presumptions.
- Explain how facial recognition might or might not be a valuable aspect of your project idea.
Readings & Media
Thematic narrative in this chapter
In the following readings and media, the authors will present the following themes:
- The history of facial recognition and its uses span to the extremes: from enhancing one’s social media experience to catching international terrorists.
- Facial recognition quietly and discretely keeps us all safe.
- Facial recognition is genie that cannot be put back in its bottle. Once it is in the hands of agents with bad intent, it will lead to oppression and tyranny.
Required Article: A brief history of facial recognition technology
FaceFirst, a corporation that develops facial recognition software and hardware systems, offers a brief description of the key points in the historical development of facial recognition technologies: “A brief history of facial recognition technology“.
Optional: For a glimpse at a very early attempt to formulate archetypes of criminality, sickliness, and other general characteristics based on facial appearance, look at the work of Victorian era (late 1800s) statistician Francis Galton who postulated that by combining photographs of multiple individuals, a composite archetype would be revealed, such as a “pianistic face” of a piano player. He did not consider it successful, though the effort is noteworthy. Would you like to play with averaging some faces like Galton? Try it out at FaceResearch.org‘s “Face Average” interactive demo.
Required Article: List of the most common uses of facial recognition
Alexis Ali – The GlobalMe Blog – “Facing the Future: Facial Recognition Uses Today” (Oct 25, 2018). The GlobalMe Blog is produced by a commercial venture that specializes in language localization – a service that enables companies to present themselves in multiple languages. This involves analysis of massive data sets of speech patterns in a way that is similar to facial recognition.
Optional: NEC corporation published an easy-to-read whitepaper that explains common use cases for facial recognition with a few FAQ about the technology and data that is collected. This is a useful document to acquaint yourself with some of the basics, but keep in mind that it is a corporate publication – not an objective report.
Optional: A more scientific document can be accessed through the Granite State College online library entitled, “Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications“.
“A more surveillant society is a safer one.”
Required Video (5:56): How Facial Recognition Technologies are Enhancing Australia’s Public Safety
NEC’s General Manager of Smart Systems, Paul Howie describes the various benefits of facial recognition.
Required List: Companies that develop facial recognition systems
FaceReader software is used to record data of people’s facial expressions for research purposes and emotional profiling.
NtechLab software “…detects and identifies people’s faces in live video streams and video footage addressing a wide range of business tasks, such as precise people count, demographic information, people flow and client behavior.” Coming soon: tracking a person’s path and recognizing a person’s ethnicity.
Faception is software that claims to detect personality types. Here are excerpts from their website:
Faception is a [technology] for profiling people and revealing their personality based only on their facial image.
Faception can analyze faces from video streams (recorded and live), cameras, or online/offline databases, encode the faces in proprietary image descriptors and match an individual with various personality traits and types with a high level of accuracy. We develop proprietary classifiers, each describing a certain personality type or trait such as an Extrovert, a person with High IQ, Professional Poker Player or a Terrorist. Ultimately, we can score facial images on a set of classifiers and provide our clients with a better understanding of their customers, the people in front of them or in front of their cameras.
“Coding bodies leads to discrimination.”
Required Essay: Chris Gilliard’s “Friction-Free Racism” from Real Life magazine
Gilliard’s essay recounts incidences where the perception of his racial identity caused people to react as if his presence was somehow out of place. He connects this experience to the reality that facial recognition is only marginally used for helpful things like unlocking smartphones or identifying a single criminal fugitive. Rather, he states, facial recognition is deployed substantially for the purpose of assigning a person to an identity category which, in a historical context, has served systems of discrimination.
He then projects a future use of facial recognition technology into a form of optics we can wear that, like augmented reality, to help identify “others” according to some identity algorithm so that we can avoid categories of people with whom we don’t want “friction”.
What to look for as you read:
- What are the connections Gilliard makes between social constructs in a society and the technologies a society embraces?
- Gilliard says that coding biometric difference – or defining “who is what” – is a form of “biometric determinism”, which is to say that “what you are classified as will determine what you will be or where you belong”. What motivates Gilliard’s concern about this?
- Given the power we already have in SM systems to select and block people that we do not want to be engaged with, why is Gilliard more concerned that facial recognition system will allow us to avoid friction with people in the real world?
About Real Life magazine: Real Life publishes essays, arguments, and narratives about living with technology. It was founded and edited by Nathan Jurgenson.
Required Article: Letter to Jeff Bezos from Congress
BuzzFeed – “Bipartisan Lawmakers Want To Talk To Amazon About Its Facial Recognition Tech“. Davey Alba, a BuzzFeed reporter on technology issues, provides background surrounding the letter written to Amazon CEO Jeff Bezos by a bipartisan group of U.S. congressional lawmakers about the affordances and risks associated with Amazon’s Rekognition facial recognition technology. The actual letter is embedded at the bottom of the article. A PDF of the primary resource can be downloaded here.
What to look for as you read:
- What do these lawmakers perceive as the risks of using facial recognition technology, and why?
- What do they believe should be the solution for controlling the technology?
Did you see this? @amazon face surveillance technology FALSELY matched me w/ someone else’s mugshot. I’m outraged & worried by the impact this tool will have on #CommunitiesOfColor when put in the hands of law enforcement! @JeffBezos: We need to talk ASAP. https://t.co/xFOy8duef1
— Rep. Jimmy Gomez (@RepJimmyGomez) July 26, 2018
Face-recognition software determined that the Mona Lisa is 83% happy, 9% disgusted, 6% fearful, and 2% angry. pic.twitter.com/07j1MznTq0
— Weird History (@weird_hist) December 28, 2018
Optional: Supplemental resources related to facial recognition
TechCrunch: “3D-printed heads let hackers – and cops – unlock your phone“.
Crain’s New York Business: “Facial-recognition startup pushes ahead despite controversy“. This is the story of an inventor who created an FR system that can alert landlords when there is suspicious activity in an apartment building and surrounding area. While this system has caused a great deal of stress about privacy issues and when/how to notify residents when it is being used, it has been embraced wholeheartedly by operators of nursing homes that have wandering patients with dementia. “It has definitely improved patient care,” they say.
Also worth noting:
https://twitter.com/graemedmoffat/status/1083531400190423040
Optional:
References
Valla, J. M., Ceci, S. J., & Williams, W. M. (2011). The accuracy of inferences about criminality based on facial appearance. Journal of Social, Evolutionary, and Cultural Psychology, 5(1), 66.Corneanu, C., Oliu, M., Cohn, J. F., & Escalera, S. (2016). Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=edsarx&AN=edsarx.1606.03237&site=eds-live
Xiong, X., & De la Torre, F. (2013). Supervised descent method and its applications to face alignment. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 532-539).
