Chapter 5
Covert Compassion
Harvard University – March, 2004
From his vantage point Brandt noticed that the auditorium was packed with more than a thousand people hurriedly filing in for the opening of the conference. The crowd having eventually made their way to their seats, a distinguished looking gentleman climbed the stairs to the podium and commenced with, “Ladies and Gentlemen, welcome to the opening session of the 29th World Congress on Applications in Computer Science. I am Dr. Weston Jackson, president of the American Society of Computing Science. Our plenary speaker needs no introduction. Dr. Brandt MacCauley is a professor in the Computer Science Department at Cal Tech. Dr. MacCauley obtained his Ph.D. from MIT, and prior to that he studied at Cambridge University.
“It would be an understatement to say that Dr. MacCauley is the wunderkind of pattern recognition on our planet today. He has published more than one hundred articles in prestigious journals. In addition, he has to date acquired four patents, and filed two additional patent disclosures dealing with pattern recognition. The title of Dr. MacCauley’s talk today is “Facial Recognition – The Last Frontier of Pattern Recognition”. Ladies and gentlemen, please welcome Dr. Brandt MacCauley.” This announcement was greeted with wild applause.
Brandt rose from his seat and, gingerly mounting the narrow staircase to the stage, he strode to the podium. The applause dying down, he commenced his speech with, “Thank you for your kind introduction, Dr. Jackson. Ladies and gentlemen, it is a pleasure to be here to speak to you on this timely and important topic. As I’m sure you are all aware, the field of pattern recognition is a rapidly expanding subject, having exploded onto the computing stage little more than two decades ago. Although the field is embryonic on a chronological timescale, it will most likely become a mature technological discipline within the next decade.
“So, where do we stand today with pattern recognition? As I’m sure you already know, the first widely utilized pattern recognition technology was the barcode, introduced in the railroad industry in the late 1960’s, and this technology essentially grew out of work in the late 1940’s by Bernard Silver at Drexel University, together with his friend Norman Woodland. Their technology, which was essentially an extension of Morse code, was awarded the first patent in this field on October 7, 1952. That technology, as well as UPC barcoding, utilizes an optical scanner to determine the widths of an array of lines that we are all familiar with, by simply observing the funny little lined box on any product that comes from our neighborhood grocery store. The first UPC barcode appeared on a pack of Wrigley’s chewing gum in 1974, and the use of barcoding is indeed ubiquitous today, spanning nearly every industry worldwide. The important point to understand here is that barcoding is limited technologically because it is one dimensional in nature; that is, the technology only recognizes a pattern in one dimension, and that dimension is perpendicular to the array of lines, whereby the optical device measures the width of each line.
“Today, pattern recognition has become one of the grand challenges in the world of computer science. An example of a pattern recognition tool is the algorithm that determines whether your e-mail is spam or useful mail. This, of course, is an example of pattern recognition of symbolic text.
“A still more complicated subset of pattern recognition is the field of image analysis, which deals with the automated recognition of images. This is a highly mathematically based field of computer science that has global implications. Imagine if you will a world where traffic flow patterns are imaged by satellites and computers make cognitive decisions from these images that redirect vehicles so as to mitigate traffic congestion. Imagine a world where a simple tape of a terrorist’s voice can be analyzed by pattern recognition software that can detect the identity of that person. Imagine a world in which an image of your fingerprint, or perhaps even more exotic, an image of your retina, can determine your identity. Imagine a world where images of patient’s organs are taken using devices based on ultrasound, X-rays, or magnetic resonance imaging, and these images are processed digitally by image analysis software that detects defects such as pre-metastasized cancer cells, providing doctors the ability to make early diagnoses and thereby save lives. Ladies and gentlemen, these are just a few of the technological breakthroughs in pattern recognition that will become a part of the world of tomorrow. And you may be surprised to know that they will all be solved within the lifetime of nearly every person in the audience today.
“Now, I may be a bit egotistical about this, but in my opinion the greatest challenge of pattern recognition for computer scientists today is facial recognition. My opinion is of course due to the fact that facial recognition is my main area of research.” At this pronouncement there were scattered twitters from the audience.
Continuing, Brandt added with an impish smile, “Now, you may be thinking to yourselves – ‘What’s the big deal with facial recognition? All you need to do is take a photo of the person, digitize it, and shuffle it into the deck, just like searching for a specific card in a deck of playing cards.’ Technically, you would be correct, except that the deck of cards has seven billion cards in it - the number of people on this planet. So it’s going to take a while for your algorithm to search through all seven billion of the cards. In fact, at the current speed of computers, it would take quite a while. My facial recognition algorithm, which incidentally is still far from capable of resolving most facial images accurately at this point in time, takes about 0.01 CPU seconds to compare two facial images on a high-end laptop. That means that it can compare about one hundred photos per second. That translates to 6,000 comparisons per minute, or 360,000 comparisons per hour. Skipping over a bit of math, that equals about sixty million comparisons per week. Thus, if you want to compare a given photo to a photo of every other person on Earth, it will take you a little under four months, given the state of technology of today’s laptop computers.
“Suppose for a moment that the person that you are attempting to identify is a suspected felon, perhaps even a potential murderer. In that case, you might not have four months to play around on your laptop searching for a psychopath who is bent on further mayhem within a much shorter time span. Unfortunately, there are essentially only two ways to speed up this search process. You need to either find a faster computer, or alternatively, you must find a way to improve the efficiency of your algorithm.
“Let’s consider the first option – improving computer speed. There are also two options within this approach. If Moore’s Law continues to work in the future, then some gains can be expected by simply waiting around a bit. Moore’s Law predicts that computer speed will double every eighteen months, and that law has now been essentially correct for just about a century. So if that law continues in effect, a year and a half from now your laptop will perform the same search in two months instead of four. In three years, it will perform the search in one month, and in less than five years the search will take two weeks. I think that we can all see where this avenue is going. It’s going to be quite some time before we get it down to an acceptable length of time from the standpoint of society’s needs for most applications. Meanwhile, the criminals continue to escape the grasp of law enforcement.
“Okay, so let’s suppose that instead of using a laptop, you use a much faster computer. For this type of problem, suppose you have access to a massively parallel computer. A reasonably available massively parallel machine today has about a thousand processers, and each of these can be utilized to run the algorithm simultaneously. This sounds promising, doesn’t it? The math tells us that this will decrease your run time from four months to a little under twenty hours. Meanwhile, your felon has in most cases nonetheless managed to slip through your fingers. Still, it sounds much more doable, until you realize how much a massively parallel machine costs today, which is in excess of twenty million dollars. In fact, they are so expensive that law enforcement agencies cannot aff
ord to dedicate them solely for the purpose of running facial recognition algorithms. What that means is that even if you can find a massively parallel machine to use, your job will be placed in a queue that will likely take at least twelve hours before it even begins to run the algorithm. Thus, the speedy computer approach is still many years away, and that’s assuming that we have a robust facial recognition algorithm, which we most certainly do not have at this point in time.
“Now, let’s consider the alternative approach – improving the efficiency of the algorithm. For example, consider using some pattern recognition technology to reduce the size of the deck of cards somewhat. For example, knowing the gender of the suspect would reduce the iterative process by a factor of two. Suddenly, your laptop will run through the deck in two months instead of four. Knowing a priori that the person is of Asian descent will further reduce the process by a factor of three, down to about twenty days. So now you’re starting to make significant progress, and these improvements are all related to the field of pattern recognition. This approach would therefore appear to be more promising than finding a faster computer.
“But hold on a minute, my opening premise was that you had a photo, and that you were going to slide it within the deck and try to match it up to an identical copy of the same photo. My friends, the world is not that simple. If you were looking for Osama Bin Laden, do you think that he would voluntarily give you a copy of his most recognizable photo? Certainly not! And therein lies the ultimate challenge of facial recognition: when searching for a match to a given photo, the deck of photos that is being searched is a deck of photos that does not contain the selfsame photo!
“The problem now becomes infinitely more complex. First of all, how many photos are extant for each of you in the audience? Any takers?” Brandt scanned the audience for volunteers. Seeing none, he continued with, “Folks, statisticians tell us that the average person on Earth has been photographed at least a thousand times. Most of you in this country have been photographed more than five thousand times in your lifetime. Now think about that for a moment. That means that the size of the deck of cards has just increased from seven billion to seven trillion cards. That means that in order for the algorithm to go through all of the available photos, the required computer time will increase from four months to four thousand months! For those of you who are not into math, that’s nearly three hundred and fifty years!
“And unfortunately, we still have not even begun to address the most difficult part of the problem. Ladies and gentlemen, the photos are all different from the one that you are attempting to match them to! The person may have aged since the last photo was taken. They may have a new hairstyle. The camera angle may be different. They may have had facial surgery. The digital resolution of the photo may be poor. The person you are searching for may have gone to the beach and acquired a sunburn just before their photo was taken, which brings us to a whole subset of this field – how to deal with the differences in wavelengths of colors that show up in the photo under different lighting conditions. Ladies and gentlemen, this is the ultimate challenge in pattern recognition – facial recognition - and that is what I have come to talk to you about today.”
Brandt then moved into the technical part of his talk, showing a dizzying array of mathematical formulas, each describing a different aspect of the challenges to be surmounted in order to solve this daunting problem. At the end of his presentation he received a standing ovation and, holding up his hands in a gesture that implied that their applause was excessive, he exclaimed, “Ladies and gentlemen, thank you for your generous, and I might add –exuberant – response, but I am afraid that your reaction is premature. While much has been done, and I count myself fortunate to be among those that have contributed to the field of pattern recognition, I feel compelled to remind you that the problem of facial recognition remains an open issue at this time. However, with continued research, I hope to be among those who contribute to the ultimate resolution of this difficult problem in the not-too-distant future. Thank you!” At this, the audience applauded yet again.
Dr. Jackson climbed back up to the stage and, approaching the podium, he said, “Please, Dr. MacCauley, don’t take your seat quite yet.” He then turned to the audience and announced, “Ladies and gentlemen, as I’m sure you are all aware, Dr. MacCauley is not only world renowned in this vital field of research, he is also a political activist and CEO of the rapidly growing worldwide organization named Restoring Patience. Please, Dr. MacCauley, could you tell us a bit about your organization?”
Smiling sheepishly, Brandt replied, “Of course, Dr. Foster, I’d be happy to say a few words about Restoring Patience,” and, turning to face the audience, he recommenced with, “Ladies and gentlemen, seven years ago a young woman named Patience was kidnapped from Lincoln, Nebraska in the middle of the night. Subsequently flown to Las Vegas, she was forced against her will to bomb the Lido Hotel. As I’m sure many of you are already aware, Patience escaped her captors that very night, and for the next four years, she found it necessary to remain in hiding and in constant fear for her life. Unfortunately, Patience perished in the North Tower of the World Trade Center on 9/11.
“Ladies and gentlemen, I was fortunate to meet Miss Walker briefly, just two days before her kidnapping. By escaping her captors immediately after the bombing, she became the very first success story for Restoring Patience. That organization has now grown to worldwide proportions. The members of Restoring Patience are committed to the principal that all women have the basic human right to be protected against exploitation. To date, this organization has helped protect the rights of more than two hundred women in the United States alone. If you are interested in this worthy cause, I invite you to visit our website. Thank you.” At this, the audience gave Brandt yet another ovation.
Dr. Foster now took the microphone, exclaiming, “Ladies and gentlemen, there you have it straight from the mouth of one the world’s greatest computer scientists, but more importantly, one of the truly great humanists of our time. Ladies and gentlemen, please join me in giving Dr. Brandt MacCauley one more measure of our appreciation for his important accomplishments!”
Los Angeles - 2005
Taking his seat on the small stage, Brandt noticed that although the set was brightly lit, the surroundings were elsewhere darkened. The two figures seated opposite one another were therefore arranged so as to make the camera angles ideal. Suddenly, the red light overhead disappearing, Brandt’s television interviewer commenced with, “Ladies and gentlemen, I am your host, William Blackburn, and you’re watching ‘The Personal Side’ on CBN. Tonight’s guest is the imminent computer scientist, Dr. Brandt MacCauley, perhaps the world’s foremost expert on pattern recognition. Dr. MacCauley, welcome to ‘The Personal Side’!”
“Thanks Bill, it’s a pleasure to be here tonight,” Brandt replied with self-assurance.
“So, Brandt - may I call you Brandt?”
Smiling graciously, Brandt responded, “Yes, of course.”
“So, you’re British, right?”
“Technically, that’s correct, Bill, but no self-respecting Scot would own up to that moniker. I’m a Scot, born in Edinburgh.”
“My mistake, Brandt. So it says here that you were educated at Cambridge, is that right?”
“Yes, that’s correct, in computer science. I followed that up with a Ph.D. at MIT. Same field, of course.”
“And for the past several years I understand you’ve been at Cal Tech.”
“That’s correct, Bill, since early 1997,” Brandt replied.
“And you have six of the most significant patents in the world in pattern recognition. Tell our viewers about that, if you will.”
“Pattern recognition is the ability to identify an object using artificial means. To put it in perspective, when we look at the face of another human being, most of us can immediately recognize that person. On the other hand, for a computer, this is not so
trivial a task. The scientific community has been attempting to sort out this problem for quite some time, and I’m happy to say that we have made great strides in recent years.”
“How so?”
“Well, you are all familiar with bar codes. That was developed in the last 1960’s, and as such it is old technology today. It was a very simple model that uses optical equipment to identify the width of line segments, from which a numeric code can be extracted that defines the object that is scanned. On the other hand, when we look at a human face, we are looking at a three dimensional image rather than a one-dimensional one such as that contained within bar codes, and that extension from one to three dimensions dramatically increases the difficulty of identifying the object.”
“I see. So what are some of the applications of pattern recognition, Dr. MacCauley?”
“Good question. Our viewers need only watch current TV shows, things like NCIS and CSI. They will see pattern recognition algorithms being applied to things like fingerprints, human organs, metastasized cancer cells, astronomical photos, manufacturing product identification, money identification such as that found in soft drink machines, and finally, perhaps the most difficult pattern recognition problem - facial recognition - as I mentioned earlier.”
At this, William Blackburn turned and, facing the camera, he announced, “Ladies and gentlemen, as I said at the beginning of our show tonight, we are talking with Dr. Brandt MacCauley, who is considered to be the world’s foremost authority on pattern recognition, and don’t ask me how it’s done, because it’s over my head, and probably yours, too!” At this, he stopped and laughed at his own weak joke, as did Brandt.
Continuing, he now added, “But that’s not what we’re here to discuss tonight folks. We’re here to talk about Dr. MacCauley’s rather unusual involvement in the rapidly growing worldwide organization called Restoring Patience.
“Dr. MacCauley, what the heck is going on here? What is a crack scientist doing leading a civic organization?”
Suppressing a chuckle, Brandt replied, “That’s a very good question, Bill. It’s not something that I just up and decided to do, nor did I volunteer to do it. I was called to it, in a manner of speaking.”
“Please enlighten, Brandt. Go on, if you will.”
“It all started on a night eight years ago. I was a young faculty member at Cal Tech, and I was invited to give a seminar at Nebraska State University. During that visit I met a young coed named Patience Walker.”
“At NSU,” Bill interrupted, not asking it as a question.
“Yes, and, I’m embarrassed to say, I was quite taken with her, so much so that I followed up by attending an International Festival that very night for the purpose of seeing her again,” Brandt replied with a sheepish grin. “Anyway, on that night she was part of an array of students who had dressed in traditional clothing from across the world. Miss Walker wore a full berka, one that displayed only her entrancing green eyes. Had I not been completely taken with her eyes, I doubt I would have noticed when, two days later, she was caught on camera in the act of bombing the Lido Hotel in Las Vegas.”
“And why on earth was she involved in such a deplorable act?” Bill queried.
“Well, the evidence all points to kidnapping. You see, Miss Walker was spirited away from Lincoln on the very night that I met her, and for reasons that are yet unknown, she was forced to participate in the bombing against her will.”
“And why is that so important?”
“Because I realized shortly thereafter that she had not only been kidnapped, but she had escaped her captors on that very night. And I somehow made it my mission to locate her. So Restoring Patience was subsequently formed by the members of the Cal Tech men’s bridge club, and today, as you well know, it has grown into a worldwide organization that is committed to overcoming exploitation of women.”
“Dr. MacCauley, that is a truly remarkable story. Let me say that I for one am most impressed with your discoveries in pattern recognition. However, your leadership of Restoring Patience is far more significant in my view. You are to be commended, sir.”
“Thank you, Bill,” Brandt replied.
“Just one more question before we cut to a commercial, Brandt. Whatever happened to Patience?”
At this Brandt announced, “After her escape Patience made her way to New York City, where she lived for four years. Unfortunately, she perished in the attack on the World Trade Center on 9/11,” and at this last pronouncement his voice cracked audibly.
“Thank you, Dr. MacCauley. And there you have it folks, the nitty-gritty on the worldwide phenomenon that is Restoring Patience. I’m William Blackburn, and this is ‘The Personal Side’ on CBN. We’ll be right back after these paid messages.”