The image changed to that of a burned-out Starbucks on an urban street. Then another photo from a newspaper showing a burned-out SUV beneath the headline SENATOR ASSASSINATED IN TERROR BOMBING. “We need only consider the recent unsolved terror bombings here in the United States to recognize how critical visual intelligence is to our future.”
Strickland scanned the faces of his audience. They were with him.
“How do we imbue machines with this ability? We do this by emulating the way humans process spatiotemporal events. Human visual cognition is closely attuned to change, and it’s these changes that create what we call ‘attention states.’ We acquire ‘attention states’ from video imagery through an algorithmic mechanism that includes notions of focus of attention, markers placed on salient objects, and the critical relationships between those objects in terms of motion and contact. These are necessary to distinguish individual events from one another. A series of attentional states over time then becomes a visual attention trace—or VAT—which begins to form the elements of a story. One that can be programmatically narrated through machine-readable text—text that can then be algorithmically searched for relevance, in real time, by an ‘audience’ of other, simpler programs. This is why we call our system ‘Raconteur’—because it tells the story of what’s happening in a way that common systems can understand. And like any good storyteller, ‘Raconteur’ remembers how the current scene fits into the whole.”
Strickland knew that his combination of youth and poise would be an advantage here. Disruptive technology was like that. Now, at twenty-two, he was leading a team that was about to revolutionize visual image processing. Although he wasn’t the driving force behind the innovations, he did know how to spot and recruit talent to his work teams. If history was any guide, that was the primary skill necessary for success in Silicon Valley. Being able to spot a good idea and knowing who could make it work. Removing obstacles and inspiring others, that was the biggest part of innovation.
“We have worked with DARPA’s technical staff to coordinate the following demonstration, in strict adherence to the Mind’s-Eye Project guidelines. Please remember that our system has not been previously exposed to the images that you—and it—are about to see. We look forward to taking your questions after the test. Until then, ladies and gentlemen, I give you ‘Raconteur,’ the storyteller. . . .”
More light applause as the screen went black.
Strickland stepped aside as two smaller screens glowed to life up front—one bearing the title “TCTO Phase 1—Recognition Test.” The other screen displayed a blinking cursor.
Strickland moved to the side to stand with his project team, bracing for whatever came next. He cast a tense look at his development lead, Vijay Prakash, but the handsome, dour Bengali ignored Strickland’s arched eyebrows and looked to the screen. The rest of the grad student crew—Sourav Chatterjee, Gerhard Koepple, Wang Bao-Rong, and Nikolay Kasheyev—nodded in acknowledgment of the moment. Then they all turned to watch the screens too.
The words “TCTO Phase 1—Recognition Test” soon appeared also on the right-hand screen. The twin projections were set up so that whatever appeared in the left-hand screen, Raconteur would have to make sense of and describe in text on the right-hand screen.
Strickland felt relief wash over him as he stood in the darkness. Failing simple character recognition while reading the title card would have killed them, but then, OCR was handled by a licensed library, not their code. Still, he knew the DARPA judges wouldn’t cut them any slack for choosing a bad library.
But the test was already moving on. No time to ponder disaster scenarios. The left-hand screen changed to black-and-white surveillance video. It depicted a woman walking down an office hallway carrying a cardboard records box.
Strickland tensed again. He’d seen the VI algorithms work a hundred thousand times and had a pretty good idea how they functioned, but they’d never been run live in front of such an important audience. What happened next would decide the next several years of his life—of their lives—and quite possibly the trajectory of Strickland’s career. He focused on the blinking cursor on the right-hand screen—the Raconteur output panel.
As the video continued, text began to appear. . . .
Person carries object along corridor.
Murmurs of approval swept through the room, but Strickland remained tense. C’mon. Do it. Do it, baby. . . .
The cursor then began expanding on the details.
Woman carries box along corridor.