What algorithms can see better than people
“Ce grand malheur, de ne pouvoir être seul.” “The great misfortune of not being able to be alone.” That is the epigraph, borrowed from the French moralist La Bruyère, of a short story Edgar Allan Poe published in 1840, and it may have something to say about the hot topic we have all been flocking to: camera surveillance in our cities.
Poe’s story follows a man recovering from an illness as he sits at the window of a London coffeehouse and watches the street. “At first my observations took an abstract and generalizing turn,” he says. “I looked at the passengers in masses, and thought of them in their aggregate relations.” He sorts them into types (clerks, gamblers, pickpockets, widows, what have you). He’s perfectly content doing so, until an old man with a strange face goes past and piques his curiosity to the point that he gets up and stalks him for a whole night and a day.
Unlike the typical Poe story, nothing too grim happens. Our cities, too, are much less macabre than they were when he was writing, and in the last few decades it’s likely that cameras have had something to do with it. Those who work in courts of law will know that jurors have been taught by our favorite detective shows to demand DNA tests and CCTV footage, even though real forensics move far more slowly than they do in fiction. Nonetheless, it’s much harder to be a serial killer than it was in the 1980s. Or to steal a painting. Or a child. There are plenty of non-safety-related benefits to cameras too. Our own lab at MIT has spent the last 20 years watching cities through lenses of one kind or another, using computer vision to measure street trees so that cities could see where to plant more, or different types.
Despite this, there is a brewing — and justified — anxiety about the eyes watching us, to the point that some people in American cities are actually tearing down their cameras.
The cameras being torn down belong mostly to a company called Flock Safety, which sells license plate readers to police departments and has more than 100,000 of them on streetlights and utility poles across the country. Since January, many a municipality across more than 20 states has canceled its contract, after officers were caught running plate searches for federal immigration agents, or after Los Angeles found the system flagged the wrong car about a third of the time. Sometimes the response has been to hide the cameras inside other objects. It’s spooky! Especially in what we consider to be the land of the free.
New York City has no Flock cameras. Well, it has no contracts with that company. But the City does, of course, have its own lensified apparatus: plate readers run by the NYPD, the Department of Transportation and the Metropolitan Transportation Authority, with the NYPD’s disclosed under a 2020 law called the POST Act.
So, naturally, we’re about to walk into a privacy-versus-safety binary. And I feel the need to advocate for the unsexy middle road.
I understand why people wish to tear the cameras down, even if a city without cameras is a bigger shift than most of us could stomach. Poe’s convalescent, remember, does two things with his eyes. He watches the crowd from the window, and then he gets up and follows one man. A camera can do either. The latter can be scary. But used in the first way, it can be quite a beneficial pursuit for a city, and New York, as it happens, revolutionized this use on an urban scale.
In the 1970s, urbanist William H. Whyte persuaded the City to let him put time-lapse cameras on rooftops above its new plazas, and then watched the film for years. He learned where people sat and for how long, whether they talked to strangers, what they did when a plaza gave them nowhere to sit. His conclusions contributed to the 1975 zoning amendments that still govern the city’s privately owned public spaces, the 590-odd plazas and arcades that developers build in exchange for extra floor space. They also went into a short, very funny book, “The Social Life of Small Urban Spaces.” Whyte’s cameras never learned a face or a name; he metaphorically stayed at the window.
A few years ago, my colleagues and I went back to his reels. With friends at Yale, Michigan State and Harvard, we trained computer vision models on Whyte’s original footage of Bryant Park, the steps of the Met, Downtown Crossing in Boston and Chestnut Street in Philadelphia, and then on footage of the same places from around 2010. The machine did in weeks what had taken Whyte and his assistants years.
We learned a lot, which I have discussed in this outlet and in others like The Guardian. People in those places now walk about 15% faster than they did in 1980. They linger 14% less. They are not lonelier on the street, at least by our numbers: the share walking alone moved from 67% to 68%, which is not a lot. In 1980, though, roughly 1 person in 20 who approached another on those streets stopped and formed a group. By 2010 it was 1 in 50.
The phone is the obvious suspect. We now arrange our meetings in advance and hold them indoors, and the plaza is somewhere we cross on the way. My colleague Ed Glaeser, an economist, has a colder theory, which is that our time has simply become more valuable and hanging around more expensive.
To be honest, our data cannot decide between these. Bryant Park in 2010 was a far safer and better-run place than the one Whyte filmed, and by every rule of urban design, it should have produced more mingling rather than less. It produced less, and we do not yet know why. However, for the first time, we have the instruments to find out. Unfortunately, these days, an architect draws a plaza, the City pays for it, the ribbon is cut, and almost nobody goes back to see whether people actually use it; Whyte did, and, because his method was expensive, almost nobody has since. Computer vision makes it cheap enough to run on every plaza in New York, every year, for less than one consultant’s report. Computer vision can give us infinite little Whytes helping us better understand our cities.
Flock’s cameras use computer vision to read a plate and to note a car’s make, color, roof rack or bumper stickers; ours use it to see a person lingering or a group forming, never to identify anyone.
My counterproposal has three parts. First, write into law the distinction Whyte kept by instinct, between data about places and data about people: the first kind the City may collect freely, the second only under conditions that strictly respect expectations of privacy. Second, clearly articulate the exceptions to the rules. The Berlin police have just begun trying this at a crime hotspot, Kottbusser Tor — with another, Warschauer Strasse, to follow — with cameras that show officers people as stick figures rather than faces. (“When someone is being beaten up, those aren’t normal movements,” Berlin’s interior senator, Iris Spranger, explained; a human decides what happens next.) Whatever identifying data the police do keep should expire in days rather than years, long enough to find a stolen car and too short to assemble anyone’s life. Every search should be logged with a reason, and the log published.
Third, the measurements the City takes of its own streets and plazas should be published routinely. New York has done this before. I remember a sign in Michael Bloomberg’s office at City Hall that read “In God we trust. Everyone else, bring data,” and under him the Taxi and Limousine Commission shared the GPS traces of the city’s 13,500 medallion cabs with researchers. Those traces led to many innovative mobility studies, including ours. Whyte published everything he saw, which is the only reason we could go back and look.
Deyan Sudjic, the former director of London’s Design Museum, wrote in the catalog of an exhibition I curated in Shenzhen in 2019 about what happens when streets and buildings acquire the ability to see, that “the possibility of anonymity is one of the most important qualities that differentiate a city from a village.” Poe’s convalescent knew this; it was the source of his happiness at the window, and he threw it away the moment he went out after one man. New York, which already has the POST Act, no Flock contract to unwind, and more of Whyte’s plazas than any other city in America, should keep the window and give up the potential stalking by making a modest change to local laws.





