Hearing on Algorithmic Transparency Reveals Rift in NYC Tech Community

Packed room at the Technology Committee meeting on algorithmic transparency. (Photo: Noel Hidalgo)

In a packed room at City Hall on Monday, Council Member James Vacca heard testimony from city officials, technologists, and civil rights advocates on his proposed legislation for transparency in the use of algorithms for city services. Vacca said that the legislation is, to the best of his knowledge, the first of its kind in the United States. Although most of the testimony was in favor of the proposed legislation, the hearing revealed a schism between the tech community as represented by the trade nonprofit Tech.NYC—whose dozens of members include giants like Facebook and Google, as well as smaller companies like Meetup and Civic Hall—and the technologists who testified as individuals.

The proposed legislation clearly touched a nerve, drawing such a crowd that latecomers had to stand shoulder to shoulder at the back of the room. “This is the largest attendance a technology meeting has ever had,” Vacca said, after asking everyone to turn their phones to vibrate. “How am I going to top this next month? This is great!”

More than a dozen individuals gave testimony, from the Deputy Commissioner for Enterprise and Solution Architecture at the Department of Information Technology and Communications (DoITT) to a hoodie-wearing technologist who did not prepare testimony in advance, but instead spoke off the cuff.

At the beginning of the hearing, Vacca gave several examples of opaque government decision-making processes that this legislation seeks to make more transparent, including the allocation of police forces and the school selection process.

The DoITT Deputy Commissioner, Don Sunderland, testified first. His words could be characterized as cautious opposition. While praising the intent of the legislation for its “laudable goal,” the bulk of the testimony consisted of objections on the grounds of “operational concerns.” Sunderland said that the legislation is too broad, and could be construed as applying to “every computer program in the city.” This would be a particular challenge for software provided by a vendor, which could be proprietary, he pointed out.

Sunderland also cited the impossibility of testing, and, of course, vague security concerns that sound a bit like fraud: “Those looking to cause damage could use knowledge of these algorithms to circumvent important criteria put in place to prevent abuse of these processes,” he said.

Representatives from The Bronx Defenders, the Brennan Center for Justice, the New York Civil Liberties Union, and the Brooklyn Defender Services were among those who spoke out in favor of the legislation.

The nonprofit trade group Tech:NYC criticized the bill in its current form, although, like Sunderland, their representative, Taline Sanassarian, pointed out that they believe in transparency and fairness, too.

“This particular proposal, however, is unworkable from the   perspective of many of our members who are engaged in the   local tech community,” Sanassarian testified. “Specifically, imposing disclosure requirements that will require the publishing of confidential and proprietary information on city websites could unintentionally provide an opportunity for bad actors to copy programs and systems. This would not only devalue the code itself, but could also open the door for those looking to compromise the security and safety of systems, potentially exposing underlying sensitive citizen data.”

She raised the specter of the Equifax breach as an illustration of the “dangers” that “public and private actors” face.

Finally, Sanassarian said that companies would be hesitant to do work with the city if making proprietary information public was a condition of the contract.

This concern is one that was addressed directly by some of the technologists in the room. Noel Hidalgo, the executive director of the civic technology organization BetaNYC, said in his written testimony that “Democracy requires transparency; copyright nor “trade secrets” should ever stand in the way of an equitable, accountable municipal government.”

Others addressed their differences with the trade group head on.

“Tech:NYC does not speak for me,” said Sumana Harihareswara, who was among the last to testify. “I am an entrepreneur and a programmer in New York City, who’s been in this community for more than a decade. And I’m an entrepreneur who works on open source tools that help governments make decisions. Open source and transparency are a way to better security. If there are businesses in our community that are making money off of citizen data and can’t show us the recipe for the decisions they’re making, they need to step up, and they need to get better, and we need to hold them accountable.”

In spite of these differences, it is clear that there is a lot of interest in making city government and the algorithms that run behind the scenes more transparent. Even those who spoke in opposition to the current form applauded the intent (and many who spoke in favor of the legislation pointed out that it could still be better). However, the technology community appears divided in its priorities, between equity, fairness, and accountability, and the good-for-business, good-for-innovation bottom line.