Facebook said that it automatically blocked 1.2 million copies at the point of upload, and removed another 300,000 copies …
YouTube’s chief product officer described the company’s response in an interview with the Washington Post.
The team worked through the night, trying to identify and remove tens of thousands of videos — many repackaged or recut versions of the original footage that showed the horrific murders. As soon as the group took down one, another would appear, as quickly as one per second in the hours after the shooting [said Neal Mohan, YouTube’s chief product officer].
As its efforts faltered, the team finally took unprecedented steps — including temporarily disabling several search functions and cutting off human review features to speed the removal of videos flagged by automated systems. Many of the new clips were altered in ways that outsmarted the company’s detection systems.
Mohan said that the murders highlighted the difficulty faced in trying to stay ahead of ghouls actively working to spread copies of this type of footage.
“This was a tragedy that was almost designed for the purpose of going viral,” Mohan said. “We’ve made progress, but that doesn’t mean we don’t have a lot of work ahead of us, and this incident has shown that, especially in the case of more viral videos like this one, there’s more work to be done.”
The uploads came more rapidly and in far greater volume than during previous mass shootings, Mohan said […]
“Every time a tragedy like this happens we learn something new, and in this case it was the unprecedented volume” of videos, Mohan said. “Frankly, I would have liked to get a handle on this earlier.”
The NZ shooting video was originally live-streamed on Facebook by the killer. The social network quickly removed the footage and blocked the account, but faced the same problem of people re-uploading it.
Facebook said it removed 1.5 million videos depicting images from the shooting in the first 24 hours after it happened — with 1.2 million of those blocked by software at the moment of upload.
Given the steps some will take to defeat automatic detection – things like cropping the video, adding watermarks, surrounding the video with a static or moving image – it’s almost impossible for AI systems to block all copies. And no feasible number of human moderators can keep up with new copies are being uploaded at the rate of one per second. The solution may lie with search.
YouTube quietly built software, called a “breaking news shelf” and a “top news shelf,” that is triggered when a major news incident occurs and people are going to YouTube to find information, either by searching for it or by coming across it on the homepage. The breaking news shelf uses signals from Google News and other sources to show content from more authoritative sources, such as mainstream media organizations, sometimes bypassing the content that everyday users upload.
YouTube also disabled the ‘recent uploads’ search filter, which won’t return until some time later, said the company.