NutzWorld SportzNutz EntertainmentNutz ComputerNutz GamezNutz TinyStart InfoTiki News

A Netflix data scientist taught an A.I. to recognize smooching scenes in movies

A senior data scientist at Netflix has taught an artificial intelligence (A.I.) algorithm to be able to recognize smooches. Amir Ziai developed the tool, which is able to watch movies and recognize scenes in which the characters lock lips, as part of his work to obtain an A.I. graduate certificate from Stanford University.

He selected kissing scenes because, he told Digital Trends, they can be tough to detect using traditional video-processing techniques. That’s because machines get easily confused and produce false positives when seeing other scenarios in which two people have their heads close to one another — such as talking scenes or ones in which characters walk in close proximity.

“Training was done using a database of Hollywood movies spanning multiple decades and genres,” Ziai said. “I annotated kissing and non-kissing segments in 100 of these movies, and used those segments to train a multimodal neural network that uses both audio and visual features from 1-second segments. The major challenge with training these models is twofold. First, I had to make sure that I’ve annotated a representative set of training examples that’ll help with generalizing to a diverse set of movies. Second, training deep-learning models on video can be very resource intensive.”

Nonetheless, he pulled off the feat, and the resulting tool turns out to be impressively accurate. The system employs a two-phase process. First, it uses a binary classifier to predict whether kissing is or isn’t taking place using features extracted from still frames and audio waves. The second component then aggregates the binary labels for “contiguous non-overlapping segments” into a set of kissing scenes. The final result achieves a validation F1 score of 0.95 on a diverse database of movies.

Ziai, it should be noted, isn’t the only person interested in getting machine intelligence to recognize kisses.

Article source:

About Michael
%d bloggers like this: