Microsoft’s new video search tech proves our sci-fi future has arrived

Image: S. Warren/AP/REX/Shutterstock

What if you could understand everything about a video without ever having to watch it?

That may sound contradictory, but Microsoft has created a tool that can analyze videos and make their contents searchable, potentially saving you the trouble of watching the footage yourself.

Called Video Indexer, the software, which debuted onstage at Microsoft’s Build developers conference, allows developers and publishers to quickly analyze huge amounts of video in surprisingly precise detail.

“The Video Indexer essentially allows a developer, or a publisher to upload a video… and then it will return a bunch of interesting insights about the video,” Microsoft’s Principal Product Manager for Cognitive Services, Irving Kwong explained to me the night before build in a preview briefing.

The Indexer can identify faces, recognize text that appears in a clip, come up with keywords, highlight text, and automatically translate speech into text captions in eight different languages. Not only that it’s actually able to pick out feelings based on what’s being said and the facial expressions of people in the video.

The Video Indexer can pick out an astonishing amount of details from video clips.

Image: microsoft

“Through the video, we use sentiment analysis to essentially look at faces and predict what the sentiment is at the time during the video,” Kwong said.

In other words: You can run a clip through the Video Indexer and it will not only identify each individual and exactly when they appear, but it will guess their emotional state at each point in the video as well. What’s more, analyzing a 45-minute video this way takes only about 4-5 minutes, according to Kwong.

Now that sounds like it’s straight out of a sci-fi flick.

Of course, Microsoft isn’t the only company working on this type of technology. Google has a similar tool that debuted earlier this year called the Cloud Video Intelligence API. Like Microsoft’s Video Indexer, it also makes the contents of videos searchable, though it’s limited to more general keywords.

It’s also important to note that Video Indexer is not intended to be a consumer application at least not not right now. Microsoft is positioning this as a service for media organizations, advertisers retailers and even government agencies and law enforcement.

This is all either incredibly cool or the start of an Orwellian nightmare, depending on where you stand. Either way, though, it’s clear our sci-fi future has finally arrived.

Read more: http://mashable.com/2017/05/10/microsoft-video-indexer/