This R&D subsidy will fund research into video feature detection and retrieval to better detect and identify deepfakes. The project has been partly financed by the RVO with an MIT grant.
Deepfakes
Visual threats are increasingly present in modern life and include all forms of manipulated images, videos and sophisticated tools, of which deepfakes are currently most common and pose the biggest threat.
Deeptrace B.V. – known as Sensity - is a world leader in the research and development of technology to detect such visual threats. They intercept visual threats on a daily basis and monitor when new malicious content is released online. To determine whether a video or image file has been altered, Sensity employs its state-of-the-art algorithms to detect even the tiniest malicious alteration, storing this information in its internal database. Due to the rapidly advancing field of visual threats production and the increasing ease of using these tools, companies worldwide are rushing to develop technology that is better at searching, detecting and preventing visual threats present online.
Advanced video embedding technology
Video, by its very nature, contains a lot of redundant information. Sensity currently employs a range of neural networks to detect certain ‘semantics’, i.e. information that indicates the nature of the video. This can be used to detect some visual threats. To future-proof the technology, Sensity wants to further develop advanced video embedding and retrieval technology. This will enable learning any relevant feature of a video, whether it is an object, a scene, an action or a series of events. Using this information enables the detection of certain flaws in the video so that it can be labelled a visual threat.
Identifying the source of the visual threat
Being able to retrieve (query) videos in any database based on a semantic or similarity search opens the door for a huge range of applications. In case of visual threats, it would allow for querying of a certain video and quickly identify any video that looks very alike (which might be a deepfake) and would help classifying the applied visual threat production technology. This will, for the first time, make it possible to gain more insight in the source of the visual threat, actually combating the problem and defending people and companies from this ever growing threat.
BrainCreators’ expertise lies in analysing any type of static data using its state-of-the-art data augmentation pipeline. This will facilitate the development of an advanced data and training pipeline or retrieval algorithms to realize video embedding and retrieval. These features lay the technological groundwork for developing a more effective and streamlined embedding, learning and retrieval approach.
The first visual threat detection platform
Successful completion of this project will result in Sensity being able to commercialize the first visual threat detection platform which is built on the latest video embedding technology and will be the only platform to support video retrieval.
