Engaging with the broader Machine Learning Community.

Anomaly Detection in Distribution Shifted environments

Even though recently there is an effervescence on the generalization capabilities in multi-environment setups, the problem is curently discussed only in supervised scenarios. We are interested in analysising unsupervised approaches and their perfomance under distribution shifted data. More, we started formalizing this setup for the Anomaly Detection task.

• S. Smeu, E. Burceanu, E. Haller, A. Nicolicioiu, Stylist: Style-Driven Feature Ranking for Robust Novelty Detection, under review, partially presented at ICCV Workshop on Out Of Distribution Generalization in Computer Vision, 2023
• S. Smeu, E. Burceanu, A. Nicolicioiu, E. Haller, Env-Aware Anomaly Detection: Ignore Style Changes, Stay True to Content!, in NeurIPS Workshop on Distribution Shifts: Connecting Methods and Applications, 2022
• M. Dragoi, E.Burceanu, E. Haller, A. Manolache, F. Brad, AnoShift: A Distribution Shift Benchmark for Unsupervised Anomaly Detection, in The Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS, Datasets and Benchmarks Track), 2022

DeepFake Detection

Deepfake detection plays a crucial role in addressing the social impact of synthetic media manipulation. With the rise of deepfake technology, there is a growing concern about its potential to spread misinformation and deceive the public. Detection efforts are essential to protect individuals and society from the harmful consequences of manipulated content. By using advanced machine learning techniques to uncover inconsistencies and anomalies within deepfake videos, audio recordings, or images, these detection methods act as a shield against the erosion of trust in digital media.

• D. Tantaru, E. Oneata, D. Oneata, Weakly-supervised deepfake localization in diffusion-generated images, WACV 2024 and *Best Short Paper* at ICCV 2023 Workshop on Adversarial Robustness In the Real World (AROW),

Unsupervised learning of objects from video sequences

We address an essential problem in computer vision, that of unsupervised foreground object segmentation in video, where a main object of interest in a video sequence should be automatically separated from its background. Video object segmentation and object discovery are strongly related tasks, but we tackle the problem from a fully unsupervised perspective, building object representations from raw video sequences. An efficient solution to this task would enable large-scale video interpretation at a high semantic level in the absence of the costly manual labeling.

• E.Burceanu, M. Leordeanu, Learning a Fast 3D Spectral Approach to Object Segmentation and Tracking over Space and Time, journal version under review, 2022
• E. Haller, A.M. Florea, M. Leordeanu, Iterative Knowledge Exchange Between Deep Learning and Space-Time Spectral Clustering for Unsupervised Segmentation in Videos, Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021
• E. Haller, M. Leordeanu, Spacetime Graph Optimization for Video Object Segmentation, arXiv, 2020
• E. Burceanu, M. Leordeanu, A 3D Convolutional Approach to Spectral Object Segmentation in Space and Time, In The International Joint Conference on Artificial Intelligence (IJCAI), 2020
• E. Haller, M. Leordeanu, Unsupervised Object Segmentation in Video by Efficient Selection of Highly Probable Positive Features, In The IEEE International Conference on Computer Vision (ICCV), 2017

Graph methods for video processing

Given a video we want to have a good understanding of the scene and to be able to identify the key events in order to extract useful information. Our goal is to capture the complex interactions between multiple entities in a scene. We improve the clasical convolutional models by proposing a graph model, that has a strong, explicit bias towards modeling relationships and at the same time being able to model long range interactions. We also design a novel method, devised at the object-level, that can discover salient regions and we quantitatively show that they correlate with objects locations and help the relational procesing

• I. Duță, A. Nicolicioiu, M. Leordeanu, Discovering Dynamic Salient Regions with Spatio-Temporal Graph Neural Networks, in the Conference on Neural Information Processing Systems (NeurIPS), 2021
• I. Duță, A. Nicolicioiu, M. Leordeanu, Dynamic Regions Graph Neural Networks for Spatio-Temporal Reasoning, Object Representations for Learning and Reasoning Workshop at NeurIPS, 2020
• A. Nicolicioiu, I. Duță, M. Leordeanu, Recurrent Space-time Graph Neural Networks, in the Conference on Neural Information Processing Systems (NeurIPS), 2019

Multi-task graph

Existing methods in multi-task graphs rely on expensive manual supervision. In contrast, our proposed solution, with consensus shift learning, relies only on pseudo-labels provided by expert models. In our graph, every node represents a task, and every edge learns to transform one input node into another. Once initialized, the graph learns by itself on virtually any novel target domain. An adaptive selection mechanism finds consensus among multiple paths reaching a given node and establishes the pseudo-ground truth at that node. Such pseudo-labels, given by ensemble pathways in the graph, are used during the next learning iteration when single edges distill this distributed knowledge.

• E. Haller, E.Burceanu, M. Leordeanu, Self-Supervised Learning in Multi-Task Graphs through Iterative Consensus Shift, in The British Machine Vision Conference (BMVC), 2021

Video Captioning

Video captioning is the task of describing a video in natural language. It lies at the intersection of computer vision, natural language processing and machine learning requiring both high level visual comprehension and the ability to produce meaningful sentences. Our goal is to detect objects and events in a video and be capable of understanding the interactions between them in spatial and temporal dimensions.

• I. Duță, A. Nicolicioiu, V. Bogolin, M. Leordeanu, Mining for meaning: from vision to language through multiple networks consensus, in The British Machine Vision Conference (BMVC), 2018

Unsupervised Object Tracking

Object tracking is one of the first and most fundamental problems that has been addressed in computer vision. Difficulties come come from severe changes in object appearance, presence of background clutter and occlusions that might take place in the video.

• E. Burceanu, SFTrack++: A Fast Learnable Spectral Segmentation Approach for Space-Time Consistent Tracking, NeurIPS 2020 - Pre-register workshop, ICCV 2021 Structured Representations for Video Understanding workshop, 2020
• E. Burceanu, M. Leordeanu, Learning a Robust Society of Tracking Parts using Co-occurrence Constraints, in The European Conference on Computer Vision (ECCV), at Visual Object Tracking workshop, 2018
• E. Burceanu, M. Leordeanu, Learning a Robust Society of Tracking Parts, 2017

Anomaly detection in Text

Recent deep methods for detecting anomalies in images learn better features of normality in an end-to-end self-supervised setting. These methods train a model to discriminate between different transformations applied to visual data and then use the output to compute an anomaly score. We use this approach for Anomaly Detection in text, by introducing a novel pretext task on text sequences. We learn our model end-to-end, enforcing two independent and complementary self-supervision signals, one at the token-level and one at the sequence-level.

• I. Pintilie, A. Manolache, F. Brad, Time Series Anomaly Detection using Diffusion-based Models, ICDM - International Conference on Data Mining, 2023
• A. Bejan, A. Manolache, M. Popescu, AD-NLP: A Benchmark for Anomaly Detection in Natural Language Processing, EMNLP, 2023
• A. Manolache, F. Brad, E. Burceanu, DATE: Detecting Anomalies in Text via Self-Supervision of Transformers, in The Proceedings of the North American Chapter of ACL (NAACL), ICML Uncertainty & Robustness in Deep Learning workshop, 2021

Domain Adaptation for Authorship Verification

The task of identifying the author of a text spans several decades and was tackled using linguistics, statistics, and, more recently, machine learning. We tackle it inspired by the impressive performance gains across a broad range of natural language processing tasks and by the recent availability of the PAN large-scale authorship dataset.

• A. Manolache, F. Brad, E. Burceanu, Antonio Barbalau, Marius Popescu, Radu Tudor Ionescu, Rethinking the Authorship Verification Experimental Setups, EMNLP, 2022
• A. Manolache, F. Brad, Antonio Barbalau, Radu Tudor Ionescu, Marius Popescu, VeriDark: A Large-Scale Benchmark for Authorship Verification on the Dark Web, in The Proceedings of the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS, Datasets and Benchmarks Track), 2022

Visual Question Answering

The VQA task refers to answering questions about an image. Existing methods fuse image and text representations and are able to exploit superficial correlations and produce the correct answer, but often for the wrong reason. The recently introduced Visual Commonsense Reasoning dataset facilitates cognition-level understanding on top of recognition, by requiring not only the correct answer, but also a justification for it. We improve existing methods by leveraging structured representations of images.

• F. Brad, Scene Graph Contextualization in Visual Commonsense Reasoning, in The IEEE International Conference on Computer Vision (ICCV) - Pre.Register Workshop, 2019

Natural Language Interface to Databases

Natural Language Interface to Databases (NLIDB) bridges the gap between technical and non-technical users by allowing the latter to query large amounts of structured data through the use of instructions written in natural language.

Despite long-standing research efforts, progress has been slow and widespread adoption has failed to pick up. Data-driven approaches have been hindered by the lack of large parallel corpora to train the models on, but recent datasets alleviate this problem. We seek to improve existing SEQ2SEQ models by leveraging syntax to guide the generation process and by using semi-supervised techniques to overcome the low parallel data regime.

• R. Iacob, F. Brad, E.-S. Apostol, C.-O. Truica, T. Rebedea, Neural Approaches for Natural Language Interfaces to Databases: A Survey, in The Proceedings of the 29th International Conference on Computational Linguistics (COLING), 2020
• I. Hosu, R. Iacob, F. Brad, S. Ruseti, T. Rebedea, Natural Language Interface for Databases Using a Dual-Encoder Model, in The Proceedings of the 27th International Conference on Computational Linguistics (COLING), 2018
• F. Brad, R. Iacob, I. Hosu, S. Ruseti, T. Rebedea, A Syntax-Guided Neural Model for Natural Language Interfaces to Databases, in The Proceedings of the IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI), 2018
• F. Brad, R. Iacob, I. Hosu, T. Rebedea, Dataset for a Neural Natural Language Interface for Databases (NNLIDB), in The Proceedings of the Eighth International Joint Conference on Natural Language Processing (IJCNLP, Volume 1: Long Papers), 2017

Learning Representations for Deep Reinforcement Learning

We are interested in furthering the research on learning good representations for Deep Reinforcement Learning towards the goals of more stable optimisation, improvede generalization and better sample complexity.

• F. Gogianu, T. Berariu, M. Roșca, C. Clopath, L. Bușoniu, R. Pașcanu, Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective, Proceedings of the 38th International Conference on Machine Learning (ICML), 2021

Malmo AI Challenge

The Microsoft Malmo AI Challenge proposed a time-extended stag hunt scneario build on top of the well-known Minecraft platform. In order to maximize its payoff in such a game an agent needs to predict the level of commitment to a collaborative strategy of the other player, decide on a specific plan and navigate through the environment to execute it.

Our submission ranked second for the AI Summer School placement prize and third for the Microsoft Azure for Research Grant prize.

• A. Nica, T. Berariu, F. Gogianu, A.M. Florea, Learning to Maximize Return in a Stag Hunt Collaborative Scenario through Deep Reinforcement Learning, in The International Conference on Machine Learning (ICML), at Reinforcement Learning Workshop, 2017

Hardness of lattice problems

Lattice-based cryptography is a great promise for post-quantum cryptography. It aims at harnessing the security of cryptographic primitives in the conjectured hardness of well-identified and well-studied algorithmic problems involving euclidean lattices. In order to build post-quantum cryptographic primitives based on lattices, we actually make use of some intermediate, more versatile problems, Learning With Errors (LWE) and Shortest Integer Solutions (SIS), which are provably at least as hard as classical lattice problems.

To obtain more efficient primitives, different structured variants of LWE and SIS have been introduced. We are interested in studying the hardness of all these problems and giving reductions between them.

• M. Bolboceanu, Z. Brakerski, R. Perlman, D. Sharma, Order-LWE and the Hardness of Ring-LWE with Entropic Secrets, in The Annual International Conference on the Theory and Application of Cryptology and Information Security (Asiacrypt), 2019
• M. Rosca, D. Stehlé and A. Wallet, On the Ring-LWE and Polynomial-LWE Problems, in the proceedings of International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT), pages 146-173, Springer, 2018
• M. Bolboceanu, Relating Different Polynomial-LWE problems, in 11th International Conference on Security for Information Technology and Communications (SECITC), 2018
• M. Rosca, A. Sakzad, D. Stehlé and R. Steinfeld, Middle-Product Learning with Errors, In the proceedings of International Cryptology Conference (CRYPTO), pages 283-297, Springer, 2017

Cryptographic primitives from (algebraic variants of) LWE

We build advanced cryptographic primitives whose security relies on the hardness of LWE and its algebraic variants. We are mainly interested in Homomorphic Encryption and Functional Encryption which allow you to run learning algorithms on encrypted data and perform statistical tests on the sensitive encrypted data that one could find for instance in a hospital or in a bank.

• S. Bai, D. Das, R. Hiromasa, M. Rosca, A. Sakzad, D. Stehlé, R. Steinfeld, Z. Zhang, MPSign: A Signature from Small-Secret Middle-Product Learning with Errors, accepted at International Conference on Practice and Theory of Public Key Cryptography (PKC), 2020
• S. Agrawal, B. Libert, M. Maitra, R. Titiu, Adaptive Simulation Security for Inner Product Functional Encryption, accepted at International Conference on Practice and Theory of Public Key Cryptography (PKC), 2020
• B.Libert, R. Țițiu, Multi-Client Functional Encryption for Linear Functions in the Standard Model from LWE, accepted at ASIACRYPT, 2019
• B. Libert, K. Nguyen, A. Passelègue, R. Țițiu, Simulation-Sound Proofs for LWE and Applications to KDM-CCA2 Security, accepted at ASIACRYPT, 2020
• R. Țițiu, B.Libert, D. Stehlé, Adaptively Secure Distributed PRFs from LWE, in The Theory of Cryptography Conference (TCC), invited to The Journal of Cryptology, 2018