Reihaneh Rabbany

Google Scholar

Shahrad Mohammadzadeh

Maîtrise recherche - McGill

Co-superviseur⋅e :

Doctorat - McGill

Maîtrise recherche - McGill

Site web

Soroush Omranpour

Maîtrise recherche - McGill

Co-superviseur⋅e :

Guillaume Rabusseau

kellin.pelrine@mila.quebec

Kellin Pelrine

Doctorat - McGill

Site web

Google Scholar

Farimah Poursafaei

Postdoctorat - McGill

Collaborateur·rice de recherche

Collaborateur·rice de recherche - McGill University

Sneheel Sarangi

Collaborateur·rice de recherche - McGill

sneheelsarangi@gmail.com

Stagiaire de recherche - McGill

Vidya Sujaya

Maîtrise recherche - McGill

Camille Thibault

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

jacob.mila.handle@tianshome.com

Collaborateur·rice de recherche - McGill

Collaborateur·rice de recherche - UdeM

Superviseur⋅e principal⋅e :

Doctorat - McGill

Jie Zan

Stagiaire de recherche - McGill

jie.zan@mail.mcgill.ca

Sveta Zhuk

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Flight-SEIR : incorporer les données de vol pour améliorer la modélisation épidémiologique et la prévention d’éclosions de maladies infectieuses

Billets de blogue

Flight-SEIR: Incorporating Flight Data to Improve Epidemiological Modelling and Disease Outbreak Prevention

3 août 2021

par

Shenyang Huang

Reihaneh Rabbany

Lire l'article

Publications

Quantifying learning-style adaptation in effectiveness of LLM teaching

Ruben Weijers

Gabrielle Fidelis de Castilho

Kellin Pelrine

This preliminary study aims to investigate whether AI, when prompted based on individual learning styles, can effectively improve comprehens… (voir plus)ion and learning experiences in educational settings. It involves tailoring LLMs baseline prompts and comparing the results of a control group receiving standard content and an experimental group receiving learning style-tailored content. Preliminary results suggest that GPT-4 can generate responses aligned with various learning styles, indicating the potential for enhanced engagement and comprehension. However, these results also reveal challenges, including the model’s tendency for sycophantic behavior and variability in responses. Our findings suggest that a more sophisticated prompt engineering approach is required for integrating AI into education (AIEd) to improve educational outcomes.

Temporal Graph Analysis with TGX

Razieh Shirzadkhani

Shenyang Huang

Elahe Kooshafar

Farimah Poursafaei

Real-world networks, with their evolving relations, are best captured as temporal graphs. However, existing software libraries are largely d… (voir plus)esigned for static graphs where the dynamic nature of temporal graphs is ignored. Bridging this gap, we introduce TGX, a Python package specially designed for analysis of temporal networks that encompasses an automated pipeline for data loading, data processing, and analysis of evolving graphs. TGX provides access to eleven built-in datasets and eight external Temporal Graph Benchmark (TGB) datasets as well as any novel datasets in the .csv format. Beyond data loading, TGX facilitates data processing functionalities such as discretization of temporal graphs and node subsampling to accelerate working with larger datasets. For comprehensive investigation, TGX offers network analysis by providing a diverse set of measures, including average node degree and the evolving number of nodes and edges per timestamp. Additionally, the package consolidates meaningful visualization plots indicating the evolution of temporal patterns, such as Temporal Edge Appearance (TEA) and Temporal Edge Trafficc (TET) plots. The TGX package is a robust tool for examining the features of temporal graphs and can be used in various areas like studying social networks, citation networks, and tracking user interactions. We plan to continuously support and update TGX based on community feedback. TGX is publicly available on: https://github.com/ComplexData-MILA/TGX.

2024-01-01

WSDM (publié)

Exhaustive Evaluation of Dynamic Link Prediction

Farimah Poursafaei

Dynamic link prediction is a crucial task in the study of evolving graphs, which serve as abstract models for various real-world application… (voir plus)s. Recent dynamic graph representation learning models have claimed near-perfect performance in this task. However, we argue that the standard evaluation strategy for dynamic link prediction overlooks the sparsity and recurrence patterns inherent in dynamic networks. Specifically, the current strategy suffers from issues such as evaluating models on a balanced set of positive and negative edges, neglecting the reassessment of frequently recurring positive edges, and lacking a comprehensive evaluation of both recurring and new edges.To address these limitations, we propose a novel evaluation strategy called EXHAUSTIVE, which takes into account all relevant negative edges and separately assesses the performance on recurring and new edges. Using our proposed evaluation strategy, we compare the performance of five state-of-the-art dynamic graph learning models on seven benchmark datasets. Compared to the previous common evaluation strategy, we observe an average drop of 62% in Average Precision for dynamic link prediction. Additionally, the ranking of the models also changes under the new evaluation setting. Furthermore, we demonstrate that while all models perform considerably worse when predicting new edges compared to recurring ones, the best performing models differ between the two scenarios. This highlights the importance of employing the proposed evaluation strategy for both the assessment and design of dynamic link prediction models. By adopting our novel evaluation strategy, researchers can obtain a more accurate understanding of model performance in dynamic link prediction, leading to improved evaluation and design of such models.

2023-12-04

2023 IEEE International Conference on Data Mining Workshops (ICDMW) (publié)

Exhaustive Evaluation of Dynamic Link Prediction

Farimah Poursafaei

2023-12-04

2023 IEEE International Conference on Data Mining Workshops (ICDMW) (publié)

SWEET - Weakly Supervised Person Name Extraction for Fighting Human Trafficking

Javin Liu

Hao Yu

Vidya Sujaya

Pratheeksha Nair

Kellin Pelrine

In this work, we propose a weak supervision pipeline SWEET: Supervise Weakly for Entity Extraction to fight Trafficking for extracting perso… (voir plus)n names from noisy escort advertisements. Our method combines the simplicity of rule-matching (through antirules, i.e., negated rules) and the generalizability of large language models fine-tuned on benchmark, domain-specific and synthetic datasets, treating them as weak labels. One of the major challenges in this domain is limited labeled data. SWEET addresses this by obtaining multiple weak labels through labeling functions and effectively aggregating them. SWEET outperforms the previous supervised SOTA method for this task by 9% F1 score on domain data and better generalizes to common benchmark datasets. Furthermore, we also release HTGEN, a synthetically generated dataset of escort advertisements (built using ChatGPT) to facilitate further research within the community.

2023-12-01

Findings of the Association for Computational Linguistics: EMNLP 2023 (publié)

openreview.net

Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4

Kellin Pelrine

Anne Imouza

Meilina Reksoprodjo

Camille Thibault

Caleb Gupta

Joel Christoph

Misinformation poses a critical societal challenge, and current approaches have yet to produce an effective solution. We propose focusing on… (voir plus) generalization, uncertainty, and how to leverage recent large language models, in order to create more practical tools to evaluate information veracity in contexts where perfect classification is impossible. We first demonstrate that GPT-4 can outperform prior methods in multiple settings and languages. Next, we explore generalization, revealing that GPT-4 and RoBERTa-large exhibit differences in failure modes. Third, we propose techniques to handle uncertainty that can detect impossible examples and strongly improve outcomes. We also discuss results on other language models, temperature, prompting, versioning, explainability, and web retrieval, each one providing practical insights and directions for future research. Finally, we publish the LIAR-New dataset with novel paired English and French misinformation data and Possibility labels that indicate if there is sufficient context for veracity evaluation. Overall, this research lays the groundwork for future tools that can drive real-world progress to combat misinformation.

2023-10-07

EMNLP/2023/Conference (accepté)

openreview.net

Temporal Graph Benchmark for Machine Learning on Temporal Graphs

Shenyang Huang

Farimah Poursafaei

Jacob Danovitch

Matthias Fey

Weihua Hu

Emanuele Rossi

Jure Leskovec

Michael M. Bronstein

Guillaume Rabusseau

We present the Temporal Graph Benchmark (TGB), a collection of challenging and diverse benchmark datasets for realistic, reproducible, and r… (voir plus)obust evaluation of machine learning models on temporal graphs. TGB datasets are of large scale, spanning years in duration, incorporate both node and edge-level prediction tasks and cover a diverse set of domains including social, trade, transaction, and transportation networks. For both tasks, we design evaluation protocols based on realistic use-cases. We extensively benchmark each dataset and find that the performance of common models can vary drastically across datasets. In addition, on dynamic node property prediction tasks, we show that simple methods often achieve superior performance compared to existing temporal graph models. We believe that these findings open up opportunities for future research on temporal graphs. Finally, TGB provides an automated machine learning pipeline for reproducible and accessible temporal graph research, including data loading, experiment setup and performance evaluation. TGB will be maintained and updated on a regular basis and welcomes community feedback. TGB datasets, data loaders, example codes, evaluation setup, and leaderboards are publicly available at https://tgb.complexdatalab.com/.

openreview.net

Party Prediction for Twitter

Kellin Pelrine

Anne Imouza

Zachary Yang

Jacob-Junqi Tian

Sacha Lévy

Gabrielle Desrosiers-Brisebois

Aarash Feizi

C'ecile Amadoro

André Blais

2023-08-25

ArXiv (prépublication)

Open, Closed, or Small Language Models for Text Classification?

Hao Yu

Zachary Yang

Kellin Pelrine

Recent advancements in large language models have demonstrated remarkable capabilities across various NLP tasks. But many questions remain, … (voir plus)including whether open-source models match closed ones, why these models excel or struggle with certain tasks, and what types of practical procedures can improve performance. We address these questions in the context of classification by evaluating three classes of models using eight datasets across three distinct tasks: named entity recognition, political party prediction, and misinformation detection. While larger LLMs often lead to improved performance, open-source models can rival their closed-source counterparts by fine-tuning. Moreover, supervised smaller models, like RoBERTa, can achieve similar or even greater performance in many datasets compared to generative LLMs. On the other hand, closed models maintain an advantage in hard tasks that demand the most generalizability. This study underscores the importance of model selection based on task requirements

2023-08-19

ArXiv (prépublication)

ToxBuster: In-game Chat Toxicity Buster with BERT

Zachary Yang

Yasmine Maricar

M. Davari

Nicolas Grenon-Godbout

Detecting toxicity in online spaces is challenging and an ever more pressing problem given the increase in social media and gaming consumpti… (voir plus)on. We introduce ToxBuster, a simple and scalable model trained on a relatively large dataset of 194k lines of game chat from Rainbow Six Siege and For Honor, carefully annotated for different kinds of toxicity. Compared to the existing state-of-the-art, ToxBuster achieves 82.95% (+7) in precision and 83.56% (+57) in recall. This improvement is obtained by leveraging past chat history and metadata. We also study the implication towards real-time and post-game moderation as well as the model transferability from one game to another.

2023-05-21

ArXiv (prépublication)

Fast and Attributed Change Detection on Dynamic Graphs with Density of States

Shenyang Huang

Jacob Danovitch

Guillaume Rabusseau

2023-05-15

ArXiv (prépublication)

Social Media as a Vector for Escort Ads:A Study on OnlyFans advertisements on Twitter

Maricarmen Arenas

Pratheeksha Nair

Golnoosh Farnadi

Online sex trafficking is on the rise and a majority of trafficking victims report being advertised online. The use of OnlyFans as a platfor… (voir plus)m for adult content is also increasing, with Twitter as its main advertising tool. Furthermore, we know that traffickers usually work within a network and control multiple victims. Consequently, we suspect that there may be networks of traffickers promoting multiple OnlyFans accounts belonging to their victims. To this end, we present the first study of OnlyFans advertisements on Twitter in the context of finding organized activities. Preliminary analysis of this space shows that most tweets related to OnlyFans contain generic text, making text-based methods less reliable. Instead, focusing on what ties the authors of these tweets together, we propose a novel method for uncovering coordinated networks of users based on their behaviour. Our method, called Multi-Level Clustering (MLC), combines two levels of clustering that considers both the network structure as well as embedded node attribute information. It focuses jointly on user connections (through mentions) and content (through shared URLs). We apply MLC to real-world data of 2 million tweets pertaining to OnlyFans and analyse the detected groups. We also evaluate our method on synthetically generated data (with injected ground truth) and show its superior performance compared to competitive baselines. Finally, we discuss examples of organized clusters as case studies and provide interesting conclusions to our study.

2023-04-30

Proceedings of the 15th ACM Web Science Conference 2023 (publié)