The information integrity project explores methods to enhance the accuracy and reliability of online information within the context of digital platforms and AI technologies. By analyzing how information is created, shared, and verified on social media and other digital ecosystems, we aim to develop tools and frameworks that support the identification of misinformation, reduce its spread, and bolster public trust in verified sources. This project contributes to our broader goal of understanding and mitigating the effects of digital misinformation on political discourse and social cohesion.
Selected Publications
A Guide to Misinformation Detection Datasets
Data is a barrier to reliable misinformation detection solutions. To address this, we curated the largest collection of (mis)information datasets in the literature, totaling 75. From these, we evaluated the quality of all of the 36 datasets that consist of statements or claims. We assessed these datasets to identify those with solid foundations for empirical work and those with flaws that could result in misleading results, such as insufficient label quality, spurious correlations, or political bias. We further show that regardless of label quality, categorical labels may no longer give an accurate evaluation of detection model performance, and discuss ways to solve this. Overall, this guide aims to provide a roadmap for obtaining higher quality data and conducting more effective evaluations in this domain.
Web Retrieval Agents for Evidence-Based Misinformation Detection
We demonstrated an effective two LLM agent architecture for misinformation detection and fact-checking. It can increase the macro F1 of misinformation detection by as much as 20 percent compared to LLMs without search. We also conducted extensive analyses on the sources our system leverages and their biases, decisions in the construction of the system like the search tool and the knowledge base, the type of evidence needed and its impact on the results, and other parts of the overall process.
Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigation
LLMs struggle with hallucinations and overconfident predictions. Uncertainty quantification can improve their reliability and helpfulness. We proposed an uncertainty quantification framework that leverages both direct confidence elicitation and sampled-based consistency methods to provide better calibration for NLP misinformation mitigation solutions.
Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4
We propose focusing on generalization, uncertainty, and how to leverage recent large language models, in order to create more practical tools to evaluate information veracity in contexts where perfect classification is impossible. This was one of the first works to study post-ChatGPT models in this domain. We first demonstrated they can outperform prior methods in multiple settings and languages, exhibit differences in failure modes, can quantify uncertainty, and other aspects of their usage. We also published the LIAR-New dataset with novel paired English and French misinformation data, and Possibility labels that indicate if there is sufficient context for veracity evaluation.
The Surprising Performance of Simple Baselines for Misinformation Detection
While many sophisticated detection models have been proposed in the literature, they were often compared with older NLP baselines such as SVMs, CNNs, and LSTMs. We showed that with basic fine-tuning, BERT-type language models were competitive with and could even significantly outperform state-of-the-art methods of the time. We further studied a comprehensive set of benchmark datasets, and discuss potential data leakage and the need for careful design of the experiments and understanding of datasets to account for confounding variables.
Technology Transfer
Veracity AI
This project combines large language models (LLMs) with web search APIs to create an AI-powered fact-checking tool. The goal is to develop a publicly accessible app that empowers individuals to verify the accuracy of information they encounter online. This app goes beyond fact-checking and aims to enhance the understanding of its users by offering credible sources.
Designed to be user-friendly and transparent, the app ensures that users can see the reasoning and evidence behind its conclusions. This combination of AI-powered analysis and credible source linking aims to build trust in the tool while addressing the growing challenges of misinformation and disinformation