Model evaluations based on LLM judgements are often biased. After debiasing, the gains achievable from access to an LLM are limited.
Florian E. Dorner, Vivian Y. Nastl, and Moritz Hardt
arxiv preprint
Human preferences about judgments in text classification are not invariant accross demographic groups
Emilia Agis Lerner, Florian E. Dorner, Elliott Ash, and Naman Goel
Annual Meeting of the Association for Computational Linguistics 2024
Recent improvements in LLM performance that go beyond compute scaling appear to be fully explained by training on benchmark-specific data
Ricardo Dominguez-Olmedo, Florian E. Dorner, and Moritz Hardt
arxiv preprint
When building a test set for binary classification from noisy label, how many labels to collect per data point? Surprisingly under a simple budget constraint, the answer is a single label.
Florian E. Dorner, Moritz Hardt
Forty-first International Conference on Machine Learning (2024)
We show how to use peer-prediction mechanisms to prevent rational clients from adversarially manipulating updates in federated learning.
Florian E. Dorner, Nikola Konstantinov, Georgi Pashaliev, Martin Vechev
NeurIPS 2023
Language models’ answers to personality tests markedly deviate from typical human responses.
Florian E. Dorner, Tom Sühr, Samira Samadi, Augustin Kelava (Equal contribution)
Socially Responsible Language Modelling Research Workshop (at NeurIPS 2023)
We use LLMs and human to generate valid constraints for individual fairness in text classification.
Florian E. Dorner, Momchil Peychev, Nikola Konstantinov, Naman Goel, Elliott Ash, and Martin Vechev
ICLR 2023 (Top 25% Spotlight)
A survey on experts’ opinions on forecasting AI progress.
Ross Gruetzemacher, Florian E. Dorner, Niko Bernaola-Alvarez, Charlie Giattino, David Manheim
Technological Forecasting and Social Change 170, 120909 (2021)
How realistic is the prospect of pricing algorithms learning to collude?
Florian E. Dorner
arxiv preprint
How has sample efficiency in deep reinforcement learning improved over time?
Florian E. Dorner
arxiv preprint