AISS Bytes: Florian Oswald

Large language models are increasingly used to generate data for research. Their outputs can vary across repeated requests even when the prompt and settings are unchanged, and setting the temperature to zero removes only part of that variation. The talk looks at where the variation comes from, how it affects downstream regressions, and proposes a reporting standard for articles and replication packages.

Joint with Guillaume Coqueret, Joan Llull, Christophe Pérignon, Christoph Scheuch and Lars Vilhuber.

Summary of the paper: floswald.github.io/publication/llm-randomness/