ELOQUENT
The ELOQUENT lab for evaluation of generative language model quality and usefulness addresses high-level quality criteria for generative language models through a set of open-ended shared tasks.
Tasks
The ELOQUENT Lab features three different tasks:
Voight-Kampff
Can machine-generated text be distinguished from human-authored text?
Robustness
Will a generative language model’s output reflect cultural variety? Will it be able to provide robust responses irrespective of interaction language?
Topical PISA Quiz
Can a generative language model create a useful topical quiz from given text? Can it score responses to such quiz questions?
Organizers
- Jussi Karlgren (AMD Silo AI)
- Marie Isabel Engels (Fraunhofer IAIS)
- Maria Barrett
- Diandra Fabre
- Pavel Šindelář (Charles University)
- Ondřej Bojar (Charles University in Prague, ÚFAL)
- Lorraine Goeuriot
- Josiane Mothe
- Philippe Mulhem
- Mario Piacentini (OECD)
- Luis Francisco Vargas Madriz (OECD)
- Didier Schwab
- Georgios Stampoulidis
- Katherina Thomas (OECD)
- Markarit Vartampetian