FinMMEval

Go to Lab page (external) Register for this lab

FinMMEval introduces the first multilingual and multimodal evaluation framework for financial large language models, assessing models’ abilities in understanding, reasoning, and decision-making across languages and modalities to promote robust, transparent, and globally inclusive financial AI systems.

Tasks

The FinMMEval Lab features three different tasks:

Financial Exam Question Answering
Evaluates models’ conceptual understanding and domain reasoning using multilingual, professional exam-style financial questions (e.g., CFA, CPA, EFPA, BBF). Performance is measured by accuracy.
Multilingual Financial Question Answering
Tests analytical financial reasoning using multilingual and multimodal information sources (e.g., SEC filings + cross-lingual news). Models generate concise, evidence-grounded answers evaluated with ROUGE and factuality metrics.
Financial Decision Making
Assesses reasoning-to-action capabilities by generating Buy/Hold/Sell trading decisions and short rationales based on textual and numerical market contexts (BTC, TSLA). Evaluated via profitability, stability, and risk metrics (e.g., Sharpe Ratio, Cumulative Return).

Organizers

  • Zhuohan Xie (Mohamed bin Zayed University of Artificial Intelligence, UAE)
  • Rania Elbadry (Mohamed bin Zayed University of Artificial Intelligence, UAE)
  • Fan Zhang (The University of Tokyo, Japan)
  • Georgi Georgiev (Sofia University “St. Kliment Ohridski”, Bulgaria)
  • Xueqing Peng (The Fin AI, USA)
  • Lingfei Qian (The Fin AI, USA)
  • Jimin Huang (The Fin AI, USA)
  • Dimitar Dimitrov (Sofia University “St. Kliment Ohridski”, Bulgaria)
  • Vanshikaa Jani (University of Arizona, USA)
  • Yuyang Dai (INSAIT, Bulgaria)
  • Jiahui Geng (Mohamed bin Zayed University of Artificial Intelligence, UAE)
  • Yuxia Wang (INSAIT, Bulgaria)
  • Ivan Koychev (Sofia University “St. Kliment Ohridski”, Bulgaria)
  • Veselin Stoyanov (Mohamed bin Zayed University of Artificial Intelligence, UAE)
  • Preslav Nakov (Mohamed bin Zayed University of Artificial Intelligence, UAE) "