QMSum

long context official site →

QMSum is a benchmark for query-based multi-domain meeting summarization consisting of 1,808 query-summary pairs over 232 meetings across academic, product, and committee domains. The dataset enables models to select and summarize relevant spans of meetings in response to specific queries. Published at NAACL 2021, QMSum presents significant challenges in long meeting summarization where models must identify and summarize relevant content based on user queries.

Methodology

Imported from llm-stats public benchmark metadata. Modality: text. Max score: 1. Categories: long_context, summarization. Language: en. Verified by llm-stats: no.

Leaderboard

  1. Phi-3.5-mini-instruct self-reported llm-stats
    21.3%
  2. Phi-3.5-MoE-instruct self-reported llm-stats
    19.9%