BigCodeBench

Multilingual: no

A benchmark that challenges LLMs to invoke multiple function calls as tools from 139 libraries and 7 domains for 1,140 fine-grained programming tasks. Evaluates code generation with diverse function calls and complex instructions, featuring two variants: Complete (code completion based on comprehensive docstrings) and Instruct (generating code from natural language instructions).

Methodology

Imported from llm-stats public benchmark metadata. Modality: text. Max score: 1. Categories: general, reasoning. Language: en. Verified by llm-stats: no.

Leaderboard

Showing 2 of 2 results

Gemini Diffusion self-reported llm-stats

45.4%
Qwen2.5-Coder 7B Instruct self-reported llm-stats

41.0%