Mistral Small 3 24B Base
Mistral Small 3 is competitive with larger models such as Llama 3.3 70B or Qwen 32B, and is an excellent open replacement for opaque proprietary models like GPT4o-mini. Mistral Small 3 is on par with Llama 3.3 70B instruct, while being more than 3x faster on the same hardware.
Benchmark results
| Benchmark | Score | Tags | Source |
|---|---|---|---|
| AGIEval | 65.8% | self-reported llm-stats | link → |
| ARC-C | 91.3% | self-reported llm-stats | link → |
| GPQA | 34.4% | self-reported llm-stats | link → |
| GSM8k | 80.7% | self-reported llm-stats | link → |
| MATH | 46.0% | self-reported llm-stats | link → |
| MBPP | 69.6% | self-reported llm-stats | link → |
| MMLU | 80.7% | self-reported llm-stats | link → |
| MMLU-Pro | 54.4% | self-reported llm-stats | link → |
| TriviaQA | 80.3% | self-reported llm-stats | link → |