RealWorldQA

vision

RealWorldQA is a benchmark designed to evaluate basic real-world spatial understanding capabilities of multimodal models. The initial release consists of over 700 anonymized images taken from vehicles and other real-world scenarios, each accompanied by a question and easily verifiable answer. Released by xAI as part of their Grok-1.5 Vision preview to test models' ability to understand natural scenes and spatial relationships in everyday visual contexts.

Methodology

Imported from llm-stats public benchmark metadata. Modality: multimodal. Max score: 1. Categories: spatial_reasoning, vision. Language: en. Verified by llm-stats: no.

Leaderboard

  1. Qwen3.6 Plus self-reported llm-stats
    85.4%
  2. Qwen3.5-122B-A10B self-reported llm-stats
    85.1%
  3. Qwen3.5-35B-A3B self-reported llm-stats
    84.1%
  4. Qwen3.6-27B self-reported llm-stats
    84.1%
  5. Qwen3.5-27B self-reported llm-stats
    83.7%
  6. Qwen3 VL 235B A22B Thinking self-reported llm-stats
    81.3%
  7. Qwen3 VL 235B A22B Instruct self-reported llm-stats
    79.3%
  8. Qwen3 VL 32B Instruct self-reported llm-stats
    79.0%
  9. Qwen3 VL 32B Thinking self-reported llm-stats
    78.4%
  10. Qwen2-VL-72B-Instruct self-reported llm-stats
    77.8%
  11. Qwen3 VL 30B A3B Thinking self-reported llm-stats
    77.4%
  12. Qwen3 VL 30B A3B Instruct self-reported llm-stats
    73.7%
  13. Qwen3 VL 8B Thinking self-reported llm-stats
    73.5%
  14. Qwen3 VL 4B Thinking self-reported llm-stats
    73.2%
  15. Qwen3 VL 8B Instruct self-reported llm-stats
    71.5%
  16. Qwen3 VL 4B Instruct self-reported llm-stats
    70.9%
  17. Qwen2.5-Omni-7B self-reported llm-stats
    70.3%
  18. Grok-1.5V self-reported llm-stats
    68.7%
  19. DeepSeek VL2 self-reported llm-stats
    68.4%
  20. DeepSeek VL2 Small self-reported llm-stats
    65.4%