MultiZebraLogic: A Multilingual Logical Reasoning Benchmark
📰 ArXiv cs.AI
arXiv:2511.03553v2 Announce Type: replace-cross Abstract: We create high-quality datasets for LLM evaluation of logical reasoning skills across nine different languages, which have been manually checked by fluent speakers. The datasets consist of so-called zebra puzzles, and we analyse different ways of tuning the difficulty of the puzzles to fit modern LLMs. This includes the size of the puzzle (number of objects and number of clues), as well as a novel addition of red herring clues containing
Full Article
Title: MultiZebraLogic: A Multilingual Logical Reasoning Benchmark
Abstract:
arXiv:2511.03553v2 Announce Type: replace-cross Abstract: We create high-quality datasets for LLM evaluation of logical reasoning skills across nine different languages, which have been manually checked by fluent speakers. The datasets consist of so-called zebra puzzles, and we analyse different ways of tuning the difficulty of the puzzles to fit modern LLMs. This includes the size of the puzzle (number of objects and number of clues), as well as a novel addition of red herring clues containing
Abstract:
arXiv:2511.03553v2 Announce Type: replace-cross Abstract: We create high-quality datasets for LLM evaluation of logical reasoning skills across nine different languages, which have been manually checked by fluent speakers. The datasets consist of so-called zebra puzzles, and we analyse different ways of tuning the difficulty of the puzzles to fit modern LLMs. This includes the size of the puzzle (number of objects and number of clues), as well as a novel addition of red herring clues containing
DeepCamp AI