R-SET GOLD-ONLY duplicate-full-row
superhero · dev_20251106-00000-of-00001 from https://huggingface.co/datasets/birdsql/bird_sql_dev_20251106/resolve/3c11fb193e5439b338e23677fa0aae11e8b85db9/data/dev_20251106-00000-of-00001.json (commit 3c11fb19, downloaded 2026-09-07)
Which superheroes have blue eyes with brown hair?
the hint the set supplies: blue eyes refers to eye_colour_id; brown hair refers to hair_colour_id; both colors are stored in the colour table
This question was audited without a prediction beside it, so there is nothing to compare the gold with. The probes below read the gold alone.
SELECT T1.superhero_name FROM superhero AS T1 INNER JOIN colour AS T2 ON T1.eye_colour_id = T2.id INNER JOIN colour AS T3 ON T1.hair_colour_id = T3.id WHERE T2.colour = 'Blue' AND T3.colour = 'Brown'this statement states no ordering of its own
sha256:1aa7da78d5c892782cf7f6a6bd62bfff8fb8cdb0d30fd8bf8c16ceaa23948be5
from evidence-gold.json, 22 rows
| superhero_nameTEXT |
|---|
| Atlas |
| Atlas |
| Big Man |
| Bloodaxe |
| Blue Beetle |
| Blue Beetle II |
| Captain Epic |
| Chewbacca |
| Fabian Cortez |
| Gravity |
| Hawkman |
| Hercules |
| Jack-Jack |
| Mister Knife |
| Molecule Man |
| Multiple Man |
| Rick Flag |
| Scarecrow |
| Scarlet Witch |
| Spider-Girl |
| Toxin |
| X-Man |
A smell is a mechanical reason to read this gold statement again. It is a heuristic: it does not state that the statement is wrong, and a maintainer decides.
this statement orders by a text column holding only numbers, and ordering it as a number gives a different answer, so the gold may be sorting 9.5 above 10
the statement states no top level ORDER BY
{
"heuristic": true,
"reason": "the statement states no top level ORDER BY"
}this statement cuts its result at a LIMIT that does not decide which rows come back, so a different but equally correct statement can return other rows and score zero
the statement states no LIMIT
{
"heuristic": true,
"reason": "the statement states no LIMIT"
}rerun over the same rows in another physical order this statement gives another answer, so its result depends on how the rows are stored and not only on the data
{
"heuristic": true,
"rule": "R-SET",
"baseline_result_hash": "sha256:1aa7da78d5c892782cf7f6a6bd62bfff8fb8cdb0d30fd8bf8c16ceaa23948be5",
"baseline_result": {
"columns": [
{
"name": "superhero_name",
"declared_type": "TEXT"
}
],
"row_count": 22,
"truncated": false,
"rows_shown": 10,
"rows": [
[
{
"type": "str",
"value": "Atlas"
}
],
[
{
"type": "str",
"value": "Atlas"
}
],
[
{
"type": "str",
"value": "Big Man"
}
],
[
{
"type": "str",
"value": "Bloodaxe"
}
],
[
{
"type": "str",
"value": "Blue Beetle"
}
],
[
{
"type": "str",
"value": "Blue Beetle II"
}
],
[
{
"type": "str",
"value": "Captain Epic"
}
],
[
{
"type": "str",
"value": "Chewbacca"
}
],
[
{
"type": "str",
"value": "Fabian Cortez"
}
],
[
{
"type": "str",
"value": "Gravity"
}
]
],
"result_hash": "sha256:1aa7da78d5c892782cf7f6a6bd62bfff8fb8cdb0d30fd8bf8c16ceaa23948be5"
},
"planner_statistics": {},
"shuffle": {
"seed": "1",
"row_limit": 300000,
"tables": [
"superhero",
"colour"
],
"tables_not_shuffled": [],
"tables_skipped_for_size": {},
"tables_not_reached_by_a_copy": {}
},
"shuffled_copies": {
"run": true,
"verdict": "equal",
"differs": false,
"result_hash": "sha256:1aa7da78d5c892782cf7f6a6bd62bfff8fb8cdb0d30fd8bf8c16ceaa23948be5",
"result": {
"columns": [
{
"name": "superhero_name",
"declared_type": "TEXT"
}
],
"row_count": 22,
"truncated": false,
"rows_shown": 10,
"rows": [
[
{
"type": "str",
"value": "Atlas"
}
],
[
{
"type": "str",
"value": "Atlas"
}
],
[
{
"type": "str",
"value": "Big Man"
}
],
[
{
"type": "str",
"value": "Bloodaxe"
}
],
[
{
"type": "str",
"value": "Blue Beetle"
}
],
[
{
"type": "str",
"value": "Blue Beetle II"
}
],
[
{
"type": "str",
"value": "Captain Epic"
}
],
[
{
"type": "str",
"value": "Chewbacca"
}
],
[
{
"type": "str",
"value": "Fabian Cortez"
}
],
[
{
"type": "str",
"value": "Gravity"
}
]
],
"result_hash": "sha256:1aa7da78d5c892782cf7f6a6bd62bfff8fb8cdb0d30fd8bf8c16ceaa23948be5"
}
},
"plan_variant": {
"run": false,
"reason": "the plan variant was not asked for"
}
}this statement returns the same whole row more than once and never says DISTINCT, so a statement answering the same question once per row disagrees on multiplicity alone
from smells.json, 1 row
| Atlas |
{
"heuristic": true,
"rows": 22,
"distinct_rows": 21,
"repeated_rows": 1,
"largest_repeat": 2,
"result_bounded": false,
"distinct_stated": false,
"set_operation": false,
"repeats_of_the_rows_shown": [
2
]
}SELECT T1.superhero_name FROM superhero AS T1 INNER JOIN colour AS T2 ON T1.eye_colour_id = T2.id INNER JOIN colour AS T3 ON T1.hair_colour_id = T3.id WHERE T2.colour = 'Blue' AND T3.colour = 'Brown'result_hash sha256:1aa7da78d5c892782cf7f6a6bd62bfff8fb8cdb0d30fd8bf8c16ceaa23948be5 recomputed from this JSON: match
record_hash sha256:c1604f60d88fef2fc6072b3993df5e270c4c21a357a6beaccb75e2609d6cec04 recomputed from this JSON: match
from evidence-gold.json, 22 rows
| superhero_nameTEXT |
|---|
| Atlas |
| Atlas |
| Big Man |
| Bloodaxe |
| Blue Beetle |
| Blue Beetle II |
| Captain Epic |
| Chewbacca |
| Fabian Cortez |
| Gravity |
| Hawkman |
| Hercules |
| Jack-Jack |
| Mister Knife |
| Molecule Man |
| Multiple Man |
| Rick Flag |
| Scarecrow |
| Scarlet Witch |
| Spider-Girl |
| Toxin |
| X-Man |
re-run this statement read-only against SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/dev/dev_databases/superhero/superhero.sqlite | size=237568 under the session settings and over the data this record's fixture digest names, and compare the two results under R-SET