R-SET GOLD-ONLY duplicate-full-row
student_club · mini_dev_pg-00000-of-00001 from https://huggingface.co/datasets/birdsql/bird_mini_dev/resolve/f65faf4ae3b638c1fa6df1d3370c8d92c8366301/data/mini_dev_pg-00000-of-00001.json (commit f65faf4a, downloaded 2026-09-14)
Was each expense in October Meeting on October 8, 2019 approved?
the hint the set supplies: event_name = 'October Meeting' where event_date = '2019-10-08'; approved = True means expenses was approved; approved = False means expenses was not approved
This question was audited without a prediction beside it, so there is nothing to compare the gold with. The probes below read the gold alone.
SELECT T3.approved FROM event AS T1 INNER JOIN budget AS T2 ON T1.event_id = T2.link_to_event INNER JOIN expense AS T3 ON T2.budget_id = T3.link_to_budget WHERE T1.event_name = 'October Meeting' AND T1.event_date LIKE '2019-10-08%'this statement states no ordering of its own
sha256:d216577fa16de9354f8180b23fc433157bac111695b90d4e5655c70e27cce55b
from evidence-gold.json, 3 rows
| approvedtext |
|---|
| true |
| true |
| true |
A smell is a mechanical reason to read this gold statement again. It is a heuristic: it does not state that the statement is wrong, and a maintainer decides.
this statement orders by a text column holding only numbers, and ordering it as a number gives a different answer, so the gold may be sorting 9.5 above 10
the statement states no top level ORDER BY
{
"heuristic": true,
"reason": "the statement states no top level ORDER BY"
}this statement cuts its result at a LIMIT that does not decide which rows come back, so a different but equally correct statement can return other rows and score zero
the statement states no LIMIT
{
"heuristic": true,
"reason": "the statement states no LIMIT"
}rerun over the same rows in another physical order this statement gives another answer, so its result depends on how the rows are stored and not only on the data
{
"heuristic": true,
"rule": "R-SET",
"baseline_result_hash": "sha256:d216577fa16de9354f8180b23fc433157bac111695b90d4e5655c70e27cce55b",
"baseline_result": {
"columns": [
{
"name": "approved",
"declared_type": "text"
}
],
"row_count": 3,
"truncated": false,
"rows_shown": 3,
"rows": [
[
{
"type": "str",
"value": "true"
}
],
[
{
"type": "str",
"value": "true"
}
],
[
{
"type": "str",
"value": "true"
}
]
],
"result_hash": "sha256:d216577fa16de9354f8180b23fc433157bac111695b90d4e5655c70e27cce55b"
},
"planner_statistics": {
"expense": {
"last_analyze": "2026-09-14 12:50:59.480202+00",
"last_autoanalyze": null,
"n_mod_since_analyze": 0
},
"event": {
"last_analyze": "2026-09-14 12:50:59.479238+00",
"last_autoanalyze": null,
"n_mod_since_analyze": 0
},
"budget": {
"last_analyze": "2026-09-14 12:50:58.001042+00",
"last_autoanalyze": null,
"n_mod_since_analyze": 0
}
},
"shuffle": {
"seed": "1",
"row_limit": 300000,
"tables": [
"expense",
"event",
"budget"
],
"tables_not_shuffled": [],
"tables_skipped_for_size": {
"laptimes": 400524,
"legalities": 427907,
"posthistory": 303155,
"trans": 1056320,
"yearmonth": 383282
},
"tables_not_reached_by_a_copy": {}
},
"shuffled_copies": {
"run": true,
"verdict": "equal",
"differs": false,
"result_hash": "sha256:d216577fa16de9354f8180b23fc433157bac111695b90d4e5655c70e27cce55b",
"result": {
"columns": [
{
"name": "approved",
"declared_type": "text"
}
],
"row_count": 3,
"truncated": false,
"rows_shown": 3,
"rows": [
[
{
"type": "str",
"value": "true"
}
],
[
{
"type": "str",
"value": "true"
}
],
[
{
"type": "str",
"value": "true"
}
]
],
"result_hash": "sha256:d216577fa16de9354f8180b23fc433157bac111695b90d4e5655c70e27cce55b"
}
},
"plan_variant": {
"run": false,
"reason": "the plan variant was not asked for"
}
}this statement returns the same whole row more than once and never says DISTINCT, so a statement answering the same question once per row disagrees on multiplicity alone
from smells.json, 1 row
| true |
{
"heuristic": true,
"rows": 3,
"distinct_rows": 1,
"repeated_rows": 1,
"largest_repeat": 3,
"result_bounded": false,
"distinct_stated": false,
"set_operation": false,
"repeats_of_the_rows_shown": [
3
]
}SELECT T3.approved FROM event AS T1 INNER JOIN budget AS T2 ON T1.event_id = T2.link_to_event INNER JOIN expense AS T3 ON T2.budget_id = T3.link_to_budget WHERE T1.event_name = 'October Meeting' AND T1.event_date LIKE '2019-10-08%'result_hash sha256:d216577fa16de9354f8180b23fc433157bac111695b90d4e5655c70e27cce55b recomputed from this JSON: match
record_hash sha256:eb848283c48d8c28e9c2c519795f1606360658b567f893c97a25236d713d981e recomputed from this JSON: match
from evidence-gold.json, 3 rows
| approvedtext |
|---|
| true |
| true |
| true |
re-run this statement read-only against PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit | server=172.17.0.2/32:5432 | database=bird under the session settings and over the data this record's fixture digest names, and compare the two results under R-SET