q220 GOLD-ONLY, dev-20251106 toxicology

The question

Please list the top three elements in the molecule TR000 in alphabetical order.

the hint the set supplies: TR000 is the molecule id.

This question was audited without a prediction beside it, so there is nothing to compare the gold with. The probes below read the gold alone.

The statements

gold

SELECT T.element FROM atom AS T WHERE T.molecule_id = 'TR000' ORDER BY T.element LIMIT 3

T.element: ascending

sha256:e0358e4b2184517a7f71f6597dac3e4c69c5b9094b2e7d5f461a8bd46b864ca2

The results

gold, 3 rows

from evidence-gold.json, 3 rows

elementTEXT
c
cl
cl

The probes

A smell is a mechanical reason to read this gold statement again. It is a heuristic: it does not state that the statement is wrong, and a maintainer decides.

ordering-over-numeric-text quiet

this statement orders by a text column holding only numbers, and ordering it as a number gives a different answer, so the gold may be sorting 9.5 above 10

what it measured

{
  "heuristic": true,
  "keys": [
    {
      "key": "T.element",
      "column": "atom.element",
      "declared_type": "TEXT",
      "not_applicable": null,
      "census": {
        "rows": 12333,
        "nulls": 0,
        "empty_strings": 0,
        "non_numeric": 12333,
        "pattern": "^-?[0-9]+(\\.[0-9]+)?$"
      },
      "every_value_is_numeric": false
    }
  ]
}

arbitrary-cut quiet

this statement cuts its result at a LIMIT that does not decide which rows come back, so a different but equally correct statement can return other rows and score zero

what it measured

{
  "heuristic": true,
  "cut": 3,
  "offset": 0,
  "distinct_kept": false,
  "unbounded_sql": "SELECT T.element, T.element AS attestql_ordering_key_0 FROM atom AS T WHERE T.molecule_id = 'TR000' ORDER BY T.element",
  "unbounded_rows": 5,
  "projected_columns": [
    "element"
  ],
  "ordering_key_columns": [
    "attestql_ordering_key_0"
  ],
  "ordering_keys": [
    {
      "key": "T.element",
      "direction": "asc",
      "nulls": "first",
      "nulls_first_in_effect": true,
      "returned_rows_null_in_this_key": 0,
      "fires": false
    }
  ],
  "tied_at_the_cut": {
    "positions": [
      1,
      2,
      3
    ],
    "tied_rows": 3,
    "distinct_projected_answers": 1,
    "rows": [
      [
        {
          "type": "str",
          "value": "cl"
        }
      ],
      [
        {
          "type": "str",
          "value": "cl"
        }
      ],
      [
        {
          "type": "str",
          "value": "cl"
        }
      ]
    ]
  },
  "case": null
}

not-a-function-of-the-data quiet

rerun over the same rows in another physical order this statement gives another answer, so its result depends on how the rows are stored and not only on the data

what it measured

{
  "heuristic": true,
  "rule": "R-ORD",
  "baseline_result_hash": "sha256:e0358e4b2184517a7f71f6597dac3e4c69c5b9094b2e7d5f461a8bd46b864ca2",
  "baseline_result": {
    "columns": [
      {
        "name": "element",
        "declared_type": "TEXT"
      }
    ],
    "row_count": 3,
    "truncated": false,
    "rows_shown": 3,
    "rows": [
      [
        {
          "type": "str",
          "value": "c"
        }
      ],
      [
        {
          "type": "str",
          "value": "cl"
        }
      ],
      [
        {
          "type": "str",
          "value": "cl"
        }
      ]
    ],
    "result_hash": "sha256:e0358e4b2184517a7f71f6597dac3e4c69c5b9094b2e7d5f461a8bd46b864ca2"
  },
  "planner_statistics": {},
  "shuffle": {
    "seed": "1",
    "row_limit": 300000,
    "tables": [
      "atom"
    ],
    "tables_not_shuffled": [],
    "tables_skipped_for_size": {},
    "tables_not_reached_by_a_copy": {}
  },
  "shuffled_copies": {
    "run": true,
    "verdict": "equal",
    "differs": false,
    "result_hash": "sha256:e0358e4b2184517a7f71f6597dac3e4c69c5b9094b2e7d5f461a8bd46b864ca2",
    "result": {
      "columns": [
        {
          "name": "element",
          "declared_type": "TEXT"
        }
      ],
      "row_count": 3,
      "truncated": false,
      "rows_shown": 3,
      "rows": [
        [
          {
            "type": "str",
            "value": "c"
          }
        ],
        [
          {
            "type": "str",
            "value": "cl"
          }
        ],
        [
          {
            "type": "str",
            "value": "cl"
          }
        ]
      ],
      "result_hash": "sha256:e0358e4b2184517a7f71f6597dac3e4c69c5b9094b2e7d5f461a8bd46b864ca2"
    }
  },
  "plan_variant": {
    "run": false,
    "reason": "the plan variant was not asked for"
  }
}

duplicate-full-row fired

this statement returns the same whole row more than once and never says DISTINCT, so a statement answering the same question once per row disagrees on multiplicity alone

from smells.json, 1 row

what it measured

{
  "heuristic": true,
  "rows": 3,
  "distinct_rows": 2,
  "repeated_rows": 1,
  "largest_repeat": 2,
  "result_bounded": false,
  "distinct_stated": false,
  "set_operation": false,
  "repeats_of_the_rows_shown": [
    2
  ]
}

The evidence records

gold: evidence-gold.json

SELECT T.element FROM atom AS T WHERE T.molecule_id = 'TR000' ORDER BY T.element LIMIT 3

statement read from: data/questions/dev_20251106-00000-of-00001.json
digest: sha256:ffd8018378ddb1a8794753e0a31cfc81862ff7318a5184c22f3dc4ce03a03feb
origin: https://huggingface.co/datasets/birdsql/bird_sql_dev_20251106/resolve/3c11fb193e5439b338e23677fa0aae11e8b85db9/data/dev_20251106-00000-of-00001.json (commit 3c11fb19, downloaded 2026-09-07), 2026-01-18T08:51:02Z

result_hash sha256:e0358e4b2184517a7f71f6597dac3e4c69c5b9094b2e7d5f461a8bd46b864ca2 recomputed from this JSON: match

record_hash sha256:02a7c48432128782756bf7acbdb8c809332f085b4dba722a82dda3681b0e722b recomputed from this JSON: match

the result this record holds, 3 rows

from evidence-gold.json, 3 rows

elementTEXT
c
cl
cl

what ran, and where

run: audit-60e8e38d-f7d5-4cd1-95e4-3c5d1bd52d16
executed at: 2026-09-14T07:24:40.643960+00:00
data as of: 2026-09-14T07:24:25.395520+00:00
backend at checkout: SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/dev/dev_databases/toxicology/toxicology.sqlite | size=2678784
backend that answered: SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/dev/dev_databases/toxicology/toxicology.sqlite | size=2678784
database role: file
replay rule: R-ORD
question set version: sha256:ffd8018378ddb1a8794753e0a31cfc81862ff7318a5184c22f3dc4ce03a03feb
validator: audit:sqlglot-sqlite-parse
checks run: parses_as_exactly_one_statement, the_one_statement_is_a_select, no_placeholder_without_a_bound_parameter
statement timeout: 30000 ms
rows: 3 rows

the session it ran under

engine: sqlite
time_zone: not stated by this engine
date_style: not stated by this engine
interval_style: not stated by this engine
extra_float_digits: not stated by this engine
database_collation: not stated by this engine
work_mem: not stated by this engine
hash_mem_multiplier: not stated by this engine

recorded beside them

sqlite_version: 3.53.4
encoding: UTF-8
reverse_unordered_selects: 0
query_only: 1
journal_mode: delete
data_version: 1
automatic_index: 1
compile_options: ATOMIC_INTRINSICS=1,COMPILER=clang-21.0.0,DEFAULT_AUTOVACUUM,DEFAULT_CACHE_SIZE=-2000,DEFAULT_FILE_FORMAT=4,DEFAULT_JOURNAL_SIZE_LIMIT=-1,DEFAULT_MMAP_SIZE=0,DEFAULT_PAGE_SIZE=4096,DEFAULT_PCACHE_INITSZ=20,DEFAULT_RECURSIVE_TRIGGERS,DEFAULT_SECTOR_SIZE=4096,DEFAULT_SYNCHRONOUS=2,DEFAULT_WAL_AUTOCHECKPOINT=1000,DEFAULT_WAL_SYNCHRONOUS=2,DEFAULT_WORKER_THREADS=0,DIRECT_OVERFLOW_READ,ENABLE_API_ARMOR,ENABLE_COLUMN_METADATA,ENABLE_DBSTAT_VTAB,ENABLE_FTS3,ENABLE_FTS3_PARENTHESIS,ENABLE_FTS5,ENABLE_GEOPOLY,ENABLE_MATH_FUNCTIONS,ENABLE_MEMORY_MANAGEMENT,ENABLE_PERCENTILE,ENABLE_PREUPDATE_HOOK,ENABLE_RTREE,ENABLE_SESSION,ENABLE_STAT4,ENABLE_UNLOCK_NOTIFY,MALLOC_SOFT_LIMIT=1024,MAX_ATTACHED=10,MAX_COLUMN=2000,MAX_COMPOUND_SELECT=500,MAX_DEFAULT_PAGE_SIZE=8192,MAX_EXPR_DEPTH=1000,MAX_FUNCTION_ARG=1000,MAX_LENGTH=1000000000,MAX_LIKE_PATTERN_LENGTH=50000,MAX_MMAP_SIZE=0x7fff0000,MAX_PAGE_COUNT=0xfffffffe,MAX_PAGE_SIZE=65536,MAX_SQL_LENGTH=1000000000,MAX_TRIGGER_DEPTH=1000,MAX_VARIABLE_NUMBER=250000,MAX_VDBE_OP=250000000,MAX_WORKER_THREADS=8,MUTEX_PTHREADS,SYSTEM_MALLOC,TEMP_STORE=1,THREADSAFE=1,USE_URI
collation_list: RTRIM,NOCASE,BINARY
case_sensitive_like: 0

the rendering and the data

version: attestql/audit/4
numeric_scale: 6
timestamp_format: %Y-%m-%dT%H:%M:%S.%fZ
timezone: UTC
null_rendering: NULL
encoding: utf-8
schema digest: sha256:d8fa71c3e79cecc54e52634cb1665f9665f1dfc8352f9d935bb3abfb1bab7464
source file sha256: sha256:f5fa7f21af1ad878ff8fef1b0582b8cb2d7ed63dbac65ff16d2ba05667650c5b
rows in main.atom: 12333

Running these again

gold

re-run this statement read-only against SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/dev/dev_databases/toxicology/toxicology.sqlite | size=2678784 under the session settings and over the data this record's fixture digest names, and compare the two results under R-ORD

This question's run

run: audit-60e8e38d-f7d5-4cd1-95e4-3c5d1bd52d16
server: SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/dev/dev_databases/toxicology/toxicology.sqlite | size=2678784
question set: dev_20251106-00000-of-00001 from https://huggingface.co/datasets/birdsql/bird_sql_dev_20251106/resolve/3c11fb193e5439b338e23677fa0aae11e8b85db9/data/dev_20251106-00000-of-00001.json (commit 3c11fb19, downloaded 2026-09-07)
replay rule: R-ORD

the run this question belongs to

The JSON this page was rendered from

evidence-gold.json the gold's evidence record
smells.json attestql/audit/smells/2

The question

The statements

◀ gold

The results

The probes

ordering-over-numeric-text quiet

arbitrary-cut quiet

not-a-function-of-the-data quiet

duplicate-full-row fired

The evidence records

gold: evidence-gold.json

recorded beside them

Running these again

gold

This question's run

The JSON this page was rendered from

gold