# dataganger 0.6.1

CRAN resubmission changes requested on 2026-07-10.

*   DESCRIPTION now quotes the package name `'shiny'` correctly in the
    Description field.
*   `read_input()` examples are now self-contained, executable, and no longer
    use commented-out code.
*   `export_synthetic()`, `make_agent_bundle()`, and
    `export_diagnostic_package()` examples now write only to temporary paths
    and no longer use `\dontrun{}`.
*   Audited exported write functions, examples, tests, and vignettes for
    home-filespace writes; package examples now stay within `tempfile()` /
    `tempdir()`.

# dataganger 0.6.0

One minimal export bundle, a Configure page with no silent defaults, and a package-wide audit pass: privacy fixes, an honest engine story, honest privacy wording, and a lighter dependency footprint.

## Privacy fixes

*   **k-anonymity now runs before column renaming.** Previously, with
    `name_strategy = "generic"`, generic renaming ran first and recorded the
    name map before `enforce_kanon()` dropped direct identifiers and applied
    suppression — so bundles could under-protect. Both engine paths now
    enforce k-anonymity first. Bundles generated with generic names on
    earlier versions may be under-protected; regenerate them.
*   The manifest and the privacy report now agree on a single exact-row-match
    number: `export_synthetic()` uses the same roles-derived exclusion rule
    as `privacy_check_post()`.
*   Seeded synthesis is fully deterministic: `decimal_places()` samples long
    columns with a deterministic stride instead of `sample()`, removing an
    RNG side effect.
*   The app's theme no longer references Google Fonts at all
    (`bslib::font_google()` replaced with plain family strings served from
    the packaged self-hosted files); the no-network source guard now also
    scans `inst/app/`.

## Bundle & agent skill

*   Export bundles use one minimal layout: `synthetic_data.csv` at the root,
    `human/` (`human.md` with the privacy report folded in, plus optional
    `comparison_report.html`), and `agent/` (`recipe.yaml` = combined
    spec + roles, `AGENT.md`, `manifest.json`). `data_dictionary.csv`,
    `load_data.R`, `analysis.qmd`, `ai-readme.md`, `privacy_report.txt`, and
    the separate `spec.yaml`/`roles.yaml` are gone. CLI `synthesize` gains
    `--recipe <recipe.yaml>`; `--spec`/`--roles` remain supported. `compact`
    and `include_dictionary` are deprecated no-ops.
*   `export_synthetic()` honors its `code_readiness` argument: when
    supplied, the bundle gains `agent/code_readiness_report.json`;
    `make_agent_bundle()` computes it automatically, so every agent bundle
    now ships the structural-compatibility report.
*   Both shipped skills (`AGENT.md` in every bundle and
    `using-dataganger-bundles`) rewritten to the minimal bundle contract.

## Engine

*   `"auto"` is a real engine alias in `synth_spec()`, `synthesize_data()`,
    the CLI (`--engine <auto|internal|synthpop>`), and spec YAML. An explicit
    `"auto"` behaves exactly like leaving the engine unset: the engine is
    derived from the objective and `dataganger.disable_synthpop` is
    respected.
*   The misleading `engine_required` spec field is retired; spec printing
    reports the explicit engine or `auto (derived from objective)`.

## App (Shiny)

*   Compare now separates Univariate and Bivariate views. The Bivariate view
    uses an X-by-synthetic interaction test to show whether predictor-outcome
    relationships changed, with outcome-specific effect sizes and p-value
    fidelity colors.
*   Exported comparison reports now include the relationship-interaction table,
    using data-column order to define predictor then outcome.
*   Synthesis controls are folded into collapsed **Advanced settings**, keeping
    the generation review focused on the effective configuration.
*   Generation guidance now invites users to review, generate, or go back to
    adjust settings, and the data panel automatically previews each newly
    generated synthetic dataset.
*   When `synthpop` is unavailable, the upload attestation recommends installing
    it for correlation-aware synthesis.
*   Configure has no silent defaults: Q1 (Points to a person?) and Q2
    (Sensitive?) start blank for every column — auto-detected values no
    longer pre-select — and generation is gated until every column has an
    explicit answer to both questions. Explicit UI answers are tracked
    separately (`user_identifies` / `user_sensitive`) so CLI synthesis is
    unaffected.
*   The upload fail-safe flags only direct-identifier candidates (ID
    patterns / free text), no longer sensitive-named columns such as
    `income`; flagged columns get a "potential identifier" pill and
    semantically-coloured actions.
*   Categorical comparisons are now inference-aware like numeric ones:
    colored by a chi-square/Fisher distributional p-value with TVD as the
    displayed effect size; the SMD definition is shown on the effect column.
*   Assorted Configure/Compare/Generate polish: bottom Confirm-and-Continue,
    calmer attestation wording (with a disable-internet note), preserve-panel
    highlight, per-question help tied to table columns, and a generation
    fidelity recap.

## Docs

*   Sensitive-column wording is honest and consistent everywhere:
    quasi-identifying columns are "grouped with k-anonymity so no rare
    combination survives"; sensitive non-identifying columns are "recreated
    from its distribution; exact values are not copied — attribute-level
    protection is not yet applied". The former "protected from linkage"
    claim is gone.
*   Engine documentation matches reality (demo → internal; development →
    synthpop when installed; analytics → synthpop + risk acknowledgement).
*   `detect_roles()` documents the two-axis columns; assorted roxygen and
    vignette corrections; the startup message is reduced to version +
    `run_app()`.

## Dependencies & internals

*   `purrr`, `tidyr`, and `vctrs` dropped from Imports (unused); `plotly`
    moved to Suggests with an install gate in the Compare module.
*   `check_code_readiness()` now documents and reports that
    `haven_labelled` → character is the expected round-trip for now.
*   Internal hygiene: unified ID-name regex, NULL/NA-safe role lookups (a
    roles object missing a column no longer errors), helper relocations, and
    dead-code removal.

# dataganger 0.5.0

Privacy gating, UI/CLI parity, an agent skill, and a provable no-network guarantee.

*   Comparison stats are now inference-aware for numeric variables: the Compare
    view shows mean SMD, SD ratio, and median standardized difference, each
    coloured by their t/F/Wilcoxon p-value bands; min/max remain value-only.
*   UI export bundles now include `spec.yaml` and `roles.yaml`, and CLI
    `synthesize --roles` can reuse the full role matrix so UI and CLI runs
    reproduce byte-identical output with the same seed.
*   The app now opens with a hard no-direct-identifiers attestation gate, then
    runs an early assistive fail-safe immediately after upload to flag possible
    direct identifiers before Objective / Configure. Once attested, Configure's
    first question collapses to `none` / `combination`. The two questions are
    framed as the remaining risks after direct identifiers: linkage (combination)
    and sensitivity.
*   Added an agents-only packaged `SKILL.md` plus `dataganger skill [--out <file>]`
    so an AI can drive the package to generate synthetic data without ever
    reading the real data; fixed `ai-readme.md` so dropped columns are not listed
    as `NA (NA)`.
*   No-network guarantee: web fonts are now self-hosted (no Google Fonts CDN), so
    the app makes no external requests; `report_issue()` prints a copy-paste
    GitHub issue instead of opening a browser (the Shiny button shows a copyable
    modal). A shipped runtime trap test and source guard prove the package makes
    no network calls, and a Linux `unshare -rn` CI job runs the suite with no
    network at all.
*   New `vignette("privacy-and-ai-workflow")` documents the privacy gating ladder,
    the two ways to use the package with AI, and the no-network guarantee.

# dataganger 0.4.0

Configure redesign around two intrinsic privacy questions.

*   The Configure step now classifies each column by answering two independent
    questions — does it point to a person (`identifies`: none / combination /
    direct) and is it sensitive (`sensitive`) — and **derives** the treatment
    rather than asking the user to pick it. The two questions are shown
    prominently above the per-column table.
*   k-anonymity membership now reads both axes, so a column that is both
    identifying-in-combination and sensitive is covered. Numeric
    quasi-identifiers are no longer coarsened into `NA` bins.
*   Each column row has an **Action override** column exposing the Action
    (synthesize / pass through / drop) and data-type overrides directly.
*   The Generate step now shows a per-column review table (points to a person?,
    sensitive?, action, and a plain-English outcome) so choices can be verified
    before generating.
*   Objective selection uses a single Protection meter, makes **development**
    the default objective, and rewrites the per-objective detail panel around
    consistent dimensions for use, values, relationships, identifiers, and
    sensitive / rare data.
*   Synthesis Settings labels are more human-readable, with matching
    `synth_spec()` documentation for the current settings surface.
*   The per-column data preview includes a filter so you can inspect one
    variable at a time while reviewing the Configure step.
*   `export_synthetic(compact = )` supports two bundle variants: the compact app
    download and the full CLI / agent bundle.
# dataganger 0.3.5

Generation, comparison, and export clarity pass:

*   Generate page now shows a read-only "Column decisions" snapshot (the Configure
    table as final, non-editable values) so choices can be reviewed before
    generating.
*   New `report_issue()` helper plus an in-app **Report a problem** button open
    a pre-filled GitHub issue with environment details, without sending anything
    automatically.
*   The engine recap resolves to the engine actually used (e.g. `synthpop (auto)`)
    after generation, instead of always showing `auto`.
*   The Regenerate button is disabled until the first generation, so it no longer
    duplicates Generate on the initial visit.
*   Exact-row-match count moved into the result stats; the redundant verbatim
    Result box was removed.
*   Compare page treats geography columns as categorical, so they get an
    original-vs-synthetic comparison instead of being skipped.
*   Export page gains a generation summary: original rows/columns, how many
    columns were synthesized, passed through, and dropped, and the final
    synthetic dimensions.
*   Internal: the cancellable-synthesis subprocess no longer uses a `:::`
    self-reference (clearing an `R CMD check` NOTE); spelling WORDLIST expanded.

# dataganger 0.3.4

Configure page clarity pass:

*   Integer-valued columns now display without spurious decimals (e.g. `127`, not
    `127.00`) by detecting whole-valued numerics, not just R integer storage.
*   The column-roles table renames "Simulation" to **Action**, folds the
    recommendation inline into the TYPE control (`... (recommended)`), drops the
    redundant `recommended_role` column, and adds a per-row info tooltip with a
    plain-English reason and the storage type.
*   Setting a column's Action to **Drop** or **Pass through** now greys out and
    disables its TYPE and DISCLOSURE selectors and no longer blocks generation;
    pass-through columns carry a "real values - verify before sharing" note.
*   Disclosure-detection reason strings rewritten in plain English.
*   Collapsible help now uses an obvious `+`/`-` affordance.

# dataganger 0.3.3

*   `pkgload` removed from `Suggests` (uses `.__DEVTOOLS__` namespace check directly).
*   CRAN-readiness: `cran-comments.md` refreshed with accurate 0/0/2 NOTE explanations.
*   Added `pkgdown` site generation via GitHub Actions (`_pkgdown.yml` + workflows).
*   Planning artifacts (`docs/superpowers/`, `todo.md`) migrated out of the package root.

# dataganger 0.3.2

## Disclosure roles
* `detect_roles()` is now conservative: it only auto-assigns a disclosure role
  when confident (a clear direct identifier, or a known-sensitive column name).
  All other columns are left **unselected** rather than defaulted to
  quasi-identifier. This fixes the root cause of the 100%-NA synthetic output:
  measures, counts, dates, and low-cardinality categoricals are no longer
  silently treated as quasi-identifiers.
* The Configure page now **requires an explicit disclosure role for every
  column** before generating. A live counter shows how many are still
  unselected. `None` is a valid explicit choice; empty is not.
* k-anonymity fires only on columns the user marks as quasi-identifiers. The
  `max_suppress_frac` feasibility backstop is retained as defense-in-depth.
* CLI: spec YAML accepts a `disclosure_roles:` mapping (column -> role) so
  disclosure decisions are reproducible from the command line.

# dataganger 0.3.1

* **Bug fix — CUSUM hang (Bug 5)**: Synthesis no longer hangs on datasets with
  character-stored date columns (e.g. "Jun 8, 2019") or other high-cardinality
  character columns. The root cause was two-fold: (1) character date strings were
  classified "unknown" and passed to synthpop as a 2000+-level factor, causing
  CART to enumerate billions of split candidates; (2) even moderate-cardinality
  character columns (>20 distinct values) used as CART *predictors* trigger the
  same 2^(k-1) blowup at k=34. Fix: `detect_roles()` now detects date strings
  (ISO, "Mon DD YYYY", MM/DD/YY) via regex and classifies them as "date"; and a
  new `synthpop_bridge_cols()` function excludes any character column with >20
  distinct values from synthpop's CART, synthesizes it independently via the
  marginal engine, and stitches it back into the output in the original column
  order. Generation on the CUSUM test file (41 k rows, 14 columns) now completes
  in under 10 seconds at both 50-row and 5000-row target sizes.

* **P1 — Configure busy indicator**: the Upload page now shows a "Profiling
  data…" / "Detecting column roles…" progress bar while the app analyses the
  uploaded file, so users know the app is working rather than frozen.

* **P2 — Row count first**: the Row count (n) input is now the first item in
  the Configure advanced-settings panel.

* **P3 — Role-reactive row suggestion**: `suggest_min_rows()` gains a `data`
  parameter; when called with `data` and `roles`, it recomputes the coverage
  estimate over only the columns that are still being synthesized. Dropping an
  ID or excluded column now immediately lowers the suggested row count on the
  Configure page.

* **P4 — Case IDs render as character**: columns detected as ID candidates are
  coerced to character in the data-panel preview, so a numeric case ID displays
  as "1078541" instead of "1,078,541.00".

* **P5 — Column summary stats**: the Configure page now shows a per-column
  summary section below the synthesis settings. Continuous columns get a
  min / Q1 / median / Q3 / max / mean / SD table; categorical columns get a
  top-5 frequency table with counts and percentages.

* **P6 — Generation progress bar + timer**: while synthesis is running, the
  Generation page displays a `MM:SS` elapsed-time counter and an animated
  progress bar. Both update every second.

# dataganger 0.3.0

* Cancellable background synthesis: the Shiny Generation step now runs the
  synthesize -> compare -> privacy pipeline in a background process (via
  `callr`), so the app stays responsive and a Cancel button can stop a long or
  stuck run. Falls back to synchronous in-process generation when `callr` is
  unavailable or the package is dev-loaded. The `dataganger.synthesis_async`
  option forces the deterministic synchronous path for tests and CI.

* Coverage-based row-count suggestion: `profile_data()` now carries
  cross-column coverage (distinct joint combinations of categorical columns
  plus the largest per-column level count), and the new `suggest_min_rows()`
  function turns that into a sufficient synthetic row count - capped at 5000,
  floored at the largest level count, and never above the original. The
  Configuration row-count slider pre-fills with the suggestion, shows an
  inline hint, and warns when set below the coverage floor. The Upload step
  shows a coverage-summary card.

* Diagnostics for long-running synthesis: `dg_log` / `dg_timeit` emit
  per-phase progress to the R console when `options(dataganger.verbose =
  TRUE)`, and `check_cancel()` polls `options(dataganger.cancel)` at column
  boundaries for cooperative cancellation. `.onAttach()` prints a startup
  hint with the package version and how to launch the app and CLI.

* Free-text detection now head-samples to 1000 rows, bounding a hot path
  that could slow the transition into Configure on wide or long-string data.

* `synthesize_marginal()` trusts the detected role instead of recomputing the
  free-text heuristic, removing a redundant pass over character columns.

* The Comparison step no longer shows a stale full table on first transition:
  the first comparable variable renders correctly without needing a click.

* The Configuration step shows inline help for each disclosure role (None,
  Direct identifier, Quasi-identifier, Sensitive) with a short example.

* `callr` and `pkgload` added to `Suggests`.

# dataganger 0.2.2

* The Column Roles step now shows a non-blocking notice when the uploaded data
  looks like an aggregated counts table rather than individual records, since
  disclosure control assumes individual-level microdata.

* New option `dataganger.disable_synthpop`: set
  `options(dataganger.disable_synthpop = TRUE)` to steer objective-derived
  synthesis onto the internal engine even when synthpop is installed. Intended
  for environments where a synthpop synthesis is undesirable or can hang
  unattended (for example continuous integration). An explicit
  `engine = "synthpop"` request is still honoured.

# dataganger 0.2.1

* Shiny app interface refinements: the Objective step shows each purpose's
  details on selection and uses consistent "more bars = stronger" meters,
  including an Anonymity meter for resistance to re-identification. Each
  synthesis engine (auto, internal, synthpop) now has a plain-language
  explainer. The Generation step shows a configuration recap (including
  advanced settings) with an "Adjust settings" shortcut back to Configuration.
  The Comparison step has a wrapping variable grid and a clearer original vs.
  synthetic explainer. The Export step downloads a single bundle (synthetic
  data as CSV, documentation, comparison report, and an analysis notebook),
  with an optional save-to-folder for sessions run locally.

* Every bundle now includes `analysis.qmd`, a Quarto report with runnable R
  code and reference Python code to read both the original and synthetic data
  and compare them (summary statistics, distribution plots, and DataGangeR's
  fidelity metrics).

* The Shiny synthesis spec exposes an engine selector (auto / internal /
  synthpop) with an inline note on whether synthpop is installed.

* Ship an agent skill (`inst/skills/using-dataganger-bundles/`) describing how
  AI agents should consume a bundle: never access the real data, ask the human
  for a go-ahead before touching the synthetic data, and where to save work.

* Fixed `interpolate()` so bundle helper files (e.g. `load_data.R`) render
  their templates instead of shipping literal placeholders.

* DataGangeR now routes relationship-preserving objectives to the optional
  synthpop engine when installed. Please cite: Nowok B, Raab GM, Dibben C
  (2016). "synthpop: Bespoke Creation of Synthetic Data in R." *Journal of
  Statistical Software*, 74(11), 1-26. doi:10.18637/jss.v074.i11

* Initial CRAN-ready scaffold.
