# Data and measurement audit — 10 September 2026

This report supersedes the measurement claims in the 12 August discovery report where they conflict. The immutable source snapshots remain dated **12 August 2026**. The September checks below establish that selected official access routes still work; they are not a new representative collection.

**Decision: a public-data feasibility study and a controlled presentation experiment are feasible. The repository does not yet establish a digital-capacity gap, AI effectiveness, or an effect on actual funding.** Present the work to a professor as a proposed study with a working data acquisition pilot, not as completed empirical findings.

## 1. What the data actually contain

The raw DREAM snapshot contains 100 unique records, all with `record_type = project`: 74 rehabilitation, 23 construction and 3 equipment projects. These are public project records. A project can have applications to financial programs, but the project record is not itself a universal grant application or a complete set of submitted, rejected and successful applications.

The collector requested the **100 most recently updated public records**, from a listing that reported 14,613 total records. Project creation dates span 5 September 2023–30 July 2026; update dates span 13 July–12 August 2026. Updated projects are a selected group. The pilot cannot support population project-creation rates, representative success rates or community rankings.

Recomputed availability from the stored raw records and rebuilt outputs:

| Observation | Projects out of 100 | What this establishes |
|---|---:|---|
| Stable DREAM project code | 100 | Traceable identifiers in this snapshot |
| Location code found in digital-index directory | 98 | An exact code join, not verified responsibility or representativeness |
| Distinct primary matched communities | 63 | Number of communities represented after the current first-code rule |
| Population profile value | 96 | Current portal value, with reference year unconfirmed |
| English abstract | 100 | Narrative present, not a verified problem statement |
| English objective | 97 | A populated source field |
| Explicit `numberOfConsumers` target | 78 | Reported service-user target; still requires substantive checking |
| At least one numeric target metric | 92 | Numeric field availability; unit and outcome meaning need validation |
| Forecast budget | 85 | Estimated project cost, not necessarily requested grant amount |
| Document metadata | 98 | Document records exist; content has not been downloaded and verified for all projects |
| Any funding-source records | 50 | Structured source records; not proof of funding success |
| Indicative source amount | 19 | Reported expected source amount |
| Active award with approved amount | 11 | Specific award evidence |
| Committed-source amount | 38 | Source status explicitly says `committed` |
| Contract amount | 41 | Procurement contract evidence |
| Procurement payment amount | 30 | Recorded transaction evidence |
| Distinct project cash-receipt field | 0 | Not available in this extraction |

Funding groups overlap. They are **not** successive levels through which every project is known to have passed. Absence in this snapshot means “not observed,” not “rejected,” “unfunded” or zero.

## 2. Official access: live checks and remaining acquisition work

The following were checked with bounded read-only requests on 10 September 2026:

1. The [official DREAM announcement of its public API](https://dream.gov.ua/ua/news/article-6) links to [Open Contracting’s DREAM API documentation](https://open-contracting.github.io/dream-api-docs/). The documentation and its [OpenAPI specification](https://open-contracting.github.io/dream-api-docs/openapi.yaml) returned HTTP 200. The specification identifies version 1.0 and the production host `public-api.dream.gov.ua`.
2. `GET https://public-api.dream.gov.ua/marketplace/public/dream/ideas?from=2026-09-09T00%3A00%3A00Z&order=asc` returned HTTP 200 with 14 changed-project identifiers. The documented `from` and `order` parameters provide a route for synchronization by modification time.
3. A single returned identifier was fetched through `/marketplace/public/dream/ideas/{id}`: HTTP 200, about 98 KB. The response is wrapped in `internal` and `cdu_response`. The latter contained project code, creation/update time, multilingual title and description, budget, metrics, documents, related processes and contracting processes. **This schema differs from the portal `/api/project/{code}` snapshot schema; a new adapter and equivalence checks are required before replacing the collector.** The wrapper also differs from the bare project shape illustrated in the specification.
4. The [Diia regional route](https://backend.hromada.gov.ua/api/front/index-group/regions-by-group?belongsTo=community&v=2) returned HTTP 200 with 26 region entries. The [Vinnytsia community route](https://backend.hromada.gov.ua/api/front/index-group/communities-by-region/1) returned HTTP 200 with 63 communities and the same fields used by the old collector: `code_3`, `isPublished`, `indexGeneral`, and four component scores. No measurement period or methodology version appeared in these community rows. This was one-region verification, not a September recollection of all 1,470 rows.

The old collector is functional evidence of a portal-based acquisition route, but its public-JavaScript bearer discovery is brittle. A documented public API **does exist**. Prefer testing that official route for a full project frame and repeatable updates. Document collection dates, source responses and hashes, handle pagination/cursors, deduplicate stable codes, and compare a small overlap against the current snapshot before scaling. Source availability is established; complete historical coverage, application coverage and redistribution terms are not established by these checks.

## 3. Digital capacity is an optional, unresolved explanatory measure

The August directory has 1,470 unique community codes: 1,379 marked published and 91 not published. Among the latter, 87 have nonzero scores and 4 have zero. The [public index page](https://hromada.gov.ua/index) still displayed 1,265 communities during the September check. These are three different counts with an unresolved meaning; none should be silently used as the number of valid contemporary observations.

The [official home page](https://hromada.gov.ua/) describes live interim regional/community measurements and anticipated Q2 2026 results. This is useful context, **not proof that every community score in either snapshot refers to Q2 2026**. The [April 2025 official instructions](https://hromada.gov.ua/post/trivaje-podannya-danix-do-indeksiv-cifrovoyi-transformaciyi) describe quarterly submission by digital leaders and four community dimensions. Thus the index includes reported administrative data and participation, not a direct measurement of grant-writing skill or staff capacity.

An official [first-measurement methodology report](https://backend.hromada.gov.ua/storage/uploads/uploads/report/Report-UKR.pdf) is discoverable. Its existence does not resolve the version or period of the current API values; the full report was not used to assert current weighting or validity in this audit.

All 98 pilot joins use location codes; no organization fallback is needed in this particular sample. One joined project has **two** matched location community codes (`DREAM-UA-140524-4A24C00C`); the existing primary-community selection is arbitrary for causal attribution. Another matched project's index row is not published. Both are now flagged. For a full study, review multi-community projects and determine the responsible applicant entity before assigning a community-level exposure. Obtain a dated, valid index export before estimating historical capacity relationships. The core presentation experiment can proceed without that index; a capacity-inequality extension cannot yet do so credibly.

## 4. Material measurement defects found and corrected

The audit changed `packages/research/build_pilot.py`, added regression checks to `tests/research/test_metric_engine.py`, and rebuilt normalized and analysis artifacts from the unchanged raw snapshots.

| Defect in previous output | Correction / result |
|---|---|
| A cumulative evidence funnel displayed 30 projects as “Received” even though no receipt field existed; it displayed 45 as “Approved” when only 11 had an approved amount. | Independent field counts now show approval 11, commitment 38, contracts 41, receipts 0 and payments 30. Counts are labelled as overlapping availability. |
| “With funding source” counted any forecast budget as a funding source. | Counts actual `funding_sources` records: 50, previously 85. |
| Beneficiaries were the maximum of metrics whose name included `capacity`, whether or not their unit represented people. | Only explicit `numberOfConsumers` targets qualify; conflicting targets return unknown. Coverage changes from 80 to 78. In three records, generic capacity previously overrode a smaller explicit service-user target. |
| A missing commitment became zero when computing `funding_gap`. | Missing remains null. Where both values exist, the difference is explicitly labelled forecast cost minus observed commitment, not verified financing need. |
| The final array element was used as current status although the source array was newest-first. | Select the dated latest status. Corrected 55 records; current counts are 75 active, 25 pending. |
| Project-level rank-based quartiles split two communities across different capacity groups. | Bins are derived from unique communities with tied scores preserved, then mapped back to projects. |
| T0/T4 automatically passed numeric validation and every treatment claimed zero unsupported additions. | Those numeric comparisons are now unassessed; unsupported additions are unknown. All 150 construction examples require source and semantic review. None is eligible for the main comparison. |
| Multi-community matching and unpublished index records were hidden in headline match coverage. | Added candidate counts and review flags. Exact-code coverage remains 98/100. |

Seven regression tests passed, covering funding observation separation, null commitment versus genuine zero, non-person capacity, conflicting people targets, latest status independent of array order, and the limits of numeric treatment checks, alongside existing staged-funding and title-cleaning tests. The artifact now includes an explicit audit notice identifying it as a feasibility pilot.

## 5. Defects and interpretation limits that remain

**Money.** `DREAM-UA-150324-B46FE29B` has forecast budget 9,230.7 UAH and procurement payments of 7,806,526.94 UAH. The source also reports approval near 9.25 million and its own anomalous coverage ratio. There is likely a scale or source-entry problem, but its correction is not known. No amount was automatically multiplied or replaced. All 85 budget records in the snapshot say `forecast`, currency `UAH`, with blank unit. Funding sources and observed contracts/transactions carry UAH; the award records do **not** carry their own currency field, so award-currency verification is incomplete. No within-project duplicate contract IDs, award IDs or payment transaction IDs were found in this snapshot; that does not establish completeness or absence of duplication across sources.

The legacy `supported_amount` still selects the highest observed evidence-stage amount, which may be a forecast, commitment, contract or payment. Its totals, per-capita values and `funded_share` must not be presented as a common financing measure. The binary `funding_success` is likewise an arbitrary mixture of different evidence types; it is not an observed selection decision. These fields and legacy model outputs are retained as diagnostic development artifacts, with warnings; the professor-facing site should use the availability audit instead.

**Timing and progression.** Fifty-five projects have two dated status observations, mainly `pending` and `active`. The median interval is between observed record statuses, not verified time to project approval, grant award, construction or completion. The negative-interval check is weak because dates are sorted first. A survival analysis needs documented lifecycle stages, entry dates and censoring rules.

**Quality.** PQS is a hand-weighted heuristic built largely from populated fields, numeric targets, document-type counts and English sentence length. It has no expert criterion validation. It includes procurement/documentation evidence that may be added after funding and may reflect project age, sector rules or translation quality. A positive association cannot be interpreted as better underlying projects or proof of a capacity gap. The existing regressions omit dated fiscal capacity, conflict exposure and several selection mechanisms; their significance should not be a research finding.

**Experiment.** The 30 examples are the first 30 linked records after sorting by sector, heuristic PQS and cost, not a random experimental sample. T1–T3 are deterministic formatting variants. T0 is the original narrative and does not share the same complete information set; T4 includes fields absent from the prose list. The core preserves at most eight numeric output metrics, and redaction/title cleaning is a convenience rule, not completed anonymity review. No AI model has produced an evaluated intervention, no expert has certified complete factual equivalence, and no evaluator outcome has been measured. A common hash and unchanged numeric tokens do not prove source truth, semantic equivalence or absence of misleading prose.

**Scope.** The code previously set every sensitive-exclusion flag false and included a configuration option for sensitive exclusions; it did not implement a completed civilian/sensitivity review. Do not cite the resulting “civilian-scope” count as a verified screening result. A public appendix should expose only selected necessary fields; local raw snapshots and rich treatment cores are not automatically appropriate public exports.

## 6. A feasible study with a clear deliverable

Start with one civilian sector and an explicit list of eligible project records. Collect the full frame or sample randomly within documented strata, retaining missingness. For a feasible pilot, build a source ledger for each selected project: claim, value, unit, date, source field/document and reviewer decision. A second reviewer resolves uncertain costs, outcomes and beneficiaries. This produces **verified information sheets**, not an assertion that the projects are substantively good investments.

Create a plain and an AI-assisted presentation from the **same verified facts**, with the same missing information and comparable length. Use expert checks to reject added claims, changed certainty, altered units or omitted caveats before evaluation. Randomly assign versions so a reviewer does not see both versions of the same project. Predefine a small rubric with outcomes such as comprehension accuracy, time to locate key facts, perceived readiness, and recommendation under a clearly hypothetical budget. Keep real grant success outside the primary claim unless actual program application/decision data and time ordering are obtained.

The resulting contribution could be an auditable project-data protocol and evidence about whether AI-assisted presentation improves understanding or shifts assessments while factual content is fixed. An improvement, a null effect or evidence of misleading confidence are all useful outcomes. Claiming a first-ever study requires a separate literature review; public data feasibility alone does not establish novelty.

## 7. Reproduce the local checks

```sh
.research-venv/bin/pytest -q tests/research
.research-venv/bin/python packages/research/build_pilot.py
```

Inputs: `data/raw/dream/2026-08-12T133338Z_pilot.json`, `data/raw/digital-capacity/2026-08-12T133338Z_portal.json`, and `data/raw/community-controls/2026-08-12T133845Z_matched.json`. Rebuilt outputs: `data/normalized/pilot-0.1.0/` and `data/analysis/pilot-0.1.0/`. The build selects the newest matching snapshots; pin these exact files for reproduction after any future harvest. Build timestamps change on regeneration. Raw source values were not rewritten in this audit.