ci: harden decomp workflow gates
This commit is contained in:
20
.github/workflows/build.yml
vendored
20
.github/workflows/build.yml
vendored
@@ -4,6 +4,12 @@ on:
|
||||
push:
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
require_complete:
|
||||
description: Fail unless 43U code, data, functions, units, and links are all 100%
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
|
||||
jobs:
|
||||
build:
|
||||
@@ -56,6 +62,9 @@ jobs:
|
||||
--binutils /binutils --compilers /compilers
|
||||
ninja all_source progress build/${{ matrix.version }}/report.json
|
||||
|
||||
- name: Test workflow guards
|
||||
run: python tools/test_workflow_guards.py
|
||||
|
||||
- name: Calculate decomp status
|
||||
if: matrix.version == '43U'
|
||||
run: |
|
||||
@@ -64,8 +73,16 @@ jobs:
|
||||
--output build/${{ matrix.version }}/decomp-status.json \
|
||||
--summary "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Enforce completion gate
|
||||
if: matrix.version == '43U' && github.event_name == 'workflow_dispatch' && inputs.require_complete == true
|
||||
run: |
|
||||
python tools/check_decomp_complete.py \
|
||||
build/${{ matrix.version }}/report.json \
|
||||
--dol build/${{ matrix.version }}/main.dol
|
||||
|
||||
# Upload map files
|
||||
- name: Upload map
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ${{ matrix.version }}_maps
|
||||
@@ -73,13 +90,14 @@ jobs:
|
||||
|
||||
# Upload progress report
|
||||
- name: Upload report
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ${{ matrix.version }}_report
|
||||
path: build/${{ matrix.version }}/report.json
|
||||
|
||||
- name: Upload decomp status
|
||||
if: matrix.version == '43U'
|
||||
if: always() && matrix.version == '43U'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: decomp_status
|
||||
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -6,6 +6,7 @@
|
||||
__pycache__
|
||||
.mypy_cache
|
||||
.cache/
|
||||
.unlazy/
|
||||
|
||||
# Original files
|
||||
orig/*/*
|
||||
|
||||
76
AGENTS.md
76
AGENTS.md
@@ -14,7 +14,8 @@ at all.
|
||||
|
||||
Concretely:
|
||||
|
||||
- `origin` is this fork, `coah80/wii-ipl`. Push here.
|
||||
- `origin` is this fork, `coah80/wii-ipl`. Only the orchestrator pushes
|
||||
validated leaf branches here.
|
||||
- `upstream` is `koopthekoopa/wii-ipl`. Its push URL is intentionally set to
|
||||
the sentinel `DISABLED_never_push_to_upstream` so an accidental
|
||||
`git push upstream` fails instead of uploading.
|
||||
@@ -22,7 +23,8 @@ Concretely:
|
||||
`--repo koopthekoopa/wii-ipl` to any `gh` command.
|
||||
- If a task appears to require contacting upstream, stop and ask the human.
|
||||
|
||||
If you are reading this because you are about to open a PR: don't.
|
||||
If you are reading this because you are about to open a PR against upstream:
|
||||
don't. PRs for validated leaf branches belong only to this fork.
|
||||
|
||||
## What this repository is
|
||||
|
||||
@@ -79,6 +81,23 @@ Helper tools live in `tools/decomp-assist/`:
|
||||
- `export_pyghidra.py` — regenerates `ghidra_decomp.txt`, decompiled C for the
|
||||
functions still missing.
|
||||
|
||||
The final completion assertion is machine-checkable:
|
||||
|
||||
```
|
||||
python3 tools/check_decomp_complete.py build/43U/report.json \
|
||||
--dol build/43U/main.dol
|
||||
```
|
||||
|
||||
It must pass before claiming full completion. GitHub Actions runs the same
|
||||
assertion when manually dispatched with `require_complete=true`.
|
||||
Normal push and pull-request builds publish progress without failing on the
|
||||
current incomplete state. Use the manual `require_complete=true` dispatch for
|
||||
the terminal all-100% gate.
|
||||
|
||||
`tools/test_workflow_guards.py` exercises the completion checker, and
|
||||
`tools/worktree_check.py` validates the main, leaf, and cleanup states before
|
||||
integration.
|
||||
|
||||
## The single most useful finding: the string pool
|
||||
|
||||
MWCC emits a translation unit's string literals into `.data` in the order the
|
||||
@@ -155,6 +174,14 @@ Codex login rather than an API key; API billing is not created by ReAgent, but
|
||||
the account's normal model or plan usage still applies. Do not start a model
|
||||
run until `doctor` reports the backend and acceptance configuration clearly.
|
||||
|
||||
If the launcher cannot import, or `doctor` reports `ready: false`, record the
|
||||
exact failure and do not claim ReAgent evidence. Repair a broken editable
|
||||
install with a fresh checkout and a non-editable install before retrying. If
|
||||
the decompile capability still lacks Java/Ghidra, workers may continue only in
|
||||
the explicitly labeled manual/objdiff fallback using fresh object builds,
|
||||
`pool_diff.py`, `ctxdiff.py`, full 43U builds, and the DOL hash. The fallback
|
||||
does not count as ReAgent execution.
|
||||
|
||||
For parallel work, orchestrate up to ten workers at once when the agent runtime
|
||||
has the capacity. Use `gpt-5.6-luna` at maximum reasoning effort with the fast
|
||||
service tier when those settings are available. If the runtime cannot host ten
|
||||
@@ -168,13 +195,16 @@ artificial assembly.
|
||||
### Ten-worker exact-match loop
|
||||
|
||||
The worker keeps iterating on its assigned function until exact-name `objdiff`
|
||||
reports `100.0%`. A worker must not stop at a fuzzy score, and must not commit
|
||||
or push to a remote. If the worker believes the remaining difference is a compiler
|
||||
tie-break, it reports the evidence and waits for the orchestrator to terminate
|
||||
or reassign the leaf; it does not self-accept a near-match. Reaching `100.0%`
|
||||
is only the handoff point; it is not acceptance.
|
||||
reports `100.0%`. A worker must not stop at a fuzzy score, push to a remote,
|
||||
open a PR, or merge. It may make a local commit on its isolated leaf branch if
|
||||
needed to preserve its work. If the worker believes the remaining difference is
|
||||
a compiler tie-break, it reports the evidence and waits for the orchestrator to
|
||||
terminate or reassign the leaf; it does not self-accept a near-match. Reaching
|
||||
`100.0%` is only the handoff point; it is not acceptance.
|
||||
|
||||
Workers are persistent across validation. After reaching a candidate result,
|
||||
Worker measurements are advisory. Only a fresh orchestrator verification can
|
||||
accept, push, open, or merge a candidate. Workers are persistent across
|
||||
validation. After reaching a candidate result,
|
||||
the worker enters a paused or awaiting-parent state while the orchestrator
|
||||
performs its checks. Keep the worker handle and its leaf assignment alive. If
|
||||
pool, ctxdiff, source review, or any other gate fails, send the failure evidence
|
||||
@@ -194,10 +224,11 @@ The orchestrator must independently rebuild the owned object and verify:
|
||||
5. The full 4.3U build passes and `build/43U/main.dol` has SHA1
|
||||
`26116613f624061ba99c8d1a299aaa6efa85670d`.
|
||||
|
||||
If any orchestrator gate fails, do not commit the candidate. Restore it or send
|
||||
the same disjoint task back to its worker with the failing evidence. Commit and
|
||||
stage the candidate branch for a PR after every candidate passes all gates, then
|
||||
regenerate the live progress report before accepting the next candidate.
|
||||
If any orchestrator gate fails, do not promote, push, open a PR, or merge the
|
||||
candidate. Restore it or send the same disjoint task back to its worker with
|
||||
the failing evidence. Stage the candidate branch for a PR only after every
|
||||
gate passes, then regenerate the live progress report before accepting the
|
||||
next candidate.
|
||||
|
||||
For each candidate, the worker must:
|
||||
|
||||
@@ -211,7 +242,7 @@ For each candidate, the worker must:
|
||||
build/43U/ok`; the worker does not push, open a PR, or merge.
|
||||
|
||||
The main agent reviews every worker result before accepting it. After each
|
||||
accepted commit, record the new report from `build/43U/report.json` and keep
|
||||
accepted merge, record the new report from `build/43U/report.json` and keep
|
||||
the DOL hash at `26116613f624061ba99c8d1a299aaa6efa85670d`. Never push to
|
||||
`upstream`; only the fork's `origin` is in scope.
|
||||
|
||||
@@ -222,9 +253,10 @@ All source-level reverse engineering, implementation, compiler experiments,
|
||||
and match iteration belong to the persistent subagents. The orchestrator may
|
||||
only select disjoint leaves, start and pause workers, resume a worker with
|
||||
failure evidence, run independent verification commands, review diffs, record
|
||||
wave state, commit accepted worker changes, push `origin/main`, and publish
|
||||
progress. If a candidate fails validation, return it to its worker; do not fix
|
||||
the candidate inline in the orchestrator.
|
||||
wave state, commit validated leaf-branch changes, push the leaf branch, open
|
||||
and merge the PR, fast-forward `origin/main`, and publish progress. If a
|
||||
candidate fails validation, return it to its worker; do not fix the candidate
|
||||
inline in the orchestrator.
|
||||
|
||||
A new thread must begin by reading this file, checking the live 43U report and
|
||||
remote branch, running the ReAgent doctor/status checks, and then dispatching
|
||||
@@ -239,6 +271,18 @@ worktree, branch, or source file. A worker may commit locally on its leaf branch
|
||||
if that is needed to preserve its exact result, but it must not push, open a PR,
|
||||
or merge.
|
||||
|
||||
Use the repository worktree checker before integration and cleanup:
|
||||
|
||||
```
|
||||
python3 tools/worktree_check.py main --synced
|
||||
python3 tools/worktree_check.py leaf <leaf-worktree> agent/<wave>/<leaf>
|
||||
python3 tools/worktree_check.py cleanup <leaf-worktree> agent/<wave>/<leaf>
|
||||
```
|
||||
|
||||
The cleanup check is read-only. It succeeds only for a clean leaf branch that
|
||||
is already an ancestor of `origin/main`; the orchestrator then performs the
|
||||
explicit `git worktree remove` and local branch deletion.
|
||||
|
||||
After the worker reaches its handoff point, the orchestrator validates the
|
||||
worker's worktree from a fresh build. If every gate passes, the orchestrator
|
||||
pushes that leaf branch to the fork, opens a PR against `coah80/wii-ipl:main`,
|
||||
|
||||
298
tools/check_decomp_complete.py
Normal file
298
tools/check_decomp_complete.py
Normal file
@@ -0,0 +1,298 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
EXPECTED_DOL_SHA1 = "26116613f624061ba99c8d1a299aaa6efa85670d"
|
||||
EXPECTED_TOTAL_UNITS = 1026
|
||||
EXPECTED_TOTAL_CODE = 2995172
|
||||
EXPECTED_TOTAL_DATA = 1832684
|
||||
EXPECTED_TOTAL_FUNCTIONS = 12563
|
||||
INTEGER_PATTERN = re.compile(r"-?(0|[1-9][0-9]*)\Z")
|
||||
CODE_SECTIONS = frozenset({".text", ".init"})
|
||||
AGGREGATE_FIELDS = (
|
||||
"total_units",
|
||||
"total_code",
|
||||
"total_data",
|
||||
"total_functions",
|
||||
"matched_code",
|
||||
"complete_code",
|
||||
"matched_data",
|
||||
"complete_data",
|
||||
"matched_functions",
|
||||
"complete_units",
|
||||
)
|
||||
|
||||
|
||||
def integer(value):
|
||||
if isinstance(value, bool):
|
||||
raise ValueError("boolean is not an integer")
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
if isinstance(value, str) and INTEGER_PATTERN.fullmatch(value):
|
||||
return int(value)
|
||||
raise ValueError(f"invalid integer: {value!r}")
|
||||
|
||||
|
||||
def exact(value):
|
||||
return (
|
||||
isinstance(value, (int, float))
|
||||
and not isinstance(value, bool)
|
||||
and math.isfinite(float(value))
|
||||
and float(value) == 100.0
|
||||
)
|
||||
|
||||
|
||||
def required(measures, field, label, failures):
|
||||
if field not in measures or measures[field] is None:
|
||||
failures.append(f"{label}: missing {field}")
|
||||
return None
|
||||
return measures[field]
|
||||
|
||||
|
||||
def count(measures, field, label, failures):
|
||||
value = required(measures, field, label, failures)
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
parsed = integer(value)
|
||||
except ValueError:
|
||||
failures.append(f"{label}: invalid integer {value!r}")
|
||||
return None
|
||||
if parsed < 0:
|
||||
failures.append(f"{label}: invalid non-negative integer {value!r}")
|
||||
return None
|
||||
return parsed
|
||||
|
||||
|
||||
def equal(measures, matched, total, label, failures):
|
||||
matched_value = count(measures, matched, label, failures)
|
||||
total_value = count(measures, total, label, failures)
|
||||
if matched_value is None or total_value is None:
|
||||
return
|
||||
if matched_value != total_value:
|
||||
failures.append(f"{label}: {matched_value} / {total_value}")
|
||||
|
||||
|
||||
def percent(measures, field, label, failures):
|
||||
value = required(measures, field, label, failures)
|
||||
if value is not None and not exact(value):
|
||||
failures.append(f"{label}: {value}")
|
||||
|
||||
|
||||
def check_complete_measures(measures, label, failures):
|
||||
for matched, total, name in (
|
||||
("matched_code", "total_code", "code match"),
|
||||
("complete_code", "total_code", "code link"),
|
||||
("matched_data", "total_data", "data match"),
|
||||
("complete_data", "total_data", "data link"),
|
||||
("matched_functions", "total_functions", "function match"),
|
||||
("complete_units", "total_units", "unit link"),
|
||||
):
|
||||
equal(measures, matched, total, f"{label} {name}", failures)
|
||||
for field in (
|
||||
"matched_code_percent",
|
||||
"complete_code_percent",
|
||||
"matched_data_percent",
|
||||
"complete_data_percent",
|
||||
"matched_functions_percent",
|
||||
"fuzzy_match_percent",
|
||||
):
|
||||
percent(measures, field, f"{label} {field}", failures)
|
||||
|
||||
|
||||
def key_for(value):
|
||||
return type(value).__name__, repr(value)
|
||||
|
||||
|
||||
def check_report(report):
|
||||
failures = []
|
||||
if not isinstance(report, dict):
|
||||
return ["report root is not an object"]
|
||||
if report.get("version") != 2:
|
||||
failures.append(f"report version: {report.get('version')}")
|
||||
|
||||
measures = report.get("measures", {})
|
||||
if not isinstance(measures, dict):
|
||||
failures.append("overall: missing measures")
|
||||
measures = {}
|
||||
expected_totals = {
|
||||
"total_units": EXPECTED_TOTAL_UNITS,
|
||||
"total_code": EXPECTED_TOTAL_CODE,
|
||||
"total_data": EXPECTED_TOTAL_DATA,
|
||||
"total_functions": EXPECTED_TOTAL_FUNCTIONS,
|
||||
}
|
||||
for field, expected in expected_totals.items():
|
||||
value = count(measures, field, "overall", failures)
|
||||
if value is not None and value != expected:
|
||||
failures.append(f"overall {field}: {value} != {expected}")
|
||||
check_complete_measures(measures, "overall", failures)
|
||||
|
||||
units = report.get("units")
|
||||
if not isinstance(units, list) or len(units) != EXPECTED_TOTAL_UNITS:
|
||||
failures.append(f"unit report length: {len(units) if isinstance(units, list) else 'missing'}")
|
||||
units = units if isinstance(units, list) else []
|
||||
names = [
|
||||
unit.get("name", "<unnamed>") if isinstance(unit, dict) else "<invalid>"
|
||||
for unit in units
|
||||
]
|
||||
if len({key_for(name) for name in names}) != len(names):
|
||||
failures.append("unit names are not unique")
|
||||
|
||||
unit_sums = {field: 0 for field in AGGREGATE_FIELDS}
|
||||
for unit in units:
|
||||
if not isinstance(unit, dict):
|
||||
failures.append("invalid unit entry")
|
||||
continue
|
||||
name = unit.get("name", "<unnamed>")
|
||||
unit_measures = unit.get("measures")
|
||||
if not isinstance(unit_measures, dict):
|
||||
failures.append(f"{name}: missing measures")
|
||||
continue
|
||||
for field in AGGREGATE_FIELDS:
|
||||
if field in unit_measures:
|
||||
value = count(unit_measures, field, name, failures)
|
||||
if value is not None:
|
||||
unit_sums[field] += value
|
||||
metadata = unit.get("metadata")
|
||||
if not isinstance(metadata, dict) or metadata.get("complete") is not True:
|
||||
failures.append(f"unit incomplete: {name}")
|
||||
|
||||
total_units = count(unit_measures, "total_units", name, failures)
|
||||
if total_units is not None and total_units != 1:
|
||||
failures.append(f"{name} total_units: {total_units}")
|
||||
equal(unit_measures, "complete_units", "total_units", f"{name} unit link", failures)
|
||||
|
||||
has_total_code = "total_code" in unit_measures
|
||||
has_total_data = "total_data" in unit_measures
|
||||
has_total_functions = "total_functions" in unit_measures
|
||||
total_functions = count(unit_measures, "total_functions", name, failures) if has_total_functions else None
|
||||
if not has_total_code and not has_total_data:
|
||||
failures.append(f"{name}: missing code and data totals")
|
||||
if has_total_code:
|
||||
equal(unit_measures, "matched_code", "total_code", f"{name} code match", failures)
|
||||
equal(unit_measures, "complete_code", "total_code", f"{name} code link", failures)
|
||||
percent(unit_measures, "matched_code_percent", f"{name} code percent", failures)
|
||||
percent(unit_measures, "complete_code_percent", f"{name} code link percent", failures)
|
||||
percent(unit_measures, "fuzzy_match_percent", f"{name} fuzzy percent", failures)
|
||||
if has_total_data:
|
||||
equal(unit_measures, "matched_data", "total_data", f"{name} data match", failures)
|
||||
equal(unit_measures, "complete_data", "total_data", f"{name} data link", failures)
|
||||
percent(unit_measures, "matched_data_percent", f"{name} data percent", failures)
|
||||
percent(unit_measures, "complete_data_percent", f"{name} data link percent", failures)
|
||||
if has_total_functions:
|
||||
equal(unit_measures, "matched_functions", "total_functions", f"{name} function match", failures)
|
||||
percent(unit_measures, "matched_functions_percent", f"{name} function percent", failures)
|
||||
|
||||
sections = unit.get("sections")
|
||||
if not isinstance(sections, list):
|
||||
failures.append(f"{name}: missing sections")
|
||||
else:
|
||||
for section in sections:
|
||||
if not isinstance(section, dict):
|
||||
failures.append(f"{name}: invalid section entry")
|
||||
continue
|
||||
section_name = section.get("name", "<unnamed>")
|
||||
if "fuzzy_match_percent" in section:
|
||||
percent(section, "fuzzy_match_percent", f"{name} section {section_name}", failures)
|
||||
elif isinstance(section_name, str) and section_name in CODE_SECTIONS:
|
||||
failures.append(f"{name} section {section_name}: missing fuzzy_match_percent")
|
||||
|
||||
functions = unit.get("functions", [])
|
||||
if total_functions is not None and total_functions > 0:
|
||||
if not isinstance(functions, list) or len(functions) != total_functions:
|
||||
failures.append(f"{name} function list: {len(functions) if isinstance(functions, list) else 'missing'} / {total_functions}")
|
||||
else:
|
||||
for function in functions:
|
||||
if not isinstance(function, dict):
|
||||
failures.append(f"{name}: invalid function entry")
|
||||
continue
|
||||
percent(function, "fuzzy_match_percent", f"{name} function {function.get('name', '<unnamed>')}", failures)
|
||||
elif has_total_functions and total_functions == 0 and functions:
|
||||
failures.append(f"{name}: unexpected functions")
|
||||
elif not has_total_functions and functions:
|
||||
failures.append(f"{name}: unexpected functions")
|
||||
|
||||
for field in AGGREGATE_FIELDS:
|
||||
expected = count(measures, field, "overall", failures)
|
||||
if expected is not None and unit_sums[field] != expected:
|
||||
failures.append(f"unit aggregate {field}: {unit_sums[field]} != {expected}")
|
||||
|
||||
categories = report.get("categories")
|
||||
if not isinstance(categories, list) or not categories:
|
||||
failures.append("missing categories")
|
||||
categories = categories if isinstance(categories, list) else []
|
||||
category_names = [
|
||||
category.get("name", "<unnamed>") if isinstance(category, dict) else "<invalid>"
|
||||
for category in categories
|
||||
]
|
||||
if len({key_for(name) for name in category_names}) != len(category_names):
|
||||
failures.append("category names are not unique")
|
||||
|
||||
category_sums = {field: 0 for field in AGGREGATE_FIELDS}
|
||||
for category in categories:
|
||||
if not isinstance(category, dict):
|
||||
failures.append("invalid category entry")
|
||||
continue
|
||||
name = category.get("name", "<unnamed>")
|
||||
category_measures = category.get("measures")
|
||||
if not isinstance(category_measures, dict):
|
||||
failures.append(f"{name}: missing measures")
|
||||
continue
|
||||
check_complete_measures(category_measures, name, failures)
|
||||
for field in AGGREGATE_FIELDS:
|
||||
if field in category_measures:
|
||||
value = count(category_measures, field, name, failures)
|
||||
if value is not None:
|
||||
category_sums[field] += value
|
||||
for field, expected in expected_totals.items():
|
||||
if category_sums[field] != expected:
|
||||
failures.append(f"category {field}: {category_sums[field]} != {expected}")
|
||||
for field in AGGREGATE_FIELDS:
|
||||
expected = count(measures, field, "overall", failures)
|
||||
if expected is not None and category_sums[field] != expected:
|
||||
failures.append(f"category aggregate {field}: {category_sums[field]} != {expected}")
|
||||
return failures
|
||||
|
||||
|
||||
def check_dol(path):
|
||||
try:
|
||||
digest = hashlib.sha1(path.read_bytes()).hexdigest()
|
||||
except OSError as error:
|
||||
return [f"DOL read failed: {error}"]
|
||||
if digest != EXPECTED_DOL_SHA1:
|
||||
return [f"DOL SHA1: {digest} != {EXPECTED_DOL_SHA1}"]
|
||||
return []
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("report", type=Path)
|
||||
parser.add_argument("--dol", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
report = json.loads(args.report.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as error:
|
||||
print("DECOMPLETE_FAIL")
|
||||
print(f"- report read failed: {error}")
|
||||
return 1
|
||||
failures = check_report(report)
|
||||
failures.extend(check_dol(args.dol))
|
||||
if failures:
|
||||
print("DECOMPLETE_FAIL")
|
||||
for failure in failures[:40]:
|
||||
print(f"- {failure}")
|
||||
if len(failures) > 40:
|
||||
print(f"- ... {len(failures) - 40} more failures")
|
||||
return 1
|
||||
print("DECOMPLETE_OK")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
122
tools/test_workflow_guards.py
Normal file
122
tools/test_workflow_guards.py
Normal file
@@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env python3
|
||||
import copy
|
||||
import json
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from check_decomp_complete import check_dol, check_report, exact
|
||||
|
||||
|
||||
class WorkflowGuardTests(unittest.TestCase):
|
||||
def test_exact_percent_is_strict(self):
|
||||
self.assertTrue(exact(100.0))
|
||||
self.assertFalse(exact(99.9999995))
|
||||
|
||||
def test_empty_report_fails_expected_schema(self):
|
||||
failures = check_report({"version": 2, "measures": {}, "units": [], "categories": []})
|
||||
self.assertTrue(any("total_units" in failure for failure in failures))
|
||||
self.assertTrue(any("missing categories" in failure for failure in failures))
|
||||
|
||||
def test_malformed_report_values_fail_without_crashing(self):
|
||||
failures = check_report(
|
||||
{
|
||||
"version": 2,
|
||||
"measures": {"total_units": 1.5},
|
||||
"units": [None],
|
||||
"categories": [None],
|
||||
}
|
||||
)
|
||||
self.assertTrue(any("invalid integer" in failure for failure in failures))
|
||||
self.assertTrue(any("invalid unit entry" in failure for failure in failures))
|
||||
self.assertTrue(any("invalid category entry" in failure for failure in failures))
|
||||
failures = check_report({"version": 2, "measures": {"total_units": -1}})
|
||||
self.assertTrue(any("invalid non-negative integer" in failure for failure in failures))
|
||||
|
||||
@unittest.skipUnless(
|
||||
(Path(__file__).resolve().parents[1] / "build/43U/report.json").is_file(),
|
||||
"generated 43U report is not available",
|
||||
)
|
||||
def test_data_section_without_code_score_is_allowed(self):
|
||||
report_path = Path(__file__).resolve().parents[1] / "build/43U/report.json"
|
||||
report = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
failures = check_report(report)
|
||||
self.assertFalse(any("enc_dummy section .sbss" in failure for failure in failures))
|
||||
|
||||
@unittest.skipUnless(
|
||||
(Path(__file__).resolve().parents[1] / "build/43U/report.json").is_file(),
|
||||
"generated 43U report is not available",
|
||||
)
|
||||
def test_complete_report_schema_passes(self):
|
||||
report_path = Path(__file__).resolve().parents[1] / "build/43U/report.json"
|
||||
report = copy.deepcopy(json.loads(report_path.read_text(encoding="utf-8")))
|
||||
|
||||
def fill(measures):
|
||||
for total, matched, linked in (
|
||||
("total_code", "matched_code", "complete_code"),
|
||||
("total_data", "matched_data", "complete_data"),
|
||||
("total_functions", "matched_functions", None),
|
||||
("total_units", None, "complete_units"),
|
||||
):
|
||||
if total in measures:
|
||||
if matched:
|
||||
measures[matched] = measures[total]
|
||||
if linked:
|
||||
measures[linked] = measures[total]
|
||||
for field in (
|
||||
"matched_code_percent",
|
||||
"complete_code_percent",
|
||||
"matched_data_percent",
|
||||
"complete_data_percent",
|
||||
"matched_functions_percent",
|
||||
"fuzzy_match_percent",
|
||||
):
|
||||
if field.endswith("data_percent") and "total_data" not in measures:
|
||||
continue
|
||||
if field.endswith("code_percent") and "total_code" not in measures:
|
||||
continue
|
||||
if field == "matched_functions_percent" and "total_functions" not in measures:
|
||||
continue
|
||||
if field == "fuzzy_match_percent" and "total_code" not in measures:
|
||||
continue
|
||||
measures[field] = 100.0
|
||||
|
||||
fill(report["measures"])
|
||||
for unit in report["units"]:
|
||||
unit["metadata"]["complete"] = True
|
||||
fill(unit["measures"])
|
||||
for section in unit["sections"]:
|
||||
section["fuzzy_match_percent"] = 100.0
|
||||
for function in unit.get("functions", []):
|
||||
function["fuzzy_match_percent"] = 100.0
|
||||
for category in report["categories"]:
|
||||
fill(category["measures"])
|
||||
self.assertEqual(check_report(report), [])
|
||||
|
||||
@unittest.skipUnless(
|
||||
(Path(__file__).resolve().parents[1] / "build/43U/report.json").is_file(),
|
||||
"generated 43U report is not available",
|
||||
)
|
||||
def test_missing_unit_data_does_not_pass(self):
|
||||
report_path = Path(__file__).resolve().parents[1] / "build/43U/report.json"
|
||||
report = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
unit = next(unit for unit in report["units"] if unit["name"] == "main/src/system/enc_dummy")
|
||||
forged = copy.deepcopy(report)
|
||||
forged_unit = next(item for item in forged["units"] if item["name"] == unit["name"])
|
||||
forged_unit["metadata"]["complete"] = True
|
||||
forged_unit["measures"].pop("matched_data", None)
|
||||
failures = check_report(forged)
|
||||
self.assertTrue(any("enc_dummy data match" in failure for failure in failures))
|
||||
|
||||
def test_dol_hash_is_checked(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "main.dol"
|
||||
path.write_bytes(b"not the Wii Menu DOL")
|
||||
self.assertTrue(check_dol(path))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
157
tools/worktree_check.py
Normal file
157
tools/worktree_check.py
Normal file
@@ -0,0 +1,157 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def git(*args, cwd=None, check=True):
|
||||
result = subprocess.run(
|
||||
["git", *args],
|
||||
cwd=cwd,
|
||||
check=check,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
return result.stdout.strip(), result.stderr.strip(), result.returncode
|
||||
|
||||
|
||||
def records(root):
|
||||
output, _, _ = git("worktree", "list", "--porcelain", cwd=root)
|
||||
result = []
|
||||
current = {}
|
||||
for line in output.splitlines() + [""]:
|
||||
if not line:
|
||||
if current:
|
||||
result.append(current)
|
||||
current = {}
|
||||
continue
|
||||
key, _, value = line.partition(" ")
|
||||
if key == "worktree":
|
||||
current["path"] = str(Path(value).resolve())
|
||||
elif key == "branch":
|
||||
current["branch"] = value.removeprefix("refs/heads/")
|
||||
elif key == "HEAD":
|
||||
current["head"] = value
|
||||
return result
|
||||
|
||||
|
||||
def record_for(items, path):
|
||||
target = str(Path(path).resolve())
|
||||
return next((item for item in items if item.get("path") == target), None)
|
||||
|
||||
|
||||
def clean(path):
|
||||
output, _, _ = git("status", "--porcelain", "--untracked-files=all", cwd=path)
|
||||
return output == ""
|
||||
|
||||
|
||||
def integration_root(root):
|
||||
item = next((item for item in records(root) if item.get("branch") == "main"), None)
|
||||
return Path(item["path"]) if item else None
|
||||
|
||||
|
||||
def is_fork_url(value):
|
||||
normalized = value.rstrip("/")
|
||||
if normalized.endswith(".git"):
|
||||
normalized = normalized[:-4]
|
||||
return normalized in {
|
||||
"https://github.com/coah80/wii-ipl",
|
||||
"git@github.com:coah80/wii-ipl",
|
||||
"ssh://git@github.com/coah80/wii-ipl",
|
||||
}
|
||||
|
||||
|
||||
def main_check(args):
|
||||
current_root = Path(git("rev-parse", "--show-toplevel")[0]).resolve()
|
||||
root = integration_root(current_root)
|
||||
if root is None:
|
||||
return "no registered main integration worktree"
|
||||
if current_root != root:
|
||||
return "run the main check from the integration worktree"
|
||||
if git("branch", "--show-current", cwd=root)[0] != "main":
|
||||
return "main worktree is not on main"
|
||||
remote, _, code = git("remote", "get-url", "origin", cwd=root, check=False)
|
||||
if code or not is_fork_url(remote):
|
||||
return "origin is not coah80/wii-ipl"
|
||||
push_url, _, code = git("remote", "get-url", "--push", "upstream", cwd=root, check=False)
|
||||
if code != 0:
|
||||
return "upstream remote is missing"
|
||||
if "DISABLED_never_push_to_upstream" not in push_url:
|
||||
return "upstream push URL is not disabled"
|
||||
if not clean(root):
|
||||
return "main worktree is dirty"
|
||||
prune, _, _ = git("worktree", "prune", "--dry-run", cwd=root)
|
||||
if prune:
|
||||
return f"orphaned worktree metadata: {prune}"
|
||||
if args.synced and git("rev-parse", "HEAD", cwd=root)[0] != git("rev-parse", "origin/main", cwd=root)[0]:
|
||||
return "main is not synchronized with origin/main"
|
||||
return None
|
||||
|
||||
|
||||
def leaf_check(args):
|
||||
root = Path(git("rev-parse", "--show-toplevel")[0]).resolve()
|
||||
item = record_for(records(root), args.path)
|
||||
if not item:
|
||||
return "leaf path is not registered as a worktree"
|
||||
if item.get("branch") != args.branch:
|
||||
return "leaf branch does not match the worktree record"
|
||||
if not args.branch.startswith("agent/"):
|
||||
return "leaf branch must use the agent/ prefix"
|
||||
if args.path.resolve() == root:
|
||||
return "leaf path is the integration worktree"
|
||||
return None
|
||||
|
||||
|
||||
def cleanup_check(args):
|
||||
root = Path(git("rev-parse", "--show-toplevel")[0]).resolve()
|
||||
item = record_for(records(root), args.path)
|
||||
if not item:
|
||||
return "leaf path is not registered as a worktree"
|
||||
if args.path.resolve() == root:
|
||||
return "integration worktree cannot be cleaned up"
|
||||
if not item.get("branch"):
|
||||
return "detached worktree cannot be cleaned up"
|
||||
if not item["branch"].startswith("agent/"):
|
||||
return "only agent branches can be cleaned up"
|
||||
if item.get("branch") != args.branch:
|
||||
return "leaf branch does not match the worktree record"
|
||||
if args.merged_ref != "origin/main":
|
||||
return "cleanup must verify ancestry against origin/main"
|
||||
if not clean(args.path):
|
||||
return "leaf worktree is dirty"
|
||||
_, _, code = git("merge-base", "--is-ancestor", args.branch, args.merged_ref, cwd=root, check=False)
|
||||
if code != 0:
|
||||
return f"leaf branch is not merged into {args.merged_ref}"
|
||||
return None
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
subparsers = parser.add_subparsers(dest="mode", required=True)
|
||||
main_parser = subparsers.add_parser("main")
|
||||
main_parser.add_argument("--synced", action="store_true")
|
||||
leaf_parser = subparsers.add_parser("leaf")
|
||||
leaf_parser.add_argument("path", type=Path)
|
||||
leaf_parser.add_argument("branch")
|
||||
cleanup_parser = subparsers.add_parser("cleanup")
|
||||
cleanup_parser.add_argument("path", type=Path)
|
||||
cleanup_parser.add_argument("branch")
|
||||
cleanup_parser.add_argument("--merged-ref", default="origin/main")
|
||||
args = parser.parse_args()
|
||||
if args.mode == "main":
|
||||
failure = main_check(args)
|
||||
elif args.mode == "leaf":
|
||||
failure = leaf_check(args)
|
||||
else:
|
||||
failure = cleanup_check(args)
|
||||
if failure:
|
||||
print(f"WORKTREE_CHECK_FAIL: {failure}")
|
||||
return 1
|
||||
print("WORKTREE_CHECK_OK")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user