Compare commits

...

1 Commits

Author SHA1 Message Date
coah
f6f77bdbeb tools: audit direct call destinations against original DOL 2026-10-02 21:04:01 -05:00
3 changed files with 802 additions and 0 deletions

View File

@@ -0,0 +1,103 @@
# Direct-call destination advisory
`call_target_diff.py` checks the callees of functions reported as objdiff100.
A 100% instruction score can still conceal a different relocation target. This
tool compares resolved destination addresses with branches decoded independently
from the original 43U DOL. It does not change source, build outputs, configuration,
CI, or the completion policy.
Requires Python and pyelftools, as do the neighboring ELF tools. Run from the
repository root after preparing the normal 43U objects and report:
```sh
python tools/decomp-assist/call_target_diff.py --output /tmp/call-targets.json
python tools/decomp-assist/call_target_diff.py src/scene/setting/iplSetting
python tools/decomp-assist/call_target_diff.py UNIT --function EXACT_SYMBOL
python -m unittest discover -s tools/decomp-assist -p test_call_target_diff.py
```
With no unit arguments, it scans the report. Unit names omit the `main/` prefix
and `.o` suffix. `--function` requires exactly one unit and still requires that
function to be reported100. JSON goes to stdout unless `--output` is supplied.
The output path cannot overwrite an input.
`--root` selects another prepared repository. `--report` and `--dol` override
input paths; otherwise they are `ROOT/build/43U/report.json` and
`ROOT/orig/43U/00000008.app`. Relative overrides are relative to the current
working directory. The DOL must match the known 43U SHA1 by default; `--dol-sha1`
accepts an explicitly verified alternative hash, including authored test inputs.
The configuration remains 43U.
## What it checks
Only exact-name functions with equal source/reference extents and
`fuzzy_match_percent == 100` are compared. Missing/ambiguous functions and
unequal extents are reported as skips. Direct unconditional PPC `bl` instructions
are decoded whether they have explicit `R_PPC_REL24`/`R_PPC_ADDR24` relocations or
already contain resolved branch displacements. Signed relative and absolute
encodings are supported.
Section-relative relocation targets and raw local branches resolve through
actual function extents. Aliases sharing one function start are retained;
overlapping distinct function starts remain unresolved. A matching reference
symbol or an unambiguous configured address maps the source callee into the
original address space. The original DOL supplies the reference call destination,
and reference-object disagreement is an input error. Symbol spelling alone is
not accepted as proof that two distinct callees are equivalent.
Rows classify:
- `different_destination`: source resolves to a different original address
- `unresolved_destination`: source has no unique original-address mapping
- `same_address_alias`: different names resolve to the same original address
- `call_shape_difference`: only one object has a direct call at that offset
Rows are candidates for inspection, not automatic bug reports. Different
constructor addresses can have identical bodies; a generated assignment or
anonymous wrapper can implement the same operation under another name. Optional
`alias_hint` evidence identifies identical original leaf bodies or a local
relocation-free body identical to the original callee. Hints require compatible
extents, equal bytes, and conservative branch checks; they do not suppress rows
or establish general semantic equivalence. Helpers containing relocations may
remain unresolved even when manual review can establish equivalence. No symbol
name, constructor prefix, or wrapper prefix is silently excluded.
An unresolved symbol is not necessarily a missing implementation: it may be a
renamed helper, external symbol, or ambiguous map entry. Check source definitions,
linker aliases, receiver types, and the actual callee before proposing a fix.
## Provenance and exits
JSON includes both observed Git heads, the original-DOL SHA1, and SHA-256 hashes
of the exact report, symbol/split maps, object files, tool, and shared ELF helper
read by the scan. Inputs are fingerprinted again at the end. A changed head or
input produces an error rather than a clean result. No binary or assembly bytes
are written into the report.
These fingerprints identify what was inspected. They do not prove that cached
objects were rebuilt from the stated source commit or that the report is fresh.
For a serious candidate, preserve its inputs and independently rebuild/verify
that specific commit before changing source.
Exit statuses are advisory:
- 0: no review rows, skips, or input errors in the selected scope
- 1: rows or skipped functions require review
- 2: invalid, missing, inconsistent, or changing input
An empty or entirely nonexact selection can check zero calls. Always inspect the
counts. A clean result does not establish completion, whole-program equivalence,
or correct callee bodies. Indirect calls, tail calls, conditional branch-and-link,
nonexact callers, and unequal extents are outside this pass. Runtime behavior,
callback effects, and floating-point evaluation order still require focused
review.
## Tests
Tests build small ELF32/PPC objects and DOL containers from authored instruction
words and metadata in temporary directories. They cover an incorrect relocation
destination despite identical instruction bytes, raw local branches, shifted
function locations, interior targets, address/constructor/wrapper aliases,
undefined destinations, malformed references, original-DOL disagreement,
selection skips, input changes, and JSON exit behavior. No original executable
bytes or assembly are embedded in the tool, documentation, or tests.

View File

@@ -0,0 +1,383 @@
"""Advisory direct-call destination audit for objdiff100, equal-size functions.
Run from the repository root: python tools/decomp-assist/call_target_diff.py [unit ...]
Checks REL24/ADDR24 relocations and raw PPC direct BL against the original DOL.
Exit 0: no review rows or skips; 1: review needed; 2: invalid/changing input.
This is not a completion gate or a proof of callee behavior. See call_target_diff.md.
"""
import argparse
from collections import Counter, defaultdict
from dataclasses import dataclass
import hashlib
import io
import json
from pathlib import Path
import re
import struct
import subprocess
import sys
from elftools.common.exceptions import ELFError
from elftools.elf.elffile import ELFFile
import jump_table_diff as elf
DOL_SHA1 = "26116613f624061ba99c8d1a299aaa6efa85670d"
def signed(value, bits):
return value - (1 << bits) if value & (1 << (bits - 1)) else value
def branch_target(word, address):
"""Return a PPC I-form direct BL destination, or None for other instructions."""
if word >> 26 != 18 or not word & 1:
return None
displacement = signed(word & 0x03FFFFFC, 26)
return (displacement if word & 2 else address + displacement) & 0xFFFFFFFF
class Inputs:
"""Fingerprint exact bytes read; detect changes before presenting a result."""
def __init__(self):
self.hashes = {}
def read(self, path):
path = Path(path)
raw = path.read_bytes()
digest = hashlib.sha256(raw).hexdigest()
previous = self.hashes.setdefault(str(path), digest)
if previous != digest:
raise ValueError("input changed while reading: " + str(path))
return raw
def changed(self):
return [path for path, digest in self.hashes.items()
if not Path(path).is_file()
or hashlib.sha256(Path(path).read_bytes()).hexdigest() != digest]
@dataclass
class CallObject:
info: elf.ObjectInfo
data: dict
def function(self, name):
symbol = self.info.symbol(name)
if symbol.kind != "STT_FUNC" or symbol.value % 4 or symbol.size % 4:
raise ValueError("expected an aligned function: " + name)
return symbol
def location(self, section, offset, mechanism):
functions = [s for s in self.info.symbols if s.section == section
and s.kind == "STT_FUNC" and s.value <= offset < s.value + s.size]
# Multiple names at one start are aliases; overlapping starts are ambiguous.
starts = {s.value for s in functions}
start = next(iter(starts)) if len(starts) == 1 else None
names = sorted({s.name for s in functions}) if start is not None else []
return {"mechanism": mechanism, "names": names,
"addend": offset - start if start is not None else 0,
"section": self.info.sections[section][0], "offset": offset}
def call(self, function, offset):
at = function.value + offset
word = int.from_bytes(self.data[function.section][at:at + 4], "big")
raw_target = branch_target(word, at)
if raw_target is None:
return None
relocation = self.info.relocs(function.section).get(at)
if relocation is not None:
expected = 2 if word & 2 else 10 # R_PPC_ADDR24 / R_PPC_REL24
if relocation.kind != expected:
raise ValueError("direct BL has an unsupported relocation kind")
location = self.info.location(relocation)
if location is not None:
section, value = location
if section not in self.data or value % 4 or not 0 <= value < len(self.data[section]):
raise ValueError("call relocation is outside an allocated section")
return self.location(section, value, "relocation")
symbol = self.info.symbols[relocation.symbol]
return {"mechanism": "relocation", "names": [symbol.name],
"addend": relocation.addend}
if word & 2:
return {"mechanism": "raw_absolute", "names": [], "addend": 0,
"address": raw_target}
return self.location(function.section, raw_target, "raw_relative")
def read_object(path, inputs):
raw = inputs.read(path)
info = elf.read_object(path)
inputs.read(path) # Shared ELF helper reads the path; reject a concurrent change.
parsed = ELFFile(io.BytesIO(raw))
data = {i: section.data() for i, section in enumerate(parsed.iter_sections())
if section["sh_type"] == "SHT_PROGBITS" and section["sh_flags"] & 2}
return CallObject(info, data)
class Dol:
def __init__(self, raw, expected_sha1):
self.sha1 = hashlib.sha1(raw).hexdigest()
if self.sha1 != expected_sha1:
raise ValueError("original DOL SHA1 does not match --dol-sha1")
if len(raw) < 256:
raise ValueError("truncated DOL header")
self.raw = raw
header = struct.unpack(">64I", raw[:256])
self.spans = []
for index in range(18):
offset, address, size = header[index], header[18 + index], header[36 + index]
if size:
if offset < 256 or offset + size > len(raw) or address + size > 0x100000000:
raise ValueError("DOL section is outside its file/address space")
if any(max(address, a) < min(address + size, a + n) for a, n, _ in self.spans):
raise ValueError("overlapping DOL address sections")
self.spans.append((address, size, offset))
def read(self, address, size):
matches = [self.raw[offset + address - start:offset + address - start + size]
for start, length, offset in self.spans
if start <= address and address + size <= start + length]
if size <= 0 or len(matches) != 1:
raise ValueError("address is not in one DOL section: 0x%X" % address)
return matches[0]
def call(self, address):
return branch_target(int.from_bytes(self.read(address, 4), "big"), address)
def read_maps(symbol_text, split_text):
names, sizes, splits = defaultdict(set), defaultdict(set), {}
for line in symbol_text.splitlines():
match = re.match(r"(.*?) = (\S+):0x([0-9a-fA-F]+);(.*)", line)
if not match or not re.search(r"type:(function|label)\b", match[4]):
continue
address = int(match[3], 16)
names[match[1]].add(address)
size = re.search(r"size:(0x[0-9a-fA-F]+|\d+)\b", match[4])
if "type:function" in match[4] and size:
sizes[address].add(int(size[1], 0))
unit = None
for line in split_text.splitlines():
if line and not line[0].isspace() and line.endswith(":") and line != "Sections:":
unit = line[:-1]
splits[unit] = {}
match = re.match(r"\s+(\S+)\s+start:0x([0-9a-fA-F]+)", line)
if match and unit:
splits[unit][match[1]] = int(match[2], 16)
return names, sizes, splits
def addresses(destination, reference, bases, names):
if destination is None:
return set()
if "address" in destination:
return {destination["address"]}
result = set()
for name in destination["names"]:
local = [s for s in reference.info.symbols if s.name == name
and s.kind == "STT_FUNC" and isinstance(s.section, int)]
if local:
for symbol in local:
section = reference.info.sections[symbol.section][0]
if section in bases:
result.add(bases[section] + symbol.value + destination["addend"])
elif len(names.get(name, ())) == 1:
result.update(address + destination["addend"] for address in names[name])
return result
def position_independent_leaf(raw):
"""Conservative filter for body-alias hints, never an equivalence proof."""
for offset in range(0, len(raw), 4):
word = int.from_bytes(raw[offset:offset + 4], "big")
op = word >> 26
branch_op = (word >> 1) & 1023
if op == 19 and branch_op in (16, 528) and (word & 1 or branch_op == 528):
return False # Indirect call or tail transfer.
if op in (16, 18):
displacement = signed(word & (0x03FFFFFC if op == 18 else 0xFFFC),
26 if op == 18 else 16)
if word & 3 or not 0 <= offset + displacement < len(raw):
return False # Link/absolute branches or transfers outside the body.
return bool(raw) and len(raw) % 4 == 0
def alias_hint(source, destination, source_addresses, target_address, dol, sizes):
if destination is None or target_address is None or len(sizes.get(target_address, ())) != 1:
return None
size = next(iter(sizes[target_address]))
target_body = dol.read(target_address, size)
if not position_independent_leaf(target_body):
return None
# Different known addresses can share a retail body (e.g. folded constructors).
if len(source_addresses) == 1:
address = next(iter(source_addresses))
if sizes.get(address) == {size} and dol.read(address, size) == target_body:
return "identical_original_leaf_bodies; current callee implementation not proved"
# Renamed local helpers need no guessed symbol pairing or name allowlist.
for name in destination["names"]:
try:
function = source.function(name)
except ValueError:
continue
if destination["addend"] or function.size != size:
continue
relocations = source.info.relocs(function.section)
if any(function.value <= at < function.value + size for at in relocations):
continue
body = source.data[function.section][function.value:function.value + size]
if body == target_body:
return "identical_relocation_free_local_body; alias requires review"
return None
def compare_function(source, reference, name, bases, names, sizes, dol):
src, ref = source.function(name), reference.function(name)
if src.size != ref.size:
raise ValueError("function extents differ")
section = reference.info.sections[ref.section][0]
if section not in bases:
raise ValueError("no original section base for " + section)
caller = bases[section] + ref.value
counts, rows = Counter(), []
for offset in range(0, ref.size, 4):
left, right = source.call(src, offset), reference.call(ref, offset)
expected = dol.call(caller + offset)
if left is None and right is None and expected is None:
continue
counts["calls"] += 1
for prefix, destination in (("source_", left), ("reference_", right)):
if destination:
counts[prefix + destination["mechanism"]] += 1
actual = addresses(left, reference, bases, names)
target = addresses(right, reference, bases, names)
if right and right.get("section") in bases:
target = {bases[right["section"]] + right["offset"]}
if bool(right) != (expected is not None) or (target and target != {expected}):
raise ValueError("reference ELF/DOL disagreement at +0x%X" % offset)
if left and right and expected is not None and actual == {expected}:
counts["same_destination"] += 1
if set(left["names"]) == set(right["names"]):
continue
classification = "same_address_alias"
elif not left or not right:
classification = "call_shape_difference"
elif len(actual) != 1:
classification = "unresolved_destination"
else:
classification = "different_destination"
hint = alias_hint(source, left, actual, expected, dol, sizes)
rows.append({"function": name, "offset": offset, "caller_address": caller,
"classification": classification, "source": left, "reference": right,
"source_addresses": sorted(actual), "reference_addresses": sorted(target),
"dol_destination": expected, "alias_hint": hint})
counts[classification] += 1
return counts, rows
def git_head(root):
result = subprocess.run(["git", "-C", str(root), "rev-parse", "HEAD"],
capture_output=True, text=True)
return result.stdout.strip() if result.returncode == 0 else None
def audit(root, report_path, dol_path, expected_sha1, units=(), function=None):
inputs = Inputs()
inputs.read(Path(__file__))
inputs.read(Path(elf.__file__))
before = git_head(root)
report = json.loads(inputs.read(report_path))
names, sizes, splits = read_maps(
inputs.read(root / "config/43U/symbols.txt").decode(),
inputs.read(root / "config/43U/splits.txt").decode())
dol = Dol(inputs.read(dol_path), expected_sha1)
counts, rows, skipped, errors = Counter(), [], [], []
selected = set(units)
found = set()
for unit in report["units"]:
key = unit["name"].removeprefix("main/")
if selected and key not in selected:
continue
found.add(key)
functions = [f for f in unit.get("functions", [])
if f.get("fuzzy_match_percent") == 100
and (function is None or function == f["name"])]
if not functions:
if function:
skipped.append({"unit": key, "function": function,
"reason": "function absent or not objdiff100"})
continue
try:
source = read_object(root / "build/43U/src" / (key + ".o"), inputs)
reference = read_object(root / "build/43U/obj" / (key + ".o"), inputs)
except (OSError, ValueError, ELFError) as error:
errors.append({"unit": key, "reason": str(error)})
continue
bases = splits.get(unit.get("metadata", {}).get("source_path"), {})
counts["units"] += 1
for entry in functions:
try:
src, ref = source.function(entry["name"]), reference.function(entry["name"])
if src.size != ref.size:
raise ValueError("function extents differ")
except ValueError as error:
skipped.append({"unit": key, "function": entry["name"], "reason": str(error)})
continue
try:
measured, result = compare_function(source, reference, entry["name"],
bases, names, sizes, dol)
except (ValueError, KeyError) as error:
errors.append({"unit": key, "function": entry["name"], "reason": str(error)})
continue
counts.update(measured)
counts["functions"] += 1
rows.extend(dict(row, unit=key) for row in result)
for missing in sorted(selected - found):
errors.append({"unit": missing, "reason": "unit absent from report"})
after = git_head(root)
changed = inputs.changed()
if before != after or changed:
errors.append({"reason": "inputs changed during audit", "paths": changed})
result = {"advisory": True, "counts": dict(counts), "rows": rows,
"skipped_functions": skipped, "errors": errors,
"provenance": {"head_before": before, "head_after": after,
"dol_sha1": dol.sha1, "sha256": inputs.hashes},
"limits": "Direct BL, objdiff100 and equal extents only. Alias hints require review; "
"no callee-behavior, build-freshness or completion proof."}
return result, 2 if errors else 1 if rows or skipped else 0
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("units", nargs="*")
parser.add_argument("--root", type=Path, default=Path("."))
parser.add_argument("--report", type=Path, help="default: ROOT/build/43U/report.json")
parser.add_argument("--dol", type=Path, help="default: ROOT/orig/43U/00000008.app")
parser.add_argument("--dol-sha1", default=DOL_SHA1)
parser.add_argument("--function", help="one exact-name objdiff100 function; requires one unit")
parser.add_argument("--output", type=Path, help="write JSON here instead of stdout")
args = parser.parse_args(argv)
if args.function and len(args.units) != 1:
parser.error("--function requires exactly one unit")
try:
result, status = audit(args.root, args.report or args.root / "build/43U/report.json",
args.dol or args.root / "orig/43U/00000008.app",
args.dol_sha1, args.units, args.function)
output = json.dumps(result, indent=2) + "\n"
if args.output:
if args.output.resolve() in {Path(p).resolve() for p in result["provenance"]["sha256"]}:
raise ValueError("output must not overwrite an audit input")
args.output.write_text(output)
else:
print(output, end="")
return status
except (OSError, ValueError, KeyError, ELFError) as error:
print("ERROR: " + str(error), file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,316 @@
"""Authored synthetic PPC/ELF/DOL fixtures only; no original executable bytes."""
import contextlib
import hashlib
import io
import json
from pathlib import Path
import struct
import tempfile
import unittest
from unittest.mock import patch
import call_target_diff as tool
BASE = 0x80001000
BLR = 0x4E800020
LI_ONE = 0x38600001
LI_TWO = 0x38600002
def put_word(data, offset, word):
data[offset:offset + 4] = word.to_bytes(4, "big")
def bl(address, destination, absolute=False):
displacement = destination if absolute else destination - address
return 0x48000001 | (displacement & 0x03FFFFFC) | (2 if absolute else 0)
def make_elf(text, symbols, relocations):
"""Minimal ELF32 big-endian PowerPC relocatable, built from authored fields."""
strings = bytearray(b"\0")
symbol_data = bytearray(16)
for name, value, size, section, kind in symbols:
name_offset = len(strings)
strings.extend(name.encode() + b"\0")
symbol_data.extend(struct.pack(">IIIBBH", name_offset, value, size,
0x10 | kind, 0, section))
relocation_data = b"".join(struct.pack(">IIi", offset, index << 8 | kind, addend)
for offset, index, kind, addend in relocations)
section_names = b"\0.text\0.symtab\0.strtab\0.rela.text\0.shstrtab\0"
entries = [("", 0, 0, b"", 0, 0, 0),
(".text", 1, 6, bytes(text), 0, 0, 0),
(".symtab", 2, 0, bytes(symbol_data), 3, 1, 16),
(".strtab", 3, 0, bytes(strings), 0, 0, 0),
(".rela.text", 4, 0, relocation_data, 2, 1, 12),
(".shstrtab", 3, 0, section_names, 0, 0, 0)]
raw = bytearray(52)
headers = []
for name, kind, flags, data, link, info, item_size in entries:
while len(raw) % 4:
raw.append(0)
offset = len(raw) if data else 0
raw.extend(data)
headers.append(struct.pack(">10I", section_names.index(name.encode() + b"\0"),
kind, flags, 0, offset, len(data), link, info,
4 if name else 0, item_size))
while len(raw) % 4:
raw.append(0)
section_offset = len(raw)
raw.extend(b"".join(headers))
ident = b"\x7fELF\x01\x02\x01" + bytes(9)
raw[:52] = ident + struct.pack(">HHIIIIIHHHHHH", 1, 20, 1, 0, 0,
section_offset, 0, 52, 0, 0, 40, 6, 5)
return bytes(raw)
def make_dol(text):
header = [0] * 64
header[0], header[18], header[36] = 256, BASE, len(text)
return struct.pack(">64I", *header) + bytes(text)
def fixture(root, source_call="B", source_raw=False, reference_raw=False,
source_start=0x20, alias_body=False, renamed=False, score=100, size=8):
text = bytearray(0x80)
for offset, value in [(0, LI_ONE), (4, BLR), (8, LI_ONE if alias_body else LI_TWO),
(12, BLR), (0x24, BLR)]:
put_word(text, offset, value)
reference_text = bytearray(text)
put_word(reference_text, 0x20, bl(0x20, 8) if reference_raw else bl(0, 0))
source_text = bytearray(text)
put_word(source_text, source_start + 4, BLR)
destination = 0 if source_call == "A" else 8
put_word(source_text, source_start,
bl(source_start, destination) if source_raw else bl(0, 0))
symbols = [("A", 0, 8, 1, 2), ("B", 8, 8, 1, 2), ("caller", 0x20, 8, 1, 2)]
source_symbols = [("renamed" if renamed and n == source_call else n,
source_start if n == "caller" else v,
size if n == "caller" else length, sec, kind)
for n, v, length, sec, kind in symbols]
source_relocs = [] if source_raw else [(source_start, 1 if source_call == "A" else 2, 10, 0)]
reference_relocs = [] if reference_raw else [(0x20, 2, 10, 0)]
source = root / "build/43U/src/toy.o"
reference = root / "build/43U/obj/toy.o"
for path, data in [(source, make_elf(source_text, source_symbols, source_relocs)),
(reference, make_elf(reference_text, symbols, reference_relocs))]:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
put_word(text, 0x20, bl(BASE + 0x20, BASE + 8))
dol = root / "original.dol"
dol.write_bytes(make_dol(text))
report = {"units": [{"name": "main/toy", "functions": [
{"name": "caller", "fuzzy_match_percent": score}],
"metadata": {"source_path": "toy.c"}}]}
report_path = root / "build/43U/report.json"
report_path.write_text(json.dumps(report))
config = root / "config/43U"
config.mkdir(parents=True, exist_ok=True)
(config / "symbols.txt").write_text(
"A = .text:0x80001000; // type:function size:0x8\n"
"B = .text:0x80001008; // type:function size:0x8\n"
"caller = .text:0x80001020; // type:function size:0x8\n")
(config / "splits.txt").write_text("toy.c:\n\t.text start:0x80001000 end:0x80001080\n")
return report_path, dol, hashlib.sha1(dol.read_bytes()).hexdigest()
class CallTargetTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
def run_fixture(self, **options):
report, dol, sha = fixture(self.root, **options)
return tool.audit(self.root, report, dol, sha)
def test_relocated_destination_matches_original_dol(self):
result, status = self.run_fixture()
self.assertEqual(status, 0)
self.assertEqual(result["counts"]["same_destination"], 1)
self.assertIn(str(self.root / "build/43U/src/toy.o"), result["provenance"]["sha256"])
def test_wrong_callee_detected_despite_identical_relocated_call_bytes(self):
result, status = self.run_fixture(source_call="A")
self.assertEqual(status, 1)
row = result["rows"][0]
self.assertEqual(row["classification"], "different_destination")
self.assertEqual(row["source_addresses"], [BASE])
self.assertEqual(row["dol_destination"], BASE + 8)
self.assertIsNone(row["alias_hint"])
def test_raw_local_calls_with_shifted_source_function(self):
result, status = self.run_fixture(source_raw=True, reference_raw=True, source_start=0x40)
self.assertEqual(status, 0)
self.assertEqual(result["counts"]["source_raw_relative"], 1)
self.assertEqual(result["counts"]["reference_raw_relative"], 1)
def rewrite_source(self, symbols, relocations):
text = bytearray(0x80)
for offset, word in [(0, LI_ONE), (4, BLR), (8, LI_TWO),
(12, BLR), (0x20, bl(0, 0)), (0x24, BLR)]:
put_word(text, offset, word)
(self.root / "build/43U/src/toy.o").write_bytes(make_elf(text, symbols, relocations))
def test_section_symbol_relocation_resolves_actual_function_start(self):
report, dol, sha = fixture(self.root)
self.rewrite_source([("A", 0, 8, 1, 2), ("B", 8, 8, 1, 2),
("caller", 0x20, 8, 1, 2), ("", 0, 0, 1, 3)],
[(0x20, 4, 10, 8)])
result, status = tool.audit(self.root, report, dol, sha)
self.assertEqual(status, 0)
self.assertEqual(result["counts"]["same_destination"], 1)
def test_same_address_alias_is_visible_without_claiming_a_bug(self):
report, dol, sha = fixture(self.root)
self.rewrite_source([("A", 0, 8, 1, 2), ("alias", 0, 0, 0, 2),
("caller", 0x20, 8, 1, 2)], [(0x20, 2, 10, 0)])
with (self.root / "config/43U/symbols.txt").open("a") as stream:
stream.write("alias = .text:0x80001008; // type:function size:0x8\n")
result, status = tool.audit(self.root, report, dol, sha)
self.assertEqual(status, 1)
self.assertEqual(result["rows"][0]["classification"], "same_address_alias")
def test_unknown_external_symbol_is_unresolved_not_guessed(self):
report, dol, sha = fixture(self.root)
self.rewrite_source([("A", 0, 8, 1, 2), ("unknown", 0, 0, 0, 2),
("caller", 0x20, 8, 1, 2)], [(0x20, 2, 10, 0)])
result, status = tool.audit(self.root, report, dol, sha)
self.assertEqual(status, 1)
self.assertEqual(result["rows"][0]["classification"], "unresolved_destination")
self.assertEqual(result["rows"][0]["source_addresses"], [])
def test_wrong_relocation_kind_or_symbol_is_an_input_error(self):
for index, kind in [(2, 1), (99, 10)]:
with self.subTest(index=index, kind=kind):
report, dol, sha = fixture(self.root)
self.rewrite_source([("A", 0, 8, 1, 2), ("B", 8, 8, 1, 2),
("caller", 0x20, 8, 1, 2)], [(0x20, index, kind, 0)])
result, status = tool.audit(self.root, report, dol, sha)
self.assertEqual(status, 2)
self.assertTrue(result["errors"])
def test_wrong_raw_local_callee_detected(self):
result, status = self.run_fixture(source_call="A", source_raw=True)
self.assertEqual(status, 1)
self.assertEqual(result["rows"][0]["source"]["names"], ["A"])
def test_decoder_signed_absolute_and_non_call_cases(self):
self.assertEqual(tool.branch_target(bl(0x20, 0), 0x20), 0)
self.assertEqual(tool.branch_target(bl(0x20, 0x40), 0x20), 0x40)
self.assertEqual(tool.branch_target(bl(0, 8, True), 0x100), 8)
self.assertEqual(tool.branch_target(bl(0, -4, True), 0), 0xFFFFFFFC)
self.assertIsNone(tool.branch_target(0x48000010, 0)) # tail branch
self.assertIsNone(tool.branch_target(0x4E800421, 0)) # indirect bctrl
def test_raw_interior_destination_retains_addend(self):
report, dol, sha = fixture(self.root, source_raw=True)
obj = tool.read_object(self.root / "build/43U/src/toy.o", tool.Inputs())
data = bytearray(obj.data[1])
put_word(data, 0x20, bl(0x20, 12))
obj.data[1] = bytes(data)
destination = obj.call(obj.function("caller"), 0)
self.assertEqual((destination["names"], destination["addend"]), (["B"], 4))
def test_renamed_relocation_free_wrapper_is_retained_for_review(self):
result, status = self.run_fixture(renamed=True)
self.assertEqual(status, 1)
self.assertEqual(result["rows"][0]["classification"], "unresolved_destination")
self.assertIn("identical_relocation_free_local_body", result["rows"][0]["alias_hint"])
def test_identical_original_bodies_do_not_silently_suppress_names(self):
result, status = self.run_fixture(source_call="A", alias_body=True)
self.assertEqual(status, 1)
self.assertEqual(result["rows"][0]["classification"], "different_destination")
self.assertIn("identical_original_leaf_bodies", result["rows"][0]["alias_hint"])
def test_nonexact_function_is_not_audited(self):
result, status = self.run_fixture(score=99.9)
self.assertEqual(status, 0)
self.assertEqual(result["counts"].get("calls", 0), 0)
def test_unequal_extents_are_explicitly_skipped(self):
result, status = self.run_fixture(size=4)
self.assertEqual(status, 1)
self.assertIn("extents differ", result["skipped_functions"][0]["reason"])
def test_missing_requested_function_or_unit_is_explicit(self):
report, dol, sha = fixture(self.root)
result, status = tool.audit(self.root, report, dol, sha, ["toy"], "missing")
self.assertEqual(status, 1)
self.assertTrue(result["skipped_functions"])
result, status = tool.audit(self.root, report, dol, sha, ["missing"])
self.assertEqual(status, 2)
self.assertTrue(result["errors"])
def test_reference_dol_disagreement_is_an_input_error(self):
report, dol, sha = fixture(self.root)
raw = bytearray(dol.read_bytes())
put_word(raw, 256 + 0x20, bl(BASE + 0x20, BASE))
dol.write_bytes(raw)
result, status = tool.audit(self.root, report, dol, hashlib.sha1(raw).hexdigest())
self.assertEqual(status, 2)
self.assertIn("ELF/DOL disagreement", result["errors"][0]["reason"])
def test_call_only_in_original_dol_is_not_missed(self):
report, dol, sha = fixture(self.root)
raw = bytearray(dol.read_bytes())
put_word(raw, 256 + 0x24, bl(BASE + 0x24, BASE))
dol.write_bytes(raw)
result, status = tool.audit(self.root, report, dol, hashlib.sha1(raw).hexdigest())
self.assertEqual(status, 2)
self.assertIn("ELF/DOL disagreement at +0x4", result["errors"][0]["reason"])
def test_removed_source_call_is_a_shape_candidate(self):
report, dol, sha = fixture(self.root)
source = self.root / "build/43U/src/toy.o"
text = bytearray(0x80)
put_word(text, 0x20, LI_ONE)
put_word(text, 0x24, BLR)
source.write_bytes(make_elf(text, [("caller", 0x20, 8, 1, 2)], []))
result, status = tool.audit(self.root, report, dol, sha)
self.assertEqual(status, 1)
self.assertEqual(result["rows"][0]["classification"], "call_shape_difference")
def test_dol_integrity_and_bounds_validation(self):
raw = make_dol(bytes(8))
with self.assertRaisesRegex(ValueError, "SHA1"):
tool.Dol(raw, "0" * 40)
with self.assertRaisesRegex(ValueError, "truncated"):
tool.Dol(b"", hashlib.sha1(b"").hexdigest())
image = tool.Dol(raw, hashlib.sha1(raw).hexdigest())
with self.assertRaisesRegex(ValueError, "not in one DOL section"):
image.read(BASE + 8, 4)
def test_no_body_alias_hint_for_position_dependent_code(self):
self.assertFalse(tool.position_independent_leaf(bl(0, 0x100).to_bytes(4, "big")))
self.assertFalse(tool.position_independent_leaf(bl(0, 0).to_bytes(4, "big")))
def test_changed_input_and_head_are_reported(self):
path = self.root / "input"
path.write_bytes(b"first")
inputs = tool.Inputs()
inputs.read(path)
path.write_bytes(b"second")
self.assertEqual(inputs.changed(), [str(path)])
with self.assertRaisesRegex(ValueError, "changed while reading"):
inputs.read(path)
report, dol, sha = fixture(self.root)
with patch.object(tool, "git_head", side_effect=["before", "after"]):
result, status = tool.audit(self.root, report, dol, sha)
self.assertEqual(status, 2)
self.assertEqual(result["errors"][-1]["reason"], "inputs changed during audit")
def test_cli_json_exit_status_and_input_overwrite_protection(self):
report, dol, sha = fixture(self.root)
output = self.root / "results.json"
args = ["--root", str(self.root), "--dol", str(dol), "--dol-sha1", sha]
self.assertEqual(tool.main(args + ["--output", str(output)]), 0)
self.assertTrue(json.loads(output.read_text())["advisory"])
with contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(tool.main(args + ["--output", str(report)]), 2)
self.assertIn("units", json.loads(report.read_text()))
if __name__ == "__main__":
unittest.main()