Files
neoLegacy-mac/tests/test_reconstructed_as3vm_imp_string_cross.py
2026-07-18 17:34:31 -05:00

368 lines
19 KiB
Python

"""Cross-platform contract tests for the reconstructed as3vm_imp_string module.
Mirrors tests/test_reconstructed_as3vm_imp_math_cross.py /
tests/test_reconstructed_as3vm_imp_array_cross.py (the freshest 4-platform
templates in the same AS3/AVM2 VM cluster). RED until as3vm_imp_string.c /
as3vm_imp_string.h (and their CMake / modules.tsv rows) exist; the
retail-evidence assertions run now and prove the nm/strings parsing.
as3vm_imp_string is Iggy's AS3 / AVM2 String class implementation: charAt /
charCodeAt / concat / fromCharCode / indexOf / lastIndexOf / localeCompare /
match / replace / search / slice / split / substr / substring /
toLocaleLowerCase / toLocaleUpperCase / toString / valueOf / get_length /
__construct, plus the file-level export as3vm_stringdata_compare_caseinsensitive.
It is a 4-PLATFORM module (durango 867 / orbis 919 / ps3 827 / vita 1001 raw
lines; retail source_functions 22 / 21 / 21 / 21 -- durango keeps the file-static
make_substring_GC as a 22nd function while orbis / ps3 / vita inline it; the
exported ABI stays a fixed 21 on all four).
How EXPECTED_* were derived (all four .obj under
work/iggy-native-decomp/<platform>-objects/as3vm_imp_string.obj, verified with
/opt/homebrew/opt/llvm/bin/llvm-nm and `nm -g`):
EXPECTED_FUNCTIONS = `llvm-nm --extern-only --defined-only <obj>` T-tagged
names, leading '.' stripped on ps3. IDENTICAL 21-name
set on durango/orbis/ps3/vita -- the exported String ABI
is invariant even though the internal helper count varies
(22/21/21/21). The .opd descriptors on ps3 surface each
export a second time as 'D'; those normalize to the
exported names and are NOT separate globals (same handling
as as3vm_convert / as3vm_imp_math / as3vm_imp_array).
EXPECTED_GLOBALS = {} -- LIKE as3vm_convert / as3vm_imp_math /
as3vm_imp_array (UNLIKE as3vm_gc / as3vm_property),
as3vm_imp_string owns NO externally named data global; nm
shows no extern B/D/C data symbol on any of the four
objects. The durango object carries six MSVC 'R'
read-only ??_C@... COMDATs -- the component-parse format
strings "?d", "s?d", "s", "sv", "?dd", "?vd" (recovered
with objdump -s) -- which are rodata artifacts, local on
orbis / ps3 / vita, NOT owned exported data. The ps3
uppercase 'D' symbols are .opd descriptors for the 21
exports and make_substring_GC is a module-LOCAL static.
EXPECTED_IMPORTS = the genuine non-scaffolding references present on ALL
FOUR retail objects (INVARIANT -- verified identical, 15
symbols):
- plain-case siblings (iggy_unicode -- ALREADY BUILT;
this module's toLocale* tail-call them):
as3vm_String_toLowerCase, as3vm_String_toUpperCase
- type conversion (as3vm_convert -- satisfied):
as3vm_ToInt32_GC, as3vm_ToString_GC
- imp_array helpers (as3vm_imp_array -- satisfied):
as3vm_array_join, as3vm_array_push_GC,
as3vm_make_array_GC
- argument parsing (as3vm_component -- satisfied):
as3vm_component_parse_arguments_GC
- interpreter re-entry / diagnostics (satisfied):
as3vm_do_func_call_vv_GC, as3vm_exception_GC
- string construction (as3vm_create -- satisfied):
as3vm_make_string_wc_GC
- GC (as3vm_gc -- satisfied):
as3vm_write_barrier_slowpath
- case-insensitive UTF-16 compare (iggy_unicode --
satisfied): rrCompareInsensitiveUTF16toUTF16
- DATA sentinels (referenced by ADDRESS; dot-UNprefixed
on ps3, confirming DATA not calls):
as3vm_global_nan, as3vm_nullundef_storage
NOTE the evidence contradicts the naive expectation that
as3vm_String_toLowerCase / toUpperCase are unbuilt: they
are already reconstructed in iggy_unicode.c (grep-verified)
-- classified here as satisfied plain-case siblings, not
genuinely-unbuilt. This real-Iggy/rr set is INVARIANT
across the four platforms.
PLATFORM_IMPORTS = the libc the reconstruction lowers. UNIFORM here
({memcmp, memcpy} on every object -- no per-target
divergence): this module memcmps every substring compare
(indexOf / lastIndexOf / search / replace / split) and
memcpys the substring / replace builders, but never
memsets, calls libm, or longjmps. Durango does NOT
__imp_-thunk memcmp/memcpy here (they appear bare), so no
normalization is needed beyond the ps3 dot prefix.
RUNTIME_ALLOWLISTS = pure toolchain scaffolding subtracted before the import
comparison. Verified against retail
`llvm-nm --undefined-only`:
durango: {} in RETAIL -- unlike its siblings the retail
durango object emits NO /GS or _fltused symbol
(its stack locals are small int arrays that do
not trip the /GS guard). The __GSHandlerCheck
/ __chkstk / __security_* / _fltused set is
nonetheless allowlisted DEFENSIVELY (a tolerant
superset) so the portable x86_64-pc-windows-msvc
reconstruction is accepted whether or not clang
re-emits any of them.
orbis: _GLOBAL_OFFSET_TABLE_ (retail-exact)
ps3: {} (retail-exact, empty)
vita: __aeabi_unwind_cpp_pr0, __stack_chk_fail,
__stack_chk_guard (retail-exact)
The retail vita object shows NO stray integer-math libcall
(__aeabi_l2d / __divsi3 / __aeabi_idiv) despite the heavy
index arithmetic -- the reconstruction must likewise avoid
synthesizing one; if a faithful port does emit one it is a
genuine import to reclassify, NOT scaffolding, so it is
deliberately left OUT of this allowlist.
PROVENANCE = the canonical Windows source path
c:\\projects\\g\\src\\flash\\as3vm_imp_string.c is the true
upstream path (orbis preserves it verbatim), so the
reconstruction embeds it and the SOURCE assertion checks
it. In the retail objects the provenance string is NOT
uniform -- follow the evidence per platform (like
as3vm_imp_math, NOT the uniform as3vm_gc):
durango: only the build path
c:\\projects\\g\\build\\...\\as3vm_imp_string.obj
(no .c string at all)
orbis: c:\\projects\\g\\src\\flash\\as3vm_imp_string.c
ps3: bare 'as3vm_imp_string.c'
vita: bare 'as3vm_imp_string.c'
Run the full suite with python3.12 (`python3.12 -m unittest discover -s tests`):
the machine python3 is 3.14, which SEGFAULTs on the full suite via an unrelated
CPython ctypes-GC bug. Single-module runs are fine on either interpreter; this
cross test's cross-compile/nm logic is python-version-independent.
"""
from pathlib import Path
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
RECONSTRUCTED = ROOT / "iggy-decompiled/src/reconstructed"
SOURCE = RECONSTRUCTED / "as3vm_imp_string.c"
HEADER = RECONSTRUCTED / "as3vm_imp_string.h"
LLVM_NM = Path("/opt/homebrew/opt/llvm/bin/llvm-nm")
VARIANTS = {
"durango": ("clang", "x86_64-pc-windows-msvc"),
"orbis": ("clang", "x86_64-unknown-freebsd"),
"vita": ("clang", "armv7a-unknown-linux-gnueabihf"),
}
PS3_CLANG = Path("/opt/homebrew/opt/llvm/bin/clang")
if PS3_CLANG.exists():
VARIANTS["ps3"] = (str(PS3_CLANG), "powerpc-unknown-linux-gnu")
# Invariant 21-name exported String ABI (leading '.' stripped on ps3).
EXPECTED_FUNCTIONS = {
"as3vm_String__construct",
"as3vm_String_charAt",
"as3vm_String_charCodeAt",
"as3vm_String_concat",
"as3vm_String_fromCharCode",
"as3vm_String_get_length",
"as3vm_String_indexOf",
"as3vm_String_lastIndexOf",
"as3vm_String_localeCompare",
"as3vm_String_match",
"as3vm_String_replace",
"as3vm_String_search",
"as3vm_String_slice",
"as3vm_String_split",
"as3vm_String_substr",
"as3vm_String_substring",
"as3vm_String_toLocaleLowerCase",
"as3vm_String_toLocaleUpperCase",
"as3vm_String_toString",
"as3vm_String_valueOf",
"as3vm_stringdata_compare_caseinsensitive",
}
# as3vm_imp_string owns no externally named data global; the durango ??_C@
# format strings are read-only rodata COMDATs, not owned exported data.
EXPECTED_GLOBALS = set()
# Genuine non-scaffolding imports present on all four retail objects (invariant).
EXPECTED_IMPORTS = {
# plain-case siblings (iggy_unicode -- ALREADY BUILT).
"as3vm_String_toLowerCase",
"as3vm_String_toUpperCase",
# type conversion (as3vm_convert -- satisfied).
"as3vm_ToInt32_GC",
"as3vm_ToString_GC",
# imp_array helpers (as3vm_imp_array -- satisfied).
"as3vm_array_join",
"as3vm_array_push_GC",
"as3vm_make_array_GC",
# argument parsing (as3vm_component -- satisfied).
"as3vm_component_parse_arguments_GC",
# interpreter re-entry / diagnostics (satisfied).
"as3vm_do_func_call_vv_GC",
"as3vm_exception_GC",
# string construction (as3vm_create -- satisfied).
"as3vm_make_string_wc_GC",
# GC (as3vm_gc -- satisfied).
"as3vm_write_barrier_slowpath",
# case-insensitive UTF-16 compare (iggy_unicode -- satisfied).
"rrCompareInsensitiveUTF16toUTF16",
# DATA sentinels (referenced by address; dot-UNprefixed on ps3).
"as3vm_global_nan",
"as3vm_nullundef_storage",
}
# The libc lowering is UNIFORM here ({memcmp, memcpy} on all four; no per-target
# divergence and no __imp_ thunk on durango), routed through PLATFORM_IMPORTS
# because it is libc rather than an Iggy symbol.
PLATFORM_IMPORTS = {
"durango": {"memcmp", "memcpy"},
"orbis": {"memcmp", "memcpy"},
"ps3": {"memcmp", "memcpy"},
"vita": {"memcmp", "memcpy"},
}
RUNTIME_ALLOWLISTS = {
# Retail durango emits NONE of these for this module; allowlisted defensively
# as a tolerant superset (see module docstring).
"durango": {"__GSHandlerCheck", "__chkstk", "__security_check_cookie",
"__security_cookie", "_fltused"},
"orbis": {"_GLOBAL_OFFSET_TABLE_"},
"ps3": set(),
"vita": {"__aeabi_unwind_cpp_pr0", "__stack_chk_fail", "__stack_chk_guard"},
}
# The provenance string embedded in each retail object differs (see the module
# docstring); assert per-platform evidence rather than a uniform path.
RETAIL_PROVENANCE = {
"durango": r"c:\projects\g\build",
"orbis": r"c:\projects\g\src\flash\as3vm_imp_string.c",
"ps3": "as3vm_imp_string.c",
"vita": "as3vm_imp_string.c",
}
def normalized_symbol(name):
name = name.removeprefix(".")
name = name.removeprefix("__imp_")
if name == "_setjmp":
return "setjmp"
return name
class ReconstructedAs3vmImpStringCrossTests(unittest.TestCase):
def test_native_provenance_counts_source_path_and_inventory(self):
# (function count, raw line count) per platform, from
# iggy_<platform>.modules.tsv column 3 and `wc -l` on the raw .c. The
# source_functions DIFFER (22/21/21/21) while the exports stay 21.
expected = {
"durango": (22, 867), "orbis": (21, 919),
"ps3": (21, 827), "vita": (21, 1001),
}
for platform, (functions, lines) in expected.items():
modules = ROOT / (
f"iggy-decompiled/symbols/iggy_{platform}.modules.tsv")
row = next(
line for line in modules.read_text(encoding="utf-8").splitlines()
if line.startswith("as3vm_imp_string\t"))
self.assertEqual(int(row.split("\t")[2]), functions, platform)
raw = ROOT / f"iggy-decompiled/raw/{platform}/as3vm_imp_string.c"
self.assertEqual(
len(raw.read_text(encoding="utf-8").splitlines()),
lines, platform)
obj = ROOT / (
f"work/iggy-native-decomp/{platform}-objects/"
f"as3vm_imp_string.obj")
symbols = subprocess.run(
["nm", "-g", str(obj)], capture_output=True, text=True)
self.assertEqual(symbols.returncode, 0, (platform, symbols.stderr))
normalized = symbols.stdout.replace(" .", " ")
for symbol in EXPECTED_FUNCTIONS | EXPECTED_GLOBALS:
self.assertIn(symbol, normalized, (platform, symbol))
strings = subprocess.run(
["strings", str(obj)], capture_output=True, text=True)
self.assertIn(RETAIL_PROVENANCE[platform], strings.stdout, platform)
if SOURCE.exists():
cmake = (RECONSTRUCTED / "CMakeLists.txt").read_text(
encoding="utf-8")
self.assertIn(
"iggy_reconstructed_as3vm_imp_string_${platform}", cmake)
self.assertIn(
"add_library(${target} STATIC as3vm_imp_string.c)", cmake)
self.assertIn(
"IGGY_RECONSTRUCT_AS3VM_IMP_STRING_${platform_upper}", cmake)
inventory = (RECONSTRUCTED / "modules.tsv").read_text(
encoding="utf-8")
# Exports are invariant (21) but retail source_functions vary
# (22/21/21/21), so the inventory uses per-platform rows exactly like
# as3vm_imp_math / as3vm_imp_array (NOT a single combined row).
for platform, functions in (
("durango", 22), ("orbis", 21),
("ps3", 21), ("vita", 21)):
self.assertIn(
f"as3vm_imp_string\t{functions}\t21\t{platform}\tcomplete",
inventory)
def test_exact_exports_globals_normalized_imports_and_c_cpp_abi(self):
self.assertTrue(SOURCE.exists(), f"missing {SOURCE}")
self.assertTrue(HEADER.exists(), f"missing {HEADER}")
self.assertIn(r"c:\\projects\\g\\src\\flash\\as3vm_imp_string.c",
SOURCE.read_text(encoding="utf-8"))
function_pointers = "".join(
f" (GenericFn)&{name},\n" for name in sorted(EXPECTED_FUNCTIONS))
probe_text = (
'#include <stdint.h>\n#include "as3vm_imp_string.h"\n'
'typedef void (*GenericFn)(void);\n'
'static GenericFn string_abis[] = {\n' + function_pointers + '};\n'
'int use_abis(void) {\n'
' for (unsigned i = 0; '
'i < sizeof(string_abis) / sizeof(string_abis[0]); ++i) {\n'
' if (!string_abis[i]) return 0;\n'
' }\n'
' return 1;\n'
'}\n')
with tempfile.TemporaryDirectory() as directory:
temporary = Path(directory)
for platform, (compiler, target) in VARIANTS.items():
extra = ["-mthumb"] if platform == "vita" else []
output = temporary / f"as3vm_imp_string_{platform}.o"
build = subprocess.run([
compiler, "-target", target, "-ffreestanding",
"-fno-builtin",
"-c", "-std=c11", "-O2", "-Wall", "-Wextra", "-Werror",
*extra,
f"-DIGGY_RECONSTRUCT_AS3VM_IMP_STRING_{platform.upper()}=1",
"-I", str(RECONSTRUCTED), str(SOURCE), "-o", str(output),
], capture_output=True, text=True)
self.assertEqual(build.returncode, 0, (platform, build.stderr))
if LLVM_NM.exists():
defined = subprocess.run([
str(LLVM_NM), "--extern-only", "--defined-only",
str(output),
], capture_output=True, text=True)
undefined = subprocess.run([
str(LLVM_NM), "--undefined-only", str(output),
], capture_output=True, text=True)
names = {normalized_symbol(parts[-1])
for line in defined.stdout.splitlines()
if len(parts := line.split()) >= 2 and
parts[-2].upper() in {"T", "D", "B", "C"}}
imports = {normalized_symbol(line.split()[-1])
for line in undefined.stdout.splitlines()
if line.split()}
self.assertEqual(
names, EXPECTED_FUNCTIONS | EXPECTED_GLOBALS, platform)
expected_imports = (
EXPECTED_IMPORTS | PLATFORM_IMPORTS[platform])
self.assertEqual(
imports - RUNTIME_ALLOWLISTS[platform],
expected_imports, (platform, imports))
for language, standard in (("c", "c11"), ("c++", "c++17")):
suffix = "c" if language == "c" else "cpp"
probe = (temporary /
f"as3vm_imp_string_abi_{platform}.{suffix}")
probe.write_text(probe_text, encoding="utf-8")
syntax = subprocess.run([
compiler, "-target", target, "-ffreestanding",
"-fsyntax-only", f"-std={standard}",
"-Wall", "-Wextra", "-Werror", *extra,
f"-DIGGY_RECONSTRUCT_AS3VM_IMP_STRING_"
f"{platform.upper()}=1",
"-I", str(RECONSTRUCTED), str(probe),
], capture_output=True, text=True)
self.assertEqual(syntax.returncode, 0,
(platform, language, syntax.stderr))
if __name__ == "__main__":
unittest.main()