Files
neoLegacy-mac/tests/test_reconstructed_as3vm_imp_array_cross.py

386 lines
18 KiB
Python

"""Cross-platform contract tests for the reconstructed as3vm_imp_array module.
Mirrors tests/test_reconstructed_as3vm_property_cross.py /
tests/test_reconstructed_as3vm_convert_cross.py (the freshest 4-platform
templates in the same AS3/AVM2 VM cluster). RED until as3vm_imp_array.c /
as3vm_imp_array.h (and their CMake / modules.tsv rows) exist; the retail-evidence
assertions run now and prove the nm/strings parsing.
as3vm_imp_array is Iggy's AS3 / AVM2 Array (and Vector) class implementation: the
push / pop / shift / unshift / splice / slice / concat / sort / sortOn /
indexOf / lastIndexOf / join / reverse / filter / map / forEach / length method
family, plus the low-level arraydata backing ops and the shared as3vm_sort
comparator engine. It is a 4-PLATFORM module (durango 3580 / orbis 3601 /
ps3 4116 / vita 3484 raw lines; retail source_functions 57 / 57 / 56 / 56 --
durango and orbis carry one extra file-static helper that ps3 / vita inline,
while the exported ABI stays a fixed 52 on all four).
How EXPECTED_* were derived (all four .obj under
work/iggy-native-decomp/<platform>-objects/as3vm_imp_array.obj, verified with
/opt/homebrew/opt/llvm/bin/llvm-nm and `nm -g`):
EXPECTED_FUNCTIONS = `llvm-nm --extern-only --defined-only <obj>` T-tagged
names, leading '.' stripped on ps3. IDENTICAL 52-name
set on durango/orbis/ps3/vita (23 Array_* + 23 Vector_* +
6 helpers) -- the exported Array/Vector ABI is invariant
even though the internal helper count varies
(57/57/56/56). The .opd descriptors on ps3 surface each
export a second time as 'D'; those normalize to the
exported names and are NOT separate globals (same handling
as as3vm_convert / as3vm_property). Note the Array side
carries sortOn (no get_fixed) and the Vector side carries
get_fixed (no sortOn); only Array has sortOn.
EXPECTED_GLOBALS = {} -- LIKE as3vm_convert / as3vm_class / as3vm_create
(UNLIKE as3vm_gc / as3vm_property), as3vm_imp_array owns
NO externally named data global; nm shows no extern
B/D/C data symbol on any of the four objects. (The
durango object carries four 'R' read-only string literals
-- three tiny join-format separators and the provenance
path -- which are neither T/D/B/C-extern nor globals.
The ps3 object's uppercase 'D' symbols are .opd function
descriptors for the 52 exports, and its lowercase 'd'
symbols -- array_makeroom_GC, comparison,
splice_or_unshift_GC, strict_equality -- are module-LOCAL
statics, not owned globals.)
EXPECTED_IMPORTS = the genuine non-scaffolding references present on ALL
FOUR retail objects (INVARIANT -- verified identical, 25
symbols):
- type conversion (as3vm_convert -- satisfied):
as3vm_ToCharArray_GC, as3vm_ToNumberDirect_GC,
as3vm_ToNumber_GC, as3vm_ToString_GC,
as3vm_ToUInt32_GC, as3vm_convert_array_to_vector,
as3vm_to_boolean_direct
- value construction (as3vm_create -- satisfied):
as3vm_make_array_GC,
as3vm_make_array_from_arraydata_GC,
as3vm_make_arraydata_GC, as3vm_make_string_GC,
as3vm_make_stringdata_GC
- naming (as3vm_names -- satisfied): as3vm_qname_find
- property / GC (satisfied): as3vm_get_property_qname_GC,
as3vm_write_barrier_slowpath
- allocator (iggy_allocation -- satisfied):
iggy_free_alloc, iggy_malloc_fileline
- genuinely-unbuilt (interpreter / component / string):
as3vm_coerce_GC,
as3vm_component_parse_arguments_GC,
as3vm_do_func_call_vv_GC, as3vm_exception_GC,
as3vm_intern_string_value,
as3vm_stringdata_compare_caseinsensitive
- DATA sentinels (referenced by ADDRESS; dot-UNprefixed
on ps3, confirming they are DATA not calls):
as3vm_nullundef_storage, as3vm_truefalse_storage
This real-Iggy set is INVARIANT across the four platforms.
PLATFORM_IMPORTS = the libc / libm the reconstruction lowers. Here the set
is UNIFORM across all four ({memcpy, rr_isnan} on every
object -- no per-target divergence, unlike as3vm_convert's
floor/memcmp split), but they are routed through
PLATFORM_IMPORTS (not EXPECTED_IMPORTS) because they are
libc/libm rather than Iggy symbols. This module memcpys
(join / shift moves) but never memsets or longjmps, and
touches rr_isnan only in the indexOf/lastIndexOf numeric
compare.
RUNTIME_ALLOWLISTS = pure toolchain scaffolding residual after subtracting the
genuine import set, verified exact against retail
`llvm-nm --undefined-only`:
durango: __GSHandlerCheck, __chkstk,
__security_check_cookie, __security_cookie,
_fltused
orbis: _GLOBAL_OFFSET_TABLE_
ps3: {} (empty)
vita: __aeabi_unwind_cpp_pr0, __stack_chk_fail,
__stack_chk_guard
(durango pulls the __GSHandler / __chkstk / cookie set
here -- the join thunk takes the address of a large
stack buffer that trips the /GS guard and probes the
stack; allowlisted so it is subtracted whether or not the
reconstruction's portable form re-emits it.)
PROVENANCE = the canonical Windows source path
c:\\projects\\g\\src\\flash\\as3vm_imp_array.c is embedded
in EVERY one of the four retail objects (durango carries
it alongside the build-temp .obj path; ps3 / vita carry it
alongside a bare 'as3vm_imp_array.c'). Because the
canonical string is uniformly present, the retail-obj
check asserts it uniformly, like as3vm_gc / as3vm_property.
Run the full suite with python3.12 (`python3.12 -m unittest discover -s tests`):
the machine python3 is 3.14, which SEGFAULTs on the full suite via an unrelated
CPython ctypes-GC bug. Single-module runs are fine on either interpreter; this
cross test's cross-compile/nm logic is python-version-independent.
"""
from pathlib import Path
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
RECONSTRUCTED = ROOT / "iggy-decompiled/src/reconstructed"
SOURCE = RECONSTRUCTED / "as3vm_imp_array.c"
HEADER = RECONSTRUCTED / "as3vm_imp_array.h"
LLVM_NM = Path("/opt/homebrew/opt/llvm/bin/llvm-nm")
VARIANTS = {
"durango": ("clang", "x86_64-pc-windows-msvc"),
"orbis": ("clang", "x86_64-unknown-freebsd"),
"vita": ("clang", "armv7a-unknown-linux-gnueabihf"),
}
PS3_CLANG = Path("/opt/homebrew/opt/llvm/bin/clang")
if PS3_CLANG.exists():
VARIANTS["ps3"] = (str(PS3_CLANG), "powerpc-unknown-linux-gnu")
# Invariant 52-name exported Array/Vector ABI (leading '.' stripped on ps3):
# 23 Array_* + 23 Vector_* + 6 helpers.
EXPECTED_FUNCTIONS = {
"as3vm_Array__construct",
"as3vm_Array_concat",
"as3vm_Array_every",
"as3vm_Array_filter",
"as3vm_Array_forEach",
"as3vm_Array_get_length",
"as3vm_Array_indexOf",
"as3vm_Array_join",
"as3vm_Array_lastIndexOf",
"as3vm_Array_map",
"as3vm_Array_pop",
"as3vm_Array_push",
"as3vm_Array_reverse",
"as3vm_Array_set_length",
"as3vm_Array_shift",
"as3vm_Array_slice",
"as3vm_Array_some",
"as3vm_Array_sort",
"as3vm_Array_sortOn",
"as3vm_Array_splice",
"as3vm_Array_toLocaleString",
"as3vm_Array_toString",
"as3vm_Array_unshift",
"as3vm_Vector__construct",
"as3vm_Vector_concat",
"as3vm_Vector_every",
"as3vm_Vector_filter",
"as3vm_Vector_forEach",
"as3vm_Vector_get_fixed",
"as3vm_Vector_get_length",
"as3vm_Vector_indexOf",
"as3vm_Vector_join",
"as3vm_Vector_lastIndexOf",
"as3vm_Vector_map",
"as3vm_Vector_pop",
"as3vm_Vector_push",
"as3vm_Vector_reverse",
"as3vm_Vector_set_length",
"as3vm_Vector_shift",
"as3vm_Vector_slice",
"as3vm_Vector_some",
"as3vm_Vector_sort",
"as3vm_Vector_splice",
"as3vm_Vector_toLocaleString",
"as3vm_Vector_toString",
"as3vm_Vector_unshift",
"as3vm_array_clone_GC",
"as3vm_array_delete_value_GC",
"as3vm_array_join",
"as3vm_array_push_GC",
"as3vm_array_set_value_GC",
"as3vm_sort",
}
# as3vm_imp_array owns no externally named data global.
EXPECTED_GLOBALS = set()
# Genuine non-scaffolding imports present on all four retail objects (invariant).
EXPECTED_IMPORTS = {
# type conversion (as3vm_convert -- satisfied).
"as3vm_ToCharArray_GC",
"as3vm_ToNumberDirect_GC",
"as3vm_ToNumber_GC",
"as3vm_ToString_GC",
"as3vm_ToUInt32_GC",
"as3vm_convert_array_to_vector",
"as3vm_to_boolean_direct",
# value construction (as3vm_create -- satisfied).
"as3vm_make_array_GC",
"as3vm_make_array_from_arraydata_GC",
"as3vm_make_arraydata_GC",
"as3vm_make_string_GC",
"as3vm_make_stringdata_GC",
# naming (as3vm_names -- satisfied).
"as3vm_qname_find",
# property / GC (satisfied).
"as3vm_get_property_qname_GC",
"as3vm_write_barrier_slowpath",
# allocator (iggy_allocation -- satisfied).
"iggy_free_alloc",
"iggy_malloc_fileline",
# genuinely-unbuilt (interpreter / component / string).
"as3vm_coerce_GC",
"as3vm_component_parse_arguments_GC",
"as3vm_do_func_call_vv_GC",
"as3vm_exception_GC",
"as3vm_intern_string_value",
"as3vm_stringdata_compare_caseinsensitive",
# DATA sentinels (referenced by address; dot-UNprefixed on ps3).
"as3vm_nullundef_storage",
"as3vm_truefalse_storage",
}
# The libc/libm lowering is UNIFORM here ({memcpy, rr_isnan} on all four; no
# per-target divergence like as3vm_convert's floor/memcmp), but is routed through
# PLATFORM_IMPORTS because it is libc/libm rather than an Iggy symbol.
PLATFORM_IMPORTS = {
"durango": {"memcpy", "rr_isnan"},
"orbis": {"memcpy", "rr_isnan"},
"ps3": {"memcpy", "rr_isnan"},
"vita": {"memcpy", "rr_isnan"},
}
RUNTIME_ALLOWLISTS = {
"durango": {"__GSHandlerCheck", "__chkstk", "__security_check_cookie",
"__security_cookie", "_fltused"},
"orbis": {"_GLOBAL_OFFSET_TABLE_"},
"ps3": set(),
"vita": {"__aeabi_unwind_cpp_pr0", "__stack_chk_fail", "__stack_chk_guard"},
}
# The canonical Windows source path is embedded in every one of the four retail
# objects, so assert it uniformly (like as3vm_gc / as3vm_property).
CANONICAL_PROVENANCE = r"c:\projects\g\src\flash\as3vm_imp_array.c"
def normalized_symbol(name):
name = name.removeprefix(".")
name = name.removeprefix("__imp_")
if name == "_setjmp":
return "setjmp"
return name
class ReconstructedAs3vmImpArrayCrossTests(unittest.TestCase):
def test_native_provenance_counts_source_path_and_inventory(self):
# (function count, raw line count) per platform, from
# iggy_<platform>.modules.tsv column 3 and `wc -l` on the raw .c. The
# source_functions DIFFER (57/57/56/56) while the exports stay 52.
expected = {
"durango": (57, 3580), "orbis": (57, 3601),
"ps3": (56, 4116), "vita": (56, 3484),
}
for platform, (functions, lines) in expected.items():
modules = ROOT / (
f"iggy-decompiled/symbols/iggy_{platform}.modules.tsv")
row = next(
line for line in modules.read_text(encoding="utf-8").splitlines()
if line.startswith("as3vm_imp_array\t"))
self.assertEqual(int(row.split("\t")[2]), functions, platform)
raw = ROOT / f"iggy-decompiled/raw/{platform}/as3vm_imp_array.c"
self.assertEqual(
len(raw.read_text(encoding="utf-8").splitlines()),
lines, platform)
obj = ROOT / (
f"work/iggy-native-decomp/{platform}-objects/as3vm_imp_array.obj")
symbols = subprocess.run(
["nm", "-g", str(obj)], capture_output=True, text=True)
self.assertEqual(symbols.returncode, 0, (platform, symbols.stderr))
normalized = symbols.stdout.replace(" .", " ")
for symbol in EXPECTED_FUNCTIONS | EXPECTED_GLOBALS:
self.assertIn(symbol, normalized, (platform, symbol))
strings = subprocess.run(
["strings", str(obj)], capture_output=True, text=True)
self.assertIn(CANONICAL_PROVENANCE, strings.stdout, platform)
if SOURCE.exists():
cmake = (RECONSTRUCTED / "CMakeLists.txt").read_text(
encoding="utf-8")
self.assertIn(
"iggy_reconstructed_as3vm_imp_array_${platform}", cmake)
self.assertIn(
"add_library(${target} STATIC as3vm_imp_array.c)", cmake)
self.assertIn(
"IGGY_RECONSTRUCT_AS3VM_IMP_ARRAY_${platform_upper}", cmake)
inventory = (RECONSTRUCTED / "modules.tsv").read_text(
encoding="utf-8")
# Exports are invariant (52) but retail source_functions vary
# (57/57/56/56), so the inventory uses per-platform rows exactly like
# as3vm_gc / as3vm_property (NOT a single combined row).
for platform, functions in (
("durango", 57), ("orbis", 57),
("ps3", 56), ("vita", 56)):
self.assertIn(
f"as3vm_imp_array\t{functions}\t52\t{platform}\tcomplete",
inventory)
def test_exact_exports_globals_normalized_imports_and_c_cpp_abi(self):
self.assertTrue(SOURCE.exists(), f"missing {SOURCE}")
self.assertTrue(HEADER.exists(), f"missing {HEADER}")
self.assertIn(r"c:\\projects\\g\\src\\flash\\as3vm_imp_array.c",
SOURCE.read_text(encoding="utf-8"))
function_pointers = "".join(
f" (GenericFn)&{name},\n" for name in sorted(EXPECTED_FUNCTIONS))
probe_text = (
'#include <stdint.h>\n#include "as3vm_imp_array.h"\n'
'typedef void (*GenericFn)(void);\n'
'static GenericFn array_abis[] = {\n' + function_pointers + '};\n'
'int use_abis(void) {\n'
' for (unsigned i = 0; '
'i < sizeof(array_abis) / sizeof(array_abis[0]); ++i) {\n'
' if (!array_abis[i]) return 0;\n'
' }\n'
' return 1;\n'
'}\n')
with tempfile.TemporaryDirectory() as directory:
temporary = Path(directory)
for platform, (compiler, target) in VARIANTS.items():
extra = ["-mthumb"] if platform == "vita" else []
output = temporary / f"as3vm_imp_array_{platform}.o"
build = subprocess.run([
compiler, "-target", target, "-ffreestanding",
"-fno-builtin",
"-c", "-std=c11", "-O2", "-Wall", "-Wextra", "-Werror",
*extra,
f"-DIGGY_RECONSTRUCT_AS3VM_IMP_ARRAY_{platform.upper()}=1",
"-I", str(RECONSTRUCTED), str(SOURCE), "-o", str(output),
], capture_output=True, text=True)
self.assertEqual(build.returncode, 0, (platform, build.stderr))
if LLVM_NM.exists():
defined = subprocess.run([
str(LLVM_NM), "--extern-only", "--defined-only",
str(output),
], capture_output=True, text=True)
undefined = subprocess.run([
str(LLVM_NM), "--undefined-only", str(output),
], capture_output=True, text=True)
names = {normalized_symbol(parts[-1])
for line in defined.stdout.splitlines()
if len(parts := line.split()) >= 2 and
parts[-2].upper() in {"T", "D", "B", "C"}}
imports = {normalized_symbol(line.split()[-1])
for line in undefined.stdout.splitlines()
if line.split()}
self.assertEqual(
names, EXPECTED_FUNCTIONS | EXPECTED_GLOBALS, platform)
expected_imports = (
EXPECTED_IMPORTS | PLATFORM_IMPORTS[platform])
self.assertEqual(
imports - RUNTIME_ALLOWLISTS[platform],
expected_imports, (platform, imports))
for language, standard in (("c", "c11"), ("c++", "c++17")):
suffix = "c" if language == "c" else "cpp"
probe = temporary / f"as3vm_imp_array_abi_{platform}.{suffix}"
probe.write_text(probe_text, encoding="utf-8")
syntax = subprocess.run([
compiler, "-target", target, "-ffreestanding",
"-fsyntax-only", f"-std={standard}",
"-Wall", "-Wextra", "-Werror", *extra,
f"-DIGGY_RECONSTRUCT_AS3VM_IMP_ARRAY_{platform.upper()}=1",
"-I", str(RECONSTRUCTED), str(probe),
], capture_output=True, text=True)
self.assertEqual(syntax.returncode, 0,
(platform, language, syntax.stderr))
if __name__ == "__main__":
unittest.main()