Files
neoLegacy-mac/tests/test_reconstructed_as3vm_imp_array.py

454 lines
22 KiB
Python

"""Runtime tests for the reconstructed as3vm_imp_array translation unit.
RED STATE: as3vm_imp_array.c / as3vm_imp_array.h do not exist yet, so setUpClass
raises "missing as3vm_imp_array runtime inputs". Every assertion below is
grounded in a specific line of iggy-decompiled/raw/durango/as3vm_imp_array.c
(cited inline) so a faithful DURANGO reconstruction turns these green.
as3vm_imp_array is Iggy's AS3 / AVM2 Array (and Vector) class: the push / pop /
shift / unshift / splice / slice / concat / sort / sortOn / indexOf /
lastIndexOf / join / reverse / filter / map / forEach / length method family,
plus the low-level arraydata backing ops and the shared as3vm_sort comparator
engine. It exports EXACTLY 52 entry points on all four platforms (23 Array_* +
23 Vector_* + 6 helpers) and -- LIKE as3vm_convert / as3vm_class / as3vm_create
but UNLIKE as3vm_gc / as3vm_property -- OWNS NO externally named data global.
ARRAY OBJECT / ARRAYDATA MODEL (raw/durango, offsets 64-bit; the runtime test
builds ONLY the durango dylib so 64-bit layout is correct -- the 32-bit vita raw
uses obj+0x1b for the arraydata pointer instead of obj+0x2f but the same
arraydata internal layout, cross-confirming the model):
- an as3value is a tagged word: (v & 3) is the tag. Tag 1 == heap cell with a
1-byte type at v[-1] (0x20 == ' ' marks an Array object); an odd non-1 word
(tag 3) is an inline int; 7 is the module's empty-slot / "hole" sentinel.
- an Array object (tag-1, type 0x20 at obj[-1]) holds its arraydata pointer at
obj+0x2f (raw:439-440, 835, 1008).
- an arraydata holds capacity (int32) at arraydata+3, length (int32) at
arraydata+7, and a dense array of 8-byte tagged as3values at arraydata+0x1f;
element[i] lives at arraydata+0x1f+i*8 (raw:2703-2729).
- the AS3 method thunks receive a call frame of four words: frame[0]=vm,
frame[1]=&result slot, frame[2]=argc (int32 at +0x10), frame[3]=&this slot
(which points at [this, arg0, arg1, ...]) (raw:433-445, 855-919).
- the VM's GC-visible operand-stack top pointer lives at vm+0x6b18; the
low-level as3vm_array_push_GC pushes the value there across the potential
grow so the collector can see it (raw:2681-2687).
WHAT IS PINNED (real, deterministic DURANGO behavior):
- as3vm_Array_get_length: result = length<<1 for a valid Array; a non-Array /
wrong-type `this` raises as3vm_exception_GC(vm, 0x78) (raw:438-444).
- as3vm_array_set_value_GC: an in-bounds write leaves length unchanged; an
extend within capacity fills [old_len .. index-1] with the 7 hole sentinel,
sets length=index+1, and writes element[index] (raw:2703-2729).
- as3vm_array_delete_value_GC: an in-bounds non-hole slot is set to 7 and
returns 1; out-of-bounds, negative, or already-hole returns 0 (raw:2551-2557).
- as3vm_array_push_GC: appends element[length]=value, length++, returns the
ARRAY OBJECT (param_2, pushed/popped through the vm+0x6b18 GC stack across the
grow so the collector can see it) -- raw:2687 `return *(gc_top-8)` is the
array object, not the value.
- as3vm_Array_push (thunk): appends each of the argc arguments, sets
length+=argc and result=length<<1 (raw:873-918).
- as3vm_Array_pop: non-empty returns the last element and decrements length;
empty leaves length 0 and returns a sentinel (raw:835-848).
- as3vm_Array_shift: non-empty returns element[0], shifts the tail down one and
decrements length (raw:1008-1023).
- as3vm_Array_reverse: reverses the dense array in place and returns `this`
(raw:937-954).
- as3vm_Array_set_length: a truncation (new_len <= length) sets length=new_len
via a controllable as3vm_ToUInt32_GC (raw:969-988).
- as3vm_Array_indexOf on an empty array returns -1 (tagged -2) after parsing
its (searchValue, fromIndex) arguments (raw:472-490).
DEFERRED (flagged for the impl / review phase, NOT exercised here):
- the full as3vm_sort quicksort + comparator dispatch (default
ToString-lexicographic vs numeric vs custom do_func_call_vv comparator) and
as3vm_Array_sort / sortOn (raw:1188-1382, 2737+) -- reaches ToString /
do_func_call_vv_GC.
- the as3vm_Array_splice interior shift + insert, and slice / concat / join
allocation (raw:1033+, 1383+, 2564+) -- reaches make_arraydata /
make_string.
- the as3vm_Array_indexOf / lastIndexOf deep strict-equality element compare
(int / double / stringdata paths, raw:498-558) beyond the empty-array miss.
- the do_func_call_vv callback family (filter / map / forEach / every / some)
and the makeroom grow path of set_value / push (real reallocating
array_makeroom_GC + make_arraydata).
- the entire Vector_* family (23 exports) -- ABI-pinned in the cross test only.
- NOTE: AS3 insertAt / removeAt are NOT separate exports in this Iggy version
(nm shows none); they are subsumed by splice.
Full-suite runs use python3.12 (the machine python3 is 3.14, which SEGFAULTs on
the full suite via an unrelated CPython ctypes-GC bug); single-module runs of
this file are fine on either interpreter.
"""
import ctypes
import shutil
from pathlib import Path
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
RECONSTRUCTED = ROOT / "iggy-decompiled/src/reconstructed"
SOURCE = RECONSTRUCTED / "as3vm_imp_array.c"
HEADER = RECONSTRUCTED / "as3vm_imp_array.h"
SUPPORT = ROOT / "tests/support/as3vm_imp_array_test_support.c"
# 64-bit DURANGO value / struct constants (cited per test).
TAG_MASK = 3 # (v & 3) is the as3value tag.
TYPE_ARRAY = 0x20 # Array object type byte at obj[-1] (raw:439/970).
OFF_OBJ_ARRAYDATA = 0x2F # Array object -> arraydata pointer (raw:439/835).
OFF_AD_CAPACITY = 3 # arraydata capacity (int32, raw:2707).
OFF_AD_LENGTH = 7 # arraydata length (int32, raw:2706/2723).
OFF_AD_ELEMS = 0x1F # arraydata dense element array (raw:2729).
HOLE_SENTINEL = 7 # empty-slot / "hole" as3value (raw:2554/2719).
OFF_VM_GC_STACK = 0x6B18 # vm -> GC operand-stack top pointer (raw:2681).
class ReconstructedAs3vmImpArrayTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
missing = [str(path) for path in (SOURCE, HEADER, SUPPORT)
if not path.exists()]
if missing:
raise AssertionError(
"missing as3vm_imp_array runtime inputs: " + ", ".join(missing))
cls.temp = tempfile.TemporaryDirectory()
cls.base = Path(cls.temp.name) / "as3vm_imp_array.dylib"
build = subprocess.run([
"clang", "-dynamiclib", "-std=c11", "-O2", "-Wall", "-Wextra",
"-Werror", "-Wl,-undefined,dynamic_lookup",
"-DIGGY_RECONSTRUCT_AS3VM_IMP_ARRAY_DURANGO=1",
"-I", str(RECONSTRUCTED),
str(SOURCE), str(SUPPORT), "-o", str(cls.base),
], capture_output=True, text=True)
if build.returncode:
raise AssertionError(build.stdout + build.stderr)
@classmethod
def tearDownClass(cls):
cls.temp.cleanup()
def setUp(self):
unique = (Path(self.temp.name) /
f"as3vm_imp_array_{self._testMethodName}.dylib")
shutil.copyfile(self.base, unique)
self.lib = ctypes.CDLL(str(unique))
self._bind(self.lib)
self.lib.as3vm_imp_array_test_reset()
self._keep = []
def _bind(self, lib):
void_p = ctypes.c_void_p
lib.as3vm_imp_array_test_reset.argtypes = []
lib.as3vm_imp_array_test_reset.restype = None
for name in ("exception", "write_barrier", "make_arraydata",
"touint32", "parse_args", "tostring", "do_func_call"):
fn = getattr(lib, f"as3vm_imp_array_test_{name}_count")
fn.restype = ctypes.c_size_t
fn.argtypes = []
lib.as3vm_imp_array_test_last_exception_code.restype = ctypes.c_int32
lib.as3vm_imp_array_test_last_exception_code.argtypes = []
lib.as3vm_imp_array_test_set_touint32_result.restype = None
lib.as3vm_imp_array_test_set_touint32_result.argtypes = [ctypes.c_uint32]
lib.as3vm_imp_array_test_set_parse_args_search.restype = None
lib.as3vm_imp_array_test_set_parse_args_search.argtypes = [void_p]
lib.as3vm_imp_array_test_set_parse_args_from.restype = None
lib.as3vm_imp_array_test_set_parse_args_from.argtypes = [ctypes.c_uint32]
# Module exports under test (durango 64-bit ABI).
lib.as3vm_Array_get_length.restype = None
lib.as3vm_Array_get_length.argtypes = [void_p]
lib.as3vm_Array_push.restype = None
lib.as3vm_Array_push.argtypes = [void_p]
lib.as3vm_Array_pop.restype = None
lib.as3vm_Array_pop.argtypes = [void_p]
lib.as3vm_Array_shift.restype = None
lib.as3vm_Array_shift.argtypes = [void_p]
lib.as3vm_Array_reverse.restype = None
lib.as3vm_Array_reverse.argtypes = [void_p]
lib.as3vm_Array_set_length.restype = None
lib.as3vm_Array_set_length.argtypes = [void_p]
lib.as3vm_Array_indexOf.restype = None
lib.as3vm_Array_indexOf.argtypes = [void_p]
lib.as3vm_array_set_value_GC.restype = None
lib.as3vm_array_set_value_GC.argtypes = [
void_p, void_p, ctypes.c_int, void_p]
lib.as3vm_array_delete_value_GC.restype = ctypes.c_int64
lib.as3vm_array_delete_value_GC.argtypes = [void_p, void_p, ctypes.c_int]
lib.as3vm_array_push_GC.restype = ctypes.c_void_p
lib.as3vm_array_push_GC.argtypes = [void_p, void_p, void_p]
def _count(self, name):
return getattr(self.lib, f"as3vm_imp_array_test_{name}_count")()
def _buffer(self, size):
buf = (ctypes.c_uint8 * size)()
self._keep.append(buf)
return ctypes.addressof(buf)
# ---- Builders. ---- #
def _tag3(self, payload):
# A tag-3 inline-int as3value: 16-aligned base | 3 so (v & 3) == 3 and
# the write barrier (which only fires on tag 1) is never taken.
base = self._buffer(0x20)
ctypes.c_uint32.from_address(base).value = payload & 0xFFFFFFFF
return base | TAG_MASK
def _arraydata(self, elements, capacity):
# arraydata: capacity int32 at +3, length int32 at +7, dense 8-byte
# tagged elements at +0x1f (raw:2703-2729).
ad = self._buffer(OFF_AD_ELEMS + (capacity + 8) * 8)
ctypes.c_int32.from_address(ad + OFF_AD_CAPACITY).value = capacity
ctypes.c_int32.from_address(ad + OFF_AD_LENGTH).value = len(elements)
for i, value in enumerate(elements):
ctypes.c_uint64.from_address(ad + OFF_AD_ELEMS + i * 8).value = value
return ad
def _array_object(self, arraydata):
# A tag-1 Array cell: obj = cell + 1, type 0x20 at obj[-1], arraydata
# pointer at obj+0x2f (raw:439-440).
cell = self._buffer(0x80)
ctypes.c_uint8.from_address(cell).value = TYPE_ARRAY
obj = cell + 1
ctypes.c_uint64.from_address(obj + OFF_OBJ_ARRAYDATA).value = arraydata
return obj
def _elem(self, arraydata, index):
return ctypes.c_uint64.from_address(
arraydata + OFF_AD_ELEMS + index * 8).value
def _length(self, arraydata):
return ctypes.c_int32.from_address(arraydata + OFF_AD_LENGTH).value
def _vm(self):
# A vm buffer large enough for the GC-stack top pointer at +0x6b18.
return self._buffer(OFF_VM_GC_STACK + 0x100)
def _install_gc_stack(self, vm):
# Point vm+0x6b18 at a scratch operand stack so the push_GC GC-root
# push/pop has somewhere real to write (raw:2681-2687).
stack = self._buffer(0x400)
ctypes.c_uint64.from_address(vm + OFF_VM_GC_STACK).value = stack
def _frame(self, vm, this_value, argc, args=()):
# frame[0]=vm, frame[1]=&result, frame[2]=argc, frame[3]=&this-slot.
# The this-slot points at [this, arg0, arg1, ...] (raw:855-882).
result = self._buffer(8)
slot = self._buffer(8 * (1 + len(args)))
ctypes.c_uint64.from_address(slot).value = this_value
for i, value in enumerate(args):
ctypes.c_uint64.from_address(slot + (i + 1) * 8).value = value
frame = self._buffer(0x20)
ctypes.c_uint64.from_address(frame + 0x00).value = vm
ctypes.c_uint64.from_address(frame + 0x08).value = result
ctypes.c_int32.from_address(frame + 0x10).value = argc
ctypes.c_uint64.from_address(frame + 0x18).value = slot
return frame, result
def _result(self, result):
return ctypes.c_int64.from_address(result).value
# ------------------------------------------------------------------ #
# as3vm_Array_get_length -- length<<1 + type guard
# ------------------------------------------------------------------ #
def test_get_length_returns_tagged_length(self):
# raw:440 a valid Array writes length<<1 (the int tag) into the result.
vm = self._vm()
ad = self._arraydata([self._tag3(i) for i in range(5)], capacity=8)
frame, result = self._frame(vm, self._array_object(ad), argc=0)
self.lib.as3vm_Array_get_length(frame)
self.assertEqual(self._result(result), 5 * 2)
self.assertEqual(self._count("exception"), 0)
def test_get_length_non_array_raises_0x78(self):
# raw:443 a non-tag-1 / wrong-type `this` raises exception 0x78.
vm = self._vm()
frame, _ = self._frame(vm, self._tag3(1), argc=0)
self.lib.as3vm_Array_get_length(frame)
self.assertEqual(self._count("exception"), 1)
self.assertEqual(
self.lib.as3vm_imp_array_test_last_exception_code(), 0x78)
# ------------------------------------------------------------------ #
# as3vm_array_set_value_GC -- in-bounds write + extend-within-capacity
# ------------------------------------------------------------------ #
def test_set_value_in_bounds_writes_element_keeps_length(self):
# raw:2725-2729 an in-bounds index writes element[i] and, because
# length > index, never enters the grow / extend block.
vm = self._vm()
value = self._tag3(0xAB)
ad = self._arraydata([self._tag3(i) for i in range(3)], capacity=8)
obj = self._array_object(ad)
self.lib.as3vm_array_set_value_GC(vm, obj, 1, value)
self.assertEqual(self._elem(ad, 1), value)
self.assertEqual(self._length(ad), 3)
def test_set_value_extend_fills_holes_and_grows_length(self):
# raw:2715-2729 index >= length but < capacity: fill [old_len..index-1]
# with the 7 hole sentinel, set length=index+1, write element[index].
vm = self._vm()
value = self._tag3(0xCD)
ad = self._arraydata([self._tag3(0), self._tag3(1)], capacity=8)
obj = self._array_object(ad)
self.lib.as3vm_array_set_value_GC(vm, obj, 4, value)
self.assertEqual(self._length(ad), 5)
self.assertEqual(self._elem(ad, 2), HOLE_SENTINEL)
self.assertEqual(self._elem(ad, 3), HOLE_SENTINEL)
self.assertEqual(self._elem(ad, 4), value)
# ------------------------------------------------------------------ #
# as3vm_array_delete_value_GC -- bounds + hole guard
# ------------------------------------------------------------------ #
def test_delete_value_in_bounds_sets_hole_and_returns_one(self):
# raw:2551-2556 an in-bounds non-hole slot becomes 7 and returns 1.
vm = self._vm()
ad = self._arraydata([self._tag3(i + 1) for i in range(3)], capacity=8)
obj = self._array_object(ad)
self.assertEqual(self.lib.as3vm_array_delete_value_GC(vm, obj, 1), 1)
self.assertEqual(self._elem(ad, 1), HOLE_SENTINEL)
def test_delete_value_already_hole_or_oob_returns_zero(self):
# raw:2553/2557 an already-hole slot, an out-of-bounds index, and a
# negative index all return 0 without mutating the array.
vm = self._vm()
ad = self._arraydata(
[self._tag3(1), HOLE_SENTINEL, self._tag3(3)], capacity=8)
obj = self._array_object(ad)
self.assertEqual(self.lib.as3vm_array_delete_value_GC(vm, obj, 1), 0)
self.assertEqual(self.lib.as3vm_array_delete_value_GC(vm, obj, 5), 0)
self.assertEqual(self.lib.as3vm_array_delete_value_GC(vm, obj, -1), 0)
self.assertEqual(self._length(ad), 3)
# ------------------------------------------------------------------ #
# as3vm_array_push_GC -- low-level append driving the GC stack
# ------------------------------------------------------------------ #
def test_array_push_gc_appends_and_returns_object(self):
# raw:2681-2687 push element[length]=value, length++. The function
# pushes the ARRAY OBJECT (param_2) onto the GC stack across the grow and
# returns it back off the stack -- `return *(gc_top-8)` == the array
# object, NOT the value. (filter uses this returned array as its
# accumulator, raw:433-445.)
vm = self._vm()
self._install_gc_stack(vm)
value = self._tag3(0x99)
ad = self._arraydata([self._tag3(0), self._tag3(1)], capacity=8)
obj = self._array_object(ad)
returned = self.lib.as3vm_array_push_GC(vm, obj, value)
self.assertEqual(returned, obj)
self.assertEqual(self._length(ad), 3)
self.assertEqual(self._elem(ad, 2), value)
# ------------------------------------------------------------------ #
# as3vm_Array_push (thunk) -- multi-arg append + tagged length result
# ------------------------------------------------------------------ #
def test_array_push_thunk_appends_all_arguments(self):
# raw:873-918 the thunk appends each of the argc arguments in order
# (capacity pre-sized so the makeroom path is not taken) and returns the
# new length<<1.
vm = self._vm()
arg_a = self._tag3(0xA1)
arg_b = self._tag3(0xB2)
ad = self._arraydata([self._tag3(0), self._tag3(1)], capacity=16)
frame, result = self._frame(
vm, self._array_object(ad), argc=2, args=(arg_a, arg_b))
self.lib.as3vm_Array_push(frame)
self.assertEqual(self._length(ad), 4)
self.assertEqual(self._elem(ad, 2), arg_a)
self.assertEqual(self._elem(ad, 3), arg_b)
self.assertEqual(self._result(result), 4 * 2)
# ------------------------------------------------------------------ #
# as3vm_Array_pop / shift / reverse
# ------------------------------------------------------------------ #
def test_pop_returns_last_and_decrements_length(self):
# raw:840-847 pop reads element[length-1], decrements length, returns it.
vm = self._vm()
values = [self._tag3(10), self._tag3(20), self._tag3(30)]
ad = self._arraydata(values, capacity=8)
frame, result = self._frame(vm, self._array_object(ad), argc=0)
self.lib.as3vm_Array_pop(frame)
self.assertEqual(self._result(result), values[2])
self.assertEqual(self._length(ad), 2)
def test_pop_empty_leaves_length_zero(self):
# raw:836-838 popping an empty array writes the undefined sentinel and
# does NOT decrement length below zero.
vm = self._vm()
ad = self._arraydata([], capacity=8)
frame, result = self._frame(vm, self._array_object(ad), argc=0)
self.lib.as3vm_Array_pop(frame)
self.assertEqual(self._length(ad), 0)
self.assertNotEqual(self._result(result), 0)
def test_shift_returns_first_and_shifts_tail_down(self):
# raw:1012-1022 shift returns element[0], moves element[i]=element[i+1]
# for the tail and decrements length.
vm = self._vm()
values = [self._tag3(10), self._tag3(20), self._tag3(30)]
ad = self._arraydata(values, capacity=8)
frame, result = self._frame(vm, self._array_object(ad), argc=0)
self.lib.as3vm_Array_shift(frame)
self.assertEqual(self._result(result), values[0])
self.assertEqual(self._length(ad), 2)
self.assertEqual(self._elem(ad, 0), values[1])
self.assertEqual(self._elem(ad, 1), values[2])
def test_reverse_reverses_in_place_and_returns_this(self):
# raw:944-954 reverse swaps the dense array in place and returns `this`.
vm = self._vm()
values = [self._tag3(i) for i in (1, 2, 3, 4)]
ad = self._arraydata(values, capacity=8)
obj = self._array_object(ad)
frame, result = self._frame(vm, obj, argc=0)
self.lib.as3vm_Array_reverse(frame)
self.assertEqual(
[self._elem(ad, i) for i in range(4)], list(reversed(values)))
self.assertEqual(self._result(result), obj)
# ------------------------------------------------------------------ #
# as3vm_Array_set_length -- truncation via controllable ToUInt32
# ------------------------------------------------------------------ #
def test_set_length_truncates(self):
# raw:978/986 new_len <= length sets length=new_len directly (the
# extend branch, which reaches array_set_value_GC, is deferred).
vm = self._vm()
ad = self._arraydata([self._tag3(i) for i in range(5)], capacity=8)
frame, _ = self._frame(vm, self._array_object(ad), argc=1,
args=(self._tag3(2),))
self.lib.as3vm_imp_array_test_set_touint32_result(2)
self.lib.as3vm_Array_set_length(frame)
self.assertEqual(self._count("touint32"), 1)
self.assertEqual(self._length(ad), 2)
self.assertEqual(self._count("exception"), 0)
# ------------------------------------------------------------------ #
# as3vm_Array_indexOf -- empty-array miss returns -1
# ------------------------------------------------------------------ #
def test_indexof_empty_array_returns_negative_one(self):
# raw:487-489 with length 0 the scan never runs and indexOf returns the
# not-found result -1, tagged as -2.
vm = self._vm()
ad = self._arraydata([], capacity=8)
frame, result = self._frame(vm, self._array_object(ad), argc=1,
args=(self._tag3(7),))
self.lib.as3vm_imp_array_test_set_parse_args_search(
ctypes.c_void_p(self._tag3(7)))
self.lib.as3vm_imp_array_test_set_parse_args_from(0)
self.lib.as3vm_Array_indexOf(frame)
self.assertEqual(self._count("parse_args"), 1)
self.assertEqual(self._result(result), -2)
if __name__ == "__main__":
unittest.main()