Attachment #308810 for bug #170215

View Details Raw Unified Return to Bug 170215

| Differences between



2017-04-28  JF Bastien  <jfbastien@apple.com>

        WebAssembly: Air::Inst::generate crashes on large binary on A64
        https://bugs.webkit.org/show_bug.cgi?id=170215

        Reviewed by NOBODY (OOPS!).

        ARM can't encode all offsets in a single instruction. We usualy
        handle this type of detail early, or the macro assembler uses a
        scratch register to take care of the large immediate. After
        register allocation we assumed that we would never get large
        offsets, and asserted this was the case. That was a fine
        assumption with JavaScript, but WebAssembly ends up generating
        stack frames which are too big to encode.

        There are two places that needed to be fixed:
            1. AirGenerate
            2. AirLowerStackArgs

        We now unconditionally pin the dataTempRegister on ARM64, and use
        it when immediates don't fit.

        Number 1. is easy: we're just incrementing SP, make sure we can
        use a scratch register when that happens.

        Number 2. is more complex: not all Inst can receive a stack
        argument whose base register isn't SP or FP. Specifically,
        Patchpoints and Stackmaps get very sad because they just want to
        know the offset value, but when we materialize the offset as
        follows:

            Move (spill337), (spill201), %r0, @8735

        Becomes (where %r16 is dataTempRegister):
            Move $1404, %r16, @8736
            Add64 %sp, %r16, @8736
            Move (%r16), 2032(%sp), %r0, @8736

        The code currently doesn't see through our little dance. To work
        around this issue we introduce a new Air Arg kind:
        ExtendedOffsetAddr. This is the same as a regular Addr, but with
        an offset which may be too big to encode. Opcodes then declare
        whether their arguments can handle such inputs, and if so we
        generate them, otherwise we generate Addr as shown above.

        None of this affects x86 because it can always encode large
        immediates.

        This patch also drive-by converts some uses of `override` to
        `final`. It makes the code easier to grok, and maybe helps the
        optimizer sometimes but really that doens't matter.

        * assembler/MacroAssembler.h:
        * assembler/MacroAssemblerARM64.h:
        * b3/B3CheckSpecial.cpp:
        (JSC::B3::CheckSpecial::admitsExtendedOffsetAddr):
        * b3/B3CheckSpecial.h:
        * b3/B3Common.cpp:
        (JSC::B3::pinnedExtendedOffsetAddrRegister): keep the CPU-specific
        pinning information in a cpp file
        * b3/B3Common.h:
        * b3/B3PatchpointSpecial.cpp:
        (JSC::B3::PatchpointSpecial::admitsExtendedOffsetAddr):
        * b3/B3PatchpointSpecial.h:
        * b3/B3StackmapSpecial.cpp:
        (JSC::B3::StackmapSpecial::isArgValidForRep):
        (JSC::B3::StackmapSpecial::repForArg):
        * b3/B3StackmapSpecial.h:
        * b3/air/AirArg.cpp:
        (JSC::B3::Air::Arg::isStackMemory):
        (JSC::B3::Air::Arg::jsHash):
        (JSC::B3::Air::Arg::dump):
        (WTF::printInternal):
        (JSC::B3::Air::Arg::stackAddrImpl): Deleted. There was only one
        use of this (in AirLowerStackArgs) and it was now confusing to
        split the logic up between these two. Inline the code that used to
        be here into its one usepoint instead.
        * b3/air/AirArg.h:
        (JSC::B3::Air::Arg::extendedOffsetAddr):
        (JSC::B3::Air::Arg::isExtendedOffsetAddr):
        (JSC::B3::Air::Arg::isMemory):
        (JSC::B3::Air::Arg::base):
        (JSC::B3::Air::Arg::offset):
        (JSC::B3::Air::Arg::isGP):
        (JSC::B3::Air::Arg::isFP):
        (JSC::B3::Air::Arg::isValidForm):
        (JSC::B3::Air::Arg::forEachTmpFast):
        (JSC::B3::Air::Arg::forEachTmp):
        (JSC::B3::Air::Arg::asAddress):
        (JSC::B3::Air::Arg::stackAddr): Deleted.
        * b3/air/AirCCallSpecial.cpp:
        (JSC::B3::Air::CCallSpecial::isValid):
        (JSC::B3::Air::CCallSpecial::admitsExtendedOffsetAddr):
        (JSC::B3::Air::CCallSpecial::generate):
        * b3/air/AirCCallSpecial.h:
        * b3/air/AirCode.cpp:
        (JSC::B3::Air::Code::Code):
        (JSC::B3::Air::Code::pinRegister): Check that the register wasn't
        pinned before pinning it. It's likely a bug to pin the same
        register twice.
        * b3/air/AirCustom.h:
        (JSC::B3::Air::PatchCustom::admitsExtendedOffsetAddr):
        (JSC::B3::Air::CCallCustom::admitsExtendedOffsetAddr):
        (JSC::B3::Air::ShuffleCustom::admitsExtendedOffsetAddr):
        (JSC::B3::Air::EntrySwitchCustom::admitsExtendedOffsetAddr):
        (JSC::B3::Air::WasmBoundsCheckCustom::admitsExtendedOffsetAddr):
        * b3/air/AirGenerate.cpp:
        (JSC::B3::Air::generate):
        * b3/air/AirInst.h:
        * b3/air/AirInstInlines.h:
        (JSC::B3::Air::Inst::admitsExtendedOffsetAddr):
        * b3/air/AirLowerStackArgs.cpp:
        (JSC::B3::Air::lowerStackArgs):
        * b3/air/AirPrintSpecial.cpp:
        (JSC::B3::Air::PrintSpecial::admitsExtendedOffsetAddr):
        (JSC::B3::Air::PrintSpecial::generate):
        * b3/air/AirPrintSpecial.h:
        * b3/air/AirSpecial.h:
        * b3/air/opcode_generator.rb:

2017-04-28  Mark Lam  <mark.lam@apple.com>

        Revert instrumentation from https://bugs.webkit.org/show_bug.cgi?id=170086 that is no longer needed.

Lines 821-827 public: a/Source/JavaScriptCore/assembler/MacroAssembler.h_sec1
- a/Source/JavaScriptCore/assembler/MacroAssembler.h -1 / +1 lines
821	return MacroAssemblerBase::branchTest8(cond, Address(address.base, address.offset), mask);	821	return MacroAssemblerBase::branchTest8(cond, Address(address.base, address.offset), mask);
822	}	822	}
823		823
824	#else // !CPU(X86_64)	824	#else // !CPU(X86_64) && !CPU(ARM64)
825		825
826	void addPtr(RegisterID src, RegisterID dest)	826	void addPtr(RegisterID src, RegisterID dest)
827	{	827	{



    static const unsigned numGPRs = 32;
    static const unsigned numFPRs = 32;
    
    static constexpr RegisterID dataTempRegister = ARM64Registers::ip0;
    static constexpr RegisterID memoryTempRegister = ARM64Registers::ip1;

    RegisterID scratchRegister()
    {

Lines 140-145 bool CheckSpecial::admitsStack(Inst& inst, unsigned argIndex) a/Source/JavaScriptCore/b3/B3CheckSpecial.cpp_sec1
- a/Source/JavaScriptCore/b3/B3CheckSpecial.cpp +7 lines
140	return admitsStackImpl(numB3Args(inst), m_numCheckArgs + 1, inst, argIndex);	140	return admitsStackImpl(numB3Args(inst), m_numCheckArgs + 1, inst, argIndex);
141	}	141	}
142		142
		143	bool CheckSpecial::admitsExtendedOffsetAddr(Inst& inst, unsigned argIndex)
		144	{
		145	if (argIndex >= 1 && argIndex < 1 + m_numCheckArgs)
		146	return false;
		147	return admitsStackImpl(numB3Args(inst), m_numCheckArgs + 1, inst, argIndex);
		148	}
		149
143	std::optional<unsigned> CheckSpecial::shouldTryAliasingDef(Inst& inst)	150	std::optional<unsigned> CheckSpecial::shouldTryAliasingDef(Inst& inst)
144	{	151	{
145	if (std::optional<unsigned> branchDef = hiddenBranch(inst).shouldTryAliasingDef())	152	if (std::optional<unsigned> branchDef = hiddenBranch(inst).shouldTryAliasingDef())



    // Constructs and returns the Inst representing the branch that this will use.
    Air::Inst hiddenBranch(const Air::Inst&) const;

    void forEachArg(Air::Inst&, const ScopedLambda<Air::Inst::EachArgCallback>&) final;
    bool isValid(Air::Inst&) final;
    bool admitsStack(Air::Inst&, unsigned argIndex) final;
    bool admitsExtendedOffsetAddr(Air::Inst&, unsigned) final;
    std::optional<unsigned> shouldTryAliasingDef(Air::Inst&) final;

    // NOTE: the generate method will generate the hidden branch and then register a LatePath that
    // generates the stackmap. Super crazy dude!

    CCallHelpers::Jump generate(Air::Inst&, CCallHelpers&, Air::GenerationContext&) final;

    void dumpImpl(PrintStream&) const final;
    void deepDumpImpl(PrintStream&) const final;

private:
    Air::Kind m_checkKind;



    return Options::logB3PhaseTimes();
}

std::optional<GPRReg> pinnedExtendedOffsetAddrRegister()
{
#if CPU(ARM64)
    return static_cast<GPRReg>(+MacroAssembler::dataTempRegister);
#elif CPU(X86_64)
    return std::nullopt;
#else
#error Unhandled architecture.
#endif
}

} } // namespace JSC::B3

#endif // ENABLE(B3_JIT)

Lines 28-33 a/Source/JavaScriptCore/b3/B3Common.h_sec1
- a/Source/JavaScriptCore/b3/B3Common.h +3 lines
28	#if ENABLE(B3_JIT)	28	#if ENABLE(B3_JIT)
29		29
30	#include "CPU.h"	30	#include "CPU.h"
		31	#include "GPRInfo.h"
31	#include "JSExportMacros.h"	32	#include "JSExportMacros.h"
32	#include "Options.h"	33	#include "Options.h"
33	#include <wtf/Optional.h>	34	#include <wtf/Optional.h>
Lines 182-187 inline unsigned defaultOptLevel() a/Source/JavaScriptCore/b3/B3Common.h_sec2
182	return Options::defaultB3OptLevel();	183	return Options::defaultB3OptLevel();
183	}	184	}
184		185
		186	std::optional<GPRReg> pinnedExtendedOffsetAddrRegister();
		187
185	} } // namespace JSC::B3	188	} } // namespace JSC::B3
186		189
187	#endif // ENABLE(B3_JIT)	190	#endif // ENABLE(B3_JIT)

Lines 130-135 bool PatchpointSpecial::admitsStack(Inst& inst, unsigned argIndex) a/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp_sec1
- a/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp +5 lines
130	return admitsStackImpl(0, 2, inst, argIndex);	130	return admitsStackImpl(0, 2, inst, argIndex);
131	}	131	}
132		132
		133	bool PatchpointSpecial::admitsExtendedOffsetAddr(Inst&, unsigned)
		134	{
		135	return true;
		136	}
		137
133	CCallHelpers::Jump PatchpointSpecial::generate(	138	CCallHelpers::Jump PatchpointSpecial::generate(
134	Inst& inst, CCallHelpers& jit, GenerationContext& context)	139	Inst& inst, CCallHelpers& jit, GenerationContext& context)
135	{	140	{



    virtual ~PatchpointSpecial();

protected:
    void forEachArg(Air::Inst&, const ScopedLambda<Air::Inst::EachArgCallback>&) final;
    bool isValid(Air::Inst&) final;
    bool admitsStack(Air::Inst&, unsigned argIndex) final;
    bool admitsExtendedOffsetAddr(Air::Inst&, unsigned) final;

    // NOTE: the generate method will generate the hidden branch and then register a LatePath that
    // generates the stackmap. Super crazy dude!

    CCallHelpers::Jump generate(Air::Inst&, CCallHelpers&, Air::GenerationContext&) final;
    
    bool isTerminal(Air::Inst&) final;

    void dumpImpl(PrintStream&) const final;
    void deepDumpImpl(PrintStream&) const final;
};

} } // namespace JSC::B3

Lines 252-258 bool StackmapSpecial::isArgValidForRep(Air::Code& code, const Air::Arg& arg, con a/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp_sec1
- a/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp -1 / +4 lines
252	case ValueRep::StackArgument:	252	case ValueRep::StackArgument:
253	if (arg == Arg::callArg(rep.offsetFromSP()))	253	if (arg == Arg::callArg(rep.offsetFromSP()))
254	return true;	254	return true;
255	if (arg.isAddr() && code.frameSize()) {	255	if ((arg.isAddr() \|\| arg.isExtendedOffsetAddr()) && code.frameSize()) {
256	if (arg.base() == Tmp(GPRInfo::callFrameRegister)	256	if (arg.base() == Tmp(GPRInfo::callFrameRegister)
257	&& arg.offset() == rep.offsetFromSP() - code.frameSize())	257	&& arg.offset() == rep.offsetFromSP() - code.frameSize())
258	return true;	258	return true;
Lines 277-282 ValueRep StackmapSpecial::repForArg(Code& code, const Arg& arg) a/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp_sec2
277	case Arg::BigImm:	277	case Arg::BigImm:
278	return ValueRep::constant(arg.value());	278	return ValueRep::constant(arg.value());
279	break;	279	break;
		280	case Arg::ExtendedOffsetAddr:
		281	ASSERT(arg.base() == Tmp(GPRInfo::callFrameRegister));
		282	FALLTHROUGH;
280	case Arg::Addr:	283	case Arg::Addr:
281	if (arg.base() == Tmp(GPRInfo::callFrameRegister))	284	if (arg.base() == Tmp(GPRInfo::callFrameRegister))
282	return ValueRep::stack(arg.offset());	285	return ValueRep::stack(arg.offset());



    };

protected:
    void reportUsedRegisters(Air::Inst&, const RegisterSet&) final;
    RegisterSet extraEarlyClobberedRegs(Air::Inst&) final;
    RegisterSet extraClobberedRegs(Air::Inst&) final;

    // Note that this does not override generate() or dumpImpl()/deepDumpImpl(). We have many some
    // subclasses that implement that.




namespace JSC { namespace B3 { namespace Air {

Arg Arg::stackAddrImpl(int32_t offsetFromFP, unsigned frameSize, Width width)
{
    Arg result = Arg::addr(Air::Tmp(GPRInfo::callFrameRegister), offsetFromFP);
    if (!result.isValidForm(width)) {
        result = Arg::addr(
            Air::Tmp(MacroAssembler::stackPointerRegister),
            offsetFromFP + frameSize);
        if (!result.isValidForm(width))
            result = Arg();
    }
    return result;
}

bool Arg::isStackMemory() const
{
    switch (kind()) {
    case Addr:
        return base() == Air::Tmp(GPRInfo::callFrameRegister)
            || base() == Air::Tmp(MacroAssembler::stackPointerRegister);
    case ExtendedOffsetAddr:
    case Stack:
    case CallArg:
        return true;

        result += m_base.internalValue();
        break;
    case Addr:
    case ExtendedOffsetAddr:
        result += m_offset;
        result += m_base.internalValue();
        break;

        out.print("(", base(), ")");
        return;
    case Addr:
    case ExtendedOffsetAddr:
        if (offset())
            out.print(offset());
        out.print("(", base(), ")");

    case Arg::Addr:
        out.print("Addr");
        return;
    case Arg::ExtendedOffsetAddr:
        out.print("ExtendedOffsetAddr");
        return;
    case Arg::Stack:
        out.print("Stack");
        return;

Lines 75-80 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec1
- a/Source/JavaScriptCore/b3/air/AirArg.h -11 / +26 lines
75	// (UseAddr) addresses.	75	// (UseAddr) addresses.
76	SimpleAddr,	76	SimpleAddr,
77	Addr,	77	Addr,
		78	ExtendedOffsetAddr,
78	Stack,	79	Stack,
79	CallArg,	80	CallArg,
80	Index,	81	Index,
Lines 546-551 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec2
546	return result;	547	return result;
547	}	548	}
548		549
		550	template<typename Int, typename = Value::IsLegalOffset<Int>>
		551	static Arg extendedOffsetAddr(Int offsetFromFP)
		552	{
		553	Arg result;
		554	result.m_kind = ExtendedOffsetAddr;
		555	result.m_base = Air::Tmp(MacroAssembler::framePointerRegister);
		556	result.m_offset = offsetFromFP;
		557	return result;
		558	}
		559
549	static Arg addr(Air::Tmp base)	560	static Arg addr(Air::Tmp base)
550	{	561	{
551	return addr(base, 0);	562	return addr(base, 0);
Lines 575-586 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec3
575	return result;	586	return result;
576	}	587	}
577		588
578	template<typename Int, typename = Value::IsLegalOffset<Int>>
579	static Arg stackAddr(Int offsetFromFP, unsigned frameSize, Width width)
580	{
581	return stackAddrImpl(offsetFromFP, frameSize, width);
582	}
583
584	// If you don't pass a Width, this optimistically assumes that you're using the right width.	589	// If you don't pass a Width, this optimistically assumes that you're using the right width.
585	static bool isValidScale(unsigned scale, std::optional<Width> width = std::nullopt)	590	static bool isValidScale(unsigned scale, std::optional<Width> width = std::nullopt)
586	{	591	{
Lines 759-764 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec4
759	return kind() == Addr;	764	return kind() == Addr;
760	}	765	}
761		766
		767	bool isExtendedOffsetAddr() const
		768	{
		769	return kind() == ExtendedOffsetAddr;
		770	}
		771
762	bool isStack() const	772	bool isStack() const
763	{	773	{
764	return kind() == Stack;	774	return kind() == Stack;
Lines 779-784 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec5
779	switch (kind()) {	789	switch (kind()) {
780	case SimpleAddr:	790	case SimpleAddr:
781	case Addr:	791	case Addr:
		792	case ExtendedOffsetAddr:
782	case Stack:	793	case Stack:
783	case CallArg:	794	case CallArg:
784	case Index:	795	case Index:
Lines 792-797 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec6
792	// stack references. The following idioms are recognized:	803	// stack references. The following idioms are recognized:
793	// - the Stack kind	804	// - the Stack kind
794	// - the CallArg kind	805	// - the CallArg kind
		806	// - the ExtendedOffsetAddr kind
795	// - the Addr kind with the base being either SP or FP	807	// - the Addr kind with the base being either SP or FP
796	// Callers of this function are allowed to expect that if it returns true, then it must be one of	808	// Callers of this function are allowed to expect that if it returns true, then it must be one of
797	// these easy-to-recognize kinds. So, making this function recognize more kinds could break things.	809	// these easy-to-recognize kinds. So, making this function recognize more kinds could break things.
Lines 943-949 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec7
943		955
944	Air::Tmp base() const	956	Air::Tmp base() const
945	{	957	{
946	ASSERT(kind() == SimpleAddr \|\| kind() == Addr \|\| kind() == Index);	958	ASSERT(kind() == SimpleAddr \|\| kind() == Addr \|\| kind() == ExtendedOffsetAddr \|\| kind() == Index);
947	return m_base;	959	return m_base;
948	}	960	}
949		961
Lines 953-959 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec8
953	{	965	{
954	if (kind() == Stack)	966	if (kind() == Stack)
955	return static_cast<Value::OffsetType>(m_scale);	967	return static_cast<Value::OffsetType>(m_scale);
956	ASSERT(kind() == Addr \|\| kind() == CallArg \|\| kind() == Index);	968	ASSERT(kind() == Addr \|\| kind() == ExtendedOffsetAddr \|\| kind() == CallArg \|\| kind() == Index);
957	return static_cast<Value::OffsetType>(m_offset);	969	return static_cast<Value::OffsetType>(m_offset);
958	}	970	}
959		971
Lines 1012-1017 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec9
1012	case BitImm64:	1024	case BitImm64:
1013	case SimpleAddr:	1025	case SimpleAddr:
1014	case Addr:	1026	case Addr:
		1027	case ExtendedOffsetAddr:
1015	case Index:	1028	case Index:
1016	case Stack:	1029	case Stack:
1017	case CallArg:	1030	case CallArg:
Lines 1047-1052 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec10
1047	return false;	1060	return false;
1048	case SimpleAddr:	1061	case SimpleAddr:
1049	case Addr:	1062	case Addr:
		1063	case ExtendedOffsetAddr:
1050	case Index:	1064	case Index:
1051	case Stack:	1065	case Stack:
1052	case CallArg:	1066	case CallArg:
Lines 1222-1227 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec11
1222	case BitImm64:	1236	case BitImm64:
1223	return isValidBitImm64Form(value());	1237	return isValidBitImm64Form(value());
1224	case SimpleAddr:	1238	case SimpleAddr:
		1239	case ExtendedOffsetAddr:
1225	return true;	1240	return true;
1226	case Addr:	1241	case Addr:
1227	case Stack:	1242	case Stack:
Lines 1247-1252 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec12
1247	case Tmp:	1262	case Tmp:
1248	case SimpleAddr:	1263	case SimpleAddr:
1249	case Addr:	1264	case Addr:
		1265	case ExtendedOffsetAddr:
1250	functor(m_base);	1266	functor(m_base);
1251	break;	1267	break;
1252	case Index:	1268	case Index:
Lines 1288-1293 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec13
1288	break;	1304	break;
1289	case SimpleAddr:	1305	case SimpleAddr:
1290	case Addr:	1306	case Addr:
		1307	case ExtendedOffsetAddr:
1291	functor(m_base, Use, GP, argRole == UseAddr ? argWidth : pointerWidth());	1308	functor(m_base, Use, GP, argRole == UseAddr ? argWidth : pointerWidth());
1292	break;	1309	break;
1293	case Index:	1310	case Index:
Lines 1326-1332 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec14
1326	{	1343	{
1327	if (isSimpleAddr())	1344	if (isSimpleAddr())
1328	return MacroAssembler::Address(m_base.gpr());	1345	return MacroAssembler::Address(m_base.gpr());
1329	ASSERT(isAddr());	1346	ASSERT(isAddr() \|\| isExtendedOffsetAddr());
1330	return MacroAssembler::Address(m_base.gpr(), static_cast<Value::OffsetType>(m_offset));	1347	return MacroAssembler::Address(m_base.gpr(), static_cast<Value::OffsetType>(m_offset));
1331	}	1348	}
1332		1349
Lines 1438-1445 public: a/Source/JavaScriptCore/b3/air/AirArg.h_sec15
1438	}	1455	}
1439		1456
1440	private:	1457	private:
1441	static Arg stackAddrImpl(int32_t, unsigned, Width);
1442
1443	int64_t m_offset { 0 };	1458	int64_t m_offset { 0 };
1444	Kind m_kind { Invalid };	1459	Kind m_kind { Invalid };
1445	int32_t m_scale { 1 };	1460	int32_t m_scale { 1 };

Lines 82-87 bool CCallSpecial::isValid(Inst& inst) a/Source/JavaScriptCore/b3/air/AirCCallSpecial.cpp_sec1
- a/Source/JavaScriptCore/b3/air/AirCCallSpecial.cpp +11 lines
82	return false;	82	return false;
83	case Arg::Tmp:	83	case Arg::Tmp:
84	case Arg::Addr:	84	case Arg::Addr:
		85	case Arg::ExtendedOffsetAddr:
85	case Arg::Stack:	86	case Arg::Stack:
86	case Arg::CallArg:	87	case Arg::CallArg:
87	break;	88	break;
Lines 117-122 bool CCallSpecial::admitsStack(Inst&, unsigned argIndex) a/Source/JavaScriptCore/b3/air/AirCCallSpecial.cpp_sec2
117	return false;	118	return false;
118	}	119	}
119		120
		121	bool CCallSpecial::admitsExtendedOffsetAddr(Inst&, unsigned argIndex)
		122	{
		123	// The callee can be on the stack.
		124	if (argIndex == calleeArgOffset)
		125	return true;
		126
		127	return false;
		128	}
		129
120	void CCallSpecial::reportUsedRegisters(Inst&, const RegisterSet&)	130	void CCallSpecial::reportUsedRegisters(Inst&, const RegisterSet&)
121	{	131	{
122	}	132	}
Lines 133-138 CCallHelpers::Jump CCallSpecial::generate(Inst& inst, CCallHelpers& jit, Generat a/Source/JavaScriptCore/b3/air/AirCCallSpecial.cpp_sec3
133	jit.call(inst.args[calleeArgOffset].gpr());	143	jit.call(inst.args[calleeArgOffset].gpr());
134	break;	144	break;
135	case Arg::Addr:	145	case Arg::Addr:
		146	case Arg::ExtendedOffsetAddr:
136	jit.call(inst.args[calleeArgOffset].asAddress());	147	jit.call(inst.args[calleeArgOffset].asAddress());
137	break;	148	break;
138	default:	149	default:



    static const GPRReg scratchRegister = GPRInfo::nonArgGPR0;

protected:
    void forEachArg(Inst&, const ScopedLambda<Inst::EachArgCallback>&) final;
    bool isValid(Inst&) final;
    bool admitsStack(Inst&, unsigned argIndex) final;
    bool admitsExtendedOffsetAddr(Inst&, unsigned) final;
    void reportUsedRegisters(Inst&, const RegisterSet&) final;
    CCallHelpers::Jump generate(Inst&, CCallHelpers&, GenerationContext&) final;
    RegisterSet extraEarlyClobberedRegs(Inst&) final;
    RegisterSet extraClobberedRegs(Inst&) final;

    void dumpImpl(PrintStream&) const final;
    void deepDumpImpl(PrintStream&) const final;

private:
    static const unsigned specialArgOffset = 0;

Lines 62-67 Code::Code(Procedure& proc) a/Source/JavaScriptCore/b3/air/AirCode.cpp_sec1
- a/Source/JavaScriptCore/b3/air/AirCode.cpp +4 lines
62	});	62	});
63	setRegsInPriorityOrder(bank, result);	63	setRegsInPriorityOrder(bank, result);
64	});	64	});
		65
		66	if (auto reg = pinnedExtendedOffsetAddrRegister())
		67	pinRegister(*reg);
65	}	68	}
66		69
67	Code::~Code()	70	Code::~Code()
Lines 82-87 void Code::setRegsInPriorityOrder(Bank bank, const Vector<Reg>& regs) a/Source/JavaScriptCore/b3/air/AirCode.cpp_sec2
82	void Code::pinRegister(Reg reg)	85	void Code::pinRegister(Reg reg)
83	{	86	{
84	Vector<Reg>& regs = regsInPriorityOrderImpl(Arg(Tmp(reg)).bank());	87	Vector<Reg>& regs = regsInPriorityOrderImpl(Arg(Tmp(reg)).bank());
		88	ASSERT(regs.contains(reg));
85	regs.removeFirst(reg);	89	regs.removeFirst(reg);
86	m_mutableRegs.clear(reg);	90	m_mutableRegs.clear(reg);
87	ASSERT(!regs.contains(reg));	91	ASSERT(!regs.contains(reg));

Lines 81-86 struct PatchCustom { a/Source/JavaScriptCore/b3/air/AirCustom.h_sec1
- a/Source/JavaScriptCore/b3/air/AirCustom.h +27 lines
81	return inst.args[0].special()->admitsStack(inst, argIndex);	81	return inst.args[0].special()->admitsStack(inst, argIndex);
82	}	82	}
83		83
		84	static bool admitsExtendedOffsetAddr(Inst& inst, unsigned argIndex)
		85	{
		86	if (!argIndex)
		87	return false;
		88	return inst.args[0].special()->admitsExtendedOffsetAddr(inst, argIndex);
		89	}
		90
84	static std::optional<unsigned> shouldTryAliasingDef(Inst& inst)	91	static std::optional<unsigned> shouldTryAliasingDef(Inst& inst)
85	{	92	{
86	return inst.args[0].special()->shouldTryAliasingDef(inst);	93	return inst.args[0].special()->shouldTryAliasingDef(inst);
Lines 157-162 struct CCallCustom : public CommonCustomBase<CCallCustom> { a/Source/JavaScriptCore/b3/air/AirCustom.h_sec2
157	{	164	{
158	return true;	165	return true;
159	}	166	}
		167
		168	static bool admitsExtendedOffsetAddr(Inst&, unsigned)
		169	{
		170	return false;
		171	}
160		172
161	static bool isTerminal(Inst&)	173	static bool isTerminal(Inst&)
162	{	174	{
Lines 221-226 struct ShuffleCustom : public CommonCustomBase<ShuffleCustom> { a/Source/JavaScriptCore/b3/air/AirCustom.h_sec3
221	}	233	}
222	}	234	}
223		235
		236	static bool admitsExtendedOffsetAddr(Inst&, unsigned)
		237	{
		238	return false;
		239	}
		240
224	static bool isTerminal(Inst&)	241	static bool isTerminal(Inst&)
225	{	242	{
226	return false;	243	return false;
Lines 255-260 struct EntrySwitchCustom : public CommonCustomBase<EntrySwitchCustom> { a/Source/JavaScriptCore/b3/air/AirCustom.h_sec4
255	{	272	{
256	return false;	273	return false;
257	}	274	}
		275
		276	static bool admitsExtendedOffsetAddr(Inst&, unsigned)
		277	{
		278	return false;
		279	}
258		280
259	static bool isTerminal(Inst&)	281	static bool isTerminal(Inst&)
260	{	282	{
Lines 296-301 struct WasmBoundsCheckCustom : public CommonCustomBase<WasmBoundsCheckCustom> { a/Source/JavaScriptCore/b3/air/AirCustom.h_sec5
296	return false;	318	return false;
297	}	319	}
298		320
		321	static bool admitsExtendedOffsetAddr(Inst&, unsigned)
		322	{
		323	return false;
		324	}
		325
299	static bool isTerminal(Inst&)	326	static bool isTerminal(Inst&)
300	{	327	{
301	return false;	328	return false;



#include "AirReportUsedRegisters.h"
#include "AirSimplifyCFG.h"
#include "AirValidate.h"
#include "AllowMacroScratchRegisterUsage.h"
#include "B3Common.h"
#include "B3Procedure.h"
#include "B3TimingScope.h"

                disassembler->startEntrypoint(jit); 

            jit.emitFunctionPrologue();
            if (code.frameSize()) {
                AllowMacroScratchRegisterUsage allowScratch(jit);
                jit.addPtr(CCallHelpers::TrustedImm32(-code.frameSize()), MacroAssembler::stackPointerRegister);
            }
            
            for (const RegisterAtOffset& entry : code.calleeSaveRegisterAtOffsetList()) {
                if (entry.reg().isGPR())

Lines 167-173 struct Inst { a/Source/JavaScriptCore/b3/air/AirInst.h_sec1
- a/Source/JavaScriptCore/b3/air/AirInst.h -1 / +4 lines
167	// This function is auto-generated by opcode_generator.rb.	167	// This function is auto-generated by opcode_generator.rb.
168	bool admitsStack(unsigned argIndex);	168	bool admitsStack(unsigned argIndex);
169	bool admitsStack(Arg&);	169	bool admitsStack(Arg&);
170		170
		171	bool admitsExtendedOffsetAddr(unsigned argIndex);
		172	bool admitsExtendedOffsetAddr(Arg&);
		173
171	// Defined by opcode_generator.rb.	174	// Defined by opcode_generator.rb.
172	bool isTerminal();	175	bool isTerminal();
173		176

Lines 111-116 inline bool Inst::admitsStack(Arg& arg) a/Source/JavaScriptCore/b3/air/AirInstInlines.h_sec1
- a/Source/JavaScriptCore/b3/air/AirInstInlines.h +5 lines
111	return admitsStack(&arg - &args[0]);	111	return admitsStack(&arg - &args[0]);
112	}	112	}
113		113
		114	inline bool Inst::admitsExtendedOffsetAddr(Arg& arg)
		115	{
		116	return admitsExtendedOffsetAddr(&arg - &args[0]);
		117	}
		118
114	inline std::optional<unsigned> Inst::shouldTryAliasingDef()	119	inline std::optional<unsigned> Inst::shouldTryAliasingDef()
115	{	120	{
116	if (!isX86())	121	if (!isX86())



    // transformation since we can search the StackSlots array to figure out which StackSlot any
    // offset-from-FP refers to.

    // FIXME: This may produce addresses that aren't valid if we end up with a ginormous stack frame.
    // We would have to scavenge for temporaries if this happened. Fortunately, this case will be
    // extremely rare so we can do crazy things when it arises.
    // https://bugs.webkit.org/show_bug.cgi?id=152530

    InsertionSet insertionSet(code);
    for (BasicBlock* block : code) {
        // FIXME We can keep track of the last large offset which was materialized in this block, and reuse the register
        // if it hasn't been clobbered instead of renetating imm+add+addr every time. https://bugs.webkit.org/show_bug.cgi?id=171387

        for (unsigned instIndex = 0; instIndex < block->size(); ++instIndex) {
            Inst& inst = block->at(instIndex);
            bool isPatch = inst.kind.opcode == Patch;

            inst.forEachArg(
                [&] (Arg& arg, Arg::Role role, Bank, Width width) {
                    auto stackAddr = [&] (Value::OffsetType offsetFromFP) -> Arg {
                        int32_t offsetFromSP = offsetFromFP + code.frameSize();

                        if (isPatch && inst.admitsExtendedOffsetAddr(arg)) {
                            // Stackmaps and patchpoints expect addr inputs relative to SP or FP only. We might as well
                            // not even bother generating an addr with valid form for these opcodes since extended offset
                            // addr is always valid.
                            return Arg::extendedOffsetAddr(offsetFromFP);
                        }

                        Arg result = Arg::addr(Air::Tmp(GPRInfo::callFrameRegister), offsetFromFP);
                        if (result.isValidForm(width))
                            return result;

                        result = Arg::addr(Air::Tmp(MacroAssembler::stackPointerRegister), offsetFromSP);
                        if (result.isValidForm(width))
                            return result;
#if CPU(ARM64)
                        ASSERT(pinnedExtendedOffsetAddrRegister());
                        Air::Tmp tmp = Air::Tmp(*pinnedExtendedOffsetAddrRegister());

                        Arg largeOffset = Arg::isValidImmForm(offsetFromSP) ? Arg::imm(offsetFromSP) : Arg::bigImm(offsetFromSP);
                        insertionSet.insert(instIndex, Move, inst.origin, largeOffset, tmp);
                        insertionSet.insert(instIndex, Add64, inst.origin, Air::Tmp(MacroAssembler::stackPointerRegister), tmp);
                        result = Arg::addr(tmp, 0);
                        return result;
#elif CPU(X86_64)
                        // Can't happen on x86: immediates are always big enough for frame size.
                        RELEASE_ASSERT_NOT_REACHED();
#else
#error Unhandled architecture.
#endif
                    };
                    
                    switch (arg.kind()) {

Lines 56-61 bool PrintSpecial::admitsStack(Inst&, unsigned) a/Source/JavaScriptCore/b3/air/AirPrintSpecial.cpp_sec1
- a/Source/JavaScriptCore/b3/air/AirPrintSpecial.cpp +6 lines
56	return false;	56	return false;
57	}	57	}
58		58
		59	bool PrintSpecial::admitsExtendedOffsetAddr(Inst&, unsigned)
		60	{
		61	return false;
		62	}
		63
59	void PrintSpecial::reportUsedRegisters(Inst&, const RegisterSet&)	64	void PrintSpecial::reportUsedRegisters(Inst&, const RegisterSet&)
60	{	65	{
61	}	66	}
Lines 71-76 CCallHelpers::Jump PrintSpecial::generate(Inst& inst, CCallHelpers& jit, Generat a/Source/JavaScriptCore/b3/air/AirPrintSpecial.cpp_sec2
71	term = Printer::Printer<MacroAssembler::RegisterID>(arg.gpr());	76	term = Printer::Printer<MacroAssembler::RegisterID>(arg.gpr());
72	break;	77	break;
73	case Arg::Addr:	78	case Arg::Addr:
		79	case Arg::ExtendedOffsetAddr:
74	term = Printer::Printer<MacroAssembler::Address>(arg.asAddress());	80	term = Printer::Printer<MacroAssembler::Address>(arg.asAddress());
75	break;	81	break;
76	default:	82	default:



    static const GPRReg scratchRegister = GPRInfo::nonArgGPR0;
    
protected:
    void forEachArg(Inst&, const ScopedLambda<Inst::EachArgCallback>&) final;
    bool isValid(Inst&) final;
    bool admitsStack(Inst&, unsigned argIndex) final;
    bool admitsExtendedOffsetAddr(Inst&, unsigned) final;
    void reportUsedRegisters(Inst&, const RegisterSet&) final;
    CCallHelpers::Jump generate(Inst&, CCallHelpers&, GenerationContext&) final;
    RegisterSet extraEarlyClobberedRegs(Inst&) final;
    RegisterSet extraClobberedRegs(Inst&) final;
    
    void dumpImpl(PrintStream&) const final;
    void deepDumpImpl(PrintStream&) const final;
    
private:
    static const unsigned specialArgOffset = 0;

Lines 55-60 public: a/Source/JavaScriptCore/b3/air/AirSpecial.h_sec1
- a/Source/JavaScriptCore/b3/air/AirSpecial.h +1 lines
55	virtual void forEachArg(Inst&, const ScopedLambda<Inst::EachArgCallback>&) = 0;	55	virtual void forEachArg(Inst&, const ScopedLambda<Inst::EachArgCallback>&) = 0;
56	virtual bool isValid(Inst&) = 0;	56	virtual bool isValid(Inst&) = 0;
57	virtual bool admitsStack(Inst&, unsigned argIndex) = 0;	57	virtual bool admitsStack(Inst&, unsigned argIndex) = 0;
		58	virtual bool admitsExtendedOffsetAddr(Inst&, unsigned argIndex) = 0;
58	virtual std::optional<unsigned> shouldTryAliasingDef(Inst&);	59	virtual std::optional<unsigned> shouldTryAliasingDef(Inst&);
59		60
60	// This gets called on for each Inst that uses this Special. Note that there is no way to	61	// This gets called on for each Inst that uses this Special. Note that there is no way to



end

def isKind(token)
    token =~ /\A((Tmp)|(Imm)|(BigImm)|(BitImm)|(BitImm64)|(SimpleAddr)|(Addr)|(ExtendedOffsetAddr)|(Index)|(RelCond)|(ResCond)|(DoubleCond)|(StatusCond))\Z/
end

def isArch(token)


    def consumeKind
        result = token.string
        parseError("Expected kind (Imm, BigImm, BitImm, BitImm64, Tmp, SimpleAddr, Addr, ExtendedOffsetAddr, Index, RelCond, ResCond, DoubleCond, or StatusCond)") unless isKind(result)
        advance
        result
    end

                        outp.puts "if (args[#{index}].isStack() && args[#{index}].stackSlot()->isSpill())"
                        outp.puts "OPGEN_RETURN(false);"
                    end
                    
                    outp.puts "if (!Arg::isValidAddrForm(args[#{index}].offset()))"
                    outp.puts "OPGEN_RETURN(false);"
                when "ExtendedOffsetAddr"
                    if arg.role == "UA"
                        outp.puts "if (args[#{index}].isStack() && args[#{index}].stackSlot()->isSpill())"
                        outp.puts "OPGEN_RETURN(false);"
                    end
                when "Index"
                    outp.puts "if (!Arg::isValidIndexForm(args[#{index}].scale(), args[#{index}].offset(), #{arg.widthCode}))"
                    outp.puts "OPGEN_RETURN(false);"

    outp.puts "return false;"
    outp.puts "}"

    outp.puts "bool Inst::admitsExtendedOffsetAddr(unsigned argIndex)"
    outp.puts "{"
    outp.puts "switch (kind.opcode) {"
    $opcodes.values.each {
        | opcode |
        if opcode.custom
            outp.puts "case Opcode::#{opcode.name}:"
            outp.puts "OPGEN_RETURN(#{opcode.name}Custom::admitsExtendedOffsetAddr(*this, argIndex));"
            outp.puts "break;"
        end
    }
    outp.puts "default:"
    outp.puts "break;"
    outp.puts "}"
    outp.puts "return false;"
    outp.puts "}"


    outp.puts "bool Inst::isTerminal()"
    outp.puts "{"
    outp.puts "switch (kind.opcode) {"

                    outp.print "args[#{index}].asTrustedImm32()"
                when "BigImm", "BitImm64"
                    outp.print "args[#{index}].asTrustedImm64()"
                when "SimpleAddr", "Addr", "ExtendedOffsetAddr"
                    outp.print "args[#{index}].asAddress()"
                when "Index"
                    outp.print "args[#{index}].asBaseIndex()"
- 

Return to Bug 170215

Lines 1-3 a/Source/JavaScriptCore/ChangeLog_sec1
- a/Source/JavaScriptCore/ChangeLog +120 lines
		1	2017-04-28 JF Bastien <jfbastien@apple.com>
		2
		3	WebAssembly: Air::Inst::generate crashes on large binary on A64
		4	https://bugs.webkit.org/show_bug.cgi?id=170215
		5
		6	Reviewed by NOBODY (OOPS!).
		7
		8	ARM can't encode all offsets in a single instruction. We usualy
		9	handle this type of detail early, or the macro assembler uses a
		10	scratch register to take care of the large immediate. After
		11	register allocation we assumed that we would never get large
		12	offsets, and asserted this was the case. That was a fine
		13	assumption with JavaScript, but WebAssembly ends up generating
		14	stack frames which are too big to encode.
		15
		16	There are two places that needed to be fixed:
		17	1. AirGenerate
		18	2. AirLowerStackArgs
		19
		20	We now unconditionally pin the dataTempRegister on ARM64, and use
		21	it when immediates don't fit.
		22
		23	Number 1. is easy: we're just incrementing SP, make sure we can
		24	use a scratch register when that happens.
		25
		26	Number 2. is more complex: not all Inst can receive a stack
		27	argument whose base register isn't SP or FP. Specifically,
		28	Patchpoints and Stackmaps get very sad because they just want to
		29	know the offset value, but when we materialize the offset as
		30	follows:
		31
		32	Move (spill337), (spill201), %r0, @8735
		33
		34	Becomes (where %r16 is dataTempRegister):
		35	Move $1404, %r16, @8736
		36	Add64 %sp, %r16, @8736
		37	Move (%r16), 2032(%sp), %r0, @8736
		38
		39	The code currently doesn't see through our little dance. To work
		40	around this issue we introduce a new Air Arg kind:
		41	ExtendedOffsetAddr. This is the same as a regular Addr, but with
		42	an offset which may be too big to encode. Opcodes then declare
		43	whether their arguments can handle such inputs, and if so we
		44	generate them, otherwise we generate Addr as shown above.
		45
		46	None of this affects x86 because it can always encode large
		47	immediates.
		48
		49	This patch also drive-by converts some uses of `override` to
		50	`final`. It makes the code easier to grok, and maybe helps the
		51	optimizer sometimes but really that doens't matter.
		52
		53	* assembler/MacroAssembler.h:
		54	* assembler/MacroAssemblerARM64.h:
		55	* b3/B3CheckSpecial.cpp:
		56	(JSC::B3::CheckSpecial::admitsExtendedOffsetAddr):
		57	* b3/B3CheckSpecial.h:
		58	* b3/B3Common.cpp:
		59	(JSC::B3::pinnedExtendedOffsetAddrRegister): keep the CPU-specific
		60	pinning information in a cpp file
		61	* b3/B3Common.h:
		62	* b3/B3PatchpointSpecial.cpp:
		63	(JSC::B3::PatchpointSpecial::admitsExtendedOffsetAddr):
		64	* b3/B3PatchpointSpecial.h:
65	* b3/B3StackmapSpecial.cpp:
66	(JSC::B3::StackmapSpecial::isArgValidForRep):
67	(JSC::B3::StackmapSpecial::repForArg):
68	* b3/B3StackmapSpecial.h:
69	* b3/air/AirArg.cpp:
70	(JSC::B3::Air::Arg::isStackMemory):
71	(JSC::B3::Air::Arg::jsHash):
72	(JSC::B3::Air::Arg::dump):
73	(WTF::printInternal):
74	(JSC::B3::Air::Arg::stackAddrImpl): Deleted. There was only one
75	use of this (in AirLowerStackArgs) and it was now confusing to
76	split the logic up between these two. Inline the code that used to
77	be here into its one usepoint instead.
78	* b3/air/AirArg.h:
79	(JSC::B3::Air::Arg::extendedOffsetAddr):
80	(JSC::B3::Air::Arg::isExtendedOffsetAddr):
81	(JSC::B3::Air::Arg::isMemory):
82	(JSC::B3::Air::Arg::base):
83	(JSC::B3::Air::Arg::offset):
84	(JSC::B3::Air::Arg::isGP):
85	(JSC::B3::Air::Arg::isFP):
86	(JSC::B3::Air::Arg::isValidForm):
87	(JSC::B3::Air::Arg::forEachTmpFast):
88	(JSC::B3::Air::Arg::forEachTmp):
89	(JSC::B3::Air::Arg::asAddress):
90	(JSC::B3::Air::Arg::stackAddr): Deleted.
91	* b3/air/AirCCallSpecial.cpp:
92	(JSC::B3::Air::CCallSpecial::isValid):
93	(JSC::B3::Air::CCallSpecial::admitsExtendedOffsetAddr):
94	(JSC::B3::Air::CCallSpecial::generate):
95	* b3/air/AirCCallSpecial.h:
96	* b3/air/AirCode.cpp:
97	(JSC::B3::Air::Code::Code):
98	(JSC::B3::Air::Code::pinRegister): Check that the register wasn't
99	pinned before pinning it. It's likely a bug to pin the same
100	register twice.
101	* b3/air/AirCustom.h:
102	(JSC::B3::Air::PatchCustom::admitsExtendedOffsetAddr):
103	(JSC::B3::Air::CCallCustom::admitsExtendedOffsetAddr):
104	(JSC::B3::Air::ShuffleCustom::admitsExtendedOffsetAddr):
105	(JSC::B3::Air::EntrySwitchCustom::admitsExtendedOffsetAddr):
106	(JSC::B3::Air::WasmBoundsCheckCustom::admitsExtendedOffsetAddr):
107	* b3/air/AirGenerate.cpp:
108	(JSC::B3::Air::generate):
109	* b3/air/AirInst.h:
110	* b3/air/AirInstInlines.h:
111	(JSC::B3::Air::Inst::admitsExtendedOffsetAddr):
112	* b3/air/AirLowerStackArgs.cpp:
113	(JSC::B3::Air::lowerStackArgs):
114	* b3/air/AirPrintSpecial.cpp:
115	(JSC::B3::Air::PrintSpecial::admitsExtendedOffsetAddr):
116	(JSC::B3::Air::PrintSpecial::generate):
117	* b3/air/AirPrintSpecial.h:
118	* b3/air/AirSpecial.h:
119	* b3/air/opcode_generator.rb:
120
1	2017-04-28 Mark Lam <mark.lam@apple.com>	121	2017-04-28 Mark Lam <mark.lam@apple.com>
2		122
3	Revert instrumentation from https://bugs.webkit.org/show_bug.cgi?id=170086 that is no longer needed.	123	Revert instrumentation from https://bugs.webkit.org/show_bug.cgi?id=170086 that is no longer needed.

Lines 39-46 public: a/Source/JavaScriptCore/assembler/MacroAssemblerARM64.h_sec1
- a/Source/JavaScriptCore/assembler/MacroAssemblerARM64.h -2 / +2 lines
39	static const unsigned numGPRs = 32;	39	static const unsigned numGPRs = 32;
40	static const unsigned numFPRs = 32;	40	static const unsigned numFPRs = 32;
41		41
42	static const RegisterID dataTempRegister = ARM64Registers::ip0;	42	static constexpr RegisterID dataTempRegister = ARM64Registers::ip0;
43	static const RegisterID memoryTempRegister = ARM64Registers::ip1;	43	static constexpr RegisterID memoryTempRegister = ARM64Registers::ip1;
44		44
45	RegisterID scratchRegister()	45	RegisterID scratchRegister()
46	{	46	{

Lines 120-137 protected: a/Source/JavaScriptCore/b3/B3CheckSpecial.h_sec1
- a/Source/JavaScriptCore/b3/B3CheckSpecial.h -7 / +8 lines
120	// Constructs and returns the Inst representing the branch that this will use.	120	// Constructs and returns the Inst representing the branch that this will use.
121	Air::Inst hiddenBranch(const Air::Inst&) const;	121	Air::Inst hiddenBranch(const Air::Inst&) const;
122		122
123	void forEachArg(Air::Inst&, const ScopedLambda<Air::Inst::EachArgCallback>&) override;	123	void forEachArg(Air::Inst&, const ScopedLambda<Air::Inst::EachArgCallback>&) final;
124	bool isValid(Air::Inst&) override;	124	bool isValid(Air::Inst&) final;
125	bool admitsStack(Air::Inst&, unsigned argIndex) override;	125	bool admitsStack(Air::Inst&, unsigned argIndex) final;
126	std::optional<unsigned> shouldTryAliasingDef(Air::Inst&) override;	126	bool admitsExtendedOffsetAddr(Air::Inst&, unsigned) final;
		127	std::optional<unsigned> shouldTryAliasingDef(Air::Inst&) final;
127		128
128	// NOTE: the generate method will generate the hidden branch and then register a LatePath that	129	// NOTE: the generate method will generate the hidden branch and then register a LatePath that
129	// generates the stackmap. Super crazy dude!	130	// generates the stackmap. Super crazy dude!
130		131
131	CCallHelpers::Jump generate(Air::Inst&, CCallHelpers&, Air::GenerationContext&) override;	132	CCallHelpers::Jump generate(Air::Inst&, CCallHelpers&, Air::GenerationContext&) final;
132		133
133	void dumpImpl(PrintStream&) const override;	134	void dumpImpl(PrintStream&) const final;
134	void deepDumpImpl(PrintStream&) const override;	135	void deepDumpImpl(PrintStream&) const final;
135		136
136	private:	137	private:
137	Air::Kind m_checkKind;	138	Air::Kind m_checkKind;

Lines 70-75 bool shouldMeasurePhaseTiming() a/Source/JavaScriptCore/b3/B3Common.cpp_sec1
- a/Source/JavaScriptCore/b3/B3Common.cpp +11 lines
70	return Options::logB3PhaseTimes();	70	return Options::logB3PhaseTimes();
71	}	71	}
72		72
		73	std::optional<GPRReg> pinnedExtendedOffsetAddrRegister()
		74	{
		75	#if CPU(ARM64)
		76	return static_cast<GPRReg>(+MacroAssembler::dataTempRegister);
		77	#elif CPU(X86_64)
		78	return std::nullopt;
		79	#else
		80	#error Unhandled architecture.
		81	#endif
		82	}
		83
73	} } // namespace JSC::B3	84	} } // namespace JSC::B3
74		85
75	#endif // ENABLE(B3_JIT)	86	#endif // ENABLE(B3_JIT)

Lines 47-65 public: a/Source/JavaScriptCore/b3/B3PatchpointSpecial.h_sec1
- a/Source/JavaScriptCore/b3/B3PatchpointSpecial.h -7 / +8 lines
47	virtual ~PatchpointSpecial();	47	virtual ~PatchpointSpecial();
48		48
49	protected:	49	protected:
50	void forEachArg(Air::Inst&, const ScopedLambda<Air::Inst::EachArgCallback>&) override;	50	void forEachArg(Air::Inst&, const ScopedLambda<Air::Inst::EachArgCallback>&) final;
51	bool isValid(Air::Inst&) override;	51	bool isValid(Air::Inst&) final;
52	bool admitsStack(Air::Inst&, unsigned argIndex) override;	52	bool admitsStack(Air::Inst&, unsigned argIndex) final;
		53	bool admitsExtendedOffsetAddr(Air::Inst&, unsigned) final;
53		54
54	// NOTE: the generate method will generate the hidden branch and then register a LatePath that	55	// NOTE: the generate method will generate the hidden branch and then register a LatePath that
55	// generates the stackmap. Super crazy dude!	56	// generates the stackmap. Super crazy dude!
56		57
57	CCallHelpers::Jump generate(Air::Inst&, CCallHelpers&, Air::GenerationContext&) override;	58	CCallHelpers::Jump generate(Air::Inst&, CCallHelpers&, Air::GenerationContext&) final;
58		59
59	bool isTerminal(Air::Inst&) override;	60	bool isTerminal(Air::Inst&) final;
60		61
61	void dumpImpl(PrintStream&) const override;	62	void dumpImpl(PrintStream&) const final;
62	void deepDumpImpl(PrintStream&) const override;	63	void deepDumpImpl(PrintStream&) const final;
63	};	64	};
64		65
65	} } // namespace JSC::B3	66	} } // namespace JSC::B3

Lines 41-65 a/Source/JavaScriptCore/b3/air/AirArg.cpp_sec1
- a/Source/JavaScriptCore/b3/air/AirArg.cpp -13 / +6 lines
41		41
42	namespace JSC { namespace B3 { namespace Air {	42	namespace JSC { namespace B3 { namespace Air {
43		43
44	Arg Arg::stackAddrImpl(int32_t offsetFromFP, unsigned frameSize, Width width)
45	{
46	Arg result = Arg::addr(Air::Tmp(GPRInfo::callFrameRegister), offsetFromFP);
47	if (!result.isValidForm(width)) {
48	result = Arg::addr(
49	Air::Tmp(MacroAssembler::stackPointerRegister),
50	offsetFromFP + frameSize);
51	if (!result.isValidForm(width))
52	result = Arg();
53	}
54	return result;
55	}
56
57	bool Arg::isStackMemory() const	44	bool Arg::isStackMemory() const
58	{	45	{
59	switch (kind()) {	46	switch (kind()) {
60	case Addr:	47	case Addr:
61	return base() == Air::Tmp(GPRInfo::callFrameRegister)	48	return base() == Air::Tmp(GPRInfo::callFrameRegister)
62	\|\| base() == Air::Tmp(MacroAssembler::stackPointerRegister);	49	\|\| base() == Air::Tmp(MacroAssembler::stackPointerRegister);
		50	case ExtendedOffsetAddr:
63	case Stack:	51	case Stack:
64	case CallArg:	52	case CallArg:
65	return true;	53	return true;
Lines 128-133 unsigned Arg::jsHash() const a/Source/JavaScriptCore/b3/air/AirArg.cpp_sec2
128	result += m_base.internalValue();	116	result += m_base.internalValue();
129	break;	117	break;
130	case Addr:	118	case Addr:
		119	case ExtendedOffsetAddr:
131	result += m_offset;	120	result += m_offset;
132	result += m_base.internalValue();	121	result += m_base.internalValue();
133	break;	122	break;
Lines 171-176 void Arg::dump(PrintStream& out) const a/Source/JavaScriptCore/b3/air/AirArg.cpp_sec3
171	out.print("(", base(), ")");	160	out.print("(", base(), ")");
172	return;	161	return;
173	case Addr:	162	case Addr:
		163	case ExtendedOffsetAddr:
174	if (offset())	164	if (offset())
175	out.print(offset());	165	out.print(offset());
176	out.print("(", base(), ")");	166	out.print("(", base(), ")");
Lines 249-254 void printInternal(PrintStream& out, Arg::Kind kind) a/Source/JavaScriptCore/b3/air/AirArg.cpp_sec4
249	case Arg::Addr:	239	case Arg::Addr:
250	out.print("Addr");	240	out.print("Addr");
251	return;	241	return;
		242	case Arg::ExtendedOffsetAddr:
		243	out.print("ExtendedOffsetAddr");
		244	return;
252	case Arg::Stack:	245	case Arg::Stack:
253	out.print("Stack");	246	out.print("Stack");
254	return;	247	return;

Lines 52-67 public: a/Source/JavaScriptCore/b3/air/AirCCallSpecial.h_sec1
- a/Source/JavaScriptCore/b3/air/AirCCallSpecial.h -9 / +10 lines
52	static const GPRReg scratchRegister = GPRInfo::nonArgGPR0;	52	static const GPRReg scratchRegister = GPRInfo::nonArgGPR0;
53		53
54	protected:	54	protected:
55	void forEachArg(Inst&, const ScopedLambda<Inst::EachArgCallback>&) override;	55	void forEachArg(Inst&, const ScopedLambda<Inst::EachArgCallback>&) final;
56	bool isValid(Inst&) override;	56	bool isValid(Inst&) final;
57	bool admitsStack(Inst&, unsigned argIndex) override;	57	bool admitsStack(Inst&, unsigned argIndex) final;
58	void reportUsedRegisters(Inst&, const RegisterSet&) override;	58	bool admitsExtendedOffsetAddr(Inst&, unsigned) final;
59	CCallHelpers::Jump generate(Inst&, CCallHelpers&, GenerationContext&) override;	59	void reportUsedRegisters(Inst&, const RegisterSet&) final;
60	RegisterSet extraEarlyClobberedRegs(Inst&) override;	60	CCallHelpers::Jump generate(Inst&, CCallHelpers&, GenerationContext&) final;
61	RegisterSet extraClobberedRegs(Inst&) override;	61	RegisterSet extraEarlyClobberedRegs(Inst&) final;
		62	RegisterSet extraClobberedRegs(Inst&) final;
62		63
63	void dumpImpl(PrintStream&) const override;	64	void dumpImpl(PrintStream&) const final;
64	void deepDumpImpl(PrintStream&) const override;	65	void deepDumpImpl(PrintStream&) const final;
65		66
66	private:	67	private:
67	static const unsigned specialArgOffset = 0;	68	static const unsigned specialArgOffset = 0;

Lines 46-51 a/Source/JavaScriptCore/b3/air/AirGenerate.cpp_sec1
- a/Source/JavaScriptCore/b3/air/AirGenerate.cpp -1 / +4 lines
46	#include "AirReportUsedRegisters.h"	46	#include "AirReportUsedRegisters.h"
47	#include "AirSimplifyCFG.h"	47	#include "AirSimplifyCFG.h"
48	#include "AirValidate.h"	48	#include "AirValidate.h"
		49	#include "AllowMacroScratchRegisterUsage.h"
49	#include "B3Common.h"	50	#include "B3Common.h"
50	#include "B3Procedure.h"	51	#include "B3Procedure.h"
51	#include "B3TimingScope.h"	52	#include "B3TimingScope.h"
Lines 216-223 void generate(Code& code, CCallHelpers& jit) a/Source/JavaScriptCore/b3/air/AirGenerate.cpp_sec2
216	disassembler->startEntrypoint(jit);	217	disassembler->startEntrypoint(jit);
217		218
218	jit.emitFunctionPrologue();	219	jit.emitFunctionPrologue();
219	if (code.frameSize())	220	if (code.frameSize()) {
		221	AllowMacroScratchRegisterUsage allowScratch(jit);
220	jit.addPtr(CCallHelpers::TrustedImm32(-code.frameSize()), MacroAssembler::stackPointerRegister);	222	jit.addPtr(CCallHelpers::TrustedImm32(-code.frameSize()), MacroAssembler::stackPointerRegister);
		223	}
221		224
222	for (const RegisterAtOffset& entry : code.calleeSaveRegisterAtOffsetList()) {	225	for (const RegisterAtOffset& entry : code.calleeSaveRegisterAtOffsetList()) {
223	if (entry.reg().isGPR())	226	if (entry.reg().isGPR())

Lines 63-89 void lowerStackArgs(Code& code) a/Source/JavaScriptCore/b3/air/AirLowerStackArgs.cpp_sec1
- a/Source/JavaScriptCore/b3/air/AirLowerStackArgs.cpp -13 / +35 lines
63	// transformation since we can search the StackSlots array to figure out which StackSlot any	63	// transformation since we can search the StackSlots array to figure out which StackSlot any
64	// offset-from-FP refers to.	64	// offset-from-FP refers to.
65		65
66	// FIXME: This may produce addresses that aren't valid if we end up with a ginormous stack frame.
67	// We would have to scavenge for temporaries if this happened. Fortunately, this case will be
68	// extremely rare so we can do crazy things when it arises.
69	// https://bugs.webkit.org/show_bug.cgi?id=152530
70
71	InsertionSet insertionSet(code);	66	InsertionSet insertionSet(code);
72	for (BasicBlock* block : code) {	67	for (BasicBlock* block : code) {
		68	// FIXME We can keep track of the last large offset which was materialized in this block, and reuse the register
		69	// if it hasn't been clobbered instead of renetating imm+add+addr every time. https://bugs.webkit.org/show_bug.cgi?id=171387
		70
73	for (unsigned instIndex = 0; instIndex < block->size(); ++instIndex) {	71	for (unsigned instIndex = 0; instIndex < block->size(); ++instIndex) {
74	Inst& inst = block->at(instIndex);	72	Inst& inst = block->at(instIndex);
		73	bool isPatch = inst.kind.opcode == Patch;
		74
75	inst.forEachArg(	75	inst.forEachArg(
76	[&] (Arg& arg, Arg::Role role, Bank, Width width) {	76	[&] (Arg& arg, Arg::Role role, Bank, Width width) {
77	auto stackAddr = [&] (Value::OffsetType offset) -> Arg {	77	auto stackAddr = [&] (Value::OffsetType offsetFromFP) -> Arg {
78	Arg result = Arg::stackAddr(offset, code.frameSize(), width);	78	int32_t offsetFromSP = offsetFromFP + code.frameSize();
79	if (!result) {	79
80	dataLog("FATAL: Could not create stack reference for offset = ", offset, " and width = ", width, "\n");	80	if (isPatch && inst.admitsExtendedOffsetAddr(arg)) {
81	dataLog("Code:\n");	81	// Stackmaps and patchpoints expect addr inputs relative to SP or FP only. We might as well
82	dataLog(code);	82	// not even bother generating an addr with valid form for these opcodes since extended offset
83	dataLog("FATAL: Could not create stack reference for offset = ", offset, " and width = ", width, "\n");	83	// addr is always valid.
84	RELEASE_ASSERT_NOT_REACHED();	84	return Arg::extendedOffsetAddr(offsetFromFP);
85	}	85	}
		86
		87	Arg result = Arg::addr(Air::Tmp(GPRInfo::callFrameRegister), offsetFromFP);
		88	if (result.isValidForm(width))
		89	return result;
		90
		91	result = Arg::addr(Air::Tmp(MacroAssembler::stackPointerRegister), offsetFromSP);
		92	if (result.isValidForm(width))
		93	return result;
		94	#if CPU(ARM64)
		95	ASSERT(pinnedExtendedOffsetAddrRegister());
		96	Air::Tmp tmp = Air::Tmp(*pinnedExtendedOffsetAddrRegister());
		97
		98	Arg largeOffset = Arg::isValidImmForm(offsetFromSP) ? Arg::imm(offsetFromSP) : Arg::bigImm(offsetFromSP);
		99	insertionSet.insert(instIndex, Move, inst.origin, largeOffset, tmp);
		100	insertionSet.insert(instIndex, Add64, inst.origin, Air::Tmp(MacroAssembler::stackPointerRegister), tmp);
		101	result = Arg::addr(tmp, 0);
86	return result;	102	return result;
		103	#elif CPU(X86_64)
		104	// Can't happen on x86: immediates are always big enough for frame size.
		105	RELEASE_ASSERT_NOT_REACHED();
		106	#else
		107	#error Unhandled architecture.
		108	#endif
87	};	109	};
88		110
89	switch (arg.kind()) {	111	switch (arg.kind()) {

Lines 229-235 def isGF(token) a/Source/JavaScriptCore/b3/air/opcode_generator.rb_sec1
- a/Source/JavaScriptCore/b3/air/opcode_generator.rb -5 / +26 lines
229	end	229	end
230		230
231	def isKind(token)	231	def isKind(token)
232	token =~ /\A((Tmp)\|(Imm)\|(BigImm)\|(BitImm)\|(BitImm64)\|(SimpleAddr)\|(Addr)\|(Index)\|(RelCond)\|(ResCond)\|(DoubleCond)\|(StatusCond))\Z/	232	token =~ /\A((Tmp)\|(Imm)\|(BigImm)\|(BitImm)\|(BitImm64)\|(SimpleAddr)\|(Addr)\|(ExtendedOffsetAddr)\|(Index)\|(RelCond)\|(ResCond)\|(DoubleCond)\|(StatusCond))\Z/
233	end	233	end
234		234
235	def isArch(token)	235	def isArch(token)
Lines 303-309 class Parser a/Source/JavaScriptCore/b3/air/opcode_generator.rb_sec2
303		303
304	def consumeKind	304	def consumeKind
305	result = token.string	305	result = token.string
306	parseError("Expected kind (Imm, BigImm, BitImm, BitImm64, Tmp, SimpleAddr, Addr, Index, RelCond, ResCond, DoubleCond, or StatusCond)") unless isKind(result)	306	parseError("Expected kind (Imm, BigImm, BitImm, BitImm64, Tmp, SimpleAddr, Addr, ExtendedOffsetAddr, Index, RelCond, ResCond, DoubleCond, or StatusCond)") unless isKind(result)
307	advance	307	advance
308	result	308	result
309	end	309	end
Lines 908-916 writeH("OpcodeGenerated") { a/Source/JavaScriptCore/b3/air/opcode_generator.rb_sec3
908	outp.puts "if (args[#{index}].isStack() && args[#{index}].stackSlot()->isSpill())"	908	outp.puts "if (args[#{index}].isStack() && args[#{index}].stackSlot()->isSpill())"
909	outp.puts "OPGEN_RETURN(false);"	909	outp.puts "OPGEN_RETURN(false);"
910	end	910	end
911
912	outp.puts "if (!Arg::isValidAddrForm(args[#{index}].offset()))"	911	outp.puts "if (!Arg::isValidAddrForm(args[#{index}].offset()))"
913	outp.puts "OPGEN_RETURN(false);"	912	outp.puts "OPGEN_RETURN(false);"
		913	when "ExtendedOffsetAddr"
		914	if arg.role == "UA"
		915	outp.puts "if (args[#{index}].isStack() && args[#{index}].stackSlot()->isSpill())"
		916	outp.puts "OPGEN_RETURN(false);"
		917	end
914	when "Index"	918	when "Index"
915	outp.puts "if (!Arg::isValidIndexForm(args[#{index}].scale(), args[#{index}].offset(), #{arg.widthCode}))"	919	outp.puts "if (!Arg::isValidIndexForm(args[#{index}].scale(), args[#{index}].offset(), #{arg.widthCode}))"
916	outp.puts "OPGEN_RETURN(false);"	920	outp.puts "OPGEN_RETURN(false);"
Lines 1076-1081 writeH("OpcodeGenerated") { a/Source/JavaScriptCore/b3/air/opcode_generator.rb_sec4
1076	outp.puts "return false;"	1080	outp.puts "return false;"
1077	outp.puts "}"	1081	outp.puts "}"
1078		1082
		1083	outp.puts "bool Inst::admitsExtendedOffsetAddr(unsigned argIndex)"
		1084	outp.puts "{"
		1085	outp.puts "switch (kind.opcode) {"
		1086	$opcodes.values.each {
		1087	\| opcode \|
		1088	if opcode.custom
		1089	outp.puts "case Opcode::#{opcode.name}:"
		1090	outp.puts "OPGEN_RETURN(#{opcode.name}Custom::admitsExtendedOffsetAddr(*this, argIndex));"
		1091	outp.puts "break;"
		1092	end
		1093	}
		1094	outp.puts "default:"
		1095	outp.puts "break;"
		1096	outp.puts "}"
		1097	outp.puts "return false;"
		1098	outp.puts "}"
		1099
		1100
1079	outp.puts "bool Inst::isTerminal()"	1101	outp.puts "bool Inst::isTerminal()"
1080	outp.puts "{"	1102	outp.puts "{"
1081	outp.puts "switch (kind.opcode) {"	1103	outp.puts "switch (kind.opcode) {"
Lines 1195-1201 writeH("OpcodeGenerated") { a/Source/JavaScriptCore/b3/air/opcode_generator.rb_sec5
1195	outp.print "args[#{index}].asTrustedImm32()"	1217	outp.print "args[#{index}].asTrustedImm32()"
1196	when "BigImm", "BitImm64"	1218	when "BigImm", "BitImm64"
1197	outp.print "args[#{index}].asTrustedImm64()"	1219	outp.print "args[#{index}].asTrustedImm64()"
1198	when "SimpleAddr", "Addr"	1220	when "SimpleAddr", "Addr", "ExtendedOffsetAddr"
1199	outp.print "args[#{index}].asAddress()"	1221	outp.print "args[#{index}].asAddress()"
1200	when "Index"	1222	when "Index"
1201	outp.print "args[#{index}].asBaseIndex()"	1223	outp.print "args[#{index}].asBaseIndex()"
1202	-