-
- Downloads
[AArch64][SME] Spill p-regs as z-regs when streaming hazards are possible (#123752)
This patch adds a new option `-aarch64-enable-zpr-predicate-spills` (which is disabled by default), this option replaces predicate spills with vector spills in streaming[-compatible] functions. For example: ``` str p8, [sp, #7, mul vl] // 2-byte Folded Spill // ... ldr p8, [sp, #7, mul vl] // 2-byte Folded Reload ``` Becomes: ``` mov z0.b, p8/z, #1 str z0, [sp] // 16-byte Folded Spill // ... ldr z0, [sp] // 16-byte Folded Reload ptrue p4.b cmpne p8.b, p4/z, z0.b, #0 ``` This is done to avoid streaming memory hazards between FPR/vector and predicate spills, which currently occupy the same stack area even when the `-aarch64-stack-hazard-size` flag is set. This is implemented with two new pseudos SPILL_PPR_TO_ZPR_SLOT_PSEUDO and FILL_PPR_FROM_ZPR_SLOT_PSEUDO. The expansion of these pseudos handles scavenging the required registers (z0 in the above example) and, in the worst case spilling a register to an emergency stack slot in the expansion. The condition flags are also preserved around the `cmpne` in case they are live at the expansion point.
Showing
- llvm/lib/Target/AArch64/AArch64FrameLowering.cpp 308 additions, 5 deletionsllvm/lib/Target/AArch64/AArch64FrameLowering.cpp
- llvm/lib/Target/AArch64/AArch64InstrInfo.cpp 15 additions, 1 deletionllvm/lib/Target/AArch64/AArch64InstrInfo.cpp
- llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp 2 additions, 2 deletionsllvm/lib/Target/AArch64/AArch64RegisterInfo.cpp
- llvm/lib/Target/AArch64/AArch64RegisterInfo.h 1 addition, 1 deletionllvm/lib/Target/AArch64/AArch64RegisterInfo.h
- llvm/lib/Target/AArch64/AArch64RegisterInfo.td 10 additions, 1 deletionllvm/lib/Target/AArch64/AArch64RegisterInfo.td
- llvm/lib/Target/AArch64/AArch64Subtarget.cpp 19 additions, 0 deletionsllvm/lib/Target/AArch64/AArch64Subtarget.cpp
- llvm/lib/Target/AArch64/AArch64Subtarget.h 2 additions, 0 deletionsllvm/lib/Target/AArch64/AArch64Subtarget.h
- llvm/lib/Target/AArch64/SMEInstrFormats.td 14 additions, 0 deletionsllvm/lib/Target/AArch64/SMEInstrFormats.td
- llvm/test/CodeGen/AArch64/spill-fill-zpr-predicates.mir 1070 additions, 0 deletionsllvm/test/CodeGen/AArch64/spill-fill-zpr-predicates.mir
- llvm/test/CodeGen/AArch64/ssve-stack-hazard-remarks.ll 12 additions, 1 deletionllvm/test/CodeGen/AArch64/ssve-stack-hazard-remarks.ll
- llvm/utils/TableGen/SubtargetEmitter.cpp 21 additions, 1 deletionllvm/utils/TableGen/SubtargetEmitter.cpp
Loading
Please register or sign in to comment