ld64 inserts 4096 nop instructions in __text section between symbols

Originator:lewurm
Number:rdar://FB11942354 Date Originated:1/13/2023
Status:Closed Resolved:
Product:Xcode Product Version:Xcode 14.2 Build version 14C18
Classification: Reproducible:yes
 
====================================================================
PLEASE PUSH A NEWER VERSION OF LD64 TO https://github.com/apple-opensource/ld64
====================================================================

See repro and description here: https://gist.github.com/lewurm/d08fa92c3d3193ef8f994d93ec3f7227


Please list the steps you took to reproduce the issue:
See https://gist.github.com/lewurm/d08fa92c3d3193ef8f994d93ec3f7227

What did you expect to happen?
Still working as it did in previous versions of ld64.

What actually happened?
Got a page (4k) full of nop instructions in the __text section that messes up relative addressing.

Comments

With the latest Xcode 16 beta I get the following output now:

% bash repro.sh helloworld-unaligned-ptrs.o

  • ld -version_details | jq .version

"1115.5.3"

  • xcodebuild -version

Xcode 16.0

Build version 16A5171c

++ xcode-select -p

  • XCODE_HOME=/Applications/Xcode16-beta1.app/Contents/Developer

++ pwd

++ pwd

  • ld -demangle -lto_library /Applications/Xcode16-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch x86_64 -platform_version macos 13.0.0 13.1 -syslibroot /Applications/Xcode16-beta1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o helloworld-unaligned-ptrs.o.binary -L/private/tmp/d08fa92c3d3193ef8f994d93ec3f7227 -L/usr/local/lib -U _darwin_check_fd_set_overflow -no_compact_unwind -exported_symbols_list /private/tmp/d08fa92c3d3193ef8f994d93ec3f7227/exported_symbols.list helloworld-unaligned-ptrs.o liblibchelper.a libdarwin.a libjvm.a libnet.a libnio.a libjava.a libfdlibm.a -ldl -lpthread -framework Foundation -lz -lSystem /Applications/Xcode16-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/16.0.0/lib/darwin/libclang_rt.osx.a /Applications/Xcode16-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/16/lib/darwin/libclang_rt.osx.a

ld: warning: no platform load command found in '/private/tmp/d08fa92c3d3193ef8f994d93ec3f7227/helloworld-unaligned-ptrs.o', assuming: macOS

ld: warning: disabling chained fixups because of unaligned pointers

ld: illegal text-relocation in '___text'+0x110258 (/private/tmp/d08fa92c3d3193ef8f994d93ec3f7227/helloworld-unaligned-ptrs.o) to '_JavaVMOperation_setQueuingThread_48eaaa57f9b892acb92c7cf8b4ac18c6832bd402'

The linker got more strict and doesn’t allow relocations for movabs anymore. Which I think is fine, according to the reloc.h headers that shouldn’t be possible anyways. It doesn’t exactly fix the issue, but just forces us to find another solution. Which we already have: instead of having a movabs $full_addr instruction, we do PC relative addressing and emit an (valid) relocation for that.

Apple Feedback

Please verify this issue with Xcode 16 Beta and update your bug report with your results by logging into https://feedbackassistant.apple.com/ or by using the Feedback Assistant app.

Xcode 16 Beta (16A5171c) https://developer.apple.com/download/ Posted Date: June 10, 2024

If resolved, you can close your feedback by choosing the Close Feedback menu item in the Actions pop-up in Feedback Assistant.

If the issue persists, please attach a new sysdiagnose captured in the latest build and attach it to the bug report. Thank you.

Xcode Sysdiagnose for macOS Instructions: https://developer.apple.com/services-account/download?path=/OS_X/OS_X_Logs/Xcode_sysdiagnose_Logging_Instructions.pdf

Simulator Diagnose for macOS Instructions: https://developer.apple.com/services-account/download?path=/OS_X/OS_X_Logs/simctl_Diagnose_Logging_Instructions.pdf

Please retry your linker invocation with “helloworld-unaligned-ptrs.o” to reproduce the 4k NOPs symptom.

With “helloworld-aligned-ptrs.o” there are no NOPs inserted indeed, however the resulting binary segfaults in dyld.

Apple Feedback

Please know that we're unable to reproduce the mis-link:

[/tmp/a]> /Applications/Xcode2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld-820.1 -dynamic -arch x86_64 -platform_version macos 13.0.0 13.1 -syslibroot /Applications/Xcode2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -o a.binary -L. -U _darwin_check_fd_set_overflow -no_compact_unwind helloworld-aligned-ptrs.o liblibchelper.a libdarwin.a libjvm.a libnet.a libnio.a libjava.a libfdlibm.a -framework Foundation -lz -lSystem

[/tmp/a]> otool -lv a.binary|grep -A8 LC_BUILD






















































  cmd LC_BUILD_VERSION

cmdsize 32

platform MACOS

minos 13.0

  sdk 13.1

ntools 1

 tool LD

version 820.1

Load command 12

[/tmp/a]> nm -nm a.binary| grep Character.toUpper



















































0000000100154520 (TEXT,text) non-external CharactertoUpperCase_32e6350ee20942d333d8e148b79c97eea2676c03

0000000100154760 (TEXT,text) non-external CharactertoUpperCaseCharArray_d3bece676e84f7b3101e5e3b92b8791bafb5cb4e

0000000100154cb0 (TEXT,text) non-external CharactertoUpperCaseEx_5e336fa9cb982976a8ba6aaa89c54c23324f695a

[/tmp/a]>

There was no 4K of NOPs inserted. But these address are slightly different than in the github readme.

The linker does break up sections at each symbol into "atoms", each with their own alignment requirements. They do during layout if the next atom's layout requirement means the next address won't work, the linker inserts padding until the address will work. For _text section for x8664, the linker inserts NOP bytes. For all other sections, it inserts zeros.

lewurm

Thank you for the quick answer!

Can you elaborate on why a 4k alignment in the section header can cause the linker to insert 4k fill bytes? The symbol (Character_toUpperCaseEx_5e336fa9cb982976a8ba6aaa89c54c23324f695a) after the 4k of fill bytes at address 0x100158c40 is certainly not 4k aligned. If anything, I would expect the linker to fill in less than 4k filler bytes, so that the symbol is 4k aligned. What is the linker trying to achieve here?

Also note that this occurrence of 4k filler bytes is placed in the middle of the __text section of one input object file; I would expect the section alignment to be used when multiple sections of the same kind but from different object files are concatenated.

Furthermore, why are earlier versions of ld64 produce a working binary with the same input?

As a side note, if I bump the section alignment to 16k this sequence of filler bytes before the symbol Character_toUpperCaseEx_5e336fa9cb982976a8ba6aaa89c54c23324f695a disappears, and the emitted binary works with the latest available ld64 version.

I would really love to understand the behaviour around this.

Apple Feedback

Please know that our engineering team has determined that this issue behaves as intended based on the information provided.

The __text section in helloworld-unaligned-ptrs.o is 4KB aligned. So, when linked with other code, ld will attempt to align it, which means it will use fill bytes, in this case they are NOPs.


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!