[Chrome] arm64: Writing a Mach-O executable via mmap produces something that incorrectly fails code signature verification and won’t run

Originator:mark
Number:rdar://FB8914231 Date Originated:2020-11-23
Status:Open Resolved:
Product:macOS Product Version:11.0.1 20B29
Classification:Application Crash Reproducible:Always
 
If a Mach-O executable is created by writing the image contents to a mmaped region, the result incorrectly fails code signature verification in xnu and won’t run. This happens when “kill” semantics are enforced, which is the default on arm64.

Steps to reproduce:

1. Create a Mach-O executable by writing its image to an mmaped region. The attached test program, mmap_copy, can do this. In this example, it’s instructed to make a copy of itself in a new location, mmap_copy_copy.

mark@arm-and-hammer zsh% clang++ -Wall -Werror t_mmap_copy.cc -o t_mmap_copy
mark@arm-and-hammer zsh% ./mmap_copy mmap_copy mmap_copy_copy

2. Attempt to run the executable.

mark@arm-and-hammer zsh% ./mmap_copy_copy

Expected behavior:

The executable should run.

mark@arm-and-hammer zsh% ./mmap_copy_copy
usage: ./mmap_copy_copy <source> <destination>

Observed behavior:

The executable does not run. It’s killed by the kernel with a SIGKILL.

mark@arm-and-hammer zsh% ./mmap_copy_copy
zsh: killed     ./mmap_copy_copy

Observing the system log during the above test, these messages are visible:

admin@arm-and-hammer zsh% sudo log stream --predicate 'sender = "kernel"'
Filtering the log data using "sender == "kernel""
Timestamp                       Thread     Type        Activity             PID    TTL  
2020-11-23 11:23:29.167443-0500 0x1a601    Default     0x0                  0      0    kernel: CODE SIGNING: cs_invalid_page(0x100250000): p=2369[mmap_copy_copy] final status 0x23020200, denying page sending SIGKILL
2020-11-23 11:23:29.167471-0500 0x1a601    Default     0x0                  0      0    kernel: CODE SIGNING: process 2369[mmap_copy_copy]: rejecting invalid page at address 0x100250000 from offset 0x0 in file "/Users/mark/mmap_copy_copy" (cs_mtime:1606148606.175188953 == mtime:1606148606.175188953) (signed:1 validated:1 tainted:1 nx:0 wpmapped:1 dirty:0 depth:0)

Observe equivalence between mmap_copy, which runs, and mmap_copy_copy, which does not:

mark@arm-and-hammer zsh% shasum mmap_copy mmap_copy_copy
e217641f45b8b2311e87276da79d81906bfa296e  mmap_copy
e217641f45b8b2311e87276da79d81906bfa296e  mmap_copy_copy

System information:

mark@arm-and-hammer zsh% sw_vers
ProductName:	macOS
ProductVersion:	11.0.1
BuildVersion:	20B29
mark@arm-and-hammer zsh% xcodebuild -version
Xcode 12.2
Build version 12B45b
mark@arm-and-hammer zsh% uname -m
arm64
mark@arm-and-hammer zsh% system_profiler SPHardwareDataType | grep 'Model Identifier'
      Model Identifier: ADP3,2

This occurs on shipping M1-based hardware as well.

To reproduce on x86_64, set the “kill” flag in the code signature.

mark@sweet16 zsh% clang++ -Wall -Werror mmap_copy.cc -o mmap_copy
mark@sweet16 zsh% codesign --sign=- --options=kill mmap_copy
mark@sweet16 zsh% ./mmap_copy mmap_copy mmap_copy_copy
mark@sweet16 zsh% shasum mmap_copy mmap_copy_copy
9a2750b2ea900767bcfe4f7ae74597ebd33e0c6f  mmap_copy
9a2750b2ea900767bcfe4f7ae74597ebd33e0c6f  mmap_copy_copy
mark@sweet16 zsh% ./mmap_copy_copy
zsh: killed     ./mmap_copy_copy

mark@sweet16 zsh% shasum mmap_copy mmap_copy_copy
58a3d23cb46d283e7a0c943e96244240646aab24  mmap_copy
58a3d23cb46d283e7a0c943e96244240646aab24  mmap_copy_copy

mark@sweet16 zsh% sw_vers
ProductName:	Mac OS X
ProductVersion:	10.15.7
BuildVersion:	19H15
mark@sweet16 zsh% xcodebuild -version
Xcode 12.2
Build version 12B45b
mark@sweet16 zsh% uname -m
x86_64
mark@sweet16 zsh% system_profiler SPHardwareDataType | grep 'Model Identifier'
      Model Identifier: MacBookPro16,1

Additional information:

The following workarounds are available:

1. You can wait for your pages to leave the UBC. If you don’t want to wait, purge can help.
2. You can put your file contents into a new vnode (via a new inode), as long as you write the file with something like write and not writing to an mmaped region. cp or equivalent is fine for this.
3. My favorite: you can call msync(…, MS_INVALIDATE) on the mmaped region, asking xnu to throw away what it knows about the vnode. If you compile mmap_copy.cc with MMAP_COPY_MSYNC_INVALIDATE defined, it will do this. You can even use this technique to “save” a broken vnode from an entirely different process by opening the file,mmaping it, and then calling msync.

ld64 is the primary producer of Mach-O executables, and has traditionally written Mach-O output by writing to an mmaped region whenever possible, as mmap_copy does. The most current source available is Xcode 11.3.1 ld64-530/src/ld/OutputFile.cpp ld::tool::OutputFile::writeOutputFile demonstrates that. But that’s old, and the new truth is that in Xcode 12.2 ld64-609.7, only on arm64, ld64 uses the write approach (essentially workaround 2 above) in every case.

This bug was discovered during the course of updating the golang linker to support mac-arm64, tracked at https://github.com/golang/go/issues/42684.

Comments

2021-11-15 from Apple

We reviewed your report and determined the behavior you experienced is currently functioning as intended.

Thank you for your feedback.

2020-12-17 from Apple

Engineering has provided the following information regarding this issue:

This is most likely due to the fact that the executable's pages have been created via an mmap() mapping, so they're marked as "wpmapped". Since we can't always easily determine if all the writable mappings are gone, a page marked as "wpmapped", i.e. a page that has been mapped for write access in the past, is considered as tainted, so it will fail code-signing verification even if it matches its signature. That's because if there's still a writable mapping somewhere, that mapping could taint the page at any time, including after we validate its signature and start executing from it.

We could potentially try and mitigate this by resetting the page's "wpmapped" when we're absolutely sure that it's safe to do so but that might not cover all the cases and could possibly open some security holes.

The work-around is to use write() instead of mmap() to create/copy the file.

Related: FB8914243

mmap_copy.cc.gz.base64

H4sIAOHhu18CA6VXZ5PiOBD97l/Rm+3biZcPMFUUTNoAU8Dm4NLaMqjOll2SSBP++0lowcJwNrs2ud 39+qn7KXB8DH6E6Oj5czh8h6JIvmPGEgZxjFLPT9LFke/DYZL9tqxHhPrRJMDQQJxjJo7GTcMmw7cM NNk0hT4V0aaJi4BQsW1LtkwR+Za3MUJHOduCH0vKdNvKBcqlmVAiYZXNHFo0ShgR49j0jEhMBDctYp FiTzAkzSo8wCGhGC5b3c6rM+/sqjvs23MH9rk+WwD2bYFDaTRAgP1IMVI5sSwn82YMpSlmHsN8Eol6 cXQCtz+fG3ZmBBckmXpp9D3MxiTCYO8GceHwFJ4+haWW1M9laSWuji4ca0nueyfr29VFt9fP961616 r3rHrHKvYrH0vCH2wV3OrYoswn5XnvofwqUViIIo71EEuvCvrK1CWDIUaE2uoLYiP/APwxYr+o79NP X2RpLF1QdQ8euPCbNgGEKZMxoc1FgBk7gIcTjka4Bk84NHgyYT5uQiPAXBCKBElo8zN9eKBhT7449S UGw2LCKJy9vxp6562rV2/6Z+rGvSXf/IRy8Z2MxvNSJMbgaozTL/Wck5Frw/NX6akGQcUKJwzA3VwH kxRT28hyAD2v3+l1X31wJFVdgSy4ASerKmDGbJO+rIPCgs8Pn/DPD+WADVBnPTYu2MQX8gOtOfFvqz yhMmfZDuDp2sUpSb0MLcmt6mCWShVDC3hHSUzHrC7v+nAnv7T7Z63hQTaAIy68OAlwVjIz/AfrZobm BiCBN5iGsprURwLn8uWocXIjqUkNF7JYg+1DZSKL+bcnNrUXq5kY5FSTUYAGnK7yb4eBC3QSRalgyy SAI46LnJma/pilDAvPR1w0VpSadiyd7O9gBxb85LVdwwpY1/3e0JOq6cCd/v6ufzU8qwD4unXtDS5b /bNOBZBN1VQAOtHKz2s/a5e7JKzEdtZZaWCnClXvSgQIcG/OiI3jgR8lPD8bnLLFQwVVVb2XJpyo3+ D+z5xQCvIEMKx2HUJH4G4rTPqtNsS134o4XwFM82s4wyiwrZxszX4W0M2cuAhqtZhQjZRZZZivZ5gm 0LSXnnQSYybv6NN3o0w7Gt5P4lhyUCerBtdwB5CEoUSt1ZRVfsRobjvOphizYphCY1PV2EI5qdLkt4 WckjIw182hzQvgahCqPvExm9B/C7D1v7ElV9cYh3bJxHAob061sUhcz7Vbsfw1lb2VX7Jt7p7SCvSR vKXP6IH9Ws3vdu/6g/d68KHb9q66b1uvrjqt4ZmzLnHMF9Tfxtq1XR3A64GJUdroJfYeC8cjTAMSWh mnCZUk9iNVzkKj7b9+mcfAwZt2+2wwqFv31n8czqgHABAAAA==


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!