The issue was addressed with improved handling of caches. This issue is fixed in tvOS 17, iOS 16.7 and iPadOS 16.7, macOS Monterey 12.7, watchOS 10, iOS 17 and iPadOS 17, macOS Sonoma 14. An app may be able to access contacts.
https://support.apple.com/en-us/HT213940
https://support.apple.com/en-us/HT213938
https://support.apple.com/en-us/HT213937
https://support.apple.com/en-us/HT213936
https://support.apple.com/en-us/HT213932
https://support.apple.com/en-us/HT213927
http://seclists.org/fulldisclosure/2023/Oct/9
http://seclists.org/fulldisclosure/2023/Oct/8
http://seclists.org/fulldisclosure/2023/Oct/6
http://seclists.org/fulldisclosure/2023/Oct/4