The TrueDepth camera on the iPad Pro M3 is primarily used for Face ID, a biometric authentication system that securely unlocks the device and authenticates transactions. It works by projecting over 30,000 invisible infrared dots onto the user's face, creating a depth map that is then analyzed by the device's neural engine to authenticate the user[4][7]. Beyond Face ID, the TrueDepth camera supports various features such as Portrait mode, Portrait Lighting, Animoji, and Memoji, which enhance the user experience in photography and video conferencing[1][4].
For Augmented Reality (AR) applications, the TrueDepth camera can be used to track the user's face, allowing for interactive AR experiences. However, simultaneous use of the front and rear cameras for both face and world tracking is limited by hardware constraints on some iPad models[6]. The ARKit framework provides tools for developing AR applications, including face tracking, but it may not support simultaneous use of both cameras on all devices[6].
In terms of third-party app development, the TrueDepth camera's capabilities are somewhat restricted. While developers can access certain face tracking features through frameworks like ARKit, they cannot directly access detailed face or eye data for advanced applications such as precise gaze detection[3]. However, some workarounds exist, such as using the Vision framework to obtain face pitch, roll, and yaw values, which can be useful for detecting the orientation of the user's face[3].
Lastly, the TrueDepth camera also supports video recording and photography features like time-lapse and cinematic video stabilization, making it versatile beyond its primary use for Face ID and AR[1].
In summary, while the TrueDepth camera on the iPad Pro M3 is primarily used for Face ID and AR, it also supports various photography and video features. However, its use in third-party apps for advanced face or eye tracking is limited by Apple's APIs.
Citations:[1] https://support.apple.com/en-us/111897
[2] https://www.macrumors.com/2017/10/09/ipad-pro-2018-truedepth-camera/
[3] https://www.reddit.com/r/iOSProgramming/comments/116fbwr/is_it_possible_for_an_application_to_use/
[4] https://www.macrumors.com/roundup/ipad-pro/
[5] https://www.apple.com/ipad-pro/specs/
[6] https://stackoverflow.com/questions/61279161/ios-arkit-3-with-ipad-pro-2020-how-to-use-front-camera-data-with-back-camera-tr
[7] https://support.apple.com/en-us/102381
[8] https://www.reddit.com/r/vtubertech/comments/18esuw4/any_ipad_with_true_depth_sensor_works_for_perfect/