iPad Pro camera/ screen resolution/ AR scanning issues

ziyangwen · 2 October 2019 16:00

Hello community ~
I have tested the 0.4.0 and 0.4.1 SDK recently and met some problems.
I found the SDK works well in my iphoneX.But when i use the same project file and export different resolution for the ipad pro.Then on the app, i can’t see anything from the camera.(The background become the Unity’s default skybox) .Though i have authorized the camera.

I am not sure is it because my ipad version is the newest one (13.1.2) or i miss some steps?Also i found i need to add some extra steps like modify the Info.plist, otherwise the ipad version will crash.（On iphone version this step is not neccessary）

nat · 2 October 2019 16:34

Great to hear from you Wenzy.

Think the need to manually add permissions in Xcode (e.g NSBluetoothAlwaysUsageDescription) is related to changes to how permissions work in iOS 13. We will look into automating this into the build process in future releases.The camera not showing up on the iPad is very strange. Would you mind answering the following questions to help us get to the bottom of this?

In Unity under Player Settings > Other Settings, is your Target device set to “iPhone + iPad”?
What version of Unity are you using?
What version of iOS is on your iPhone?
If you are happy to send the project to my email (nat@litho.cc) - am happy to have a look myself.

Would you mind providing a screenshot of the problem when it occurs on the iPad (and possibly of the XCode logs too, if there’s anything of note in there.)

ziyangwen · 2 October 2019 18:27

Thanks nat.

I already set to “iPhone + iPad”.And My unity version is 2019.2.2f1.My iphone version is 12.3.1.

Today i try to export the file again.I found the camera can read now~ weird

But i met another problem.The first thing is when it goes to “AR SCANNING” interface,the number will stuck no matter how i move my camera(also i can’t press the continue button to switch to next page).The second thing is i found the ipad pro resolution provide from the SDK. It’s 2224 X 1668(maybe it’s base on old 10.5 inch’s ipad pro).That’s the closest resolution to the new 11 inch ipad pro (2388×1668),So when i apply it to the app.I can’t touch the button below

I can press the close button to skip these steps and the object can be interactive in the scene.(Just lack of accuracy.)

I have restart the app and reimport the app.But this problem still exist.

lewis · 3 October 2019 10:53

Hi Wenzy,

Glad you got it up and running

Notes about your AR scanning progress issues:

- What number did the scanning progress get stuck on?

- Does the progress always get stuck on the same value?

- I suspect the value getting stuck might be due to an issue with AR Foundation, or how AR planes are being interpreted by Litho - we will be investigating and improving the AR scanning process over the next few weeks

- The ‘CONTINUE’ button on the AR Scanning tab only becomes active when scanning is complete, however:

- You can skip the AR Scanning tab at any time by swiping to the left (to get to the next tab) - the scanning process is just a suggestion, as interactions can feel a bit strange if the environment is not scanned well

Notes about your resolution issues:

- The Game view in Unity is only used for previewing the app in the Unity Editor; when you build to a device, it should adjust the app to match the native resolution of the device (so it doesn’t matter which resolution you use in the Game view), however:

- If you want to preview the app at the correct resolution, you should be able to add a new resolution option as shown in the pictures below:

- You can also skip the Onboarding tabs at any time by swiping to the left (to get to the next tab) - these are simply used to teach new users how to wear and calibrate Litho. From SDK v0.4.1, the Onboarding tabs only show the first time a new build of the app is run (the ‘TUTORIAL’ button will show them again)

- The fastest way to fix the UI layout (iPad support is due to be added over the next few months) is to delete or scale down the images in the middle of the UI cards - if you remove these, the button and other content that usually falls off the bottom of the screen should move up and become visible. Do this for any tabs that don’t fit onscreen at iPad resolution. You can test this using the Unity Game view after setting it to the new resolution as shown in the pictures above.

Thanks again for the feedback, and let us know how you get on

ziyangwen · 3 October 2019 14:57

Hi,lewis.Thanks to your reply and suggestion.It’s helpful.And u can also consider update some new resolution on that list int the next SDK,that would be more convenient for other developers:)

I found the progress number is random.Each time will have different number(usually is above 0).I suspect is it becasue it will start to scan the plane when this app is open(I can see the yellow checking points are shining and the borderline is changing).And when it goes to the “AR SCANNING” page,it stucks and shows the previous progress.I can’t confirm that because i don’t have my device nearby right now.But i remember if i rotate my camera more before i enter that page.The progress number usually will be higher.

lewis · 3 October 2019 20:36

Hi Wenzy,

- Thanks for the suggestion - I think the new resolutions will likely be included with newer versions of the ARKit package, which we will upgrade our support for when they are released (non-preview versions).

- Does the app stop recognising AR scan progress when you connect to your Litho, or does it specifically stop only when you swipe to look at the AR Scanning tab? (You can prevent the menu from automatically moving to the AR Scanning tab after connection by holding the Device Selection tab in place with your finger on the screen)

- Would you be able to provide some debugging info for us to take a look at? The content of the ‘All Output’ window in Xcode when running the app on your iPad (whilst debugging with Xcode) could be useful.

ziyangwen · 13 October 2019 04:50

Hi,lewis.Sorry for the late reply.I get my device now~

Before i connect to Litho.I can still see some shining yellow points and i think that is the scan progress right?If in this step i scan long enough,i found when it enter to the AR Scanning tab.It shows 100% and i can jump into next tab.

I have uploaded a testing video.Hope that could help.
(https://drive.google.com/open?id=1luCtCKw0D6Bejulr8nG2UXaef9dD8ULr)
In some cases when i am in the AR scanning tab,after a very long time scanning.The number will suddenly jump from like 27% to 100%.But this is not always happen.It usually stuck.

Here is my Debugging message：

2019-10-13 12:06:39.488260+0800 NewLithoSDKtesting[1542:796354] Built from ‘2019.2/release’ branch, Version ‘2019.2.2f1 (ab112815d860)’, Build type ‘Release’, Scripting Backend ‘il2cpp’

2019-10-13 12:06:39.492071+0800 NewLithoSDKtesting[1542:796354] -> registered mono modules 0x1014c32d0

-> applicationDidFinishLaunching()

2019-10-13 12:06:39.532588+0800 NewLithoSDKtesting[1542:796354] Metal GPU Frame Capture Enabled

-> applicationDidBecomeActive()

[XR] Discovering subsystems at path /private/var/containers/Bundle/Application/53BF6FB4-424F-43C3-AC1A-BBEAD2275F60/NewLithoSDKtesting.app/Data/UnitySubsystems

[XR] No descriptors matched for examples in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] 1 ‘inputs’ descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json

[XR] No descriptors matched for cameras in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for displays in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for depths in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for meshings in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for planes in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for raycasts in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for referencePoints in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for sessions in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for experience in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for gestures in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

GfxDevice: creating device client; threaded=1

Initializing Metal device caps: Apple A12X GPU

Initialize engine version: 2019.2.2f1 (ab112815d860)

2019-10-13 12:06:39.954210+0800 NewLithoSDKtesting[1542:796354] Unbalanced calls to begin/end appearance transitions for <SplashScreenController: 0x104427510>.

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)