iPad Pro camera/ screen resolution/ AR scanning issues

Hello community ~
I have tested the 0.4.0 and 0.4.1 SDK recently and met some problems.
I found the SDK works well in my iphoneX.But when i use the same project file and export different resolution for the ipad pro.Then on the app, i can’t see anything from the camera.(The background become the Unity’s default skybox) .Though i have authorized the camera.

I am not sure is it because my ipad version is the newest one (13.1.2) or i miss some steps?Also i found i need to add some extra steps like modify the Info.plist, otherwise the ipad version will crash.(On iphone version this step is not neccessary)

2 Likes

Great to hear from you Wenzy.

Think the need to manually add permissions in Xcode (e.g NSBluetoothAlwaysUsageDescription) is related to changes to how permissions work in iOS 13. We will look into automating this into the build process in future releases.The camera not showing up on the iPad is very strange. Would you mind answering the following questions to help us get to the bottom of this?

  • In Unity under Player Settings > Other Settings, is your Target device set to “iPhone + iPad”?
  • What version of Unity are you using?
  • What version of iOS is on your iPhone?
  • If you are happy to send the project to my email (nat@litho.cc) - am happy to have a look myself.

Would you mind providing a screenshot of the problem when it occurs on the iPad (and possibly of the XCode logs too, if there’s anything of note in there.)

1 Like

Thanks nat.

I already set to “iPhone + iPad”.And My unity version is 2019.2.2f1.My iphone version is 12.3.1.

Today i try to export the file again.I found the camera can read now~ weird :sweat_smile:

But i met another problem.The first thing is when it goes to “AR SCANNING” interface,the number will stuck no matter how i move my camera(also i can’t press the continue button to switch to next page).The second thing is i found the ipad pro resolution provide from the SDK. It’s 2224 X 1668(maybe it’s base on old 10.5 inch’s ipad pro).That’s the closest resolution to the new 11 inch ipad pro (2388×1668),So when i apply it to the app.I can’t touch the button below

I can press the close button to skip these steps and the object can be interactive in the scene.(Just lack of accuracy.)

I have restart the app and reimport the app.But this problem still exist.

2 Likes

Hi Wenzy,

Glad you got it up and running :slight_smile:

Notes about your AR scanning progress issues:

:question: - What number did the scanning progress get stuck on?

:question: - Does the progress always get stuck on the same value?

:thought_balloon: - I suspect the value getting stuck might be due to an issue with AR Foundation, or how AR planes are being interpreted by Litho - we will be investigating and improving the AR scanning process over the next few weeks

:bulb: - The ‘CONTINUE’ button on the AR Scanning tab only becomes active when scanning is complete, however:

:point_right: - You can skip the AR Scanning tab at any time by swiping to the left (to get to the next tab) - the scanning process is just a suggestion, as interactions can feel a bit strange if the environment is not scanned well

Notes about your resolution issues:

:bulb: - The Game view in Unity is only used for previewing the app in the Unity Editor; when you build to a device, it should adjust the app to match the native resolution of the device (so it doesn’t matter which resolution you use in the Game view), however:

:point_right: - If you want to preview the app at the correct resolution, you should be able to add a new resolution option as shown in the pictures below:

:bulb: - You can also skip the Onboarding tabs at any time by swiping to the left (to get to the next tab) - these are simply used to teach new users how to wear and calibrate Litho. From SDK v0.4.1, the Onboarding tabs only show the first time a new build of the app is run (the ‘TUTORIAL’ button will show them again)

:point_right: - The fastest way to fix the UI layout (iPad support is due to be added over the next few months) is to delete or scale down the images in the middle of the UI cards - if you remove these, the button and other content that usually falls off the bottom of the screen should move up and become visible. Do this for any tabs that don’t fit onscreen at iPad resolution. You can test this using the Unity Game view after setting it to the new resolution as shown in the pictures above.

Thanks again for the feedback, and let us know how you get on :slight_smile:

1 Like

Hi,lewis.Thanks to your reply and suggestion.It’s helpful.And u can also consider update some new resolution on that list int the next SDK,that would be more convenient for other developers:)

I found the progress number is random.Each time will have different number(usually is above 0).I suspect is it becasue it will start to scan the plane when this app is open(I can see the yellow checking points are shining and the borderline is changing).And when it goes to the “AR SCANNING” page,it stucks and shows the previous progress.I can’t confirm that because i don’t have my device nearby right now.But i remember if i rotate my camera more before i enter that page.The progress number usually will be higher.

1 Like

Hi Wenzy,

:thought_balloon: - Thanks for the suggestion - I think the new resolutions will likely be included with newer versions of the ARKit package, which we will upgrade our support for when they are released (non-preview versions).

:question: - Does the app stop recognising AR scan progress when you connect to your Litho, or does it specifically stop only when you swipe to look at the AR Scanning tab? (You can prevent the menu from automatically moving to the AR Scanning tab after connection by holding the Device Selection tab in place with your finger on the screen)

:question: - Would you be able to provide some debugging info for us to take a look at? The content of the ‘All Output’ window in Xcode when running the app on your iPad (whilst debugging with Xcode) could be useful.

1 Like

Hi,lewis.Sorry for the late reply.I get my device now~

Before i connect to Litho.I can still see some shining yellow points and i think that is the scan progress right?If in this step i scan long enough,i found when it enter to the AR Scanning tab.It shows 100% and i can jump into next tab.

I have uploaded a testing video.Hope that could help.
(https://drive.google.com/open?id=1luCtCKw0D6Bejulr8nG2UXaef9dD8ULr)
In some cases when i am in the AR scanning tab,after a very long time scanning.The number will suddenly jump from like 27% to 100%.But this is not always happen.It usually stuck.

Here is my Debugging message:

2019-10-13 12:06:39.488260+0800 NewLithoSDKtesting[1542:796354] Built from ‘2019.2/release’ branch, Version ‘2019.2.2f1 (ab112815d860)’, Build type ‘Release’, Scripting Backend ‘il2cpp’

2019-10-13 12:06:39.492071+0800 NewLithoSDKtesting[1542:796354] -> registered mono modules 0x1014c32d0

-> applicationDidFinishLaunching()

2019-10-13 12:06:39.532588+0800 NewLithoSDKtesting[1542:796354] Metal GPU Frame Capture Enabled

-> applicationDidBecomeActive()

[XR] Discovering subsystems at path /private/var/containers/Bundle/Application/53BF6FB4-424F-43C3-AC1A-BBEAD2275F60/NewLithoSDKtesting.app/Data/UnitySubsystems

[XR] No descriptors matched for examples in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] 1 ‘inputs’ descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json

[XR] No descriptors matched for cameras in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for displays in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for depths in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for meshings in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for planes in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for raycasts in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for referencePoints in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for sessions in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for experience in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

[XR] No descriptors matched for gestures in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.

GfxDevice: creating device client; threaded=1

Initializing Metal device caps: Apple A12X GPU

Initialize engine version: 2019.2.2f1 (ab112815d860)

2019-10-13 12:06:39.954210+0800 NewLithoSDKtesting[1542:796354] Unbalanced calls to begin/end appearance transitions for <SplashScreenController: 0x104427510>.

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

UnloadTime: 0.828833 ms

[XR] Loading plugin UnityARKit for subsystem ARKit-Input…

[XR] UnityARKit successfully registered Provider for ARKit-Input

2019-10-13 12:06:41.946869+0800 NewLithoSDKtesting[1542:796354] [CoreBluetooth] API MISUSE: <CBCentralManager: 0x2805689a0> can only accept this command while in the powered on state

Setting up 2 worker threads for Enlighten.

Thread -> id: 171187000 -> priority: 1

Thread -> id: 171213000 -> priority: 1

The referenced script on this Behaviour (Game Object ‘’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

LITHO.Demo.ShowcaseManager:Update()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

LITHO.Demo.ShowcaseManager:Update()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

LITHO.Demo.ShowcaseManager:Update()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

LITHO.Demo.ShowcaseManager:Update()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

LITHO.Demo.ShowcaseManager:Update()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

LITHO.Demo.ShowcaseManager:Update()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

Litho device found: Litho FA4F

LITHO.DeviceEventHandler:Invoke(String)

LITHO.Litho:HandleEvents()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

The referenced script on this Behaviour (Game Object ‘’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

The referenced script on this Behaviour (Game Object ‘Button’) is missing!

UnityEngine.Object:Internal_InstantiateSingleWithParent(Object, Transform, Vector3, Quaternion)

UnityEngine.Object:Instantiate(Object, Vector3, Quaternion, Transform)

UnityEngine.Object:Instantiate(T, Vector3, Quaternion, Transform)

LITHO.Demo.ShowcaseItem:ResetItem()

[./Runtime/Mono/ManagedMonoBehaviourRef.cpp line 334]

(Filename: ./Runtime/Mono/ManagedMonoBehaviourRef.cpp Line: 334)

-> applicationWillResignActive()

-> applicationDidBecomeActive()

State changed to Connecting from NoneConnected

LITHO.UI.DeviceSelectUI:ChangeState(ConnectionState)

LITHO.UI.DeviceSelectUI:HandleButtonClick(DeviceOptionUI)

UnityEngine.Events.UnityAction:Invoke()

UnityEngine.Events.UnityEvent:Invoke()

UnityEngine.Events.UnityAction:Invoke()

UnityEngine.Events.UnityEvent:Invoke()

UnityEngine.EventSystems.EventFunction`1:Invoke(T1, BaseEventData)

UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)

UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchPress(PointerEventData, Boolean, Boolean)

UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchEvents()

UnityEngine.EventSystems.StandaloneInputModule:Process()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

Firmware: 45b1cf97279039136950fb2141ff083e0713ba5e

LITHO.Litho:GetDeviceInfo(String)

LITHO.Litho:b__103_0(String)

LITHO.DeviceEventHandler:Invoke(String)

LITHO.Litho:HandleEvents()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

Hardware: 1.3

LITHO.Litho:GetDeviceInfo(String)

LITHO.Litho:b__103_0(String)

LITHO.DeviceEventHandler:Invoke(String)

LITHO.Litho:HandleEvents()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

Model: Litho

LITHO.Litho:b__103_0(String)

LITHO.DeviceEventHandler:Invoke(String)

LITHO.Litho:HandleEvents()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

State changed to Connected from Connecting

LITHO.UI.DeviceSelectUI:ChangeState(ConnectionState)

LITHO.UI.DeviceSelectUI:HandleDeviceConnected(String)

LITHO.DeviceEventHandler:Invoke(String)

LITHO.Litho:HandleEvents()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

-> applicationWillResignActive()

-> applicationDidBecomeActive()

-> applicationWillResignActive()

-> applicationDidEnterBackground()

2019-10-13 12:08:15.726484+0800 NewLithoSDKtesting[1542:796354] Can’t end BackgroundTask: no background task exists with identifier 2 (0x2), or it may have already been ended. Break in UIApplicationEndBackgroundTaskError() to debug.

1 Like

Hi Wenzy,

Thanks for the extra details and video.

:thought_balloon: - Judging from the video, I’d say that this is a combination of two things - limitations of AR Foundation, and lack of clarity on how Litho is interpreting the AR Foundation data. This should be resolved (and significantly sped up) by an upcoming update to the SDK (v0.5.0). The following solutions apply to SDK versions before v0.5.0:

:point_right: - The easiest solution is to just skip the AR Scanning tab by swiping to the left.

:point_right: - However, to get 100% scanning accuracy, Litho needs to know where the ground is - specifically, you’ll need to scan a 5m^2 area of the surface you are standing on (e.g. a 2.5m x 2m rectangle on the floor). In your video, you are scanning the table (which is also useful, as this will allow you to interact with that table), however the main part of scanning for Litho is to detect the floor. The reason for this is that Litho is designed for room-scale interactions.

:bulb: - The reason why scan progress appears to freeze is that once you have scanned the table, scan progress will not increase until you scan an area on the floor that is larger than the table.

:bulb: - When scanning the floor (particularly plain, slightly reflective floors - we have issues with ours in the office), you may have to hold the camera a little closer to the ground and move more slowly than when scanning tables (tables tend to have more easily-tracked features).

I hope this resolves your issues, and looking forward to getting the next update to you soon!

1 Like