ARStateModel

This object is how you interact with Remodel-AR functionality from SwiftUI.

Declaration

class ARStateModel

Overview

This class is organized by task type.

Topics

common

Functions and variables that are used by all the AR approaches.

let wallPaintedCount: PassthroughSubject<Int, Never> Number of times the user has painted a wall in the current session.

let coachingVisible: PassthroughSubject<Bool, Never> Shows if the ARKit library currently thinks the coaching view should be visible or not

let cameraAimInfo: PassthroughSubject<CameraAimInfo?, Never> Reflects where the current user camera is pointing as well as an estimate of what surface it is pointing at (wall, floor, ceiling).

let trackingReady: PassthroughSubject<Bool, Never> Reflects if the AR library thinks the scene has a sufficiently good solution to start placing geometry. For the different solutions, specifically, this means: Lidar - not used, the user determines when the scan is done. Legacy - indicates 8 seconds of determing the AR solution has been completed. Floorplan - indicates a floor plane has been found and added to the scene.

let paintInfo: PassthroughSubject<PaintInfo?, Never> This is the variable that gets updated when getPaintInfo() is called and contains all the information about painted walls in the current scene.

let capturedPhoto: PassthroughSubject<UIImage?, Never> This is the variable that gets updated when savePhoto() is called. This returns a rendered image of the scene including any geometry and shaders.

let rawCameraImage: PassthroughSubject<UIImage?, Never> This is the variable that gets updated when retrieveRawCameraImage() is called. This returns only the raw camera image without any geometry or shaders.

func pickColor(paint: WallPaint) Use this to set the current selected color that will be applied to the next mesh that is tapped Parameters: - paint: the paint to set as the active color

func pickTexture(texture: UIImage?) Use this to set the current selected texture that will be applied to the next mesh that is tapped Parameters: - texture: the texture to set as the active texture. If you want to remove the texture, pass in nil.

func clearColor(id: String) Removes color from the surface and returns it to the default state. Parameters: - id: The id of the surface color is to be removed from. Id can be obtained from the getPaintInfo() command.

func sharePhoto() Captures the current camera image including any AR effects currently applied and displays the share dialog.

func savePhoto() Saves an image of the current scene to the capturedPhoto variable. Image is returned via the capturedPhoto PassthroughSubject.

func retrieveRawCameraImage() Retrieves a raw image from the camera feed without any geometry or processing.

func resetScene() Resets the AR scene, deleting any geometry. Affects all settings and parameters.

func pauseScene() Pauses the AR scene, used for app background mode.

func startScene(reset: Bool?) Starts/resumes the AR scene. Parameters: - reset: if set to true, the scene will reset and delete all geometry.

func getPaintInfo() Retrieves the list of painted walls along with any important information. Results are returned via the paintInfo PassthroughSubject.

User Interaction

Functions specific to the user experience of interacting with the AR scene.

func dragStart(point: CGPoint) Call when the user starts dragging their finger on the screen Parameters: - point: current location of the users finger

func dragMove(point: CGPoint) Call when the user is dragging their finger on the screen Parameters: - point: current location of the users finger

func dragEnd() Call when the current drag gesture is finished.

func enableBrightnessTapAdjustment(enabled: Bool) Enables or disables the ability to set wall brightness by tapping on different parts of the wall. Enabled by default. Parameters: - enabled: determines if tap to adjust brightness will be enabled.

func enableBrightnessDragAdjustment(enabled: Bool) Enables or disables the ability to set wall brightness by dragging on a wall after it’s created. Enabled by default. Parameters: - enabled: determines if drag to adjust brightness will be enabled.

Lidar

The Lidar approach scans the room geometry using the Lidar sensor. After the user has scanned the room, they can optionally pause the room meshing by sending the setScanMode(scanMode: ScanMode) command. The user can now tap to paint the walls.

let planarMeshCount: PassthroughSubject<Int, Never> The number of planar meshes currently in the scene. Only used for Lidar.

func setScanMode(scanMode: ScanMode) Changes the current scanning mode, used to pause and resume Lidar meshing of the room. Only used for Lidar. Parameters: - scanMode: If set to "paused", Lidar mesh updates will be frozen as they are. Meshing can be resumed by passing in "scanning".

func save3DModel() Saves the current mesh of the room to an OBJ file and an USDZ file to the documents directory. Also displays a share dialog to enable easy sharing of the model files. Only used for Lidar.

Legacy

The Legacy approach allows users to manually create wall patches to be painted. The process to follow is first to prompt the user to place their phone against the wall with the top of the phone against the wall and the screen facing the ceiling, then send the placeWallBasePlane() command. A grid will appear on the wall with a dot that is projected onto the grid. The user should then be prompted to place the first wall point, triggered by the setFirstCorner() command. At this point, a wall patch will be rendered on the grid between the first point and the current projected point. Now the user should be prompted to place the second wall point, triggered by the setSecondCorner() command. At this point the user can paint the wall with different colors. The wall can be modified by tapping the wall to select it, then either dragging the wall with your finger or dragging the corners to adjust the size.

let placeWallState: PassthroughSubject<PlaceWallState, Never> This variable reflects the current placeWallState inside of the legacy state machine. Only used for Legacy.

let wallState: PassthroughSubject<WallState, Never> This variable reflects the current wallState inside of the legacy state machine. Only used for Legacy.

func placeBasePlane() Adds the base plane (grid plane) to the scene at the current location of the phone.

func updateBasePlane() Updates the base plane (grid plane) to the current location of the phone.

func cancelAddWall() Cancels/finishes the current wall being added and removes all related elements.

func setFirstCorner() Sets the first corner of the current wall being placed. After the first corner is set, the wall patch is visible and is created between the first corner and the current projected point from the center of the screen.

func setSecondCorner() Sets the second corner of the current wall being placed. After the second corner is placed, the mesh will only be editable by dragging the corners to resize and reposition the mesh.

Floorplan

The Floorplan approach allows users to draw the floorplan of their room on the floor, then drag to set the height of the walls. This creates geometry that covers all the room surfaces which can now be painted and textured. When the Floorplan approach is loaded, it automatically starts scanning until it finds a floor plane. At that point, it displays a grid on the floor and a target indicator. Every time the user taps on the screen, a point is placed. Once all the corners of the room have been selected, the user can either place a last point on top of the first point, or send the finishCorners(closeShape: true) command. The library automatically switches to setting the wall height. At this point, the user should be prompted to drag the wall up using their finger to set the height. Once the height is set, the finishHeight() command should be sent. At this point, the walls may be painted by tapping.

func finishCorners(closeShape: Bool) Finishes the process of placing corners. Moves the user experience on to the step of setting the height. Parameters: - closeShape: If this is true, the last and first points will be connected, forming a closed shape. Defaults to false.

func finishHeight() This is to be called after the room height is at the desired position.

func showUnpaintedWalls(visible: Bool) Toggles the visibility of unpainted walls. Only used for Floorplan. Parameters: - visible: A boolean that determines whether or not to display unpainted walls

Shader Paint

The Shader Paint approach allows paints to be previewed on older devices that don't support AR. This approach allows the user to select up to three colors from the wall that is to be painted and an adjustable color threshold. Any color that is close enough to the selected colors will be painted the selected color. Brightness can be set by either tapping or dragging a finger on the screen.

func setColorThreshold(threshold: Float) Sets the current threshold used for masking foreground objects using colors. Parameters: - threshold: the threshold to use to remove occlusions. Higher values paint more of the wall and occlusions. Lower values paint less of the wall.

func setTouchMode(mode: TouchMode) Used for occluding foreground objects based on the color of the wall. Parameters: - mode: The current mode to apply to touches. Determines whether touches will behave as hit-testing, or for selecting colors for color based occlusions.

Abnormalities

The Abnormalities approach scans the room using the Lidar sensor, then allows the user to capture images that can be passed into a ML model. Passio provides an SDK that allows wall defects to be classified that works hand in hand with this feature. If a defect is recognized, then the defect can be added to the Remodel-AR library, which will then project a bounding box onto the meshed wall with the name of the defect.

let selectedAbnormality: PassthroughSubject<Abnormality?, Never> If the user taps on an abnormality label in the 3D view, this variable is updated with the abnormality that was tapped. Only used for Abnormalities.

let addedAbnormalityId: PassthroughSubject<String?, Never> After adding a new abnormality, this variable contains the ID of the newly added abnormality. Only used for Abnormalities.

let capturedAbnormalityImage: PassthroughSubject<ImageCapture?, Never> This variable contains the most recent captured image after calling captureAbnormalityImage(). Only used for Abnormalities.

let abnormalitiesInfo: PassthroughSubject<AbnormalitiesInfo?, Never> This variable is updated when retrieveAbnormalitiesInfo() is called. This variable contains an array of all the abnormalities currently contained in the scene. Only used for Abnormalities.

func setScanArea(rect: CGRect) Sets the area of the UI that the scan graphic is covering. Also determines which part of the screen is projected onto the wall to represent any detected abnormalities. Parameters: - rect: In screen coordinates, determines which are of the screen to return the image from for capturedAbnormalityImage(), also determines where in 3D space the bounding box will be projected onto the wall.

func captureAbnormalityImage() Triggers a capture of the next raw camera image, returned in the abnormalitySelected() callback. This is different from the normal retrieveRawCameraImage() function in that it also captures the location of the scanArea projected onto the wall if addAbnormality() is subsequently called.

func addAbnormality(name: String) Adds an abnormality to the AR scene and returns the created abnormality ID. Parameters: - name: The name of the abnormality as reported by the SDK.

func updateAbnormality(abnormality: Abnormality) Updates an abnormality type in the AR scene. Parameters: - abnormality: The object containing the updated abnormality information.

func deleteAbnormality(id: String) Deletes an abnormality from the AR scene Parameters: - id: The identifier of the abnormality to be deleted

func retrieveAbnormalitiesInfo() Retrieves the list of abnormalities along with any important information.

UI Customization

The UI elements and colors can be customized as desired to match branding and design.

func setLegacyUIImages(cornerTextures: [Corners: UIImage], centerDot: UIImage) Overrides the texture used for UI images. Only used for Legacy. Parameters: - cornerTextures: A dictionary of images to use for the different corners, as indicated by the Corners enum. - centerDot: the texture to use for the center dot

func setGridImage(gridImage: UIImage) Overrides the texture used for the grid background. Only used for Legacy and Floorplan. Parameters: - gridImage: the texture to use for the grid background image

func setDeselectedColor(color: UIColor) Overrides the color used for mesh highlighting. Only used for Floorplan. Parameters: - color: the color to use for mesh highlights

func setSelectedColor(color: UIColor) Overrides the color used for the target indicator and the ceiling mesh highlights. Only used for Floorplan. Parameters: - color: the color to use for target indicator and ceiling highlights

func setMeshColor(color: UIColor) Overrides the color used for the generated mesh lines. Only used for Lidar and Abnormalities. Parameters: - color: the color to use for the mesh lines

If at any point you need help from the Passio team, please reach out to us at [e-mail]