Getting to know Shader Paint

In this tutorial, you will learn how to paint walls using the Remodel-AR framework's Shader Paint approach.

Before starting this tutorial, make sure you've completed Setting up Remodel-AR in UIKit. A link to the starting project is provided here for your convenience.

37KB

Setting up RemodelAR Code.zip

archive

Step 1

Add a new ViewController in Main.storyboard. Next create a new ViewController.swift file and name it ShaderPaintViewController. Now set the ViewController's class to ShaderPaintViewController in Main.storyboard.

Step 2

Open ShaderPaintViewController.swift and import the RemodelAR and ARKit framework.

import ARKit
import RemodelAR

Step 3

Open Main.storyboard and add ARSCNView to the storyboard. Next, connect the ARSCNView outlet to ShaderPaintViewController.swift.

Step 4

In ShaderPaintViewController.swift create a lazy ARController property and assign it using the RemodelARLib.makeShaderARController(with: arscnView) method.

• makeShaderARController(with: arscnView) is a method that takes an ARSCNView's instance as an argument.

• You have to provide an ARSCNView instance that you have connected from the storyboard in this method or one you created and added programmatically.

private lazy var arController: ARController = {
    wreturn RemodelARLib.makeShaderARController(with: arscnView)
}()

Step 5

Create a method for Configuring UI and call this method in ShaderPaintViewController’s ViewDidLoad() method. Also create a function for creating an ARController object.

override func viewDidLoad() {
    super.viewDidLoad()

    configureUI()
}

private func createARView() {
    // Creates an ARController that uses colors to mask room walls for 
    // painting instead of geometry. Can be used with any device.
    arController = RemodelARLib.makeLidarARController(with: arscnView)
}

private func configureUI() {
    createARView()
}

Step 6

Run the project and allow camera permission, then test that the camera feed displays correctly.

Step 7

Open Main.storyboard and add a CollectionView, then name it ColorPickerCollectionView. Don't forget to connect its Outlet.

Update the bottom constraint to safe area. Also, add any other necessary constraints. Your CollectionView should now be placed at the bottom of the ViewController. This CollectionView will work as a ColorPicker.

In the CollectionView's cell, add a UIView. Next, add all the necessary implementation details for the CollectionView and its cell.

Step 8

Now create an array of WallPaint

let colors = [WallPaint(id: "0", color: .orange),
              WallPaint(id: "1", color: .red),
              WallPaint(id: "2", color: .yellow),
              WallPaint(id: "3", color: .green),
              WallPaint(id: "4", color: .black),
              WallPaint(id: "5", color: .blue),
              WallPaint(id: "7", color: .brown),
              WallPaint(id: "8", color: .lightGray),
              WallPaint(id: "9", color: .darkGray),
              WallPaint(id: "10", color: .systemPink),
              WallPaint(id: "11", color: .purple),
              WallPaint(id: "12", color: .cyan),
              WallPaint(id: "14", color: .magenta),
              WallPaint(id: "15", color: .gray)]

Set this array as the CollectionView's datasource. Set WallPaint's color as the UIView's background color in the cell.

After finishing the CollectionView's setup, run the app and verify that the CollectionView displays different colored cells you can scroll through.

Step 9

Open Main.storyboard and add five buttons, then change the first four buttons title to Average, Light, Dark and Brightness. Clear the remaining button's title to "" and update its image to the reset image.

Embed these buttons in a Horizontal StackView. Now place this StackView on top of the ColorPickerCollectionView. Update the StackView's constraints as needed.

Now, connect the button's IBActions to ShaderPaintViewController.swift. Next, modify the IBActions as shown below.

@IBAction func onAverageTapped(_ sender: PaintyButton) {
    arController.setTouchMode(mode: TouchMode(rawValue: 0)!)
}

@IBAction func onDarkTapped(_ sender: PaintyButton) {
    arController.setTouchMode(mode: TouchMode(rawValue: 1)!)
}

@IBAction func onLightTapped(_ sender: PaintyButton) {
    arController.setTouchMode(mode: TouchMode(rawValue: 2)!)
}

@IBAction func onBrightnessTapped(_ sender: PaintyButton) {
    arController.setTouchMode(mode: TouchMode(rawValue: 3)!)
}

@IBAction func onResetTapped(_ sender: PaintyButton) {
    arController.resetScene()
}

Step 10

Open Main.storyboard and add another three buttons, then change the buttons titles to Record, Stop and Idle.

Embed these buttons in a Horizontal StackView. Now place this StackView on top of the above-created buttons. Update the StackView's constraints as needed.

Next, connect the buttons' IBActions to ShaderPaintViewController.swift, then modify the IBActions as shown below.

@IBAction func onRecordTapped(_ sender: PaintyButton) {
    arController.setABTestingMode(mode: 0)
}
    
@IBAction func onStopTapped(_ sender: PaintyButton) {
    arController.setABTestingMode(mode: 1)
}
    
@IBAction func onIdleTapped(_ sender: PaintyButton) {
   arController.setABTestingMode(mode: 2)
}

Step 11

Open Main.storybarod and create a UISlider, then place it on top of above-created buttons. Connect its outlet to ShaderPaintViewController.swift and name it thresholdSlider.

Now, modify the configureUI() method as shown below:

private func configureUI() {
   createARView()
   thresoldSlider.value = 10
}

Next, create an IBAction for the UISlider in ShaderPaintViewController.swift.

@IBAction func onThresoldSliderChanged(_ sender: UISlider) {
    arController.setColorThreshold(threshold: sender.value) //Sets the current threshold used for masking foreground objects using colors
}

Step 12

Open Main.storyboard and add a new button on top of the above-created Slider. Clear the button's title and set its image to camera.fill.

Next, create an IBAction for the button in ShaderPaintViewController.swift.

@IBAction func onSaveToCameraTapped(_ sender: PaintyButton) {
    let savedImage = arController.savePhoto()
    UIImageWriteToSavedPhotosAlbum(savedImage, self, nil, nil)
}

Step 13

Open ShaderPaintViewController.swift and create two properties as shown below:

private var outerCenterPoint = CenterPoint()
private var innerCenterPoint = CenterPoint()

Next, Create two methods in same file.

private func createCenterPoint() {

    outerCenterPoint = CenterPoint(frame: CGRect(x: view.frame.width/2 - 30, y: view.frame.height/2 - 30, width: 60, height: 60))
    innerCenterPoint = CenterPoint(frame: CGRect(x: view.frame.width/2 - 8, y: view.frame.height/2 - 8, width: 16, height: 16))
        
    view.addSubview(outerCenterPoint)
    view.addSubview(innerCenterPoint)
}

private func showCenterPoint(index: Int) {
        
   outerCenterPoint.isHidden = index == 0 ? false : true
   innerCenterPoint.isHidden = index == 0 ? false : true
}

Next, modify the createARView() method:

private func createARView() {
    arController = RemodelARLib.makeShaderARController(with: arscnView)
    createCenterPoint()
}

Step 14

Create two methods for adding a UIPanGestureRecognizer on AR.

private func addGestureOnARView() {
    let dragGesture = UIPanGestureRecognizer(target: self, action: #selector(onDraggingARView(_:)))
    arscnView.isUserInteractionEnabled = true
    arscnView.addGestureRecognizer(dragGesture)
}
    
@objc private func onDraggingARView(_ sender: UIPanGestureRecognizer) {
    let gestureState = sender.state
        
    switch gestureState {
        case .changed:
            arController.dragStart(point: sender.location(in: arscnView))
            arController.dragMove(point: sender.location(in: arscnView))
        case .ended:
            arController.dragEnd()
        default:
            break
   }
}

Next, modify the createARView() method as shown below:

private func createARView() {
    arController = RemodelARLib.makeShaderARController(with: arscnView)

    addGestureOnARView()

    arController.setColor(paint: colorPicker[0].color) //Use this to set the current selected color that will be applied to the next mesh that is tapped
    arController.setTouchMode(mode: TouchMode(rawValue: 3)!)
        
    createCenterPoint()
}

Step 15

Finally, Override the touchesEnded(_ touches: Set, with event: UIEvent?) method in ShaderPaintViewController.swift.

public override func touchesEnded(_ touches: Set<UITouch>, with event: UIEvent?) {
   guard let touch = touches.first else { return }

   let point = touch.location(in: arscnView)
   // Call this method when the user touches the ARSCNView for mesh 
   // hit testing to determine which mesh to color when the users 
   // taps on the view
   arController.handleTouch(point: point)
}

Now, you should be able to use colors to mask room walls for painting instead of geometry.

If at any point you need help from the Passio team, please reach out to us at [e-mail]