Barcode scanning
Real-time barcode scanning
The SDK can detect barcodes located on the packaging of packaged foods. To implement real-time barcode scanning, the camera setup for continuous image recognition is needed.
Camera preview
To start using camera detection the app must first acquire the permission to open the camera from the user. This permission is not handled by the SDK.
Add the UI element that is responsible for rendering the camera frames:
var videoLayer: AVCaptureVideoPreviewLayer?
func setupPreviewLayer() {
guard videoLayer == nil else { return }
if let videoLayer = passioSDK.getPreviewLayer() {
self.videoLayer = videoLayer
videoLayer.frame = view.bounds
view.layer.insertSublayer(videoLayer, at: 0)
}
}To start using the camera in your Activity/Fragment, implement the PassioCameraViewProvider interface. By implementing this interface the SDK will use that component as the lifecycle owner of the camera (when that component calls onPause() the camera will stop) and also will provide the Context in order for the camera to start. The component implementing the interface must have a PreviewView in its view hierarchy.
Start by adding the PreviewView to your view hierarchy. Go to your layout.xml and add the following.
<androidx.camera.view.PreviewView
android:id="@+id/myPreviewView"
android:layout_width="match_parent"
android:layout_height="match_parent" />Using the PassioCameraViewProvider
This approach is more manual but gives you more flexibility. You need to implement the PassioCameraViewProvider interface and supply the needed LifecycleOwner and the PreviewView added in the initial step.
class MainActivity : AppCompatActivity(), PassioCameraViewProvider {
override fun requestCameraLifecycleOwner(): LifecycleOwner {
return this
}
override fun requestPreviewView(): PreviewView {
return myPreviewView
}
}After the user has granted permission to use the camera, start the SDK camera
override fun onStart() {
super.onStart()
if (!hasPermissions()) {
ActivityCompat.requestPermissions(
this,
REQUIRED_PERMISSIONS,
REQUEST_CODE_PERMISSIONS
)
return
} else {
PassioSDK.instance.startCamera(this /*reference to the PassioCameraViewProvider*/)
}
}Using the PassioCameraFragment
PassioCameraFragment is an abstract class that handles Camera permission at runtime as well as starting the Camera process of the SDK. To use the PassioCameraFragment simply extend it in your own fragment and supply the PreviewView that has been added to the view hierarchy in the previous step.
class MyFragment : PassioCameraFragment() {
override fun getPreviewView(): PreviewView {
return myPreviewView
}
override fun onCameraReady() {
// Proceed with initializing the recognition session
}
override fun onCameraPermissionDenied() {
// Explain to the user that the camera is needed for this feature to
// work and ask for permission again
}
}import {
PassioSDK,
DetectionCameraView,
} from '@passiolife/nutritionai-react-native-sdk-v2';To show the live camera preview, add the DetectionCameraView to your view
<DetectionCameraView style={{flex: 1, width: '100%'}} />@override
Widget build(BuildContext context) {
return Scaffold(
body: Stack(
children: [
const PassioPreview(),
...
],
),
);
}Start barcode scanning session
To start a barcode scanning session, a callback for the results needs to be registered. Once the camera preview is active, if there is a registered barcode callback, the SDK will start analysing the frames and providing the results:
/// Use this function to detect barcodes by pointing the camera at a barcode
/// - Parameters:
/// - recognitionDelegate: ``BarcodeRecognitionDelegate``, Add self to implement the BarcodeRecognitionDelegate
/// - completion: If success, it will return array of `BarcodeCandidate` objects
public func startBarcodeScanning(recognitionDelegate: BarcodeRecognitionDelegate,
completion: @escaping (Bool) -> Void)
/// Implement the BarcodeRecognitionDelegate protocol to receive delegate method from the startBarcodeScanning
public protocol BarcodeRecognitionDelegate: AnyObject {
/// Delegate function for food recognition
/// - Parameters:
/// - candidates: Barcode candidates if available
func recognitionResults(barcodeCandidates: [BarcodeCandidate]?)
}fun startBarcodeScanning(
listener: BarcodeScanningListener
): Boolean
interface BarcodeScanningListener {
fun onBarcodeResult(candidates: List<BarcodeCandidate>)
} /**
* Start the camera and begin detecting barcode items.
*/
startBarcodeScanning(
callback: (detection: BarcodeScanEvent) => void
): Subscription
/**
* A collection of food candidates detected by the models.
*/
export interface BarcodeScanEvent {
/**
* Food candidate results from barcode scanning.
*/
barcodeCandidates?: BarcodeCandidate[]
}Future<void> startBarcodeScanning(BarcodeScanningListener listener) {
return NutritionAIPlatform.instance.startBarcodeScanning(listener);
}
/// Used as a callback to receive results from the barcode scanning process.
abstract class BarcodeScanningListener {
/// Method called to deliver barcode results.
void onBarcodeResult(List<BarcodeCandidate> candidates);
}Once the session is over, invoke the stopBarcodeScanning to deregister the listener.
Barcode scanning from an image
To recognise a barcode in an image, refer to the recognizeImageRemote function. The result of the barcode scanning process will come in the packagedFoodItem attribute, and the resultType will be Barcode.
UI Example
Set up the camera preview and detection session to recognise barcodes
Create a result view with two states: search and result
If the candidate.barcodeCandidates is empty, show the search state. Else, fetch the PassioFoodItem, and use it's name and icon to show the result
Example of an image that produces multiple BarcodeCandidates:
Last updated