Visual, Barcode and Packaged Food detection

Note: Some of these detections might not be accessible to you if your license does not include that module

Visual food detection

A DetectedCandidate represents the result from running Passio's neural network, specialized for detecting foods like apples, salads, burgers etc.

class DetectedCandidate(
    val passioID: PassioID,
    val confidence: Float,
    val boundingBox: RectF,
    val croppedImage: Bitmap?
)

The PassioID is a unique identifier for food that is used to query our Nutritional Database. See section Nutritional Database for more details.

Barcode detection

Try to opt in for BARCODE by enabling the flag detectBarcodes and point your device at the image of the barcodes below. The SDK's barcode detector will recognize the barcode and it will return the barcode number inside the BarcodeCandidate object.

data class BarcodeCandidate(val barcode: Barcode, val boundingBox: RectF)

Use the barcode field to query Passio's backend and if the scanned food can be found in Passio's database, a PassioIDAttributes object will be returned.

PassioSDK.instance.fetchPassioIDAttributesForBarcode(barcode) { passioIDAttribute ->
    if (passioIDAttribute == null) {
        return@fetchPassioIDAttributesForBarcode
    }
    Toast.makeText(requestContext(), passioIDAttribute.name, Toast.LENGTH_SHORT).show()
}

Packaged food detection

Passio uses OCR to recognize names of branded foods. Opt in with FoodDetectionConfiguration.detectPackagedFood and if the SDK has recognized a branded food that is present in Passio's Nutritional Database it will return the PackagedFoodCandidate of that food item. To retrieve full information about the item call:

PassioSDK.instance.fetchPassioIDAttributesForPackagedFood(
    packagedFoodCandidate.packagedFoodCode
) { passioIDAttributes ->    
    if (passioIDAttributes == null) {        
        return@fetchPassioIDAttributesForPackagedFood    
    }    
    Toast.makeText(requestContext(), passioIDAttribute.name, Toast.LENGTH_SHORT).show()
}

The FoodCandidate returns a list of VisualCandidates, which object should I choose to display to the user?

The Visual Detection process can detect up to 5 objects per frame. Depending on your UI/UX you can display all the results to the end user, or just choose one of them. The criteria to choose the best candidate from a list is up to you, but the two most common strategies are the biggest bounding box or the highest confidence. This biggest bounding box represents the biggest object on the frame and the highest confidence points to the object that the neural network is most sure about.

If at any point you need help from the Passio team, please reach out to us at [e-mail]