Food detection session

The SDK can detect 3 different categories: VISUAL, BARCODE and PACKAGED. The VISUAL recognition is powered by Passio's neural network and is used to recognize over 3000 food classes. BARCODE, as the name suggests, can be used to scan a barcode located on a branded food. Finally, PACKAGED can detect the name of a branded food. To choose one or more types of detection, a FoodDetectionConfiguration object is defined and the corresponding fields are set. The VISUAL recognition works automatically.

val options = FoodDetectionConfiguration().apply {
    detectBarcodes = true
}

To start the Food Recognition process a FoodRecognitionListener also has to be defined. The listener serves as a callback for all the different food detection processes defined by the FoodDetectionConfiguration.

private val foodRecognitionListener = object : FoodRecognitionListener {
    override fun onRecognitionResults(
        candidates: FoodCandidates,
        image: Bitmap?,
        nutritionFacts: PassioNutritionFacts?
    ) {
        val detectedCandidates = candidates.detectedCandidates!!
        val barcodeCandidates = candidates.barcodeCandidates!!
        val passioID = detectedCandidates.first().passioID
        val foodName = PassioSDK.instance.lookupNameFor(passioID)
        Toast.makeText(requestContext(), foodName, Toast.LENGTH_SHORT).show()
    }
}

Only the corresponding candidate lists will be populated (e.g. if you define detection types VISUAL and BARCODE, you will never receive a packagedFoodCandidates list in this callback).

Using the listener and the detection options start the food detection by calling the startFoodDetection method of the SDK.

override fun onStart() {
    super.onStart()
    PassioSDK.instance.startFoodDetection(foodRecognitionListener)
}

Stop the food recognition in the onStop() lifecycle callback.

override fun onStop() {
    PassioSDK.instance.stopFoodDetection()
    super.onStop()
}

Try to run the code containing the foodListener defined above. Point the phone at the image below and see if you are getting the correct food printed on the screen (it should be red apples, but if it isn't don't worry, we'll cover the data structure later).

apple_img

When starting the food detection with the FoodRecognitionListener as the callback, on every frame analyzed you will receive a FoodCandidates object. This is the structure of that data class:

data class FoodCandidates(
    val detectedCandidates: List<DetectedCandidate>? = null,
    val barcodeCandidates: List<BarcodeCandidate>? = null,
    val packagedFoodCandidates: List<PackagedFoodCandidate>? = null
)

You can see the structure of all the detection classes in the file called PassioAPI.kt

Depending on how you structured the FoodDetectionConfiguration object, some fields of the FoodCandidates object might be null. If the field is null it means that that type of detection has not been run. On the other hand, if the detection process ran to completion and the list is empty, then that type of detection could not recognize anything from the given frame.

If at any point you need help from the Passio team, please reach out to us at [e-mail]