RecognizeImageRemote

Create an ImageRecognitionsFragment to handle image capture, gallery selection, and food item recognition.

Step 1: Select or Capture Image

For selecting or capturing an image, use PhotoPicker and PhotoCapture functions in ImageRecognitionsFragment. Refer to Android's official documentation for detailed instructions:

package com.passioai.quickstart.ui.recognitions

import ai.passio.passiosdk.passiofood.PassioSDK
import android.graphics.Bitmap
import android.net.Uri
import android.os.Bundle
import androidx.fragment.app.Fragment
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import android.widget.Toast
import androidx.core.view.isVisible
import com.passioai.quickstart.databinding.FragmentImageRecognitionsBinding
import com.passioai.quickstart.util.PhotoCaptureManager
import com.passioai.quickstart.util.PhotoPickerManager
import com.passioai.quickstart.util.uriToBitmap

class ImageRecognitionsFragment : Fragment() {

    private lateinit var _binding: FragmentImageRecognitionsBinding
    private val binding get() = _binding
    private val photoPickerManager = PhotoPickerManager()
    private val photoCaptureManager = PhotoCaptureManager()
    private var bitmap: Bitmap? = null

    override fun onCreateView(inflater: LayoutInflater, container: ViewGroup?, savedInstanceState: Bundle?): View {
        _binding = FragmentImageRecognitionsBinding.inflate(inflater, container, false)
        return binding.root
    }

    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        photoPickerManager.init(this) { uris -> onImagePicked(uris) }
        photoCaptureManager.init(this) { uri -> onImageCaptured(uri) }

        binding.btnCamera.setOnClickListener { photoCaptureManager.capturePhoto() }
        binding.btnGallery.setOnClickListener { photoPickerManager.pickSingleImage() }
        binding.btnSubmit.setOnClickListener { fetchResult() }
    }

    private fun onImagePicked(uris: List<Uri>) {
        if (uris.isNotEmpty()) {
            uriToBitmap(requireContext(), uris[0])?.let {
                bitmap = it
                binding.ivPhoto.load(bitmap)
            }
        }
    }

    private fun onImageCaptured(uri: Uri) {
        uriToBitmap(requireContext(), uri)?.let {
            bitmap = it
            binding.ivPhoto.load(bitmap)
        }
    }
}

Step 2: Recognize Food Item

The fetchResult() method sends the selected image to the Passio SDK for recognition.

private fun fetchResult() {
    if (bitmap == null) {
        Toast.makeText(requireContext(), "Please select an image", Toast.LENGTH_SHORT).show()
        return
    }
    binding.loader.isVisible = true
    PassioSDK.instance.recognizeImageRemote(bitmap!!) { result ->
        binding.loader.isVisible = false
        val adapter = ImageRecognitionsAdapter(result) { info -> onShowDetails(info) }
        binding.list.adapter = adapter
    }
}

PassioSDK.instance.recognizeImageRemote

The recognizeImageRemote function takes an image in Bitmap format, analyzes it remotely using Passio's servers, and returns a list of food items recognized within the image. This function is asynchronous and utilizes a callback to return the result once the analysis is complete.

Output

The function returns a list of PassioAdvisorFoodInfo objects through a callback

The structure of PassioAdvisorFoodInfo is as follows:

  • recognisedName (String): The name of the recognized food item.

  • portionSize (String): Suggested portion size for the recognized food.

  • weightGrams (Double): Estimated weight of the food item in grams.

  • foodDataInfo (PassioFoodDataInfo?): An optional field containing more detailed nutritional information if available.

  • packagedFoodItem (PassioFoodItem?): An optional field containing packaged food item data if the recognized item is a packaged product.

  • resultType (PassioFoodResultType): The type of result, indicating if the food is freshly prepared, packaged, or a general item.

Step 3: Display Food Details

Navigate to a details screen upon item selection.

private fun onShowDetails(passioAdvisorFoodInfo: PassioAdvisorFoodInfo) {
        if (passioAdvisorFoodInfo.foodDataInfo != null) {

            PassioSDK.instance.fetchFoodItemForDataInfo(passioAdvisorFoodInfo.foodDataInfo!!, passioAdvisorFoodInfo.foodDataInfo!!.nutritionPreview.servingQuantity, passioAdvisorFoodInfo.foodDataInfo!!.nutritionPreview.servingUnit){ passioFoodItem->
                passioFoodItem?.let {
                    navigateToDetails(it)
                }
            }
        }
        else if (passioAdvisorFoodInfo.packagedFoodItem != null) {
            navigateToDetails(passioAdvisorFoodInfo.packagedFoodItem!!)

        }

}

private fun navigateToDetails(foodItem: PassioFoodItem) {
    FoodDetailsFragment.setPassioFoodItem(foodItem)
    findNavController().navigate(ImageRecognitionsFragmentDirections.imageRecognitionsToFoodDetails())
}

PassioSDK.instance.fetchFoodItemForDataInfo

The fetchFoodItemForDataInfo function retrieves detailed food information from Passio's database based on provided data fields. This function is also asynchronous and accepts a callback that returns a PassioFoodItem object with the full nutritional profile of the requested food item.

Input Parameters

  • foodDataInfo (PassioFoodDataInfo): A data object containing metadata about the food item, including its ID and a preview of its nutrition details.

  • servingQuantity (Double): The quantity or portion size of the food.

  • servingUnit (String): The unit of measurement for the serving size, such as grams, ounces, or pieces.

Output

The function returns a PassioFoodItem through the callback. This object contains the full nutritional breakdown of the food item, which can include:

  • Nutritional values (e.g., calories, protein, fat, carbohydrates)

  • Serving size details

  • Icons and images associated with the item

Check out the full code here. FULL CODE

PreviousConfigure SDK and handle the resultNextFood Details

Last updated