The Passio Android SDK is shipped in the form of an .aar file.

Download the .aar from our distribution repository

• Visit the releases page of the Android-Passio-SDK-Distribution GitHub repository. Download the passiolib-release.aar file from the latest release.

• If you don't have access to the repository please contact our support team at support@passiolife.com

Include the .aar in your Android Studio project

To include the .aar file into your project go to the module that will contain the food recognition feature. Place the .aar file in the libs folder of the module. If the libs folder is missing, create one. Reference the .aar file in the build.gradle file of the implementing module.

implementation files('libs/passiolib-release.aar')

Sync Project with Gradle Files and be sure that you can reference the PassioSDK class within your code.

Dependencies

Passio Android SDK is powered by TensorFlow and FirebaseVision with the camera being managed by CameraX. Add the dependencies to these three projects by adding these lines to the module's build.gradle file.

dependencies {
    // TensorFlow
    implementation 'org.tensorflow:tensorflow-lite:2.8.0'
    implementation 'org.tensorflow:tensorflow-lite-metadata:0.4.0'

    // CameraX
    def camerax_version = "1.3.0-alpha12"
    implementation "androidx.camera:camera-core:$camerax_version"
    implementation "androidx.camera:camera-camera2:$camerax_version"
    implementation "androidx.camera:camera-lifecycle:$camerax_version"
    implementation 'androidx.camera:camera-view:1.3.0-alpha02'
    implementation 'androidx.camera:camera-extensions:1.3.0-alpha02'

    // Barcode and OCR
    implementation 'com.google.android.gms:play-services-mlkit-text-recognition:18.0.2'
    implementation 'com.google.android.gms:play-services-mlkit-barcode-scanning:18.1.0'
}

In order for the SDK to work add the following lines to the android section of the module's build.gradle file.

android {
    aaptOptions {
        noCompress "passiosecure"
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
}

Running Sync Project with Gradle Files will enable access to the SDK.

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

To start using the camera in your Activity/Fragment, implement the PassioCameraViewProvider interface. By implementing this interface the SDK will use that component as the lifecycle owner of the camera (when that component calls onPause() the camera will stop) and also will provide the Context in order for the camera to start. The component implementing the interface must have a PreviewView in its view hierarchy.

Start by adding the PreviewView to your view hierarchy. Go to your layout.xml and add the following.

<androidx.camera.view.PreviewView
    android:id="@+id/myPreviewView"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

The SDK requires the device's Camera to work. To ensure that the Camera is working and sending frames to analyze, you need the user's permission to access the camera. As of now, only the back camera is supported by the SDK. There are two ways to implement Camera usage along with the permission request. Review both of these strategies and choose what best suits your code.

Using the PassioCameraFragment

PassioCameraFragment is an abstract class that handles Camera permission at runtime as well as starting the Camera process of the SDK. To use the PassioCameraFragment simply extend it in your own fragment and supply the PreviewView that has been added to the view hierarchy in the previous step.

class MyFragment : PassioCameraFragment() {
	
    override fun getPreviewView(): PreviewView {
	return myPreviewView
    }

    override fun onCameraReady() { 
    	// Proceed with initializing the recognition session
    }

    override fun onCameraPermissionDenied() { 
    	// Explain to the user that the camera is needed for this feature to
        // work and ask for permission again
    }
}

Using the PassioCameraViewProvider

This approach is more manual but gives you more flexibility. You need to implement the PassioCameraViewProvider interface and supply the needed LifecycleOwner and the PreviewView added in the initial step.

class MainActivity : AppCompatActivity(), PassioCameraViewProvider {
	
    override fun requestCameraLifecycleOwner(): LifecycleOwner {
        return this
    }

    override fun requestPreviewView(): PreviewView {
        return myPreviewView
    }
}

Acquire the user's permission to use the Camera with these steps

Finally, after the user has granted permission to use the camera, start the SDK camera

override fun onStart() {
    super.onStart()
    if (!hasPermissions()) {
        ActivityCompat.requestPermissions(
            this,
            REQUIRED_PERMISSIONS,
            REQUEST_CODE_PERMISSIONS
        )
        return
    } else {
        PassioSDK.instance.startCamera(this /*reference to the PassioCameraViewProvider*/)
    }
}

Run the code to make sure that the preview is working

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

Welcome to the Passio Nutrition-AI Android SDK!

Two things are required for the SDK to work: a proper license key and the neural network files that the SDK uses to run the recognition process.

• The license key can be requsted at support@passiolife.com

• The files are shipped inside the PassioSDKSandbox project on the releases page of the Android-Passio-SDK-Distribution repository. Unzip the PassioSDKSandbox project and locate the files in the assets folder: /PassioSDKSandbox/app/src/main/assets. Files that the SDK needs have an extension .passiosecure and are encrypted

There are several ways to initialize the SDK depending on the location of the needed files:

1. Files are in the assets folder of your Android project

2. Let the SDK manage file download and update

3. Host the files on your own server, download the files, store them on the device's memory and point the SDK to the URIs of the needed files

1. Files are located in the Assets folder

• If the assets folder is not already present in your Android project, create one in the module where the SDK dependencies are located under the main directory (e.g. app/src/main/assets)

• Transfer the .passiosecure files to the assets folder

• Create a PassioConfiguration object with the application context and the license key

• Call the configure method of the PassioSDK without changing the default PassioConfiguration object

val passioConfiguration = PassioConfiguration(
    this.applicationContext,
    "Your license key here"
)

PassioSDK.instance.configure(passioConfiguration) { passioStatus ->
    // Handle PassioStatus states
}

2. The SDK will manage the download and updates

val passioConfiguration = PassioConfiguration(
    this.applicationContext,
    "Your license key here"
).apply {
    sdkDownloadsModels = true
}

PassioSDK.instance.configure(passioConfiguration) { passioStatus ->
    // Handle PassioStatus states
} 

To track the download process you can implement a PassioStatusListener (recommended to define after the PassioStatus mode returns IS_DOWNLOADING_MODELS)

3. Files are located in local device storage

This approach is recommended if you are responsible for the download of the files. In this case the SDK does not care where the files are hosted and how they are downloaded, just where they are stored after the download has completed. Use the PassioConfiguration object to modify the initialization process with local files and pass the URIs of the files to the localFiles field.

val passioConfiguration = PassioConfiguration(
    this.applicationContext,
    "Your license key here"
).apply {
    localFiles = /*List of files URIs*/
}

PassioSDK.instance.configure(passioConfiguration) { passioStatus ->
    // Handle PassioStatus states
} 

Handle the configuration response

When you call the configure method of the Passio SDK with the PassioConfiguration object, you will need to define a callback to handle the result of the configuration process. This result is comprised in the PassioStatus object.

class PassioStatus {
    var mode: PassioMode
    var missingFiles: List<FileName>?
    var debugMessage: String?
    var activeModels: Int?
}

The mode of PassioStatus defines what the current status of the configuration process is. There are 5 different modes, and they all should be handled by the implementing side.

enum class PassioMode {
    NOT_READY,
    FAILED_TO_CONFIGURE,
    IS_BEING_CONFIGURED,
    IS_DOWNLOADING_MODELS,
    IS_READY_FOR_DETECTION
}

• NOT_READY -> The configuration process hasn't started yet.

• FAILED_TO_CONFIGURE -> The configuration process resulted in an error. Be sure to check the debugMessage of the PassioStatus object to get more insight into the nature of the error.

• IS_BEING_CONFIGURED -> The SDK is still in the configuration process. Normally, you shouldn't receive this mode as a callback to the configure method.

• IS_DOWNLOADING_MODELS -> The files required for the SDK to work are not present and are currently being downloaded.

• IS_READY_FOR_DETECTION -> The configuration process is over and all the SDKs functionalities are available.

The missingFiles variable contains a list of all the files missing for the SDK to work optimally. When this list is not null, this doesn't mean that the SDK is not operational. The SDK may be initialized with older files, but it will advocate the download of the newest files through this list.

The debugMessage usually gives more verbose and human-readable information about the configuration process. It will always give an error message if the mode is FAILED_TO_CONFIGURE.

The activeModels will give the version of the files installed and ran by the SDK if the configuration process ran successfully i.e. if the mode is IS_READY_FOR_DETECTION.

Best practices

1. The PassioStatus mode is NOT_READY -> check debugMessage for the error, usually the wrong license key has been supplied or the files are missing.

2. The PassioStatus mode is IS_DOWNLOADING_MODELS -> register a PassioStatusListener through the PassioSDK.instance.setPassioStatusListener() method as it will enable tracking of the download progress.

3. The PassioStatus mode is IS_READY_FOR_DETECTION -> still check the missingFiles because the SDK might be running with an older version of the files and newer need to be downloaded.

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

To start the Food Recognition process a FoodRecognitionListener also has to be defined. The listener serves as a callback for all the different food detection processes defined by the FoodDetectionConfiguration.

Using the listener and the detection options start the food detection by calling the startFoodDetection method of the SDK.

Stop the food recognition in the onStop() lifecycle callback.

Try to run the code containing the foodListener defined above. Point the phone at the image below and see if you are getting the correct food printed on the screen (it should be red apples, but if it isn't don't worry, we'll cover the data structure later).

When starting the food detection with the FoodRecognitionListener as the callback, on every frame analyzed you will receive a FoodCandidates object. This is the structure of that data class:

Depending on how you structured the FoodDetectionConfiguration object, some fields of the FoodCandidates object might be null. If the field is null it means that that type of detection has not been run. On the other hand, if the detection process ran to completion and the list is empty, then that type of detection could not recognize anything from the given frame.

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

Visual food detection

A DetectedCandidate represents the result from running Passio's neural network, specialized for detecting foods like apples, salads, burgers etc.

Barcode detection

Try to opt in for BARCODE by enabling the flag detectBarcodes and point your device at the image of the barcodes below. The SDK's barcode detector will recognize the barcode and it will return the barcode number inside the BarcodeCandidate object.

Use the barcode field to query Passio's backend and if the scanned food can be found in Passio's database, a PassioIDAttributes object will be returned.

Packaged food detection

Passio uses to recognize names of branded foods. Opt in with FoodDetectionConfiguration.detectPackagedFood and if the SDK has recognized a branded food that is present in Passio's Nutritional Database it will return the PackagedFoodCandidate of that food item. To retrieve full information about the item call:

The FoodCandidate returns a list of VisualCandidates, which object should I choose to display to the user?

The Visual Detection process can detect up to 5 objects per frame. Depending on your UI/UX you can display all the results to the end user, or just choose one of them. The criteria to choose the best candidate from a list is up to you, but the two most common strategies are the biggest bounding box or the highest confidence. This biggest bounding box represents the biggest object on the frame and the highest confidence points to the object that the neural network is most sure about.

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

With every shipped version of the SDK there will also be a PassioSDKSandbox containing the latest files and showcasing the basic setup of an app using the SDK. In this section we will cover the architecture of the small app and all the calls to the SDK keeping in mind where they are located and when they are called.

The app consists of one MainActivity that holds 2 fragments: RecognizerFragment and PermissionFragment. The latter will be displayed to the user if they deny access to the device camera. The RecognitionFragment is used to display the camera preview, start the recognition session and display the results through a view called FoodResultView.

PassioSDK library

The .aar file has already been imported as a separate module called passiolib-release. If you want to see how Android Studio structures the module, you can right-click on the passiolib-release module in the project viewer and choose the option "Open in". The passiolib-release module has been added as a dependency to the app modules, where we will use the SDK.

Camera

As mentioned above, the camera preview will be used in the RecognizerFragment. If you follow that fragment's onCreateView method you will see that the fragment inflates the view hierarchy found in the file fragment_recognizer.xml. Located in that view hierarchy we have the CameraX's PreviewView with an ID of "recognizerPreviewView". Taking a look back to the RecognizerFragment we can see that this component implements the PassioCameraViewProvider interface. Also, in the fragment's onStart lifecycle callback we are calling the

to start the preview and link the camera to the SDK. We don't need to stop the camera on onStop because we are passing the LifecycleObserver reference to the startCamera method. The camera will stop automatically when it needs to.

SDK Initialization

The SDK is initialized and configured in the onCreate method in the MainActivity.

This configure method means that we are using the files located in the assets folder of the app module. Verify that there are 4 files in the assets folder with the extension .passiosecure.

Detection session

The session is defined in the onStart and onStop lifecycle callbacks of the RecognizerFragment.

As described in the section Food detection session, we define a FoodDetectionConfiguration object and through it opt in for VISUAL and BARCODE detection. We then pass this object to startFoodDetection to receive results through the foodListener. When this fragment's onStop is called we are stopping the food detection process so that any results generated in the SDK after this point won't be delivered causing leaks.

The foodListener above only takes into account the candidates from the detectedCandidates and barcodeCandidate lists because we only opted in for VISUAL and BARCODE detection. The other files of the candidates object received in the onRecognitionResults method will be null. Depending on the candidates within these two lists we will change the state of the FoodResultView denoted as recognizerFoodResultView.

Taking a closer look at the FoodResultView we can see how to get food details such as the name or the icon depending on the source of the detection. If the passioID comes from the FOOD detection we can obtain the PassioIDAttributes through the

On the other hand, if the results come from the BARCODE detection, we can get the PassioIDAttributes from the networking call

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

Check out the full code here. FULL CODE

Once the food detection process has returned a PassioID, it is time to query the database and see what information Passio has on the detected food.

The most detailed information can be found in the PassioIDAttributes object. To retrieve this object from a given PassioID, simply call

PassioIDAttributes

Here we will cover the fields that are not self-explanatory:

1. imageName - An identifier for the image of the food. To retrieve the for the given image name use

The image for a food can also be retrieved using the PassioID and the method

1. parents, children, siblings - All the foods in Passio's Nutritional Database are organized in a tree-like structure with each PassioID being a node in that tree. Let's say we scanned a "mandarin". A child of a "mandarin" is a "tangerine", and its parent is a "whole citrus fruit". We can offer these alternatives to the user in case the user is pointing to a similar food to a mandarin.

2. passioFoodItemDataForDefault, passioFoodRecipe - The SDK differentiates single food items and food recipes. A single item represents just one particular food like an apple or peas. A food recipe consists of more than one single food item. For example a recipe is a Caprese salad which consists of mozzarella, tomatoes, basil, salt, black pepper and olive oil.

PassioFoodItemData

PassioFoodItemData holds the nutritional information for a food item like calories, carbs, protein, fat etc. It also has a reference weight on which these values are calculated and contains serving sizes and serving units. Serving sizes offer you quick options to select the size of the food with a given quantity and unit.

PassioFoodRecipe

Recipes should be viewed as a collection of PassioFoodItemData objects.

Other useful queries

If you have a PassioID returned from the detection process and you are just interested in the name of the corresponding food, use

To query the names of the foods from the database by a filter use

The returned list contains a PassioID paired with the name of the food.

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

Before you continue, please make sure you have a new key for version 2.x. The 1.4.x will not work with 2.x SDK!

Dependencies

Changes to the build.gradle file:

API Changes

• PassioMode.IS_AUTO_UPDATING renamed to PassioMode.IS_DOWNLOADING_MODELS

• PassioMode.IS_READY_FOR_NUTRITION was removed.

• PassioSDK.instance.setDownloadListener was removed. Instead, the download process is traced through the PassioStatusListener

• FoodDetectionConfiguration.detectOCR renamed to detectPackagedFood

• FoodDetectionConfiguration.detectLogos was removed.

• PassioSDK.instance.lookupImageForFilename, lookupPassioIDFor, lookupAlternativesFor, lookupAllSiblingsFor, lookupParentFor, lookupAllFoodItemsInDatabase, lookupAllPassioIDs, lookupAllVisuallyRecognizableFoodItems were removed

• PassioSDK.instance.searchForFood now returns a list of Pair<passioID, food name>

• PassioSDK.instance.lookupImageFor was separated into two functions: 1. Local image lookup: lookupIconFor 2. Network image fetch: fetchIconFor

• Nutrients in the PassioFoodItemData have been refactored from Double to UnitMass

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

In this example, we'll use a Fragment to set up the SDK configuration. Add a new Fragment class named ConfigFragment.kt to handle the configuration process.

Here is the ConfigFragment that configures the Passio SDK:

package com.passioai.quickstart.ui.config

import ai.passio.passiosdk.core.config.PassioConfiguration
import ai.passio.passiosdk.core.config.PassioMode
import ai.passio.passiosdk.passiofood.PassioSDK
import android.os.Bundle
import androidx.fragment.app.Fragment
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import androidx.navigation.fragment.findNavController
import com.passioai.quickstart.BuildConfig
import com.passioai.quickstart.databinding.FragmentConfigBinding

class ConfigFragment : Fragment() {

    private lateinit var _binding: FragmentConfigBinding
    private val binding get() = _binding

    override fun onCreateView(
        inflater: LayoutInflater, container: ViewGroup?,
        savedInstanceState: Bundle?
    ): View {
        _binding = FragmentConfigBinding.inflate(inflater, container, false)
        return _binding.root
    }

    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        configureSDK()
    }

    private fun configureSDK() {
        val passioConfiguration = PassioConfiguration(
            requireActivity().applicationContext,
            BuildConfig.SDK_KEY
        ).apply {
            sdkDownloadsModels = true
            debugMode = 0
            remoteOnly = true
        }

        PassioSDK.instance.configure(passioConfiguration) { passioStatus ->
            when (passioStatus.mode) {
                PassioMode.NOT_READY -> onSDKError("Not ready")
                PassioMode.FAILED_TO_CONFIGURE -> onSDKError(getString(passioStatus.error!!.errorRes))
                PassioMode.IS_READY_FOR_DETECTION -> onSDKReadyToUse()
                PassioMode.IS_BEING_CONFIGURED -> {
                    binding.tvSDKStatus.text = "Configuring..."
                }
                PassioMode.IS_DOWNLOADING_MODELS -> {
                    binding.tvSDKStatus.text = "Downloading models..."
                }
            }
        }
    }

    private fun onSDKError(message: String) {
        binding.tvSDKStatus.text = message
    }

    private fun onSDKReadyToUse() {
        binding.tvSDKStatus.text = "Configured and ready to use"
        findNavController().navigate(ConfigFragmentDirections.configToImageRecognitions())
    }
}

Key Points to Note

1. PassioConfiguration:

   • Set up PassioConfiguration with your context and SDK key.

   • Enable sdkDownloadsModels to allow the SDK to download necessary models.

   • remoteOnly = true enables the SDK to access remote resources if required.

2. PassioMode States: The SDK configuration callback provides different statuses:

   • NOT_READY: SDK is not ready, usually due to an issue.

   • FAILED_TO_CONFIGURE: Configuration failed.

   • IS_READY_FOR_DETECTION: SDK is configured and ready.

   • IS_BEING_CONFIGURED: SDK is currently being configured.

   • IS_DOWNLOADING_MODELS: SDK is downloading required models.

3. Navigation on Ready:

   • On successful configuration, the app navigates to ImageRecognitionsFragment.

Layout Setup

To display the configuration status, add a TextView (e.g., tvSDKStatus) in the fragment’s XML layout file.

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    tools:context=".ui.config.ConfigFragment">


    <androidx.appcompat.widget.AppCompatTextView
        android:id="@+id/tvSDKStatus"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        tools:text="Hello World!" />

</androidx.constraintlayout.widget.ConstraintLayout>

Error Handling

The SDK setup can fail due to configuration or network issues. The onSDKError function in the ConfigFragment updates the UI with the relevant error message to help troubleshoot.

Check out the full code here. FULL CODE

1. Add the SDK dependency using Maven: Ensure you have the correct repository and dependency in your build.gradle file.

In the build.gradle file add the PassioSDK dependency

dependencies {
    implementation 'ai.passio.passiosdk:nutrition-ai:3.2.1-3'
}

Dependencies

dependencies {
    // TensorFlow
    implementation 'org.tensorflow:tensorflow-lite:2.8.0'
    implementation 'org.tensorflow:tensorflow-lite-metadata:0.4.0'

    // CameraX
    def camerax_version = "1.3.4"
    implementation "androidx.camera:camera-core:$camerax_version"
    implementation "androidx.camera:camera-camera2:$camerax_version"
    implementation "androidx.camera:camera-lifecycle:$camerax_version"
    implementation "androidx.camera:camera-view:$camerax_version"
    implementation "androidx.camera:camera-extensions:$camerax_version"

    // Barcode and OCR
    implementation 'com.google.android.gms:play-services-mlkit-text-recognition:19.0.0'
    implementation 'com.google.android.gms:play-services-mlkit-barcode-scanning:18.3.0'
}

1. Set up SDK Key: Obtain your SDK key from Passio’s Developer Portal. Add it to your BuildConfig or securely store it as an environment variable in your project.

Check out the full code here. FULL CODE

The FoodDetailsFragment is a core component designed to display detailed information about a selected food item, including its name, icon, calories, macronutrients, and a list of micronutrients. Users can adjust serving sizes and quantities to view nutrient values accordingly.

The primary components in FoodDetailsFragment include:

1. Food Item Information: Displays the name, icon, and serving size details.

2. Serving Size Selector: Allows users to select a preferred serving size or unit.

3. Nutritional Information: Shows calorie count and macronutrient breakdown (Carbs, Protein, Fat).

4. Micronutrient List: Provides detailed nutrient information for vitamins, minerals, and other nutrients.

PassioFoodItem

The PassioFoodItem model is the foundation of food details and includes the following properties:

Example:

The food item name, icon, and details are rendered using PassioFoodItem's properties.

PassioFoodAmount

This model defines possible serving sizes and units. PassioFoodAmount includes:

Example:

The servingSizes property is used to populate the RecyclerView adapter ServingSizeAdapter, allowing users to select different serving sizes.

PassioServingSize and PassioServingUnit

These classes are components of PassioFoodAmount that define the quantity and units for serving sizes:

PassioServingSize

PassioServingUnit

Setting Serving Size Quantity

The setQuantityAndUnit function allows users to specify a desired serving quantity and unit, updating the displayed nutrient values dynamically.

FoodDetailsFragment.kt

Check out the full code here.

Create an ImageRecognitionsFragment to handle image capture, gallery selection, and food item recognition.

Step 1: Select or Capture Image

For selecting or capturing an image, use PhotoPicker and PhotoCapture functions in ImageRecognitionsFragment. Refer to Android's official documentation for detailed instructions:

• Image Picker (Android)

• Camera Intent (Android)

package com.passioai.quickstart.ui.recognitions

import ai.passio.passiosdk.passiofood.PassioSDK
import android.graphics.Bitmap
import android.net.Uri
import android.os.Bundle
import androidx.fragment.app.Fragment
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import android.widget.Toast
import androidx.core.view.isVisible
import com.passioai.quickstart.databinding.FragmentImageRecognitionsBinding
import com.passioai.quickstart.util.PhotoCaptureManager
import com.passioai.quickstart.util.PhotoPickerManager
import com.passioai.quickstart.util.uriToBitmap

class ImageRecognitionsFragment : Fragment() {

    private lateinit var _binding: FragmentImageRecognitionsBinding
    private val binding get() = _binding
    private val photoPickerManager = PhotoPickerManager()
    private val photoCaptureManager = PhotoCaptureManager()
    private var bitmap: Bitmap? = null

    override fun onCreateView(inflater: LayoutInflater, container: ViewGroup?, savedInstanceState: Bundle?): View {
        _binding = FragmentImageRecognitionsBinding.inflate(inflater, container, false)
        return binding.root
    }

    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        photoPickerManager.init(this) { uris -> onImagePicked(uris) }
        photoCaptureManager.init(this) { uri -> onImageCaptured(uri) }

        binding.btnCamera.setOnClickListener { photoCaptureManager.capturePhoto() }
        binding.btnGallery.setOnClickListener { photoPickerManager.pickSingleImage() }
        binding.btnSubmit.setOnClickListener { fetchResult() }
    }

    private fun onImagePicked(uris: List<Uri>) {
        if (uris.isNotEmpty()) {
            uriToBitmap(requireContext(), uris[0])?.let {
                bitmap = it
                binding.ivPhoto.load(bitmap)
            }
        }
    }

    private fun onImageCaptured(uri: Uri) {
        uriToBitmap(requireContext(), uri)?.let {
            bitmap = it
            binding.ivPhoto.load(bitmap)
        }
    }
}

Step 2: Recognize Food Item

The fetchResult() method sends the selected image to the Passio SDK for recognition.

private fun fetchResult() {
    if (bitmap == null) {
        Toast.makeText(requireContext(), "Please select an image", Toast.LENGTH_SHORT).show()
        return
    }
    binding.loader.isVisible = true
    PassioSDK.instance.recognizeImageRemote(bitmap!!) { result ->
        binding.loader.isVisible = false
        val adapter = ImageRecognitionsAdapter(result) { info -> onShowDetails(info) }
        binding.list.adapter = adapter
    }
}

PassioSDK.instance.recognizeImageRemote

The recognizeImageRemote function takes an image in Bitmap format, analyzes it remotely using Passio's servers, and returns a list of food items recognized within the image. This function is asynchronous and utilizes a callback to return the result once the analysis is complete.

Output

The function returns a list of PassioAdvisorFoodInfo objects through a callback

The structure of PassioAdvisorFoodInfo is as follows:

• recognisedName (String): The name of the recognized food item.

• portionSize (String): Suggested portion size for the recognized food.

• weightGrams (Double): Estimated weight of the food item in grams.

• foodDataInfo (PassioFoodDataInfo?): An optional field containing more detailed nutritional information if available.

• packagedFoodItem (PassioFoodItem?): An optional field containing packaged food item data if the recognized item is a packaged product.

• resultType (PassioFoodResultType): The type of result, indicating if the food is freshly prepared, packaged, or a general item.

Step 3: Display Food Details

Navigate to a details screen upon item selection.

private fun onShowDetails(passioAdvisorFoodInfo: PassioAdvisorFoodInfo) {
        if (passioAdvisorFoodInfo.foodDataInfo != null) {

            PassioSDK.instance.fetchFoodItemForDataInfo(passioAdvisorFoodInfo.foodDataInfo!!, passioAdvisorFoodInfo.foodDataInfo!!.nutritionPreview.servingQuantity, passioAdvisorFoodInfo.foodDataInfo!!.nutritionPreview.servingUnit){ passioFoodItem->
                passioFoodItem?.let {
                    navigateToDetails(it)
                }
            }
        }
        else if (passioAdvisorFoodInfo.packagedFoodItem != null) {
            navigateToDetails(passioAdvisorFoodInfo.packagedFoodItem!!)

        }

}

private fun navigateToDetails(foodItem: PassioFoodItem) {
    FoodDetailsFragment.setPassioFoodItem(foodItem)
    findNavController().navigate(ImageRecognitionsFragmentDirections.imageRecognitionsToFoodDetails())
}

PassioSDK.instance.fetchFoodItemForDataInfo

The fetchFoodItemForDataInfo function retrieves detailed food information from Passio's database based on provided data fields. This function is also asynchronous and accepts a callback that returns a PassioFoodItem object with the full nutritional profile of the requested food item.

Input Parameters

• foodDataInfo (PassioFoodDataInfo): A data object containing metadata about the food item, including its ID and a preview of its nutrition details.

• servingQuantity (Double): The quantity or portion size of the food.

• servingUnit (String): The unit of measurement for the serving size, such as grams, ounces, or pieces.

Output

The function returns a PassioFoodItem through the callback. This object contains the full nutritional breakdown of the food item, which can include:

• Nutritional values (e.g., calories, protein, fat, carbohydrates)

• Serving size details

• Icons and images associated with the item

Check out the full code here. FULL CODE