arrow-left

All pages
gitbookPowered by GitBook
1 of 5

Loading...

Loading...

Loading...

Loading...

Loading...

Getting Started

hashtag
Welcome to the Passio Nutrition-AI React Native SDK!

This project provides React Native bindings for the Passio SDK.

Please note that the SDK will currently not run in the iOS simulator. We hope to change this in the future, but an iOS test device is required for now.

hashtag
Requirements:

  • React Native v0.60.0 or higher

  • Xcode 13.4 or higher

  • iOS 13 or higher

If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

Android API level 26 or higher
  • Cocoapods 1.10.1 or higher

  • Start food detection

    Once the SDK is ready, you can start food detection.

    hashtag
    Camera

    Android Platform: You need the user's permission to access the camera.

    <uses-permission android:name="android.permission.CAMERA">

    IOS Platform: Enter a value for NSCameraUsageDescription in your Info.plist so the camera may be utilized.

    hashtag
    Import the SDK

    To show the live camera preview, add the DetectionCameraView to your view

    hashtag
    The SDK can detect 4 different categories: VISUAL, BARCODE, PACKAGED FOOD and NUTRITION LABEL

    Try to run the above code in component and Point the phone to the image below and see if you are getting the correct food in console log Got food detection event

    If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

    import {
      PassioSDK,
      DetectionCameraView,
    } from '@passiolife/nutritionai-react-native-sdk-v2';
    apple_img
    <DetectionCameraView style={{flex: 1, width: '100%'}} />
    const config: FoodDetectionConfig = {
       /**
       * Detect barcodes on packaged food products. Results will be returned
       * as `BarcodeCandidates` in the `FoodCandidates` property of `FoodDetectionEvent`
       */
      detectBarcodes: true,
     
      /**
       * Results will be returned as DetectedCandidate in the `FoodCandidates`and 
       * property of `FoodDetectionEvent`
       */
    
      detectPackagedFood: true,
    
      /**
       * Detect barcodes on packaged food products. Results will be returned
       * under the `nutritionFacts` property on `FoodDetectionEvent`.
       */
      
      detectNutritionFacts: true,
    };
        
    useEffect(() => {
      if (!isReady) {
        return;
      }
      const subscription = PassioSDK.startFoodDetection(
        config,
        async (detection: FoodDetectionEvent) => {
    
          const { candidates, nutritionFacts } = detection
    
          if (candidates?.barcodeCandidates?.length) {
             
             // show barcode candidates to the user
    
          } else if (candidates?.packagedFoodCode?.length) {
            
            // show OCR candidates to the user
    
          } else if (candidates?.detectedCandidates?.length) {
            
            // show visually recognized candidates to the user
    
          } else if (nutritionFacts) {
    
            // Show scanned nutrition facts to the user
          }
        },
      );
      // stop food detection when component unmounts
      return () => subscription.remove(); 
    
    }, [isReady]);

    FoodDetectionEvent

    When starting food detection with PassioSDK.startFoodDetection you will receive a FoodDetectionEvent. This is the structure of that data class:

    export interface FoodDetectionEvent {
        /**
         * A collection of food candidates detected by the models.
         */
        candidates?: FoodCandidates;
        /**
         * Detected nutrition facts when scanning a nutrition label.
         */
        nutritionFacts?: NutritionFacts;
    }
    /**
     * A collection of food candidates detected by the models.
     */
    export interface FoodCandidates {
      /**
       * Food candidate results from visual scanning. The array is sorted by confidence, with the most confident result at index 0.
       */
      detectedCandidates: DetectedCandidate[]
    
      /**
       * Food candidate results from barcode scanning.
       */
      barcodeCandidates?: BarcodeCandidate[]
    
      /**
       * Food candidate results from packagedFoodCode scanning.
       */
      packagedFoodCode?: PackagedFoodCode[]
    }

    If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

    /**
     * Nutrition facts scanned from the nutrition label on a package food item
     */
    export interface NutritionFacts {
      servingSizeQuantity?: number
      servingSizeUnitName?: string
      servingSizeGram?: number
      servingSizeUnit: ServingSizeUnit
      calories?: number
      fat?: number
      carbs?: number
      protein?: number
      saturatedFat?: number
      transFat?: number
      cholesterol?: number
      sodium?: number
      dietaryFiber?: number
      sugars?: number
      sugarAlcohol?: number
    }

    Installation

    1. Create a Github Personal Access Token (classic)arrow-up-right with read:packages access selected.

    2. Create an .npmrc file in the root of your project with the following linesarrow-up-right replacing GITHUB_ACCESS_TOKEN with the token you've created.

    //npm.pkg.github.com/:_authToken=GITHUB_ACCESS_TOKEN
    @passiolife:registry=https://npm.pkg.github.com

    3. Add the Passio SDK dependency to your package.json and run npm install or yarn.

    npm install @passiolife/nutritionai-react-native-sdk-v2 

    or

    4. Ensure the native dependencies are linked to your app.

    For iOS, run pod install.

    If pod install is failed with error message Specs satisfying the ReactNativePassioSDK (from ../node_modules/@passiolife/nutritionai-react-native-sdk-v2) dependency were found, but they required a higher minimum deployment target. , ensure Podfile is configured with minimum ios version '13.0'

    For Android, add this implementation line to the dependencies section on app/build.gradle file.

    Also for Android, make sure the android project is configured with minSdkVersion 26 or above (check the project's build.gradle file).

    If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

    SDK Initialization and Configuration

    For the SDK to work, it requires a proper license key.

    • The license key can be requested at support@passiolife.com

    hashtag
    SDK Initialization

    When your component mounts, configure the SDK using your Passio provided developer key and start food detection.

    autoUpdate: true : Set to true to enable the download of AI models from Passio's servers.

    The possible states of the SDK after calling configure. Switch on sdkStatus.mode to access the data associated with each state.

    MODE: notReady, isReadyForDetection and error

    notReady : SDK is not ready due to missing model files. Please download the specified file and call configure again, passing in the localFileURLs of the downloaded files.

    isReadyForDetection: SDK configured successfully. This status much be reached before calling startFoodDetection

    error: SDK failed to configure in an unrecoverable way. Please read the error message for more information

    If at any point you need help from the Passio team, please reach out to us at support@passiolife.com

    yarn add @passiolife/nutritionai-react-native-sdk-v2
    cd ios; pod install
    const [isReady, setIsReady] = useState(false);
    
    // Effect to configure the SDK and request camera permission
    useEffect(() => {
      Promise.all([
        PassioSDK.configure({
          key: 'your-developer-key',
          autoUpdate: true,
        }),
        PassioSDK.requestCameraAuthorization(),
      ]).then(([sdkStatus, cameraAuthorized]) => {
        console.log(
          `SDK configured: ${sdkStatus.mode} Camera authorized: ${cameraAuthorized}`,
        );
        setIsReady(sdkStatus.mode === 'isReadyForDetection' && cameraAuthorized);
      });
    }, []);
    
    platform :ios, '13.0'
    dependencies {
        // Add this line below for Passio SDK library
        implementation files("$rootDir/../node_modules/@passiolife/nutritionai-react-native-sdk-v2/android/libs/passiolib-release.aar")
        ...
    }
    buildscript {
        ext {
            ...
            minSdkVersion = 26
            ...
        }
    }