Passio Nutrition-AI
  • Nutrition-AI SDK Overview
  • Guides
    • Nutrition AI SDK
      • SDK Key and minimum requirements
      • Installation
      • Configure the SDK
      • Use Cases
        • Food recognition
        • Nutrition data
        • Barcode scanning
        • Nutrition Facts scanning
        • Search, Food Icons, RefCode
        • Speech recognition
        • Nutrition Advisor
        • Suggestions and Meal Plans
        • User created foods and reports
    • iOS SDK Docs
      • Before getting started
      • Getting the ml models to the device
      • Run the demos first
      • Adding Passio SDK into your project
      • Initialize and configure the SDK
      • Start/Stop food detection
      • Food Recognition Delegate
      • Migration from SDK 1.4.X to 2.x
      • SDK API
      • Quick Start Guide
        • Installation
        • Configure the SDK (UIKit)
        • Recognise food using image
        • Food Details
    • Android SDK Docs
      • Getting started
        • Include the library
        • Camera
        • SDK Initialization and Configuration
        • Food detection session
        • Visual, Barcode and Packaged Food detection
        • Nutritional Database
        • Sandbox app
      • Troubleshooting on Android
      • Migration from SDK version 1.4.x to 2.x
      • Quick Start Guide
        • Importing the Android SDK to a project
        • Configure SDK and handle the result
        • RecognizeImageRemote
        • Food Details
    • React Native SDK Docs
      • Getting Started
        • Installation
        • SDK Initialization and Configuration
      • API Reference
        • configure & requestCameraAuthorization
        • recognizeImageRemote
        • searchForFood & searchForFoodSemantic
        • startFoodDetection
          • FoodDetectionEvent
        • fetchFoodItemForProductCode
        • fetchFoodItemForRefCode
        • fetchFoodItemForDataInfo
        • startNutritionFactsDetection
        • fetchFoodItemForPassioID
        • recognizeSpeechRemote
        • Fetch Nutrients
        • onDowloadingPassioModelCallBacks
        • detectFoodFromImageURI
        • addToPersonalization
        • updateLanguage
        • fetchHiddenIngredients
        • fetchVisualAlternatives
        • fetchPossibleIngredients
      • Properties
        • PassioFoodItem
        • PassioFoodDataInfo
        • PassioFoodAmount
        • PassioIngredient
        • PassioNutrients
        • UnitMass
        • ServingUnit
        • ServingSize
        • PassioSearchResult
        • NutritionFacts
        • PassioNutritionPreview
        • PassioSpeechRecognitionModel
        • PassioLogAction
        • PassioAdvisorFoodInfo
        • FoodCandidates
        • FoodDetectionEvent
        • NutritionDetectionEvent
        • DetectedCandidate
        • BarcodeCandidate
        • PackagedFoodCode
        • AmountEstimate
        • ImagesInfo
        • PassioStatus
        • FoodDetectionEvent
        • NutritionFacts
        • PassioMealPlan
        • PassioMealPlanItem
      • Nutriton Advisor
        • initConversation
        • sendMessage
        • sendImage
        • Guide
          • useNutritionAdvisor
        • Properties
          • PassioAdvisorResponse
          • PassioAdvisorMessageResultStatus
          • PassioAdvisorResultStatus
      • Quick Start Guide
        • Installation
        • Configure the SDK
        • RecognizeImageRemote
        • Food Detail
      • Guide
        • Integrate SDK Setup
        • Integrate Quick Scan
        • Integrate Food Search
        • Integrate Food Editor with Recipe
        • Integrate MealPlan
        • Integrate Suggestions
        • Integrate recognizeImageRemote
      • Components
        • DetectionCameraView
        • PassioIconView
      • Changelog
      • More
        • Display Macro Chart
          • MockFood
          • Utils PassioNutrient
        • Display Micro Progress
        • Getting nutrition value for a FoodItem
        • Getting nutrition value for a Recipe
        • Recalculate the nutrition value based on serving size options.
      • Migrations
        • Migrating from SDK 2.X to version 3.X
        • Structure Migrations From SDK 3.X to SDK 2.X
        • [Deprecated] Migrating from SDK 1.X to version 2.X
        • [Deprecated] Getting Started (v1)
          • Installation
          • SDK Initialization and Configuration
          • Start food detection
          • FoodDetectionEvent
          • Nutritional Database
      • Troubleshooting on RN
      • V2
        • Getting Started
          • Installation
          • SDK Initialization and Configuration
          • Start food detection
          • FoodDetectionEvent
        • RN SDK API
          • Properties
            • PersonalizedAlternative
            • FoodSearchResult
            • PassioNutrient
            • FoodDetectionEvent
            • DownloadModelCallBack
            • UPCProduct
            • ServingUnit
            • PassioStatus
            • PassioIDAttributes
            • PassioIDEntityType
            • PassioFoodItem
            • PassioRecipe
            • ServingSize
            • Measurement
            • UnitMass
            • NutritionFacts
        • Food Image
        • Search Food Item
        • Quick Scan
        • Multi Scan
        • MealPlan
        • Recipe
      • How do I Integrate a Passio SDK in EXPO?
    • Flutter SDK Docs
      • Getting Started
    • Before You Continue
    • Setup For Android
    • Initialize and configure the SDK
  • Fundamentals
    • Nutrition-AI Developer FAQ
      • Nutrition Data
      • Supported Phones
      • Security
      • Testing Volume Estimation
      • Testing Nutrition-AI SDK
      • Nutrition-AI Test Methodology
    • Nutrition API - Mobile SDK interoperability
      • JSON Response parsing
  • Versions
    • 3.2.4
      • SDK Key and minimum requirements
      • Installation
      • Configure the SDK
      • Use Cases
        • Food recognition
        • Nutrition data
        • Barcode scanning
        • Nutrition Facts scanning
        • Search, Food Icons, RefCode
        • Speech recognition
        • Nutrition Advisor
        • Suggestions and Meal Plans
        • User created foods and reports
    • 3.2.2
      • SDK Key and minimum requirements
      • Installation
      • Configure the SDK
      • Use Cases
        • Food recognition
        • Nutrition data
        • Barcode scanning
        • Nutrition Facts scanning
        • Search, Food Icons, RefCode
        • Speech recognition
        • Nutrition Advisor
        • Suggestions and Meal Plans
        • User created foods and reports
    • 3.2.0
      • SDK Key and minimum requirements
      • Installation
      • Configure the SDK
      • Use Cases
        • Food recognition
        • Nutrition data
        • Barcode scanning
        • Nutrition Facts scanning
        • Search, Food Icons, RefCode
        • Speech recognition
        • Nutrition Advisor
        • Suggestions and Meal Plans
    • 3.1.4
      • SDK Key and minimum requirements
      • Installation
      • Configure the SDK
      • Use Cases
        • Food recognition
        • Nutrition data
        • Barcode scanning
        • Search, Food Icons, RefCode
        • Speech recognition
        • Nutrition Advisor
        • Suggestions and Meal Plans
Powered by GitBook
On this page
  • Conversation flow
  • Extract ingredients
  • UI Example
Export as PDF
  1. Versions
  2. 3.2.0
  3. Use Cases

Nutrition Advisor

Starting from the SDK version 3.1.4 the NutritionAdvisor is only accessible using the NutritionAI Hub key. A key bought for the previous NutritionAI product won't work with this feature.

The Nutrition Advisor is an intelligent chatbot designed to offer personalised guidance on nutrition, diets, and healthy eating habits. Whether you're looking to explore a new diet, understand the nutritional value of certain foods, or find delicious and balanced recipes, the Nutrition Advisor is here to assist. Users can interact with the Nutrition Advisor through text messages, engaging in a conversational experience where they can ask questions about various diets like keto, vegan, Mediterranean, and more. The chatbot provides detailed insights into the benefits, potential drawbacks, and suitability of these diets based on individual goals and preferences.

As opposed to the rest of the SDK, the Nutrition Advisor has a different API entry point: NutritionAdvisor

Conversation flow

Use these steps to initialise the conversation and enable users to send messages:

  1. To start a conversation initConversation has to be invoked. The response will either indicate a success state, which means message can be sent to the advisor. In case of an error, an error message will detail the cause.

  2. Text messages are sent using the sendMessage API. The reply (if successful) will contain a PassioAdvisorResponse response. This object consist of:

  • threadId - identifies the conversation

  • messageId - identifies the specific message

  • markupContent - text response from the advisor denoted with markup tags. This can be used to style the text when rendering

  • rawConent - text response without the markup tags

  • tools - list of actions that can be executed by the advisor. Only should be used by the SDK

  • extracedIngredients - will contain a list of foods if the request to the advisor was an image recognition request or extract ingredients from text message request

  1. Image are sent to recognition using sendImage. As mentioned above, in the PassioAdvisorResponse object, the extractedIngredients property will hold the recognised foods.

NutritionAdvisor.shared.initConversation { advisorResult in
    switch advisorResult {
        case .success:
            print("The advisor can start acceping messages")
        case .failure(let error):
            print("The advisor can not start acceping messages. Error:- \(error.errorMessage)")
    }
}
let message = "Hi! I would like a recipe for a healthy burger"
NutritionAdvisor.shared.sendMessage(message: message) { advisorResponse in
    switch advisorResponse {
        case .success(let response): // Show text in a chat bubble
            print("Response:- \(response.rawContent)")
        case .failure(let error): // Handle error
            print("Chat Error:- \(error.errorMessage)")
    }
}
let image = Bundle.main.path(forResource: "image1", ofType: "png")
NutritionAdvisor.shared.sendImage(image: image) { advisorResponse in
    switch advisorResponse {
        case .success(let response): // Show list of foods
            print("Image Response:- \(response.extractedIngredients)")
        case .failure(let error): // Handle error
            print("Image Chat Error:- \(error.errorMessage)")
    }
}
NutritionAdvisor.instance.initConversation { result ->
    when (result) {
        is PassioResult.Success -> /* the advisor can start acceping messages */
        is PassioResult.Error -> /* handle conversation init error */
    }
}

NutritionAdvisor.instance.sendMessage("Hi! I would like a recipe for a healthy burger") { result ->
    when (result) {
        is PassioResult.Error -> {
            val errorMessage = result.message
            // Handle error
        }
        is PassioResult.Success -> {
            val text = result.value.rawContent
            // Show text in a chat bubble
        }
    }
}

val bitmap = loadBitmap()
NutritionAdvisor.instance.sendImage(bitmap) { result ->
    when (result) {
        is PassioResult.Error -> {
            val errorMessage = result.message
            // Handle error
        }
        is PassioResult.Success -> {
            val foods = result.value.extractedIngredients
            // Show list of foods
        }
    }
}
import { NutritionAdvisor } from '@passiolife/nutritionai-react-native-sdk-v3'

const [loadingState, setLoadingState] = useState<SDKStatus>('Init')
  useEffect(() => {
    async function configure() {
      try {
            NutritionAdvisor.initConversation()
              .then((item) => {
                setLoadingState(
                  item?.status === 'Success' ? 'Success' : 'Error'
                )
              })
              .catch((error) => {
                setLoadingState(error)
              })
      } catch (err) {
        console.error(`PassioSDK Error ${err}`)
        setLoadingState('Error')
      }
    }
    configure()
  }, [])


 const response = await NutritionAdvisor.sendMessage(message)
 switch (response?.status) {
          case 'Success':
            // get your response here {response.response}
            return
          case 'Error':
          // get your erorr response here {response.message}
            return
        }

const response = await NutritionAdvisor.sendImage(imageURI)
 switch (response?.status) {
          case 'Success':
            // get your response here {response.response}
            return
          case 'Error':
          // get your erorr response here {response.message}
            return
        }
final result = await NutritionAdvisor.instance.initConversation();
switch (result) {
  case Success():
    // Initialization succeeded, the advisor can start accepting messages
    break;
  case Error():
    // Handle the init error here
    break;
}

final result = await NutritionAdvisor.instance.sendMessage('Hi! I would like a recipe for a healthy burger');
switch (result) {
  case Success():
    // Parse the response from result.value
    break;
  case Error():
    // Handle the error that occurred while sending the message
    break;
}

Uint8List bytes = loadImage();
final result = await NutritionAdvisor.instance.sendImage(bytes);
switch (result) {
  case Success():
    // The food items are found in imageResult.value.extractedIngredients
    break;
  case Error():
    // Handle the error that occurred while sending the message
    break;
}

Extract ingredients

In the case that a PassioAdvisorResponse contains a tool called searchMatchIngredient, the Advisor has the ability to parse the text response and return the list of foods. If a message that has this tool is passed to the fetchIngredients API, the returned response will contain foods in the extracedIngredients field.

This feature is particularly useful when trying to log foods when directly communicating with the chatbot. A common use case is when user ask for a specific recipe. They can then initiate the extraction of foods from the message, and see their nutrition value and finally log the entire meal in one action.

UI Example

  1. Create a screen where the user can type messages and send images to the NutritionAdvisor. Upon initiating the screen, set up the conversation by calling initConversation.

  2. When the user types a message in the input text field and taps "Send", invoke the sendMessage function. Because this is an asynchronous function, add an animation to indicate this behaviour.

  3. If the response from the advisor contains a tool called searchMatchIngredient show an action button "Find foods" to initiate extraction of the foods. Invoke the fetchIngredients function using the response from the Advisor.

  4. Create a view where the user can see the extracted foods, their serving size and have the ability to log them.

  1. Add the ability for the user to select an image from the library of the phone. Once the image is obtained, invoke the sendImage to get the list of foods. If foods are recognised, the response will contain a list of foods, use the same result view to show them.

PreviousSpeech recognitionNextSuggestions and Meal Plans

Last updated 8 months ago