Nutrition Advisor

Starting from the SDK version 3.1.4 the NutritionAdvisor is only accessible using the NutritionAI Hub key. A key bought for the previous NutritionAI product won't work with this feature.

The Nutrition Advisor is an intelligent chatbot designed to offer personalised guidance on nutrition, diets, and healthy eating habits. Whether you're looking to explore a new diet, understand the nutritional value of certain foods, or find delicious and balanced recipes, the Nutrition Advisor is here to assist. Users can interact with the Nutrition Advisor through text messages, engaging in a conversational experience where they can ask questions about various diets like keto, vegan, Mediterranean, and more. The chatbot provides detailed insights into the benefits, potential drawbacks, and suitability of these diets based on individual goals and preferences.

As opposed to the rest of the SDK, the Nutrition Advisor has a different API entry point: NutritionAdvisor

Conversation flow

Use these steps to initialise the conversation and enable users to send messages:

1. To start a conversation initConversation has to be invoked. The response will either indicate a success state, which means message can be sent to the advisor. In case of an error, an error message will detail the cause.

2. Text messages are sent using the sendMessage API. The reply (if successful) will contain a PassioAdvisorResponse response. This object consist of:

• threadId - identifies the conversation

• messageId - identifies the specific message

• markupContent - text response from the advisor denoted with markup tags. This can be used to style the text when rendering

• rawConent - text response without the markup tags

• tools - list of actions that can be executed by the advisor. Only should be used by the SDK

• extracedIngredients - will contain a list of foods if the request to the advisor was an image recognition request or extract ingredients from text message request

3. Image are sent to recognition using sendImage. As mentioned above, in the PassioAdvisorResponse object, the extractedIngredients property will hold the recognised foods.

iOS

NutritionAdvisor.shared.initConversation { advisorResult in
    switch advisorResult {
        case .success:
            print("The advisor can start acceping messages")
        case .failure(let error):
            print("The advisor can not start acceping messages. Error:- \(error.errorMessage)")
    }
}
let message = "Hi! I would like a recipe for a healthy burger"
NutritionAdvisor.shared.sendMessage(message: message) { advisorResponse in
    switch advisorResponse {
        case .success(let response): // Show text in a chat bubble
            print("Response:- \(response.rawContent)")
        case .failure(let error): // Handle error
            print("Chat Error:- \(error.errorMessage)")
    }
}
let image = Bundle.main.path(forResource: "image1", ofType: "png")
NutritionAdvisor.shared.sendImage(image: image) { advisorResponse in
    switch advisorResponse {
        case .success(let response): // Show list of foods
            print("Image Response:- \(response.extractedIngredients)")
        case .failure(let error): // Handle error
            print("Image Chat Error:- \(error.errorMessage)")
    }
}

Android

NutritionAdvisor.instance.initConversation { result ->
    when (result) {
        is PassioResult.Success -> /* the advisor can start acceping messages */
        is PassioResult.Error -> /* handle conversation init error */
    }
}

NutritionAdvisor.instance.sendMessage("Hi! I would like a recipe for a healthy burger") { result ->
    when (result) {
        is PassioResult.Error -> {
            val errorMessage = result.message
            // Handle error
        }
        is PassioResult.Success -> {
            val text = result.value.rawContent
            // Show text in a chat bubble
        }
    }
}

val bitmap = loadBitmap()
NutritionAdvisor.instance.sendImage(bitmap) { result ->
    when (result) {
        is PassioResult.Error -> {
            val errorMessage = result.message
            // Handle error
        }
        is PassioResult.Success -> {
            val foods = result.value.extractedIngredients
            // Show list of foods
        }
    }
}

React Native

import { NutritionAdvisor } from '@passiolife/nutritionai-react-native-sdk-v3'

const [loadingState, setLoadingState] = useState<SDKStatus>('Init')
  useEffect(() => {
    async function configure() {
      try {
            NutritionAdvisor.initConversation()
              .then((item) => {
                setLoadingState(
                  item?.status === 'Success' ? 'Success' : 'Error'
                )
              })
              .catch((error) => {
                setLoadingState(error)
              })
      } catch (err) {
        console.error(`PassioSDK Error ${err}`)
        setLoadingState('Error')
      }
    }
    configure()
  }, [])


 const response = await NutritionAdvisor.sendMessage(message)
 switch (response?.status) {
          case 'Success':
            // get your response here {response.response}
            return
          case 'Error':
          // get your erorr response here {response.message}
            return
        }

const response = await NutritionAdvisor.sendImage(imageURI)
 switch (response?.status) {
          case 'Success':
            // get your response here {response.response}
            return
          case 'Error':
          // get your erorr response here {response.message}
            return
        }

Flutter

final result = await NutritionAdvisor.instance.initConversation();
switch (result) {
  case Success():
    // Initialization succeeded, the advisor can start accepting messages
    break;
  case Error():
    // Handle the init error here
    break;
}

final result = await NutritionAdvisor.instance.sendMessage('Hi! I would like a recipe for a healthy burger');
switch (result) {
  case Success():
    // Parse the response from result.value
    break;
  case Error():
    // Handle the error that occurred while sending the message
    break;
}

Uint8List bytes = loadImage();
final result = await NutritionAdvisor.instance.sendImage(bytes);
switch (result) {
  case Success():
    // The food items are found in imageResult.value.extractedIngredients
    break;
  case Error():
    // Handle the error that occurred while sending the message
    break;
}

Extract ingredients

In the case that a PassioAdvisorResponse contains a tool called searchMatchIngredient, the Advisor has the ability to parse the text response and return the list of foods. If a message that has this tool is passed to the fetchIngredients API, the returned response will contain foods in the extracedIngredients field.

This feature is particularly useful when trying to log foods when directly communicating with the chatbot. A common use case is when user ask for a specific recipe. They can then initiate the extraction of foods from the message, and see their nutrition value and finally log the entire meal in one action.

UI Example

1. Create a screen where the user can type messages and send images to the NutritionAdvisor. Upon initiating the screen, set up the conversation by calling initConversation.

2. When the user types a message in the input text field and taps "Send", invoke the sendMessage function. Because this is an asynchronous function, add an animation to indicate this behaviour.

3. If the response from the advisor contains a tool called searchMatchIngredient show an action button "Find foods" to initiate extraction of the foods. Invoke the fetchIngredients function using the response from the Advisor.

4. Create a view where the user can see the extracted foods, their serving size and have the ability to log them.

5. Add the ability for the user to select an image from the library of the phone. Once the image is obtained, invoke the sendImage to get the list of foods. If foods are recognised, the response will contain a list of foods, use the same result view to show them.