Nutrition-AI Developer FAQ
Frequently Asked Questions
SDK Configuration
Can I bundle models and files with the app instead of letting the SDK to download them?
You can. When implementing the Passio sdk configuration, use the sdkDownloadModels = false. This will increase the size of the app, but it removes the need for the models to be downloaded later. However, keep in mind that you still need WiFi or LTE connectivity to use the models because they require a valid license key, which is checked regularly against Passio's license key API.
What is the size of the SDK?
The core SDK is lightweight, coming in at under 10MB on both Android and iOS. The SDK does not include models or data files by default. Instead, it pulls models from Passio's cloud servers once the SDK is activated with a valid key. These models are loaded in the background, similar to how levels are loaded in mobile games. The models and data files typically range from 40MB to 60MB.
What are the minimum iOS and Android environments you can support?
For iOS, we support iOS 13 and above. We recommend building the SDK with the latest version of Xcode, though earlier versions may be supported if necessary. For specific inquiries, please contact us at support@passiolife.com.
For Android, we support Android API level 26 and above. We recommend using the latest version of Kotlin, though we can support older versions as well (e.g., we recently deployed for Kotlin 1.4.21).
How often do you release the SDK?
We generally release an updated version of our SDK every two to three months. In cases where critical updates are needed, we may shift to bi-weekly releases. Updating the SDK is typically straightforward and requires a simple drag-and-drop replacement of the model. Occasionally, a few lines of code may need to be adjusted, but this is rare, and our technical team is always available to assist with any changes.
What are the implications of having an expired/revoked token for SDKs already published in the app?
There are two scenarios to consider:
If your payments are not up to date and the token expires, the SDK will stop functioning.
If you've received an updated SDK and associated keys, the old SDK keys will continue to work as long as your contract is current. This ensures there will be no impact on users who haven’t updated to the new SDK.
Where does the image recognition happen? Is it performed on the client-side SDK or through an API call to Passio’s server?
Image recognition is performed entirely on the client side (on the user's device). Our client-side SDK includes a database of around 40,000 common foods and nutritional references, enabling visual logging without requiring API calls to the cloud. Additionally, we offer a larger database of over 2.5 million foods available via API calls to Passio’s server, primarily used for logging foods through UPC/barcodes or augmenting manual searches.
Are there any other API calls the SDK makes to the Passio server regarding analytics or usage statistics?
Yes. We collect limited performance statistics to improve the SDK, but we will always share the specifics with you before doing so. This provision is included in the contract. One key metric we track is the scan/log ratio, which helps us assess the average performance of the model across users. No personally identifiable information (PII) is captured.
Additionally, if you're using our UPC functionality, the packaged food data is stored in the cloud, requiring an API call to access that information.
What is the response time threshold for the APIs?
Since computer vision runs on the device, response times are nearly instantaneous, processing image input at 2 frames per second. For packaged food recognition via API, response times depend on network conditions but are typically under one second.
How does learning and correlation work with client’s data and business logic?
Learning, correlations, and downstream activities can be approached using the user logs generated by the SDK. These logs are stored within your application and are fully controlled and managed by you on the client side—no logs or user data are sent to Passio.
You can leverage these logs to build models, generate recommendations, or conduct studies that correlate nutrition logs with metabolic and other data. You also have the flexibility to send these logs from the client side to your servers or third-party servers, just like any other data in your app.
Is there a way for the app to save the image as well?
Passio does not collect or store user-scanned images of food items. The SDK only handles the recognition process. If your application requires image capture and storage for additional context or manual logging, it will need to be implemented within your app. Your team can design and manage this feature to capture and store images as needed.
What does the data model look like? What is the role of parent, sibling, and child relationships?
The data model is based on two main types of objects: nutrition references and visual labels.
Nutrition references contain nutritional data such as nutrients and measurements. These references come in two forms:
Branded Products: Nutrition references tied to a specific product code.
Generic Nutrition References: Average data for general foods (e.g., “raw spinach”).
Visual Labels exist within a visual hierarchy. They represent visual objects but are too ambiguous to include nutrition data on their own. Below the leaf nodes of the visual hierarchy, there is a many-to-many mapping to nutrition references. This ensures every visual label can be linked to nutritional data. For example:
The visual label "almond milk" can have multiple branded almond milk nutrition references.
A branded almond milk nutrition reference can be linked to both "almond milk (in a glass)" and "boxed milk."
Some visual labels are also linked to recipes, which are defined by a set of nutrition references and measurements that compose the recipe.
The recognition process can follow different paths:
OCR or Barcode Scanning: Recognizes a nutrition reference by its product code. Parent, sibling, and child relationships are not relevant here.
Visual Scanning: Recognizes a visual label within the visual hierarchy:
The parent represents the object one level higher (e.g., the parent of "raw spinach leaves" might be "raw greens").
Siblings are visually similar alternatives (e.g., “baby kale” for "raw spinach leaves").
Children can be either:
Another set of visual labels, representing alternatives when high-level visual labels are recognized.
Nutrition references, which provide the final nutritional data (e.g., children of "raw spinach leaves" would be nutrition references).
For text searches on nutrition references, there are no child relationships, as nutrition references are the final layer of information. The parent is the visual label the nutrition reference maps to, and siblings are other nutrition references attached to the same visual label, and can hence be considered alternative nutrition references.
How do we identify healthy food items?
While we have a team of Registered Dietitians who create curated lists, identifying healthy food items has not been our primary focus until recently. We typically allow our customers to define what is healthy or unhealthy based on their own models and criteria. For example, some dietitians consider keto to be unhealthy, while others may recommend it. Additionally, "healthy" can vary greatly depending on individual conditions, dietary restrictions, or allergies, making it a highly personalized determination.
Can you provide a sample of meals and recipes?
In the Nutrition AI SDK, we use the concept of a recipe, which consists of specific nutrition references and measurements that define the composition of a recipe. Recipes can also be linked to visual labels, which is particularly useful when a specific branded product or generic reference isn't available, such as for "bread with cream cheese."
Meal plans are curated sets of nutrition references or recipes designed for a specific dietary purpose. These plans can help users follow personalized nutrition guidelines based on their health goals or dietary needs.
Can we get alternatives for specific items in meals as part of the API?
Yes, you can. By making an internal client-side API call with the name of the food item, you will receive an object containing alternatives. This allows you to drill down through the food hierarchy to explore different options.
How can we interact with the API and learn user preferences?
User preferences can be learned by analyzing the user logs and behavior within your app’s client-side logging experience. This process is entirely controlled by your application, and you do not need to interact with the API to gather this information.
With respect to supplements, can a user search for multiple nutrients to populate a multivitamin option?
The search functionality currently only looks for food or supplement names. If a supplement’s name includes the nutrients (e.g., “Brand X - Vitamin C, Iron, and Calcium Supplement”), it will appear in the search results. However, if the supplement's name does not explicitly mention those nutrients, it will not show up, as nutrient-specific searching is not supported at this time.
Would it be possible for Passio to provide a list of all the parameters associated with supplements?
Currently, supplements are not separated from other packaged foods, so the same parameters apply to both. When you scan a supplement, you can expect the same information provided for any packaged food. In the future, if there is demand, we may introduce additional nutrient information specifically for supplements beyond what is currently available in the SDK.
Is the supplement information stored in the OpenFoodFacts database that is supplied with the SDK or in a separate database?
The information we provide comes from our own database, which is a curated result of importing and merging data from sources such as the USDA FDC database, Open Food Facts, and additional data collected by our internal team (e.g., through research on manufacturer websites). However, the data surfaced by the SDK may not exactly match what’s found in FDC or Open Food Facts, as we apply our own curation logic and may include additional details like measurement names.
When scanning a food, the SDK will indicate whether the original data came from Open Food Facts or the USDA FDC database.
If a user scans a multivitamin nutrition label, will they see all the nutrients and their quantities?
The SDK currently surfaces a specific set of about 20-30 nutrients. If these nutrients are present on the scanned multivitamin, they will be displayed along with their quantities. While we store all nutrient data in our backend database, only the currently supported nutrients are provided in the SDK. In the future, we may expand this to include all available nutrients.
How can we use our local Nutrient File for nutrient analysis. Please let us know what we need to tweak in the Passio SDK for this?
Passio’s nutrition references work as follows:
Barcodes: If your local Nutrient File doesn’t have as extensive a barcode mapping as sources like OpenFood Facts (which we use), you may not be able to map all barcodes to your database.
USDA FDC: Similarly, not all foods will map easily to your local database, as is the case with barcodes.
Passio-created foods and recipes: The only potential mapping aid here is the name of the food.
The Passio SDK returns product codes, barcodes, and FDC IDs for foods (when available). However, any mapping between the Passio SDK and your local Nutrient File would need to be done outside of the SDK, within your application.
Additionally, your team would need to manage how to store and present nutrient information from your local file, as the format might differ from Passio’s data. This would likely involve creating your own UI on top of the mapping.
Would it be possible for Passio to provide the high-level food groups for customers to use?
Yes, we populate high-level food groups on the backend as tags. Below is the current list of food group tags:
Alcoholic beverages
Baby foods
Baked goods
Beverages
Breakfast cereals
Candies and sweets
Cereal grains and pasta
Cooking and baking ingredients
Dairies and products
Dairy substitutes
Desserts
Eggs and products
Fats and oils
Food additives
Fried foods
Fruits and products
Legumes and products
Meals
Meat and products
Nuts and seeds products
Pickles
Sauces, dips, spreads, gravies
Seafood and products
Seaweeds and products
Snack and protein bars
Snacks
Soups
Spices and condiments
Stuffing
Supplements
Sweeteners
Vegetables and products
Does the Passio SDK provide tags for NOVA scores related to food?
Yes, the Passio SDK offers NOVA score tags related to food. Below is the list of NOVA score categories:
NOVA1: Whole foods with little or no processing, such as fruits, vegetables, nuts, and seeds.
NOVA2: Ingredients extracted from foods, such as oils, fats, sugar, and salt.
NOVA3: Processed foods made by combining unprocessed or minimally processed foods with Group 2 ingredients (e.g., canned vegetables, cheese).
NOVA4: Highly processed foods that often contain additives, preservatives, and flavorings (e.g., packaged snacks, soft drinks, fast food).
Reference: https://world.openfoodfacts.org/nova
With both IU and mcg RAE being used for vitamin A, how should I decide which unit to rely on when interpreting the data? Should I be concerned about the differences in conversion between these units?
Vitamin A Unit Conversions: Historically, vitamin A was measured in IU (International Units), but since 2016, the default unit has shifted to mcg RAE (micrograms of Retinol Activity Equivalents). You may still encounter both units in various data sources. Because the conversion between IU and mcg RAE depends on the source of vitamin A, it is not always straightforward.
We recommend surfacing both values (IU and mcg RAE) and letting users decide which to use or how to present the information.
In addition to what we already surface, we need to add:
Vitamin A, RAE (mcg) (FDC ID 1106)
Conversion Complexities:
Animal Source (Retinol): 1 mcg RAE = 3.33 IU
Plant Source (Beta-Carotene from Food): 1 mcg RAE = 20 IU
Plant Source (Beta-Carotene from Supplement): 1 mcg RAE = 3.33 IU
Plant Source (Alpha-Carotene or Beta-Cryptoxanthin): 1 mcg RAE = 40 IU
References:
How can we display small food icons in a rounded corner square instead of a circle in our app?
The Passio SDK provides the original image in three different resolutions (90, 180, and 360 pixels). While the SDK handles image retrieval, adding rounded corners or changing the shape (e.g., from a circle to a rounded square) is managed on the app’s side. You can easily achieve this with your app's image rendering.
For example, in Swift, you can modify the image's corner radius to create a rounded square. Here's a helpful guide for doing this in Swift: Create a Circular Profile Image (Swift). You can adapt the same approach to make rounded corners instead of a full circle.
In the image recognition flow, what is the current limit for the number of images processed?
In the image recognition flow, the current limit is 7 images when capturing multiple images. This limit is set to prevent any slowdown and ensure a smooth user experience.
Can you clarify the process for integrating user-created foods with the SDK? Specifically, how should we handle storing and syncing this data with the SDK for barcode scanning or search functions? Additionally, when will the API for uploading user-created foods to the SDK be available?
Currently, the SDK does not support user-created foods directly, so this functionality needs to be handled on the app side. You will need to manage the storage and syncing of user-created food data within your app, then integrate it with the SDK for barcode scanning or search purposes.
Once the API for uploading user-created foods is available, you’ll be able to upload this data to our servers. If you need any assistance during this process, feel free to reach out to us at support@passiolife.com.
Last updated