Nutrition-AI Developer FAQ
Frequently Asked Questions
SDK Configuration
You can. When implementing the Passio sdk configuration, use the sdkDownloadModels = false. This will increase the size of the app, but it removes the need for the models to be downloaded later. However, keep in mind that you still need WiFi or LTE connectivity to use the models because they require a valid license key, which is checked regularly against Passio's license key API.
The core SDK is lightweight, coming in at under 10MB on both Android and iOS. The SDK does not include models or data files by default. Instead, it pulls models from Passio's cloud servers once the SDK is activated with a valid key. These models are loaded in the background, similar to how levels are loaded in mobile games. The models and data files typically range from 40MB to 60MB.
For iOS, we support iOS 13 and above. We recommend building the SDK with the latest version of Xcode, though earlier versions may be supported if necessary. For specific inquiries, please contact us at support@passiolife.com.
For Android, we support Android API level 26 and above. We recommend using the latest version of Kotlin, though we can support older versions as well (e.g., we recently deployed for Kotlin 1.4.21).
We generally release an updated version of our SDK every two to three months. In cases where critical updates are needed, we may shift to bi-weekly releases. Updating the SDK is typically straightforward and requires a simple drag-and-drop replacement of the model. Occasionally, a few lines of code may need to be adjusted, but this is rare, and our technical team is always available to assist with any changes.
There are two scenarios to consider:
If your payments are not up to date and the token expires, the SDK will stop functioning.
If you've received an updated SDK and associated keys, the old SDK keys will continue to work as long as your contract is current. This ensures there will be no impact on users who haven’t updated to the new SDK.
Image recognition is performed entirely on the client side (on the user's device). Our client-side SDK includes a database of around 40,000 common foods and nutritional references, enabling visual logging without requiring API calls to the cloud. Additionally, we offer a larger database of over 2.5 million foods available via API calls to Passio’s server, primarily used for logging foods through UPC/barcodes or augmenting manual searches.
Yes. We collect limited performance statistics to improve the SDK, but we will always share the specifics with you before doing so. This provision is included in the contract. One key metric we track is the scan/log ratio, which helps us assess the average performance of the model across users. No personally identifiable information (PII) is captured.
Additionally, if you're using our UPC functionality, the packaged food data is stored in the cloud, requiring an API call to access that information.
Since computer vision runs on the device, response times are nearly instantaneous, processing image input at 2 frames per second. For packaged food recognition via API, response times depend on network conditions but are typically under one second.
Learning, correlations, and downstream activities can be approached using the user logs generated by the SDK. These logs are stored within your application and are fully controlled and managed by you on the client side—no logs or user data are sent to Passio.
You can leverage these logs to build models, generate recommendations, or conduct studies that correlate nutrition logs with metabolic and other data. You also have the flexibility to send these logs from the client side to your servers or third-party servers, just like any other data in your app.
Passio does not collect or store user-scanned images of food items. The SDK only handles the recognition process. If your application requires image capture and storage for additional context or manual logging, it will need to be implemented within your app. Your team can design and manage this feature to capture and store images as needed.
The data model is based on two main types of objects: nutrition references and visual labels.
Nutrition references contain nutritional data such as nutrients and measurements. These references come in two forms:
Branded Products: Nutrition references tied to a specific product code.
Generic Nutrition References: Average data for general foods (e.g., “raw spinach”).
Visual Labels exist within a visual hierarchy. They represent visual objects but are too ambiguous to include nutrition data on their own. Below the leaf nodes of the visual hierarchy, there is a many-to-many mapping to nutrition references. This ensures every visual label can be linked to nutritional data. For example:
The visual label "almond milk" can have multiple branded almond milk nutrition references.
A branded almond milk nutrition reference can be linked to both "almond milk (in a glass)" and "boxed milk."
Some visual labels are also linked to recipes, which are defined by a set of nutrition references and measurements that compose the recipe.
The recognition process can follow different paths:
OCR or Barcode Scanning: Recognizes a nutrition reference by its product code. Parent, sibling, and child relationships are not relevant here.
Visual Scanning: Recognizes a visual label within the visual hierarchy:
The parent represents the object one level higher (e.g., the parent of "raw spinach leaves" might be "raw greens").
Siblings are visually similar alternatives (e.g., “baby kale” for "raw spinach leaves").
Children can be either:
Another set of visual labels, representing alternatives when high-level visual labels are recognized.
Nutrition references, which provide the final nutritional data (e.g., children of "raw spinach leaves" would be nutrition references).
For text searches on nutrition references, there are no child relationships, as nutrition references are the final layer of information. The parent is the visual label the nutrition reference maps to, and siblings are other nutrition references attached to the same visual label, and can hence be considered alternative nutrition references.
While we have a team of Registered Dietitians who create curated lists, identifying healthy food items has not been our primary focus until recently. We typically allow our customers to define what is healthy or unhealthy based on their own models and criteria. For example, some dietitians consider keto to be unhealthy, while others may recommend it. Additionally, "healthy" can vary greatly depending on individual conditions, dietary restrictions, or allergies, making it a highly personalized determination.
In the Nutrition AI SDK, we use the concept of a recipe, which consists of specific nutrition references and measurements that define the composition of a recipe. Recipes can also be linked to visual labels, which is particularly useful when a specific branded product or generic reference isn't available, such as for "bread with cream cheese."
Meal plans are curated sets of nutrition references or recipes designed for a specific dietary purpose. These plans can help users follow personalized nutrition guidelines based on their health goals or dietary needs.
Yes, you can. By making an internal client-side API call with the name of the food item, you will receive an object containing alternatives. This allows you to drill down through the food hierarchy to explore different options.
User preferences can be learned by analyzing the user logs and behavior within your app’s client-side logging experience. This process is entirely controlled by your application, and you do not need to interact with the API to gather this information.
The search functionality currently only looks for food or supplement names. If a supplement’s name includes the nutrients (e.g., “Brand X - Vitamin C, Iron, and Calcium Supplement”), it will appear in the search results. However, if the supplement's name does not explicitly mention those nutrients, it will not show up, as nutrient-specific searching is not supported at this time.
Currently, supplements are not separated from other packaged foods, so the same parameters apply to both. When you scan a supplement, you can expect the same information provided for any packaged food. In the future, if there is demand, we may introduce additional nutrient information specifically for supplements beyond what is currently available in the SDK.
The information we provide comes from our own database, which is a curated result of importing and merging data from sources such as the USDA FDC database, Open Food Facts, and additional data collected by our internal team (e.g., through research on manufacturer websites). However, the data surfaced by the SDK may not exactly match what’s found in FDC or Open Food Facts, as we apply our own curation logic and may include additional details like measurement names.
When scanning a food, the SDK will indicate whether the original data came from Open Food Facts or the USDA FDC database.
The SDK currently surfaces a specific set of about 20-30 nutrients. If these nutrients are present on the scanned multivitamin, they will be displayed along with their quantities. While we store all nutrient data in our backend database, only the currently supported nutrients are provided in the SDK. In the future, we may expand this to include all available nutrients.
Passio’s nutrition references work as follows:
Barcodes: If your local Nutrient File doesn’t have as extensive a barcode mapping as sources like OpenFood Facts (which we use), you may not be able to map all barcodes to your database.
USDA FDC: Similarly, not all foods will map easily to your local database, as is the case with barcodes.
Passio-created foods and recipes: The only potential mapping aid here is the name of the food.
The Passio SDK returns product codes, barcodes, and FDC IDs for foods (when available). However, any mapping between the Passio SDK and your local Nutrient File would need to be done outside of the SDK, within your application.
Additionally, your team would need to manage how to store and present nutrient information from your local file, as the format might differ from Passio’s data. This would likely involve creating your own UI on top of the mapping.
Yes, we populate high-level food groups on the backend as tags. Below is the current list of food group tags:
Alcoholic beverages
Baby foods
Baked goods
Beverages
Breakfast cereals
Candies and sweets
Cereal grains and pasta
Cooking and baking ingredients
Dairies and products
Dairy substitutes
Desserts
Eggs and products
Fats and oils
Food additives
Fried foods
Fruits and products
Legumes and products
Meals
Meat and products
Nuts and seeds products
Pickles
Sauces, dips, spreads, gravies
Seafood and products
Seaweeds and products
Snack and protein bars
Snacks
Soups
Spices and condiments
Stuffing
Supplements
Sweeteners
Vegetables and products
Yes, the Passio SDK offers NOVA score tags related to food. Below is the list of NOVA score categories:
NOVA1: Whole foods with little or no processing, such as fruits, vegetables, nuts, and seeds.
NOVA2: Ingredients extracted from foods, such as oils, fats, sugar, and salt.
NOVA3: Processed foods made by combining unprocessed or minimally processed foods with Group 2 ingredients (e.g., canned vegetables, cheese).
NOVA4: Highly processed foods that often contain additives, preservatives, and flavorings (e.g., packaged snacks, soft drinks, fast food).
Reference: https://world.openfoodfacts.org/nova
Vitamin A Unit Conversions: Historically, vitamin A was measured in IU (International Units), but since 2016, the default unit has shifted to mcg RAE (micrograms of Retinol Activity Equivalents). You may still encounter both units in various data sources. Because the conversion between IU and mcg RAE depends on the source of vitamin A, it is not always straightforward.
We recommend surfacing both values (IU and mcg RAE) and letting users decide which to use or how to present the information.
In addition to what we already surface, we need to add:
Vitamin A, RAE (mcg) (FDC ID 1106)
Conversion Complexities:
Animal Source (Retinol): 1 mcg RAE = 3.33 IU
Plant Source (Beta-Carotene from Food): 1 mcg RAE = 20 IU
Plant Source (Beta-Carotene from Supplement): 1 mcg RAE = 3.33 IU
Plant Source (Alpha-Carotene or Beta-Cryptoxanthin): 1 mcg RAE = 40 IU
References:
The Passio SDK provides the original image in three different resolutions (90, 180, and 360 pixels). While the SDK handles image retrieval, adding rounded corners or changing the shape (e.g., from a circle to a rounded square) is managed on the app’s side. You can easily achieve this with your app's image rendering.
For example, in Swift, you can modify the image's corner radius to create a rounded square. Here's a helpful guide for doing this in Swift: Create a Circular Profile Image (Swift). You can adapt the same approach to make rounded corners instead of a full circle.
In the image recognition flow, the current limit is 7 images when capturing multiple images. This limit is set to prevent any slowdown and ensure a smooth user experience.
Currently, the SDK does not support user-created foods directly, so this functionality needs to be handled on the app side. You will need to manage the storage and syncing of user-created food data within your app, then integrate it with the SDK for barcode scanning or search purposes.
Once the API for uploading user-created foods is available, you’ll be able to upload this data to our servers. If you need any assistance during this process, feel free to reach out to us at support@passiolife.com.
Last updated