RecognizeImageRemote

The RecognizeImageRemote component is responsible for recognizing food items from an image, either by taking a photo using the camera or selecting an image from the gallery. It integrates with the Passio Nutrition SDK to fetch food details and displays the results in a bottom sheet.

Install required Libraries

yarn add react-native-image-picker && expo-camera && @gorhom/bottom-sheet

useCameraPermissions: It seems like you're using a custom hook to handle camera permissions, but it's not a native hook from libraries like expo-camera Ensure that the hook is correctly implemented or imported.

const [permission, requestPermission] = useCameraPermissions();  

useEffect(() => {
    requestPermission();
  }, []);

cameraViewRef: You need to ensure that this ref is properly created and assigned to the camera view. If you're using expo-camera, ensure you're following the right method for capturing images.

  const cameraViewRef = useRef<CameraView>(null);

setPictureUri: This is the state updater function. Whenever a new picture is captured, you'll call setPictureUri with the URI of the picture to update the state. This triggers a re-render of the component with the updated picture URI.

 const [pictureUri, setPictureUri] = useState<string>("");

Take a Picture

  
const takePicture = useCallback(async () => {
    try {
      const picture = await cameraViewRef.current?.takePictureAsync();
      setPictureUri(picture?.uri ?? "");
    } catch (_) {
      Alert.alert("error while capture image");
    }
  }, []);

Pick Picture from Gallery

  const pickImage = useCallback(async () => {
    try {
      const { assets } = await launchImageLibrary({
        mediaType: "photo",
      });
      setPictureUri(assets?.[0].uri ?? "");
    } catch (err) {
      setLoading(false);
    }
  }, []);

Let's Recognize the picture using SDK

The onScanImage function is responsible for sending a captured image to the Passio SDK for remote image recognition . It processes the image, sends it to a remote server for recognition, and updates the local state based on the response.

  const [passioSpeechRecognitionModel, setPassioSpeechRecognitionModel] =
    useState<PassioAdvisorFoodInfo[] | null>();

This part sends the image to PassioSDK for remote recognition.

• pictureUri?.replace("file://", ""): This removes the file:// prefix from the pictureUri, ensuring it's in a valid format for the SDK to process.

• The third argument, "RES_512", likely specifies the resolution or processing mode for the image.

import {
  PassioAdvisorFoodInfo,
  PassioSDK,
} from "@passiolife/nutritionai-react-native-sdk-v3";

const onScanImage = () => {
    setPassioSpeechRecognitionModel(null);
    PassioSDK.recognizeImageRemote(
      pictureUri?.replace("file://", "") ?? "",
      undefined,
      "RES_512"
    )
      .then((candidates) => {
        if (candidates?.length === 0){
          Alert.alert("unable to find any food.");
        }else{
          setPassioSpeechRecognitionModel(candidates);
        }
      })
      .catch((e) => {
        Alert.alert("Unable to recognized this image");
      })
      .finally(() => {
        setPictureUri("");
      });
  };

Display the recognition results in the UI

In this UI component using a BottomSheet with dynamic sizing that contains a list of recognized items (from passioSpeechRecognitionModel) and a close button.

          <BottomSheet ref={sheetRef} enableDynamicSizing>
             ...
              <BottomSheetFlatList
                data={passioSpeechRecognitionModel}
                keyExtractor={(_, index) => `${index}`}
                renderItem={renderItem}
                contentContainerStyle={{ paddingTop: 80 }}
                ListFooterComponent={() => <View style={{ height: 50 }} />}
              />              />
          </BottomSheet>

  const renderItem = ({ item }: { item: PassioAdvisorFoodInfo }) => {
    const info = `${item.portionSize} | ${
      item.foodDataInfo?.nutritionPreview?.calories ??
      item?.packagedFoodItem?.ingredients?.[0].referenceNutrients?.calories
        ?.value
    } kcal`;
    const name =
      item?.foodDataInfo?.foodName ??
      item?.packagedFoodItem?.name ??
      item?.recognisedName;
    const iconID = item?.foodDataInfo?.iconID ?? item?.packagedFoodItem?.iconId;

    return (
      <TouchableOpacity
        style={{
          padding: 12,
          flex: 1,
          marginVertical: 4,
          marginHorizontal: 16,
          backgroundColor: "rgba(238, 242, 255, 1)",
          flexDirection: "row",
          borderRadius: 8,
        }}
        onPress={async () => {
         // Pass food detail info
         // Later you can pass food detial on press of item
          if (item.packagedFoodItem) {
            props.onFoodDetail(item.packagedFoodItem);
          } else {
            if (item.foodDataInfo) {
              const dataInfo = await PassioSDK.fetchFoodItemForDataInfo(
                item?.foodDataInfo
              );
              if (dataInfo) {
                props.onFoodDetail(dataInfo);
              }
            }
          }
        }}
      >
        <View
          style={{
            height: 46,
            width: 46,
            borderRadius: 30,
            overflow: "hidden",
          }}
        >
          <PassioIconView
            style={{
              height: 46,
              width: 46,
            }}
            config={{
              passioID: iconID ?? "",
              iconSize: IconSize.PX360,
            }}
          />
        </View>
        <View style={{ flex: 1 }}>
          <Text
            style={[
              {
                flex: 1,
                marginHorizontal: 8,
                fontSize: 16,
              },
              { textTransform: "capitalize" },
            ]}
          >
            {name}
          </Text>
          <Text
            style={{
              flex: 1,
              marginHorizontal: 8,
              fontSize: 16,
            }}
          >
            {info}
          </Text>
        </View>
      </TouchableOpacity>
    );
  };

Check out the full code here. FULL CODE