RecognizeImageRemote
The RecognizeImageRemote
component is responsible for recognizing food items from an image, either by taking a photo
using the camera or selecting an image from the gallery
. It integrates with the Passio Nutrition SDK to fetch food details and displays the results in a bottom sheet.
Install required Libraries
yarn add react-native-image-picker && expo-camera && @gorhom/bottom-sheet
useCameraPermissions
: It seems like you're using a custom hook to handle camera permissions, but it's not a native hook from libraries like expo-camera
Ensure that the hook is correctly implemented or imported.
const [permission, requestPermission] = useCameraPermissions();
useEffect(() => {
requestPermission();
}, []);
cameraViewRef
: You need to ensure that this ref is properly created and assigned to the camera view. If you're using expo-camera
, ensure you're following the right method for capturing images.
const cameraViewRef = useRef<CameraView>(null);
setPictureUri
: This is the state updater function. Whenever a new picture is captured, you'll call setPictureUri
with the URI of the picture to update the state. This triggers a re-render of the component with the updated picture URI.
const [pictureUri, setPictureUri] = useState<string>("");
Take a Picture
const takePicture = useCallback(async () => {
try {
const picture = await cameraViewRef.current?.takePictureAsync();
setPictureUri(picture?.uri ?? "");
} catch (_) {
Alert.alert("error while capture image");
}
}, []);
Pick Picture from Gallery
const pickImage = useCallback(async () => {
try {
const { assets } = await launchImageLibrary({
mediaType: "photo",
});
setPictureUri(assets?.[0].uri ?? "");
} catch (err) {
setLoading(false);
}
}, []);
Let's Recognize the picture using SDK
The onScanImage
function is responsible for sending a captured image to the Passio SDK for remote image recognition . It processes the image, sends it to a remote server for recognition, and updates the local state based on the response.
const [passioSpeechRecognitionModel, setPassioSpeechRecognitionModel] =
useState<PassioAdvisorFoodInfo[] | null>();
This part sends the image to PassioSDK
for remote recognition.
pictureUri?.replace("file://", "")
: This removes thefile://
prefix from thepictureUri
, ensuring it's in a valid format for the SDK to process.The third argument,
"RES_512"
, likely specifies the resolution or processing mode for the image.
import {
PassioAdvisorFoodInfo,
PassioSDK,
} from "@passiolife/nutritionai-react-native-sdk-v3";
const onScanImage = () => {
setPassioSpeechRecognitionModel(null);
PassioSDK.recognizeImageRemote(
pictureUri?.replace("file://", "") ?? "",
undefined,
"RES_512"
)
.then((candidates) => {
if (candidates?.length === 0){
Alert.alert("unable to find any food.");
}else{
setPassioSpeechRecognitionModel(candidates);
}
})
.catch((e) => {
Alert.alert("Unable to recognized this image");
})
.finally(() => {
setPictureUri("");
});
};
Display the recognition results in the UI
In this UI component using a BottomSheet
with dynamic sizing that contains a list of recognized items (from passioSpeechRecognitionModel
) and a close button.
<BottomSheet ref={sheetRef} enableDynamicSizing>
...
<BottomSheetFlatList
data={passioSpeechRecognitionModel}
keyExtractor={(_, index) => `${index}`}
renderItem={renderItem}
contentContainerStyle={{ paddingTop: 80 }}
ListFooterComponent={() => <View style={{ height: 50 }} />}
/> />
</BottomSheet>
const renderItem = ({ item }: { item: PassioAdvisorFoodInfo }) => {
const info = `${item.portionSize} | ${
item.foodDataInfo?.nutritionPreview?.calories ??
item?.packagedFoodItem?.ingredients?.[0].referenceNutrients?.calories
?.value
} kcal`;
const name =
item?.foodDataInfo?.foodName ??
item?.packagedFoodItem?.name ??
item?.recognisedName;
const iconID = item?.foodDataInfo?.iconID ?? item?.packagedFoodItem?.iconId;
return (
<TouchableOpacity
style={{
padding: 12,
flex: 1,
marginVertical: 4,
marginHorizontal: 16,
backgroundColor: "rgba(238, 242, 255, 1)",
flexDirection: "row",
borderRadius: 8,
}}
onPress={async () => {
// Pass food detail info
// Later you can pass food detial on press of item
if (item.packagedFoodItem) {
props.onFoodDetail(item.packagedFoodItem);
} else {
if (item.foodDataInfo) {
const dataInfo = await PassioSDK.fetchFoodItemForDataInfo(
item?.foodDataInfo
);
if (dataInfo) {
props.onFoodDetail(dataInfo);
}
}
}
}}
>
<View
style={{
height: 46,
width: 46,
borderRadius: 30,
overflow: "hidden",
}}
>
<PassioIconView
style={{
height: 46,
width: 46,
}}
config={{
passioID: iconID ?? "",
iconSize: IconSize.PX360,
}}
/>
</View>
<View style={{ flex: 1 }}>
<Text
style={[
{
flex: 1,
marginHorizontal: 8,
fontSize: 16,
},
{ textTransform: "capitalize" },
]}
>
{name}
</Text>
<Text
style={{
flex: 1,
marginHorizontal: 8,
fontSize: 16,
}}
>
{info}
</Text>
</View>
</TouchableOpacity>
);
};
Check out the full code here. FULL CODE
Last updated