Welcome to the future, where not only can ChatGPT talk, but it can also see! Mind-blowing, isn’t it?
Here are 10 super useful ways you can use the multimodal image recognition feature inside the ChatGPT chat window.
1. Language Translation
Imagine you’re traveling, and you come across a signboard that looks important, but alas! It’s in a language that seems as decipherable as the enigma code to you. We’ve all been there, haven’t we?
Strolling down the cobbled streets of an old European town, you stumble upon an ancient-looking signboard. It’s intriguing, it’s mysterious, it’s…in Italian. And your Italian, let’s face it, is as rusty as the signboard itself. What do you do? Take a snap and toss it to ChatGPT!
This isn’t just about deciphering foreign menus so we don’t accidentally order snails (unless you’re into that sort of thing, no judgment here!). It’s about breaking barriers, making connections, and ensuring that language is no longer a roadblock, but a gateway.
Also, Imagine if you are a student or researcher and need to translate an entire page of text.
2. Healthcare Guidance
We all have those “uh-oh” moments when a new spot, bump, or rash decides to set up camp on our bodies. Rather than jumping down the WebMD rabbit hole – you know, the one that convinces you you’ve got a rare, unpronounceable disease from the 18th century – let’s hit up ChatGPT.
ChatGPT will give you the usual lecture that it’s not a doctor blah blah, but it can then certainly point you in the right direction, giving you a nudge before you consult the real doctors. You’ve got to love the future!
It’s all about that initial step, the “hey, should I be worried about this?” moment. Because who wants to spend hours in a waiting room only to be told it’s a bug bite? Not me!
IMPORTANT: just make sure you don’t include any faces in the image as ChatGPT has been restricted from analyzing images with human faces.
3. Educational Tutoring
We’ve all been there, staring at a math problem that seems to be written in hieroglyphics, or a historical map that might as well be a treasure map, because, where’s the gold? Aha! Snap a pic, shoot it over to ChatGPT, and watch the magic happen!
Note: Just make sure you have the Wolfram plugin enabled if you’re using it for math.
Isn’t learning supposed to be fun?
Useful for trading chart analysis too – as Twitter user @youraimarketer demonstrates below:
4. Fashion Recommendations
Ready to step out but your outfit’s screaming “Help!” louder than a horror movie victim?
We’ve all faced the closet conundrum. Stacks of clothes but nothing to wear. It’s like being hungry with a fridge full of food and still ordering takeout.
Who knew your grandma’s brooch and those forgotten boots could look so chic together?
5. Fitness Technique Correction
Flexing in front of the mirror but something feels off? Maybe your plank looks more like a saggy bridge than a sturdy board. Don’t sweat it (well, keep sweating, but you know what I mean).
You snap a pic of your form – no faces, we’re keeping it anonymous! ChatGPT doesn’t need to see your game face.
In no time, ChatGPT is analyzing your form, making sure you’re not heading to Snap City. We get it, everyone’s fitness journey is unique. Whether you’re lifting, running, or getting down with downward dog, proper form is key!
6. Cooking Assistance
ChatGPT has already been incredible with recipes and cooking guidance, especially when you give it a list of ingredients. Now you don’t have stand in front of your fridge, eyes darting between a lonely carrot and a half-eaten block of cheese, wondering if there’s a culinary masterpiece lurking in there?
Snap a picture and ChatGPT can get to work.
ChatGPT turns those ingredients into a symphony of flavors.
7. Home Improvement Advice
Staring at your living room, feeling like it’s more “meh” than “yeah”? Dreaming of a space that screams “you” but stuck in a décor dilemma?
Snap a pic of your space and let ChatGPT be your interior design guru! Whether you’re in a cozy corner apartment or a sprawling suburban dwelling, it’s about to get a whole lot homier in there!
Before you know it, ChatGPT is dishing out advice like an HGTV star. Color palettes? Check! Furniture arrangement that flows? Double-check! Lighting that makes everything glow? Triple-check! It’s like having a home makeover show right in your living room?
8. Wildlife and Pet Behavior Understanding
Ever watched your cat perform Cirque du Soleil stunts at 3 AM and wondered, “What’s up, Whiskers?” Or stumbled upon a creature during a hike and thought, “Who’s this little fella?” Snap a pic and ask away, because ChatGPT is about to become your wildlife whisperer and pet psychologist!
ChatGPT is dropping knowledge bombs about species traits, dishing out fun facts, and serving up tailored tips to understand and care for your furry, feathered, or scaly friends!
9. Identifying Plant Species
Stumbling upon a mysterious leafy fellow in your garden, eyes squinting, scratching your head? Or, maybe you’ve got this potted buddy at home looking a bit, well, under the weather?
Take a picture and….
Before you can say “chlorophyll”, ChatGPT dives into its botanical brain, spitting out species names and care instructions like a horticultural hip-hop artist. Whether it’s a Sunflower or a Sneezewort, you’re getting the 411 on suitable environments, watering rhythms, and maybe, just maybe, how to finally keep that basil alive!
10. OCR Feature
Imagine this – you’ve got a pile of old receipts, documents, and, oh, maybe a cryptic note or two. Instead of diving into deciphering that jumbled jungle, snap a pic, upload, and let ChatGPT do the heavy lifting!
And here’s the kicker – got a test question that looks like it’s written in ancient runes? Snap it, send it, and watch ChatGPT turn into your personal tutor, breaking down the question, serving up the answers, all while making you feel like the genius! I pity teachers though.
FAQs about ChatGPT’s New Multimodal Image Feature
Voice and image capabilities. Now, you can have a voice conversation or show ChatGPT images to make interactions more intuitive and fun.
You’re traveling and snap a photo of a landmark. Now, you can have a live conversation with ChatGPT about it! Or, you’re at home pondering dinner; a quick pic of your fridge and pantry, and voila, dinner suggestions coming your way!
Plus and Enterprise users. Voice will be available on iOS and Android (just opt-in in your settings), and image capabilities will be available on all platforms.
Easy peasy! Tap the photo button to capture or choose an image. If you’re on iOS or Android, hit the plus button first. You can discuss multiple images or use our drawing tool to guide the conversation.
The image understanding is fueled by multimodal GPT-3.5 and GPT-4. These models apply their language reasoning skills to a wide range of images, including photographs, screenshots, and documents containing both text and images.
Vision-based models can sometimes misinterpret images, especially in high-stakes domains
ChatGPT has been working closely with Be My Eyes, an app for blind and low-vision people, to understand the uses and limitations of the new features. This collaboration has helped it make the feature both “useful and safe”.
No OpenAI has significantly limited ChatGPT’s ability to analyze and make direct statements about people in images to respect individuals’ privacy.
Yes. Users find it valuable to have general conversations about images, even if they contain people in the background. But remember, detailed analysis of individuals is a no-go!