image.mdx 1.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445
  1. ---
  2. title: "🖼️ Image"
  3. ---
  4. To use an image as data source, just add `data_type` as `image` and pass in the path of the image (local or hosted).
  5. We use [GPT4 Vision](https://platform.openai.com/docs/guides/vision) to generate meaning of the image using a custom prompt, and then use the generated text as the data source.
  6. You would require an OpenAI API key with access to `gpt-4-vision-preview` model to use this feature.
  7. ### Without customization
  8. ```python
  9. import os
  10. from embedchain import App
  11. os.environ["OPENAI_API_KEY"] = "sk-xxx"
  12. app = App()
  13. app.add("./Elon-Musk.webp", data_type="image")
  14. response = app.query("Describe the man in the image.")
  15. print(response)
  16. # Answer: The man in the image is dressed in formal attire, wearing a dark suit jacket and a white collared shirt. He has short hair and is standing. He appears to be gazing off to the side with a reflective expression. The background is dark with faint, warm-toned vertical lines, possibly from a lit environment behind the individual or reflections. The overall atmosphere is somewhat moody and introspective.
  17. ```
  18. ### Customization
  19. ```python
  20. import os
  21. from embedchain import App
  22. from embedchain.loaders.image import ImageLoader
  23. image_loader = ImageLoader(
  24. max_tokens=100,
  25. api_key="sk-xxx",
  26. prompt="Is the person looking wealthy? Structure your thoughts around what you see in the image.",
  27. )
  28. app = App()
  29. app.add("./Elon-Musk.webp", data_type="image", loader=image_loader)
  30. response = app.query("Describe the man in the image.")
  31. print(response)
  32. # Answer: The man in the image appears to be well-dressed in a suit and shirt, suggesting that he may be in a professional or formal setting. His composed demeanor and confident posture further indicate a sense of self-assurance. Based on these visual cues, one could infer that the man may have a certain level of economic or social status, possibly indicating wealth or professional success.
  33. ```