vector-databases.mdx 4.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224
  1. ---
  2. title: 🗄️ Vector databases
  3. ---
  4. ## Overview
  5. Utilizing a vector database alongside Embedchain is a seamless process. All you need to do is configure it within the YAML configuration file. We've provided examples for each supported database below:
  6. <CardGroup cols={4}>
  7. <Card title="ChromaDB" href="#chromadb"></Card>
  8. <Card title="Elasticsearch" href="#elasticsearch"></Card>
  9. <Card title="OpenSearch" href="#opensearch"></Card>
  10. <Card title="Zilliz" href="#zilliz"></Card>
  11. <Card title="LanceDB" href="#lancedb"></Card>
  12. <Card title="Pinecone" href="#pinecone"></Card>
  13. <Card title="Qdrant" href="#qdrant"></Card>
  14. <Card title="Weaviate" href="#weaviate"></Card>
  15. </CardGroup>
  16. ## ChromaDB
  17. <CodeGroup>
  18. ```python main.py
  19. from embedchain import Pipeline as App
  20. # load chroma configuration from yaml file
  21. app = App.from_config(yaml_path="config1.yaml")
  22. ```
  23. ```yaml config1.yaml
  24. vectordb:
  25. provider: chroma
  26. config:
  27. collection_name: 'my-collection'
  28. dir: db
  29. allow_reset: true
  30. ```
  31. ```yaml config2.yaml
  32. vectordb:
  33. provider: chroma
  34. config:
  35. collection_name: 'my-collection'
  36. host: localhost
  37. port: 5200
  38. allow_reset: true
  39. ```
  40. </CodeGroup>
  41. ## Elasticsearch
  42. Install related dependencies using the following command:
  43. ```bash
  44. pip install --upgrade 'embedchain[elasticsearch]'
  45. ```
  46. <CodeGroup>
  47. ```python main.py
  48. from embedchain import Pipeline as App
  49. # load elasticsearch configuration from yaml file
  50. app = App.from_config(yaml_path="config.yaml")
  51. ```
  52. ```yaml config.yaml
  53. vectordb:
  54. provider: elasticsearch
  55. config:
  56. collection_name: 'es-index'
  57. es_url: http://localhost:9200
  58. allow_reset: true
  59. api_key: xxx
  60. ```
  61. </CodeGroup>
  62. ## OpenSearch
  63. Install related dependencies using the following command:
  64. ```bash
  65. pip install --upgrade 'embedchain[opensearch]'
  66. ```
  67. <CodeGroup>
  68. ```python main.py
  69. from embedchain import Pipeline as App
  70. # load opensearch configuration from yaml file
  71. app = App.from_config(yaml_path="config.yaml")
  72. ```
  73. ```yaml config.yaml
  74. vectordb:
  75. provider: opensearch
  76. config:
  77. opensearch_url: 'https://localhost:9200'
  78. http_auth:
  79. - admin
  80. - admin
  81. vector_dimension: 1536
  82. collection_name: 'my-app'
  83. use_ssl: false
  84. verify_certs: false
  85. ```
  86. </CodeGroup>
  87. ## Zilliz
  88. Install related dependencies using the following command:
  89. ```bash
  90. pip install --upgrade 'embedchain[milvus]'
  91. ```
  92. Set the Zilliz environment variables `ZILLIZ_CLOUD_URI` and `ZILLIZ_CLOUD_TOKEN` which you can find it on their [cloud platform](https://cloud.zilliz.com/).
  93. <CodeGroup>
  94. ```python main.py
  95. import os
  96. from embedchain import Pipeline as App
  97. os.environ['ZILLIZ_CLOUD_URI'] = 'https://xxx.zillizcloud.com'
  98. os.environ['ZILLIZ_CLOUD_TOKEN'] = 'xxx'
  99. # load zilliz configuration from yaml file
  100. app = App.from_config(yaml_path="config.yaml")
  101. ```
  102. ```yaml config.yaml
  103. vectordb:
  104. provider: zilliz
  105. config:
  106. collection_name: 'zilliz-app'
  107. uri: https://xxxx.api.gcp-region.zillizcloud.com
  108. token: xxx
  109. vector_dim: 1536
  110. metric_type: L2
  111. ```
  112. </CodeGroup>
  113. ## LanceDB
  114. _Coming soon_
  115. ## Pinecone
  116. Install pinecone related dependencies using the following command:
  117. ```bash
  118. pip install --upgrade 'embedchain[pinecone]'
  119. ```
  120. In order to use Pinecone as vector database, set the environment variables `PINECONE_API_KEY` and `PINECONE_ENV` which you can find on [Pinecone dashboard](https://app.pinecone.io/).
  121. <CodeGroup>
  122. ```python main.py
  123. from embedchain import Pipeline as App
  124. # load pinecone configuration from yaml file
  125. app = App.from_config(yaml_path="config.yaml")
  126. ```
  127. ```yaml config.yaml
  128. vectordb:
  129. provider: pinecone
  130. config:
  131. metric: cosine
  132. vector_dimension: 1536
  133. collection_name: my-pinecone-index
  134. ```
  135. </CodeGroup>
  136. ## Qdrant
  137. In order to use Qdrant as a vector database, set the environment variables `QDRANT_URL` and `QDRANT_API_KEY` which you can find on [Qdrant Dashboard](https://cloud.qdrant.io/).
  138. <CodeGroup>
  139. ```python main.py
  140. from embedchain import Pipeline as App
  141. # load qdrant configuration from yaml file
  142. app = App.from_config(yaml_path="config.yaml")
  143. ```
  144. ```yaml config.yaml
  145. vectordb:
  146. provider: qdrant
  147. config:
  148. collection_name: my_qdrant_index
  149. ```
  150. </CodeGroup>
  151. ## Weaviate
  152. In order to use Weaviate as a vector database, set the environment variables `WEAVIATE_ENDPOINT` and `WEAVIATE_API_KEY` which you can find on [Weaviate dashboard](https://console.weaviate.cloud/dashboard).
  153. <CodeGroup>
  154. ```python main.py
  155. from embedchain import Pipeline as App
  156. # load weaviate configuration from yaml file
  157. app = App.from_config(yaml_path="config.yaml")
  158. ```
  159. ```yaml config.yaml
  160. vectordb:
  161. provider: weaviate
  162. config:
  163. collection_name: my_weaviate_index
  164. ```
  165. </CodeGroup>
  166. <Snippet file="missing-vector-db-tip.mdx" />