vector-databases.mdx 5.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232
  1. ---
  2. title: 🗄️ Vector databases
  3. ---
  4. ## Overview
  5. Utilizing a vector database alongside Embedchain is a seamless process. All you need to do is configure it within the YAML configuration file. We've provided examples for each supported database below:
  6. <CardGroup cols={4}>
  7. <Card title="ChromaDB" href="#chromadb"></Card>
  8. <Card title="Elasticsearch" href="#elasticsearch"></Card>
  9. <Card title="OpenSearch" href="#opensearch"></Card>
  10. <Card title="Zilliz" href="#zilliz"></Card>
  11. <Card title="LanceDB" href="#lancedb"></Card>
  12. <Card title="Pinecone" href="#pinecone"></Card>
  13. <Card title="Qdrant" href="#qdrant"></Card>
  14. <Card title="Weaviate" href="#weaviate"></Card>
  15. </CardGroup>
  16. ## ChromaDB
  17. <CodeGroup>
  18. ```python main.py
  19. from embedchain import Pipeline as App
  20. # load chroma configuration from yaml file
  21. app = App.from_config(config_path="config1.yaml")
  22. ```
  23. ```yaml config1.yaml
  24. vectordb:
  25. provider: chroma
  26. config:
  27. collection_name: 'my-collection'
  28. dir: db
  29. allow_reset: true
  30. ```
  31. ```yaml config2.yaml
  32. vectordb:
  33. provider: chroma
  34. config:
  35. collection_name: 'my-collection'
  36. host: localhost
  37. port: 5200
  38. allow_reset: true
  39. ```
  40. </CodeGroup>
  41. ## Elasticsearch
  42. Install related dependencies using the following command:
  43. ```bash
  44. pip install --upgrade 'embedchain[elasticsearch]'
  45. ```
  46. <Note>
  47. You can configure the Elasticsearch connection by providing either `es_url` or `cloud_id`. If you are using the Elasticsearch Service on Elastic Cloud, you can find the `cloud_id` on the [Elastic Cloud dashboard](https://cloud.elastic.co/deployments).
  48. </Note>
  49. You can authorize the connection to Elasticsearch by providing either `basic_auth`, `api_key`, or `bearer_auth`.
  50. <CodeGroup>
  51. ```python main.py
  52. from embedchain import Pipeline as App
  53. # load elasticsearch configuration from yaml file
  54. app = App.from_config(config_path="config.yaml")
  55. ```
  56. ```yaml config.yaml
  57. vectordb:
  58. provider: elasticsearch
  59. config:
  60. collection_name: 'es-index'
  61. cloud_id: 'deployment-name:xxxx'
  62. basic_auth:
  63. - elastic
  64. - <your_password>
  65. verify_certs: false
  66. ```
  67. </CodeGroup>
  68. ## OpenSearch
  69. Install related dependencies using the following command:
  70. ```bash
  71. pip install --upgrade 'embedchain[opensearch]'
  72. ```
  73. <CodeGroup>
  74. ```python main.py
  75. from embedchain import Pipeline as App
  76. # load opensearch configuration from yaml file
  77. app = App.from_config(config_path="config.yaml")
  78. ```
  79. ```yaml config.yaml
  80. vectordb:
  81. provider: opensearch
  82. config:
  83. collection_name: 'my-app'
  84. opensearch_url: 'https://localhost:9200'
  85. http_auth:
  86. - admin
  87. - admin
  88. vector_dimension: 1536
  89. use_ssl: false
  90. verify_certs: false
  91. ```
  92. </CodeGroup>
  93. ## Zilliz
  94. Install related dependencies using the following command:
  95. ```bash
  96. pip install --upgrade 'embedchain[milvus]'
  97. ```
  98. Set the Zilliz environment variables `ZILLIZ_CLOUD_URI` and `ZILLIZ_CLOUD_TOKEN` which you can find it on their [cloud platform](https://cloud.zilliz.com/).
  99. <CodeGroup>
  100. ```python main.py
  101. import os
  102. from embedchain import Pipeline as App
  103. os.environ['ZILLIZ_CLOUD_URI'] = 'https://xxx.zillizcloud.com'
  104. os.environ['ZILLIZ_CLOUD_TOKEN'] = 'xxx'
  105. # load zilliz configuration from yaml file
  106. app = App.from_config(config_path="config.yaml")
  107. ```
  108. ```yaml config.yaml
  109. vectordb:
  110. provider: zilliz
  111. config:
  112. collection_name: 'zilliz_app'
  113. uri: https://xxxx.api.gcp-region.zillizcloud.com
  114. token: xxx
  115. vector_dim: 1536
  116. metric_type: L2
  117. ```
  118. </CodeGroup>
  119. ## LanceDB
  120. _Coming soon_
  121. ## Pinecone
  122. Install pinecone related dependencies using the following command:
  123. ```bash
  124. pip install --upgrade 'embedchain[pinecone]'
  125. ```
  126. In order to use Pinecone as vector database, set the environment variables `PINECONE_API_KEY` and `PINECONE_ENV` which you can find on [Pinecone dashboard](https://app.pinecone.io/).
  127. <CodeGroup>
  128. ```python main.py
  129. from embedchain import Pipeline as App
  130. # load pinecone configuration from yaml file
  131. app = App.from_config(config_path="config.yaml")
  132. ```
  133. ```yaml config.yaml
  134. vectordb:
  135. provider: pinecone
  136. config:
  137. metric: cosine
  138. vector_dimension: 1536
  139. collection_name: my-pinecone-index
  140. ```
  141. </CodeGroup>
  142. ## Qdrant
  143. In order to use Qdrant as a vector database, set the environment variables `QDRANT_URL` and `QDRANT_API_KEY` which you can find on [Qdrant Dashboard](https://cloud.qdrant.io/).
  144. <CodeGroup>
  145. ```python main.py
  146. from embedchain import Pipeline as App
  147. # load qdrant configuration from yaml file
  148. app = App.from_config(config_path="config.yaml")
  149. ```
  150. ```yaml config.yaml
  151. vectordb:
  152. provider: qdrant
  153. config:
  154. collection_name: my_qdrant_index
  155. ```
  156. </CodeGroup>
  157. ## Weaviate
  158. In order to use Weaviate as a vector database, set the environment variables `WEAVIATE_ENDPOINT` and `WEAVIATE_API_KEY` which you can find on [Weaviate dashboard](https://console.weaviate.cloud/dashboard).
  159. <CodeGroup>
  160. ```python main.py
  161. from embedchain import Pipeline as App
  162. # load weaviate configuration from yaml file
  163. app = App.from_config(config_path="config.yaml")
  164. ```
  165. ```yaml config.yaml
  166. vectordb:
  167. provider: weaviate
  168. config:
  169. collection_name: my_weaviate_index
  170. ```
  171. </CodeGroup>
  172. <Snippet file="missing-vector-db-tip.mdx" />