chat.mdx 5.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175
  1. ---
  2. title: '💬 chat'
  3. ---
  4. `chat()` method allows you to chat over your data sources using a user-friendly chat API. You can find the signature below:
  5. ### Parameters
  6. <ParamField path="input_query" type="str">
  7. Question to ask
  8. </ParamField>
  9. <ParamField path="config" type="BaseLlmConfig" optional>
  10. Configure different llm settings such as prompt, temprature, number_documents etc.
  11. </ParamField>
  12. <ParamField path="dry_run" type="bool" optional>
  13. The purpose is to test the prompt structure without actually running LLM inference. Defaults to `False`
  14. </ParamField>
  15. <ParamField path="where" type="dict" optional>
  16. A dictionary of key-value pairs to filter the chunks from the vector database. Defaults to `None`
  17. </ParamField>
  18. <ParamField path="session_id" type="str" optional>
  19. Session ID of the chat. This can be used to maintain chat history of different user sessions. Default value: `default`
  20. </ParamField>
  21. <ParamField path="citations" type="bool" optional>
  22. Return citations along with the LLM answer. Defaults to `False`
  23. </ParamField>
  24. ### Returns
  25. <ResponseField name="answer" type="str | tuple">
  26. If `citations=False`, return a stringified answer to the question asked. <br />
  27. If `citations=True`, returns a tuple with answer and citations respectively.
  28. </ResponseField>
  29. ## Usage
  30. ### With citations
  31. If you want to get the answer to question and return both answer and citations, use the following code snippet:
  32. ```python With Citations
  33. from embedchain import App
  34. # Initialize app
  35. app = App()
  36. # Add data source
  37. app.add("https://www.forbes.com/profile/elon-musk")
  38. # Get relevant answer for your query
  39. answer, sources = app.chat("What is the net worth of Elon?", citations=True)
  40. print(answer)
  41. # Answer: The net worth of Elon Musk is $221.9 billion.
  42. print(sources)
  43. # [
  44. # (
  45. # 'Elon Musk PROFILEElon MuskCEO, Tesla$247.1B$2.3B (0.96%)Real Time Net Worthas of 12/7/23 ...',
  46. # {
  47. # 'url': 'https://www.forbes.com/profile/elon-musk',
  48. # 'score': 0.89,
  49. # ...
  50. # }
  51. # ),
  52. # (
  53. # '74% of the company, which is now called X.Wealth HistoryHOVER TO REVEAL NET WORTH BY YEARForbes ...',
  54. # {
  55. # 'url': 'https://www.forbes.com/profile/elon-musk',
  56. # 'score': 0.81,
  57. # ...
  58. # }
  59. # ),
  60. # (
  61. # 'founded in 2002, is worth nearly $150 billion after a $750 million tender offer in June 2023 ...',
  62. # {
  63. # 'url': 'https://www.forbes.com/profile/elon-musk',
  64. # 'score': 0.73,
  65. # ...
  66. # }
  67. # )
  68. # ]
  69. ```
  70. <Note>
  71. When `citations=True`, note that the returned `sources` are a list of tuples where each tuple has two elements (in the following order):
  72. 1. source chunk
  73. 2. dictionary with metadata about the source chunk
  74. - `url`: url of the source
  75. - `doc_id`: document id (used for book keeping purposes)
  76. - `score`: score of the source chunk with respect to the question
  77. - other metadata you might have added at the time of adding the source
  78. </Note>
  79. ### Without citations
  80. If you just want to return answers and don't want to return citations, you can use the following example:
  81. ```python Without Citations
  82. from embedchain import App
  83. # Initialize app
  84. app = App()
  85. # Add data source
  86. app.add("https://www.forbes.com/profile/elon-musk")
  87. # Chat on your data using `.chat()`
  88. answer = app.chat("What is the net worth of Elon?")
  89. print(answer)
  90. # Answer: The net worth of Elon Musk is $221.9 billion.
  91. ```
  92. ### With session id
  93. If you want to maintain chat sessions for different users, you can simply pass the `session_id` keyword argument. See the example below:
  94. ```python With session id
  95. from embedchain import App
  96. app = App()
  97. app.add("https://www.forbes.com/profile/elon-musk")
  98. # Chat on your data using `.chat()`
  99. app.chat("What is the net worth of Elon Musk?", session_id="user1")
  100. # 'The net worth of Elon Musk is $250.8 billion.'
  101. app.chat("What is the net worth of Bill Gates?", session_id="user2")
  102. # "I don't know the current net worth of Bill Gates."
  103. app.chat("What was my last question", session_id="user1")
  104. # 'Your last question was "What is the net worth of Elon Musk?"'
  105. ```
  106. ### With custom context window
  107. If you want to customize the context window that you want to use during chat (default context window is 3 document chunks), you can do using the following code snippet:
  108. ```python with custom chunks size
  109. from embedchain import App
  110. from embedchain.config import BaseLlmConfig
  111. app = App()
  112. app.add("https://www.forbes.com/profile/elon-musk")
  113. query_config = BaseLlmConfig(number_documents=5)
  114. app.chat("What is the net worth of Elon Musk?", config=query_config)
  115. ```
  116. ### With Mem0 to store chat history
  117. Mem0 is a cutting-edge long-term memory for LLMs to enable personalization for the GenAI stack. It enables LLMs to remember past interactions and provide more personalized responses.
  118. In order to use Mem0 to enable memory for personalization in your apps:
  119. - Install the [`mem0`](https://docs.mem0.ai/) package using `pip install mem0ai`.
  120. - Prepare config for `memory`, refer [Configurations](docs/api-reference/advanced/configuration.mdx).
  121. ```python with mem0
  122. from embedchain import App
  123. config = {
  124. "memory": {
  125. "top_k": 5
  126. }
  127. }
  128. app = App.from_config(config=config)
  129. app.add("https://www.forbes.com/profile/elon-musk")
  130. app.chat("What is the net worth of Elon Musk?")
  131. ```
  132. ## How Mem0 works:
  133. - Mem0 saves context derived from each user question into its memory.
  134. - When a user poses a new question, Mem0 retrieves relevant previous memories.
  135. - The `top_k` parameter in the memory configuration specifies the number of top memories to consider during retrieval.
  136. - Mem0 generates the final response by integrating the user's question, context from the data source, and the relevant memories.