evaluate.mdx 1.2 KB

1234567891011121314151617181920212223242526272829303132333435363738394041
  1. ---
  2. title: '📝 evaluate'
  3. ---
  4. `evaluate()` method is used to evaluate the performance of a RAG app. You can find the signature below:
  5. ### Parameters
  6. <ParamField path="question" type="Union[str, list[str]]">
  7. A question or a list of questions to evaluate your app on.
  8. </ParamField>
  9. <ParamField path="metrics" type="Optional[list[Union[BaseMetric, str]]]" optional>
  10. The metrics to evaluate your app on. Defaults to all metrics: `["context_relevancy", "answer_relevancy", "groundedness"]`
  11. </ParamField>
  12. <ParamField path="num_workers" type="int" optional>
  13. Specify the number of threads to use for parallel processing.
  14. </ParamField>
  15. ### Returns
  16. <ResponseField name="metrics" type="dict">
  17. Returns the metrics you have chosen to evaluate your app on as a dictionary.
  18. </ResponseField>
  19. ## Usage
  20. ```python
  21. from embedchain import App
  22. app = App()
  23. # add data source
  24. app.add("https://www.forbes.com/profile/elon-musk")
  25. # run evaluation
  26. app.evaluate("what is the net worth of Elon Musk?")
  27. # {'answer_relevancy': 0.958019958036268, 'context_relevancy': 0.12903225806451613}
  28. # or
  29. # app.evaluate(["what is the net worth of Elon Musk?", "which companies does Elon Musk own?"])
  30. ```