These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Langfuse
Langfuse is an open-source engineering platform for applications built on large language models (LLMs). It helps teams develop, monitor, evaluate and debug AI applications, including agents and retrieval-augmented generation systems. Langfuse makes the behaviour of these applications visible and measurable. This supports transparency, quality assurance and accountability in their development and operation.
Developers instrument their applications through software development kits (SDKs) for Python and JavaScript/TypeScript, or through integrations with frameworks such as LangChain, LlamaIndex, CrewAI and the OpenAI SDK. Langfuse then records traces of each interaction. Traces capture LLM calls and related steps such as retrieval, embeddings and agent actions. Teams can inspect these traces to understand application behaviour, investigate failures and monitor user sessions. Prompt management lets teams store, version and collaboratively refine prompts in one place.
Evaluation features help teams measure output quality. These include LLM-as-a-judge scoring, code-based evaluators, user feedback collection, manual labelling and custom evaluation pipelines. Datasets support test sets and benchmarks for pre-deployment testing and structured experiments. A playground lets users test prompts and model settings, starting directly from problematic results found in traces. An application programming interface (API) supports custom workflows. Langfuse can be self-hosted or used as a managed cloud service with a free tier. The code is released under the MIT licence, except for enterprise folders.
About the tool
You can click on the links to see the associated tools
Developing organisation(s):
Tool type(s):
Objective(s):
Impacted stakeholders:
Purpose(s):
Target sector(s):
Country/Territory of origin:
Lifecycle stage(s):
Type of approach:
Maturity:
Usage rights:
License:
Target groups:
Target users:
Stakeholder group:
Validity:
Geographical scope:
Required skills:
Risk management stage(s):
Technology platforms:
Github stars:
- 36850
Github forks:
- 4140
Use Cases
Would you like to submit a use case for this tool?
If you have used this tool, we would love to know more about your experience.
Add use case
Partnership on AI
