Skip to content

Commit 7b1f469

Browse files
nirganirga
authored
Merge pull request #1 from traceloop/skeleton
feat: basic prompt logging
2 parents a16a3b9 + 56cb1be commit 7b1f469

15 files changed

Lines changed: 814 additions & 2 deletions

File tree

‎Gemfile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# frozen_string_literal: true
2+
3+
source "https://rubygems.org"
4+
5+
git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
6+
7+
# gem "rails"

‎README.md‎

Lines changed: 133 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,133 @@
1-
# openllmetry-ruby
2-
Sister project to OpenLLMetry, but in Ruby. Open-source observability for your LLM application, based on OpenTelemetry
1+
<p align="center">
2+
<a href="https://www.traceloop.com/">
3+
<img width="300" src="https://raw.githubusercontent.com/traceloop/openllmetry/main/img/logo.png">
4+
</a>
5+
</p>
6+
<h1 align="center">OpenLLMetry Ruby</h1>
7+
<p align="center">
8+
<p align="center">Open-source observability for your LLM application</p>
9+
</p>
10+
<h4 align="center">
11+
<a href="https://traceloop.com/docs/openllmetry/getting-started-ruby"><strong>Get started »</strong></a>
12+
<br />
13+
<br />
14+
<a href="https://join.slack.com/t/traceloopcommunity/shared_invite/zt-1plpfpm6r-zOHKI028VkpcWdobX65C~g">Slack</a> |
15+
<a href="https://traceloop.com/docs/openllmetry/introduction">Docs</a> |
16+
<a href="https://www.traceloop.com">Website</a>
17+
</h4>
18+
19+
<h4 align="center">
20+
<a href="https://github.com/traceloop/openllmetry/blob/main/LICENSE">
21+
<img src="https://img.shields.io/badge/license-Apache 2.0-blue.svg" alt="OpenLLMetry is released under the Apache-2.0 License">
22+
</a>
23+
<a href="https://www.ycombinator.com/companies/traceloop"><img src="https://img.shields.io/website?color=%23f26522&down_message=Y%20Combinator&label=Backed&logo=ycombinator&style=flat-square&up_message=Y%20Combinator&url=https%3A%2F%2Fwww.ycombinator.com"></a>
24+
<a href="https://github.com/traceloop/openllmetry/blob/main/CONTRIBUTING.md">
25+
<img src="https://img.shields.io/badge/PRs-Welcome-brightgreen" alt="PRs welcome!" />
26+
</a>
27+
<a href="https://github.com/traceloop/openllmetry/issues">
28+
<img src="https://img.shields.io/github/commit-activity/m/traceloop/openllmetry" alt="git commit activity" />
29+
</a>
30+
<a href="https://join.slack.com/t/traceloopcommunity/shared_invite/zt-1plpfpm6r-zOHKI028VkpcWdobX65C~g">
31+
<img src="https://img.shields.io/badge/chat-on%20Slack-blueviolet" alt="Slack community channel" />
32+
</a>
33+
<a href="https://twitter.com/traceloopdev">
34+
<img src="https://img.shields.io/badge/follow-%40traceloopdev-1DA1F2?logo=twitter&style=social" alt="Traceloop Twitter" />
35+
</a>
36+
</h4>
37+
38+
OpenLLMetry is a set of extensions built on top of [OpenTelemetry](https://opentelemetry.io/) that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Datadog, Honeycomb, and others.
39+
40+
It's built and maintained by Traceloop under the Apache 2.0 license.
41+
42+
The repo contains standard OpenTelemetry instrumentations for LLM providers and Vector DBs, as well as a Traceloop SDK that makes it easy to get started with OpenLLMetry, while still outputting standard OpenTelemetry data that can be connected to your observability stack.
43+
If you already have OpenTelemetry instrumented, you can just add any of our instrumentations directly.
44+
45+
## 🚀 Getting Started
46+
47+
The easiest way to get started is to use our SDK.
48+
For a complete guide, go to our [docs](https://traceloop.com/docs/openllmetry/getting-started-ruby).
49+
50+
Install the SDK:
51+
52+
```bash
53+
gem install traceloop-sdk
54+
```
55+
56+
Then, to start instrumenting your code, just add this line to your code:
57+
58+
```python
59+
from traceloop.sdk import Traceloop
60+
61+
Traceloop.init()
62+
```
63+
64+
That's it. You're now tracing your code with OpenLLMetry!
65+
If you're running this locally, you may want to disable batch sending, so you can see the traces immediately:
66+
67+
```python
68+
Traceloop.init(disable_batch=True)
69+
```
70+
71+
Now, you need to decide where to export the traces to.
72+
73+
## ⏫ Supported (and tested) destinations
74+
75+
- [x] [Traceloop](https://www.traceloop.com/docs/openllmetry/integrations/traceloop)
76+
- [x] [Dynatrace](https://www.traceloop.com/docs/openllmetry/integrations/dynatrace)
77+
- [x] [Datadog](https://www.traceloop.com/docs/openllmetry/integrations/datadog)
78+
- [x] [New Relic](https://www.traceloop.com/docs/openllmetry/integrations/newrelic)
79+
- [x] [Honeycomb](https://www.traceloop.com/docs/openllmetry/integrations/honeycomb)
80+
- [x] [Grafana Tempo](https://www.traceloop.com/docs/openllmetry/integrations/grafana)
81+
- [x] [HyperDX](https://www.traceloop.com/docs/openllmetry/integrations/hyperdx)
82+
- [x] [SigNoz](https://www.traceloop.com/docs/openllmetry/integrations/signoz)
83+
- [x] [OpenTelemetry Collector](https://www.traceloop.com/docs/openllmetry/integrations/otel-collector)
84+
85+
See [our docs](https://traceloop.com/docs/openllmetry/integrations/exporting) for instructions on connecting to each one.
86+
87+
## 🪗 What do we instrument?
88+
89+
OpenLLMetry can instrument everything that [OpenTelemetry already instruments](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation) - so things like your DB, API calls, and more. On top of that, we built a set of custom extensions that instrument things like your calls to OpenAI or Anthropic, or your Vector DB like Pinecone, Chroma, or Weaviate.
90+
91+
### LLM Providers
92+
93+
- [x] OpenAI / Azure OpenAI
94+
- [x] Anthropic
95+
- [x] Cohere
96+
- [ ] Replicate
97+
- [x] HuggingFace
98+
- [ ] Vertex AI (GCP)
99+
- [ ] Bedrock (AWS)
100+
101+
### Vector DBs
102+
103+
- [x] Pinecone
104+
- [x] Chroma
105+
- [ ] Weaviate
106+
- [ ] Milvus
107+
108+
### Frameworks
109+
110+
- [x] LangChain
111+
- [x] [Haystack](https://haystack.deepset.ai/integrations/traceloop)
112+
- [x] [LiteLLM](https://docs.litellm.ai/docs/observability/traceloop_integration)
113+
- [ ] LlamaIndex
114+
115+
## 🌱 Contributing
116+
117+
Whether it's big or small, we love contributions ❤️ Check out our guide to see how to [get started](https://traceloop.com/docs/openllmetry/contributing/overview).
118+
119+
Not sure where to get started? You can:
120+
121+
- [Book a free pairing session with one of our teammates](mailto:nir@traceloop.com?subject=Pairing%20session&body=I'd%20like%20to%20do%20a%20pairing%20session!)!
122+
- Join our <a href="https://join.slack.com/t/traceloopcommunity/shared_invite/zt-1plpfpm6r-zOHKI028VkpcWdobX65C~g">Slack</a>, and ask us any questions there.
123+
124+
## 💚 Community & Support
125+
126+
- [Slack](https://join.slack.com/t/traceloopcommunity/shared_invite/zt-1plpfpm6r-zOHKI028VkpcWdobX65C~g) (For live discussion with the community and the Traceloop team)
127+
- [GitHub Discussions](https://github.com/traceloop/openllmetry/discussions) (For help with building and deeper conversations about features)
128+
- [GitHub Issues](https://github.com/traceloop/openllmetry/issues) (For any bugs and errors you encounter using OpenLLMetry)
129+
- [Twitter](https://twitter.com/traceloopdev) (Get news fast)
130+
131+
## 🙏 Special Thanks
132+
133+
To @patrickdebois, who [suggested the great name](https://x.com/patrickdebois/status/1695518950715473991?s=46&t=zn2SOuJcSVq-Pe2Ysevzkg) we're now using for this repo!

‎sample-app/Gemfile‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# frozen_string_literal: true
2+
3+
source "https://rubygems.org"
4+
5+
gem "ruby-openai"
6+
gem "traceloop-sdk"

‎sample-app/Gemfile.lock‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
base64 (0.1.1)
5+
event_stream_parser (0.3.0)
6+
faraday (2.7.11)
7+
base64
8+
faraday-net_http (>= 2.0, < 3.1)
9+
ruby2_keywords (>= 0.0.4)
10+
faraday-multipart (1.0.4)
11+
multipart-post (~> 2)
12+
faraday-net_http (3.0.2)
13+
google-protobuf (3.25.0-arm64-darwin)
14+
googleapis-common-protos-types (1.9.0)
15+
google-protobuf (~> 3.18)
16+
multipart-post (2.3.0)
17+
opentelemetry-api (1.2.3)
18+
opentelemetry-common (0.20.0)
19+
opentelemetry-api (~> 1.0)
20+
opentelemetry-exporter-otlp (0.26.1)
21+
google-protobuf (~> 3.14)
22+
googleapis-common-protos-types (~> 1.3)
23+
opentelemetry-api (~> 1.1)
24+
opentelemetry-common (~> 0.20)
25+
opentelemetry-sdk (~> 1.2)
26+
opentelemetry-semantic_conventions
27+
opentelemetry-registry (0.3.0)
28+
opentelemetry-api (~> 1.1)
29+
opentelemetry-sdk (1.3.1)
30+
opentelemetry-api (~> 1.1)
31+
opentelemetry-common (~> 0.20)
32+
opentelemetry-registry (~> 0.2)
33+
opentelemetry-semantic_conventions
34+
opentelemetry-semantic_conventions (1.10.0)
35+
opentelemetry-api (~> 1.0)
36+
opentelemetry-semantic_conventions_ai (0.0.3)
37+
opentelemetry-api (~> 1.0)
38+
ruby-openai (5.2.0)
39+
event_stream_parser (>= 0.3.0, < 1.0.0)
40+
faraday (>= 1)
41+
faraday-multipart (>= 1)
42+
ruby2_keywords (0.0.5)
43+
traceloop-sdk (0.0.1)
44+
opentelemetry-exporter-otlp (~> 0.26.1)
45+
opentelemetry-sdk (~> 1.3.1)
46+
opentelemetry-semantic_conventions_ai (~> 0.0.3)
47+
48+
PLATFORMS
49+
arm64-darwin-23
50+
51+
DEPENDENCIES
52+
ruby-openai
53+
traceloop-sdk
54+
55+
BUNDLED WITH
56+
2.4.10

‎sample-app/openai.rb‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
require "openai"
2+
require "traceloop/sdk"
3+
4+
OpenAI.configure do |config|
5+
config.access_token = ENV.fetch("OPENAI_API_KEY")
6+
end
7+
8+
client = OpenAI::Client.new
9+
10+
traceloop = Traceloop::SDK::Traceloop.new
11+
12+
traceloop.llm_call() do |tracer|
13+
tracer.log_prompt(model="gpt-3.5-turbo", user_prompt="Tell me a joke about OpenTelemetry")
14+
response = client.chat(
15+
parameters: {
16+
model: "gpt-3.5-turbo",
17+
messages: [{ role: "user", content: "Tell me a joke about OpenTelemetry" }]
18+
})
19+
tracer.log_response(response)
20+
puts response.dig("choices", 0, "message", "content")
21+
end

‎semantic_conventions_ai/Gemfile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# frozen_string_literal: true
2+
3+
source "https://rubygems.org"
4+
5+
git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
6+
7+
# gem "rails"

0 commit comments

Comments
 (0)