Lot's of analytics products, in my opinion, focus on the testing/evals part of building an LLM product. What Zond does is try to tell you how people are using your product.
At the moment the way it works is like a middleware: you swap out your LLM endpoint, then Zond starts capturing your LLM interactions and gives you:
- Clustering: so what are the key types of interactions people are having,
- Frustration tracking: how often are people saying things like "No, that's stupid" etc.
- Power user tracking: who's asking really complicated questions or trying to push your product.
- Confidence tracking: how often does the LLM generate responses that have low confidence, as measured by logprobs (only for OpenAI at the moment)
- Conversation completion: how often are people giving up
- Normal analytics stuff! An integration with Hubspot/Attio is on the roadmap.
Looking for beta users so if anyone is interested please let me know!