ai-metrics

Inspect Codex / Claude logs, verify metering samples and re-count supported billing exports. Free local tools; bounded $10 diagnosis and $20 QA.

WakaTime plugin for GitHub Copilot CLI - Track your AI usage, lines of code generated, and time spent prompting Copilot AI

VerifyAI is a verification harness for testing, auditing, and evaluating responses from AI assistants and coding agents.

Honest metrics for AI tools and features. Agent skills for Claude Code, Codex, Cursor, Copilot, Gemini CLI and 30+ others: designs outcome metrics from your business context, audits dashboards against 20 named vanity-metric anti-patterns, generates instrumentation.

AI observability Grafana plugin tracking real-time LLM metrics — latency, cost, tokens, insights, and provider performance (Cerebras, LLAMA, MCP) with live 5s updates.

A simple Spring Boot application demonstrating AI chat client observability with token usage metrics, prompt logging, and performance monitoring.

Modern data exploration interface for analyzing LLM response patterns and performance metrics - Built with React, TypeScript, and TanStack Router

A posthog wrapper for laravel with support for: product analytics, Feature flags, Error tracking, LLM analytics & Events

Multi-model LLM benchmarking and evaluation toolkit

Agentic Workflow Evaluation: Text Summarization Agent. This project includes an AI agent evaluation workflow using a text summarization model with OpenAI API and Transformers library. It follows an iterative approach: generate summaries, analyze metrics, adjust parameters, and retest to refine AI agents for accuracy, readability, and performance.

为 AI 产品设计事件模型、转化漏斗、质量成本指标与数据看板。

Interactive dashboard with visualizations across AI development, geographic distribution, innovation, investment, and public perception.

Measure whether an AI deployment improves the operation it joined.

Open framework for measuring AI value across engineering, support, analytics, ops, and internal copilots.

Measurement method for AI Foundations source-line fidelity, drift, override, and non-merge testing.

Experimental framework for testing and measuring AI system capabilities, reasoning patterns, and emergent behaviors

📊 Ultra-fast lock-free token, latency, cost tracking and evaluation engine for LLMs and AI agents.

AI Security Metrics - Accuracy Monitoring and Data Drift Detection

Weekly Reading Group in AI measurement science