Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2a312b219f | ||
|
|
5630c7eda3 | ||
|
|
2f62d46a10 | ||
|
|
39b05f2a41 | ||
|
|
ea89f8e0ed | ||
|
|
6025a6a74e | ||
|
|
8f2270c6c0 | ||
|
|
cd43a1ad7a | ||
|
|
e6f6f2c909 | ||
|
|
62047676bd | ||
|
|
b9ecbc7feb |
Vendored
+4
-4
@@ -2,9 +2,9 @@
|
||||
title: "Link Check"
|
||||
task: ""
|
||||
lineage_type: import
|
||||
upstream_source: https://github.com/ARUNAGIRINATHAN-K/awesome-ai-agents-2026/blob/539ec1dc/.github/workflows/link-check.yml
|
||||
upstream_sha: 539ec1dc
|
||||
imported_at: 2026-06-26
|
||||
upstream_source: https://github.com/ARUNAGIRINATHAN-K/awesome-ai-agents-2026/blob/117f624b/.github/workflows/link-check.yml
|
||||
upstream_sha: 117f624b
|
||||
imported_at: 2026-07-25
|
||||
prompt_class: unknown
|
||||
upstream_changes: accepted
|
||||
author: upstream
|
||||
@@ -40,7 +40,7 @@ jobs:
|
||||
args: >-
|
||||
--verbose
|
||||
--no-progress
|
||||
--accept 200,204,301,302,403,429
|
||||
--accept 200,204,301,302,308,403,429
|
||||
--exclude-path node_modules
|
||||
--exclude "star-history.com"
|
||||
--exclude "shields.io"
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
title: "Awesome AI Agents 2026"
|
||||
task: ""
|
||||
lineage_type: import
|
||||
upstream_source: https://github.com/ARUNAGIRINATHAN-K/awesome-ai-agents-2026/blob/0aa994bf/README.md
|
||||
upstream_sha: 0aa994bf
|
||||
imported_at: 2026-07-15
|
||||
upstream_source: https://github.com/ARUNAGIRINATHAN-K/awesome-ai-agents-2026/blob/7d483c59/README.md
|
||||
upstream_sha: 7d483c59
|
||||
imported_at: 2026-08-13
|
||||
prompt_class: catalogue
|
||||
upstream_changes: accepted
|
||||
author: upstream
|
||||
@@ -68,6 +68,8 @@ validated: false
|
||||
- [Changelog](#changelog)
|
||||
- [Star History](#star-history)
|
||||
|
||||
 
|
||||
|
||||
## Orchestration Frameworks
|
||||
|
||||
- [Agency Swarm](https://github.com/VRSEN/agency-swarm) `🚀` `[Python]` `[Multi-Agent]` - Orchestrates multi-agent systems built on the OpenAI Assistants API with role-based collaboration.
|
||||
@@ -93,6 +95,7 @@ validated: false
|
||||
- [Modus](https://github.com/hypermodeinc/modus) `🔬` `[WebAssembly]` `[Serverless]` - Serverless framework for high-throughput agent workloads with minimal cold starts.
|
||||
- [Open-AutoGLM](https://github.com/zai-org/Open-AutoGLM) `🔬` `[Python]` `[Mobile]` - Open-source phone agent framework for building mobile device automation agents.
|
||||
- [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) `🚀` `[Python]` `[Multi-Agent]` - Lightweight multi-agent SDK with tracing and guardrails from OpenAI.
|
||||
- [OpenProgram](https://github.com/Fzkuji/OpenProgram) `🔬` `[Python]` `[Multi-Agent]` - Self-programming framework whose agents create, run, and refine workflows across models, tools, memory, and context.
|
||||
- [PraisonAI](https://github.com/MervinPraison/PraisonAI) `🚀` `[Python]` `[MCP]` - Production multi-agent framework with self-reflection, MCP integration, and workflow automation.
|
||||
- [PydanticAI](https://github.com/pydantic/pydantic-ai) `🌱` `[Python]` `[Pydantic]` - Type-safe agent framework from the Pydantic team with a FastAPI-style developer experience.
|
||||
- [Semantic Kernel](https://github.com/microsoft/semantic-kernel) `🚀` `[C#]` `[Microsoft]` - Microsoft enterprise SDK for Python, C#, and Java with modular plugins, memory, and goal planning.
|
||||
@@ -114,15 +117,19 @@ validated: false
|
||||
- [Cursor](https://cursor.com) `🚀` `[TypeScript]` `[IDE]` - AI-native IDE with parallel Cloud Agents and 100K+ enterprise users.
|
||||
- [Devika](https://github.com/stitionai/devika) `🌱` `[Python]` `[Docker]` - Open-source AI software engineer that understands high-level instructions and writes full-featured code autonomously.
|
||||
- [Devin](https://devin.ai) `🚀` `[Cloud]` `[GitHub]` - Fully autonomous AI software engineer showing 8-12x efficiency gains in production (Nubank case study), cloud sandbox with persistent state, GitHub integration, and enterprise adoption growing 40% MoM.
|
||||
- [fractal](https://github.com/plasma-ai/fractal) `🌱` `[Python]` `[Multi-Agent]` - Orchestrates Claude Code, Codex, and other agents in per-node Git worktrees with recursive delegation and a live TUI.
|
||||
- [gemini-cli](https://github.com/google-gemini/gemini-cli) `🌱` `[Go]` `[CLI]` - Open-source CLI agent bringing Gemini AI directly into your terminal.
|
||||
- [GitHub Copilot](https://github.com/features/copilot) `🌱` `[TypeScript]` `[VS Code]` - Agent Mode in VS Code with Copilot Workspace for issue-to-PR workflows across Claude, GPT, and Gemini.
|
||||
- [Goose](https://github.com/aaif-goose/goose) `🌱` `[Rust]` `[CLI]` - Extensible open-source CLI agent that installs, executes, edits, and tests with any LLM.
|
||||
- [JetBrains AI](https://www.jetbrains.com/ai/) `🌱` `[Kotlin]` `[JetBrains]` - Deep AI integration across all JetBrains IDEs with context-aware completions and refactoring.
|
||||
- [Juggler](https://github.com/juggler-ai/juggler) `🌱` `[Desktop]` `[Local]` - Multi-client desktop/remote GUI agent with inspectable tool calls, branching-thread editable context, and plugin extensibility.
|
||||
- [Kiro](https://kiro.dev) `🚀` `[Cloud]` `[IDE]` - Spec-driven development agent that writes specs, auto-generates tasks, implements code, and automates DevOps workflows.
|
||||
- [Open Interpreter](https://github.com/openinterpreter/open-interpreter) `🌱` `[Python]` `[CLI]` - Execute code locally via natural-language model instructions with a ChatGPT-like interface.
|
||||
- [Open Interpreter](https://github.com/openinterpreter/openinterpreter) `🌱` `[Python]` `[CLI]` - Execute code locally via natural-language model instructions with a ChatGPT-like interface.
|
||||
- [opencode](https://github.com/anomalyco/opencode) `🌱` `[TypeScript]` `[Desktop]` - Open-source coding agent available as a desktop app with a visual interface.
|
||||
- [OpenHands](https://github.com/OpenHands/OpenHands) `🌱` `[Python]` `[Docker]` - AI-driven development platform that writes, tests, and deploys code autonomously.
|
||||
- [Ouroboros](https://github.com/Q00/ouroboros) `🌱` `[Python]` `[MCP]` - Pins an acceptance spec before the run and verifies the result, hiding grading commands from the executing agent.
|
||||
- [PR-Agent](https://github.com/The-PR-Agent/pr-agent) `🚀` `[Python]` `[GitHub]` - Open-source AI PR reviewer that auto-describes, reviews, and improves pull requests.
|
||||
- [Prime Agent](https://github.com/PrimeIntellect-ai/prime-agent) `🚀` `[TypeScript]` `[CLI]` - Open-source RLM coding and research agent designed for long-running autonomous tasks.
|
||||
- [Qodo](https://www.qodo.ai) `🚀` `[Cloud]` `[Security]` - AI code review platform with context-aware PR validation and security analysis.
|
||||
- [RooCode](https://github.com/RooCodeInc/Roo-Code) `🌱` `[TypeScript]` `[VS Code]` - Cline fork with structured modes and reduced hallucinations for more reliable code generation.
|
||||
- [Snyk Code](https://snyk.io/product/snyk-code/) `🌱` `[Cloud]` `[Security]` - AI-powered security scanner with real-time vulnerability detection in agent-generated code.
|
||||
@@ -145,12 +152,13 @@ validated: false
|
||||
- [LanceDB](https://github.com/lancedb/lancedb) `🌱` `[Rust]` `[Vector DB]` - Serverless vector search database embedded directly in the agent process with no infrastructure needed.
|
||||
- [Langmem](https://github.com/langchain-ai/langmem) `🌱` `[Python]` `[LangChain]` - Helps agents learn and adapt from their interactions over time with persistent memory.
|
||||
- [Mem0](https://github.com/mem0ai/mem0) `🌱` `[Python]` `[Vector DB]` - Memory layer for AI applications with long-term, short-term, and semantic memory extraction.
|
||||
- [Remio](https://remio.ai/) `🔬` `[Desktop]` `[Memory]` - Local-first AI memory desktop app that parses files into searchable vector indexes.
|
||||
- [Remio](https://www.remio.ai/) `🔬` `[Desktop]` `[Memory]` - Local-first AI memory desktop app that parses files into searchable vector indexes.
|
||||
- [Memoir](https://github.com/zhangfengcdt/memoir) `🔬` `[Python]` `[Memory]` - Git-like versioned semantic memory for AI agents with branching and commits.
|
||||
- [Memvid](https://github.com/memvid/memvid) `🌱` `[Python]` `[RAG]` - Replace complex RAG pipelines with a serverless, single-file memory layer for instant retrieval.
|
||||
- [Milvus](https://github.com/milvus-io/milvus) `🌱` `[Go]` `[Vector DB]` - Scales vector search to billions of embeddings for large-scale agent knowledge bases.
|
||||
- [Mori (森)](https://github.com/fjwood69/mori) `🌱` `[Python]` `[MCP]` - Sovereign shared memory layer for AI coding agents with zero-instrumentation capture via lifecycle hooks, a dream pipeline that distills sessions into curated governed memories, and support for Claude Code, Cursor, Codex, and Antigravity.
|
||||
- [Motorhead](https://github.com/getmetal/motorhead) `🌱` `[Rust]` `[Multi-Agent]` - Manages conversation context windows for agents with automatic background summarization.
|
||||
- [Open Index](https://github.com/DrDroidLab/open-index) `🔬` `[Python]` `[MCP]` - Builds typed knowledge graphs with hybrid search and read/write MCP tools for domain-specific agents.
|
||||
- [Pathway](https://github.com/pathwaycom/pathway) `🌱` `[Python]` `[RAG]` - Live data RAG engine with real-time streaming for agents that need up-to-the-second knowledge.
|
||||
- [Pinecone](https://www.pinecone.io) `🚀` `[Cloud]` `[Vector DB]` - Managed vector database with agent namespaces for multi-tenant isolation, hybrid search (vector + keyword), serverless auto-scaling, and $11B valuation.
|
||||
- [Qdrant](https://github.com/qdrant/qdrant) `🌱` `[Rust]` `[Vector DB]` - High-performance vector similarity search engine with rich payload filtering for agent memory.
|
||||
@@ -200,7 +208,7 @@ The protocol layer that enables agents to discover tools, communicate with each
|
||||
- [mcp-nest](https://github.com/CharanBharathula/mcp-nest) `🌱` `[Python]` `[MCP]` - Unified Model Context Protocol (MCP) server for executing code and managing files.
|
||||
- [NotFair](https://notfair.co) `🚀` `[Cloud]` `[MCP]` - Hosted Google Ads MCP server for diagnosing, optimizing, and executing campaign changes via the Google Ads API with a human-approval gate.
|
||||
- [NotFair Skills](https://github.com/nowork-studio/NotFair) `🚀` `[TypeScript]` `[MCP]` - Open-source Claude Code skills for SEO, GEO, Google Ads, and Meta Ads, connecting to live data through Google Ads MCP, Meta Ads MCP, Google Search Console MCP, and Google Analytics (GA4) MCP.
|
||||
- [Toolhouse](https://toolhouse.ai) `🌱` `[Python]` `[Multi-Agent]` - Cloud-hosted tool infrastructure for agents with optimized execution and low-latency access.
|
||||
- [Toolhouse](https://toolhouse.ai/en/) `🌱` `[Python]` `[Multi-Agent]` - Cloud-hosted tool infrastructure for agents with optimized execution and low-latency access.
|
||||
- [Zapier MCP Server](https://zapier.com/mcp) `🌱` `[Cloud]` `[MCP]` - Connect agents to 7,000+ app integrations via MCP, powered by Zapier's automation platform.
|
||||
- [zero-api-key-web-search](https://github.com/wd041216-bit/zero-api-key-web-search) `🌱` `[Python]` `[MCP]` - Free web search toolkit for AI agents with no API keys, MCP server support.
|
||||
|
||||
@@ -230,9 +238,8 @@ Sandboxes, web scrapers, browser automation, and networking layers that agents d
|
||||
|
||||
- [Agent Bounties](https://github.com/NSPG13/agent-bounties) `🔬` `[Rust]` `[MCP]` - Coordinates verifiable digital bounty workflows designed for agents to post, fund, claim, solve, verify, and earn.
|
||||
- [AgentDock](https://github.com/agentdock/agentdock) `🚀` `[Python]` `[Docker]` - Framework for building and deploying production-ready AI agents with composable node architecture.
|
||||
- [Agent Starter](https://github.com/raintree-technology/agent-starter) `🌱` `[TypeScript]` `[MCP]` - Project-local config manager that syncs one `agent.json` manifest into Claude Code, Codex, Cursor, and MCP setup while preserving manual edits and detecting drift.
|
||||
- [AgentServices](https://agentservices.to) `🚀` `[Python]` `[x402]` - Paid data APIs for AI agents with 54 services, 37 MCP tools, and x402 nanopayments on Base. Market data, onchain analytics, AI inference, and research.
|
||||
- [codex-profiles](https://github.com/Ducksss/codex-profiles) `🚀` `[Python]` `[OpenAI]` - Bash CLI for switching OpenAI Codex CLI and Desktop profiles with isolated CODEX_HOME directories.
|
||||
- [CompozyOS](https://github.com/compozy/compozy) `🚀` `[Go]` `[Multi-Agent]` - Runs agent CLIs as a team on loops and schedules, with shared memory, permissions and approvals in one self-hosted daemon.
|
||||
- [Crawl4AI](https://github.com/unclecode/crawl4ai) `🌱` `[Python]` `[Multi-Agent]` - Extracts structured data from web pages using LLM-friendly output formats optimized for agent ingestion.
|
||||
- [Docling](https://github.com/docling-project/docling) `🌱` `[Python]` `[IDE]` - Parses PDFs, DOCX, and slides into structured text with deep layout understanding for document agents.
|
||||
- [E2B](https://github.com/e2b-dev/e2b) `🌱` `[TypeScript]` `[Multi-Agent]` - Cloud sandboxes for AI agents to run code securely in isolated environments.
|
||||
@@ -242,7 +249,7 @@ Sandboxes, web scrapers, browser automation, and networking layers that agents d
|
||||
- [LlamaParse](https://github.com/run-llama/llama_cloud_services) `🌱` `[Python]` `[RAG]` - GenAI-native document parser designed to extract complex tables and layouts for RAG pipelines.
|
||||
- [Marker](https://github.com/datalab-to/marker) `🌱` `[Python]` `[CLI]` - Converts PDF documents to markdown with high accuracy for tables, equations, and figures.
|
||||
- [Notte](https://github.com/nottelabs/notte) `🚀` `[Python]` `[Pipeline]` - Browser automation engine optimized for production AI pipelines.
|
||||
- [Pilot Protocol](https://github.com/TeoSlayer/pilotprotocol) `🌱` `[Go]` `[Multi-Agent]` - Networking stack for distributed agent systems with encrypted tunnels.
|
||||
- [Pilot Protocol](https://github.com/pilot-protocol/pilotprotocol) `🌱` `[Go]` `[Multi-Agent]` - Networking stack for distributed agent systems with encrypted tunnels.
|
||||
- [Playwright](https://github.com/microsoft/playwright) `🌱` `[TypeScript]` `[Testing]` - Automates Chromium, Firefox, and WebKit browsers with a single cross-language API for agent-driven testing.
|
||||
- [ScrapeGraphAI](https://github.com/ScrapeGraphAI/Scrapegraph-ai) `🌱` `[Python]` `[LangChain]` - Python web-scraping library that uses LLMs to build intelligent scraping pipelines from natural-language instructions.
|
||||
- [Surya](https://github.com/datalab-to/surya) `🌱` `[Python]` `[CLI]` - Runs OCR and layout detection on documents in 90+ languages for multilingual document agents.
|
||||
@@ -259,7 +266,8 @@ Sandboxes, web scrapers, browser automation, and networking layers that agents d
|
||||
- [Flowise](https://github.com/FlowiseAI/Flowise) `🌱` `[TypeScript]` `[RAG]` - Open-source drag-and-drop UI to build customized LLM flows, RAG pipelines, and agent systems.
|
||||
- [Langflow](https://github.com/langflow-ai/langflow) `🌱` `[Python]` `[RAG]` - Visual drag-and-drop builder for LLM workflows, RAG agents, and multi-step pipelines.
|
||||
- [Lindy](https://www.lindy.ai) `🌱` `[Cloud]` `[No-Code]` - No-code AI agent platform with 3000+ app integrations for business workflow automation.
|
||||
- [LLMGraph](https://llmgraph.ai) `🔬` `[Cloud]` `[RAG]` - Visual canvas for building RAG chatbots and AI agents with one-click deploy to REST API or chat widget.
|
||||
- [LlamaIndex Workflows](https://developers.llamaindex.ai/python/llamaagents/workflows/) `🌱` `[Python]` `[RAG]` - Event-driven orchestration framework for building complex agentic systems.
|
||||
- [LLMGraph](https://llmgraph.ai/) `🔬` `[Cloud]` `[RAG]` - Visual canvas for building RAG chatbots and AI agents with one-click deploy to REST API or chat widget.
|
||||
- [Make](https://www.make.com/en) `🌱` `[Cloud]` `[RAG]` - Visual workflow automation platform with AI capabilities and drag-and-drop scenario builder.
|
||||
- [n8n](https://github.com/n8n-io/n8n) `🌱` `[TypeScript]` `[Docker]` - Open-source workflow automation with AI agent nodes combining visual and code-based orchestration.
|
||||
- [Relevance AI](https://relevanceai.com) `🌱` `[Cloud]` `[No-Code]` - No-code AI agent builder for sales, support, and research use cases with team collaboration.
|
||||
@@ -274,7 +282,9 @@ Sandboxes, web scrapers, browser automation, and networking layers that agents d
|
||||
- [LiveKit Agents](https://github.com/livekit/agents) `🌱` `[Python]` `[IDE]` - Framework for building real-time, multimodal AI agents with voice, video, and data channels.
|
||||
- [Nuance AI](https://dragon.nuance.com/en-us/home) `🚀` `[Cloud]` `[CLI]` - Enterprise speech and conversational AI platform for clinical and contact-center workflows with HIPAA-capable deployments.
|
||||
- [Google Cloud Speech-to-Text v2](https://cloud.google.com/speech-to-text) `🚀` `[Cloud]` `[Pipeline]` - Google Cloud streaming and batch speech recognition API v2 with improved accuracy, streaming, and noise suppression for real-time agent pipelines.
|
||||
- [Personal Jarvis](https://github.com/PersonalJarvis/PersonalJarvis) `🔬` `[Python]` `[Voice]` - Voice-driven desktop assistant that takes mouse and keyboard and delegates heavy tasks to agent harnesses like Claude Code, Codex, and MCP.
|
||||
- [Pipecat](https://github.com/pipecat-ai/pipecat) `🚀` `[Python]` `[Streaming]` - Production-grade voice AI framework with sub-250ms latency, WebRTC support, multimodal (voice+vision+text), real-time streaming, and 70+ language support.
|
||||
- [qwen-audio-agent](https://github.com/QwenAudio/qwen-audio-agent) `🌱` `[Desktop]` `[Voice]` - Full-duplex voice runtime that drives coding agents (Claude Code, Codex, OpenCode, Kimi Code, and more) over ACP, keeping conversations going while background tasks run, with barge-in and a local wake word.
|
||||
- [Rasa](https://github.com/RasaHQ/rasa) `🌱` `[Python]` `[Self-Hosted]` - Open-source conversational AI framework with self-hosted NLU training and dialogue management.
|
||||
- [simulate-sdk](https://github.com/future-agi/simulate-sdk) `🌱` `[Python]` `[Voice]` - Persona- and scenario-driven SDK for simulating voice and text AI agents.
|
||||
- [Vapi](https://github.com/VapiAI/server-sdk-python) `🌱` `[Python]` `[Voice]` - Platform for building voice AI agents with low-latency speech-to-speech capabilities.
|
||||
@@ -293,6 +303,7 @@ Sandboxes, web scrapers, browser automation, and networking layers that agents d
|
||||
- [Braintrust](https://www.braintrust.dev) `🌱` `[TypeScript]` `[Evaluation]` - Eval-driven development platform with experiment tracking and prompt optimization for agent quality.
|
||||
- [ElevenAgents](https://elevenlabs.io/agents) `🚀` `[Cloud]` `[Voice]` - Voice agent platform from ElevenLabs for customer support automation with HIPAA compliance and multi-language support.
|
||||
- [DriftGuard](https://github.com/sujal-maheshwari2004/DriftGuard) `🌱` `[Python]` `[Multi-Agent]` - Semantic memory guardrails using causal graphs to prevent agents from repeating past failures.
|
||||
- [Galley](https://github.com/shinpr/galley) `🔬` `[Go]` `[Multi-Agent]` - Pairs independently configured executors and supervisors with repository-defined quality gates and inspectable evidence for each coding attempt.
|
||||
- [Guardrails AI](https://github.com/guardrails-ai/guardrails) `🌱` `[Python]` `[Multi-Agent]` - Adds structural, type, and quality guarantees to LLM outputs for reliable agent responses.
|
||||
- [Helicone](https://github.com/Helicone/helicone) `🌱` `[TypeScript]` `[Observability]` - Open-source LLM observability with one-line integration for cost tracking and prompt analytics.
|
||||
- [Lakera Guard](https://www.lakera.ai) `🌱` `[Cloud]` `[Security]` - Real-time protection against prompt injection, data leakage, and toxicity in agent interactions.
|
||||
@@ -301,12 +312,15 @@ Sandboxes, web scrapers, browser automation, and networking layers that agents d
|
||||
- [LangSmith](https://smith.langchain.com) `🚀` `[Cloud]` `[LangChain]` - LangChain platform for tracing, testing, and evaluating agent performance with production monitoring.
|
||||
- [LLM Guard](https://github.com/protectai/llm-guard) `🌱` `[Python]` `[Security]` - Security toolkit for scanning LLM inputs and outputs to prevent prompt injection and data leaks.
|
||||
- [Logfire](https://github.com/pydantic/logfire) `🌱` `[Python]` `[Pydantic]` - Python-native observability from the Pydantic team with deep integration for high-performance agent monitoring.
|
||||
- [MartinLoop](https://github.com/Keesan12/martin-loop) `🔬` `[TypeScript]` `[Compliance]` - Governs autonomous coding agents, turning open-ended runs into budgeted, verified software work with signed outcome receipts.
|
||||
- [NeMo Guardrails](https://github.com/NVIDIA-NeMo/Guardrails) `🌱` `[Python]` `[Multi-Agent]` - NVIDIA programmable guardrails toolkit for controlling and securing LLM-powered agent conversations.
|
||||
- [Orchard Kit](https://github.com/OrchardHarmonics/orchard-kit) `🌱` `[Python]` `[Security]` - Modules for agent runtime security, self-audit trails, and collective cognition patterns.
|
||||
- [OWASP Top 10 for Agentic Apps](https://owasp.org/www-project-top-10-for-large-language-model-applications/) `🌱` `[Python]` `[Security]` - Security framework covering goal hijacking, tool misuse, and cascading failure mitigations for agents.
|
||||
- [Pluribus](https://github.com/caioribeiroclw-pixel/pluribus) `🔬` `[TypeScript]` `[Observability]` - Generates cross-tool agent context and privacy-safe evidence receipts for loaded authority, handoffs, and skill use.
|
||||
- [Rebuff](https://github.com/protectai/rebuff) `🌱` `[Python]` `[Security]` - Self-hardening prompt injection detection system for securing agent inputs against adversarial attacks.
|
||||
- [ai-evaluation](https://github.com/future-agi/ai-evaluation) `🌱` `[Python]` `[Evaluation]` - LLM evaluation framework with 50+ metrics, LLM-as-Judge, and guardrail scanners (jailbreak, PII, injection).
|
||||
- [Agent Learning Kit](https://github.com/future-agi/agent-learning-kit) `🌱` `[Python]` `[Evaluation]` - LLM evaluation framework with 50+ metrics, LLM-as-Judge, and guardrail scanners (jailbreak, PII, injection).
|
||||
- [Shipmoor](https://shipmoor.dev) `🔬` `[Python]` `[Testing]` - Local, deterministic verification layer for AI agent code: scans, test evidence, and a binding merge verdict without uploading source.
|
||||
- [SourceryKit](https://github.com/ProvablyAI/sourcerykit) `🔬` `[Python]` `[Security]` - Verifies an agent's outbound requests and MCP handoffs against a source of truth using zero-knowledge proofs, logging each call and blocking anything off the trusted-endpoint allow-list.
|
||||
- [Future AGI](https://github.com/future-agi/future-agi) `🌱` `[Python]` `[Self-Hosted]` - Self-hostable end-to-end agent engineering platform with tracing, evals, guardrails, and gateway.
|
||||
|
||||
## Agent Interfaces and UIs
|
||||
@@ -315,12 +329,15 @@ Frontend workspaces and chat interfaces with built-in agent plugins and tool-use
|
||||
|
||||
- [AionUi](https://github.com/iOfficeAI/AionUi) `🚀` `[Desktop]` `[Multi-Agent]` - Connects 20+ AI CLIs and any API key in a local, open-source multi-agent desktop workspace.
|
||||
- [AnythingLLM](https://github.com/Mintplex-Labs/anything-llm) `🌱` `[TypeScript]` `[RAG]` - All-in-one AI application with RAG, agents, and multi-model support for desktop and Docker.
|
||||
- [Better Agent](https://github.com/ofekron/better-agent) `🔬` `[Desktop]` `[Multi-Agent]` - Manages Claude, Codex, and Gemini coding-agent sessions with parallel forks, delegation, persistence, and restart recovery.
|
||||
- [DB-GPT](https://github.com/eosphoros-ai/DB-GPT) `🌱` `[Python]` `[Database]` - Data interaction platform with local LLM support for 100% private database and analytics agents.
|
||||
- [LibreChat](https://github.com/danny-avila/LibreChat) `🌱` `[TypeScript]` `[IDE]` - Self-hosted multi-model chat interface supporting all major AI providers with access control.
|
||||
- [LobeHub](https://lobehub.com/) `🌱` `[TypeScript]` `[Multi-Agent]` - Modern platform for hybrid work and AI-driven collaboration with extensible agent teams and rapid integration.
|
||||
- [LobeChat](https://github.com/lobehub/lobehub) `🌱` `[TypeScript]` `[Multi-Agent]` - Modern, open-source AI chat framework with a massive plugin ecosystem for autonomous agent capabilities.
|
||||
- [OpenHuman](https://github.com/tinyhumansai/openhuman) `🚀` `[Rust]` `[Memory]` - Self-hosted local-first personal AI assistant with a Rust core, desktop apps, knowledge-graph memory, skills, voice, and multi-channel messaging.
|
||||
- [Orkas](https://github.com/Orkas-AI/Orkas) `🔬` `[Desktop]` `[Multi-Agent]` - Runs parallel AI agents in a local-first desktop workspace with shared files, BYOK providers, and optional sync.
|
||||
- [OpenWebUI](https://github.com/open-webui/open-webui) `🌱` `[TypeScript]` `[RAG]` - Extensible local AI interface with built-in RAG, tool use, and support for multi-agent workflows.
|
||||
- [lucinate](https://github.com/lucinate-ai/lucinate) `🌱` `[Go]` `[TUI]` - Multi-backend terminal AI chat client for OpenClaw, Hermes, Ollama, and OpenAI-compatible APIs with routines, multi-agent switching, and local agent skills.
|
||||
|
||||
## Agent Deployment and Hosting
|
||||
|
||||
@@ -338,6 +355,7 @@ Frontend workspaces and chat interfaces with built-in agent plugins and tool-use
|
||||
|
||||
- [AgentBench](https://github.com/THUDM/AgentBench) `🌱` `[Python]` `[Benchmark]` - Comprehensive benchmark for evaluating LLMs as agents across 8 distinct environments.
|
||||
- [ARC-AGI-2](https://arcprize.org) `🌱` `[Python]` `[Benchmark]` - Frontier benchmark for measuring general intelligence capabilities in AI agents beyond pattern matching.
|
||||
- [ClawBench](https://github.com/TIGER-AI-Lab/ClawBench) `🔬` `[Python]` `[Benchmark]` - Evaluates web agents on 283 real-world tasks across 163 live websites with interception and trace-based scoring.
|
||||
- [GAIA Benchmark](https://huggingface.co/papers/2311.12983) `🌱` `[Python]` `[Benchmark]` - Benchmark for General AI Assistants measuring real-world reasoning and tool use.
|
||||
- [Inspect AI](https://github.com/UKGovernmentBEIS/inspect_ai) `🌱` `[Python]` `[Evaluation]` - Framework for evaluating large language models with composable tasks and scoring.
|
||||
- [SWE-bench](https://github.com/SWE-bench/SWE-bench) `🚀` `[Python]` `[GitHub]` - Benchmark for evaluating LLMs on real-world software engineering tasks from GitHub issues.
|
||||
@@ -370,6 +388,8 @@ Curated list of vertical agent solutions for finance, healthcare, legal, manufac
|
||||
- [JPMorgan AlphaBlue](https://www.jpmorgan.com/global) `🚀` `[Cloud]` `[Multi-Agent]` - Trading and market analysis agent integrations for institutional workflows.
|
||||
- [Goldman Sachs Marquee AI](https://www.goldmansachs.com) `🚀` `[Cloud]` `[Multi-Agent]` - Market data and analytics agents built on Marquee platform.
|
||||
- [Morgan Stanley AdvisorBot](https://www.morganstanley.com) `🚀` `[Cloud]` `[CLI]` - Financial advisory assistant for advisors and retail clients.
|
||||
- [AgentPump](https://agentpump.app) `🔬` `[Cloud]` `[CLI]` - Runs autonomous on-chain memecoin trading agents on Solana that trade on a schedule, operable from the terminal via the @agentpump/cli.
|
||||
|
||||
|
||||
### Healthcare
|
||||
|
||||
@@ -391,6 +411,7 @@ Curated list of vertical agent solutions for finance, healthcare, legal, manufac
|
||||
### Government & Compliance
|
||||
|
||||
- Anthropic Government Agents - Policy analysis and public sector agents for regulated workflows (🏷️ `Cloud` `Government` `Enterprise`).
|
||||
- [Leyna](https://www.atom.com/name/Leyna.ai) `🌱` `[Cloud]` `[Government]` - Public records request agent for government agencies to parse, redact, and track requests.
|
||||
- Tax & Insurance Agent Platforms - Generic category placeholder for compliance-focused tax and underwriting agents (🏷️ `Cloud` `Compliance` `Enterprise`).
|
||||
|
||||
## Learning Resources
|
||||
@@ -411,6 +432,7 @@ Curated list of vertical agent solutions for finance, healthcare, legal, manufac
|
||||
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) `🌱` `[Python]` `[Paper]` - The foundational paper behind the ReAct prompting pattern used in most agent frameworks.
|
||||
- [Reflexion](https://github.com/noahshinn/reflexion) `🌱` `[Python]` `[Paper]` - Research framework letting agents learn from past mistakes via iterative verbal self-reflection loops.
|
||||
- [Tree of Thoughts](https://github.com/princeton-nlp/tree-of-thought-llm) `🌱` `[Python]` `[Paper]` - Explores multiple parallel reasoning paths before committing to a final answer for complex problems.
|
||||
- [What's Missing in Autonomous Research?](https://haizhaoyang.github.io/research/autoresearch-survey.html) `🔬` `[Python]` `[Research]` - Systematizes 56 autonomous research systems across seven axes, showing most can generate but few can defend research artifacts.
|
||||
|
||||
## Agent Communication
|
||||
|
||||
@@ -520,7 +542,7 @@ Tools for generating images, video, music, audio, and 3D assets using AI models.
|
||||
- [LTX Video](https://github.com/Lightricks/LTX-Video) `🌱` `[Python]` `[IDE]` - Commercially safe open-source video generation model trained on fully licensed data.
|
||||
- [Luma Dream Machine](https://lumalabs.ai/app) `🌱` `[Cloud]` `[IDE]` - Creates physics-accurate 4K HDR video scenes with 3D-style output for cinematic quality.
|
||||
- [Pika 2.5](https://pika.art/login) `🌱` `[Cloud]` `[IDE]` - Beginner-friendly video generation tool with fast turnaround and intuitive editing controls.
|
||||
- [Runway Gen-4.5](https://runwayml.com) `🌱` `[Cloud]` `[IDE]` - Professional video generation with Motion Brush and the highest benchmark scores for visual quality.
|
||||
- [Runway Gen-4.5](https://runway.com) `🌱` `[Cloud]` `[IDE]` - Professional video generation with Motion Brush and the highest benchmark scores for visual quality.
|
||||
- [Seedance 2.0](https://seed.bytedance.com/en/seedance) `🌱` `[Cloud]` `[IDE]` - Quad-modal input video generator with built-in lip-sync and 2K resolution output.
|
||||
- [Sora 2](https://sora.com) `🚀` `[Cloud]` `[OpenAI]` - Generates 25-second cinematic video clips with strong narrative coherence from text prompts by OpenAI.
|
||||
- [Wan 2.1](https://github.com/Wan-Video/Wan2.1) `🌱` `[Python]` `[IDE]` - Open-source self-hosted video generation model with unlimited length and no per-video costs.
|
||||
@@ -554,6 +576,7 @@ AI agents that automate customer support, CRM workflows, sales outreach, and tic
|
||||
|
||||
- [Ada](https://www.ada.cx) `🚀` `[Cloud]` `[Multi-Agent]` - Resolves 60% of customer support tickets automatically with complex multi-turn query understanding.
|
||||
- [Assembled](https://www.assembled.com) `🚀` `[Cloud]` `[Multi-Agent]` - Routes support tickets with workforce-aware scheduling and intelligent handoff to human agents.
|
||||
- [ChatBotKit](https://chatbotkit.com) `🌱` `[Cloud]` `[RAG]` - Deploys AI agents that answer customer questions from business data and hand conversations to humans.
|
||||
- [Dixa](https://www.dixa.com) `🚀` `[Cloud]` `[Multi-Agent]` - CRM-first conversational support platform with AI-powered routing and customer context enrichment.
|
||||
- [Freshdesk Freddy AI](https://www.freshworks.com/freshdesk/omni/freddy-ai-automation/) `🌱` `[Cloud]` `[Multi-Agent]` - Auto-triages and routes support tickets with smart AI suggestions for budget-conscious SMB teams.
|
||||
- [Intercom Fin](https://fin.ai) `🚀` `[Cloud]` `[Multi-Agent]` - Resolves 50% of SaaS support tickets by learning directly from your help center and knowledge base.
|
||||
@@ -578,7 +601,8 @@ AI agents that automate customer support, CRM workflows, sales outreach, and tic
|
||||
- [BuyWhere](https://buywhere.ai) `🌱` `[Cloud]` `[MCP]` - AI shopping agent for real-time product search and price comparison across SG/SEA/US markets using an MCP-native server with 8 tools.
|
||||
- [Clay](https://www.clay.com) `🌱` `[Cloud]` `[IDE]` - Enriches leads from 70+ data providers and generates hyper-personalized outreach at scale.
|
||||
- [Instantly](https://instantly.ai) `🌱` `[Cloud]` `[Multi-Agent]` - Generates AI cold emails with smart sender rotation and built-in domain warmup for deliverability.
|
||||
- [Lavender](https://www.lavender.ai) `🌱` `[Cloud]` `[Multi-Agent]` - Coaches email writing in real-time with AI response scoring and recipient intelligence.
|
||||
- [Lavender](https://lavender.ai/) `🌱` `[Cloud]` `[Multi-Agent]` - Coaches email writing in real-time with AI response scoring and recipient intelligence.
|
||||
- [OutreachAgent](https://outreachagent.dev/for-agents) `🔬` `[Cloud]` `[Event-Driven]` - Runs reply-aware outbound email workflows with webhooks, sender pacing, approvals, and deliverability guardrails.
|
||||
- [Overloop CLI](https://overloop.com) `🌱` `[Cloud]` `[CLI]` - AI outbound CLI agent that sources 450M+ contacts and runs email plus LinkedIn campaigns with JSON output.
|
||||
|
||||
## Voice Agent Platforms
|
||||
@@ -603,6 +627,7 @@ Platforms for building, deploying, and scaling voice-based AI agents across call
|
||||
- [Synthesia](https://www.synthesia.io) `🌱` `[Cloud]` `[IDE]` - Generates AI video avatars that speak in 120+ languages for training and communication agents.
|
||||
- [Synthflow](https://synthflow.ai) `🌱` `[Cloud]` `[No-Code]` - No-code voice agent builder with pre-built templates for SMBs to deploy phone agents quickly.
|
||||
- [Voiceflow](https://www.voiceflow.com) `🌱` `[Cloud]` `[No-Code]` - No-code builder for voice and chat agents with visual conversation design and team collaboration.
|
||||
- [Workforce Wave](https://www.workforcewave.com/) `🔬` `[Cloud]` `[Voice]` - AI voice receptionist for SMBs handling 24/7 call answering, appointment booking, and lead capture.
|
||||
|
||||
## Deep Research Agents
|
||||
|
||||
@@ -752,6 +777,7 @@ Key data points on the AI agent market covering adoption, growth, valuations, an
|
||||
|
||||
Tools for running LLMs locally and self-hosting AI agent platforms with full privacy and control.
|
||||
|
||||
- [Avelina](https://avelina.ai) `🌱` `[TypeScript]` `[Telegram]` - Self-hosted personal AI assistant on your own VPS that lives in Telegram, with persistent long-term memory, voice, and Claude-powered reasoning.
|
||||
- [GPT4All](https://github.com/nomic-ai/gpt4all) `🌱` `[C++]` `[Local]` - Open-source local chat application that runs LLMs on consumer hardware without cloud dependencies.
|
||||
- [Jan](https://github.com/janhq/jan) `🌱` `[TypeScript]` `[Local]` - Open-source ChatGPT alternative that runs 100% offline with local model management and privacy guarantees.
|
||||
- [Llamafile](https://github.com/mozilla-ai/llamafile) `🌱` `[C++]` `[Local]` - Distributes LLMs as single executable files with zero setup required from Mozilla.
|
||||
@@ -810,9 +836,9 @@ Curated newsletters, podcasts, and communities for staying current with AI agent
|
||||
- [r/ClaudeAI](https://www.reddit.com/r/ClaudeAI/) `🌱` `[Python]` `[Multi-Agent]` - Reddit community for Claude users sharing agent workflows, prompts, and integration patterns.
|
||||
- [r/LangChain](https://www.reddit.com/r/LangChain/) `🌱` `[Python]` `[LangChain]` - Reddit community for agent developers using LangChain, LangGraph, and related frameworks.
|
||||
- [r/LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/) `🌱` `[Cloud]` `[IDE]` - Reddit community for self-hosted LLM users sharing local deployment and agent setup guides.
|
||||
- [The Agents Index](https://theagentsindex.com) `🔬` `[Cloud]` `[Multi-Agent]` - Compares AI agent tools with sourced pricing, verdicts, and pros/cons in a researched, quality-gated directory.
|
||||
- [The Rundown AI](https://www.therundown.ai) `🌱` `[Python]` `[RAG]` - Daily AI digest reaching 600K+ subscribers with concise coverage of agent news and launches.
|
||||
|
||||
- [Agents Launchpad](https://launchpad.smartbizcalc.com) `🌱` `[Python]` `[Multi-Agent]` - Community-curated directory of indie AI agents and tools.
|
||||
## Changelog
|
||||
|
||||
See [CHANGELOG.md](CHANGELOG.md) for the full update history.
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
title: "Awesome AI Agent Papers"
|
||||
task: ""
|
||||
lineage_type: import
|
||||
upstream_source: https://github.com/VoltAgent/awesome-ai-agent-papers/blob/aa50c0c3/README.md
|
||||
upstream_sha: aa50c0c3
|
||||
imported_at: 2026-07-03
|
||||
upstream_source: https://github.com/VoltAgent/awesome-ai-agent-papers/blob/c8502b6a/README.md
|
||||
upstream_sha: c8502b6a
|
||||
imported_at: 2026-08-08
|
||||
prompt_class: catalogue
|
||||
upstream_changes: accepted
|
||||
author: upstream
|
||||
@@ -69,12 +69,13 @@ Hundreds of papers are published on arXiv every week, and a growing number of th
|
||||
<br>
|
||||
|
||||
<details open id="multi-agent">
|
||||
<summary><h3 style="display:inline">Multi-Agent (53)</h3></summary>
|
||||
<summary><h3 style="display:inline">Multi-Agent (54)</h3></summary>
|
||||
|
||||
<br>
|
||||
|
||||
| Paper | arXiv ID |
|
||||
|---|:---:|
|
||||
| **[Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy](https://arxiv.org/pdf/2606.24177)** - Treats prompt engineering as an engineering discipline rather than an art, minimizing human time while expecting maximum output. Carries the smallest prompt footprint among cross-disciplinary autoresearch systems (18 roles, 230.6 KiB total), has run across 10+ research fields without modification, and the longest observed run went 30 days unattended. | <a href="https://arxiv.org/abs/2606.24177"><img src="https://img.shields.io/badge/arXiv-2606.24177-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[PerceptUI: LLM Agents as Human-Aligned Synthetic Users for UI/UX Evaluation](https://arxiv.org/pdf/2606.05697)** - A persona-conditioned framework that predicts how a specific user would answer UI/UX evaluation questions and explains why in natural language. Trained via contrastive reflection fine-tuning and reflective prompt evolution, reaching human-level realism and generalizing to unseen questions and personas. | <a href="https://arxiv.org/abs/2606.05697"><img src="https://img.shields.io/badge/arXiv-2606.05697-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computing](https://arxiv.org/pdf/2602.17607)** - A multi-agent pipeline that reads a PDE problem description in plain text and writes, debugs, and validates a classical numerical solver end-to-end. Generates spectral and finite-difference code (no neural networks), scoring ~6 orders of magnitude below FNO and CodePDE baselines. | <a href="https://arxiv.org/abs/2602.17607"><img src="https://img.shields.io/badge/arXiv-2602.17607-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[Beyond Offline A/B Testing: Context-Aware Agent Simulation for Recommender System Evaluation](https://arxiv.org/abs/2604.09549)** - Evaluates recommender systems via agent-RS interactions. | <a href="https://arxiv.org/abs/2604.09549"><img src="https://img.shields.io/badge/arXiv-2602.06039-b31b1b.svg" alt="arXiv" /></a> |
|
||||
@@ -206,7 +207,7 @@ Hundreds of papers are published on arXiv every week, and a growing number of th
|
||||
<br>
|
||||
|
||||
<details id="eval--observability">
|
||||
<summary><h3 style="display:inline">Eval & Observability (80)</h3></summary>
|
||||
<summary><h3 style="display:inline">Eval & Observability (81)</h3></summary>
|
||||
|
||||
<br>
|
||||
|
||||
@@ -214,6 +215,7 @@ Hundreds of papers are published on arXiv every week, and a growing number of th
|
||||
|---|:---:|
|
||||
| **[PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting](https://arxiv.org/pdf/2606.08878)** - A 110-scenario benchmark testing whether LLMs can compose orchestration prompts that distribute context to sub-agents without information leakage. Across 10 topologies and 27 commercial models, GPT-5.5 leads with 62% pass rate while the average is 14.9%. | <a href="https://arxiv.org/abs/2606.08878"><img src="https://img.shields.io/badge/arXiv-2606.08878-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[ClawBench: Evaluating Browser Agents on Live Production Websites with Submission-Interception](https://arxiv.org/abs/2604.08523)** - Benchmarks browser agents on 283 everyday tasks (V1 153 + V2 130) across 163 live production sites, with a Chrome-extension plus CDP layer that blocks only the final write request so agents can run end-to-end on real sites without real-world side effects. Two-stage scoring (interception + LLM judge); leaderboard at https://claw-bench.com. | <a href="https://arxiv.org/abs/2604.08523"><img src="https://img.shields.io/badge/arXiv-2604.08523-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs](https://arxiv.org/pdf/2505.20139)** - Benchmarks structured-output generation and cross-format conversion across 18 text and renderable formats, with syntax, structural, and visual evaluation checks. | <a href="https://arxiv.org/abs/2505.20139"><img src="https://img.shields.io/badge/arXiv-2505.20139-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[From Features to Actions: Explainability in Traditional and Agentic AI Systems](https://arxiv.org/pdf/2602.06841v1)** - Compares attribution-based explanations with trace-based diagnostics across static and agentic settings to study how explainability methods translate to multi-step agent trajectories. | <a href="https://arxiv.org/abs/2602.06841v1"><img src="https://img.shields.io/badge/arXiv-2602.06841-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[Agentic Uncertainty Reveals Agentic Overconfidence](https://arxiv.org/pdf/2602.06948v1)** - Investigates whether agents can accurately predict their own success rates in agentic tasks. | <a href="https://arxiv.org/abs/2602.06948v1"><img src="https://img.shields.io/badge/arXiv-2602.06948-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents](https://arxiv.org/pdf/2602.06855v1)** - Introduces 20 research tasks from real ML papers covering idea generation, experiments, and refinement for benchmarking science agents. | <a href="https://arxiv.org/abs/2602.06855v1"><img src="https://img.shields.io/badge/arXiv-2602.06855-b31b1b.svg" alt="arXiv" /></a> |
|
||||
@@ -258,6 +260,7 @@ Hundreds of papers are published on arXiv every week, and a growing number of th
|
||||
| **[The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution](https://arxiv.org/pdf/2601.15075v2)** - Proposes a hierarchical framework for general agentic attribution that identifies internal factors driving agent actions through temporal likelihood dynamics and perturbation-based analysis. | <a href="http://arxiv.org/abs/2601.15075v2"><img src="https://img.shields.io/badge/arXiv-2601.15075-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering](https://arxiv.org/pdf/2601.14470v1)** - Analyzes token consumption patterns across software development lifecycle stages in a multi-agent system to identify where tokens are consumed and which stages drive cost. | <a href="http://arxiv.org/abs/2601.14470v1"><img src="https://img.shields.io/badge/arXiv-2601.14470-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[APEX-Agents](https://arxiv.org/pdf/2601.14242v2)** - Introduces a benchmark of 480 long-horizon, cross-application productivity tasks created by investment banking analysts, consultants, and lawyers for evaluating AI agent capabilities in realistic work environments. | <a href="http://arxiv.org/abs/2601.14242v2"><img src="https://img.shields.io/badge/arXiv-2601.14242-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[DRIFT: Detecting Representational Inconsistencies for Factual Truthfulness](https://arxiv.org/pdf/2601.14210)** - Lightweight probes (3M-37M params) trained on middle-layer hidden states catch factually wrong generations before they reach the user, at less than 0.1% overhead. Up to 13 AUROC points above final-layer baselines, and the probes transfer across datasets without retraining. | <a href="https://arxiv.org/abs/2601.14210"><img src="https://img.shields.io/badge/arXiv-2601.14210-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[CooperBench: Why Coding Agents Cannot be Your Teammates Yet](https://arxiv.org/pdf/2601.13295v2)** - Introduces a benchmark of 600+ collaborative coding tasks to evaluate whether coding agents can coordinate as effective teammates under various coordination structures. | <a href="http://arxiv.org/abs/2601.13295v2"><img src="https://img.shields.io/badge/arXiv-2601.13295-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[Insider Knowledge: How Much Can RAG Systems Gain from Evaluation Secrets?](https://arxiv.org/pdf/2601.13227v1)** - Investigates how RAG systems can game nugget-based LLM judge evaluations through metric overfitting, demonstrating near-perfect scores when evaluation elements are leaked or predictable. | <a href="http://arxiv.org/abs/2601.13227v1"><img src="https://img.shields.io/badge/arXiv-2601.13227-b31b1b.svg" alt="arXiv" /></a> |
|
||||
| **[Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents](https://arxiv.org/pdf/2601.15322v1)** - Introduces the Determinism-Faithfulness Assurance Harness for measuring trajectory determinism and evidence-conditioned faithfulness in tool-using LLM agents across 74 configurations and 12 models. | <a href="http://arxiv.org/abs/2601.15322v1"><img src="https://img.shields.io/badge/arXiv-2601.15322-b31b1b.svg" alt="arXiv" /></a> |
|
||||
|
||||
Reference in New Issue
Block a user