add

README.md

@@ -9,87 +9,39 @@ hf_oauth: true
 short_description: Agent arena for software engineering tasks
 ---
 
-# SWE-Agent-Arena: An Interactive Platform for Evaluating CLI Coding Agents in Software Engineering
+# SWE-Agent-Arena
 
-Welcome to **SWE-Agent-Arena**, an open-source platform designed for evaluating CLI coding agents on real software engineering (SE) tasks. SWE-Agent-Arena benchmarks agents through blind pairwise comparisons — two random agents work on the same task, each in its own isolated environment, and users compare agent output and git diffs to vote on performance.
+An open-source platform for evaluating CLI coding agents on real software engineering tasks. Two anonymous agents tackle the same task in isolated environments — you compare their output and git diffs, then vote.
 
-## Key Features
+**[Try it on HF Spaces](https://huggingface.co/spaces/SE-Arena/SWE-Agent-Arena)**
 
-- **Blind Pairwise Comparison**: Two anonymous agents tackle the same task — vote without knowing which agent is which.
-- **Multi-Round Conversational Workflows**: Send follow-up messages to each agent independently across multiple rounds, mirroring real-world iterative SE workflows.
-- **Live Streaming Output**: Watch agent stdout stream in real-time as agents work on your task.
-- **Side-by-Side Git Diffs**: Compare exactly what each agent changed with side-by-side diff views.
-- **RepoChat Integration**: Automatically inject repository context (issues, PRs, commits, file contents, and more) from GitHub, GitLab, or HuggingFace URLs into agent workspaces for more realistic evaluations.
-- **Advanced Evaluation Metrics**: Assess agents using a comprehensive suite of metrics including:
-  - **Traditional ranking metrics**: Elo ratings and win rates to measure overall agent performance
-  - **Efficiency metrics**: Conversation Efficiency Index (CEI) — fewer rounds to win = higher score
-  - **Consistency metrics**: Model Consistency Score (MCS) — draw rate in self-matches to quantify agent determinism and reliability
-  - **Probabilistic metrics**: Bradley-Terry iterative MLE coefficients for pairwise comparison modeling
-  - **Network-based metrics**: PageRank, eigenvector centrality to identify influential agents in head-to-head comparisons
-  - **Community detection metrics**: Newman modularity to reveal clusters of agents with similar capabilities
-- **Transparent, Open-Source Leaderboard**: View real-time agent rankings across diverse SE workflows with full transparency.
-- **Intelligent Request Filtering**: Employ `gpt-oss-safeguard-20b` as a guardrail to automatically filter out non-software-engineering-related requests, ensuring focused and relevant evaluations.
+## Key Capabilities
 
-## Why SWE-Agent-Arena?
-
-Existing evaluation frameworks often don't address the complex, iterative nature of SE tasks performed by CLI coding agents. SWE-Agent-Arena fills critical gaps by:
-
-- Supporting context-rich, multi-turn evaluations to capture iterative agent workflows
-- Integrating repository-level context to simulate real-world development scenarios
-- Providing multidimensional metrics for nuanced agent comparisons
-- Comparing end-to-end CLI agents — not just language models — on actual code changes
+- **Blind pairwise comparison** with live-streaming output and side-by-side git diffs
+- **Multi-round conversations** — send follow-ups to each agent independently, mirroring real iterative workflows
+- **RepoChat** — auto-inject repo context (issues, PRs, files) from GitHub / GitLab / HuggingFace URLs
+- **Rich leaderboard** — Elo, Bradley-Terry MLE, PageRank, CEI (conversation efficiency), MCS (consistency), and Newman modularity
 
 ## How It Works
 
-1. **Submit a Task**: Sign in and input your SE-related task (optional: include a GitHub/GitLab/HuggingFace URL for repository context)
-2. **Watch Agents Work**: Two anonymous agents work on the task in parallel, each in an isolated temp directory — watch live output as they run
-3. **Compare Diffs**: Side-by-side git diffs show what each agent changed
-4. **Continue the Conversation**: Send follow-up messages to each agent independently to test contextual understanding over multiple rounds
-5. **Vote**: Choose the better agent — Agent A, Agent B, Tie, or Tie (Both Bad)
-
-## Getting Started
-
-### Prerequisites
-
-- A [Hugging Face](https://huggingface.co) account
-
-### Usage
-
-1. Navigate to the [SWE-Agent-Arena platform](https://huggingface.co/spaces/SE-Arena/SWE-Agent-Arena)
-2. Sign in with your Hugging Face account
-3. Enter your SE task prompt (optionally include a repository URL for context)
-4. Watch agents work, compare diffs, engage in multi-round interactions, and vote on agent performance
-
-## Contributing
-
-We welcome contributions from the community! Here's how you can help:
-
-1. **Submit SE Tasks**: Share your real-world SE problems to enrich our evaluation dataset
-2. **Report Issues**: Found a bug or have a feature request? Open an issue in this repository
-3. **Enhance the Codebase**: Fork the repository, make your changes, and submit a pull request
+1. **Submit a task** — sign in, describe an SE task (optionally paste a repo URL for context)
+2. **Watch agents work** — two anonymous agents run in parallel with live stdout
+3. **Compare diffs** — side-by-side view of what each agent changed
+4. **Vote** — Agent A, Agent B, Tie, or Tie (Both Bad)
 
 ## Terms of Service
 
-- The service is a **research preview**. It only provides limited safety measures and may generate offensive content.
-- It must not be used for any **illegal, harmful, violent, racist, or sexual** purposes.
-- Please do not upload any **private** information.
-- The service collects user dialogue data and reserves the right to distribute it under a **Creative Commons Attribution (CC-BY)** or similar license.
-
-## Future Plans
+- **Research preview** — limited safety measures; may generate offensive content.
+- Must not be used for illegal, harmful, violent, racist, or sexual purposes.
+- Do not upload private information.
+- Collected dialogue data may be distributed under a **CC-BY** or similar license.
 
-- **Analysis of Real-World SE Workloads**: Identify common patterns and challenges in user-submitted tasks
-- **Multi-Round Evaluation Metrics**: Develop specialized metrics for assessing agent adaptation over successive turns
-- **Expanded Agent Coverage**: Include additional CLI coding agents as they become available
-- **Advanced Context Integration**: Support richer repository context injection for more realistic evaluation scenarios
-
-## Contact
+## Contributing
 
-For inquiries or feedback, please [open an issue](https://github.com/Software-Engineering-Arena/SWE-Agent-Arena/issues/new) in this repository. We welcome your contributions and suggestions!
+Issues, tasks, and PRs welcome — [open an issue](https://github.com/Software-Engineering-Arena/SWE-Agent-Arena/issues/new) to get started.
 
 ## Citation
 
-Made with ❤️ for SWE-Agent-Arena. If this work is useful to you, please consider citing our vision paper:
-
 ```bibtex
 @inproceedings{zhao2025se,
   title={SE Arena: An Interactive Platform for Evaluating Foundation Models in Software Engineering},

README_ref.md

@@ -1,96 +0,0 @@
----
-title: SWE-Chatbot-Arena
-emoji: 🎯
-colorFrom: green
-colorTo: blue
-sdk: gradio
-sdk_version: 5.50.0
-app_file: app.py
-hf_oauth: true
-pinned: false
-short_description: Chatbot arena for software engineering tasks
----
-
-# SWE-Chatbot-Arena: An Interactive Platform for Evaluating Foundation Models in Software Engineering
-
-Welcome to **SWE-Chatbot-Arena**, an open-source platform designed for evaluating software engineering-focused foundation models (FMs), particularly large language models (LLMs). SWE-Chatbot-Arena benchmarks models in iterative, context-rich workflows that are characteristic of software engineering (SE) tasks.
-
-## Key Features
-
-- **Multi-Round Conversational Workflows**: Evaluate models through extended, context-dependent interactions that mirror real-world SE processes.
-- **RepoChat Integration**: Automatically inject repository context (issues, commits, PRs) into conversations for more realistic evaluations.
-- **Advanced Evaluation Metrics**: Assess models using a comprehensive suite of metrics including:
-  - **Traditional ranking metrics**: Elo ratings and win rates to measure overall model performance
-  - **Network-based metrics**: Eigenvector centrality and PageRank to identify influential models in head-to-head comparisons
-  - **Community detection metrics**: Newman modularity to reveal clusters of models with similar capabilities
-  - **Consistency metrics**: Self-play match analysis to quantify model determinism and reliability
-  - **Efficiency metrics**: Conversation efficiency index to measure response quality relative to length
-- **Transparent, Open-Source Leaderboard**: View real-time model rankings across diverse SE workflows with full transparency.
-- **Intelligent Request Filtering**: Employ `gpt-oss-safeguard-20b` as a guardrail to automatically filter out non-software-engineering-related requests, ensuring focused and relevant evaluations.
-
-## Why SWE-Chatbot-Arena?
-
-Existing evaluation frameworks (e.g. [LMArena](https://lmarena.ai)) often don't address the complex, iterative nature of SE tasks. SWE-Chatbot-Arena fills critical gaps by:
-
-- Supporting context-rich, multi-turn evaluations to capture iterative workflows
-- Integrating repository-level context through RepoChat to simulate real-world development scenarios
-- Providing multidimensional metrics for nuanced model comparisons
-- Focusing on the full breadth of SE tasks beyond just code generation
-
-## How It Works
-
-1. **Submit a Prompt**: Sign in and input your SE-related task (optional: include a repository URL for RepoChat context)
-2. **Compare Responses**: Two anonymous models provide responses to your query
-3. **Continue the Conversation**: Test contextual understanding over multiple rounds
-4. **Vote**: Choose the better model at any point, with ability to re-assess after multiple turns
-
-## Getting Started
-
-### Prerequisites
-
-- A [Hugging Face](https://huggingface.co) account
-
-### Usage
-
-1. Navigate to the [SWE-Chatbot-Arena platform](https://huggingface.co/spaces/SE-Arena/SWE-Chatbot-Arena)
-2. Sign in with your Hugging Face account
-3. Enter your SE task prompt (optionally include a repository URL for RepoChat)
-4. Engage in multi-round interactions and vote on model performance
-
-## Contributing
-
-We welcome contributions from the community! Here's how you can help:
-
-1. **Submit SE Tasks**: Share your real-world SE problems to enrich our evaluation dataset
-2. **Report Issues**: Found a bug or have a feature request? Open an issue in this repository
-3. **Enhance the Codebase**: Fork the repository, make your changes, and submit a pull request
-
-## Privacy Policy
-
-Your interactions are anonymized and used solely for improving SWE-Chatbot-Arena and FM benchmarking. By using SWE-Chatbot-Arena, you agree to our Terms of Service.
-
-## Future Plans
-
-- **Analysis of Real-World SE Workloads**: Identify common patterns and challenges in user-submitted tasks
-- **Multi-Round Evaluation Metrics**: Develop specialized metrics for assessing model adaptation over successive turns
-- **Expanded FM Coverage**: Include multimodal and domain-specific foundation models
-- **Advanced Context Compression**: Integrate techniques like [LongRope](https://github.com/microsoft/LongRoPE) and [SelfExtend](https://github.com/datamllab/LongLM) to manage long-term memory in multi-round conversations
-
-## Contact
-
-For inquiries or feedback, please [open an issue](https://github.com/SE-Arena/SWE-Chatbot-Arena/issues/new) in this repository. We welcome your contributions and suggestions!
-
-## Citation
-
-Made with ❤️ for SWE-Chatbot-Arena. If this work is useful to you, please consider citing our vision paper:
-
-```bibtex
-@inproceedings{zhao2025se,
-  title={SE Arena: An Interactive Platform for Evaluating Foundation Models in Software Engineering},
-  author={Zhao, Zhimin},
-  booktitle={2025 IEEE/ACM Second International Conference on AI Foundation Models and Software Engineering (Forge)},
-  pages={78--81},
-  year={2025},
-  organization={IEEE}
-}
-```