Skip to content

Add coder-eval plugin (Code Quality Testing) - #415

Open
uipreliga wants to merge 1 commit into
ccplugins:mainfrom
uipreliga:add-coder-eval
Open

Add coder-eval plugin (Code Quality Testing)#415
uipreliga wants to merge 1 commit into
ccplugins:mainfrom
uipreliga:add-coder-eval

Conversation

@uipreliga

Copy link
Copy Markdown

Adds coder-eval to Code Quality Testing.

Disclosure: I maintain this project. Happy to reword, shorten, or move the entry — just say which.

What it is

Most quality tooling checks the files a plugin ships. coder-eval checks the behavior: given a labelled set of prompts, does the skill actually fire when it should and stay quiet when it shouldn't? It reports suite-level precision / recall / F1 and can fail a GitHub Actions run when the numbers drop, so a description: edit that quietly breaks activation gets caught in review.

It runs the same suite against Claude Code, Codex, OpenCode and Antigravity, which is how you tell "the skill is wrong" apart from "this harness didn't route to it."

Two things I left out on purpose, tell me if you want them

  1. External Marketplaces — coder_eval is also a marketplace (claude plugin marketplace add UiPath/coder_eval), so it would qualify for that table too. I did not add a second row, because putting myself in two places in one PR seemed presumptuous. Say the word and I'll add it.
  2. README-zh.md — I did not touch it. It currently carries 1 of the 6 external entries that README.md has, so it looks like you sync it yourself rather than expecting contributors to. If you'd prefer the row there too, here it is:
- [coder-eval](https://github.com/UiPath/coder_eval) - 面向技能与插件的行为评测。基于带标签的提示集给出技能激活的精确率/召回率,并可作为 CI 门禁。同一套用例可在 Claude Code、Codex 与 OpenCode 上运行。

Diff is one line. Thanks for maintaining the list.

Signed-off-by: uipreliga <tomasz.religa@uipath.com>

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant