From 21b396abe14bd35d821c346d10961a97de4f2d56 Mon Sep 17 00:00:00 2001 From: Simon Mo Date: Tue, 7 Jul 2026 08:16:08 -0700 Subject: [PATCH] AGENTS MD: Add suggestion on how to incorporate tests (#47784) Signed-off-by: Simon Mo Co-authored-by: Cursor Agent Co-authored-by: Benjamin Chislett --- AGENTS.md | 47 +++++++++++++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 241eab38818..a53b81873cf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -29,6 +29,7 @@ Do not open one-off PRs for tiny edits (single typo, isolated style change, one - PR descriptions for AI-assisted work **must** include: - Why this is not duplicating an existing PR. - Test commands run and results. + - Model evaluation results when the change affects output, accuracy, or serving. - Clear statement that AI assistance was used. ### Fail-closed behavior @@ -66,23 +67,38 @@ VLLM_USE_PRECOMPILED=1 uv pip install -e . --torch-backend=auto uv pip install -e . --torch-backend=auto ``` -### Running tests +### Tests > Requires [Environment setup](#environment-setup) and [Installing dependencies](#installing-dependencies). ```bash -# Install test dependencies. -# requirements/test/cuda.txt is pinned to x86_64; on other platforms, use the -# unpinned source file instead: -uv pip install -r requirements/test/cuda.in # resolves for current platform -# Or on x86_64: -uv pip install -r requirements/test/cuda.txt +# Install test dependencies (use cuda.in on non-x86_64): +uv pip install -r requirements/test/cuda.in -# Run a specific test file (use .venv/bin/python directly; -# `source activate` does not persist in non-interactive shells): +# Run a specific test file: .venv/bin/python -m pytest tests/path/to/test_file.py -v ``` +When adding tests: + +- **Design before you write.** Answer four questions first: what is the module + for, what is its I/O contract, what failure am I guarding against, and what is + the cheapest level that catches it (unit over integration over e2e)? +- **Reuse before create.** Extend existing test files, `conftest.py` fixtures, and + helpers; add a new file only when no nearby suite fits. +- **Test behavior with intent.** Assert observable outcomes through public APIs; + state why in the name or docstring. Skip trivial wiring; flaky tests are worse + than no tests. +- **Keep it minimal.** One behavior per test and the smallest setup that + triggers it; if the test diff dwarfs the code change, cut scope. +- **No one-off kernel benchmarks in `tests/`.** Put kernel perf work in + `benchmarks/kernels/`; prove correctness in existing pytest suites. +- **Run model evals for model-affecting changes.** Search `tests/evals/` or use + `vllm bench` and include results in the PR — do not wait for reviewers to ask. + +For model-specific requirements, see +[`docs/contributing/model/tests.md`](docs/contributing/model/tests.md). + ### Running linters > Requires [Environment setup](#environment-setup). @@ -107,23 +123,18 @@ Use [Google-style docstrings](https://google.github.io/styleguide/pyguide.html#3 ### Coding style guidelines -Follow these rules for all code changes in this repository: - -- Try to match existing code style. -- Code should be self-documenting and self-explanatory. -- Keep comments and docstrings minimal and concise. +- Match existing code style +- Minimize use of comments. Eliminate comments which are redundant, preferring legible and self-documenting code. When used, keep docstrings and comments brief and direct. - Assume the reader is familiar with vLLM. ### Commit messages -Add attribution using commit trailers such as `Co-authored-by:` (other projects use `Assisted-by:` or `Generated-by:`). For example: +Add attribution using commit trailers such as `Co-authored-by:` (other projects use `Assisted-by:` or `Generated-by:`): ```text Your commit message here -Co-authored-by: GitHub Copilot -Co-authored-by: Claude -Co-authored-by: gemini-code-assist +Co-authored-by: Agent Name Here Signed-off-by: Your Name ```