Skills

Tests AI agent tool use with realistic fixtures, malformed inputs, permission boundaries, deterministic assertions, and recovery checks for failed or partially completed actions.

Defines repeatable quality evaluation for LLM outputs using representative datasets, scoring rubrics, model-graded checks, human review sampling, and regression thresholds.

Создаёт воспроизводимые оценки AI-агентов, измеряющие успешность задач, корректность использования инструментов, обоснованность, задержку, стоимость и безопасное восстановление в реалистичных сквозных сценариях.

Diagnoses failing browser and end-to-end tests by correlating assertions, traces, screenshots, network activity, timing, and application state into reproducible fixes.

Designs and implements visual regression testing for web UIs. Recommends a tooling approach (Playwright snapshots, Storybook + a diffing service, or a dedicated platform), writes screenshot tests with stable selectors and masked dynamic regions, sets sensible diff thresholds, and integrates the suite into CI with baseline management to reduce flaky failures.

Generates property-based tests that assert invariants across randomly generated inputs using frameworks like Hypothesis, fast-check, or jqwik. Identifies properties (round-trip, idempotence, invariants, oracle comparison), defines generators and shrinking, and sets up stateful testing for complex APIs. Surfaces edge cases that example-based tests miss.

Sets up and interprets mutation testing to measure real test-suite effectiveness beyond line coverage. Configures tools like Stryker, PIT, or mutmut, explains surviving mutants, recommends targeted tests to kill them, and tunes performance for CI. Helps teams find tests that assert nothing and coverage that lies.

Generates consumer-driven contract tests between services so that API providers and consumers stay compatible as they evolve independently. Produces Pact-style contracts, provider verification stubs, and CI wiring, and flags breaking changes before they reach production.