Skills

Evaluates retrieval quality for search and RAG systems using grounded test sets, relevance metrics, failure taxonomy, chunking experiments, and actionable remediation.

Guides containment, evidence collection, root-cause analysis, customer communication, and remediation when an AI agent takes an unsafe, incorrect, or unauthorized action.

Tests AI agent tool use with realistic fixtures, malformed inputs, permission boundaries, deterministic assertions, and recovery checks for failed or partially completed actions.

Designs traces, metrics, structured events, privacy controls, dashboards, and alerts for LLM applications so teams can diagnose quality, cost, latency, and tool-use behavior.

Создаёт воспроизводимые оценки AI-агентов, измеряющие успешность задач, корректность использования инструментов, обоснованность, задержку, стоимость и безопасное восстановление в реалистичных сквозных сценариях.

Diagnoses failing browser and end-to-end tests by correlating assertions, traces, screenshots, network activity, timing, and application state into reproducible fixes.

Reviews cloud and AI infrastructure spend to identify measurable savings while preserving reliability, performance, security, ownership, and product requirements.

Plans safe database schema and data migrations with backwards compatibility, staged rollout, validation, rollback, performance safeguards, and application coordination.

Designs and implements secure, ergonomic Model Context Protocol servers with precise tool schemas, transport selection, authentication boundaries, error contracts, and tests.

Converts product safety requirements into practical agent and LLM policies with abuse cases, escalation paths, safe defaults, audit events, and acceptance criteria.

Optimizes prompts and agent workflows for finite context windows by prioritizing evidence, compressing history, managing retrieval budgets, and measuring token-cost trade-offs.

Defines repeatable quality evaluation for LLM outputs using representative datasets, scoring rubrics, model-graded checks, human review sampling, and regression thresholds.

Designs durable memory for AI agents, including what to retain, how to retrieve it, privacy boundaries, expiration policies, and evaluation criteria for useful recall.

Designs low-risk secret rotation for applications and agents, covering inventory, overlapping credentials, automation, audit evidence, emergency revocation, and rollback.

Designs durable API evolution plans with compatibility guarantees, deprecation policy, version negotiation, consumer communication, migration tooling, and operational telemetry.

Reviews Model Context Protocol servers for excessive permissions, unsafe tool design, prompt injection exposure, secret handling, authorization gaps, and auditability.

Converts a design specification or screenshot into accessible, responsive production UI while preserving component boundaries, design tokens, semantic structure, and states.

Simulates realistic adversarial behavior against AI features to uncover prompt injection, data exfiltration, unsafe tool use, jailbreak, and authorization failures before launch.

Builds disaster recovery and business continuity plans for systems and data. Defines RTO and RPO targets, maps critical dependencies, chooses backup and replication strategies, documents failover and restore procedures, and designs a testing cadence with game days. Produces a plan that balances resilience against cost and operational complexity.

Designs caching strategies across the stack to cut latency and load. Chooses cache layers (browser, CDN, application, database), picks patterns (cache-aside, read-through, write-through, write-behind), sets TTLs and eviction policies, and plans invalidation to avoid staleness and stampedes. Produces a layered plan with keys, TTLs, and invalidation rules.

Designs and implements visual regression testing for web UIs. Recommends a tooling approach (Playwright snapshots, Storybook + a diffing service, or a dedicated platform), writes screenshot tests with stable selectors and masked dynamic regions, sets sensible diff thresholds, and integrates the suite into CI with baseline management to reduce flaky failures.