feat: v1 — boilerplate WxO + web
Some checks failed
CI — Lint + Evals / lint (push) Has been cancelled
CI — Lint + Evals / smoke (push) Has been cancelled

Boilerplate completo para construir soluciones agénticas sobre IBM
watsonx Orchestrate (ADK 2.x) con una capa web encima.

Destilado de cotemar-poc-n1 y dun-casos-prueba. Incluye:

- 11 docs (best practices, architecture patterns, ADK cheatsheet,
  observability, tool authoring, deployment, RUNBOOK, DEPLOY_TO_NEW_WOX,
  known-issues con 16 errores reales y fix, eval strategy, INDEX)
- 13 templates WxO (orchestrator/specialist/single-meta agents,
  3 connections, KB + runbook, observable_tool decorator, coercion
  helpers, Python tools, OpenAPI spec, backend filter endpoint, webhook
  validator HMAC, MCP connection)
- 6 scripts (deploy idempotente con fallback ADK, undeploy, reset,
  check-adk-version, new-specialist scaffold, eval-agents runner)
- 4 eval pieces (linter de best practices, runner, smoke-test, direct
  backend probe) + scenario templates
- 8 subagentes Claude (.claude/agents/) — wxo-architect, wxo-agent-author,
  wxo-tool-author, runbook-author, mock-builder, backend-tool-builder,
  eval-author, web-layer-builder
- Skill bundle fit-wxo-bootstrap/ (SKILL.md + 3 templates) listo para
  copiar a ~/.claude/skills/
- Web layer default FastAPI+HTMX con vista timeline observable + endpoint
  receptor de trazas
- Docker compose Coolify-ready (healthcheck wget -qO-, sin labels Traefik,
  network split internal:true)
- CI Gitea workflow con lint + smoke

Best practices enforced por evals/lint_wxo_yaml.py:
- A1: máx 10 tools por agente
- A3: orchestrator sin tools de remediación
- A6: agente sin propósito (react sin tools ni KB)
- T1: @observable_tool obligatorio, no @tool directo
- T3: _compat shim inline en tools.py
- D1: docker-compose sin wget --spider
- I-005: OpenAPI con description per-operation
- I-009: sin labels Traefik manuales

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Felipe Arentsen
2026-05-16 14:59:44 +00:00
commit b089c2ef18
70 changed files with 6739 additions and 0 deletions

72
evals/eval-agents.sh Executable file
View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
# Runner completo de evals. Corre las 4 capas en orden.
# Exit 0 si todo pasa, 1 si alguna falla.
set -uo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
FAILED=0
# ─── Capa 0: Lint ───────────────────────────────────────────────────────────
echo "[lint] best practices"
if python3 evals/lint_wxo_yaml.py; then
echo " ✓ lint PASS"
else
echo " ✗ lint FAIL"
FAILED=$((FAILED + 1))
fi
echo ""
# ─── Capa 1: Native (ADK) ───────────────────────────────────────────────────
echo "[native] orchestrate agents test"
NATIVE_PASS=0; NATIVE_FAIL=0
for f in evals/scenarios/*.input.json; do
[ -f "$f" ] || continue
# Derivar nombre del agente (convención: scenario_X_agentName.input.json o leer del JSON)
agent=$(python3 -c "import json,sys; d=json.load(open('$f')); print(d.get('_agent',''))" 2>/dev/null)
if [ -z "$agent" ]; then
echo "$f sin '_agent' field, skip"
continue
fi
if orchestrate agents test "$agent" --input-file "$f" > /tmp/eval_native_$$.log 2>&1; then
NATIVE_PASS=$((NATIVE_PASS + 1))
else
NATIVE_FAIL=$((NATIVE_FAIL + 1))
echo "$f failed (see /tmp/eval_native_$$.log)"
fi
done
echo " native: $NATIVE_PASS pass / $NATIVE_FAIL fail"
[ "$NATIVE_FAIL" -gt 0 ] && FAILED=$((FAILED + 1))
echo ""
# ─── Capa 2 + 3: Behavior + Final state (runner.py) ─────────────────────────
echo "[behavior+final] custom runner"
if python3 evals/runner.py; then
echo " ✓ runner PASS"
else
echo " ✗ runner FAIL"
FAILED=$((FAILED + 1))
fi
echo ""
# ─── Capa 4: Smoke test ─────────────────────────────────────────────────────
echo "[smoke] end-to-end"
if ./evals/smoke-test.sh > /tmp/eval_smoke_$$.log 2>&1; then
echo " ✓ smoke PASS"
else
echo " ✗ smoke FAIL (see /tmp/eval_smoke_$$.log)"
FAILED=$((FAILED + 1))
fi
echo ""
# ─── Resumen ────────────────────────────────────────────────────────────────
echo "════════════════════════════════════════"
if [ "$FAILED" -eq 0 ]; then
echo " ✓ ALL EVALS PASS"
exit 0
else
echo "$FAILED layer(s) FAILED"
exit 1
fi