ci(uat): measure the full-depth conformance cost nightly and raise the step budget #3609

Part of #3601. Scope set by #3619 (CNCF bar).

Summary

Nightly UAT keeps running aicr validate --phase all with every conformance check the recipe lists, at the depth the CNCF bar requires. No weekly lane, no depth tiers, no nightly --skip-check list. This issue measures the added cost and raises the step budget.

Constraint

The "UAT - validate (all phases)" step in .github/workflows/uat-aws.yaml has a 40-minute budget. The UAT cells have two GPU nodes (AWS desired: 2, GCP a3-megagpu-8g), so capacity is not the blocker. After #3603, #3604 and #3428 land, nightly adds one CPU gang Job, one HPA cycle on 2 GPUs, one TrainJob and gateway traffic on CPU backends, estimated at 5 to 8 minutes with the HPA scale-down as the long pole. Recipes that gain checks through #3605's wiring (EKS, AKS and NVKE training bases gain gang, HPA, secure access and cluster autoscaling) see the largest delta.

Change

• Record the per-check durations from the nightly CTRF after each parity PR lands and attach them here.
• Raise timeout-minutes on the validate step (and the job cap it sits under) to the measured need plus headroom.
• --skip-check remains only for lanes that cannot host a check at all (Kind and KWOK leaves) and is never set on GPU cells.
• The conformance checks run serially on a cell so one check's GPU hold is not misread by another.

Acceptance

• Nightly passes within the raised budget for two consecutive nights with the full check set; durations are recorded on this issue.