Repository navigation
aws-healthomics: add S3 run storage, EXPRESS run mode, runtime platform, local scratch, image staging, run diagnosis/optimization and metrics steering docs - #216
Draft
ThandojuSruthi wants to merge 2 commits into
Conversation
…platform, local scratch, image staging, run diagnosis/optimization and metrics steering docs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available: N/A
Description of changes:
Adds new steering docs to the
aws-healthomicspower and updates existing ones.New steering docs
steering/s3-run-storage.md— S3 run storage (storageType: S3,workDirUri,taskWorkDirS3Uri): prerequisites, service-role permissions (work directory, output, and input buckets), starting and monitoring runs, run cache with S3 storage, reruns, local storage on S3 runs, and troubleshooting S3 storage failures.steering/express-run-mode.md— EXPRESS run mode (runMode): when to use it, requirements (S3 run storage, runs under 8 hours, RESTRICTED networking), starting and verifying an EXPRESS run, and troubleshooting.steering/runtime-platform.md— Choosing X86_64 or ARM64 compute withruntimePlatformPreferenceand task-levelruntimePlatform(WDL, Nextflow, CWL): precedence, image-based inference, GPU and Region limitations, and troubleshooting.steering/ephemeral-storage.md— Local scratch (ephemeral) storage for workflow tasks:scratchStorageMode, sizing withdisk/disks/tmpdirMin, monitoring, billing, and troubleshooting/tmpexhaustion.steering/image-staging.md— Staging container images from registries not supported by ECR pull-through cache (for example Seqera Wave, NVIDIA NGC, Google Artifact Registry).steering/diagnose-run-failure.md— Diagnosing a specific failed run and guiding remediation, including routing feature-specific failure reasons to the matching steering doc.steering/optimize-run.md— Analyzing a completed run for cost and performance optimizations and right-sizing.steering/metrics-analysis.md— Answering run and workflow metrics questions with vended CloudWatch metrics: utilization, bottlenecks, stuck runs, right-sizing, and run comparisons.Updated steering docs
POWER.md— Routing entries for all new steering docs.steering/running-a-workflow.md— New "Storage Type, Run Mode, and Architecture" section with tooling guidance.steering/batch-runs.md—storageType: S3,workDirUri, andruntimePlatformPreferenceindefaultRunSetting; note thatrunModeis not supported for batch runs andscratchStorageModehas no effect on S3 batches.steering/workflow-development.md— Build images with an ARM64 variant (multi-architecture preferred) when ARM64 is wanted; x86_64 remains the default.steering/migration-guide-for-wdl.md— Fix the container-tool check so it verifies every required binary, not just the first.steering/regional-capabilities.md— Add g5.12xlarge and larger availability in Tokyo and Ohio.Testing: Imported the power into Kiro IDE as a custom power and verified routing and answers for S3 run storage, EXPRESS run mode, ARM64, batch runs, run caching, local scratch, and run-failure diagnosis prompts.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.