Feature request
Please expose Amazon FSx for Lustre enhanced metadata performance configuration for ParallelCluster-managed Persistent 2 filesystems.
Use case
We create short-lived sequencing-analysis clusters with FSx Lustre storage and S3 data repository associations. We want ParallelCluster and its CloudFormation stack to own filesystem creation and deletion so that the storage lifecycle follows the cluster. Creating FSx separately and attaching it by FileSystemId makes that lifecycle harder to manage and increases the risk of leaving billable resources behind.
Current limitation
In ParallelCluster 3.16.1, FsxLustreSettings does not expose MetadataConfiguration, and the managed FSx CloudFormation renderer does not pass it to the filesystem resource.
FSx requires the metadata configuration to be specified when the filesystem is created. It cannot be added later to a filesystem created without that configuration. Consequently, creating a filesystem through ParallelCluster and updating it after creation does not recover this capability.
AWS documentation: https://docs.aws.amazon.com/fsx/latest/LustreGuide/managing-metadata-performance.html
Requested behavior
Support a configuration such as the following (proposed syntax):
SharedStorage:
- Name: analysis-fsx
StorageType: FsxLustre
MountDir: /fsx
FsxLustreSettings:
DeploymentType: PERSISTENT_2
FileSystemTypeVersion: "2.15"
StorageType: SSD
StorageCapacity: 12000
PerUnitStorageThroughput: 250
MetadataConfiguration:
Mode: AUTOMATICAlso support USER_PROVISIONED with an explicit Iops value, subject to FSx service constraints.
Please include:
- Schema/model support and rendering into AWS::FSx::FileSystem LustreConfiguration.MetadataConfiguration.
- Validation for supported deployment types, Lustre versions, storage types, modes, and IOPS combinations.
- Documented update behavior consistent with FSx restrictions.
- No change to existing behavior when the field is omitted.
This is about metadata operation throughput and later metadata scalability; it is separate from S3 lazy hydration, data import policies, and cache release.
Feature request
Please expose Amazon FSx for Lustre enhanced metadata performance configuration for ParallelCluster-managed Persistent 2 filesystems.
Use case
We create short-lived sequencing-analysis clusters with FSx Lustre storage and S3 data repository associations. We want ParallelCluster and its CloudFormation stack to own filesystem creation and deletion so that the storage lifecycle follows the cluster. Creating FSx separately and attaching it by FileSystemId makes that lifecycle harder to manage and increases the risk of leaving billable resources behind.
Current limitation
In ParallelCluster 3.16.1, FsxLustreSettings does not expose MetadataConfiguration, and the managed FSx CloudFormation renderer does not pass it to the filesystem resource.
FSx requires the metadata configuration to be specified when the filesystem is created. It cannot be added later to a filesystem created without that configuration. Consequently, creating a filesystem through ParallelCluster and updating it after creation does not recover this capability.
AWS documentation: https://docs.aws.amazon.com/fsx/latest/LustreGuide/managing-metadata-performance.html
Requested behavior
Support a configuration such as the following (proposed syntax):
Also support USER_PROVISIONED with an explicit Iops value, subject to FSx service constraints.
Please include:
This is about metadata operation throughput and later metadata scalability; it is separate from S3 lazy hydration, data import policies, and cache release.