Repository navigation
HDDS-17374. Handle old clients not passing StoragePolicy in allocateBlock - #11716
Conversation
|
The behaviour of |
|
This change is need to handle the scenario mentioned in the pr-comment of the (PR). |
|
Please review @devmadhuu , @ashishkumar50 , @amaliujia ! |
There was a problem hiding this comment.
Thanks @anuragp010 for the patch. The code changes look good. Could we also add an acceptance test using a real older client binary to cover the bucket-policy change during block allocation?
The existing cross-version compatibility tests provide these references:
| Reference | What it does |
|---|---|
| xcompat/test-new.sh | Runs older clients against the current cluster. |
| xcompat/clients.yaml | Defines an actual 2.2.1 client image, alongside other released versions. |
| compatibility/write.robot | Uses the selected client binary to write keys, including RATIS/EC cases. |
| compatibility/read.robot | Reads keys and compares their contents with the original file. |
For this PR, the test could cover the following scenario:
| Step | Action | Expected result |
|---|---|---|
| 1 | Start the updated, finalized cluster. Use a new client to create a WARM bucket. | Bucket policy is WARM. |
| 2 | A real older client starts writing a key and allocates its first block. | First block uses DISK. |
| 3 | Pause that writer. A new client changes the bucket policy to COLD. | The open key retains WARM. |
| 4 | Resume the same older writer and force another block allocation. | The next block still uses DISK. |
| 5 | Complete the write and read the key back. | Content matches, and every block follows WARM. |
Please use an explicit pause/resume signal between the writer and test controller to make the policy change deterministic.
|
Thanks for the review and detailed pointers, @devmadhuu . I agree this scenario needs an acceptance test with a real older client, but can we take it up as part of a follow-up JIRA (HDDS-17828) rather than in this PR? The existing xcompat tests run one command to completion in one container. This scenario needs an old-client write that stops after the first block is allocated. While it is stopped, a new client changes the bucket policy, and then the old client resumes. That needs:
That is a larger change than this fix. |
Agree. Thanks, lets handle in follow up PR. |
devmadhuu
left a comment
There was a problem hiding this comment.
Thanks @anuragp010 for the patch. Changes LGTM +1
amaliujia
left a comment
There was a problem hiding this comment.
The core idea is simple and easy to understand.
Just have a minor comment.
| ACLType.WRITE, allocateBlockRequest.getClientID()); | ||
|
|
||
| // An older client does not send the storage policy in allocateBlock requests. Using the bucket policy for them | ||
| // is wrong when the bucket policy changes while the key is being written. |
There was a problem hiding this comment.
Nit: the comment in this two lines read odd.
I am wondering if the following is better
| // is wrong when the bucket policy changes while the key is being written. | |
| // Older clients, and keys opened without a policy, do not send the policy in allocateBlock. In such cases, | |
| // use the policy stored on the open key so that a bucket policy change during the write does not move later blocks to another tier. |
There was a problem hiding this comment.
Thanks for the review, @amaliujia !
I have reworded the comment with one minor tweak to your suggestion: I omitted "keys opened without a policy".
Once the storage policy feature is finalized, if a key is created without the client explicitly passing a policy, it is resolved during the createKey/createFile request's preExecute phase on the OM. As a result, the OMKeyInfo object sent back to the client already contains a storage policy. This same OMKeyInfo object is then used for subsequent allocateBlock calls. Therefore, the scenario addressed by this PR will only occur with older clients.
amaliujia
left a comment
There was a problem hiding this comment.
LGTM. Probably can just merge this PR.
|
Thanks @anuragp010 for the patch. Thanks @amaliujia for the review. |
|
Thanks @devmadhuu and @amaliujia for the review ! |
What changes were proposed in this pull request?
With an upgraded cluster containing all OMs supporting storage policy, an older client without the Storage Policy changes will not send the
storagePolicyfield in any request made to the OM. This is of consequence toallocateBlockas it allocates block based on the storage policy sent by the client. If the client doesn't send a storage policy, it uses the bucket's policy and falls back to the default. But in the scenario where the bucket policy changes from the time key is created to the subsequentallocateBlockcall, thenallocateBlockwill not follow the key's storage policy. This PR fixes that by getting the key's storage policy from theopenKeyTableif not passed. Should the openKey not have a policy (Key is created pre upgrade on the OM), thenallocateBlockwill use the bucket's storage policy, falling back to the default if bucket's policy is null.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17374
How was this patch tested?
testOldClientAllocateBlockKeepsKeyPolicyWhenBucketPolicyChanges