add MaxText GRPO / RL post-training sample on GKE TPU v7x - #2148
Conversation
moficodes
left a comment
There was a problem hiding this comment.
Few file paths are pointing to your local file system.
There are a lot of variable in yaml and commands that needs to be replaces manually by user. Look into using envsubst or let users know how and when to replace them.
…ded paths with relative links, and clean placeholders
@moficodes - Thanks for your feedbacks. I have updated the PR with all the recommended changes. Let me know if there are any further changes needed. Thanks! |
moficodes
left a comment
There was a problem hiding this comment.
Couple more clarifications.
no worries @moficodes - I just made few changes. PTAL and let me know if any changes are needed further. Thanks! |
@## Description
This PR introduces a comprehensive new sample for running MaxText GRPO (Reinforcement Learning) post-training on Google Kubernetes Engine (GKE) using Cloud TPUs.
It provides a production-tested foundation for aligning Large Language Models (like Llama-3.1-8B-Instruct) using Group Relative Policy Optimization on TPU v7x infrastructure.
Key Additions:
2x2x1TPU v7x topology) and Kueue cluster queue configuration.batch_sizevs completions are counted, and sequence length padding behaviors to prevent HBM OOMs.custom_rewards_template.py) and a custom dataset integration guide.calculate_mfu.py) to measure hardware utilization.co-contributor: @rick-c-goog