Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -128,8 +128,7 @@ Use it to send a sensitive run to an isolated bucket, point at a bucket with dif

{{< variant flyte >}}
{{< markdown >}}
> [!NOTE]
> The open-source Flyte backend doesn't apply `raw_data_path` to remote runs. It accepts the value, but writes offloaded data under each action's output location, next to its `outputs.pb`.
On the open-source Flyte backend, remote runs apply `raw_data_path` only with flyteorg/flyte `v2.0.54` or later.
{{< /markdown >}}
{{< /variant >}}

Expand Down Expand Up @@ -164,9 +163,3 @@ For how retention policies are configured in your deployment, see [BYOC data ret
- **Values** = every task input/output is a **literal**, stored either *inline* in `inputs.pb`/`outputs.pb` or *by reference* to **raw data** offloaded in the bucket.
- **"Metadata" in docs** usually means database-side records. **"Metadata bucket" in Helm/ops** is legacy naming for the data plane bucket. It does *not* hold database metadata.
- **`flyte.with_runcontext(raw_data_path=...)`** is your knob to send offloaded data elsewhere per run.

{{< variant flyte >}}
{{< markdown >}}
The open-source Flyte backend doesn't apply `raw_data_path` to remote runs; see [the note above](#per-run-customization-raw_data_path).
{{< /markdown >}}
{{< /variant >}}
8 changes: 1 addition & 7 deletions content/user-guide/run-scaling/data-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ When a task returns data:

{{< variant flyte >}}
{{< markdown >}}
The open-source Flyte backend doesn't apply `raw_data_path` to remote runs; see [Where your data lives](../get-started/core-concepts/where-data-lives#per-run-customization-raw_data_path).
On the open-source Flyte backend, remote runs apply `raw_data_path` only with flyteorg/flyte `v2.0.54` or later.
{{< /markdown >}}
{{< /variant >}}

Expand Down Expand Up @@ -161,9 +161,3 @@ run = flyte.with_runcontext(
```

This allows you to control where raw data (files, directories, DataFrames) is stored for specific runs. See [Run context](../tasks/task-deployment/run-context) for the full set of options.

{{< variant flyte >}}
{{< markdown >}}
The open-source Flyte backend doesn't apply `raw_data_path` to remote runs; see [Where your data lives](../get-started/core-concepts/where-data-lives#per-run-customization-raw_data_path).
{{< /markdown >}}
{{< /variant >}}
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,12 @@ Use a local directory for development:
flyte run --local --raw-data-path ./output/ my_example.py my_task
```

{{< variant flyte >}}
{{< markdown >}}
On the open-source Flyte backend, remote runs apply `--raw-data-path` only with flyteorg/flyte `v2.0.54` or later.
{{< /markdown >}}
{{< /variant >}}

### Use cases

- **Custom storage locations**: Direct outputs to specific S3 buckets or paths
Expand Down
2 changes: 1 addition & 1 deletion content/user-guide/tasks/task-deployment/run-context.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ All parameters are optional. Unset parameters inherit from the configuration fil

{{< variant flyte >}}
{{< markdown >}}
The open-source Flyte backend doesn't apply `raw_data_path` (or `--raw-data-path`) to remote runs; see [Where your data lives](../../get-started/core-concepts/where-data-lives#per-run-customization-raw_data_path).
On the open-source Flyte backend, remote runs apply `raw_data_path` (or `--raw-data-path`) only with flyteorg/flyte `v2.0.54` or later.
{{< /markdown >}}
{{< /variant >}}

Expand Down
Loading