Skip to content

Add content loader for json-lines and ztsd, gzip and zlib encoding - #356

Merged
jeparlefrancais merged 3 commits into
mainfrom
json-lines-content-loader
Jun 10, 2026
Merged

jeparlefrancais merged 3 commits into
mainfrom
json-lines-content-loader

Conversation

@jeparlefrancais

@jeparlefrancais jeparlefrancais commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Closes #353

Adds new content loaders and encoding options for embedding file content.

  • json_lines loader to support line-delimited json
  • compression encodings zstd, gzip and zlib: new variants for string, buffer, and bytes loaders

Just like the existing base64, the content is compressed at build time by darklua and embedded as-is. Decoding is left to the runtime.

  • add entry to the changelog
  • update docs

@github-actions

Copy link
Copy Markdown

Coverage after merging json-lines-content-loader into main will be

84.92%

Coverage Report for Changed Files
FileStmtsBranchesFuncsLinesUncovered Lines
src/frontend
   configuration.rs82.26%100%80.60%82.55%114–117, 121–125, 129–132, 136–140, 143–146, 149–154, 157–159, 241–260, 262–263, 345–349, 81–83, 94–97
   content_loader.rs67.88%100%47.37%70.11%113–116, 118–120, 123–125, 245–247, 287, 294–295, 315–317, 334–335, 337–339, 341, 344–347, 349–352, 354, 357–360, 362–365

@jeparlefrancais
jeparlefrancais merged commit 4a64350 into main Jun 10, 2026
9 of 10 checks passed
@jeparlefrancais
jeparlefrancais deleted the json-lines-content-loader branch June 10, 2026 19:59
@jeparlefrancais jeparlefrancais mentioned this pull request Jun 10, 2026
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
darklua 0.19.0

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre>The big change is **content loaders**. darklua now accepts glob patterns that map to configurable loaders in your config file. Most loaders are already there by default, but you can override or extend them:

```json5
{
  generator: "retain_lines",
  rules: [
    // ...
  ],
  loaders: {
    "**/*.model.json": "copy",
    "**/*.json": "json",
    "**/*.md": "string",
  },
}
```

This solves a few issues I know people had and it opens up new possibilities too:
- Rojo model files can now be copied when processing files (no more manual `cp` workaround) with the `copy` loader
- when bundling, darklua could already load data files (json, toml, yaml) and inline the converted data as Lua code, **but now** there are actual loaders to make that work in the regular `process` flow too
- there is now a path to support more Lua parsers, since there is a way to add loaders like a future `lua53` loader. (this is not currently planned, but it could be sponsored)

Available loaders: `luau`, `copy`, `skip`, `string`, `buffer`, `bytes`, data loaders (`json`, `json_lines`, `toml`, `yaml`), and encoding variants for embedding content (`/base64`, `/zstd`, `/gzip`, `/zlib`)

Other changes:
- `const` declarations are now supported, with a new `make_assignment_local` rule to convert them to `local` assignments
- type instantiation prefixes and methods are now supported (e.g. `func<<string>>()`)
- the `rename_variables` rule can now detect globals automatically and avoid renaming them

---

## What's Changed
* Support type instantiation prefixes and methods by @jeparlefrancais in seaofvoices/darklua#345
* Add `make_assignment_local` rule by @notpoiu in seaofvoices/darklua#346
* Add `detect_globals` parameter to `rename_variables` rule by @jeparlefrancais in seaofvoices/darklua#348
* Fix file watcher to handle rename events by @jeparlefrancais in seaofvoices/darklua#351
* Create content loaders by @jeparlefrancais in seaofvoices/darklua#354
* Support infinity for generators `column_span` parameter by @jeparlefrancais in seaofvoices/darklua#355
* Add content loader for json-lines and `ztsd`, `gzip` and `zlib` encoding by @jeparlefrancais in seaofvoices/darklua#356

## New Contributors
* @notpoiu made their first contribution in seaofvoices/darklua#346

**Full Changelog**: https://github.com/seaofvoices/darklua/compare/v0.18.0...v0.19.0</pre>
  <p>View the full release notes at <a href="https://github.com/seaofvoices/darklua/releases/tag/v0.19.0">https://github.com/seaofvoices/darklua/releases/tag/v0.19.0</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!11206
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New Content Loader for json-lines (ndjson, jsonl)

1 participant