Skip to content

Drain autoreleased objects between unzipped entries - #406

Open
Aminark wants to merge 1 commit into
weichsel:developmentfrom
Aminark:fix/unzip-autorelease-pool
Open

Aminark wants to merge 1 commit into
weichsel:developmentfrom
Aminark:fix/unzip-autorelease-pool

Conversation

@Aminark

@Aminark Aminark commented Oct 23, 2026

Copy link
Copy Markdown

Fixes #391

Changes proposed in this PR

On Apple platforms, FileManager.unzipItem can retain autoreleased Foundation objects until the caller's autorelease pool drains. When extracting many small entries, peak memory grows with the number of entries.

Extract each entry inside its own autoreleasepool when Objective-C is available, following the workaround described in #391. Other platforms call the same extraction helper directly. The helper retains the existing containment checks, symlink restrictions, progress handling, and CRC32 validation; the checksum check is inlined into it.

Tests performed

  • All 131 existing tests passed on macOS 15.7.7 (arm64), Xcode 26.1 / Swift 6.2.1, in release mode with -Xswiftc -enable-testing --enable-code-coverage.
  • The 126 non-performance tests and five performance tests were run in separate processes because running them all together exhausted local temporary disk space. No tests were omitted.
  • Merged coverage, using the CI exclusions: 100% line coverage (2340/2340); changed file 100% (283/283).
  • SwiftLint 0.62.1: swiftlint lint --strict --quiet Sources Tests passed.
  • Generated ZIPs containing 1,000 and 10,000 deflated 1,184-byte files. A release-built SwiftPM executable called unzipItem inside one outer autorelease pool, without a Progress, in a fresh process per run. /usr/bin/time -l reported:
Entries Base e7a17d5 peak RSS This change peak RSS
1,000 9,338,880 bytes 8,732,672 bytes
10,000 23,101,440 bytes 9,437,184 bytes

The 10,000-entry case used about 59% less peak RSS. Every extracted file was compared byte-for-byte with its ZIP payload. These are macOS measurements; an iOS device OOM scenario and non-Apple builds have not been tested locally.

Reproduce the memory measurement

Generate each input archive with Python:

import zipfile

for count in (1000, 10000):
    with zipfile.ZipFile(f"{count}.zip", "w", compression=zipfile.ZIP_DEFLATED) as archive:
        for i in range(count):
            archive.writestr(
                f"file-{i:05d}.txt",
                b"ZIPFoundation memory regression probe\n" * 32,
            )

Use this main.swift in a SwiftPM executable that depends on the checkout being measured:

import Foundation
import ZIPFoundation

let source = URL(fileURLWithPath: CommandLine.arguments[1])
let destination = URL(fileURLWithPath: CommandLine.arguments[2], isDirectory: true)
try autoreleasepool {
    try FileManager.default.unzipItem(at: source, to: destination)
}
let files = try FileManager.default.contentsOfDirectory(atPath: destination.path)
precondition(files.count == Int(CommandLine.arguments[3])!)
print("Extracted \(files.count) files")

Build with swift build -c release, then use a fresh destination directory for each execution:

/usr/bin/time -l .build/release/ZipMemoryProbe 1000.zip output-1000 1000
/usr/bin/time -l .build/release/ZipMemoryProbe 10000.zip output-10000 10000

Compare the maximum resident set size values before and after the change.

Replaces #405 after renaming the source branch; the commits are unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Running out of memory on ios devices with big zip file with a lot of files inside it

1 participant