Skip to content

keepAlive JSDoc points at stop() for a footgun #4146

Description

@janbuchar
  • The keepAlive option says to "use crawler.stop() to exit the crawler gracefully". stop() only halts dispatch; whatever is still queued stays queued. For the case keepAlive is usually reached for — a crawler fed by another crawler or an external producer — calling stop() when the producer finishes drops everything the consumer had not picked up yet (in a local test, 20 of 20).
  • Point the JSDoc at taskLoopOptions.isFinishedFunction with (await manager.checkReadiness()).status === 'finished' plus the producer's running flag, and say plainly what stop() does. stop({ drain: true }) would be nice but the doc fix is the important part.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions