Enforce byte and time limits while streaming imported documents

Project context

A fictional knowledge service imports documents from customer-entered URLs. The importer follows redirects and trusts response metadata too broadly.

Setup prerequisites

• Build a local fetch adapter and controlled HTTP test servers.
• Use synthetic documents and deny network access outside the test allowlist.

Preceding work

Complete these dependencies, or supply their agreed outputs before taking this ticket.

• AFETCH-101 · Accept only explicitly supported document URL schemes
• AFETCH-102 · Bind document import requests to tenant-owned destination policies
• AFETCH-103 · Reject private and special-use destination addresses before connection
• AFETCH-104 · Revalidate every document redirect before following it

Acceptance criteria

• Bound actual streamed bytes regardless of headers.
• Enforce connect, idle and overall deadlines.
• Cancel the upstream stream and remove incomplete staging objects on failure.

Implementation constraints

• Choose explicit local test budgets and an injectable clock.

Verification to include

• Import a document within the declared byte and deadline limits.
• Stream beyond the byte cap or stall and verify cancellation plus cleanup.

Deliverables

• Bounded stream adapter and timeout fixtures

Rollout and recovery

Canary small imports; lower admitted limits while investigating failures.

Value of the work

For the engineer: Practice defensive URL handling and bounded untrusted-input processing.

For the team: Review whether integrations can import content without expanding infrastructure access.

Evidence boundaries

Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.

Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.