Stream large note exports with bounded intermediate storage
A member with many notes causes the worker to build one large JSON string and exceed its resource budget.
- Focused work estimate
- 3h 30m + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Streaming · Artifact integrity
Estimated field mix
- Performance engineering40%
- Privacy engineering30%
- Storage systems30%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional learning workspace offers a member a portable copy of their own profile, notes and activity settings. Shared documents contain contributions from other people. The product export policy specifies included fields and shared-record handling; this is an engineering exercise, not a legal portability claim.
Setup prerequisites
- Create synthetic members, private notes and shared-document contributions in a local database.
- Provide fake queue and object-storage adapters with controllable expiry and failure.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- REXPORT-101 · List exportable member fields before serializing database rows
- REXPORT-102 · Define how shared document contributions appear in a personal export
- REXPORT-103 · Bind export creation to the current authorized subject
- REXPORT-104 · Freeze the export selection without holding a long database transaction
Acceptance criteria
- Stream the chosen format with bounded buffering and valid escaping.
- Keep ordering and manifest counts deterministic for the frozen selection.
- Remove incomplete temporary artifacts on controlled failure while preserving retry metadata.
Implementation constraints
- Use a synthetic large-note fixture and declared memory/disk limits; do not require real user content.
Verification to include
- Export the same selection through small and large chunk sizes and compare parsed records and digest.
- Interrupt output after a multibyte value and verify no incomplete artifact becomes downloadable.
Deliverables
- Streaming exporter, resource report and interrupted-write regression
Rollout and recovery
Keep completed artifacts unpublished until final validation; retry writes to a new temporary object identity.
Value of the work
For the engineer: Practice export scoping, consistency, streaming and expiring access without leaking other members’ data.
For the team: Create a reviewable export contract and cross-subject regression suite for an approved product policy.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.