Rupato
RUPATO FIELD NOTES · File transfer

How to plan a reliable upload and download workflow for 40,000 files

At 40,000 files, a transfer needs a file list, a recovery plan and a way to prove that everything arrived. Start with the data volume and a measured pilot before choosing a transfer method.

Published 6 August 2026 · Updated 24 October 2026 · Rupato · 3 min read

Estimate the volume honestly

Here is a hypothetical planning example using decimal units: 40,000 files averaging 20 MB contain about 800 GB of data. At a sustained 100 Mbps upload rate, the theoretical transfer time is roughly 17.8 hours: 800 GB × 8 ÷ 0.1 Gbps. That excludes protocol overhead, pauses, retries and any service limits.

Your broadband download speed is not the relevant figure. Measure the upload rate to the actual destination using representative files. Also allow local storage for originals, working files and downloads; extracting an archive can need additional space.

Create a manifest before sending

Use one row per expected file. Record relative path, filename, size, product identifier and view. Where the selected tools support it, add a checksum and name its algorithm. A checksum comparison is meaningful only when both sides use the same method.

Example manifest fields, using hypothetical assets
PathProductViewStatus
batch-01/SH104_side.tifSH104SideVerified
batch-01/SH104_sole.tifSH104SoleRetry required

A total file count alone is insufficient. One missing file and one unintended duplicate can leave the count unchanged. Reconcile identity as well as quantity.

Test failure before committing the catalogue

Run a pilot containing large files, nested folders, repeated filenames in different folders and any non-ASCII names you use. Confirm what happens after an interrupted connection. Does the chosen transfer resume a file, retry it or restart the whole archive?

Ask the receiving service which methods and limits it actually supports. Browser uploads, cloud imports and SFTP are different options, not capabilities to assume. Confirm authentication, access expiry, duplicate handling and who monitors failures.

Split work into verifiable batches

Choose batches that fit how you will reconcile and release the work. Keep each manifest with its batch. Do not upload the whole set again after an uncertain failure; identify what arrived, verify it and retry only the unresolved items.

Agree how replacements are named. Overwriting an existing filename without a version record makes it difficult to tell which source entered production.

Plan the return journey

Delivery needs its own manifest: source identifier, approved output, format and status. Test a representative download and open the files before scheduling the full return. Preserve the verified sources until you have confirmed the agreed retention and backup arrangements. A successful upload message is a milestone, not proof of a complete, usable delivery.

About the author

Rupato publishes practical guidance on product photo editing, production briefs and quality checks. Rupato is operated by Liditech Ltd in the United Kingdom and was founded by Chandran Iyemperumal. Meet the company behind these guides.