When you want to upload a large file,
you can split it into smaller parts and
upload them using the Upload Part API.
Create Upload Session
First, . The resulting
object defines the size of each part and the number of parts to upload.
Split File
Split the file into parts to be uploaded.
If you want to use the command line,
use the split command:
For example:
This will result in your file divided into several files.
Get SHA Digest
To get the value for the SHA digest,
use the following openSSL command
to encode the file part:
For example:
The result is a base-64 encoded message used to verify the upload.
Upload Part
Upload the bytes for the part you want to upload, specifying the byte
range for the part and the SHA digest to ensure the content is
uploaded correctly.
Content Range
Each part’s size must be exactly equal in size to the part size
specified in the upload session that you created.
One exception is the last part of the file, as
this can be smaller. The Content-Range parameter
definition follows this pattern:
When providing the value for Content-Range, remember that:
- The lower bound of each part’s byte range must be a multiple of the part size.
- The higher bound must be a multiple of the part size - 1.
For example, if the part size is 8388608,
the content range for the first two parts will be:
Response
After each upload, the resulting response includes
the ID and SHA of the part uploaded.
Keep all the JSON responses from all part
uploads as they are needed to .
Range Overlap
If a part upload request fails with any error code
range_overlaps_existing_part then the application
made a mistake in cutting up the file into parts
and tried to upload a part into a range that already had
content uploaded for it. The application should assume
that this last part was not persisted to the session.
Parallel uploads
Although you can upload the parts in parallel, try to upload them in
order as much as is possible. Parts with a lower byte offset should be uploaded
before parts with a higher byte offset.
The recommended approach is to upload 3 to 5 parts in parallel from a queue
of parts, ordered by byte offset. If a part upload fails, retry it
before you upload further parts.