Multipart Upload
In this article:
Multipart Upload#
We recommend uploading large files in parts. This allows you to upload file parts in parallel and avoid the need to restart uploading of the entire file if the process fails. After uploading, the whole object is assembled from the parts.
Multipart upload is recommended when the file size exceeds 100 MB. Each part of the file (except for the last) must not be less than 5 MB.
CreateMultipartUpload#
Initiates multipart uploading.
The request returns Upload, the multipart upload ID, which must be used in all subsequent operations on the multipart object.
Sample Request#
Request
POST /bucket1/os-image.iso?uploads=null HTTP/1.1
Host: s3.k2.cloud
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220602T125526Z
Authorization: AWS4-HMAC-SHA256 Credential=project:user@company/20220602/{region}/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=15b7af0199009f18d33e9b94641ca5bc216dba6da083bdfa6de563571a604788
Sample Response#
Response
HTTP/1.1 200 OK
x-amz-request-id: tx000000000000009a52dc0-006298b33e-41e627a3-default
content-type: application/xml
content-length: 268
date: Thu, 02 Jun 2022 12:55:26 GMT
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Bucket>bucket1</Bucket>
<Key>os-image.iso</Key>
<UploadId>2~uBeFr6eZPc5DhCXYI-hAua_wGd4Ggwi</UploadId>
</InitiateMultipartUploadResult>
UploadPart#
Uploads a part of the object into the specified bucket. You must specify the part number and upload ID in the request. Individual parts can be uploaded in any order.
The response returns an ETag. It, along with the part number, must be specified when assembling the object from parts.
Sample Request#
Request
PUT /bucket1/os-image.iso?partNumber=1&uploadId=2~uBeFr6eZPc5DhCXYI-hAua_wGd4Ggwi HTTP/1.1
Host: s3.k2.cloud
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220602T130547Z
Authorization: AWS4-HMAC-SHA256 Credential=project:user@company/20220602/{region}/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=3633648f05c9b8d542618071aaa64d30f166ff75b8a8a804d52b84b74943c8ad
Content-Type: application/octet-stream
Content-Length: 22
"<file contents here>"
Sample Response#
Response
HTTP/1.1 200 OK
content-length: 0
etag: "aa2d29fda013392e244bb34a2c3acf66"
accept-ranges: bytes
x-amz-request-id: tx000000000000009a870f3-006298b56a-41e7b39b-default
date: Thu, 02 Jun 2022 15:29:35 GMT
ListMultipartUploads#
Lists all current multipart uploads into the specified bucket.
Sample Request#
Request
GET /bucket_1?uploads=null HTTP/1.1
Host: s3.k2.cloud
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220606T145616Z
Authorization: AWS4-HMAC-SHA256 Credential=project:user@company/20220606/{region}/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=afa0b0d06e69620bed2eb300919be37c5b5840b9af9d9934ac3beec844024417
Sample Response#
Response
HTTP/1.1 200 OK
transfer-encoding: chunked
x-amz-request-id: tx00000000000000f8a1079-00629e1590-41e627a3-default
content-type: application/xml
date: Mon, 06 Jun 2022 14:56:16 GMT
<?xml version="1.0" encoding="UTF-8"?>
<ListMultipartUploadsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Bucket>bucket1</Bucket>
<NextKeyMarker>os-image.iso</NextKeyMarker>
<NextUploadIdMarker>2~YvXN5XszdRN4lMHQOctMjjTbI-w-BC0</NextUploadIdMarker>
<MaxUploads>1000</MaxUploads>
<IsTruncated>false</IsTruncated>
<Upload>
<Key>os-image.iso</Key>
<UploadId>2~BmzJSmosTIF8H9_FyP_2UKPLgjPLJU7</UploadId>
<Initiator>
<ID>b5e1b8d4-4886-4d03-a1b4-e03682a4ed8e</ID>
<DisplayName>user@company</DisplayName>
</Initiator>
<Owner>
<ID>b5e1b8d4-4886-4d03-a1b4-e03682a4ed8e</ID>
<DisplayName>user@company</DisplayName>
</Owner>
<StorageClass>STANDARD</StorageClass>
<Initiated>2022-06-02T12:54:00.879Z</Initiated>
</Upload>
</ListMultipartUploadsResult>
ListParts#
Returns the list of the object parts with their ETags.
Sample Request#
Outputs information about the first two parts of an object.
Request
GET /bucket1/os-image.iso?max-parts=2&uploadId=2~uBeFr6eZPc5DhCXYI-hAua_wGd4Ggwi HTTP/1.1
Host: s3.k2.cloud
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220602T132141Z
Authorization: AWS4-HMAC-SHA256 Credential=project:user@company/20220602/{region}/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=cd88e3c159bd450f3b2cb428ee28bfdca8944ac28247fe9ec5ab8b9ca17def6a
Sample Response#
Response
HTTP/1.1 200 OK
transfer-encoding: chunked
x-amz-request-id: tx000000000000005361844-006298b965-41e72c37-default
date: Thu, 02 Jun 2022 13:21:41 GMT
<?xml version="1.0" encoding="UTF-8"?>
<ListPartsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Bucket>bucket1</Bucket>
<Key>os-image.iso</Key>
<UploadId>2~uBeFr6eZPc5DhCXYI-hAua_wGd4Ggwi</UploadId>
<StorageClass>STANDARD</StorageClass>
<PartNumberMarker>0</PartNumberMarker>
<NextPartNumberMarker>2</NextPartNumberMarker>
<MaxParts>2</MaxParts>
<IsTruncated>true</IsTruncated>
<Owner>
<ID></ID>
<DisplayName></DisplayName>
</Owner>
<Part>
<LastModified>2022-06-02T13:05:47.910Z</LastModified>
<PartNumber>1</PartNumber>
<ETag>"aa2d29fda013392e244bb34a2c3acf66"</ETag>
<Size>104857600</Size>
</Part>
<Part>
<LastModified>2022-06-02T13:13:04.044Z</LastModified>
<PartNumber>2</PartNumber>
<ETag>"9ca5cd796145086b1e0fd66db4821463"</ETag>
<Size>104857600</Size>
</Part>
</ListPartsResult>
CompleteMultipartUpload#
Completes the object uploading by concatenating all its parts. Part numbers and their ETags must be specified.
Sample Request#
Request
POST /bucket1/os-image.iso?uploadId=2~uBeFr6eZPc5DhCXYI-hAua_wGd4Ggwi HTTP/1.1
Host: s3.k2.cloud
X-Amz-Date: 20220602T133803Z
Authorization: AWS4-HMAC-SHA256 Credential=project:user@company/20220602/{region}/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=0792421fd7e56be60d7d5e121fdff8f90033401662da359c5ed9f6b52e0d72da
Content-Type: text/plain
Content-Length: 363
<CompleteMultipartUpload>
<Part>
<PartNumber>1</PartNumber>
<ETag>"aa2d29fda013392e244bb34a2c3acf66"</ETag>
</Part>
<Part>
<PartNumber>2</PartNumber>
<ETag>"9ca5cd796145086b1e0fd66db4821463"</ETag>
</Part>
<Part>
<PartNumber>3</PartNumber>
<ETag>"19e038d58285587d3de4355544aa8117"</ETag>
</Part>
</CompleteMultipartUpload>
Sample Response#
Response
HTTP/1.1 200 OK
x-amz-request-id: tx00000000000000538ba7a-006298bd3b-415d5bdc-default
content-type: application/xml
content-length: 342
date: Thu, 02 Jun 2022 13:21:41 GMT
<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Location>s3.k2.cloud/bucket1/os-image.iso</Location>
<Bucket>bucket1</Bucket>
<Key>os-image.iso</Key>
<ETag>66df11da26f6095f81882b8574f6c977-3</ETag>
</CompleteMultipartUploadResult>
AbortMultipartUpload#
Aborts the multipart upload. Uploaded parts are deleted from the object storage.
If any parts are being uploaded at the time when you run the operation, then you may need to re-run it to delete the newly uploaded parts. To verify that all parts have been deleted, run ListParts.
Sample Request#
Request
DELETE /bucket1/os-image.iso?uploadId=2~lwX4Y6971D50TbFwoRedaSl8pGvFW-c HTTP/1.1
Host: s3.k2.cloud
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20220602T155018Z
Authorization: AWS4-HMAC-SHA256 Credential=project:user@company/20220602/{region}/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=42ed169db8b5c7123d3650a1c15df5592752284a6cd87e7b2f05f6cbbd571c59
Sample Response#
Response
HTTP/1.1 204 No Content
x-amz-request-id: tx000000000000009be9d8c-006298db1e-41e627a3-default
date: Thu, 02 Jun 2022 15:45:35 GMT