S3 Gateway

Multipart upload

The gateway implements the S3 multipart protocol. Large files are split into parts, uploaded in parallel, and then assembled in the provider's storage. Multipart is the recommended way to push files larger than the single-part cap.

Flow

1. Create: POST /<bucket>/<key>?uploads → the gateway returns an uploadId.
2. Upload parts: PUT /<bucket>/<key>?partNumber=N&uploadId=… with each part as the body. Each returns its own ETag.
3. Complete: POST /<bucket>/<key>?uploadId=… with a body listing all parts in order:

   <?xml version="1.0" encoding="UTF-8"?>
   <CompleteMultipartUpload>
     <Part><PartNumber>1</PartNumber><ETag>"…"</ETag></Part>
     <Part><PartNumber>2</PartNumber><ETag>"…"</ETag></Part>
     <Part><PartNumber>3</PartNumber><ETag>"…"</ETag></Part>
   </CompleteMultipartUpload>

   The gateway returns a composite ETag of the form <md5>-<N>.
4. Abort (optional): DELETE /<bucket>/<key>?uploadId=… removes the upload row and any uploaded part files.

Limits

• Part numbers: 1 through 10000.
• Single-part object size: 5 GB.
• Uploads expire after 7 days of inactivity.

aws-chunked encoding

The gateway transparently decodes Content-Encoding: aws-chunked PUT bodies. The AWS CLI uses this for files > 8 MB; clients don't need to do anything special.