What is AWS S3 Bucket?

Amazon Simple storage service allows you to store objects or any sizes securely and with good performance, scalability and securely. You can ideally store unlimited data into AWS S3 bucket. Lets get into some of the important features of AWS S3 bucket.

There are various S3 storage classes which can be used according to the requirements.
You can also configure Storage lifecycle which allows you to manage your objects efficiently and you can move the objects to different storage classes.
S3 object lock: you can add a object lock for a particular time so that the objects are not deleted by mistake.
S3 replication: you can replicate objects to different destinations may be in different buckets or different regions accordingly.
S3 batch operations: you can manage lot of objects in a single API request using batch operations.
You can block public access to S3 buckets and object. By default, Block Public Access settings are turned on at the account and bucket level.
You can apply IAM policy to users or roles to access 3 bucket securely. You can also apply resource based policy on AWS s3 buckets and objects.
You can also apply access control list on a particular bucket or a particular objects.
You can disable ACL and take ownership of every object in your bucket. As a bucket error you have rides on every object in your bucket.
You can also use access analyzer for S3 two evaluate all the access policies
You can have up to 100 buckets in your AWS account
When is the bucket is created you are not allowed to change the name afterwards or the region.
Every object is identified by a name that is a key and a version ID and every object in bucket has exactly one key.

You can access your bucket using the Amazon S3 console using both virtual-hosted–style and path-style URLs to access a bucket.

https://bucket-name.s3.region-code.amazonaws.com/key-name (Virtual Hosted )

https://bucket-name.s3.region-code.amazonaws.com/key-name ( Path Based )

AWS S3 Bucket Access Control List

You can set the bucket ownership and S3 object ownership in AWS S3 bucket level settings and can disable ACL so that you are owners of every object.
When any other AWS account upload the objects in AWS S3 in your account then that account owns the bucket and has access to it but if you disable ACL then bucket owner automatically owns every object in your bucket.

S3 Object Ownership is an Amazon S3 bucket-level setting that you can use to disable access control lists (ACLs) and take ownership of every object in your bucket, simplifying access management for data stored in Amazon S3.

AWS S3 Object Encryption

Amazon AWS S3 encryption is done in transit and at rest. Server-side encryption encrypts the object before saving it and decrypts when you download it.

Server-side encryption with Amazon S3 managed keys (SSE-S3)
Server-side encryption with AWS Key Management Service (AWS KMS) keys (SSE-KMS)
Server-side encryption with customer-provided keys (SSE-C)

Client side encryption can be done before sending objects to as 3 bucket.

AWS S3 Bucket Policy

In AWS S3 bucket policy is a resource-based policy which allows you to grant permission to your bucket and objects only bucket owner of that account can associate a policy with the bucket and bucket policies a based on access policies.

AWS s3 bucket policy examples

In this section we will go through some of the examples of bucket policy. With bucket policy you can secure access to objects in your buckets, so that only users with the appropriate permissions can access them

s3 bucket policy to encrypt each object with server-side encryption using AWS Key Management Service (AWS KMS) keys (SSE-KMS)

To require server-side encryption of all objects in a particular Amazon S3 bucket, you can use a bucket policy.

{
   "Version":"2012-10-17",
   "Id":"PutObjectPolicy",
   "Statement":[{
         "Sid":"DenyUnEncryptedObjectUploads",
         "Effect":"Deny",
         "Principal":"*",
         "Action":"s3:PutObject",
         "Resource":"arn:aws:s3:::DOC-EXAMPLE-BUCKET1/*",
         "Condition":{
            "StringNotEquals":{
               "s3:x-amz-server-side-encryption":"aws:kms"
            }
         }
      }
   ]
}

s3 bucket policy which require SSE-KMS with a specific AWS KMS key for all objects written to a bucket

{
"Version": "2012-10-17",
"Id": "PutObjPolicy",
"Statement": [{
  "Sid": "DenyObjectsThatAreNotSSEKMSWithSpecificKey",
  "Principal": "*",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*",
  "Condition": {
    "ArnNotEqualsIfExists": {
      "s3:x-amz-server-side-encryption-aws-kms-key-id": "arn:aws:kms:us-east-2:111122223333:key/01234567-89ab-cdef-0123-456789abcdef"
    }
  }
}]
}

Grant cross-account permissions to upload objects while ensuring that the bucket owner has full control

{
   "Version":"2012-10-17",
   "Statement":[
     {
       "Sid":"PolicyForAllowUploadWithACL",
       "Effect":"Allow",
       "Principal":{"AWS":"111122223333"},
       "Action":"s3:PutObject",
       "Resource":"arn:aws:s3:::DOC-EXAMPLE-BUCKET/*",
       "Condition": {
         "StringEquals": {"s3:x-amz-acl":"bucket-owner-full-control"}
       }
     }
   ]
}

How to remove bucket content completely using aws s3 rm

To remove bucket content completely run the below command.

aws s3 rm s3://bucket-name –recursive

Deleting a AWS S3 bucket – How you can delete an empty Amazon S3 bucket run the below command.

aws s3 rb s3://bucket-name --force

How to transform data with S3 object Lambda

To Transform the data with AWS S3 Object Lambda follow the below steps:

Prerequisites
Step 1: Create an S3 bucket
Step 2: Upload a file to the S3 bucket
Step 3: Create an S3 access point
Step 4: Create a Lambda function
Step 5: Configure an IAM policy for your Lambda function’s execution role
Step 6: Create an S3 Object Lambda Access Point
Step 7: View the transformed data
Step 8: Clean up

List S3 Bucket using using the AWS S3 CLI command ( aws s3 list bucket or AWS S3 ls )

To list the bucket using AWS CLI then use the below command. The below command lists all prefixes and objects in a bucket

aws s3 ls s3<strong>:</strong>//mybucket

AWS S3 Sync

Syncs directories and S3 prefixes. Recursively copies new and updated files from the source directory to the destination. Only creates folders in the destination if they contain one or more files.

The following sync command syncs objects under a specified prefix and bucket to files in a local directory by uploading the local files to s3

aws s3 sync . s3://mybucket

AWS S3 cp recursive

To list the bucket using AWS CLI then use the below command.

aws s3 mv

Moves a local file or S3 object to another location locally or in S3. The following mv command moves a single file to a specified bucket and key.

aws s3 mv test.txt s3://mybucket/test2.txt

Conclusion

In this tutorial we learned important concepts of AWS S3 such as its use, bucket policy and features of AWS S3 bucket.

DevOps Automateinfra Learning

Most Trending and Quality DevOps Cloud Automation tutorials, guides and learning

What is AWS S3 Bucket?

Table of content