At reInvent 2021, AWS introduced a preview of AWS Backup support for Amazon S3. In February 2022 AWS announced the general availability of the feature. We had some time to evaluate this new functionality, so let’s have a look at how this powerful feature is a gamechanger for your S3 backup strategy.
AWS Backup is a service that allows you to centralize and automate data protection across different AWS services. You can leverage AWS Organizations with AWS Backup to deploy data protection policies to govern backup activities. Before diving deeper, we should start with a quick overview of the four main components used in AWS Backup.
In a backup plan, you define Backup rules and select which resources the backup should contain. In the Backup rules you configure which backup vault will be used to store the recovery points, the backup frequency and the retention period of your recovery point. You also specify which resources the backup plan should cover. AWS Backup supports tag-based resource selection. This means that the resources that should be backed up by the backup plan can be tagged with a specific tag and are then automatically backed up by AWS Backup.
A backup job is the result of a Backup Plan rule and resource selection. It will attempt to backup the selected resources, within the time window defined in the Backup Plan rule. A backup job will fail when it does not finish in the allotted time or when having incorrect permissions. You can use AWS EventBridge to capture failure events emitted from AWS Backup. When AWS Backup is unable to create a continuous backup, it tries to create a snapshot backup instead. AWS advises the use of both continuous and snapshot backups to protect your data.
Is a data store containing the backed up data andupports resource policies that control access to the vault and the data it contains. It has a Lock feature that blocks all delete operations and any updates that alter the retention period of the stored recovery points.
A recovery point is either a snapshot backup or Point-In-Time-Recovery backup. You can use a recovery point to restore data. The data inside the recovery point cannot be edited. Tags and retention period can be changed, if the Backup Vault allows it. A recovery point can have a retention period of up to 100 years.
Overview of the protected resources
Recovery Time Objective (RTO) is the metric that defines the time in which a business process must be restored after a disaster. AWS Backup for Amazon S3 supports continuous backups. With these continuous backups, you can restore your S3 bucket or S3 objects up within 1 second of precision and that for a maximum of 35 days in the past. Note that a restore job is not instant, so restoring data from a large backup can take some time.
A Recovery Point Objective (RPO) is a measure of the maximum tolerable amount of data that a business can afford to lose during a disaster. Using hourly snapshots and continuous backups, you can create a shorter RPO window.
You can restore up to 5 individual items using the S3 URI or the entire bucket. Having the ability to restore an individual item allows for more granular restores. Talking about restoring the data, you can choose to restore to an existing bucket or create a new bucket with the restored data. A bucket can only be restored in the same region as the original bucket. Another trade-off is that only the current version of an object will be restored and none of the previous versions.
Continuous Data Protection is a backup and recovery storage system in which all the data is backed up whenever any change is made. While AWS Backup for S3 supports continuous backups, it does not capture the changes immediately.
AWS S3 backups are encrypted using an AWS KMS key associated with the backup vault. The AWS KMS key can either be a customer-managed key or AWS-managed key. AWS Backup encrypts all backups, even if the source buckets are not encrypted. AWS Backup also encrypts the recovery points when at rest.
Another security feature is AWS Backup Vault Lock. This enforces a write-once, read-many setting for all the backups in a backup vault. AWS Backup Vault Lock adds a layer of defence that protects a backup vault from inadvertent or malicious deletes and of updates to the retention period. Alternatively, resource policies can be configured to protect your backup vaults.
When restoring S3 data, you can choose the original encryption keys. In case that there was no original encryption, the object is restored without encryption. You can also choose a substitute encryption key when the original key is not available. Finally, you can pick an Amazon S3 Key or KMS key.
AWS Backup for S3 only supports Versioned Buckets. This is listed as a best practice in the S3 best practices. When using versioned buckets, it is also considered a best practice to set the lifecycle configuration to manage your non-current versions.
AWS Backup creates a full backup as the first backup and incremental backups on top of that full backup. It’s important to have a retention period that is at least as long as the time it takes to take a backup. If not, then AWS Backup will keep creating full backups instead of incremental backups. If the original full backup has reached the end of its lifecycle and has been deleted, the incremental backups still retain the necessary reference data to allow a full restore.
With AWS Backup for S3, you can manually create an on-demand backup. These backups are not automated and need to be initiated by the user. This can jumpstart your S3 backup strategy if you don’t have the resources to automate this process. Note that there is no support for on-demand continuous backups. This is because on-demand backup records a point in time whereas continuous backup records changes over a period of time.
At $0.05/GB-Month, the storage cost of AWS Backup is more than twice the price of S3 Standard Storage Cost ($0.023/GB). When backing up a bucket with a cheaper storage tier like S3 Infrequent Access, then the difference is even bigger. Looking at the cold storage cost of other data sources supported by AWS Backup like EFS ($0.05/GB-month warm storage vs $0.01/GB-month cold storage), it seems like S3 could benefit from the cold storage options as well.
Cross-account or cross-region replication still requires additional configuration. AWS Backups supports Cross-Region and Cross-Account backups for other data sources like AWS EFS and EBS. This can be achieved for S3 using S3 ReplicationConfiguration, but it would be great if this is integrated with AWS Backup. You can find more information on S3 replication here.
Disabling S3 ACL by configuring the ACL configuration as Bucket Owner Enforced fails all restore jobs for the bucket. This ACL configuration is a recommended configuration for AWS S3 buckets, so it’s sad to see that the configuration is causing AWS Backup failures. However, setting the ACL configuration less strict as Bucket Owner Preferred, will not fail the restore operation.
There is an example repository. In this example repository, you will find a CDK application to deploy a sample application. The AWS Backup for S3 documentation mentions the IAM policies required to create and restore a backup. This has been included in the application.
The application creates a sample S3 bucket with two AWS Backup Plans and two AWS Backup vaults. To follow the steps of this demo, you will need to have uploaded some data to the S3 bucket created by the application and wait for approximately 1 hour to have a usable recovery point with status Available.
We’ll restore a folder that we “accidentally” deleted from our S3 bucket. Our bucket contains two folders, representing the data from 2 tenants of our AWS account.
We’ll delete the data of user1. We can create a restore job from our AWS Backup Vault containing a recovery point. Furthermore, we provide the S3 URL path of our S3 folder and wait until the job is completed.
We see that the restore job is completed and that our item should have been restored.
Looking at the versions of one of the restored objects, we can see that a new current version has been set and so reverting the previous delete operation.
AWS S3 plays a key role to provide organizations reliable and highly durable object storage. AWS Backup for S3 enables you to automate and securely store S3 backups in vaults protected by AWS Backup Vault Lock or resource policies. You can set a retention period for your backups to comply with compliance requirements. AWS Backup can assist you in defining a RPO and RTO for your S3 data in your organization.
Contact Cloudway for more information.