Core Concepts of AWS Storage Gateway
AWS Storage Gateway is a hybrid storage service that connects on-premises applications with AWS cloud storage, addressing the challenge of integrating existing on-premises infrastructure with cloud storage services without application refactoring.
Seamlessly integrate existing on-premises infrastructure with AWS cloud storage services (like Amazon S3) without rewriting applications. Facilitates secure and efficient interaction with AWS cloud storage for applications using traditional protocols (SMB, NFS, iSCSI).
AWS Storage Gateway acts as a bridge and translator, providing a local cache and reducing the need for expensive on-premises hardware.
The service reduces costs by migrating off expensive on-premises hardware and improves backup and disaster recovery processes. It provides low-latency access to data stored in the cloud, combines infinite scalability and durability of AWS storage with local performance needs, encrypts data in transit and at rest, and provides local caching for frequently accessed data.
Storage Gateway can be deployed as a Virtual Machine, typically on local hypervisors (VMware ESXi, Microsoft Hyper-V, Linux KVM), or as a Hardware Appliance, which are physical hardware appliances available for environments without virtualized platforms.
Technical Specs: Deployment options: Virtual Machine (on VMware ESXi, Microsoft Hyper-V, Linux KVM), Hardware Appliance (physical)
The gateway presents standard storage interfaces to applications, manages data transfer to AWS, handles networking, caching, and communicates with AWS services like Amazon S3, S3 Glacier, EBS, and AWS Backup.
Hybrid storage solutions, like AWS Storage Gateway, offer flexibility by integrating on-premises and cloud storage. Cloud storage itself refers to the on-demand availability of computer storage space and data access over a network, eliminating the need for physical data centers.
AWS Storage Gateway Types
AWS Storage Gateway offers several specialized types, each designed for specific on-premises storage needs and integrating with different AWS cloud storage services.
Storage Gateway supports three key storage interfaces: File, Volume, and Tape, which correspond to its various gateway types.
S3 File Gateway
The S3 File Gateway is designed for Network Attached Storage (NAS) needs, presenting file shares to on-premises users and applications. When a user saves a file, the gateway uploads it to Amazon S3 as an object, while metadata is cached locally for fast lookups. It sits on-premises and integrates with the AWS cloud via Direct Connect, Site-to-Site VPN, or the public internet, encrypting data in transit.
protocols
NFS, SMB
connectivity
Direct Connect, Site-to-Site VPN, or public internet
aws_storage_tiers_supported
S3 Standard, S3 Standard-IA, S3 One Zone-IA, S3 Intelligent-Tiering. Supports S3 Glacier for archiving.
authentication
Can integrate with Active Directory (AD)
Use Cases:
- Data migration and ingestion of on-premises file data to Amazon S3 (for data lakes, archiving)
- Offloading infrequently accessed data from on-premises NAS to S3, reducing local storage costs
- Providing a centralized, scalable, cost-effective cloud storage solution for on-premises file storage needs
- Enabling hybrid cloud workflows for analytics, machine learning, or other cloud-based processing of on-premises generated data
Amazon FSx File Gateway
The Amazon FSx File Gateway provides low-latency and efficient access to in-cloud Amazon FSx for Windows File Server from on-premises applications. It acts as a gateway to a fully managed, highly reliable FSx for Windows File Server in AWS, caching frequently accessed data locally for low-latency access.
connectivity
Requires AWS Direct Connect or Site-to-Site VPN connectivity. Does not work with public internet.
supported_windows_capabilities
SMB, NTFS, Active Directory authentication
Volume Gateway
The Volume Gateway provides block-level storage via the iSCSI protocol, acting like a Storage Area Network (SAN). It operates in two modes: Cache Mode and Stored Mode.
protocol
iSCSI
cache_mode_operation
Only frequently accessed data is kept on local cache disk, main copies in S3 as EBS snapshots.
cache_mode_on_premises_storage
Requires lesser on-premises volumes (for hot data caching).
stored_mode_operation
Primary data stored locally on provided disks; asynchronous, point-in-time snapshots saved as EBS snapshots in S3 for disaster recovery.
stored_mode_on_premises_storage
Requires provisioning and configuring all local storage.
Use Cases:
- Cache Mode: Low-latency access for most used data, combined with S3 durability/cost-effectiveness.
- Stored Mode: Applications requiring extremely low latency and protection of local databases with offsite snapshots.
Tape Gateway
The Tape Gateway replaces physical tape backups for long-term archiving, often driven by compliance requirements. It eliminates physical tape libraries by using an iSCSI interface (Virtual Tape Library - VTL). Backup servers write to the virtual tape library, and data is copied over HTTPS to a tape library in Amazon S3. When a virtual tape is ejected, it can be automatically pushed to S3 Glacier or S3 Glacier Deep Archive for cost-effective long-term retention. Data is first staged in S3, it does NOT directly write to S3 Glacier or Deep Archive.
interface
iSCSI (Virtual Tape Library - VTL)
data_copy_protocol
HTTPS
workflow_storage_in_aws
Virtual tapes first stored in Amazon S3, then can be automatically pushed to S3 Glacier or S3 Glacier Deep Archive upon ejection.
Use Cases:
- Replacing physical tape-based backup with a simple cloud solution
- Long-term data archival meeting compliance requirements
- Cost-effective off-site backup storage
Physical Hardware Appliance Option
AWS offers an alternative deployment option for environments where virtualization is not feasible.
AWS offers physical hardware appliances for branch offices without a virtualization environment.
These appliances can be ordered directly from Amazon or AWS.
They are pre-validated, optimized servers with appropriate CPU, RAM, and SSD caching for performance. Essentially, they are servers running a hypervisor to host the gateway VM.
Technical Specs: Pre-validated, optimized servers with appropriate CPU, RAM, and SSD caching
The physical hardware appliance simplifies procurement and deployment in environments where local virtualization management is not feasible.