Introduction to Amazon FSx
Amazon FSx is a family of fully managed file systems designed to address specialized application needs beyond general-purpose cloud storage services.
Addresses the gap between generic cloud storage (S3, EFS) and specific commercial application requirements, particularly those needing specialized file system types like Windows SMB shares or high-performance Lustre.
Provides fully managed, specific file systems without the need to manage underlying infrastructure, hardware failures, or OS patching.
Amazon FSx Service Offerings
Amazon FSx offers several distinct file system types, each optimized for different workloads and use cases.
Amazon FSx for Windows File Server
For migrating existing Windows-based file workloads without code rewrites. It is 100% compatible with Windows environments and presents standard SMB file shares.
compatibility
100% compatible with Windows environments
protocols
Standard SMB file shares
file_systems
Windows NTFS or New Technology File System
integration
Microsoft Active Directory for authentication and permissions
storage_options
SSD (for high-performance, latency-sensitive workloads e.g., SQL Server, VDI), HDD (for general-purpose, cost-effective storage e.g., user home directories)
scalability
Up to hundreds of petabytes of data and millions of IOPS
backup
Daily backups to Amazon S3
Use Cases:
- Migrating existing Windows-based file workloads
- SQL Server
- VDI
- User home directories
Amazon FSx for Lustre
For high-performance computing (HPC) workloads requiring massive throughput and speed. It is based on the popular open-source Lustre parallel file system and is POSIX compliant, working seamlessly with Linux-based applications. It can link directly to S3 buckets, allowing data in S3 to be processed as a mounted file system with high speed and write data back to S3.
file_system
Lustre parallel file system
compatibility
POSIX compliant, works seamlessly with Linux-based applications
s3_integration
Can link directly to S3 buckets
deployment_options
Scratch (for temporary data processing or bursty workloads; data is not replicated), Persistent (for long-term storage, speed, and parallelism for massive jobs)
storage_classes
SSD, intelligent tiering, HDD
scalability
Up to hundreds of gigabytes, millions of IOPS, sub-millisecond latency
Use Cases:
- High-performance computing
- machine learning training
- video rendering
- big data analytics
Amazon FSx for NetApp ONTAP
For enterprises with existing NetApp on-premises infrastructure, providing a fully managed ONTAP experience in the cloud.
ontap_description
Proprietary operating system from NetApp for data storage management, organization, retrieval, and access
protocols
Supports NAS (SMB, NFS) and SAN (iSCSI)
features
Data compression, deduplication, SnapMirror replication
compatibility
Diversified operating systems and services (Linux, Windows, Mac, VMware Cloud on AWS, Amazon WorkSpaces, AppStream 2.0, EC2, ECS, EKS)
Use Cases:
- Migrating existing ONTAP environments
- lift-and-shift of SAN/NAS from on-premises to cloud
Amazon FSx for OpenZFS
For high-performance, data-intensive Linux and Unix environments. OpenZFS is an open-source enterprise-grade file system and logical volume manager known for data integrity, high performance, and flexible storage management.
data_integrity
Generates checksums for data blocks; verifies checksums on read and attempts data healing using redundant copies to prevent 'bit rot.'
performance
Up to 2 million IOPS with latency of hundreds of microseconds
features
Instant point-in-time snapshots, data cloning
compatibility
Broadly accessible from Linux, Windows, Macs, VMware, Cloud services (WorkSpaces, AppStream 2.0, EC2, ECS, EKS)
protocols
Compatible with industry-standard NFS protocols (v3, v4.1, v4.2)
Use Cases:
- Replacing on-premises file servers
- building and running high-performance data-intensive applications
FSx Deployment Options
FSx file systems can be deployed in different configurations to balance cost and availability.
Single AZ
A deployment option that is cheaper and offers faster failover within the availability zone.
cost
Cheaper
failover
Faster failover within the availability zone
Use Cases:
- Development or test environments
Multi-AZ
A deployment option that offers higher availability and is standard for production workloads. It synchronously replicates data across two Availability Zones for high availability and durability. Automatic failover typically occurs under 60 seconds.
cost
Higher
availability
Standard for production workloads
replication
Synchronously replicates data across two Availability Zones for high availability and durability
failover
Automatic failover typically under 60 seconds
scalability
Dynamic scaling of storage capacity and performance without downtime
Use Cases:
- Production workloads requiring high availability
FSx Management Console Demonstration
procedure
This section outlines the steps for creating different FSx file systems via the AWS Management Console.
Demonstration snippets for creating FSx file systems.
Prerequisites
- AWS Management Console access
1
Search for 'FSX' in the AWS Management Console search bar.
To access the Amazon FSx service dashboard.
2
Select the desired FSx file system type (e.g., FSX for NetApp ONTAP, FSX for OpenZFS, FSX for Windows File Server, FSX for Lustre).
To initiate the creation wizard for a specific FSx offering.
3
Configure FSx for Windows File Server: Provide a Name (optional, e.g., CS FSX demo).
4
Configure FSx for Windows File Server: Select Deployment Type: Multi-AZ or Single-AZ (Single-AZ1, Single-AZ2 with NVMe cache).
5
Configure FSx for Windows File Server: Choose Storage Type: SSD or HDD.
6
Configure FSx for Windows File Server: Specify Storage Capacity: Mandatory (e.g., 300GB).
7
Configure FSx for Windows File Server: Configure Provisioned IOPS (for SSD): Automatic (3 IOPS/GB) or user-provisioned (e.g., 10,000).
8
Configure FSx for Windows File Server: Select Throughput Capacity: Selectable (e.g., 256 MB/sec recommended).
9
Configure FSx for Windows File Server: Define Network: VPC, preferred subnet, standby subnet.
10
Configure FSx for Windows File Server: Choose Network Type: IPv4 or dual-stack.
11
Configure FSx for Windows File Server: Configure Windows Authentication: Requires Windows Active Directory (AWS Managed or Self-Managed). (Note: This was a blocker for the demo).
12
Configure FSx for Lustre Creation: Select Deployment and Storage Class: Persistent SSD.
To set up a high-performance file system for HPC workloads.
13
Configure FSx for Lustre Creation: Configure Throughput: Configurable (e.g., 250 MB/sec per terabyte).
14
Configure FSx for Lustre Creation: Specify Storage Capacity: Minimum required based on throughput (e.g., 19.2 TB).
15
Configure FSx for Lustre Creation: Configure Metadata Configuration: Automatic.
16
Configure FSx for Lustre Creation: Select Encryption Option: AWS Key or customer-provided key.
17
Configure FSx for Lustre Creation: Specify Data Repository: Path to S3 bucket.
18
Configure FSx for Lustre Creation: Review and Create File System.