Search This Blog

Monday, November 28, 2016

Part 3 - AWS - EBS (Elastic Block Storage)

EBS: (Elastic Block Storage) (Disk in a Cloud)

Provides block-level storage volumes: -
Amazon EBS allows you to create storage volumes and attach them to Amazon EC2 instances. Once attached, you can create a file system on top of these volumes, run a database, or use them in any other way you would use a block devices.  Supports up to 16 TB.
Cost is calculated from GB/month of total allocated size
·        Even if nothing is stored on it.
Ability to store information off an EC2 instance.
·        Volumes can be attached and detached to EC2 instances
·        Volumes persist after terminating an instance.

Amazon EBS volumes are placed in a specific Availability Zone
Can only be attached to instances within the same zone
·        Can only be attached to one instance at a time
·        Many volumes can be attached to a single instance
Volumes are automatically replicated across AWS servers within the zone
·        To improve reliability, availability, and durability
·        Prevents data loss due to single hardware failure
o   Not designed for durability or considered highly durable
o   Designed to be better than storing data on a single hard drive
·        Snapshots can be used to provide highly durable backups

EBS Volumes Types

·        General Purpose SSD (GP2)
o   Designed for 99.999% availability
o   Ratio of 3 IOPS per GB with up to 10000 IOPS and the ability to burst up to 3000 IOPS for short periods for volumes under 1 Gib.
·        Provisioned IOPS SSD (I01) (Input output Per Second)
o   Designed for I/O intensive applications such as large relational or NoSQL databases. Use if you need more than 10,000 IOPS
·        Magnetic (Standard)
o   Lowest cost per gigabyte to all EBS volumes types. Magnetic volumes are ideal for workloads where data is accessed infrequently, and applications where the lowest storage cost is important.

Creating EBS Volume

Attaching an EBS Volume


Mounting an EBS Volume

·        Once attached, a volume must be mounted in the actual instance
o   Depends on OS
·        Volume must also be formatted before use
o   New volume appear as zeroed-out raw block-level storage
o   Can format as any file system desired
§  Depends on OS

Snapshot

·        EBS volumes can be backed up
        A point-in-time snapshot
·        Stored in S3
        Provides very high durability
        More on S3 shortly
·        Snapshots are incremental backups to reduce storage requirements
        Snapshots only store data that changed since last snapshot
        All snapshots have access to all data to restore to that snapshot
        Deleting a snapshot only deletes data unique to that snapshot
·        A snapshot can be restored to a new EBS volume

Volume Vs Snapshots

·       Volumes exist on EBS
o   Virtual Hard Disk
·        Snapshots exist on S3
·        You can take a snapshot of volume, this will store that volume on S3.
·        Snapshots are point in time copies of Volumes.
·        Snapshots are incremental, this means that only the blocks that have changed since your last snapshot are moved to S3.
·        If this is your first snapshot, it may take some time to create.

Volumes vs Snapshots - Security

·       Snapshots of encrypted volumes are encrypted automatically.
·        Volumes restored from encrypted snapshots are encrypted automatically.
·        You can share snapshots, but only if they are unencrypted.
o   These snapshots can be shared with other AWS accounts or made public.

Snapshots of Root Device Volumes

·       To create a snapshot for Amazon EBS volumes that server as root devices, you should stop the instance before taking the snapshot.

RAID, Volume & Snapshots

·       RAID=Redundant Array of Independent Disks
o   RAID 0 - Striped, No Redundancy, Good Performance.
o   RAID 1 - Mirrored, Redundancy
o   RAID 5 - Good for reads, bad of writes, AWS does not recommend every putting RAID 5's on EBS
o   RAID 10 - Striped & Mirrored, Good Redundancy, Good Performance.

How can I take a Snapshot of a RAID Array?

·       Problem - Take a snapshot, the snapshot excludes data held in the cache by applications and the OS. This tends not to matter on a single volume, however using multiple volumes in a RAID array, this can be a problem due to interdependencies of the array.
·        Solution - Take an application consistent snapshot.
o   Stop the application from writing to disk
o   Flush all caches to the disk
·        How can we do this?
o   Freeze the file system
o   Unmount the RAID array
o   Shutting down the associated EC2 instances

Creating Snapshots

·        Management Console
        Three ways to create a snapshot
       From EBS Volumes page, right-click volume and select Create Snapshot
       From EBS Volumes page, select Actions | Create Snapshot
       From EBS Snapshots page, select the Create Snapshot button


·        Command line:
·        Creating Snapshots
Syntax: #aws ec2 create-snapshot –-volume-id <vol_id> –-description <description>
Example: #aws ec2 create-snapshot –-volume-id vol-cf674da4 –-description "June05Snap"

Creating Volumes from Snapshots

·        From Management Console
        Can specify a snapshot when creating a new volume (shown earlier)
        Or

        Same as before, except add a –-snapshot-id parameter
# aws ec2 create-volume --size <size> --snapshot-id <snapshot> --availability-zone <zone> --volume-type <type>

Increasing EBS Volumes Size

·        A volume size cannot be increased directly, but it can be done easily
·        Increasing volume size requires four steps
o   Create a snapshot of the volume
o   Create a new volume from that snapshot with a larger size
o   Attach the new volume to an instance
o   Configure the file system for the new volume size
       This is the only new step
·        Performing Step 4 will depend on the operating system
o   The following slides assume that a new 3-GB volume was created from a 2-GB snapshot

Reducing EBS Volume Size

·        Reducing volume size is different than increasing size
o   Volumes created from a snapshot cannot be smaller than the snapshot
·        Reducing volume size is more of a manual process
·        Create a new EBS volume of the desired size
o   The new volume must be big enough to hold the amount of data!
·        Attach both the original and new volume to the same instance
o   Format the new volume as needed
·        Copy the contents from the original volume to the new volume
o   Linux: Use cp or rsync 
o   Windows: Use Windows Explorer or xcopy
·        Recommended to then create a snapshot of the new volume

Moving EBS Volumes across Zones

Moving volumes across availability zones within the same region is easy
·        Create a snapshot of the original volume
·        Create a new volume in a different zone from the snapshot
o   Snapshots are not associated with an availability zone

Moving EBS Volumes Across Region

·        Moving volumes across regions is also easy
o   Create a snapshot of the original volume
·        From the Snapshots screen, select the snapshot, click Actions, and select Copy
o   Select the region to copy to
·        This will incur bandwidth charges

Encrypting EBS Volumes

·        EBS supports encrypting volumes
o   From the Create Volume screen, select “Encrypt this volume”
§  Available keys are managed from the IAM console
·        Encryption/decryption performed by Amazon
·        Can also encrypt the volume manually
o   EBS volumes are just block-level devices
o   Can use operating system modules to encrypt files or entire file systems
§  Windows Encrypting File System (EFS)
§  Various Linux/UNIX cryptographic file systems

EBS Limits

·        Each account has an EBS limit of a total size of 20 TB for each volume type
·        Higher limits can be requested at
o   http://aws.amazon.com/contact-us/ebs_volume_limit_request
·        Requests are for a specific region

Exam Tips EBS

·        EBS Consists of;
o   General Purpose SSD - GP2 - (Up to 10,000 IOPS)
o   Provisioned IOPS SSD - IO1-(More than 10,000 IOPS)
o   Magnetic - cheap, infrequently accessed storage)

·        You cannot mount 1 EBS volume to multiple EC2 instances, instead use EFS.

No comments:

Post a Comment