Tuesday, April 4, 2017

3 Million Storage IOPS on AWS Cloud Instance

I3 Instance Family
NVMe Technology
Linux Block layer
I3 Storage Benchmark Results

AWS I3 Instance Family

AWS has always been at forefront of adopting new and advanced Intel technologies into their fleet of cloud instances. Introduction of next generation of I3 instance family has continued this tradition by offering low latency and high performance IO features for both storage and network. I3 instances comes with direct attached NVMe (Non-Volatile Memory PCIe) SSD storage. Due to no virtualization overhead and direct access to NVMe storage, I3 instances can able to achieve unprecedented 15GB/s read storage throughput and over 3 million read IOPS. Direct Memory Access (DMA) to storage keeps latencies low (< 100 us) even during moderate IO load.
NVMe SR-IOV extension allows splitting the storage drives across VM (instances). Instead of using Xen virtualized split driver model that is prone to higher latencies, cloud instance runs native nvme storage driver to access subset of PCI resources on a physical PCI IO board. Data transfer between driver and hardware is handled via low latency DMA path that does not require hypervisor intervention. Also, Intel VT-d support for re-mapping of Device DMA access and device generated interrupts helps cloud provider to isolate and partition IO resources and assign them to a specific cloud instance without compromising the integrity of the underlying hardware.
Other noteworthy I3 instance family features are:

  • Advanced Intel Broadwell processors
  • Support for 64 vcpus, 488 GB DDR4-based memory, 15 TB of NVMe local storage
  • SR-IOV based networking using Elastic Network Adaptor (ENA) offers 20 Gbps net throughput and over 2 Million Packets Per Second processing for low latency networking
  • EBS-optimized instance

NVMe Technology and Features

NVMe protocol supports multiple hardware queues, which is an advancement over traditional SAS and SATA protocols. Typical SAS devices support up to 256 commands and SATA devices support up to 32 commands in a single hardware queue. NVMe supports 64K commands per queue and up to 64K queues per device. NVMe queues are designed such that I/O commands and responses operate on the same processor core and thus can take advantage of warm cpu caches, locality as well as parallel processing capabilities of multi-core processors. Each application or thread running on the cpu gets a separate queue bound to that cpu, and with Linux block layer multi-queue support, no I/O locking is required to process IO. NVMe also supports MSI-X and interrupt steering, to distribute interrupt processing across multiple CPU, that improves scalability.

In addition, NVMe uses simple command set that take half the number of CPU instructions to process an I/O request that SAS or SATA does, providing higher IOPS per CPU instruction cycle and lower I/O latency.


Linux Block Layer

When application issues IO requests, Linux block layer moves requests from per-cpu submission queues into hardware queues, up to the maximum number specified by the driver. NVMe devices supports multiple queues (AWS sets different number of hardware queues for each I3 instance type).
Instance Type
Hardware Queues per Device
Linux Software Queues
Number of nvme Devices
Capacity
Total HW Queues
i3.xl
4
4
1
0.8T
4
i3.2xl
8
8
1
1.7T
8
i3.4xl
16
16
2
3.5T
32
i3.8xl
16
32
4
6.9T
64
i3.16xl
31
64
8
13.8T
248
With mq feature of block layer, it is now possible to run entire submission and completion path for IO processing on the same CPU where the process scheduled I/O, maximizing cache locality and performance. nvme driver was updated in Linux kernel version 3.19 to use Linux multiqueue feature
multi-queue feature implementation was completed in recent kernel version (3.19 and above) that improves Linux block layer scalability to achieve maximum IOPS (15 Million) to accommodate today's and future high performance NVMe devices. NVMe uses optimized block layer path and that reduces software overhead by over 50%: SCSI/SAS (6.0 us, 19500 cycles), NVMe (2.8 us, 9100 cycles). Before mq feature block layer has a single request queue per device that is protected by spinlock resulting in higher contention and lower scalability. Linux block layer mq features splits the request queue into two queues:
  • Number of separate per-cpu software queues. Each cpu submits IO operations into its own queue, with no interaction or locking requirements with other cpus.
  • One or more hardware queues managed by driver

I3 NVMe Storage Benchmark Results

I3 instance setup:
Instance Type
Hardware Queues per Device
Number of nvme Devices
Capacity
Total HW Queues
i3.xl
4
1
0.8T
4
i3.2xl
8
1
1.7T
8
i3.4xl
16
2
3.5T
32
i3.8xl
16
4
6.9T
64
i3.16xl
31
8
13.8T
248

I3 NVMe storage IOPS

Instance Type
Access pattern
Block size
Read iops
i3.xl
random
4k
205k
i3.2xl
random
4k
413k
i3.4xl
random
4k
830k
i3.8xl
random
4k
1.65m
i3.16xl
random
4k
3.3m

I3 NVMe storage Throughput

Instance Type
Access pattern
Block size
rtput(MB)
i3.xl
sequential
128k
980
i3.2xl
sequential
128k
1910
i3.4xl
sequential
128k
3814
i3.8xl
sequential
128k
7641
i3.16xl
sequential
128k
15302

Thanks to Intel hardware virtualization extensions, VT-x (cpu virtualization), EPT (translation tables or memory virtualization), VT-d (I/O virtualization), and SR-IOV server virtualization has evolved from a slow software only solution to an efficient hardware assisted one. Large chunks of compute work are now offloaded to hardware to achieve optimum performance, bypassing hypervisor layer. With reduced virtualization overhead, performance gap between hypervisor controlled and bare-metal systems continue to shrink. 

35 comments:

  1. Any chance the code used for the benchmarks is available somewhere? Would be curious to try to replicate your results.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  2. Thanks a lot very much for the high quality and results-oriented help.
    I won’t think twice to endorse your blog post to anybody who wants
    and needs support about this area.


    AWS Training in Chennai


    AWS Training in Bangalore


    AWS Training in Bangalore

    ReplyDelete
    Replies
    1. Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Front end developer learn from Javascript Training in Chennai . or learn thru Javascript Training in Chennai. Nowadays JavaScript has tons of job opportunities on various vertical industry. JavaScript Training in Chennai

      Delete
  3. I wish to show thanks to you just for bailing me out of this particular trouble.As a result of checking through the net and meeting techniques that were not productive, I thought my life was done

    AWS Training chennai

    ReplyDelete
  4. I wish to show thanks to you just for bailing me out of this particular trouble.As a result of checking through the net and meeting techniques that were not productive, I thought my life was done

    AWS Training chennai

    ReplyDelete
  5. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.
    "Dotnet Training in Marathahalli"

    ReplyDelete
  6. Useful post, I have learned a lot. Keep sharing more like this.
    AWS course in Chennai | AWS Training Chennai

    ReplyDelete
  7. Public self storage companies with containers located within a building typically only allow access during business hours, which decreases the risk for theft and other mischievous activity.Luton Removals

    ReplyDelete
  8. And indeed, I’m just always astounded concerning the remarkable things served by you. Some four facts on this page are undeniably the most effective I’ve had.

    AWS Training in Bangalore|
    AWS Training in chennai|

    ReplyDelete
  9. Looking for Backblaze? Backblaze is favorite online backup service. Backing up your files to an off-site cloud server is an easy, affordable, and safe way to make sure that your most important files are safe.
    If you want to know more, Please check out here: backblaze

    ReplyDelete
  10. I read this article. I think You put a lot of effort to create this article. I appreciate your work.
    Dissertation Writing Services

    ReplyDelete
  11. My rather long internet look up has at the end of the day been compensated with pleasant insight to talk about with my family and friends.


    Aws Training in Porur


    ReplyDelete
  12. informative blog thanks for providing such a great information.
    Aws Training in Hyderabad

    ReplyDelete
  13. Thank you for your informative post on me projects chennai . It helped a lot in training my students during our Embeddes training Chennai sessions. We at DLK CDC, provide me project centers chennai for students who are interested in choosing a career in embedded.

    ReplyDelete
  14. Thanks for sharing the information, Salesforce experts a a lot of openings in multi-level companies, for more information n
    Best Salesforce Training in Hyderabad
    Salesforce online training in india

    ReplyDelete
  15. This was an nice and amazing and the given contents were very useful and the precision has given here is good.
    Java Training in Chennai

    ReplyDelete

  16. Thankyou for sharing this good information.
    really nice blog and very useful.hadoop training in chennai

    ReplyDelete
  17. Thanks for the blog and it is really very useful one.hadoop training in chennai

    ReplyDelete
  18. This was an nice and amazing and the given contents were very useful and the precision has given here is good.


    Bigdata training institute in bangalore

    ReplyDelete
  19. The Post seems to be good i really gather lot of information from the post thanks for sharing this awesome post AWS Training in Chennai


    ReplyDelete
  20. The Post seems to be good i really gather lot of information from the post thanks for sharing this awesome post AWS Training in Chennai

    ReplyDelete
  21. This comment has been removed by the author.

    ReplyDelete
  22. I would like to thank you for your nicely written post, its informative and your writing style encouraged me to read it till end. Thanks

    angularjs-Training in annanagar

    angularjs Training in chennai

    angularjs Training in chennai

    angularjs Training in bangalore

    ReplyDelete
  23. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.
    rpa training in Chennai | rpa training in bangalore

    ReplyDelete
  24. Very delicious food share you always, I like your all informative posts very much.

    Phoebe

    ReplyDelete
  25. Resources like the one you mentioned here will be very useful to me ! I will post a link to this page on my blog. I am sure my visitors will find that very useful
    python training in tambaram | python training in annanagar | python training in jayanagar

    ReplyDelete
  26. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.
    advanced excel training in bangalore

    ReplyDelete
  27. A very nice guide. I will definitely follow these tips. Thank you for sharing such detailed article. I am learning a lot from you.

    rpa training in electronic-city | rpa training in btm | rpa training in marathahalli | rpa training in pune

    ReplyDelete
  28. Have you been thinking about the power sources and the tiles whom use blocks I wanted to thank you for this great read!! I definitely enjoyed every little bit of it and I have you bookmarked to check out the new stuff you post
    Java training in Indira nagar | Java training in Rajaji nagar

    Java training in Marathahalli | Java training in Btm layout

    ReplyDelete
  29. Amazon has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. Amazon Web Services (AWS) is a secure cloud services platform, offering compute power, database storage, content delivery and other functionality to help businesses scale and grow.For more information visit.
    aws online training
    aws training in hyderabad
    aws online training in hyderabad

    ReplyDelete

  30. Howdy, would you mind letting me know which web host you’re utilizing? I’ve loaded your blog in 3 completely different web browsers, and I must say this blog loads a lot quicker than most. Can you suggest a good internet hosting provider at a reasonable price?


    Best AWS Training Institute in BTM Layout Bangalore ,AWS Coursesin BTM


    Best AWS Training in Marathahalli | AWS Training in Marathahalli

    Amazon Web Services Training in Jaya Nagar | Best AWS Training in Jaya Nagar


    AWS Training in BTM Layout |Best AWS Training in BTM Layout

    ReplyDelete
  31. Your post is really awesome. Your blog is really helpful for me to develop my skills in a right way. Thanks for sharing this unique information with us.
    - Learn Digital Academy

    ReplyDelete