Introduction

While server virtualization is being widely deployed in an effort to reduce costs and optimize data center resource usage, an additional key area where virtualization has an opportunity to shine is in the area of I/O performance and its role in enabling more efficient application execution. The advent of the Single Root I/O Virtualization (SR-IOV) by the PCI-SIG organization provides a step forward in making it easier to implement virtualization within the PCI bus itself. SR-IOV provides additional definitions to the PCI Express® (PCIe®) specification to enable multiple Virtual Machines (VMs) to share PCI hardware resources.

Using virtualization provides several important benefits to system designers:

  • It makes it possible to run a large number of virtual machines per server, which reduces the need for hardware and the resultant costs of space and power required by hardware devices
  • It creates the ability to start or stop and add or remove servers independently, increasing flexibility and scalability
  • It adds the capability to run different operating systems on the same host machine, again reducing the need for discreet hardware

In this paper, we will explore why designing systems that have been natively built on SR-IOV-enabled hardware may be the most cost-effective way to improve I/O performance and how to easily implement SR-IOV in PCIe devices.

Traditional Virtualized System Overview

A virtualized system (Figure 1) is a discreet system which contains:

  • Several virtual machines (VM)
  • A supervisor, also referred to as the Virtual Machine Manager (VMM)
  • PCI Express hierarchy

Within this virtualized system, the Supervisor plays a crucial role; it provides the interface between the hardware and the virtual machines, it is responsible for security, and ensures that there is no possible interaction between virtual machines.

Any system can be virtualized without specific SR-IOV technology. In a more traditional virtualization scenario, the Supervisor must emulate virtual devices and perform resource sharing on their behalf by instantiating a virtual Ethernet controller for each virtual machine (Figure 2). This creates an I/O bottleneck and often results in poor performance. In addition, it creates a tradeoff between the number of virtual machines a physical server can realistically support and the system’s I/O performance. Adding more VMs can aggravate the bottleneck.

Providing a Better Way – The Benefits of SR-IOV Hardware Implementation

Designing systems with hardware that incorporates SR-IOV support allows virtual devices to be implemented in hardware and enables resource sharing to be handled by a PCI Express® device such as an Ethernet controller (Figure 3).

The benefit of using SR-IOV over more traditional network virtualization is that in SR-IOV virtualization, the VM is talking directly to the network adapter through Direct Memory Access (DMA). Using DMA allows the VM to bypass virtualization transports such as the VM Bus and avoids requiring any processing in the management partition. By avoiding the use of a switch, the best possible system performance is attained, providing close to “bare-metal” performance.

Implementing SR-IOV in an Adapter Card

The SR-IOV specification enables a hardware provider to modify their PCI card to define itself as independent devices to a VMM. To achieve this, the SR-IOV architecture distinguishes two types of functions (Figure 4):

Physical Functions (PFs): 
Physical Functions (PFs) are full-featured PCIe functions; they are discovered, managed, and manipulated like any other PCIe device and PFs have full configuration space. It is possible to configure or control the PCIe device via the PF and in turn, the PF has the complete ability to move data in and out of the device. Each PCI Express device can have from one (1) and up to eight (8) physical PFs. Each PF is independent and is seen by software as a separate PCI Express device, which allows several devices in the same chip and makes software development easier and less costly.

Virtual Functions (VFs):
Virtual Functions (VFs) are ‘lightweight’ PCIe functions designed solely to move data in and out. Each VF is attached to an underlying PF and each PF can have from zero (0) to one (1) or more VFs. In addition, PFs within the same device can each have a different number of VFs. While VFs, are similar to PFs, they intentionally have a reduced configuration space because they inherit most of their settings from their PF.

In order to effectively implement SR-IOV, it is necessary to access the VF configuration space.

Traditional routing allows 8 functions and the device number is always “0” in PCIe.

So alternate routing is used with the standard fields to extend the number of functions. This change allows up to 256 functions. (Figure 5)

In addition, SR-IOV extends this further by allowing the use of several consecutive bus numbers for a single device, enabling more than 256 functions.

VFs are light-weight functions and their configuration space is significantly different from PFs. Significant changes are as follows:

  • Most registers are hardwired. They are set to “0”, “1” or take the same value as their PF.

    • No Base Address Register is implemented
  • Only a few RW or RWC registers are implemented in each VF:
    • A few PCI/PCI Express “enable” and “status” bits
    • MSI/MSI-X registers
    • Optionally, some capability registers such as AER may be enabled

In order to access the VF memory spaces, up to 6 VF BARs are implemented in the PF SR-IOV capability. They are similar to normal BAR registers except that their settings apply to all VFs.

During initial SR-IOV set-up and initialization VFs are not enabled and are invisible. The Supervisor then detects the device and configures PFs. If the host system and device driver detect SR-IOV capability then they will:

  • Configure the number of VFs
  • Assign addresses to VF BARs
  • Enable VFs

Once set up, each VM can be assigned a virtual device and can access it directly via its VF driver. There must be at least one VF per VM, otherwise the Supervisor will need to perform some or all sharing management, reducing the benefits of utilizing SR-IOV. The current market trend is to have 64 VMs, so a SR-IOV capable device should support 64 VFs.

Choosing the Right Partners for SR-IOV Success

Many PCI card designers are realizing that the PCI IP they choose is crucial to the success of their SR-IOV implementation. By choosing PCI IP vendors who understand and design for SR-IOV inherently, the PCI cards provide a more seamless integration with the host system.

PLDA, a long-time leader in PCIe IP innovation, provides its XpressRICH3® PCIe Gen 3 IP (Figure 6) to many of the leading PCI Express hardware vendors. PLDA’s IP provides native SR-IOV support and the PLDA IP delivers industry-leading specifications to fully enable SR-IOV functionality.

PLDA’s XpressRICH3 PCIe 3.0 IP delivers:

  • The ability for each Physical Function (PF) to support up to 64 Virtual Functions (VF)
  • Native support for up to 8 PFs
  • Enablement of up to a total of 512 functions
  • VFs share the same configuration access, status and error reporting interface as their PF
  • VFs are mapped after PFs, using several bus numbers if necessary
  • Application checks tl_rx_bardec to figure out which VF is receiving a packet
  • Same RTL for ASIC and high-end FPGA applications
  • x16, x8, x4, x2, x1 at GEN3 (8Gbps) speed
  • Backward compatible to GEN2 (5Gbps) and GEN1 (2.5Gbps)

Because PLDA is the industry leader in PCI Express IP, with over 2,500 designs in working silicon, it is also an assurance of ease of integration and first-time right functionality. In addition, PLDA offers free evaluation to enable a hands-on trial of their IP before purchase and provides a comprehensive, SR-IOV Reference Design enabling quick implementation and reducing time-to-market. To schedule a demo of the XpressRICH3 IP running SR-IOV, in which the IP performs both “Read DMA” and “Write DMA”, visit PLDA at www.plda.com.

Summary:

In summary, the key benefits of using SR-IOV to achieve virtualization include:

    • Enabling efficient sharing of PCIe devices, optimizing performance and capacity
    • Creating hundreds of VFs associated with a single PF, extending the capacity of a device and lowering hardware costs
    • Dynamic control by the PF through registers designed to turn on the SR-IOV capability, eliminating the need for time-intensive integration
    • Increased performance via direct access to hardware from the virtual machine environment

转载:Why using Single Root I/O Virtualization (SR-IOV) can help improve I/O performance and Reduce Costs的更多相关文章

  1. OpenStack for NFV applications: enabling Single Root I/O virtualization and PCI-Passthrough

    http://superuser.openstack.org/articles/openstack-for-nfv-applications-enabling-single-root-i-o-virt ...

  2. Carrier-Grade Mirantis OpenStack (the Mirantis NFV Initiative), Part 1: Single Root I/O Virtualization (SR-IOV)

    The Mirantis NFV initiative aims to create an NFV ecosystem for OpenStack, with validated  hardware ...

  3. [转载]Linux下非root用户如何安装软件

    [转载]Linux下非root用户如何安装软件 来源:https://tlanyan.me/work-with-linux-without-root-permission/ 这是本人遇到的实际问题,之 ...

  4. 【转载】Centos7修改root密码

    参考: https://blog.csdn.net/wcy00q/article/details/70570043 知道root密码,需要修改密码 以root登录系统输入passwd命令默认修改roo ...

  5. CNA, FCoE, TOE, RDMA, iWARP, iSCSI等概念及 Chelsio T5 产品介绍 转载

    CNA, FCoE, TOE, RDMA, iWARP, iSCSI等概念及 Chelsio T5 产品介绍 2016年09月01日 13:56:30 疯子19911109 阅读数:4823 标签:  ...

  6. Dynamic device virtualization

    A system and method for providing dynamic device virtualization is herein disclosed. According to on ...

  7. DPDK support for vhost-user

    转载:http://blog.csdn.net/quqi99/article/details/47321023 X86体系早期没有在硬件设计上对虚拟化提供支持,因此虚拟化完全通过软件实现.一个典型的做 ...

  8. KVM 介绍(4):I/O 设备直接分配和 SR-IOV [KVM PCI/PCIe Pass-Through SR-IOV]

    学习 KVM 的系列文章: (1)介绍和安装 (2)CPU 和 内存虚拟化 (3)I/O QEMU 全虚拟化和准虚拟化(Para-virtulizaiton) (4)I/O PCI/PCIe设备直接分 ...

  9. KVM虚拟化网络优化技术总结

    https://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/sr-iov-nfv-tech-brief ...

随机推荐

  1. 【Java面试题】21 Java中的异常处理机制的简单原理和应用。

    异常指Java程序运行时(非编译)所发生的非正常情况或错误. java对异常进行了分类,不同类型的异常使用了不同的java类,所有异常的根类为java.lang.Throwable.Throwable ...

  2. SQL 2005 如何只安装客户端?

    在“注册信息”输入合适的用户名和公司后,点击[下一步],在“要安装的组件”界面中选择要安装的组件:根据我们的需要来选择要安装的组件(各组件的说明如下表),这里我勾选所有的组件:我们可以点击   [高级 ...

  3. Asp.net 程序优化js,css合并与压缩

    访问时将js和css压缩并且缓存在客户端,采用的是Yahoo.Yui.Compressor组件还完成的,从这里可下载 创建一个IHttpHandler来处理文件 ) }; )              ...

  4. HTML&CSS精选笔记_列表与超链接

    列表与超链接 列表标记 无序列表ul 无序列表的各个列表项之间没有顺序级别之分,是并列的 <ul> <li>列表项1</li> <li>列表项2< ...

  5. org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Can

    org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Can ...

  6. idea & datagrip 注册码

    CNEKJPQZEX-eyJsaWNlbnNlSWQiOiJDTkVLSlBRWkVYIiwibGljZW5zZWVOYW1lIjoibGFuIHl1IiwiYXNzaWduZWVOYW1lIjoiI ...

  7. Django学习笔记 开发环境搭建

    为什么使用django?1.支持快速开发:用python开发:数据库ORM系统,并不需要我们手动地构造SQL语句,而是用python的对象访问数据库,能够提升开发效率.2.大量内置应用:后台管理系统a ...

  8. 【Thinkphp5 】部署nginx时nginx.conf配置文件修改

    背景:thinkphp5项目 服务器环境: lnmp 1 打开路径  /usr/local/nginx/conf/vhost/ 此路径下会有你添加的域名文件夹..找到对应的域名打开. 2  代码如下, ...

  9. 微信红包随机生成算法(PHP版)

    /** * 求一个数的平方 * @param $n */ function sqr($n){ return $n*$n; } /** * 生产min和max之间的随机数,但是概率不是平均的,从min到 ...

  10. str += "a" + "b" & str = str + "a" + "b"的性能比较

    str += "a" + "b"在浏览器中的执行过程: 1.创建临时字符串, 2.将临时字符串设置为“ab”, 3.将临时字符串和str进行连接, 4.将结果赋 ...