From 2597ba763fc44e247e462177abfe92d95bc5e6d5 Mon Sep 17 00:00:00 2001 From: Donald Dutile Date: Tue, 27 Nov 2012 22:31:37 -0500 Subject: [PATCH] PCI: SRIOV control and status via sysfs (documentation) Add documentation of new sysfs files and new pci_driver SRIOV configuration interface. [bhelgaas: changelog] Signed-off: Donald Dutile Signed-off-by: Bjorn Helgaas --- Documentation/ABI/testing/sysfs-bus-pci | 34 ++++++++++++++++++ Documentation/PCI/pci-iov-howto.txt | 48 ++++++++++++++++++++++--- 2 files changed, 78 insertions(+), 4 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci index dff1f48d252d..1ce5ae329c04 100644 --- a/Documentation/ABI/testing/sysfs-bus-pci +++ b/Documentation/ABI/testing/sysfs-bus-pci @@ -222,3 +222,37 @@ Description: satisfied too. Reading this attribute will show the current value of d3cold_allowed bit. Writing this attribute will set the value of d3cold_allowed bit. + +What: /sys/bus/pci/devices/.../sriov_totalvfs +Date: November 2012 +Contact: Donald Dutile +Description: + This file appears when a physical PCIe device supports SR-IOV. + Userspace applications can read this file to determine the + maximum number of Virtual Functions (VFs) a PCIe physical + function (PF) can support. Typically, this is the value reported + in the PF's SR-IOV extended capability structure's TotalVFs + element. Drivers have the ability at probe time to reduce the + value read from this file via the pci_sriov_set_totalvfs() + function. + +What: /sys/bus/pci/devices/.../sriov_numvfs +Date: November 2012 +Contact: Donald Dutile +Description: + This file appears when a physical PCIe device supports SR-IOV. + Userspace applications can read and write to this file to + determine and control the enablement or disablement of Virtual + Functions (VFs) on the physical function (PF). A read of this + file will return the number of VFs that are enabled on this PF. + A number written to this file will enable the specified + number of VFs. A userspace application would typically read the + file and check that the value is zero, and then write the number + of VFs that should be enabled on the PF; the value written + should be less than or equal to the value in the sriov_totalvfs + file. A userspace application wanting to disable the VFs would + write a zero to this file. The core ensures that valid values + are written to this file, and returns errors when values are not + valid. For example, writing a 2 to this file when sriov_numvfs + is not 0 and not 2 already will return an error. Writing a 10 + when the value of sriov_totalvfs is 8 will return an error. diff --git a/Documentation/PCI/pci-iov-howto.txt b/Documentation/PCI/pci-iov-howto.txt index fc73ef5d65b8..cfaca7e69893 100644 --- a/Documentation/PCI/pci-iov-howto.txt +++ b/Documentation/PCI/pci-iov-howto.txt @@ -2,6 +2,9 @@ Copyright (C) 2009 Intel Corporation Yu Zhao + Update: November 2012 + -- sysfs-based SRIOV enable-/disable-ment + Donald Dutile 1. Overview @@ -24,10 +27,21 @@ real existing PCI device. 2.1 How can I enable SR-IOV capability -The device driver (PF driver) will control the enabling and disabling -of the capability via API provided by SR-IOV core. If the hardware -has SR-IOV capability, loading its PF driver would enable it and all -VFs associated with the PF. +Multiple methods are available for SR-IOV enablement. +In the first method, the device driver (PF driver) will control the +enabling and disabling of the capability via API provided by SR-IOV core. +If the hardware has SR-IOV capability, loading its PF driver would +enable it and all VFs associated with the PF. Some PF drivers require +a module parameter to be set to determine the number of VFs to enable. +In the second method, a write to the sysfs file sriov_numvfs will +enable and disable the VFs associated with a PCIe PF. This method +enables per-PF, VF enable/disable values versus the first method, +which applies to all PFs of the same device. Additionally, the +PCI SRIOV core support ensures that enable/disable operations are +valid to reduce duplication in multiple drivers for the same +checks, e.g., check numvfs == 0 if enabling VFs, ensure +numvfs <= totalvfs. +The second method is the recommended method for new/future VF devices. 2.2 How can I use the Virtual Functions @@ -40,13 +54,22 @@ requires device driver that is same as a normal PCI device's. 3.1 SR-IOV API To enable SR-IOV capability: +(a) For the first method, in the driver: int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn); 'nr_virtfn' is number of VFs to be enabled. +(b) For the second method, from sysfs: + echo 'nr_virtfn' > \ + /sys/bus/pci/devices//sriov_numvfs To disable SR-IOV capability: +(a) For the first method, in the driver: void pci_disable_sriov(struct pci_dev *dev); +(b) For the second method, from sysfs: + echo 0 > \ + /sys/bus/pci/devices//sriov_numvfs To notify SR-IOV core of Virtual Function Migration: +(a) In the driver: irqreturn_t pci_sriov_migration(struct pci_dev *dev); 3.2 Usage example @@ -88,6 +111,22 @@ static void dev_shutdown(struct pci_dev *dev) ... } +static int dev_sriov_configure(struct pci_dev *dev, int numvfs) +{ + if (numvfs > 0) { + ... + pci_enable_sriov(dev, numvfs); + ... + return numvfs; + } + if (numvfs == 0) { + .... + pci_disable_sriov(dev); + ... + return 0; + } +} + static struct pci_driver dev_driver = { .name = "SR-IOV Physical Function driver", .id_table = dev_id_table, @@ -96,4 +135,5 @@ static struct pci_driver dev_driver = { .suspend = dev_suspend, .resume = dev_resume, .shutdown = dev_shutdown, + .sriov_configure = dev_sriov_configure, };