Commit graph

150 commits

Author SHA1 Message Date
Dave Jiang 9ca1c5f2ab dmaengine: ioatdma: workaround for incorrect DMACAP register
BDX-DE IOATDMA reports incorrect DMACAP register for PQ related
ops. Ignoring those bits.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2015-03-05 14:32:02 +05:30
Nicholas Mc Guire 12385f458a ioat: fail self-test if wait_for_completion times out
wait_for_completion_timeout reaching timeout was being ignored,
fail the self-test if timeout condition occurs.

v2: fixup of coding style issues.

Signed-off-by: Nicholas Mc Guire <der.herr@hofr.at>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Prarit Bhargava <prarit@redhat.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2015-02-04 22:54:22 -08:00
Dave Jiang 68a8cc9e9e ioatdma: Adding support for BDX-DE ioatdma.
Adding PCI device IDs and hooks in workarounds for Broadwell DE ioatdma.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2015-02-03 17:47:40 -08:00
Dave Jiang abf538ae03 dmaengine: ioatdma: PQ err descriptors should callback with err results
The err completion callback is missing from the error handler. Two
reasons we never hit this. On Xeon because the hw err workaround, the
completion happens on a NULL descriptor so we don't do callback on the
PQ descriptor. On Atom we have DWBES support and thus the callback already
happened or we don't halt on error, so that was take cared of. But this code
needs to be corrected for future error handlers.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2014-12-22 12:22:56 +05:30
Prarit Bhargava 4ff2fd839c dmaengine: ioatdma: fix dma mapping errors
Several systems are showing the following stack trace:

WARNING: CPU: 0 PID: 2352 at lib/dma-debug.c:1140 check_unmap+0x4ee/0x9e0()
ioatdma 0000:00:04.0: DMA-API: device driver failed to check map error[device address=0x0000000465bad000] [size=4096 bytes] [mapped as page]
Modules linked in: ioatdma(E+) nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache cfg80211 rfkill x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel cdc_ether ses ghash_clmulni_intel usbnet mii enclosure aesni_intel lrw gf128mul glue_helper iTCO_wdt shpchp ablk_helper iTCO_vendor_support cryptd pcspkr ipmi_devintf sb_edac lpc_ich edac_core mfd_core ipmi_si i2c_i801 wmi ipmi_msghandler nfsd auth_rpcgss nfs_acl lockd sunrpc xfs libcrc32c sd_mod crc_t10dif crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt drm_kms_helper ttm igb drm ptp pps_core dca i2c_algo_bit i2ccore megaraid_sas dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ioatdma]
CPU: 0 PID: 2352 Comm: insmod Tainted: G            E  3.17.0-rc4+ #14
Hardware name: HP ProLiant m300 Server Cartridge/, BIOS H02 01/30/2014
 0000000000000009 ffff88007994b7d8 ffffffff816e7225 ffff88007994b820
 ffff88007994b810 ffffffff8107e51d ffff88045fc56c00 ffff88046643ee90
 ffffffff8338ccd0 0000000000000286 ffffffff81956629 ffff88007994b870
Call Trace:
 [<ffffffff816e7225>] dump_stack+0x4d/0x66
 [<ffffffff8107e51d>] warn_slowpath_common+0x7d/0xa0
 [<ffffffff8107e58c>] warn_slowpath_fmt+0x4c/0x50
 [<ffffffff81381e6e>] check_unmap+0x4ee/0x9e0
 [<ffffffff813823bf>] debug_dma_unmap_page+0x5f/0x70
 [<ffffffffa04546d8>] ioat_xor_val_self_test+0x498/0xcf0 [ioatdma]
 [<ffffffff81204f0a>] ? kfree+0xda/0x2b0
 [<ffffffffa044d510>] ? ioat_dma_setup_interrupts+0x120/0x2d0 [ioatdma]
 [<ffffffffa0454f4e>] ioat3_dma_self_test+0x1e/0x30 [ioatdma]
 [<ffffffffa044f904>] ioat_probe+0xf4/0x110 [ioatdma]
 [<ffffffffa04550f8>] ioat3_dma_probe+0x198/0x3a0 [ioatdma]
 [<ffffffffa044d18e>] ioat_pci_probe+0x11e/0x1b0 [ioatdma]
 [<ffffffff81393a15>] local_pci_probe+0x45/0xa0
 [<ffffffff81394be5>] ? pci_match_device+0xe5/0x110
 [<ffffffff81394d29>] pci_device_probe+0xd9/0x130
 [<ffffffff81462860>] driver_probe_device+0x90/0x3c0
 [<ffffffff81462c63>] __driver_attach+0x93/0xa0
 [<ffffffff81462bd0>] ? __device_attach+0x40/0x40
 [<ffffffff8146080b>] bus_for_each_dev+0x6b/0xb0
 [<ffffffff814622ce>] driver_attach+0x1e/0x20
 [<ffffffff81461ed8>] bus_add_driver+0x188/0x260
 [<ffffffffa0423000>] ? 0xffffffffa0423000
 [<ffffffff81463734>] driver_register+0x64/0xf0
 [<ffffffff813933a0>] __pci_register_driver+0x60/0x70
 [<ffffffffa0423089>] ioat_init_module+0x89/0x1000 [ioatdma]
 [<ffffffff8100212c>] do_one_initcall+0xbc/0x200
 [<ffffffff811e8b22>] ? __vunmap+0xd2/0x120
 [<ffffffff8111e73c>] load_module+0x14ec/0x1b50
 [<ffffffff81119970>] ? store_uevent+0x40/0x40
 [<ffffffff8111ef36>] SyS_finit_module+0x86/0xb0
 [<ffffffff816f1469>] system_call_fastpath+0x16/0x1b
---[ end trace 1052ccbbc3db4d08 ]---
Mapped at:
 [<ffffffff81380be1>] debug_dma_map_page+0x91/0x140
 [<ffffffffa045440e>] ioat_xor_val_self_test+0x1ce/0xcf0 [ioatdma]
 [<ffffffffa0454f4e>] ioat3_dma_self_test+0x1e/0x30 [ioatdma]
 [<ffffffffa044f904>] ioat_probe+0xf4/0x110 [ioatdma]
 [<ffffffffa04550f8>] ioat3_dma_probe+0x198/0x3a0 [ioatdma]

This happens because the current ioatdma DMA test code does not check the return
value of dma_map_page() calls with dma_mapping_error().  In addition, it was
noticed that mapping for the variable dest_dma is free'd before the last use.

This patch fixes these errors by initializing the dma_srcs[] array and checking
the returns with dma_mapping_error().

Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2014-12-09 14:52:05 +05:30
Dan Williams 7bced39751 net_dma: simple removal
Per commit "77873803363c net_dma: mark broken" net_dma is no longer used
and there is no plan to fix it.

This is the mechanical removal of bits in CONFIG_NET_DMA ifdef guards.
Reverting the remainder of the net_dma induced changes is deferred to
subsequent patches.

Marked for stable due to Roman's report of a memory leak in
dma_pin_iovec_pages():

    https://lkml.org/lkml/2014/9/3/177

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: David Whipple <whipple@securedatainnovations.ch>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
Cc: <stable@vger.kernel.org>
Reported-by: Roman Gushchin <klamm@yandex-team.ru>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2014-09-28 07:05:16 -07:00
Manuel Schölling e628ce70ca ioat: Use time_before_jiffies()
To be future-proof and for better readability the time comparisons are modified
to use time_before_jiffies() instead of plain, error-prone math.

Signed-off-by: Manuel Schölling <manuel.schoelling@gmx.de>
[djbw: use time_before_jiffies() to make argument order more clear]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2014-08-21 11:39:22 -07:00
Alexander Gordeev 368da992b9 ioat: Use pci_enable_msix_exact() instead of pci_enable_msix()
As result of deprecation of MSI-X/MSI enablement functions
pci_enable_msix() and pci_enable_msi_block() all drivers
using these two interfaces need to be updated to use the
new pci_enable_msi_range()  or pci_enable_msi_exact()
and pci_enable_msix_range() or pci_enable_msix_exact()
interfaces.

Function pci_enable_msix() returns a tri-state value while
pci_enable_msi_exact() is a canonical zero/-errno variant.
The former is being phased out in favor of the latter.
In case of 'ioat' there (should be) no difference.

Cc: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2014-04-10 12:17:35 -07:00
Rashika 2358b820d4 drivers: dma: Include appropriate header file in dca.c
Includes an appropriate header file dma_v2.h in ioat/dca.c because
functions ioat2_dca_init() and ioat3_dca_init() have their function
declarations in dma_v2.h.

This eliminates the following warning in ioat/dca.c:
drivers/dma/ioat/dca.c:410:22: warning: no previous prototype for ‘ioat2_dca_init’ [-Wmissing-prototypes]
drivers/dma/ioat/dca.c:624:22: warning: no previous prototype for ‘ioat3_dca_init’ [-Wmissing-prototypes]

Signed-off-by: Rashika Kheria <rashika.kheria@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2014-04-10 12:17:35 -07:00
Rashika 8d1d32767c drivers: dma: Mark functions as static in dma_v3.c
Mark the functions ioat3_prep_xor_val(), ioat3_prep_pq_val() and
ioat3_prep_pqxor_val() as static in dma_v3.c because they are not used
outside this file.

This eliminates the following warnings in dma_v3.c:
drivers/dma/ioat/dma_v3.c:741:1: warning: no previous prototype for ‘ioat3_prep_xor_val’ [-Wmissing-prototypes]
drivers/dma/ioat/dma_v3.c:1092:1: warning: no previous prototype for ‘ioat3_prep_pq_val’ [-Wmissing-prototypes]
drivers/dma/ioat/dma_v3.c:1134:1: warning: no previous prototype for ‘ioat3_prep_pqxor_val’ [-Wmissing-prototypes]

Signed-off-by: Rashika Kheria <rashika.kheria@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2014-04-10 12:17:35 -07:00
Yijing Wang 1fde254846 ioat/dca: Use dev_is_pci() to check whether it is pci device
Use PCI standard marco dev_is_pci() instead of directly compare
pci_bus_type to check whether it is pci device.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2014-04-10 12:17:34 -07:00
Dan Williams da87ca4d4c ioat: fix tasklet tear down
Since commit 7787380336 "net_dma: mark broken" we no longer pin dma
engines active for the network-receive-offload use case.  As a result
the ->free_chan_resources() that occurs after the driver self test no
longer has a NET_DMA induced ->alloc_chan_resources() to back it up.  A
late firing irq can lead to ksoftirqd spinning indefinitely due to the
tasklet_disable() performed by ->free_chan_resources().  Only
->alloc_chan_resources() can clear this condition in affected kernels.

This problem has been present since commit 3e037454bc "I/OAT: Add
support for MSI and MSI-X" in 2.6.24, but is now exposed. Given the
NET_DMA use case is deprecated we can revisit moving the driver to use
threaded irqs.  For now, just tear down the irq and tasklet properly by:

1/ Disable the irq from triggering the tasklet

2/ Disable the irq from re-arming

3/ Flush inflight interrupts

4/ Flush the timer

5/ Flush inflight tasklets

References:
https://lkml.org/lkml/2014/1/27/282
https://lkml.org/lkml/2014/2/19/672

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: <stable@vger.kernel.org>
Reported-by: Mike Galbraith <bitbucket@online.de>
Reported-by: Stanislav Fomichev <stfomichev@yandex-team.ru>
Tested-by: Mike Galbraith <bitbucket@online.de>
Tested-by: Stanislav Fomichev <stfomichev@yandex-team.ru>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2014-02-25 09:44:20 -08:00
Jiang Liu 3532e5660f drivers/dma/ioat/dma.c: check DMA mapping error in ioat_dma_self_test()
Check DMA mapping return values in function ioat_dma_self_test() to get
rid of following warning message.

  ------------[ cut here ]------------
  WARNING: CPU: 0 PID: 1203 at lib/dma-debug.c:937 check_unmap+0x4c0/0x9a0()
  ioatdma 0000:00:04.0: DMA-API: device driver failed to check map error[device address=0x000000085191b000] [size=2000 bytes] [mapped as single]
  Modules linked in: ioatdma(+) mac_hid wmi acpi_pad lp parport hidd_generic usbhid hid ixgbe isci dca libsas ahci ptp libahci scsi_transport_sas meegaraid_sas pps_core mdio
  CPU: 0 PID: 1203 Comm: systemd-udevd Not tainted 3.13.0-rc4+ #8
  Hardware name: Intel Corporation BRICKLAND/BRICKLAND, BIOS BRIVTIIN1.86B.0044.L09.1311181644 11/18/2013
  Call Trace:
    dump_stack+0x4d/0x66
    warn_slowpath_common+0x7d/0xa0
    warn_slowpath_fmt+0x4c/0x50
    check_unmap+0x4c0/0x9a0
    debug_dma_unmap_page+0x81/0x90
    ioat_dma_self_test+0x3d2/0x680 [ioatdma]
    ioat3_dma_self_test+0x12/0x30 [ioatdma]
    ioat_probe+0xf4/0x110 [ioatdma]
    ioat3_dma_probe+0x268/0x410 [ioatdma]
    ioat_pci_probe+0x122/0x1b0 [ioatdma]
    local_pci_probe+0x45/0xa0
    pci_device_probe+0xd9/0x130
    driver_probe_device+0x171/0x490
    __driver_attach+0x93/0xa0
    bus_for_each_dev+0x6b/0xb0
    driver_attach+0x1e/0x20
    bus_add_driver+0x1f8/0x2b0
    driver_register+0x81/0x110
    __pci_register_driver+0x60/0x70
    ioat_init_module+0x89/0x1000 [ioatdma]
    do_one_initcall+0xe2/0x250
    load_module+0x2313/0x2a00
    SyS_init_module+0xd9/0x130
    system_call_fastpath+0x1a/0x1f
  ---[ end trace 990c591681d27c31 ]---
  Mapped at:
    debug_dma_map_page+0xbe/0x180
    ioat_dma_self_test+0x1ab/0x680 [ioatdma]
    ioat3_dma_self_test+0x12/0x30 [ioatdma]
    ioat_probe+0xf4/0x110 [ioatdma]
    ioat3_dma_probe+0x268/0x410 [ioatdma]

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Cc: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-01-02 14:40:30 -08:00
Vinod Koul df12a3178d Merge commit 'dmaengine-3.13-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/dmaengine
Pull dmaengine changes from Dan

1/ Bartlomiej and Dan finalized a rework of the dma address unmap
   implementation.

2/ In the course of testing 1/ a collection of enhancements to dmatest
   fell out.  Notably basic performance statistics, and fixed / enhanced
   test control through new module parameters 'run', 'wait', 'noverify',
   and 'verbose'.  Thanks to Andriy and Linus for their review.

3/ Testing the raid related corner cases of 1/ triggered bugs in the
   recently added 16-source operation support in the ioatdma driver.

4/ Some minor fixes / cleanups to mv_xor and ioatdma.

Conflicts:
	drivers/dma/dmatest.c

Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-11-16 12:02:36 +05:30
Dan Williams 779e561ae2 ioat: fix ioat3_irq_reinit
The implementation of ioat3_irq_reinit has two bugs:

1/ The mode is incorrectly set to MSIX for the MSI case

2/ The 'dev_id' parameter to free_irq is the ioatdma_device not the channel in
   the msi and intx case

Include a small cleanup to clarify that ioat3_irq_reinit is only for bwd
hardware

Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:42 -08:00
Dan Williams 4c5d9619e0 ioat: kill msix_single_vector support
Once we have determined that we will not have all of our desired msix
vectors there is no point in attempting a single msix allocation.  The
driver will already need to read registers to determine the source of
the interrupt the fact that it is msix is moot.  Fallback directly to
msi.

Reported-by: Alexander Gordeev <agordeev@redhat.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:42 -08:00
Dan Williams 59056e85d7 ioatdma: clean up sed pool kmem_cache
Use a single cache for all sed allocations.  No need to make it per
channel.  This also avoids the slub_debug warnings for multiple caches
with the same name.

Switching to dmam_pool_create() to fix leaking the dma pools on
initialization failure and lets us kill ioat3_dma_remove().

Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:41 -08:00
Dan Williams 21e96c7313 ioatdma: fix selection of 16 vs 8 source path
When performing continuations there are implied sources that need to be
added to the source count. Quoting dma_set_maxpq:

/* dma_maxpq - reduce maxpq in the face of continued operations
 * @dma - dma device with PQ capability
 * @flags - to check if DMA_PREP_CONTINUE and DMA_PREP_PQ_DISABLE_P are set
 *
 * When an engine does not support native continuation we need 3 extra
 * source slots to reuse P and Q with the following coefficients:
 * 1/ {00} * P : remove P from Q', but use it as a source for P'
 * 2/ {01} * Q : use Q to continue Q' calculation
 * 3/ {00} * Q : subtract Q from P' to cancel (2)
 *
 * In the case where P is disabled we only need 1 extra source:
 * 1/ {01} * Q : use Q to continue Q' calculation
 */

...fix the selection of the 16 source path to take these implied sources
into account.

Note this also kills the BUG_ON(src_cnt < 9) check in
__ioat3_prep_pq16_lock().  Besides not accounting for implied sources
the check is redundant given we already made the path selection.

Cc: <stable@vger.kernel.org>
Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:41 -08:00
Dan Williams 5d48b9b5d8 ioatdma: fix sed pool selection
The array to lookup the sed pool based on the number of sources
(pq16_idx_to_sedi) is 16 entries and expects a max source index.
However, we pass the total source count which runs off the end of the
array when src_cnt == 16.  The minimal fix is to just pass src_cnt-1,
but given we know the source count is > 8 we can just calculate the sed
pool by (src_cnt - 2) >> 3.

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: <stable@vger.kernel.org>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:41 -08:00
Dave Jiang ac7d631f7d ioatdma: Fix bug in selftest after removal of DMA_MEMSET.
Commit 48a9db4 (3.11) removed the memset op in the xor selftest for ioatdma.
The issue is that with the removal of that op, it never replaced the memset
with a CPU memset. The memory being operated on is expected to be zeroes but
was not. This is causing the xor selftest to fail.

Cc: <stable@vger.kernel.org>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:41 -08:00
Bartlomiej Zolnierkiewicz 0776ae7b89 dmaengine: remove DMA unmap flags
Remove no longer needed DMA unmap flags:
- DMA_COMPL_SKIP_SRC_UNMAP
- DMA_COMPL_SKIP_DEST_UNMAP
- DMA_COMPL_SRC_UNMAP_SINGLE
- DMA_COMPL_DEST_UNMAP_SINGLE

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Jon Mason <jon.mason@intel.com>
Acked-by: Mark Brown <broonie@linaro.org>
[djbw: clean up straggling skip unmap flags in ntb]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:38 -08:00
Bartlomiej Zolnierkiewicz 54f8d501e8 dmaengine: remove DMA unmap from drivers
Remove support for DMA unmapping from drivers as it is no longer
needed (DMA core code is now handling it).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
[djbw: fix up chan2parent() unused warning in drivers/dma/dw/core.c]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-14 11:04:38 -08:00
Dan Williams d38a8c622a dmaengine: prepare for generic 'unmap' data
Add a hook for a common dma unmap implementation to enable removal of
the per driver custom unmap code.  (A reworked version of Bartlomiej
Zolnierkiewicz's patches to remove the custom callbacks and the size
increase of dma_async_tx_descriptor for drivers that don't care about
raid).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
[bzolnier: prepare pl330 driver for adding missing unmap while at it]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2013-11-13 16:25:06 -08:00
Vinod Koul 2f16f802c3 dmaengine: ioat: use DMA_COMPLETE for dma completion status
Acked-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-10-25 11:16:05 +05:30
Paul Bolle e6a5fa6340 ioatdma: silence GCC warnings
Building dma_v3.o triggers a GCC warning:
    drivers/dma/ioat/dma_v3.c: In function ‘__ioat3_prep_pq16_lock’:
    drivers/dma/ioat/dma_v3.c:264:11: warning: array subscript is below array bounds [-Warray-bounds]
    drivers/dma/ioat/dma_v3.c:264:11: warning: array subscript is below array bounds [-Warray-bounds]

This warning is caused by pq16_set_src(). It uses "int idx" as an index
to an eight element array. Changing "idx" to "unsigned" silences this
warning. Apparently GCC can then determine that "idx" will never be
negative.

Signed-off-by: Paul Bolle <pebolle@tiscali.nl>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <djbw@fb.com>
2013-08-23 00:21:44 -07:00
Brice Goglin c4dcf0e2dd ioatdma: disable RAID on non-Atom platforms and reenable unaligned copies
Disable RAID on non-Atom platform and remove related fixups such as the
64-byte alignement restriction on legacy DMA operations (introduced in
commit f26df1a1 as a workaround for silicon errata).

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Jon Mason <jon.mason@intel.com>
Signed-off-by: Dan Williams <djbw@fb.com>
2013-08-22 22:57:39 -07:00
Bartlomiej Zolnierkiewicz 48a9db462d drivers/dma: remove unused support for MEMSET operations
There have never been any real users of MEMSET operations since they
have been introduced in January 2007 by commit 7405f74bad ("dmaengine:
refactor dmaengine around dma_async_tx_descriptor").  Therefore remove
support for them for now, it can be always brought back when needed.

[sebastian.hesselbarth@gmail.com: fix drivers/dma/mv_xor]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Olof Johansson <olof@lixom.net>
Cc: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-07-03 16:07:42 -07:00
Linus Torvalds 1763e735b0 Merge branch 'for-linus' of git://git.infradead.org/users/vkoul/slave-dma
Pull slave-dmaengine updates from Vinod Koul:
 "This time we have dmatest improvements from Andy along with dw_dmac
  fixes.  He has also done support for acpi for dmanegine.

  Also we have bunch of fixes going in DT support for dmanegine for
  various folks.  Then Haswell and other ioat changes from Dave and
  SUDMAC support from Shimoda."

* 'for-linus' of git://git.infradead.org/users/vkoul/slave-dma: (53 commits)
  dma: tegra: implement suspend/resume callbacks
  dma:of: Use a mutex to protect the of_dma_list
  dma: of: Fix of_node reference leak
  dmaengine: sirf: move driver init from module_init to subsys_initcall
  sudmac: add support for SUDMAC
  dma: sh: add Kconfig
  at_hdmac: move to generic DMA binding
  ioatdma: ioat3_alloc_sed can be static
  ioatdma: Adding write back descriptor error status support for ioatdma 3.3
  ioatdma: S1200 platforms ioatdma channel 2 and 3 falsely advertise RAID cap
  ioatdma: Adding support for 16 src PQ ops and super extended descriptors
  ioatdma: Removing hw bug workaround for CB3.x .2 and earlier
  dw_dmac: add ACPI support
  dmaengine: call acpi_dma_request_slave_channel as well
  dma: acpi-dma: introduce ACPI DMA helpers
  dma: of: Remove unnecessary list_empty check
  DMA: OF: Check properties value before running be32_to_cpup() on it
  DMA: of: Constant names
  ioatdma: skip silicon bug workaround for pq_align for cb3.3
  ioatdma: Removing PQ val disable for cb3.3
  ...
2013-05-09 09:46:45 -07:00
Fengguang Wu e6a30fec08 ioatdma: ioat3_alloc_sed can be static
Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-16 18:49:40 +05:30
Dave Jiang 75c6f0ab48 ioatdma: Adding write back descriptor error status support for ioatdma 3.3
v3.3 provides support for write back descriptor error status. This allows
reporting of errors in a descriptor field. In supporting this, certain
errors such as P/Q validation errors no longer halts the channel. The DMA
engine can continue to execute until the end of the chain and allow software
to report the "errors" up the stack. We are also going to mask those error
interrupts and handle them when the "chain" has completed at the end.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 22:46:15 +05:30
Dave Jiang d302398da9 ioatdma: S1200 platforms ioatdma channel 2 and 3 falsely advertise RAID cap
This workaround checks for channel 2&3 and remove RAID cap.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 22:46:15 +05:30
Dave Jiang 7727eaa449 ioatdma: Adding support for 16 src PQ ops and super extended descriptors
v3.3 introduced 16 sources PQ operations. This also introduced super extended
descriptors to support the 16 srcs operations. This patch adds support for
the 16 sources ops and in turn adds the super extended descriptors for those
ops.

5 SED pools are created depending on the descriptor sizes. An SED can be a 64
bytes sized descriptor or larger and must be physically contiguous. A kmem
cache pool is created for allocating the software descriptor that manages the
hardware descriptor. The super extended descriptor will take place of extended
descriptor under certain operations and be "attached" to the op descriptor
during operation. This is a new feature for ioatdma v3.3.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 22:46:15 +05:30
Dave Jiang e0884772d3 ioatdma: Removing hw bug workaround for CB3.x .2 and earlier
CB3.2 and earlier hardware has silicon bugs that are no longer needed with
the new hardware. We don't have to use a NULL op to signal interrupt for
RAID ops any longer. This code make sure the legacy workarounds only happen on
legacy hardware.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 22:46:14 +05:30
Dave Jiang eceec44ecd ioatdma: skip silicon bug workaround for pq_align for cb3.3
The alignment workaround is only necessary for cb3.2 or earlier platforms.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:20 +05:30
Dave Jiang 3f09ede423 ioatdma: Removing PQ val disable for cb3.3
The PQ Val ops work on the newer hardware so we should actually provide support
for it and remove the disabling bits.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:20 +05:30
Dave Jiang 6ead7e4849 ioatdma: skip legacy reset bits since v3.3 plattform doesn't need it
Make it so only 3.2 and earlier platform need the PCI config register
clearings since this implementation does not have the registers.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:20 +05:30
Dave Jiang 8a52b9ff11 ioatdma: channel reset scheme fixup on Intel Atom S1200 platforms
The Intel Atom S1200 family ioatdma changed the channel reset behavior.
It does a reset similar to PCI FLR by resetting all the MSIX
registers. We have to re-init msix interrupts because of this. This
workaround is only specific to this platform and is not expected to carry
over to the later generations.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:20 +05:30
Dave Jiang d92a8d7cbb ioatdma: Add 64bit chansts register read for ioat v3.3.
The channel status register for v3.3 is now 64bit. Use readq if available
on v3.3 platforms.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:20 +05:30
Dave Jiang 0132bcef76 ioatdma: Adding PCI IDs for Intel Atom S1200 product family ioatdma devices
These should be good for the IOAT DMA devices on the Intel Atom S1269,
S1279, and S1289 platforms. We are also adding IOAT v3.3 definition for
the new DMA engine.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:20 +05:30
Dave Jiang 570727b552 ioatdma: Adding Haswell devid for ioatdma
Adding Haswell PCI device IDs for ioatdma and simplify the detection of
certain Xeon CPUs that has alignment bugs so that modifications can be
changed at a single place going forward.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:19 +05:30
Dave Jiang 9a37f64404 ioatdma: allow all channels to have irq coalescing support
Looks like only the RAID channels are allowed to have irq coalescing support
in the existing code. Fixing that. The ioat3 cleanup code can handle memcpy
ops anyways

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:16 +05:30
Dave Jiang 50f9f97e70 ioatdma: make debug output more readable
Making OP field a hex instead of integer to make it more readable. Also add
the dump out of the NEXT field.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-04-15 09:51:16 +05:30
Alexander Duyck f3c78f8515 ioat/dca: Update DCA BIOS workarounds to use TAINT_FIRMWARE_WORKAROUND
This patch is meant to be a follow-up for a patch originally submitted under
the title "ioat: Do not enable DCA if tag map is invalid".  It was brought to
my attention that the preferred approach for BIOS workarounds is to set the
taint flag for TAINT_FIRMWARE_WORKAROUND for systems that require BIOS
workarounds.

This change makes it so that the DCA workarounds for broken BIOSes will now
use WARN_TAINT_ONCE(1, TAINT_FIRMWARE_WORKAROUND, ...) instead of just
printing a message via dev_err.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-22 12:55:38 -04:00
Linus Torvalds 5115f3c19d Merge branch 'next' of git://git.infradead.org/users/vkoul/slave-dma
Pull slave-dmaengine updates from Vinod Koul:
 "This is fairly big pull by my standards as I had missed last merge
  window.  So we have the support for device tree for slave-dmaengine,
  large updates to dw_dmac driver from Andy for reusing on different
  architectures.  Along with this we have fixes on bunch of the drivers"

Fix up trivial conflicts, usually due to #include line movement next to
each other.

* 'next' of git://git.infradead.org/users/vkoul/slave-dma: (111 commits)
  Revert "ARM: SPEAr13xx: Pass DW DMAC platform data from DT"
  ARM: dts: pl330: Add #dma-cells for generic dma binding support
  DMA: PL330: Register the DMA controller with the generic DMA helpers
  DMA: PL330: Add xlate function
  DMA: PL330: Add new pl330 filter for DT case.
  dma: tegra20-apb-dma: remove unnecessary assignment
  edma: do not waste memory for dma_mask
  dma: coh901318: set residue only if dma is in progress
  dma: coh901318: avoid unbalanced locking
  dmaengine.h: remove redundant else keyword
  dma: of-dma: protect list write operation by spin_lock
  dmaengine: ste_dma40: do not remove descriptors for cyclic transfers
  dma: of-dma.c: fix memory leakage
  dw_dmac: apply default dma_mask if needed
  dmaengine: ioat - fix spare sparse complain
  dmaengine: move drivers/of/dma.c -> drivers/dma/of-dma.c
  ioatdma: fix race between updating ioat->head and IOAT_COMPLETION_PENDING
  dw_dmac: add support for Lynxpoint DMA controllers
  dw_dmac: return proper residue value
  dw_dmac: fill individual length of descriptor
  ...
2013-02-26 09:24:48 -08:00
Fengguang Wu a20702b8d7 dmaengine: ioat - fix spare sparse complain
>> drivers/dma/ioat/dma_v3.c:371:6: sparse: symbol 'ioat3_timer_event' was not declared.

Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-02-13 08:10:09 -08:00
Dave Jiang 4dec23d771 ioatdma: fix race between updating ioat->head and IOAT_COMPLETION_PENDING
There is a race that can hit during __cleanup() when the ioat->head pointer is
incremented during descriptor submission. The __cleanup() can clear the
PENDING flag when it does not see any active descriptors. This causes new
submitted descriptors to be ignored because the COMPLETION_PENDING flag is
cleared. This was introduced when code was adapted from ioatdma v1 to ioatdma
v2. For v2 and v3, IOAT_COMPLETION_PENDING flag will be abandoned and a new
flag IOAT_CHAN_ACTIVE will be utilized. This flag will also be protected under
the prep_lock when being modified in order to avoid the race.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2013-02-12 08:27:21 -08:00
Linus Torvalds 3f58e0945e Merge branch 'fixes' of git://git.infradead.org/users/vkoul/slave-dma
Pull slave-dmaengine fixes from Vinod Koul:
 "A few fixes on slave dmanengine.  There are trivial fixes in imx-dma,
  tegra-dma & ioat driver"

* 'fixes' of git://git.infradead.org/users/vkoul/slave-dma:
  dma: tegra: implement flags parameters for cyclic transfer
  dmaengine: imx-dma: Disable use of hw_chain to fix sg_dma transfers.
  ioat: Fix DMA memory sync direction correct flag
2013-01-24 10:17:49 -08:00
Dave Jiang 6decffd5f6 ioat: remove chanerr mask setting for IOAT v3.x
The existing code set a value in the PCI_CHANERRMSK_INT register
for a workaround to address a pre-silicon bug on the Intel 5520 IO hub that
has been fixed when the hardware was released. There is no need for this
code.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <djbw@fb.com>
2013-01-07 22:05:13 -08:00
Dave Jiang 1a363068dc ioat: Add alignment workaround for IVB platforms
The PCI IDs for IvyBridge IOAT DMA needs to go into a header file since
dma_v3.c looks them up for certain hardware workarounds. Need to add to the
alignment workaround for IOAT 3.2 since it wasn't fixed in IVB.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <djbw@fb.com>
2013-01-07 22:05:13 -08:00
Bartlomiej Zolnierkiewicz 7369f56e3e ioat3: add missing DMA unmap to ioat_xor_val_self_test()
Make ioat_xor_val_self_test() do DMA unmapping itself and fix handling
of failure cases.

Cc: Dan Williams <djbw@fb.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <djbw@fb.com>
2013-01-07 22:05:08 -08:00