Bug "6430966 SATA NCQ support is needed in SATA framework and hba drivers" enabled NCQ
but still only had us issue just 1 command at a time.
Bug "6539787 marvell88sx driver needs to support multiple block PIO commands" followed
that and changed it to issuing 32 (NCQ's protocol limitation) commands at a time.
Disabing NCQ (set sata:sata_func_enable = 0x5 in /etc/system and reboot) will get us
back to only issuing one command at a time, but also disables NCQ (which re-introduces
6430966). In certain situations (see 6543178 sequential read performance regression
on multi-disk volumes), sending out multiple concurrent commands at a time can cause
performance regressions.
It would be nice to be able (even if its an ugly /etc/system hack) to go back to having
NCQ enabled but only issue one command at a time.
This bug will also set the default # of commands to issue (1, 32, or heck something
in between).
The limit of 32 NCQ commands was in the driver from the very beginning. The limitation of 1 NCQ command was artificially created by allowing SATA module and marvel driver to use FPDMA R/W commands (NCQ commands) but telling the target driver that queueing is not supported. Hence, sd was issuing a single command at a time.
This is just a note for external people who only have access to the "Description field"...
To set the number of concurrent I/Os to just 1 (effectively disabling NCQ), add
the following to /etc/system and reboot:
set sata:sata_max_queue_depth = 0x1
Work Around
set zfs_vdev_max_pending (assuming you are only using zfs :-)