Exercise those storage devices
Data-communications equipment doesn't live in a silo. It lives in a network or other system with other devices and it most communicate with them. In other words, devices on a network or other communications link must interoperate with each other. Storage products that use PCIe (PCI Express) as a communications bus are no exception.
Before you even can test for interoperability, you must test a product or system to be sure it needs design specifications and it confirms to communications standards. Comprehensive test coverage can help you achieve that goal. Even if you have the right test tools, deploying them to the product test specifications can still result in to incomplete test coverage unless you understand the test tools' limitations, especially with PCIe.
Storage devices that communicate over PCIe taken a tremendous jump. In an effort to increase performance, protocol structures have become more advanced. Unlike the previous generation of storage devices, these new protocol structures use encoded data protection mechanisms that ensure reliable data transmission. For example, storage command structures and data for protocols like Non-Volatile Memory (NVM) Express, SATA Express, and SCSI Express are now encoded and contained in the data payload of PCIe (Figure 1. This very change that makes testing more challenging.
Over the years, designers and interoperability test engineers have used many test methodologies to determine how their new products behave when placed in real-world environments. Differences in test techniques and tool selection for each methodology are based on the unique attributes of the protocol and the reliability of the test stand to perform a task. A good example of this would be the use of "in-line error injectors" by storage engineers and technicians to identify various real-time issues associated with protocols such as SAS (Serial Attached SCSI).
In-line error injectors or "jammers," as they became known, are devices that manipulate or impair protocol traffic on a connection between two nodes of a communication link. Jammers deliberately create controlled error conditions, letting designers test error detection and recovery routines for new devices. It is important to prevent additional system behavioral changes or modification of transmission characteristics beyond the actual error insertion, for that changes the test result, which render it inconclusive.
A common test plan could include a list of tests describing how to create important error conditions and the criteria that will be used to determine proper or improper responses to the error stimulus. These error scenarios are then implemented in the tool and executed on the system under test. Simple tests may span insertion/deletion of protocol frames to substitution of a "good" response status with an "error" response status. The test help you see if the system recognizes the error and correctly responds to the condition. Through careful test design, error recovery mechanisms can be steadily refined until the device provides reliable operation under normal and stressed operating conditions. The overall goal is to ensure that the SUT meets design and quality specifications through comprehensive test coverage.
Jammer Test Capabilities
The jammer is usually connected by cabling directly between an initiator and target. Ideally jammers are electrically invisible and non-intrusive to the SUT. By programmatically controlling protocol traffic and introducing specific error conditions, the system response to various scenarios of failed communication can be tested. The real benefit of the jammer is that it can perform error testing including the following functions:
- Inject errors into a real world situation in-line.
- Bit error injection
- CRC modification
- Frame modification
- Link connect or disconnect
- Primitive modification
- Out of Band (OOB) and Speed Negotiation Window(SNW) modification
- Verify if SUT recovers from error conditions without data loss or corruption
This straightforward test methodology makes the tool indispensable for communication protocols used in storage systems. Parallel ATA, SCSI, Fibre Channel, and SAS/SATA test environments need this type of tool to determine how well the device performs in the system and how gracefully it responds when operating under less than ideal conditions.
Jammers were so effective for traditional storage applications because they grew out of a more simplistic test methodology. Early in the computer industry, storage communications between a host initiator and target devices was less complicated. When a host sends a command sequence, the target returns a status code byte indicating the command was successful, encountered an error, or indicated that the target was busy servicing other requests. It then moved onto the next command in the sequence or retransmitted until the message was finally successful. Rigid ribbon cables were used to transmit low speed data communications and lengths were limited by crosstalk noise from the cables themselves. It was during this time that the jammer became a standard tool for storage test applications. The process of setting up an error scenario (jam) was uncomplicated with few protocol safeguard mechanisms to interfere with line rate traffic modification on the test set up.