Serial attached SCSI simulator aids design choices

& -February 08, 2013

Here’s a look at how a SAS simulator developed by University of New Hampshire’s InterOperability Laboratory  can scale enterprise storage area networks in the age of big data.

Serial attached SCSI (SAS) currently uses an expander-based networking architecture. The expander acts as a pure port extender with no switching backplane involved, meaning there is minimal flow control and processing taking place. This architecture is severely limited for enterprises requiring larger storage area networks. In fact, once the number of connections starts to increase from tens and hundreds to thousands of connections, the entire network performance begins to decline.

T10, the standards body that defines the SAS standards, is working on defining switching support to address these limitations with larger networks [1]. Current simulators do not accurately model the SAS technology. With the advent of increasing device speeds, there is currently no way to evaluate corresponding performance gains. Thus, either extensive changes are required to existing simulators or an entirely new simulator needs to be created. In this article, we present a new SAS simulator that is used to understand performance implications of design choices.

SAS is a point-to-point and full duplex storage area network technology. It competes with Fibre Channel, Fibre Channel over Ethernet (FCoE) and Internet Small Computer System Interface (iSCSI) in the enterprise storage space [2]. SAS was designed with simplicity as a main goal, differentiating it from competing technologies. The challenge is that scalability is severely limited in order to avoid complexity.

There are three categories of devices used in the SAS domain: initiators, expanders, and targets. Initiators, sometimes called "filers," make requests to read and write data from targets, which contain the physical disks. An initiator is either directly attached to a target or networked through an expander. Expanders act as multiplexers, allowing one initiator to connect to several different targets. Often, expanders are located within targets. For our study, we considered each a separate device; however, by adjusting latency characteristics, one can model a single device containing both target and expander functionality.

Each device includes one or more ports, and each port includes one or more physical lanes. Ports that contain multiple physical lanes are called wide ports, and ports that contain just one physical lane are called narrow ports [3]. SAS communication partners are identified through 64-bit SAS addresses. Though vendors have the freedom to assign each physical lane its own SAS address, the SAS Simulator allows the assignment of only one SAS address to each port. The idea of simulating physical lanes with individual SAS addresses can still be achieved by creating multiple narrow ports, each with its own address.

The SAS protocol was designed such that expanders can remain low-cost and easy to implement. Each expander is composed of ports containing one or more physical lanes. Each physical lane in an expander is allowed to serve only one logical connection at a time, and a virtual circuit is achieved using crossbar circuitry. Within an expander, it is possible that several connections request the same physical lane. To decide which link gains the virtual circuit, expanders use an arbitration state machine. Logical connections possess an arbitration wait time (AWT), which is used to help ensure fairness. Expanders continuously increment the AWT for a connection between reception and transmission [4]. A connection with a larger AWT wins arbitration over connections with lower ones. This allows the expander to give priority to the competing connection that has been in the arbitration process longest. The winning logical connection is progressively decided using the following rules:

  1. The connection with the highest AWT is chosen.
  2. If the AWTs are equal, the connection with the highest source SAS address is chosen. This is the SAS address of the initiator opening the connection.
  3.  If the AWTs are equal and the SAS addresses are equal, then the device with the highest connection rate is chosen.
  4. If all of the above fail to determine a winner, any connection is chosen [4].
Whichever connection wins the arbitration gains the physical media in contest. Losing connections forward an OPEN_REJECT codeword back to the initiator. The SCSI Trade Association, the marketing arm of the SAS industry, releases training slides which further explain the SAS link layer [4].
Next: Title-1

Loading comments...

Write a Comment

To comment please Log In