What is MSCP?

MSCP is Mass Storage Control Protocol, which has disk and tape versions. It is a high-level protocol for talking to disks and tapes of arbitrary physical nature. Tape MSCP is known as TMSCP, but disk MSCP doesn't have a special name.

From the viewpoint of a PDP-11 user or a VAX user ignorant to anything other than UNIBUS and Q-bus, as the original 4.3BSD authors were, MSCP and TMSCP are different protocols, and each is just yet another kind of controller interface they can find on a UNIBUS or Q-bus, not conceptually different from, say, RK07 disks or TS11 tapes. This is the attitude to MSCP used in all versions of UNIX up through the 1986 4.3BSD release, and it still remains pretty much the same in 4.3BSD-Tahoe and the currently existing 4.3BSD-Quasijarus code. As we shall see momentarily, however, in a VAX OS this attitude doesn't lead to anything except mediocricity, and it is the Quasijarus Consortium's job to do away with it and make the MSCP code in 4.3BSD rise on new foundations.

Cluster Interfaces and the Systems Communication Architecture (SCA)

At exactly the same time when the first crate of UDA50s, UNIBUS MSCP controllers, shipped to anxiously waiting PDP-11 users, the DEC engineers were cutting the red tape on something much more exciting, namely, the Systems Communication Architecture (SCA).

SCA is an architecture for intercomputer communication and mass storage. At the very foundation of SCA lies the revolutionary concept that the communication between a CPU and a mass storage controller is a form of intercomputer communication. In principle this is nothing new, as DEC mass storage controllers had on-board control microprocessors on them long before the SCA was invented, but SCA makes this very explicit and raises this to much higher levels.

By making intercomputer communication and the communication between a CPU and a mass storage controller one and the same thing, SCA creates an exciting possibility. Instead of equipping computers with local mass storage, computers and mass storage controllers can be made into nodes on a very high-speed intercomputer bus communicating via SCA. If this bus has more than one CPU node attached to it, all storage controller nodes on it become equally shared among all CPUs.

Computer Interconnect (CI), a transformer-coupled star topology coaxial cable interconnect, is one such intercomputer bus. Digital Storage System Interconnect (DSSI) is another. DSSI runs exactly the same protocols as CI and is identical to CI from the SCA viewpoint, but uses a much cheaper physical layer. The DSSI physical layer is a SCSI-like linear 50-conductor interconnect.

SCA doesn't stop here, however. As we shall see below, the SCA concept of making the communication between a CPU and a mass storage controller a form of intercomputer communication applies even to the more traditional hardware configurations without CI or DSSI.

The Relationship between MSCP and SCA

MSCP is the protocol defined by SCA for communication between an operating system's device drivers and mass storage controllers. Despite what PDP-11 and U/Q-confined VAX users may think, MSCP never existed separately from SCA and has been an integral part of SCA from day 0. As explained below, MSCP is an SCA SYSAP.

The only reason why PDP-11 users can think of MSCP as something not much different from RK07s and TS11s is because DEC has taken special steps to create a subset of SCA/MSCP that can be implemented within the confines of the primitive PDP-11 architecture, whereas the full SCA requires a VAX. This will be explained in detail momentarily.

First, a few SCA definitions. A port is a computer's hardware connection to other nodes that it can communicate with via SCA. A storage controller is an entity that controls mass storage devices, communicates with computers via SCA, and speaks standard protocols defined by SCA. A system application (SYSAP) is some end-user function that SCA is serving. Examples of SYSAPs are MSCP$DISK (disk MSCP), MSCP$TAPE (tape MSCP), and INET$RFC_790 (a SYSAP for routing IP traffic over cluster interfaces).

An SCA implementation within an operating system must, in addition to the common SCA code, include the following components: class drivers, port drivers (PDs), and port-to-port drivers (PPDs). A class driver supports a SYSAP, a PD supports a port, and a PPD supports the communication with other nodes over a port.

The original SCA design defined three ways to build a mass storage system based on SCA. These are:

Cluster storage

With this approach, the storage controller is a separate node on an intercomputer bus. Each computer attached to the bus has an interface port that does this attachment. Neither the bus nor the port knows or cares about what communication takes place on the bus, whether it is between different computers or between a computer and a storage controller, and what SYSAPs are being used. This is the most generic approach.

At the time when SCA was first defined, DSSI didn't exist and CI was the only intercomputer bus. Storage controller nodes attached to CI are called Hierarchical Storage Controllers (HSCs). VAXen were the only computers for which CI adapters were natively made (PDP-11s are too weak), and all CI adapters use Generic VAX Port (GVP). See Buses and Peripherals for more information about GVP. In order to support CI, an SCA implementation needs a port driver for the CI adapter and a port-to-port driver for the CI bus protocols.

BVP Storage System Port (BVP SSP)

For users who do not need to share a storage controller between multiple computers it becomes advantageous to attach it directly to one of the computer's internal buses, combining the SCA functions of the port and the storage controller into one device. Such combined ports/controllers will hereafter be called local MSCP ports. For systems where this bus is VAXBI (DEC's best bus at the time), BVP SSP is ideal. As its name indicates, BVP SSP is based on BVP, a form of GVP with four standard longword registers, meaning that just like a CI adapter, it is specific to the VAX architecture and takes full advantage of it.

Since BVP SSP combines the functions of the port and the storage controller, the BVP SSP port driver is also the port-to-port driver.

UNIBUS/Q-bus Storage System Port (UQSSP)

Just like BVP SSP, UQSSP combines the functions of the port and the storage controller. However, unlike VAXBI, UNIBUS and Q-bus are PDP-11 buses, not VAX ones. Not only does this mean that the port functions must be much simpler, but the entire design must accomodate the primitive PDP-11 architecture. In order to mimic the traditional Chinese wall between disks and tapes that exists in the PDP-11 world (but not in SCA), a UQSSP port is normally either disk only or tape only, and disk and tape UQSSP ports appear at different CSRs. (Since this separation of disks and tapes is so much against the spirit of SCA, the UQSSP spec doesn't enforce it, and there were a few UQSSP implementations made that support both disks and tapes on the same port.)

UQSSP was designed so that a PDP-11 OS or a simple-minded VAX OS that's oblivious to anything except UNIBUS and Q-bus can support UQSSP devices by writing an "MSCP" driver that implements UQSSP and the MSCP$DISK SYSAP with all SCA stuff sealed within the driver, writing a "TMSCP" driver that implements UQSSP and the MSCP$TAPE SYSAP, again with all SCA stuff sealed within it, and keeping the rest of the OS totally oblivious to SCA. It goes without saying, however, that doing this in a general-purpose VAX OS that is supposed to be extendable and portable, which is what the original 4.3BSD developers did, is nothing but a demonstration of ignorance and short-sightedness.

In a real VAX OS with a full SCA framework, UQSSP is supported by a port driver, which just like the BVP SSP one also acts as a port-to-port driver.

The Real World

According to the information available so far, the above model was defined some time between the shipment of VAX-11/730, the last VAX shipped with non-MSCP mass storage, and the shipment of VAX 8600, the first VAX to ship standard in the CI VAXcluster configuration. At that time the VAXBI bus was in its earliest design stage, and the two BI bus VAXen (8200 and 8800) just had their dev teams started and SID codes assigned to them. Two or three years passed before these two VAXen actually shipped, and there were two more VAXen, MicroVAX I and II, that were both invented and shipped while VAXBI was in development.

As a result of this, BI and XMI local MSCP ports ended up being a little different from the way they were originally envisioned. Originally all BI local MSCP ports were envisioned as using BVP SSP, and UQSSP was left specifically for PDP-11s and VAXen with PDP-11 buses (UNIBUS and Q-bus). The original BI disk controller was codenamed HSB (note the similarity with HSC) and used BVP SSP. However, by the time the BI bus VAXen actually shipped, for some reason, presumably cost, DEC decided to replace the HSB with the more modest KDB50. As evident from its name, KDB50 is a reworking of the UNIBUS UDA50 and Q-bus KDA50, both of which are UQSSP controllers. It tacks a bit of nexus glue on top of the UQSSP port (namely, the logic to grok VAX page tables and do paged DMA instead of the straight U/Q one) and effectively ports UQSSP to the BI bus. KDM70 does the same for XMI. A VAX OS should support this by extending the UQSSP port driver to handle these pseudo-UQSSP ports too. (Incidentally, KDM70 is a good example of a UQSSP port that supports both disks and tapes. This is OK because a VAX OS with a full SCA implementation has only one UQSSP port driver, and MSCP$DISK and MSCP$TAPE are class drivers.)

This flashback to UNIBUS is even more explicit on the BI KLESI. Instead of tacking VAX nexus glue on top of a hacked UQSSP port, it takes the absolutely unchanged UQSSP port and the absolutely unchanged UNIBUS adapter and slaps them on the same board. Except for a different BI device type code, a BI KLESI is a UNIBUS adapter with a UNIBUS KLESI sitting behind it at the hardwired CSR of 174500. Supporting this is a one line change to recognize KLESI's BI device type code as being another code for the UNIBUS adapter.

Fortunately, the original BVP SSP isn't dead either. BI TK50 and TK70 ports use it, and there may be others that do. And who knows, maybe they did actually make some HSBs at some point, and there may be some happy hacker out there who has one.

This brings us to the list of three different types of BI/XMI local MSCP ports:

A robust open-ended and continuously maintainable VAX OS should support all three, and it shouldn't make any a priori assumptions about any of the three port types being used only by specific devices. An OS that supports UNIBUS, BI, and XMI and has full UQSSP and BVP SSP port drivers in its SCA framework will support arbitrary UNIBUS, BI, and XMI local MSCP ports using any of the three port types.

As it happens, both CI adapters and BVP SSP ports use GVP. Therefore, the CI and BVP SSP port drivers have a lot of common code. Normally code duplication is avoided by making a common port driver component called the GVP port driver, used by both CI and BVP SSP PDs.

Finally, the last change that SCA has undergone is the introduction of DSSI. DSSI runs exactly the same protocols as CI and thus shares the same port-to-port driver (PPD). However, it's physical layer is completely different, and there were several different hardware DSSI adapters made.

The first DSSI adapter made was the DEC SII chip. The DSSI physical layer is really hacked SCSI, and SII is just a somewhat special SCSI chip that can work in DSSI mode. Although apparently it is hacked significantly enough to make it impossible to use a standard SCSI chip as a DSSI adapter, the same SII chip implements both SCSI and DSSI. SII is just as dumb as an off-the-shelf SCSI chip. It only implements the lowest-level stuff and does DMA to a static RAM buffer. This means that SII requires a new port driver in SCA. For historical reasons, this port driver is called the MSI PD. MSI stands for Mayfair Storage Interconnect, which is a reference to the Mayfair (MicroVAX 3xxx) series where DSSI and SII were first invented.

SII was quickly succeeded by SHAC (Single Host Adapter Chip), which is much better. Unlike SII, SHAC is very intelligent. SHAC's design goal was to create a DSSI chip with the software interface of a CI adapter and reap all benefits of GVP. SHAC meets this goal perfectly, as its software interface is so close to that of CI adapters that it is supported by the CI port driver. This feat has cost DEC a lot of transistors. Whereas SII is just a hacked SCSI chip, SHAC includes a VAX host interface that attaches directly to the CDAL bus and implements GVP with all of its memory management and interlocked queue bells and whistles, an interface to the SCSI-like physical layer, and an embedded RISC microprocessor, all on-chip.

What does this mean for 4.3BSD-Quasijarus?

Unfortunately, being ignorant to everything except the UDA50s they had sitting in front of their noses, the original 4.3BSD developers have abused DEC's friendliness to PDP-11s, and instead of implementing SCA generically, they have made a single monolithic driver for the UQSSP/MSCP$DISK combination and didn't support anything else.

This was in the days of 4.2BSD, at the same time when DEC stole the BSD code to make Ultrix. They have almost certainly cursed Berkeley's attitude to VAX hardware support, but the goal of shipping Ultrix with MicroVAXen I and II and their TK50s requiring getting TMSCP working with no time wasted. As they didn't have time to write a full SCA framework, they copied Berkeley's uda (UQSSP/MSCP$DISK) driver and created a new tmscp driver for the UQSSP/MSCP$TAPE combination. This driver was donated back to Berkeley and made its way onto the 4.3BSD tape. As a result, the original 4.3BSD release only supported UNIBUS and Q-bus local MSCP ports, with different drivers for the UQSSP/MSCP$DISK and UQSSP/MSCP$TAPE combinations. The same was true for Ultrix V1.x.

After that Berkeley and DEC went their own separate ways. In 4.3BSD-Tahoe Chris Torek hacked the UQSSP/MSCP$DISK driver to support KDB50 as well. To the user it now looks like two different drivers, uda for UNIBUS MSCP ports and kdb for KDB50, but they share some common code. uda supports the 4.3BSD-Tahoe disk labels, but kdb doesn't. This situation remains in the current 4.3BSD-Quasijarus code. Also Chris didn't add the one-liner to the BI device type switch that would recognize BI KLESI as a UNIBUS adapter and automagically support it (as he told me, he wasn't aware of the BI KLESI being a UNIBUS adapter in disguise). I did it in the 4.3BSD-Quasijarus0a release, though, so BI KLESI is now fully supported.

DEC did much better. First, when they supported VAXBI in Ultrix V2.0, they also needed to extend the old uda driver to grok KDB50. However, unlike Chris Torek, they were already making plans for a full SCA implementation, and they knew that in the new system KDB50 would be supported by the UQSSP driver, so in Ultrix V2.0 KDB50 was supported by the same uda driver as U/Q MSCP ports, not like Chris Torek's separate uda and kdb drivers.

And yes, they did support BI KLESI too. Incidentally, they didn't do it with a one-liner as I did, instead they have a whole bunch of screwy code making BI KLESI appear as a different device to the user than a UNIBUS adapter. The only reason I can think of why they did it this way is pure marketing, i.e., they didn't want people to know that BI KLESI is UNIBUS in disguise. Quasijarus Project is free from such dogmas and took the much more straightforward approach.

In Ultrix V3.0 they finally did away with the uda and tmscp drivers and implemented SCA properly. Ever since V3.0 Ultrix has had 4 port drivers: UQSSP, BVP SSP, CI, and MSI. The CI and MSI PDs use the CI PPD, and the UQSSP and BVP SSP PDs act as PPDs themselves. Ultrix has MSCP$DISK and MSCP$TAPE class drivers and supports MSCP disks and tapes seen either over a cluster interface or on a local MSCP port. Ultrix also has an INET$RFC_790 class driver and can route IP traffic over cluster interfaces. However, Ultrix isn't VMS, and it doesn't support the VMS-specific clustering SYSAPs or participate in VMS clusters.

4.3BSD-Quasijarus should do the same. The IFCTF possesses the complete sources for Ultrix V4.2, and the Ultrix SCA framework should be ported to 4.3BSD and incorporated into 4.3BSD-Quasijarus in its entirity. It will take some work, though, since Ultrix internals are quite different from 4.3BSD ones. It will also be very important to be able to understand the code being ported, so any SCA manuals will be very helpful.

KA410 MFM Controller

One final issue related to MSCP is the MFM disk controller on KA410 BabyVAX. Back in the days of MicroVAX II, DEC has made an MSCP controller called RQDX3. It controlled MFM disks via an SMC HDC9224 chip, and it had an on-board T-11 control microprocessor. The T-11 communicated with the host CPU via SCA/MSCP and controlled the HDC9224. Then they made the KA410 BabyVAX. KA410 used the same MicroVAX II CPU chip, MFM disks, and TK50 tapes as MicroVAX II, and it was targeted for the same marketing space as MicroVAX II. However, it was significantly cost-reduced. KA410 is a BabyVAX, meaning that it is a single board lacking any expansion bus, it is a poor man's VAX, meaning that I/O devices can't do scatter-gather DMA to main memory, and it doesn't have any microprocessors on it except the VAX CPU itself. The last point means that the HDC9224 controlling the MFM disks is exposed directly to the CPU, without MSCP standing in between.

Since there are no (micro)processors other than the CPU itself anywhere in sight, there is no one to communicate with, and therefore the SCA framework is of no help on KA410. KA410 HDC9224 has to be supported by a "plain" device driver, without involving SCA. This is what Ultrix does. This driver is somewhat tricky, however, since DEC has made the MFM disks interchangeable between RQDX3 and KA410, meaning that the KA410 HDC9224 driver has to emulate MSCP disk identification and bad block handling.

The last point about KA410 HDC9224 that requires explanation is how VMS handles it. To an unsuspecting VMS user, KA410 MFM disks appear as DU devices, just like MSCP disks under VMS. This has misled many people, including myself originally, into thinking that VMS somehow manages to attach the HDC9224 driver to the SCA framework.

SCA stands for Systems Communication Architecture, and with no (micro)processors other than the CPU itself anywhere in sight, there is no one to communicate with. A normal SCA port driver doesn't do anything except passing messages and datagrams over to the other party. The way VMS makes it look to the user is as if the HDC9224 driver was an SCA port driver that did in software on KA410 what the T-11 does in hardware on RQDX3. In other words, it looks as if it implements an MSCP controller in software.

On a closer look, however, implementing an MSCP controller in software is not as simple as it looks. SCA is highly symmetric, and whenever two parties communicate via SCA, each has a full SCA implementation in it. A MicroVAX II with an RQDX3 running a full VAX OS has two SCA implementations running: one in the OS running on the CPU and the other in the firmware running on the T-11. Duplicating this with a "software MSCP controller" would mean writing an OS with TWO SCA implementations in it, communicating with each other. Not only is this insane, but it would also severely degrade performance.

And no, VMS doesn't do this. It merely makes it look like it does to fool the poor unsuspecting users. Why does it do this? The only reason I can think of is to satisfy the marketing department's demand to make MicroVAX II and KA410 look and feel the same. In a similar vein, VMS makes the SCSI TK50Z tape drive look like an MU device on the KA410, again fooling the user into thinking that it has something to do with MSCP, which in this case is even more nonsense than in the case of MFM disks, as TK50Z is just a normal SCSI tape drive. Fortunately, Ultrix doesn't try to sell KA410 MFM disks and TK50Z tapes as MSCP (probably only because UNIX is not as tolerant to such crap as VMS is), but it still doesn't support arbitrary SCSI devices on KA410 (see SCSI).

4.3BSD-Quasijarus will support HDC9224 just like Ultrix does, probably by borrowing the driver from Ultrix. The SCSI port will, however, be supported as a normal SCSI port, see SCSI.

This page has been last updated on 22-MAY-2000.

@(#)mscp_clu.html 1.1 03/12/18

Michael Sokolov
msokolov@ivan.Harhan.ORG