#
Upgrading and Repairing Networks

# - 5 -

# The Server Platform

The file server is the focal point of the client/server network model. It is the means by which users on the network gain access to the resources that they need to accomplish their daily tasks. The operative concept behind the local area network is to spread a company's computing resources around the network, instead of having everything centralized as in a mainframe system. The file server functions as a gateway to all these resources, whether they've been located in the server computer or not. The server also functions as a security device, allowing users access only to the resources that they need and protecting the core operating system from intervention that could interrupt the operations of other users.

Once access is granted to the necessary network resources, users are actually performing their computing tasks using the hardware within their workstations. This is the basis of the client/server model. Processing tasks are performed by every computer on the network, decreasing the load on the central systems. Unlike a mainframe system in which all processing takes place within one computer, file servers allow individual computers to perform the same functions without the need for redundant resources at every workstation. Thus, one centrally located copy of a particular software application can be accessed by dozens of users, each using the processor and memory of their individual machines to run it and outputting their final results to a single, centrally located printer without placing an excessive burden on the server. The file server can be substantially smaller than a mainframe in size, processing capability, and most importantly cost. For a far smaller price than the "big iron," a local area network can be assembled that will provide users the same (or greater) functionality, along with far greater flexibility and fault tolerance.

Many people who are new to networking believe that there is an inherent difference between a file server and a normal workstation PC. From a pure hardware standpoint, this is nearly always not the case. The primary difference between a file server and a workstation computer is the software that is running on it. As far as the hardware is concerned, the differences are mostly a matter of degree and marketing. A file server is simply a PC with the same resources as a workstation but in greater abundance. It usually has a faster processor, more memory, a greater amount of disk space, and a wider array of peripherals--but this need not be so. The primary difference is that this machine runs a network operating system (NOS) such as Novell NetWare, as opposed to a client operating system (OS) such as DOS. See part III, "Software Platforms: NOSs and Clients," for an in-depth examination of the differences between NOSs and other OSs.

## How Many Servers?

As stated earlier, the basic concept of the LAN is to spread resources around the network. In many cases, this results in the use of multiple file servers that provide network access to different areas of the enterprise. Depending on the size and needs of the company, the use of multiple servers can provide several advantages as well as disadvantages. The primary alternative to this is a server configuration that has gained increasing popularity in recent years, colloquially called the super-server. This consists of a single file server that takes the place of several smaller servers. This machine has a far greater amount of hardware resources than a normal server does. It ends up functioning in almost the same way as a mainframe, in that it is the single contact point of the entire network.

The primary disadvantage of the super-server is the same as that of the mainframe--fault tolerance. When operating in a distributed network environment (that is, one utilizing multiple servers spread around the network) users can, in the case of a server failure, log on to another server and continue working, as long as the network has been designed and configured to accommodate this sort of redundancy. In a super-server environment, users tend to be completely dependent on a single machine. Should it fail for any reason, productivity often comes to a halt until repairs are made.

One of the primary advantages of the super-server, though, is the fact that administrative tasks are centralized. Only one set of user accounts needs to be maintained, and the server hardware is all located in one place, rather than having multiple file servers located in different areas of the office, building, or campus.

One of the first questions to be considered when planning an upgrade or expansion of an existing network is whether to augment the capabilities of existing file servers or to add additional servers to the network. Hardware costs are certainly a major factor in this decision, but there are other criteria to be taken into account as well. One of these is the skill level of your user base. When administering a large network, having knowledgeable users or workgroup administrators spread throughout the enterprise can be a tremendous aid to the MIS department. If these people can be counted upon to responsibly administer the small parts of the network local to their department, then a distributed network might be more desirable. If the user base is familiar only with the operations of a workstation and has little knowledge of networking, then having fewer file servers of greater capacity allows for simpler administration and less traveling time between servers or departments.

Other criteria include the working habits and the geographical organization of the enterprise. Discrete departmental workgroups in remote locations, with little interaction and different computing requirements, would be better candidates for multiple servers than a large group of identically tasked workers in one location. Of course, circumstances are rarely this clear-cut, and the decision as to the number of servers needed is likely to be a compromise between several of these factors.

As we examine the various components of the network server, we will cover the ways in which the hardware can be assembled to provide a smooth passage of data from source to destination that is free of bottlenecks. Other sections of this book examine the ways in which a suitable level of fault tolerance can be provided for a selected hardware configuration and also accommodate the needs of the organization that use it. We will also attempt to identify the components that are the most critical to particular types of network use. This way, you should be able to find information that allows you to create a network or upgrade into something that suits the way your business works--instead of the other way around--and provide additional performance or fault tolerance in the most critical areas and in the most economical manner.

## About the Motherboard

Although the microprocessor or CPU is often thought of as defining the capabilities of a computer, it is useless without an architecture designed to support it properly. At the most basic level, a computer operates on data by performing three fundamental operations: storage, transfer, and calculations. Data may be stored in memory or on a hard or floppy disk drive, and calculations are performed within the microprocessor. The task of providing the means by which the data is transferred from the storage medium to the processor and back again is accomplished by the motherboard.

In years past, several different means were used to connect the disparate parts of a computer. Minicomputers and mainframe systems used backplanes or wiring systems of a modular design that allowed for the simple replacement of individual parts. The space considerations of the modern PC, however, have led to the use of a single board onto which the processor, system memory, I/O bus, and other vital circuitry are all integrated. This is referred to as the system board, main board, or motherboard (see fig. 5.1).

Figure 5.1
This is a typical PC motherboard.

In this chapter, we examine this crucial component to a well-designed file server, as well as all the basic components that are mounted on a motherboard. We look at the means by which the different parts of the computer communicate and at the little-known ancillary components that can be crucial to the assembly of a system with true compatibility. A time will come when it seems as though technology has passed your servers by, and the only alternatives will be to upgrade or replace them. It is important to know what you have before you improve it, however, and with this book, you should be able to judge whether or not you need "the latest thing" in the networking industry and whether or not you can easily adapt your existing system to it.

### Documentation

If there is one piece of advice that you retain from this entire book, it should be this one: Document Your File Server's Motherboard! Do not buy a PC for use as a file server (or even as a workstation, for that matter) without finding out specific information about the motherboard. If the vendor cannot tell you who makes their motherboards, hang up the phone! If they cannot tell you what chipset or BIOS is on their motherboards, hang up the phone! If they cannot furnish detailed documentation for their motherboards (by this I mean more than an 8-page pamphlet), do not buy from them! Hardware incompatibilities are a major cause of file server outages, and if you do not know what you have inside the box, then no one can help you.

When purchasing PCs as a unit, as opposed to assembling them yourself, you must be aware of the places in which unscrupulous vendors are most liable to cut corners. They realize that potential customers are likely to ask what make of hard drive is in the machine, or how much memory it has in it, but they are far less likely to ask about the speed of the memory, or the chipset on the motherboard. Any reputable vendor should be able to supply you with a spec sheet containing all the technical details concerning a particular system. If they can't, look for a different vendor!

If you are dealing with legacy equipment for which you have no documentation, then use a notebook, spreadsheet, or text file to keep your own documentation. As you identify aspects of the hardware in a machine, write them down in a safe place. This will save you from repeatedly having to open the system to see what's inside and will help prevent you from sounding like a fool when you're on the phone with vendors, salespeople, and consultants.

Many simple upgrades can end up being a nightmare. Always budget what you believe to be a sufficient amount of downtime when upgrading a server, and then double it. Murphy's Law always seems to be in force when you've got the case open, and the guts of your file server are all over the floor. A properly documented system often means the difference between a successful upgrade and having to put the server back the way it was because you ordered the wrong parts.

## Microprocessors

The microprocessor, sometimes known (rather inaccurately) as the central processing unit or CPU, is the place where all the actual calculations within a computer take place. Hard drive space, memory, and even caching RAM are all merely storage media of different types and speeds that facilitate the delivery of data to the CPU, where the actual processing takes place. A microprocessor is built around thin silicon wafers that have been grown like crystals and subsequently exposed to intense heat. During the heating process, the wafers are exposed to gases containing particles that are impregnated into the silicon, thus creating different degrees of conductivity in precisely specified areas of the chip. Since the late 1950s, when the first techniques for clustering multiple transistors on a single chip were patented, these integrated circuits have evolved in complexity and capability at a phenomenal rate, resulting in today's microprocessors, each of which represents the equivalent computing power of a football stadium full of 1950s vacuum tubes.

On the simplest level, a microprocessor is a component that, when furnished with a particular set of electrical signals input at specified points, will always return exactly the same response. The phenomenal achievement of these devices is the speed at which millions of these responses can be generated and the way in which sequential responses can be made to interact, based on previous responses. These input signals correspond to the actions that a program requests from a processor and the data that the actions are performed upon. Every microprocessor has a vocabulary of these actions, called its instruction set or command set, which is used to form the language by which programmers create their code. The code, which is stored on the computer's memory chips, on a hard drive, or on some other storage device, is then reduced to binary electrical signals that are the only means of communication with the microprocessor and are fed to its input/output (I/O) unit.

The I/O unit of the microprocessor is essentially the staging area for the electrical input, precisely controlling the rate and timing by which signals are fed to the other two parts of the chip, the control unit and arithmetic-logic unit (ALU). In strict technical terms, it is these two units, exclusive of the input/output unit, that compose the CPU. The I/O unit consists of two kinds of connections to the motherboard of the computer: the address bus, which furnishes the processor with information about the memory location of the commands and data to be processed, and the data bus, through which the information itself travels. Once the signals are fed to the control unit, this part of the chip's architecture then relays the data and command instructions to the ALU, where the actual calculations are performed at the proper time. The control unit contains its own internal clock which is used to control the rate at which all the units function, but the entire processor is also highly reliant on the motherboard's system clock, which is used to ensure that the vast number of electrical signals entering and exiting the microprocessor are not garbled or confused.

All three of these basic units, however, have an influence on the overall speed of the processor. The width and speed of the I/O unit's bus control the rate at which data enters and exits the processor, and the ALU controls the amount of data that can be operated on at any one time by the size and number of its registers. Registers are holding areas where data is temporarily stored while calculations are being performed by the ALU (see fig. 5.2). Because even simple arithmetic calculations require multiple steps when they are reduced to binary electrical signals, the registers are an integral part of the decision-making process, retaining the interim results of calculations within the processor itself, instead of having to offload this information through the I/O bus to the computer's memory chips.

Figure 5.2
Microprocessors store interim calculation results in their internal buffers to avoid delays caused by repeated memory writes.

### CISC and RISC Processors

Although different microprocessor platforms, on a high level, may be performing what are fundamentally the same operations, they often go about them in very different ways. Even a simple arithmetic calculation, when it is reduced to binary electrical signals, is performed in a series of steps, the number and complexity of which may vary greatly in different types of processors. This is due primarily to differences in the various processors' instruction sets. This can be compared to two sentences with the exact same meaning, expressed in two different languages, using two different alphabets. The meaning transmitted is the same, but the intermediate steps can be utterly unlike each other.

The first microprocessors executed many of the instructions in their command sets through the use of numerous intermediate steps. The various combinations of these intermediate steps allowed a large number of highly complex instructions to be added to the command set. These intermediate steps, by which instructions were carried out, came to be known as microcode. The processor's microcode is essentially the alphabet upon which its command set (or language) is based. It runs on a separate operational level within the microprocessor, called a nanoprocessor.

This technique allows the processor a rich and varied instruction set but also introduces an added layer of processing overhead that can lessen the overall speed at which instructions are executed. This has become known as complex instruction set computing or CISC, and is the operational model for the entire Intel PC microprocessor family. In the 1970s, it was discovered during studies of the actual usage of CISC microprocessors of the day, that roughly 20% of the instructions in the command set were being used to perform 80% of the work. In other words, only a small subset of the existing vocabulary was being used, most of the time. This discovery led to the development of microprocessors that were designed to optimize the performance of the most frequently used commands, at the expense of the seldom used ones. This resulted in what became known as reduced instruction set computing (RISC). RISC processors eliminate the microcode layer that slows down the CISC processing method, substituting a relatively small vocabulary of simpler commands that can be combined to adequately emulate the seldom-used, more complex commands that comprise the other 80% of the typical CISC processor instruction set.

At the time of their introduction, RISC processors, by streamlining the command set and reducing processor overhead in other ways, significantly reduced the number of clock cycles required to execute a specific instruction and were clearly faster than their CISC counterparts in many operations. However, the continued refinement of both techniques has resulted today in microprocessors of both types that are far more comparable than their predecessors were. CISC command sets have also been streamlined to some extent, and RISC command sets have been enlarged to accommodate today's needs. Comparing the state-of-the-art in both schools of microprocessor design today demonstrates that, while the fastest processors available today still use the RISC model, both processor types offer the speed and the flexibility required by contemporary operating systems.

In addition, the ever-expanding market for these processors has resulted in development cycles that are significantly shorter than they were only a few years ago. Faster and more powerful processors are continually being introduced, and the benchmark ratings of the RISC processors used in many UNIX and Windows NT machines can now be all but matched by the Intel platform, with its Pentium and Pentium Pro processors. When considering an operating system that can run on machines using either CISC and RISC processors, such as Windows NT or SunSoft's Solaris, the RISC solution is likely to provide more pure speed, but this should not necessrily be the deciding factor in the purchase. Costs between the two processor types can vary widely, and care should be taken to decide whether the additional performance gained is worth the additional expenditure. The availability and cost of application software should also be a part of the equation.

## File Server Processing

In a network file server, due to the nature of the client/server model, the speed and effectiveness of the microprocessor often do not have a major impact on the overall performance of the network, beyond a minimum required level of adequacy. On most networks, the majority of application processing is carried out by the workstation's processor. While it is obviously an essential part of the system, the microprocessor in the average file server should not be overtaxed by the administrative functions (such as network communications, file sharing, and printer sharing) that are its primary everyday tasks, unless the processor is grossly underpowered or the server is heavily overloaded. In these cases, upgrading the processor can provide little return for what might be a large expenditure in time, money, and effort.

However, in the case of servers that run database applications or other processor- intensive tasks, it is important to ensure that the processor is not overburdened because delays or even stoppages in logons and network file access can result. Machines such as these can more accurately be called application servers, rather than file servers. If your business requires the use of processor-intensive server modules such as these, then this is a good reason to consider using multiple servers to separate these functions from the traditional network tasks, rather than overburdening a single machine's microprocessor.

As the networking industry progresses, we should begin to see a clearer division between application and file servers. This will result naturally from the trend toward heterogeneous networks composed of mixed platforms, and the continued development of network operating systems and hardware configurations that are clearly better suited for one or another of these tasks. In the next few sections, we examine some non-Intel processors available for network use today (see chapter 6, "The Workstation Platform," for a detailed discussion of the Intel family of microprocessors). We also discuss the future in which the use of multiple processors in a single server will allow for even more powerful applications to be run across a network, using the client/server model to spread the resource load more efficiently between server and workstation.

### Novell NetWare

While different NOSs are designed for use on machines running different types of microprocessors, and while some NOSs--such as Microsoft Windows NT and Sunsoft Solaris--are available in versions compiled for various different processors, the most commonly used NOS, Novell NetWare, is currently limited to operating solely on the Intel processor platform. NetWare is used on such a vast majority of network file servers primarily because of the ease and flexibility with which it provides resource sharing and security features, two basic requirements of a network. Although it is likely that the future will see something of a reduction in the overall market share that NetWare currently enjoys, it is equally likely that many networks will continue to run NetWare servers for basic networking tasks.

While the NetWare versions currently shipping can run on any PC with an Intel 80386 microprocessor or better, it is economically and technologically impractical, given the state of today's market, to consider running a NetWare file server on anything less than an Intel 80486 processor. Most of the 386 file servers constructed during the time when that was the cutting edge of processor technology have long since been relegated to the status of workstation.

I have no doubt that there are still many sites running 386 servers, and as long as the services required from the network remain unchanged, this is fine, but attempts to upgrade these machines are probably not worth the effort. Technology has advanced considerably in the past few years, not only in processors, but in memory, storage systems, and expansion buses as well. To get one of these machines up-to-speed for today's definition of a file server, you probably would be left with nothing from the original machine but the case, and even this might be inadequate.

486 and Pentium machines do, however, have processor upgrade paths, but due to the many different chips on the market, these paths may be somewhat convoluted, to say the least. The process of upgrading a NetWare server's microprocessor is no different from that of a workstation. For this reason, details of the upgrade process, as well as material on the history and architecture of the Intel microprocessor line, may be found in chapter 6, "The Workstation Platform."

### Buying New Processors

Although processor upgrades may be complex, if there is one component in a new NetWare file server whose purchase can be considered a no-brainer, it is that of the microprocessor. Buy a Pentium. While I would not have said this during the initial release period of the 60MHz and 66MHz models, the technology is now more than stable enough to warrant reliance on the currently available family of chips, ranging in speed from 75MHz to 133MHz. Sufficient care is now being taken to ensure that the chips are properly cooled. At this point in time, however, I recommend a certain amount of caution concerning the new Pentium Pro units, based solely on the fact that they are the result of a new manufacturing process and have not yet been fully proven in the marketplace. Remember that your file servers are not a place to experiment. Buy the fastest, most reliable products that you can afford--and since you probably will end up paying less for a processor than for nearly any other major component of a server, why not get the best available?

### Other Microprocessors

While the Intel platform dominates the file server microprocessor market by a tremendous margin--due primarily to the enduring popularity of Novell NetWare--it is by no means the only game in town. The growing popularity of Windows NT and the explosive growth of the Internet have boosted sales of several alternative processor platforms. The cross-platform support provided by Windows NT is a clear indication of the direction in which the networking industry is progressing. SunSoft Corporation also supports several different processors with its Solaris operating system, and rumors and announcements have been circulating for years concerning a processor-independent version of NetWare. While nothing is likely to disturb Intel's reign in the desktop and home PC markets, several of the cross-platform NOSs now available might be better served by non-Intel processors.

While many of the UNIX platforms available are more mature, they have also become more specialized. Our focus in this section is on platforms supporting the newer NOSs that have attracted the attention of many exclusive NetWare shops, due to the growing number of powerful client/server applications now available and in development, which run on them. The future of networking will be in open, mixed environments in which compatibility is a given and platforms can be chosen for their suitability to the task at hand, rather than their ability to conform to the needs of a restricting, proprietary architecture. This philosophy has given a new lease on life to many processor manufacturers who at one time faced a limited future in the commercial networking industry.

MIPS Technologies. Like the other major processor manufacturers who rival Intel, MIPS Technologies markets a line of RISC-based microprocessors that are used for a wide range of applications, from embedded electronics and desktop workstations to high-end graphics terminals and supercomputers. Their R4x00 line of processors is used in workstations that can run Windows NT 3.5 or UNIX V.4 as well as specialized online transaction processing systems (OLTP). Windows NT support allows for compatibility with the entire line of x86-based 16-bit DOS and Windows software, as well as the newer 32-bit applications being designed for the NT platform, thus opening the door to a whole new world of users.

The MIPS R4x00 microprocessors are based on a true 64-bit architecture, with full 64-bit registers, virtual address space, and integer and floating-point operations. An extended 36-bit address bus gives them a 64G physical address space. The capability for a 64-bit-wide virtual address space is what makes them true 64-bit processors with a virtual address space of 1 terabyte. Able to execute one instruction per clock cycle, the R4400 has separate 16K level 1 write-back caches for instructions and data. The caches are virtually indexed, to allow for simultaneous data accesses and the data cache has a two-entry store buffer so that two store operations per cycle can be executed without latency penalties or the need for instruction pairing, as in the Pentium. There is also a one-line write buffer that allows processing to continue while output is waiting to be written to memory. Support is also provided for communication with a level 2 cache up to 4M in size over a 128-bit data bus. Using standard static RAM chips, the level 2 cache can be configured as a unified cache or split into instruction and data caches.

Other processors in the MIPS line are designed for more specialized applications. For example, the R8000 utilizes a separate FPU that, in combination with its integer unit, can execute up to four instructions per clock cycle. It is now being used in 3D and graphics workstations manufactured by firms like Silicon Graphics.

DEC Alpha AXP. Digital Equipment Corporation, in designing its Alpha AXP architecture, set about plotting the growth of microprocessor technology for the next 25 years. By quantifying the advancements made since the introduction of the first microprocessors, DEC has predicted that the processors available 25 years from now will have to be 1,000 times faster than current technology to keep pace with other developments. They have, therefore, eliminated anything from their designs that they felt would become a bottleneck at any time during that period. For example, since they saw the possibility for an ultimate limit in clock speed, they concentrated on executing more instructions per clock cycle. Indeed, this has become a primary design restriction for the Alpha processors. Everything that could have a negative effect on their multiple-instruction-issue technology has been eliminated from their designs, resulting in an architecture intended to eventually sustain the execution of up to ten instructions per clock cycle.

DEC has made significant inroads toward this goal. The Alpha AXP 21164 microprocessor, available in speeds of 266MHz and 300MHz, is the first processor able to execute over 1 billion instructions per second (bips). It is listed in the Guinness Book of World Records as the world's fastest microprocessor. Manufactured using DEC's own fourth-generation CMOS technology, the Alpha AXP processors, like the MIPS, are also of 64-bit RISC architecture, and are designed for the pipelining of instructions, in which execution of subsequent instructions is begun before the first is completed.

As with MIPS, DEC's processor platform is supported by Windows NT, as well as DEC's own OSF/1 and OpenVMS operating systems. Prior to the release of Windows NT, DEC's networking solutions were all highly proprietary and not at all designed for compatibility with other systems. They have recently expanded their presence in the conventional LAN world, though, by introducing lines of PCs based on both Intel and Alpha AXP processors. All processor manufacturers have vast amounts of benchmark data readily available to prove that their processors are the fastest, and DEC is no exception. I have deliberately avoided using this information because benchmarks rarely have any significance in the real world of computing, but I think it is safe to say that DEC has the fastest processors on the market. This performance comes at a heavy cost, though, as their top-of-the-line processors are priced well above the $2,000 mark, while the fastest Pentium is available for under $1,000. For a heavy-duty application server managing multiple databases or other processor intensive tasks, however, a single Alpha AXP processor might be able to replace a significantly more complex, and more costly, multiprocessor system.

PowerPC. Unlike the other processors discussed in this section, the PowerPC is not the product of a single company. It is a microprocessor designed in accordance with a standard developed jointly by IBM, Motorola, and Apple Computer. Although based on the POWER processor architecture used in IBM RS/6000 workstations, the standard calls for a common instruction set architecture that allows any manufacturer to design and build chips running the same code.

PowerPC microprocessors currently are available only in 32-bit implementations, but the standard is scaleable up to a full 64-bit data path. Motorola's chips range in clock speed from 50MHz to 100MHz, with all but the lowest-end processors being manufactured using a 0.5 micron CMOS technology and running separate instruction and data caches up to 16K. A prototype processor called the MPC620, manufactured by the same process but not yet in production, has 64-bit data and bus widths and runs at a 133MHz clock speed. Incorporating seven million transistors and including 32K data and instruction caches, this is the first in what is intended to be a line of high-performance processors that will rival the Alpha and MIPS chips for the workstation and application server market.

Many hardware and software companies have expressed interest in the PowerPC, due in part to the fact that an open standard will eliminate the possibility of a single chip manufacturer dominating the industry with a technological monopoly, as Intel currently does with the x86 platform. PowerPC processors are currently being manufactured by both IBM and Motorola for use in IBM RS/6000 servers and workstations and in Apple Power Macintosh computers. Power Macs use a software emulation routine that allow them to run the standard Apple System 7.x operating system, and the RS/6000s run IBM's AIX flavor of UNIX without alteration, as the processor is completely binary- compatible with the POWER-based RS/6000s.

The RS/6000 systems are also PReP compliant. The PowerPC Reference Platform (PReP) is a standard created by IBM that allows systems built by different manufacturers to be compatible with each other. All PReP-compliant machines should be able to run AIX, as well as the PowerPC versions of OS/2, Solaris, and Windows NT. Another open platform standard, called the Common Hardware Reference Platform (CHRP), has been developed by Apple, IBM, and Motorola and is designed to be compatible with both PReP and Power Mac machines. Thus, CHRP systems (which are not yet available) should be able to run the Mac OS, AIX, and the PowerPC versions of OS/2 and Windows NT. A version of NetWare for the PowerPC is in development, and plans are also in the works for the use of PowerPC processors in future generations of video game systems as well as automobiles.

The PowerPC microprocessors themselves have been available for some time, and desktop systems based on the PowerPC are available from a handful of manufacturers, including IBM. However, attempts to gain a foothold into the Intel market have been hampered by the platform's failure to deliver a persuasive reason to switch. Windows NT for the PowerPC is available, but existing NT applications must be recompiled to run on the processor. In addition, the performance improvements that were touted as being provided by the new processor turn out to hover somewhere around 15% over an Intel processor of the same clock speed, as opposed to the 30% originally estimated. And finally, the price advantage that was also supposed to facilitate the acceptance of the platform is virtually non-existent. Entry level IBM PowerPC systems are priced at $3,700 and up, hardly an incentive to experiment on a new processor with limited OS and application support.

Up to this point, the PowerPC platform has been blessed with a surfeit of good intentions and exciting speculation, but outside of the Power Macintosh, there is as yet no good reason to believe that they will have a major impact on the computing industry anytime soon.

While its potential has only been realized in a limited fashion so far, the PowerPC has garnered tremendous interest from both hardware and software manufacturers. Open systems are the way of the future in the networking industry, fostering increased competition and the technological advances that always accompany such competition.

### Multiple Processors

After examining the way in which manufacturers have incorporated such remarkable technology into today's microprocessors, the prospects seem almost unlimited. The fact is, however, that software designers keep adding additional power and capabilities to their programs, and it is up to hardware designers to continue making machines that can run them faster and better. The next step in increasing the speed and capability of client/server computing in the LAN environment, as previously demonstrated in the mainframe world, is to use multiple processors in a single machine, accessing a single memory array to share an application's processing tasks evenly among them. This is known as symmetric multiprocessing (SMP), and hardware features facilitating SMP have been designed into all the contemporary microprocessor designs discussed in this section.

The use of multiple processors requires significant alterations in both the hardware of the computer and the software running on it. Among LANs, it is primarily on various flavors of UNIX that multiprocessing has been realized effectively. High-performance workstations used for graphics, CAD, and financial work, as well as large database and OLTP servers, have long made use of multiple microprocessors manufactured by IBM, DEC, and MIPS. It was only with the multiprocessor support built into Windows NT, however, that the possibilities of multiprocessor systems were made available to the primarily Intel-based commercial desktop LAN environment. In addition to this, Novell has recently released an SMP add-on package for NetWare 4.1 that will further assert the viability of this technology in the marketplace.

The existing multiprocessor LAN technologies are mostly the products of individual UNIX development companies, utilizing proprietary hardware with operating systems written specifically for that hardware. Systems range from dual processor UNIX graphics workstations to the Massively Parallel Processing (MPP) supercomputers recently marketed by Cray Research, which can utilize up to 2,048 Alpha AXP multiprocessors (this top-of-the-line model can be had for a mere $31 million--call Cray at (612)452-6650 for more information). With the introduction of multiprocessing on the Intel platform by Windows NT, however, and the subsequent NetWare release, it has been quickly realized that there will be a commercial market for this technology in the near future. Intel has therefore set about creating a hardware standard for Intel-based multiprocessor systems that will make it financially feasible for the developers of operating systems (the traditional laggards in the realization of new technologies) to modify their environments to accommodate the special needs of multiprocessing systems. Without a standard such as this, it would be necessary to customize operating systems and applications to specific hardware designs, all but relegating this technology to a niche market that would never gain widespread acceptance.

The Intel multiprocessing hardware standard is based on SMP. An SMP system is one in which all the processors are functionally identical. Each processor is capable of communicating with every other processor and with shared memory and I/O systems. Note that some multiprocessing systems are not symmetric. These asymmetric multiprocessing systems usually function by assigning specific tasks to individual processors. For example, the three-stage multiprocessor development effort undertaken by Novell for its NetWare product eventually will allow individual domains created by the NetWare 4.x NOS to be assigned to specific processors, thus allowing for an added measure of fault tolerance. The Intel standard, however (which may or may not be utilized by Novell), is designed to allow an operating system to distribute tasks dynamically among processors for increased speed. This is not to say, however, that two processors deliver twice the speed of a uniprocessor system. In fact, this is almost never the case, and if the jury remains out on this technology, it is because overall speed in some systems increases at a diminishing rate with the addition of each extra processor.

The goal of the Intel standard is to allow SMP systems to run all existing shrink-wrapped uniprocessor software as well as multiprocessor-enabled applications with a minimal amount of modification to the hardware of the system. The essential difference in hardware will be the inclusion of an advanced programmable interrupt controller (APIC) that allows communications between individual processors as well as between processors and the I/O system over a separate bus called the interrupt controller communications (ICC) bus. It is through this conduit that interrupts are delivered from source to destination anywhere in the SMP system. In this way, no additional interrupt traffic is created on the system's memory bus, and the processors have a dedicated communications conduit to facilitate the sharing of tasks among themselves.

Although it functions as a single unit, the APIC uses a distributed architecture and can actually be divided among several chips. The local APIC is the module associated with the microprocessor itself. All Pentiums of the second generation or later (that is, 75MHz or above) have a local APIC integrated into the chip; other processors require the use of the Intel 82489DX interrupt controller, which contains the local APIC as well as the I/O APIC. The I/O APIC is available also as part of an I/O chipset, such as Intel's 82430 PCI-EISA bridge chipset. Each local and I/O APIC has a unit ID register that functions as the physical name of the unit for purposes of internal communications over the ICC bus, as well as for identification of specific I/O and interprocessor interrupts by software.

Intel has also incorporated a cache consistency protocol, named MESI, into the second-generation Pentium's internal data cache design. This protocol assigns one of four states to each line in the data cache, which allow determinations to be made regarding the read/write status of each line. This is done to ensure that data in the cache remains consistent with data in memory, despite access to the same memory addresses by multiple processors.

Apart from the APIC, the primary specific hardware modifications that have to be made are in the system BIOS. It is necessary to have a BIOS that can identify and initialize all the microprocessors and other MP-related hardware in the system. Specifications other than these are more general and fall easily under the realm of common sense. Obviously, a system with more processing power creates a larger burden on other subsystems, particularly on memory. Intel recommends the use of a level 2 cache with high- performance features such as write-back technology and error correction but does not include any specifications other than that it be completely software-transparent. The Intel specification also does not recommend a specific system bus type, but the company clearly has plans to make PCI the bus of choice for these implementations, either in place of or in addition to one of the other AT standard buses.

Other standards are in development that rival Intel's standard, including one by their traditional competitor Cyrix. It is too early to tell how popular the multiple processor concept will become in the corporate network, but it is a technology that certainly merits watching. Digital's 25-year plan that predicts microprocessors becoming 1,000 times faster than those available today, specifies higher instruction-per-clock-cycle ratios as the source of the first tenfold speed increase. The second tenfold speed increase is allocated to the use of multiple processors. As our technologies approach the ultimate physical limits of their component parts, it will be replication that provides the next step in efficiency. Whether this concept is marketable on a large scale remains to be seen, but multiple processor systems are one of the most viable methods of pushing the performance envelope that forms the core of networking technology today.

Of course, for any appreciable gain in speed to be realized, application software must also be altered to accommodate SMP. NetWare 4.1 SMP, which is an add-on to the standard NetWare 4.1 product, allows for backwards compatibility with all existing NetWare NLMs but provides only a marginal performance improvement in their operation, as well as in the standard OS file and print services. For significant gains to be realized, server-based applications will have to be modified to utilize the functionality provided by the new OS. NetWare has also released an API to facilitate this process, but the extent to which application developers will commit their efforts to the new environment remains to be seen.

At this time, I would say that if you are already using a Windows NT or NetWare application (such as a database engine) that has become available in an SMP-enabled version, and you are having problems achieving the level of performance that you require, an SMP server may be worth looking into. Otherwise, until the technology is ratified by a greater level of acceptance and commitment in the Intel community, it remains experimental, and therefore risky.

## Memory

During our examination of the microprocessors used in today's network file servers, we have seen the current state of an evolutionary developmental process that began with the first PC and has progressed rapidly ever since. This section is concerned with memory, one of the other primary components of a file server, and one that has always lagged behind the rest of the computer in technological development. Intel microprocessors have advanced in speed from 4.77MHz to 133MHz, with more speed to come, and potential I/O bus throughput has increased equally dramatically, but the standard dynamic random access memory (DRAM) chips that populate the average PC have all but topped out, technologically, at a maximum refresh rate of 54 nanoseconds (ns=one billionth of a second), which works out to an approximate peak transfer speed of 18.5MHz on a bus that can run at speeds of up to 66MHz.

Additional speed can be added, though, through a layer of static RAM (SRAM) cache between the main memory banks and the microprocessor. SRAM is available with refresh rate speeds as fast as 10ns, more than enough for today's high speed buses, but it is still just a cache of 512K at most. While a system that uses SRAM as its primary memory array is possible, SRAM chips are not only much faster, but unfortunately are also much larger, run much hotter, and are ten times more expensive than normal DRAM.

Fortunately, there may be a solution in sight for this memory bottleneck. There are several new RAM technologies entering the market that promise a performance rate similar to that of SRAM, but at only a fraction of the size, heat, and cost. While it is too early to say which, if any, of these new memory types will revolutionize the industry, be assured that the players who have the most interest in the new microprocessor and I/O bus technologies (in particular, Intel) are very interested in supporting these new memory types as they become available. In this section, we examine the types of memory used in today's file servers, cover the essential aspects of memory upgrades, and then take a look at the new technologies that hopefully will bring RAM technology up to speed with the rest of the motherboard.

### The Primary Memory Array (DRAM)

The basic hardware component of the dynamic RAM chip has changed little since the first PCs hit the market. They are essentially composed of capacitors that hold a charge, and transistors that read the state of the capacitors' charges and report them in binary format (1=charge, 0=no charge). The chips themselves have gotten faster and contain more memory in a single package thanks to the use of semiconductor equivalents on an integrated circuit instead of actual capacitors, but the inherent problems of the design remain unchanged.

Capacitors have a natural tendency to leak off their charge over time, causing a 1 to become a 0, and data corruption occurs. To prevent this, a transistor must be dedicated to each capacitor for the purpose of checking the state of the charge and refreshing it before it can decay. This refresh overhead is what imposes a speed limitation on this technology.

What has changed over the years is the hardware used to mount the chips in a computer system. In the earliest PCs, individual DRAM chips of far smaller capacity than today's models were mounted by hand onto a board. If you wanted to add more memory to your system, you bought more chips and popped them into any free sockets available on the memory board. This became increasingly impractical, due to both the increase in the average system's memory requirements and the larger amount of space that was required to make the chips accessible to human fingers or tools.

PCs today all use memory chips that have been premounted on small circuit boards called SIMMs, or single inline memory modules (see fig. 5.3). SIMMs make it much easier to handle large amounts of memory because several chips can be mounted on one board, and since they are permanently soldered in place, they can be placed much closer together. The primary drawback to this concept is that if one chip on a SIMM goes bad, the entire unit must be replaced (unless you are very good with a soldering iron). Older PCs may use SIPPs, or single inline pin package modules, which substitute pins for the SIMM's edge connectors, or even the individual chips previously mentioned, called dual inline pin chips (DIPs). Since we are concerned only with file servers in this chapter, we continue to assume that a minimum of an 80486-based PC is being used.

Adding and Replacing Memory. SIMMs are mounted roughly perpendicular to a PC's motherboard in slots specifically designed to hold them. The process of adding or removing SIMMs is quite easy, but knowing which ones to buy as well as how many and where to put them is highly dependent on the rest of the system and can be rather confusing.

The main concerns when purchasing SIMMs are

- The total amount of memory on each SIMM
- The speed of the memory chips
- The use of a parity bit
- The number of connector pins on the SIMM board itself
- The number of chips on the SIMM

Figure 5.3
This is a typical SIMM.

If, as recommended earlier, you have obtained the documentation for your motherboard, the upgrade process should be easy. The manual should tell you what type of SIMMs to buy, and how you can safely array them in the slots provided. If you don't have the documentation, the best thing to do is to examine the SIMMs already in the machine and duplicate them as closely as possible.

Memory is arrayed on the motherboard in banks that correspond to the width of the microprocessor's data bus. A single bank may consist of up to four individual SIMM slots. Thus, a motherboard designed for a 486 is likely to have 36-bit banks (with a parity-checking mechanism accounting for the extra 4 bits above 32). A Pentium motherboard therefore has 72-bit banks (64 bits plus 8 for parity). Since SIMMs are available either as 30-pin (9-bit) or 72-pin (36-bit) modules, the 486 motherboard has either four 30-pin slots per bank (9 bits times 4 slots for a 36-bit bank) or one 72-pin slot. Most newer 486 systems use 72-pin slots and virtually all Pentium motherboards use two 72-pin slots per bank.

It is important to identify how many slots per bank your system has because all the SIMMs in any one bank must be identical. Depending on the configuration of the motherboard, this in itself can impose severe limitations on the amount of memory that you may add. If, for example, you have a 486 PC with two banks of four 30-bit slots each, you are limited to a relatively small number of memory configurations because SIMMs are only available in 1M, 4M, 16M, and 64M sizes. Since each bank must consist of four identical SIMMs, you must have either 4M, 16M, 64M, or 256M of memory in any one bank. If your system has more than one bank, then you can safely put 1M SIMMs in one bank and 4M SIMMs in the other or even leave one bank completely empty. This leaves you a limited growth path. In addition to this, you must take the possibility of future upgrades into account. If all your slots are full, and you wish to add more memory, then some of the old SIMMs must be replaced with higher capacity models. The SIMMs that are removed can be used in another machine or else go to waste. This is one of several good reasons to maintain a measure of uniformity in your PC purchases. Virtually every network has servers or workstations that could use a few more megabytes of memory; as long as other machines use the same type, SIMMs can easily be transferred.

For reasons of flexibility, then, it is preferable that you purchase a motherboard that uses the 72-pin SIMMs. In a 486 machine, this means one slot per bank, and many more possible ways to configure the server's memory to your needs, rather than the other way around.

Identifying SIMMs. SIMMs generally are looked at in one of two ways, both of which can be confusing: on paper or in your hand. That is, looking at one of the SIMMs currently installed in your PC and at the listings of available SIMM types in catalogs or advertisements and correlating the two to figure out what you already have and what you need can be difficult.

Catalogs usually do not provide illustrations of individual SIMM types since they're not much to look at, and you probably wouldn't be able to read the printing on the chips in a photograph, anyway. This section provides the means for identifying the specifications of the memory in a PC in order to purchase additional chips that are compatible. When you lack a definitive source for documentation of your motherboard, however, it is impossible to be absolutely sure that a SIMM will be compatible, unless you are able to purchase exact duplicates of those already in the machine--and this is unlikely. Therefore, be sure you know your selected chip vendor's return policies in case you have problems. Most memory vendors allow SIMMs to be returned, primarily because inserting them into an incompatible motherboard rarely results in damage. They simply do not work. Also, there is little or no packaging overhead in the sale of SIMMs. Usually, they arrive loose, in a small anti-static bag, so there are no concerns about shrink-wrapping or resale of returned modules.

Removing SIMMs from the Motherboard. It can be difficult to identify SIMMs while they are mounted on the motherboard, so the best way to know what type of SIMMs are in the machine is to remove one of them. In many cases, SIMM sockets hold the SIMMs at an angle slightly less than perpendicular to the motherboard. This is because when they are installed, they are inserted straight in and then tilted a few degrees to engage two small clips that secure the SIMM at either end (see fig. 5.4). Some SIMMs, however, are inserted at the angle and drawn perpendicular to lock in place. The locking clips are made of metal or plastic. Be particularly careful with plastic clips--if you break one, you have to choose from unattractive alternatives: replacing the motherboard (very expensive and a lot of work), or holding the SIMM in place with a rubber band or a piece of tape (unreliable at best).

Before touching the motherboard or SIMMs at all, unplug the PC from its power source, and then ground yourself to prevent a static discharge while you're working. Many components of a PC are sensitive to static, but none are more sensitive than DRAM chips, so it is advisable to wear a grounding strap or work in an environment not prone to static buildup. Avoid carpeting and wool sweaters, and be sure to touch metal before actually handling SIMMs. Also, be sure to store any loose SIMMs in an anti-static bag, even if they will be needed in just a few minutes.

Figure 5.4
Closeup view of a SIMM installed on a motherboard and the clips holding it in place.

Since SIMMs usually have to be installed in order, with each one overlapping the previous one, only the first one is immediately accessible. To remove it, release the clips at either side of the first module so that you can freely move the entire chip into an upright position, perpendicular to the motherboard. Once it is free of the clips, you should be able to lift the entire SIMM out of the socket and remove it from the system.

Number of Pins. When you look closely at a SIMM, you will see that it looks like a very small expansion card. A row of 30 or 72 gold connectors runs across the bottom of a printed circuit board that has a number of chips soldered onto one or both sides. There might be a small "30" or "72" printed on the circuit board itself, to identify the number of pins. The number of pins obviously is an indication of the type of sockets that are installed on your motherboard, but also indicate the width of the chips. 72-pin SIMMs are 32 bits wide (or 36 bits with parity checking), while 30-pin SIMMs are 8 bits wide (or 9 with parity checking).

Chip Speed. Looking at the memory chips that are mounted on the SIMM you have removed, you might see the manufacturer's name, identified by word or symbol, and a part number. A relatively small number of manufacturers make memory chips. Many more companies, however, buy the chips and mount them on SIMMs. If you see a manufacturer's name on the SIMM board, do not confuse it with the maker of the chips.

The last digit in a chip's part number usually is a 6, 7, or 8, and often has a dash in front of it. This, with a zero appended to it, indicates the speed of the chip. This is measured in nanoseconds and is a measurement of the refresh rate of the chips; the smaller the number, the faster the refresh rate, and the faster the memory performance. 60ns DRAM chips are the fastest that are commercially available. It usually is not a problem to install faster memory into a PC than the motherboard specifications require. It is not recommended, though, that different speeds be used in the same bank, and you should never use slower memory than a system specifies.

Parity. In their first PCs, IBM introduced a basic error checking concept called the parity bit; it has remained in popular use ever since. The system works by including one extra bit for every 8 bits (1 byte) of memory. Most SIMMs do this by including an extra memory chip on the board. Non-parity SIMMs have two or eight chips installed on them, while parity SIMMs have three or nine. As data is written to each byte of memory, this extra bit is set to a 0 or 1 by a dedicated circuit on the motherboard, depending on the state of the other eight bits. If the total value of the other eight bits is an even number, then the parity bit is turned on to make the number odd. If the eight bits add up to an odd number, then the parity bit is left off, keeping the number odd. As each byte is read from memory, the same mechanism checks to see if the total value of the byte is an odd number. If it is not, then the data on that chip is considered corrupted. Most systems generate a non-maskable interrupt (NMI) when a parity error is detected and immediately halt the system to prevent the tainted data from propagating itself into any other media. An error message usually is displayed, supplying the location of the error in memory and giving the option of continuing to work or rebooting the system. It should be emphasized that a single parity error does not necessarily indicate the failure of a memory chip. Environmental conditions can cause isolated errors that seldom or never reoccur. Repeated parity errors, however, usually indicate that a SIMM needs to be replaced.

Obviously, the parity bit is a very limited mechanism for error checking. While it should not be able to cause a false positive result, it obviously could fail 50% of the time to detect cases of data corruption in more than one bit (when the data is corrupted but the total remains odd). It is, nevertheless, an added measure of security that creates little overhead for the system. SIMMs containing the parity bit, however, cost more than those that do not, and some vendors have been known to disable parity checking on a motherboard to save a few dollars by populating the machine with cheaper SIMMs. If you wish to use parity checking, make sure that the system's motherboard is capable of supporting it (usually through DIP switch or jumper settings) and that all SIMMs installed in the system have a parity bit.

> NOTE: Bear in mind that the relatively infrequent occurrences of corruption errors in today's DRAM chips makes parity checking something of a moot point in most new purchasing decisions. Do not pass up an otherwise well-equipped system merely because it lacks memory parity checking--many vendors and motherboard manufacturers are beginning to leave this feature out of their newest products.

Error Checking and Correcting Memory. While parity checking has been available for a long time, a newer technique takes the same concept further by being able to actually correct memory errors as well as check for them. On servers in which system halts due to non-maskable interrupts are unacceptable or intolerable, error checking and correcting (ECC) memory systems utilize a checksum technique to effectively detect errors when they occur in any two bits of a data word and then correct one of the two.

To perform these functions, a coprocessor or application-specific integrated circuit (ASIC) is used to compute a checksum on a data block of designated size. The checksum is appended to the block and sent to its destination. There, the checksum is computed again and compared with the original results. The nature of the algorithm is such that, should an error in a single bit be detected, the exact location of the bit can be derived from the checksum and corrected. Remember that since we are dealing with binary data at this point, simply locating the exact bit in which the error occurs is all that is needed to correct it. Errors in two bits generate an NMI and halt the system. Compared to parity checking, this technique requires much more overhead in both data storage and processing cycles. The amount of overhead is dependent on the size of the block used and the location of the ECC mechanism in the system. ECC features have been incorporated into devices ranging from tape backup drives to processor caches and everywhere in-between.

For example, in a 32-bit 486-based system, ECC increases the data path from 32 to 39 bits wide, as compared to parity, which needs a 36-bit width. This requires a significant change in the memory architecture that might not be worth the effort and expense. In 64-bit Pentium systems, however, as well as in other systems using 64-bit processors, the data overhead is only one bit, the same as for parity checking. For little more than the price of a processor, occasional memory errors that previously would have halted the system can be corrected on the fly.

There is a certain amount of disagreement in the industry regarding the real benefits of ECC memory, beyond peace of mind. System outages due to transient memory corruption are far less common than hard drive or power regulation problems, and many people feel that they occur so seldom as to be a negligible risk. If you want to have every measure of protection that you can get for your server's data, however, ECC is a giant step beyond parity checking.

Purchasing Memory. When shopping for memory, you will see SIMMs listed in ads and catalogs using an XxY format--for example, 1x9--along with a speed listing. The first numeral identifies the amount of memory on the SIMM, and the second indicates the width of the chips, the presence of parity, and the number of pins on the SIMM. In the example above, the 1 indicates that it is a 1M SIMM, and the 9 indicates that it uses 8-bit-wide chips and therefore has a 30-pin connector, plus one bit for parity.

Since 72-pin SIMMs have four times the density of their 30-pin counterparts, the second numeral is always a 32 or a 36 (with parity). For the same reason, the first numeral is one-fourth of what normally would be shown for an 8-bit-wide SIMM. Thus, a 1x36 SIMM consists of 4M of parity memory on a 72-pin board. The same SIMM without parity would be a 1x32. If the width aspect confuses you, it is always possible to tell by the price whether you are getting 4M or just 1M.

You might also see memory described using three numerals, such as 1x9x3. This indicates that the manufacturer has utilized higher-width DRAM chips to make a lower-width SIMM. Though statistically identical to a 1x9 SIMM, this configuration has only three chips mounted on the board instead of nine; two 4 bit chips plus one 1 bit chip for parity. This same configuration has also been expressed as 1x3 by some vendors. While these are functionally equivalent to normal 1x9 SIMMs, some motherboards are intolerant of this configuration. Do not attempt to use such SIMMs if your motherboard manufacturer recommends against it.

Installing Memory. Installing new SIMMs into the motherboard is simply the opposite of the removal procedure outlined above. Follow the same precautions regarding static, and note also the way that the SIMMs should lean when fully installed. Very often, you have to insert SIMMs in order, or else previously installed SIMMs block access to other sockets. Insert a SIMM into its slot in the proper manner, either perpendicular or at an angle to the motherboard. One side of the module has a notched corner to prevent you from inserting it the wrong way. Once the SIMM is seated firmly in the slot, tilt it gently until the clips fasten themselves around the board (see fig. 5.5). You might have to open the clips by hand to accomplish this, rather than force the SIMM into place. Unlike some expansion boards and processor chips, a great amount of force is not required here, so remain gentle.

Figure 5.5
This is a correctly installed SIMM.

Once all your new memory is installed, double-check that all the banks being used are completely filled with identical SIMMs, connect the power cord, and boot the machine. If the memory check display of the BIOS (basic input/output system) is enabled, you should immediately see the results of your efforts as the system counts up to the total amount of memory installed in the machine. If the total does not reflect the additional memory you have installed, reflects only part of it, or the machine fails to boot at all, then one or more of the SIMMs is defective or is not fully seated in its slot, or you have not properly filled a memory bank.

If, however, the memory check displays the proper amount, yet you receive an error message from the BIOS indicating a "memory error" or "memory size mismatch," then everything is all right. What has happened is that the computer's BIOS has detected a different amount of memory in the system than is specified in its configuration ROM. Simply run the BIOS setup program (usually by holding down a certain key combination during the boot process) and save the BIOS configuration as though you had altered some of its settings. After you reboot the machine again, you should have no further problems.

Adding additional memory is one of the simplest and most effective upgrades that can be performed on a server. NetWare, as well as other NOSs, always runs better when it has enough memory to utilize because many of its processes are flexible enough to take advantage of additional memory as soon as it is supplied. For example, with more memory, greater amounts of file system information can be cached, resulting in faster file access. Therefore, while NOSs often have precise formulas to determine the amount of memory required in a server that depend on the amount of disk space in the machine and other variables, it is always recommended that some extra DRAM be added. Even as much as 25% or 50% more will not go to waste. It is not expensive, and it often makes a noticeable difference in performance.

### RAM Caches (SRAM)

The word "cache" is one that is much overused in the computer industry today. Modern computers can have several different caches operating at the same time in different places. Simply put, a cache is an area of memory that is used to buffer input or output between two resources of different speeds. Your system might be running a software-based cache, like the MS-DOS Smartdrive program, that utilizes part of your computer's memory to buffer I/O from the hard drive. Hard drives or I/O controllers might have memory chips integrated into their construction to provide for onboard caching. Multiprocessors like the Intel 80486 and Pentium chips have small internal caches where data moving to or from memory can be temporarily stored. This section, though, concerns separate RAM caches that function much the same as those within the processors previously listed.

As stated earlier, the DRAM technology that is used for the primary memory arrays in modern PCs is far slower than today's processors and I/O buses. This means that the faster components often sit idle as they wait for data to be transferred to or from memory. Caching is an attempt to minimize these memory latency periods by storing a small subset of the data residing in DRAM memory in a separate, smaller, but much faster memory array. Thus, a RAM cache is a completely separate memory bank with its own controller that is located between the microprocessor and the primary DRAM array. The memory chips comprising this cache are called static random access memory (SRAM). While DRAM has a refresh rate of no less than 54ns, SRAM technology can provide refresh rates of as little as 10ns.

How Caches Work. When a call is made by the microprocessor for data that is present in memory, the microprocessor first checks its own internal cache arrangement (designated the level 1 cache or primary cache); if the data is not found, the request leaves the processor chip on its way to the memory array. When an SRAM cache (called a level 2 cache or secondary cache) is present, however, the data request is intercepted by the cache's controller chip. This controller's function is to maintain a continuously updated listing of the data that is currently in the cache, indexed by the data's memory address in the primary array. If the data is not found in the cache, then a cache miss is declared, and the request is passed on to DRAM, where it is processed in the usual manner. If the data is found in the cache, however, this is called a cache hit, and the data is immediately sent to the processor at its much higher speed, eliminating the delay caused by access to the slower DRAM.

While DRAM chips use equal numbers of capacitors (to hold a charge) and transistors (to read and relay the status of the charge for each capacitor), SRAM chips utilize a pair of transistors for each memory bit. There is no leak-off of the charge in a transistor as there is in a capacitor, so there is no refresh overhead. This is why such dramatically faster speeds are possible. As a tradeoff to the greater speeds, however, SRAM chips are larger than DRAM chips, they run hotter, consume more power, and are far more expensive to manufacture. This is why SRAM is used only in small amounts, for caching, rather than as a primary memory resource.

> NOTE: Because of the extra heat and power consumption, SRAM caches are rarely found in laptops or portables.

Cache Designs. Different cache controller designs utilize different methods of data storage and indexing within the cache, providing various levels of performance. A direct mapped cache splits the main memory array into virtual blocks the same size as the cache itself, and then splits each block into lines. Data from any particular line in any DRAM block can be stored only in its corresponding line in the cache. This method allows for a very small index, but severely limits the possible locations in which a datum from DRAM can be stored in the cache, resulting in many more cache misses.

A full associative cache can store data from any location in the primary memory array into any location on the cache. This causes an improved cache hit ratio, but also creates a much larger index that has to be checked by the cache controller for every request, increasing overhead and decreasing overall performance levels. A set associative cache is something of a compromise between the previous two methods. This design calls for the splitting of the cache into a number of discrete direct mapped areas. A two-way set associative cache is thus split into two areas, a four-way into four, and so on. In this way, there is greater flexibility regarding where a particular piece of data can be stored, without creating an index as large as that of the full associative cache. Difficulties imple-menting this design have made it more expensive to manufacture than the other two methods, but this nevertheless is the most popular design today.

Another factor to take into account in cache design is the two-directional nature of memory access. The microprocessor is continually sending data to the main memory array and reading data from it. Wait states can be caused by memory latency in either direction, and caches can operate on data being read only from memory or bidirectionally. A write-through cache is one that operates only in a single direction. Data being written to memory by the processor is saved in the cache and immediately passed to main memory. If main memory signals its inability to write at that exact moment, a wait state occurs until main memory signals its readiness. A write-back cache operates in both directions. Data written to the cache is held there until main memory signals its readiness to receive it. This allows other operations to continue in the interim, increasing overall performance. As you can imagine, a write-back cache design essentially doubles the complexity of the caching operation, adding additional overhead to the controller and additional expense to the unit. For maximum performance, though, most PC manufacturers have accepted that a four-way set-associative write-back cache is the best possible design.

Static RAM caches range in size from 64K to 512K. They sometimes are integrated into motherboard designs, but more often are packaged as small daughterboards that plug into a socket on the motherboard. This allows for varying amounts of cache to be provided in a server. The general rule of thumb in caching is "more is better." While some other types of caches might run into the law of diminishing returns, in which the addition of more memory beyond a certain point leads to a disproportionate return in performance, SRAM caches utilize small enough amounts of memory for them not to be affected by this. The only drawback is system design and cost. As with most things, it is not wise to use more cache than your motherboard manufacturer indicates as a maximum. It is important also to check the width of the bus between the microprocessor and the RAM cache. Some motherboards include a dedicated 128-bit bus for this purpose, which can provide an extra measure of added performance.

The SRAM cache is the most common technique in use today for overcoming the speed limitations of DRAM memory arrays. A well-designed cache can provide a tremendous increase in a file server's performance. Care should be taken, however, to utilize only the cache design recommended by the manufacturer of the server's motherboard. There are far fewer manufacturing "standards" in place for SRAM cache design than there are for most other PC components, so the odds of buying the SRAM cache unit that you want and having it work in a particular motherboard are slim. When not purchased with the motherboard itself, cache upgrades are best purchased directly from the motherboard vendor or manufacturer. When purchasing a new file server or motherboard, however, a measure of care taken in selecting a model that provides a fast, efficient SRAM cache can provide a great deal of extra performance for the cost.

### Memory Interleaving

Another technique for speeding up memory access times is interleaving. This is a technique in which two or more memory banks of equal capacity are used in parallel. Data bits are written alternately to the two banks, so that sequential memory reads alternate their access between the two.

While one bank is being accessed, the other is being refreshed, effectively reducing wait states. When non-sequential accesses are required, the overall efficiency diminishes, as the laws of probability determine whether or not the next bit required is located on another bank or not. This can be mitigated, however, by the use of more banks, and there is no performance penalty (beyond traditional DRAM access times) for sequential reads from the same bank. Requiring no alterations to the DRAM chips, this is probably the most inexpensive means of reducing memory access delays and can be used in conjunction with a cache for additional performance gains. This technique, however, is a property of the motherboard, not the DRAM chips themselves. Check the motherboard documentation to find out if memory interleaving is possible on your server.

### New Memory Technologies

While SRAM caching provides a measure of relief for the slow performance of standard DRAM chips, it is no more than a stopgap measure. Processor and bus speeds continue to rise, and the added cost of additional SRAM chips and more powerful caching controllers will become increasingly unmanageable.

An alternative is clearly required that will provide performance near that of SRAM for the entire main memory array at a manageable cost, preferably without a major change in motherboard architecture. This sounds like a tall order, but there are several new technologies in the works that may provide just that.

As is usually the case in the computer industry, however, one problem generates several possible solutions, and the requisite period of corporate bickering over standards will no doubt ensue before any technology emerges as the dominant solution.

Nevertheless, steps are being taken in the right direction, and the first systems utilizing some of these technologies are already on the market. Never a trailblazer as far as my file servers are concerned, I do not recommend the use of any of these technologies until they have been adequately tested under real world conditions, but I certainly will be keeping a watchful eye on their progress.

EDO DRAM. Enhanced data output (EDO) DRAM is the first of these new memory technologies to be available on the open market today. Several giant semiconductor corporations, including Samsung, Mitsubishi, Hyundai, and particularly Micron Technologies, have announced that they will manufacture the chips, and systems and chipset manufacturers are also beginning to endorse this new technology, which is said to deliver zero wait states at bus speeds of up to 66MHz. Pentium systems using EDO DRAM are already available from Micron Computer and Toshiba. These systems contain motherboards that utilize the new Intel Triton core-logic chipset, the only chipset on the market at this time to support EDO DRAM (and synchronous DRAM as well).

EDO DRAM is based on traditional fast page mode DRAM, is pin-compatible to the old RAM chips, and does not require any substantial alterations to the system board architecture. A small modification to the DRAM chips themselves allows for data to be passed through them at increased rates, and a subsequent level of this technology, called burst EDO, uses a pipelining technique that allows an operation to begin before the previous one has been completed, adding additional speed.

Synchronous DRAM. Another new technology that could end up rivaling EDO for the first solid niche in the marketplace is synchronous DRAM (SDRAM). Current DRAM technology uses asynchronous row and column addressing, and the SDRAM concept simply alters this arrangement so that inputs and outputs can both be conducted with each clock cycle. Column access is also pipelined, allowing a second request to begin execution before the first has finished. The result is performance equivalent to 10ns SRAM, at prices comparable to traditional DRAM.

The Triton Pentium chipset from Intel contains built-in support for both SDRAM and EDO DRAM. With this level playing field on which to compete, it remains to be seen which technology takes hold in the marketplace.

Cached DRAM. Two other manufacturers have taken separate approaches toward essentially the same end. Mitsubishi's cached DRAM (CDRAM) and Ramtron's enhanced DRAM (EDRAM) each include a small amount of SRAM cache memory along with DRAM on each chip. They differ in that CDRAM uses a set-associative cache design, while EDRAM uses a direct-mapped cache design. The SRAM portions of both chips have a refresh rate of approximately 15ns, and the apparent advantage held by CDRAM due to its superior cache design is effectively nullified by the faster refresh rate of the EDRAM chip (35ns versus 70ns). Both designs are estimated to cost $50 per megabyte.

RDRAM. While all the technologies previously listed have attempted to retain the basic memory architecture of the PC, a company named Rambus has designed a completely revolutionary scheme that uses the system's processor or a custom-designed ASIC to control the transfer of data over a proprietary Rambus channel in packets of up to 256 bytes. Twin 250MHz clocks and the ability to transfer data on both clock edges yields a maximum transfer rate of 500 M/sec, equivalent to a 2ns refresh rate. This fantastic performance is strongly mitigated, however, by the need for major hardware design changes, which motherboard manufacturers will be very reluctant to adopt. This is likely to end up as a proprietary design which, if adopted by system manufacturers with sufficient clout in the industry, could be successful, but unless the innovations described earlier prove to be unusable, the performance improvement of RDRAM over EDO DRAM will not justify the investment of millions of dollars for new motherboard designs and manufacturing techniques.

## I/O Bus Types

It is fairly easy to predict the trials and tribulations that the new memory technologies will go through before a definitive standard is realized. This is because the personal computer industry has seen the same process occur several times, most notably in designs for the I/O bus that were developed as a means of improving the performance standard set by the archaic ISA bus from the original IBM PC designs. It's difficult to believe that so many systems still are being sold today with high-powered processors like the 486 and the Pentium that still rely so heavily on this antiquated architecture.

The fundamental reason that these persistent difficulties plague the PC industry is not technological but economic. The original PC was more of a marketing coup than a technological one. Many other computer designs existed that were as good or better than the PC, and in the same way, many bus designs have come and gone over the years, but they didn't last because they were proprietary systems. At one time, a giant conglomerate like IBM could create a standard just by making a decision, and this was how the ISA bus came about. When IBM said, "This is how we are going to build our machines," many other companies were more than willing to design, manufacture, and market expansion boards that would work in those machines. Unfortunately, this eventually became the problem rather than the solution. As technology moved forward, the old standards became inadequate, and new ones had to be developed that were compatible with the old so that the investments made by other companies would not be lost. This is where IBM found out that they were no longer the sole trendsetter in the PC industry. This led to a battle of competing I/O bus standards that is being waged to this day, and although the players have changed, the game remains the same.

### ISA

The Industry Standard Architecture (ISA) I/O bus was developed for the original IBM PCs. Using early Intel 8088 microprocessors that ran at 4.77MHz, this 8-bit-wide bus was designed to run at the same speed. Indeed, it functioned more as an extension to the processor itself than as a separate subsystem. The processor's clock was the centralized timing source for the whole system, forcing even simple data transfers that required no calculation to be routed through the processor. In essence, this was a true local bus, with the expansion slots hardwired directly to the microprocessor's I/O pins.

It became apparent quickly that this would not do. Within months, processors were being manufactured that ran at unheard of speeds, like 6MHz or even 8MHz. The expansion cards designed by other manufacturers for the original PC bus would not run at these increased speeds, so it became necessary to unlink the bus from its direct connection to the processor clock. In order to relieve the processor of some of its routine tasks, a separate controller was added, called a direct memory access (DMA) controller. The original DMA controller in IBM machines ran at 5MHz and could access only 1M of memory, which was the addressable limit of an 8-bit architecture. These early systems still had a bus speed roughly equivalent to the processor's clock speed, but they were two separate clocks, and the bus was no longer dependent on the processor for timing information.

By the time IBM released the PC-AT in 1984, the company also had chosen to expand the existing bus design to accommodate the new Intel 80286 processors. They added an additional connector to the original 8-bit slot (see fig. 5.6) that allowed for a 16-bit data width while retaining compatibility with earlier designs. Still running at 8MHz despite processor speeds of 10MHz or 12MHz, the AT bus could now address up to 16M of memory, the limit for a 16-bit system, and had a theoretical maximum data transfer rate of 8 M/sec. This figure, of course, represents at least double the actual transfer rate that would be achieved under normal conditions. Factor in the need to accomodate the 32-bit-wide data path of today's microprocessors, and the result is a bus that is generally inadequate for many of today's computing needs. Nevertheless, this is the same ISA bus that is still used today in the vast majority of the world's PCs.

Figure 5.6
This is an adapter card's 16-bit ISA bus connector.

For use in a network file server, the ISA bus should not even be considered, except as a secondary bus type used for non-networked resources such as video adapters. ISA expansion boards still use DMA transfers and still cannot address memory above 16M. Furthermore, they force the processor to spend part of its time operating at the bus's top speed of 8MHz and still use a 16-bit data path, so that the performance capabilities of the high-speed 32- or 64-bit processor are largely going to waste.

As we consider alternatives throughout the rest of this section, bear in mind that any of them is preferable to the ISA bus for use in a file server. I don't mean to condemn it out of hand, for the ISA bus is still a valid choice for workstation use and has led a surprisingly long and useful life in a highly volatile industry. The primary function of a file server, however, is data transfer--when I see someone spend a great deal of money on a server with the latest processor and fastest drives and then run them through a 16-bit SCSI card and wonder why it's no faster than the old machine, I have only one answer.

### MCA

When IBM realized the ultimate inadequacy of even their expanded AT bus, they set about creating a new bus standard that would be able to accommodate the continual improvements being made in other components of the PC. Still the dominant player in the desktop PC market, they proved their skills by developing a bus type using what they called Micro Channel architecture (MCA) for the PS/2 systems that went to market in 1987.

MCA was remarkably visionary in its design, capabilities, and expandability, but IBM proceeded to let the euphoria over this accomplishment go to their heads and made two drastic mistakes that forever prevented the MCA bus from becoming an industry standard. First, they decided that their requirements for the new bus overshadowed the need for backward compatibility with existing expansion cards--ISA cards cannot run in MCA slots. Second, they implemented a licensing scheme that attempted to force other manufacturers to pay retroactive royalty fees on the ISA bus technology to market systems using the MCA bus.

This caused enough of an uproar within the industry that a "gang of nine"--other PC manufacturers, usually cutthroat competitors with each other--got together to develop their own standard, which resulted in the EISA bus. As a result, the MCA bus never enjoyed the popularity that it deserved outside of IBM shops, and far fewer MCA expansion cards were made by third-party manufacturers than EISA and ISA cards. IBM continued to market and maintain MCA as a proprietary standard for nearly a decade, and only recently announced that they would discontinue further development of MCA and gradually phase out its use in favor of PCI (see the "PCI" section below). This is not due to unviability of the standard or low quality of resulting products but solely to antiquated business practices of a former industry leader now relegated to a position farther back in the crowd.

The most obvious improvement of the MCA bus was its 32-bit data path. Designed to accommodate the new 80386 processors that were just being released, the address path was also widened to 32 bits, allowing a full 4G of memory to be addressed. However, 32 bits are not required. The bus reads a signal from an expansion card that informs it of the card's capabilities. Thus, while MCA cards can be 32-bit, they do not have to be. This should be considered whenever purchasing Micro Channel adapters. The Adaptec AHA-1640 Micro Channel SCSI host adapter, for example, is a 16-bit MCA card that suffers from the same 16M memory limitation as an ISA adapter using DMA does.

From an installation standpoint, MCA expansion cards are well ahead ahead of ISA, although they do not approach the Plug and Play standards that we are striving for today. A reference disk containing the configuration program is needed, but gone are the jumpers and DIP switches that plagued the older technology. Just plug an MCA card into the slot, and the hardware aspect of the installation is finished.

The MCA bus runs at 10MHz, not blazingly fast by today's standards, but with a slight edge over ISA and even EISA systems. The primary innovation of the bus that made it incompatible with previous designs, however, was hardware-mediated bus arbitration. In a remarkably foresighted move, IBM created a bus that would be quite capable of supporting today's multiprocessor systems. The MCA bus can arbitrate between eight microprocessors and up to eight other devices, such as DMA controllers or graphics coprocessors directly at the hardware level. This is done through the inclusion of a controller on the bus called the central arbitration point (CAP) that determines which device in the system gets control of the bus at any given time. By utilizing a hierarchy that gives added weight to more critical processes (such as memory refreshes or NMIs, for example), the CAP is able to satisfy the needs of all its "customers" without monopolizing the system's microprocessor. For this to function properly, however, it is necessary for all devices involved to have comparable arbitration circuitry.

The bus also utilizes level-sensitive interrupts, which are incompatible with the edge-triggered interrupts of the ISA bus. Level-sensitive interrupts can be shared, while edge-triggered interrupts cannot, and level-sensitive ones can be sensed during the actual interrupt, while the AT bus interrupts can be sensed only at the moment that the interrupt request changes state.

Also included in the original MCA bus design was a burst mode that was a direct response to the extensive microprocessor overhead that is required for data transfers in the AT bus. Instead of the two step "addressing and mailing" process of the older technology, which required the processor to be involved in the transfer of every byte, MCA is able to transfer larger blocks of data, or bursts, without processor intervention by utilizing its own hardware to detect the status of the transfer.

Two years after the release of the initial PS/2 machines with the Micro Channel bus, IBM enhanced its specification to allow better performance. The original MCA design, running at 10MHz and using 32-bit words, had a maximum theoretical transfer rate of 20 M/sec. Compared to the 8 M/sec of the ISA bus, this is a vast improvement, but the Micro Channel 2 specification goes even further. While the original burst mode design was created to facilitate the transfer of random data bytes that required individual addresses, the enhanced MCA bus incorporates a process called streaming data mode that greatly enhances the transfer of sequential blocks of data.

It was recognized by studying the data access patterns of normal system use that larger, contiguous blocks of data are often transferred. Executables and large data files, for example, might reside in one large block on a drive and have to be read into memory by the methodical access of each subsequent byte. In this situation, it is not necessary to transfer an address location for each byte because the source and the destination are both a series of contiguous addresses. Streaming data mode eliminates the clock cycles devoted to the individual addressing of data bytes. Since a normal transfer utilizes one cycle for addressing and one for the actual transfer, the elimination of the former effectively doubles the maximum transfer rate, to 40 M/sec. However, since the bus design allots 32 bus lines for addressing as well as 32 for the transfer, streaming data mode leaves address lines idle after the original address for the entire block is sent. MCA multiplexes the data transfer, using the 32 idle lines to double the throughput again to a 64-bit transfer, running at 80 M/sec. Designed for use with bus masters and slaves, the best part of this extension to the MCA specification is that it is completely optional. Hardware that cannot support these techniques simply performs data transfers using the original method.

As you can see, when combined with hardware that can support its extended features, the Micro Channel architecture makes quite a robust bus that can more than adequately service the needs of the average server. Since IBM is now combining this technology in machines with the PCI bus--and eventually phasing it out altogether--it would be unwise to purchase a server that relies on MCA at this time. In addition, there never developed as wide a range of third-party hardware that supported MCA as there did for EISA and other bus types. If you already possess servers that use this bus, though, you can be sure that they will perform at least as well as EISA and far better than ISA.

### EISA

As stated earlier, the EISA bus was a direct reaction to the incompatibility and licensing practices of IBM's MCA bus. Therefore, the EISA design holds compatibility with existing ISA adapters as its highest priority. Beyond that, the standard amounts to a collection of most of the desirable attributes of the Micro Channel bus and other proprietary standards. Perhaps due in part to the fact that they had the MCA bus as a model to work from, the committee successfully created a standard that has endured for many years. Although not as fast and capable as some newer emerging standards, such as VLB and PCI, EISA definitely is thoroughly tested, completely reliable, and well-suited for use in the average server. I do not hesitate to recommend purchasing an EISA-based server, as EISA promises to remain a viable standard for some time to come.

One of the cleverest aspects of the EISA bus design is that of the connector itself. Unlike an MCA slot, into which an ISA card could not hope to be inserted, an EISA slot has connectors of two lengths. The upper level of a connector corresponds exactly to the edge connector of a standard ISA adapter. This allows all existing ISA cards to be plugged into an EISA slot just as though it were an AT bus. Adapter cards designed to the EISA standard, however, have shorter connectors that plug into the same ISA-compatible slot, as well as longer connectors that plug into a series of contacts located deeper in the EISA slot. Special cutouts on EISA cards allow them to be fully inserted into the slot at both levels, while ISA cards are blocked from extending down to the EISA connectors and possibly causing a short circuit. Thus, the additional advantages of the EISA bus are made available without utilizing any more motherboard real estate than that taken by an ISA card.

Of course, as with MCA, EISA has taken care of the primary requirement of expanding the bus to use a 32-bit data path. 32 address lines are used, allowing the bus to address the full 4G of memory that 32-bit processors like the 386 and 486 can support. The EISA specification also takes great pains to accommodate the varying power needs of the large array of ISA and EISA adapters that are to be supported. The design calls for 45 watts at four possible voltages per EISA slot. This is far more than most ISA or EISA cards will ever need, which is lucky, because a fully populated EISA bus containing eight adapters, each using the maximum amount of power available, would require a power supply of over 300 watts for the I/O bus alone, discounting the needs of the motherboard, processor, and other devices in the machine. Obviously, this is more than the average PC power supply can furnish, but it is another indication of the designers' forward-looking intentions for the EISA bus.

In the interest of backward compatibility, though, it was necessary to keep the speed of the EISA bus the same as that of the AT bus: 8.33MHz. This speed amounts to one-fourth of 33MHz, the highest processor speed available at the time. This is because, when running in its basic mode, EISA is essentially a synchronous bus, coordinating its speed with that of the system processor. EISA, however, like MCA, has special bus mastering capabilities that greatly enhance the speed of data transfers under certain conditions. Bus mastering is a system in which an adapter contains its own processor for managing data transfers, thus reducing overhead and improving performance. This prevents the adapter from having to add to the burden of the system CPU or rely on the motherboard's DMA controller for processing. The EISA counterpart of Micro Channel's CAP is called the Integrated System Peripheral (ISP) chip.

Aside from managing the transfer of bus control between the various devices in the computer, the ISP chip allows for a compressed transfer mode that uses a special timing signal that runs at double the normal bus speed, as well as a burst mode. Unlike the burst mode of the MCA bus, though, EISA's furnishes an address with each byte, allowing for transfers of non-sequential data in a single burst. Overhead is reduced by transferring only the 10 lowest bits of each address. This limits the contents of a single burst to bytes that reside with a block of 1,024 32-bit double words. Each burst is also limited to either read or write data--the two cannot be mixed. Using burst mode, a theoretical transfer rate of 33 M/sec can be achieved (although, to swipe a disclaimer from the automobile ads, your mileage may vary).

Some EISA adapters utilize bus mastering and others do not. Those that do, usually offer increased performance and, not surprisingly, carry a higher price. Also, not every EISA slot is capable of supporting a bus mastering card. As always, consult your motherboard documentation before making a purchase decision.

EISA also expands considerably on the capabilities of DMA transfers. In addition to supporting the capabilities of the standard DMA transfer from the AT bus, EISA's DMA controller allows access to the full 4G of memory supported by the specification, instead of just 16M. Three additional DMA transfer modes are also added in the EISA specification, reducing the number of clock cycles needed to perform a transfer. The original ISA bus required eight bus clock cycles for each 8- or 16-bit transfer, most of this being superfluous. EISA type A transfers eliminate two of the eight bus cycles and type B transfers eliminate four. These modes are designated for use by legacy ISA adapters, many of which have the capability to run at one or another of these accelerated speeds. Type C transfers, synonymous with the burst mode described above, are designed solely for use by EISA adapters, and eliminate all but a single bus cycle from the sequence, with the same 1,024 double word address limitation.

The ability to recognize and translate between different bus widths is also built into the EISA specification. An EISA bus controller is responsible for breaking down 32-bit transfers into two or four sequential streams, 16- or 8-bits wide respectively, so that no data is lost when transferred between devices of different bus widths.

One of the design decisions that made the Micro Channel architecture incompatible with the AT bus was the use of level-sensitive interrupts that could be shared by several devices. The 15 edge-triggered interrupts of the ISA specification (which already had been raised from eight on the original PC) were clearly insufficient, and the prospect of adding more became overly complex and did not seem to be a good overall solution. The quandary was resolved by making each interrupt in an EISA machine individually configurable to edge- or level-triggered operation. The same limitations of the ISA design still apply. Edge-triggered interrupts cannot be shared with other edge- or level-triggered interrupts. However, EISA cards effectively can share level-sensitive interrupts with other EISA cards.

Installing EISA expansion cards into a PC is not quite as simple as installing MCA cards but is still a great improvement over installing ISA cards. Rather than having a number of possible memory addresses that can be used by any adapter in the system, the EISA specification calls for each slot to have an assigned range of addresses that can only be used by that slot. In addition, all EISA cards are assigned a product identifier that makes them uniquely recognizable to the system. Configuration is performed through the use of software (although DIP switches or jumpers are still present on many EISA cards as a backup), with settings saved in the CMOS system ROM, which has been enlarged for this purpose. Usually, it is necessary only to identify the slot into which a particular card has been inserted, and the software is able to manage the selection of all other necessary parameters.

It took several years for EISA to become as stable a standard as it is today. Software support for its full capabilities was slow in coming, and the extra circuitry involved in building an EISA machine raised the price of a PC by as much as $1,000. Both of these problems have long since been overcome, though. An EISA bus adds approximately $300 to the price of a PC today and full software support, as well as a wide range of expansion cards, has been available for some time.

The next sections examine two newer bus types that significantly expand on the capabilities of those already considered. These up-and-coming bus types are very new technology, however, so while they hold great promise, there are definite indications that all the kinks are not yet worked out. As stated earlier, your file servers are not a place to experiment. For this reason, you should make a detailed study of every aspect of the new local bus types before purchasing a server that relies exclusively upon them.

### VESA Local Bus

While the Micro Channel and EISA designs both made significant increases in the overall speed of the PC's I/O expansion bus, there were aspects unrelated to their fundamental design that prevented them from being the complete solution that users were looking for. Aside from the licensing and marketing problems of the MCA bus, it was quite some time before software drivers were released that allowed the full capabilities of the EISA and MCA technologies to be exercised. Particularly in the case of the EISA bus--which without its bus mastering and burst mode was only slightly faster than ISA--the average user was not able to see an appreciable difference in transfer speed.

So, as the 1990s began, work continued on the creation of faster buses, and developers hit upon a solution that amounted more to a step backward than forward. Since microprocessors were being produced by then in speeds as fast as 33MHz, while the EISA and MCA buses were still hampered by their 8MHz and 10MHz speeds, someone hit upon the idea of relinking the I/O bus to the processor clock, as in the original IBM PC. By having both subsystems run at speeds up to 33MHz on a 32-bit-wide data path, surely this was a way to realize a noticeable increase in speed. This was also the time at which Windows 3.0 was released, and the PC computing world began to participate in the GUI revolution.

The primary bottleneck in running the new generation of graphical applications and operating systems was the video subsystem, and the newly revived concept of the local bus was put to work in speeding up graphics performance. Utilized at first in proprietary designs that linked video circuitry to the processor clock right on the motherboard, the concept got off to a shaky start, as no design proved dominant enough to claim a market share worthy of third-party hardware development. The speed enhancement was there, but the integration of the video circuitry onto the motherboard prevented systems from being upgraded to the latest graphic accelerator designs that were just coming into play. There was enough interest in the concept, however, that in 1992, the Video Electronics Standards Association (VESA) began work on a draft standard for a local I/O expansion bus that was ratified in August of that year.

Called the VESA local bus, VL-bus, or simply VLB, this standard--unlike those that had come before it--was designed to augment an existing bus design, not replace it. Using a standard 112-pin MCA connector, the bus interface was located as an extension to an existing ISA, EISA, or MCA slot. For example, an ISA-based VLB slot would consist of the original 8-bit connector, then the 16-bit extension, and the VLB connector immediately beyond. Local bus expansion cards could be designed with pins for all three connections, or just one set for the VLB connection. Likewise, a VLB slot could be utilized for a normal ISA, EISA, or MCA card, with the added connector simply left vacant, if desired. This was an ingenious piece of design work, sacrificing nothing in backward compatibility while contributing the potential for tremendous improvement. Because the specification calls for an expansion bus linked to the processor's clock as well as motherboard integration of local bus circuitry, the design became a viable one for more than just video acceleration. NICs and SCSI controllers also could be adapted for use on the VLB, and this piqued the interest of network administrators who were always looking for ways to improve hard drive and network throughput in their file servers.

Unlike the MCA and EISA standards, which set out to be revolutionary designs, the beauty and effectiveness of the VLB standard is in its simplicity. In its most basic form, it is little more than a series of address, data, and control signals extended from the Intel 80486 microprocessor to an expansion device. Using a single hardware interrupt control line to interface with the system's existing ISA, EISA, or MCA bus, VLB peripherals can take advantage of attributes provided by the other designs, such as DMA control and level-sensitive interrupts. In addition, the VESA local bus is entirely software transparent. Most of its speed is an inherent property of the basic design, which adopts the electrical and timing specifications directly from the 486 processor bus, and not the result of sophisticated bus mastering techniques that require customized software support to be effective, as with EISA.

However, the fact that the VLB was designed around the Intel 80486 microprocessor has become its primary weakness as well as its primary strength. The first VESA local bus standard called for a system bus running at a speed no greater than 40MHz and had 32 data lines and 30 address lines, enabling it to address up to 4G of memory. At this speed, the VLB could perform a write operation with no wait states and a read with one wait state. At speeds over 40MHz, penalties accrue in the form of two wait states for both reads and writes. Like the standards discussed earlier, VLB also has a burst mode that can provide a maximum theoretical throughput of 132 M/sec.

Another problem that arose due to the speed at which the bus functioned was the amount of electrical interference generated by the signal. An increase in the number of connectors causes an increase in electrical capacitance, and raising the bus frequency by almost 400% exacerbates this considerably. For this reason, the first version of the standard called for no more than three local bus devices on a 33MHz system. As clock speed increased, the number of devices was reduced. A 50MHz 486 could run only one local bus device, and it had to be integrated into the motherboard to support the specification. The version 2.0 specification allows for three devices in a 40MHz machine and two in a 50MHz machine, provided that the systems utilize a low-capacitance design. This in itself was not unduly limiting, however, as nearly all PCs could effectively utilize the increased speed only in three functions: video, disk access, and network access. For use in an average server, video is inconsequential, leaving enough bandwidth for network and disk storage connections.

Overall processor speed is also a consideration. Although the 2.0 standard allows for bus speeds of up to 66MHz, the electrical capacitance problems effectively limit bus speeds to an absolute maximum of no more than 50MHz. For use in most 486 systems, this is acceptable, as the clock-doubled and -tripled processors all interface with the system bus at speeds of no more than 33MHz. For Pentium systems, despite the fact that the system bus runs at no more than 33MHz, the VLB is not a recommended solution. Various workarounds have been proposed as a means of adapting the bus design to the new breed of processors, but a definitive solution has not yet been accepted by the industry.

The second version of the VLB standard, which was created to accommodate the release of the Pentium, calls for a 64-bit-wide path, while retaining backward compatibility with the earlier standard. When used in a Pentium system, however, or with any of the other 64-bit processors listed in the specification (such as MIPS and the Power PC), the VLB standard requires the inclusion of a bridge between the bus and the processor that technically prevents it from being a true local bus. Obviously, the revised standard is a calculated response to the double punch of Intel's Pentium processor and its PCI specification. It adds support for write-back caching and introduces an additional signal that can be used to automatically identify the 486 systems that are capable of supporting VLB's advanced burst mode features. It remains to be seen whether the VLB design can be successfully adapted to 64-bit systems and remain competitive to PCI, which was designed from the ground up as a Pentium-based specification.

When implemented in the environment for which it was intended, the VESA local bus standard provides a performance increase that is easily perceptible and undeniable. Although the gain is more obvious when it is used for video, rather than for disk or network interfaces, the gains over EISA and MCA systems are significant enough to warrant advocacy. Therefore, in any 486 file server--except one based on the 80486DX-50 processor (which runs the system bus at 50MHz)--I do not hesitate to recommend the VLB, preferably in combination with EISA, thus allowing the new standard to take advantage of the shared interrupts and advanced DMA transfers of the older one. Its simplicity and directness results in little added cost to the price of the system board, and its integration with the 486 architecture provides excellent performance. I am just as quick, however, to recommend against utilizing the VLB in a Pentium system. The expansions to the specification seem to be too much of a retrofitted solution for some fundamental inadequacies of the design, when it is run in a Pentium. This has, in recent months, become virtually a non-issue, though, as few Pentium machines are even being offered with VLB as an option anymore.

As far as configuration of the actual hardware is concerned, the VESA local bus specification makes no attempt to govern the manner in which expansion cards are configured. The decision whether to use traditional jumpers or DIP switches, a software-based configuration program, or an auto-detecting Plug and Play design is left solely up to the board manufacturers. In addition, care must be taken in the treatment of the VLB expansion cards themselves. Criticism has been leveled at several manufacturers of VLB cards regarding difficulties when inserting the card into the VLB slot. ISA and EISA cards typically are beveled and rounded at the corners of the edge connectors. VLB cards often are not, and the connectors themselves are thinner and more fragile than those on older adapters (see fig. 5.7). When adding a VLB card to a PC, make sure that the path of the card into the bus slot is unobstructed, so that it can be inserted straight down from directly above the motherboard.

Figure 5.7
These are bus connectors of a VLB adapter. To the right are the standard 16-bit connectors and to the left are the smaller, local bus connectors.

### PCI

The VESA local bus standard was conceived and ratified in less than eight months. Clearly, it was an attempt to get products using this technology to market as quickly as possible. The further development of the standard was undertaken much more slowly. This was because the VESA 2 specification was not only a response to the introduction of the Pentium but was also a direct response to the release of the Intel PCI local bus specification, a far more detailed and comprehensive specification for a different bus design, built around the Pentium processor. In fact, the PCI bus cannot truly be called a local bus at all.

Since it is a self-contained system, utilizing a bridge to establish a 33MHz synchronous connection to the microprocessor bus--rather than connecting directly to it--the peripheral component interconnect (PCI) bus is technically referred to as a mezzanine bus. A buffered PCI controller examines all signals being delivered from the processor and passes them along, as needed, to the appropriate slot--this might be PCI, or one of the slots used by the existing system bus design. This is because, like VLB, PCI is designed to augment an existing bus, not replace it. The two are dissimilar, however, in that the PCI design is not tethered to the clock speed of the microprocessor. The bus can be slowed to accommodate legacy devices or even stopped completely to save power. This is all performed completely independently from the operation of the system processor, which makes the design ideal for the new generation of bus mastering controllers that can operate with the PCI buffer as an isolated subsystem. Data transfers can be sent from the CPU to the buffer, from which they can be transferred to the destination device without any further intervention from the CPU. If the target signals a not-ready state, then the data waits in the PCI buffer until it is free to accept data--all the while, the processor continues working on other tasks. This makes PCI inherently suited to the burst mode capabilities of the Pentium processor.

The PCI connector itself, like that of the VLB, is a 124-pin MCA-type slot that is offset from the other bus slots on the motherboard. Unlike VLB, there is no aspect of the design that can accommodate the connection of one adapter to both expansion buses. Indeed, as we shall see, this is a technique that is expressly to be avoided. Some systems are available in which PCI slots share a backplane with slots for other bus types, but these are for use by one type of adapter or the other, not both.

In a manner similar to that of the MCA bus design, the pinout of the PCI connector is expressly designed to offset the same high electrical capacitance problems that have plagued VLB systems when running the bus at high speeds. Every live signal line on the connector is either next to or opposite a redundant power supply or ground line, and the specification even specifies standards for the location of the PCI chips on the motherboard, to minimize the overall length of the bus. Despite this, however, PCI systems--like those using VLB--usually are limited to a maximum of three bus mastering PCI devices (whether they are slotted or located on the motherboard), although additional slave slots up to a total of 10 can be supplied.

Some high-end servers have begun to appear with motherboards containing dual, bridged PCI subsystems that increase the number of available slots, but this should not be necessary for anything other than a "super-server" configuration, as multiport PCI NICs and SCSI adapters are currently available. Accommodations have also been made for the two different voltages currently used by Intel processors. Separate connector types are specified for 3.3v or 5v operation that prevent a card from being inserted into a slot of the wrong voltage, and an additional design is included for expansion boards that are individually keyed to fit either slot when they are capable of operating at both voltage levels.

PCI is a more forward-thinking specification than VLB. It anticipates the increasing use of high-end multimedia hardware (such as that for videoconferencing, which Intel coincidentally has a hand in developing) and multiple processors in desktop computers, and is expressly designed to be adaptable to many current and future systems. Indeed, the bus is almost completely self-contained. Protocols for arbitrated bus mastering are furnished using a dedicated 4-bit command code that makes it completely processor-independent. Although it has not yet been fully realized, PCI is very likely to set a standard for motherboard design that will accommodate all new microprocessor designs for some time to come, including the Intel Pentium Pro, and even high-performance 64-bit RISC processors designed by other manufacturers including MIPS, Alpha AXP, and the PowerPC.

PCI is also designed to be a major step toward the long-awaited but never realized Plug and Play specification. Like EISA and Micro Channel, PCI adapters utilize no jumper or DIP switches. They are required by the specification to contain 256 registers for the storage of configuration data, which is relayed to and from the bus via a signal dedicated to that purpose. Once the new breed of operating systems has fulfilled the promise of an operational Plug and Play design, then PCI, along with the use of an expanded system BIOS, will be able to meet them halfway in their goal of completely eliminating the need for manual configuration by either hardware or software means.

Despite the differences in their intents and designs, VESA local bus and PCI systems deliver performance levels that are remarkably similar. Both have a theoretical maximum throughput of around 130 M/sec, although these figures represent peak burst mode rates; neither will provide anything even close to this speed in normal use. Both technologies are perceptibly faster than either MCA or EISA, though, and each seems particularly well-suited to a specific platform. I would even go so far as to say, without hesitation, that when buying a 486-based file server, use VLB, and when buying a Pentium, include some PCI slots. In both cases, however, I would strongly recommend the inclusion of EISA slots as a subsidiary bus. In the case of the 486, the VESA local bus will be able to take advantage of EISA's attributes, and in both cases, the EISA slots provide an effective fallback. Should any problems or incompatibilities arise with these new technologies, it would be possible to revert to EISA performance levels for no more than the price of an additional adapter card.

It is unfortunate to see, though, that as 486 machines become increasingly rare in the file-server market, support for the VESA local bus is waning with them. I maintain that VLB has always had as much potential as the PCI bus but was lacking primarily in a powerful corporate advocate, such as PCI has in Intel. Now that IBM and Apple are planning to phase out their long standing MCA and NuBus designs in favor of PCI, and even non-Intel platforms such as Digital's Alpha AXP are using it, PCI's future seems assured, despite continuing compatibility problems.

Combining VLB and PCI. One innovation that is being marketed by some vendors is a line of motherboards containing both VLB and PCI slots. This is an abortive design that should be recognized immediately by anyone familiar with both technologies as a blatant attempt to appease both camps that ends up satisfying neither. PCI is, by definition, a buffered architecture that is designed to remain separate from the processor bus. When VLB is present in the same system, you essentially are attaching the PCI bus directly to the processor. This creates an additional load for the system processor that PCI was specifically designed to eliminate and degrades the performance of the PCI bus at the same time. In other words, each of the two architectures detracts from the performance of the other. They are fundamentally incompatible and should not be present in the same system.

Motherboards and Chipsets. The proliferation of PCI and other standards has caused users to take a greater interest in what is perhaps the most crucial single component in the PC, the motherboard. Many savvy users are able to quote disk drive specifications from memory and elaborate the advantages of this or that new processor but still will purchase a computer without any investigation into the manufacturer and capabilities of the motherboard. PCI, in a rather backhanded way, has changed this.

As with any new technology, the benefits tend to look far more attractive on paper than when they are realized in silicon. The first PCI adapters that went to market were fraught with a large number of incompatibility problems that still are being resolved. In fact, the problems seldom were due solely to the design of the expansion card but more often were attributable to driver software and especially to the core-logic chipset designs used on PCI motherboards. Even the early Pentium motherboard designs by Intel, using their 3.3v Neptune chipset, have been known to cause compatibility problems with certain adapters.

Intel recently has released the Triton chipset, which is reported to offer better PCI support (including an enhanced burst mode) and cache management than Neptune--it's cheaper as well. It is aimed at laying the groundwork for the introduction of the Pentium Pro processor. Other chipset designs being used by higher-end motherboard manufacturers are made by SIS and Opti, but Intel is clearly the dominant player in the field. At this time, the only way to avoid known compatibility problems is to gather as much information as possible before making any purchases. Ask motherboard vendors what chipsets they are using, and try to find out from manufacturers of expansion cards about any hardware incompatibilities they might be experiencing.

> NOTE: Finding out about hardware incompatibilities is usually better accomplished by contacting the manufacturer's technical support department rather than salespeople, and it usually is best to have specific information at hand, rather than asking general questions. For example, "I'm setting up a new server and I want to use your PCI SCSI adapter. Have you encountered any problems with chipsets made by Brand X?" is more likely to get you an honest, detailed answer than "Are there any problems with your PCI SCSI card?"

## Disk Subsystems

Despite the tremendous advances in microprocessors and memory discussed thus far, no component of the PC has advanced as far and as quickly as hard drive technology has. In the space of 12 years, the maximum capacity of hard disk drives has increased over 400 times and access speeds more than 10 times. Some of the same users who wondered if they would ever fill a 10M or 20M drive now look casually at a 4G model and wonder if it will be enough. These improvements, of course, are a direct response to the needs of users and the feats of programmers, who always manage to keep the hardware industry on its toes by continually devising applications that are larger and more capable than their predecessors.

Network file servers, of course, are where this technology is most desperately needed. A substantial portion of NetWare's resources is devoted to the management of the file system, and when dozens or hundreds of users are all accessing files on one server's drives at the same time, the throughput required of the hardware can overwhelm the capabilities of even the cutting edge in storage technology. It is not just a question of drive capacity or even of raw throughput. Network environments can generate hundreds of file requests at the same time, far more than any stand-alone operating system, even those of the multithreaded, multitasking variety such as OS/2 and Windows 95.

What is required is a network storage subsystem that can support multiple devices, and manage these requests without placing an undue burden on the server's processor. The system also should be able to multitask the requests to different devices so that files can be accessed from different drives simultaneously. As a result, the system should have sufficient bandwidth to accommodate not just the maximum throughput of each individual drive but the combined transfer rates of all the devices in case they are all accessed at once.

It should not surprise you to find out that these ideals often are not fully realized in the average network file server. The purpose of this section is to identify the various components of such a subsystem, locate the areas in which bottlenecks typically occur, describe ways in which these bottlenecks can be reduced or eliminated, and finally, to describe the process of installing and configuring the hardware of such a subsystem. It should be noted that this chapter deals primarily with a basic server storage system and concentrates on throughput. Many new technologies are being used in the networking industry to provide for data redundancy on the hardware level. RAID arrays are covered in appendix C, "RAID," but these function essentially as extensions of the basic system defined here.

### Storage Technologies

Advances in hard drive technology are some of the primary stepping stones that have made networking possible. When the first IBM PC arrived in the early 1980s, with its Seagate ST-506 hard drive, 10M of storage and a 120ms access time seemed like a lot, but it really didn't give the average businessperson any ideas about accessing it from across the room, let alone from across the country. By the late 1980s, however, drive technology had progressed through several incarnations (see chapter 6, "The Workstation Platform," for the whole story), resulting in two basic drive types that provided the relative speed and capacity needed for use in file servers. These were ESDI and SCSI.

The MFM and RLL drives--which ultimately developed into the IDE standard that is the most common hard drive type found in workstation and stand-alone PCs today--could not provide sufficient capacity, either in single drives or by combining multiple units. The common PC BIOS configuration--still used to this day--can support no more than two of these drives, and its addressing scheme limits them to no more than 528M each, making them unsuitable for use in file servers.

ESDI. The enhanced small device interface (ESDI) was developed in the early 1980s, primarily by Maxtor Corporation. It offered several advantages over the ST-506 standard that it sought to replace, among them a maximum theoretical transfer rate of 24 megabits per second. By the late 1980s, ESDI had become the dominant standard for network file servers. This did not last long, however. Forced out of the workstation market by the overwhelming popularity of IDE drives, ESDI's advocates found the network server market dwindling as well, due mostly to the increased popularity of SCSI, which offered greater expandability, support for more device types, and better performance. There also were compatibility problems that often made combining ESDI drives from different makers problematic.

ESDI drives are virtually obsolete today. They can be found in some older file servers, and are easily identifiable by the fact that they utilize two connections to the bus: a 34-pin control connector and a 20-pin data connector. These probably would be 386-based servers, as the drives had all but disappeared from mainstream use by the time the 80486 processor came into popular use. If you have a server that is utilizing ESDI drives and the performance is satisfactory for your needs, the best advice that I can give is to use them until they drop.

Most 386s have long since been reduced from servers to workstations, where they can continue to be useful for some time. It would be difficult to locate sources for new ESDI drives and controllers today, making their continued use unreliable at best. If an ESDI drive in such a machine dies, and the rest of the machine is worth saving, an IDE drive can be added for workstation use--but for a server, the smart move is to get rid of ESDI drives, as the potential for extended downtime when they fail is very high.

### SCSI

The small computer systems interface (SCSI, pronounced "scuzzy") is far and away the predominant file server storage interface in use today. The new Enhanced IDE interfaces may prove to be suitable for smaller file servers as well as workstations (see the next chapter), but for large installations, SCSI has no rival and is not likely to have any in the near future. The standard SCSI-2 interface provides for throughput rates of up to 10 M/sec and supports up to seven devices on a single host adapter.

The devices can be hard disk, tape, optical, or CD-ROM drives, as well as autochangers and jukeboxes, providing almost unlimited storage capacity. Devices are assigned unique SCSI IDs at the physical level which are utilized for the logical addressing through which each device can be accessed using an arbitration protocol controlled by the host adapter. This makes it an ideal interface for the demands of network use, as users' requests for files on different drives can be queued and executed by the host adapter without draining the resources of the system's microprocessor. SCSI devices are in use today on a great number of different computing platforms. Apple Macintosh systems provide native SCSI access and many mini-computer and microcomputer platforms use SCSI host adapter expansion boards to take advantage of the interface.

SCSI also has a reputation for being difficult to install and configure, on both the hardware and software sides. At one time, this reputation was well-earned, but the standards by which SCSI devices are manufactured have matured considerably, and with the knowledge of some basic operational principles and a few hardware rules to live by, installing or expanding a SCSI subsystem can be a fairly easy procedure.

The SCSI Standards. SCSI is not just a controller that provides access to devices but is actually a completely self-contained system bus that is accessed through the computer's expansion bus. It has its own hardware standards, signaling protocol, and command language. It has its origins in an interface called Shugart Associates Systems Interface (SASI) that was developed by Shugart Associates and the NCR Corporation in 1981. SASI's basic goal was to develop a standardized peripheral interface that would provide high performance while remaining independent from the host system's processor. This would allow the manufacturers of peripheral devices to have a set of guidelines for the design of new products, ensuring that there would be an interface that could support them and that users would not have to re-engineer their systems to take advantage of every new hardware development.

The original standard called for up to eight devices connected by an 8-bit-wide bus and a maximum transfer rate of 5 M/sec. Each device must contain a SCSI bus controller that facilitates the communications between the hardware device itself and the bus. At least one of the eight devices must be a SCSI host adapter. The job of the host adapter (also referred to as an initiator) is to arbitrate all the communications taking place on the SCSI bus, as well as to provide the interface to the system bus, through which the data going to and from the SCSI devices (or targets) reaches the rest of the computer. The standard calls for the capability to include multiple initiators on a single bus, either in the form of multiple host adapters or by the use of target devices that can themselves function as initiators at specified times. In this manner, data transfers can take place between separate devices wholly within the confines of the SCSI bus, requiring no intervention from the system processor. This proposed standard was submitted to the American National Standards Institute (ANSI) for ratification, where it underwent extensive revisions before it finally was approved in 1986.

By the time approval took place, certain shortcomings in the original standard had already been identified and work had already begun on the SCSI-2 standard, which originated from the development of a core group of 18 basic SCSI commands that became known as the common command set (CCS). A number of other optional components were added to the specification that contributed to the overall speed and efficiency of the interface. In fact, now that SCSI-2 has been ratified, work has already begun on SCSI-3. (There is an entire subculture within the computer industry that revolves around the continuous development and ratification of standards such as this).

SCSI-2. The most important thing to know about the differences between the original SCSI and the SCSI-2 standard is that the latter is a superset of the former. In other words, every SCSI-1 device is also a SCSI-2 device and is completely supported by the newer standard. SCSI-2 peripherals can be connected safely to a SCSI-1 host adapter, but they will be unable to take advantage of any SCSI-2 features that they contain. Do not be fooled into judging the performance levels of a SCSI device merely by the SCSI-2 appellation. If there is a problem inherent in both SCSI standards, it is that too many of the features they contain are listed as optional. A vendor is perfectly within his rights to take a carload of hard drives made in 1987, slap SCSI-2 labeling on them, and sell them as such, although they contain none of the features that make SCSI-2 an improvement over the original standard.

As stated earlier, a SCSI host adapter communicates with the peripherals on its bus by means of its own proprietary command language. Unlike most communications protocols used by the average computer network today, the SCSI language is composed primarily of English words, most of which are easily comprehensible. There are many software packages that provide user access to the basic command language (usually for debugging purposes), and it is relatively easy to identify when a drive is being sent, for example, a READ or a WRITE command. The original SCSI standard contained commands sufficient primarily for the operation of hard drives. The CCS that was included as part of the SCSI-2 standard, however, was primarily designed to furnish standardized support for other types of devices that required commands not available in the original command set.

> NOTE: The original standard allowed for vendor-specific additions to its command set, but this practice was likely to result in incompatibilities between devices made by different manufacturers, thwarting the very reason for the standard.

> For example, a tape or CD-ROM drive might be able to utilize a software function to eject the media from the drive. If two manufacturers of tape drives utilize two different commands to perform this same task, then the manufacturer of host adapters is forced to choose between them, or try to support both. This would quickly get out of hand. Customers would have to buy drives not knowing if their host adapters supported them, or be locked into purchasing products from only one manufacturer. This is the way that standards fail, and the committee that controlled the SCSI standard was quick to recognize this and take action.

Incidentally, manufacturers generally do not wait for a standard to be ratified before they start producing equipment that conforms to it. The standards are made available to the public while they are in development and a great many SCSI-2 devices were on the market long before the standard was officially ratified. This is to be expected when it takes eight years for a standard to be approved. In the case of SCSI-2, there were very few differences between the document proposed in 1986 and the final approved version in 1994, so these premature devices are not likely to cause any serious incompatibility problems (at least, not due to the standard).

SCSI-2 Options. SCSI-2 also outlines a number of distinct enhancements to the interface, nearly all of which were specified as optional. Some of them have come into common usage, while others are almost never seen in the PC world. These options provide for a large array of products on today's market, with varying capabilities. Many of these variants will function on a basic level even when incorrectly combined with mismatched equipment. As stated earlier, a Fast SCSI drive will function when attached to a host adapter that doesn't support Fast mode. That is, you are able to access the data on the drive, but the device does not function in its Fast mode unless the host adapter is compliant, so the extra money spent on the more capable drive is wasted. When purchasing a new storage subsystem or even augmenting an existing one, it is important to know the capabilities of all your equipment before making any purchasing decisions. This way you can get the performance that you expect from a product, and the most for your company's money.

Differential SCSI. One of the most fundamental options for the SCSI interface is known as differential SCSI. Almost never utilized in the PC networking industry, differential SCSI is opposed to single-ended SCSI, which is the normal implementation of the interface used by PCs. The essential difference between the two is the length of the SCSI bus. Normal single-ended SCSI is limited to an overall bus length of six meters. This includes the entire aggregate length of the bus connectors, both inside and outside the computer case. Differential SCSI utilizes two wires for each signal on the bus, while normal SCSI uses one. The use of inverted signals on a differential bus reduces the amount of noise during communications and allows for a bus length of up to 25 meters.

Differential SCSI devices are identified by a standard symbol that appears somewhere on the device (see fig. 5.8). Even though differential SCSI is very rarely seen in the PC world, I mention it because it is one of the few cases in which SCSI devices of different types cannot be intermixed. The two device types utilize the same connectors and cabling, but connecting a differential device to a single-ended SCSI bus, or vice versa, will cause a smell you've probably never encountered before and probably will smoke the entire bus.

Tagged I/O Process Queuing. The SCSI-1 standard included a specification for the transfer of individual commands from an initiator to a specified target device. This was known as untagged queuing because only a single process request could be sent to a device. The next request could not be sent until the process had been completed. SCSI-2 allows for optional tagged queuing, in which up to 256 individual process requests can be sent by one or more initiators to a single target. The processes are held in the target's buffers until executed. In addition, the processes, referred to by their queue tags, can be executed by the target in a sequence differing from that in which they were received to achieve maximum efficiency in accordance with the native logic of the target device.

Fast SCSI. Fast SCSI is another optional feature of the SCSI-2 standard. The term is used to refer to the synchronous transfer mode that doubles the overall transfer rate of the interface from 5 M/sec to 10 M/sec. The basic difference between synchronous transfer mode and its asynchronous equivalent is that an asynchronous transfer requires a acknowledgment (or ACK signal) for every byte transferred, while a synchronous transfer can send a series of bytes for every ACK. Again, both the SCSI host adapter and the target device must support this feature for the increase in throughput to be realized.

Figure 5.8
These are the universal symbols for single-ended and differential SCSI.

> NOTE: Bear in mind that transfer rates cited throughout this section define a theoretical maximum burst mode rate and are furnished primarily for comparison purposes. Actual throughput under normal conditions will be considerably less.

Wide SCSI. Wide SCSI, as the name implies, is the optional practice of increasing the throughput of the interface by widening the bus. A Wide SCSI bus can also accommodate up to 15 Wide SCSI devices instead of the usual seven devices. Specifications have been made in the standard for 16-bit-wide as well as 32-bit-wide SCSI buses. A 16-bit bus, logically, doubles the throughput of the basic SCSI protocol from 5 M/sec to 10 M/sec. When used in combination with Fast SCSI, the overall transfer rate reaches 20 M/sec. This, quite logically, is referred to in the marketplace as Fast & Wide SCSI. The 32-bit implementation of the bus can double this again to a 40 M/sec transfer rate. The implementation of the Wide SCSI standard has led to the different types of SCSI cable connectors that are used to create the bus. In practice, however, while Fast & Wide SCSI has found some acceptance in the marketplace, Wide SCSI by itself is not often seen, and the 32-bit variant is virtually non-existent.

The use of Wide SCSI requires that both the host adapter and the target device be capable of utilizing the increased bandwidth. Normal 8-bit SCSI devices can be attached to a Wide SCSI host adapter, however, with an appropriate cable. The Wide SCSI functionality is implemented during the handshake between the initiator (usually the host adapter) and each target device. If either of the two is configured to utilize Wide SCSI, and both are capable of it, then the additional bandwidth is utilized. This is different behavior from that of Fast SCSI, in which both devices must be configured to use synchronous transfers.

Table 5.1 provides a brief comparison between the various SCSI busses.

#### Table 5.1 SCSI Variants and Specifications

SCSI Variant | Bus Width | Bus Speed | Cable | Max. Bus Length
SCSI-1 | 8 bit | 5 M/sec | A-cable | 6 meters
Fast SCSI | 8 bit | 10 M/sec | A-cable+ | 3 meters
Wide SCSI | 16 bit | 10 M/sec | P-cable+ | 6 meters
Fast/Wide SCSI | 16 bit | 20 M/sec | P-cable+ | 3 meters
Fast/Wide SCSI | 32 bit | 40 M/sec | P-cable+ Q-cable | 3 meters

SCSI Cables. The basic SCSI interface calls for the use of a 50-connector Centronics-type cable that is referred to in the standard as the A-cable. The original 16-bit-wide SCSI specification calls for a 68-connector B-cable that must be used in addition to the A-cable. This was not greeted with much enthusiasm by users, and one of the first acts of the SCSI-3 committee was to drop the B-cable concept in favor of a single 68-connector P-cable of D-shell design that replaces both the A-cable and B-cable.

The 32-bit-wide SCSI bus, which has never caught on in the marketplace, requires a P-cable and an additional 68-conductor Q-cable. The initial failure of 32-bit-wide SCSI to gain market acceptance was due in part to the fact that drives were not yet available to take full advantage of this much throughput. The need for a second cable, though, was definitely a contributing factor to the unpopularity of 32-bit-wide SCSI and probably will continue to be in the future.

Other cable designs are used in some SCSI implementations. Both Apple and IBM utilize proprietary cables for their SCSI connections (what a surprise!). IBM utilizes a 60-pin mini-Centronics connector. To attach a non-IBM external SCSI device to an IBM server, you must obtain a special cable from (you guessed it!) IBM. Leave it to Big Blue to de-standardize the standards. Apple's 25-pin D-shell connector also differs from the norm, looking more like a serial cable than a SCSI cable.

> CAUTION: Even when connectors differ, a SCSI device is a SCSI device, no matter what platform or operating system it is used with. Don't be fooled by unscrupulous vendors into thinking that you need to buy particular drive because "it's a Macintosh SCSI drive" or "it's an IBM SCSI drive." This probably only means that the package includes a different cable.

While we are on the subject, it should be mentioned that buying off-brand or generic SCSI cables should be avoided. After spending thousands of dollars on SCSI drives, don't ruin your bus (and your day) with cheap cables. I have known people to spend hours trying to troubleshoot a SCSI installation--or worse yet, pay someone a hefty hourly fee to do it for them, only to find that a $20 cable was the cause of their aggravation. Remember that a SCSI bus functions as a unit. If you leave a gap anywhere in your defenses, data is sure to find a way to leak out of it.

Termination. As with other bus types, the SCSI bus must be electrically terminated at both ends of its length. This is to ensure that signals that have been propagated throughout the bus are promptly and completely removed from it. Otherwise, they can be reflected back, causing increased noise and possible data corruption. This is particularly crucial when there are two or more target devices on the bus, as signals destined for a particular target can be mixed with other signals, affecting the performance of all the targets. I have seen cases in which the terminators were removed from a bus containing one host adapter and one device, with no ill effect, but this is not recommended.

Termination of the bus can be achieved in several ways. The simplest and most common form is called passive termination and calls for the placement of resistor packs at both ends of the bus. The resistors negate the electrical charges that reach them, effectively removing signals from the bus. These resistors can be supplied in many different ways. External SCSI devices are either internally terminated or utilize an external resistor pack in the form of a plug. Internally terminated devices always should have some means to disable the termination, when desired. This can be through the use of DIP switches, jumpers, or a user interface to the device, such as an LED display. Devices using an external resistor pack usually have two SCSI connectors on them to allow for daisy chaining of devices. In this case, your SCSI cable plugs into one of the connectors, and the resistor pack plugs into the other. It makes no difference which goes into which connector. Some devices have only a single connector, and utilize a pass-through terminator plug. This is a resistor pack containing both male and female connectors that plugs into the SCSI device's single socket and contains an additional connector for the SCSI cable.

Internal SCSI devices, including host adapters, usually have terminating resistors integrated into their circuitry and utilize DIP switches or jumpers to enable and disable termination. The host adapter is a SCSI device just like any other and must be considered when terminating the bus. Most host adapters ship with termination enabled, but if you have both internal and external devices connected to the adapter, then it is not at the end of the bus, and termination must be disabled. Remember that termination must be provided at the physical ends of the SCSI bus. SCSI ID numbers have no bearing on this whatsoever. Some people make the mistake of assuming that the devices using SCSI IDs 0 and 7 are the ends of the bus, but this is not the case. It is also important that termination be furnished only at the ends of the bus. Other devices may ship with termination enabled and must be adjusted if they are going to be located in the middle of the bus.

Another form of termination can be used in place of the passive resistor type usually furnished with SCSI target devices. Active termination is a means by which an inverse voltage of the signal, instead of resistors, is used to negate the transmissions on the SCSI bus. For example, when a 5v signal is traveling along the bus and reaches a resistor pack, it is dissipated by the added resistance over the course of a period of time that, in electronic terms, would be considered quite slow, and signal fluctuations may occur. An active terminator instead supplies a signal of -5v that negates the original 5v signal far more quickly and efficiently than resistance dissipates it. The SCSI-2 specification calls for active termination whenever Fast SCSI or Wide SCSI is used. It should also be considered when the overall length of the SCSI bus is more than one or two meters. Active termination is used already on some SCSI devices available today, particularly host adapters. In other cases, active terminators must be purchased separately, but for a high-performance system they definitely are recommended.

One of the remarkable aspects of troubleshooting SCSI installations is that the root causes of problems often are found to be the simplest components of the system, and improper termination ranks very high on the list. Many of the most common symptoms of SCSI problems, such as data corruption, dismounting of drives, and excessive read or write errors can be caused by missing or improperly installed terminators. A SCSI subsystem in a file server is likely to be worked harder than one located in any workstation. It must be as bulletproof as possible, and the attention you pay to small items like terminators and cables will pay off in the long run. Additional SCSI troubleshooting checklists can be found in chapter 17, "Backup Technology: Programs and Data."

SCSI-3. No sooner had the SCSI-2 standard been ratified in 1994 than work began on the SCSI-3 specification. Judging by this standards committee's track record, the new standard should be entered into the books sometime in the year 2002. You can open a magazine today, however, and see advertisements offering SCSI-3 hard drives and controllers. Although the proposed SCSI-3 standard contains support for up to 32 devices, and work is being done on a proposed "serial SCSI" implementation, none of this is supported by these devices. SCSI-3 has come to be used solely as a reference to a Fast & Wide SCSI-2 device that utilizes a single P-cable instead of the two cables called for in the SCSI-2 specification (see the preceding section, "SCSI Cables," for more information on Fast & Wide).

SCSI Hardware. When purchasing SCSI hardware for use in a file server, the requirements are quite different from those of a workstation. The object, obviously, is to provide the greatest possible performance for the least amount of money, but it is important to account for the unique requirements of file server access when planning for an upgraded or new server. The first decision to be made is what kind of SCSI you intend to run on the machine, because all your devices must support the type of bus you choose to assemble.

When we discussed the differences between regular, Fast, and Wide SCSI earlier, throughput rates of 5 M/sec, 10 M/sec, and as much as 20 M/sec were bandied about, but remember that this is only an estimate of the best possible throughput that the bus can achieve in its fastest burst mode. If you purchase a single 4G hard drive that can transfer data at 8 M/sec and put it on a Fast & Wide SCSI bus that is capable of 20 M/sec transfers, you will get no more than 8 M/sec of throughput. If you purchase four 1G drives of the same speed, however, and install them on the same bus, their combined maximum throughput is 32 M/sec, which can more than fill the capacity of the SCSI bus. Remember that a NOS like NetWare must be treated like any other multitasking OS. If you can picture a Windows (or maybe an OS/2) machine running dozens or even hundreds of applications at the same time, you get an idea of what the file system of a NetWare server must contend with every minute of every working day. SCSI is particularly well suited to this kind of environment because each drive on the bus contains its own controller. The computer can send multiple file requests through the host adapter to each of the individual drives, which can queue them, execute them, and furnish the required data as each drive gains control of the bus.

This is not to say, however, that every hard drive will be continuously running at its peak transfer rate. That is why the second configuration described earlier is more than sufficient for general use. There is no need to furnish sufficient bandwidth on the bus to accommodate the maximum transfer rates of all the target devices at once. What is important is to make sure to avoid bottlenecks that ultimately will waste the investment you've made in high-performance peripherals.

SCSI Hard Disk Drives. A SCSI hard disk drive is basically a normal disk drive assembly with an integrated controller, to which circuitry has been added that allows it to communicate over the SCSI bus. If you compare product lists or advertisements, you will find that most hard drive manufacturers have SCSI drive lines that are identical to their IDE drive lines in speed and capacity. This is because they are fundamentally the same drive, with a few extra chips added to the SCSI model. Like an IDE drive, the controller is wholly dedicated to communicating with this one device, so the manufacturers can make any proprietary adjustments that they wish. The only requirement is that the controller be able to communicate with the other devices on the SCSI bus using the common SCSI command language.

You might wonder what performance advantage SCSI provides over IDE, since they are fundamentally the same drive. For a stand-alone, single-user workstation, the qualified answer is: none. SCSI provides greater expandability, but essentially adds an extra amount of communications overhead to the device that can actually lessen the overall efficiency of the unit. In a file server environment, however, SCSI provides a definite advantage because of its command queuing and multitasking capabilities--IDE can only process one request at a time, and must finish that processing before executing the next one. SCSI also provides the expandability and large drive capacity needed for a network server, as well as the ability to support many different types of devices.

Most of the major SCSI hard drive manufacturers today market Fast SCSI-2 drives as their base product line, and the same is true for the manufacturers of host bus adapters. Wide SCSI-2 drives are also available from some manufacturers. I advise against purchasing new drives that do not support Fast synchronous transfers, even if you can find such drives. A Fast SCSI drive functions quite well on a 5 M/sec bus using a vintage SCSI -1 host adapter, if necessary, and this leaves you an open path for future upgrades. If you happen to come upon a source of dirt-cheap SCSI-1 hard drives, buy them, but use them in workstations instead of servers. If you are running a Wide SCSI host adapter, remember that you must buy Wide SCSI hard drives to take advantage of the extra bandwidth. In addition, Wide components have the 68-pin connectors that the feature requires--adapters or custom cables are needed to connect 8-bit SCSI peripherals to the bus.

As far as configuration is concerned, hard drives have come a long way from the early days when you had to be concerned with low-level formatting, interleave, and head skew. As with IDE, SCSI hard disk drives are low-level formatted at the factory, and since the controller is integrated and dedicated solely to the support of that one device, communications between the two have also been optimized by the manufacturer. There are a few SCSI-related settings, however, that must be considered. The following sections discuss these settings.

SCSI ID and Termination. As with any SCSI device, a hard drive must be assigned a SCSI ID different from all other devices on the bus. While different hard drives may use DIP switches or housing switches to set the ID, this is frequently accomplished by setting a combination of three jumpers on the drive according to the binary values of the numbers 0 through 7. (These are the same switch combinations that I used to input the octal code needed to boot the first computer I ever worked with, a Data General Nova 1200 mini, circa 1976. Some skills never die.) Table 5.2 displays the values of the possible jumper combinations, but be sure to consult the documentation for the unit to determine "which way is up."

#### Table 5.2 Jumper Settings for Specifying SCSI IDs on Hard Drives

SCSI ID | J1 | J2 | J3
0 | 0 | 0 | 0
1 | 0 | 0 | 1
2 | 0 | 1 | 0
3 | 0 | 1 | 1
4 | 1 | 0 | 0
5 | 1 | 0 | 1
6 | 1 | 1 | 0
7 | 1 | 1 | 1

If, however, you purchase a drive array housing that holds several hard disk drives, the housing itself may provide selector mechanisms for the individual drives in the form of a button or rotary knob that is used to set the ID.

There are some special concerns to be noted with hard drives in respect to SCSI IDs. Some older implementations of the SCSI interface required hard drives to be assigned SCSI IDs 0 and 1. This was true particularly when the hard drive was being used as a boot device. This limitation has supposedly been eliminated, but I have run into many situations in which reassigning the SCSI IDs of the various devices to make the hard drives 0 and 1 has resolved problems that could not be remedied any other way. On the other hand, the bus arbitration protocol of the SCSI interface assigns task priorities based on SCSI IDs, with 7 having the highest priority and 0 the lowest. This is why host adapters are almost always assigned SCSI ID 7, so that they will win in any arbitration. Therefore, if you want your hard drives to receive the highest priority attention from the host adapter (and in a file server, who doesn't?), I recommend assigning them the highest possible IDs below 7, and lowering them only if absolutely necessary.

Termination, as well, is usually activated and deactivated through the use of jumper settings. Be aware that some drives ship with no terminating resistors at all--in this case, resistors must be purchased separately and mounted on the drive.

Other SCSI Hard Drive Settings. Most of the other SCSI-related settings on hard disk drives are used to enable or disable certain aspects of the communication protocol between the target device and the host adapter. In these cases, the most important thing to remember is that these features must be enabled on both devices for them to function properly.

SCSI parity is a rudimentary error checking protocol similar to the parity checking done at the memory level by many computers. Virtually all drives and host adapters on the market today support it. The ability to disable the function is usually present simply for reasons of backward compatibility with older host adapters that do not have the feature. Unlike the parity checking done by memory chips, which many people consider unnecessary due to the comparative rarity of memory corruption errors today, parity checking on the disk level is definitely recommended, as disk errors are far more prevalent.

Synchronous negotiation is the process by which a host adapter and a device perform a handshake to determine how a data transfer is going to be performed. Each device is essentially informing the other of its capabilities so that the speediest possible transfer rate can be achieved. As stated earlier, nearly all SCSI devices on the market today support Fast SCSI-2 and are capable of running in the faster synchronous mode. This feature should be enabled on all SCSI devices, except in certain cases when older drives not supporting it are negatively affected by the host adapter's attempts to negotiate. I have also seen cases in which this feature is disabled on the entire SCSI bus to accommodate certain software packages (most notably backup software) that became overwhelmed by the faster transfers provided. This, however, results in a slowdown of all devices on the bus to accommodate one device, and should be avoided if possible.

Terminator power is a setting that controls whether or not a particular device furnishes the power needed by the bus terminators. This power is required whether the termination is active or passive, and is nearly always furnished by the host adapter. No harm can be done, however, by having multiple devices furnish terminator power, so many users simply enable this feature on all devices as a fail-safe mechanism.

Delayed start is a means by which the power-up cycle of a disk drive can be delayed for a short period after the computer is powered on. Hard disk drives require far more power to start up than to actually run. Spinning the platters up to the high speeds at which they run is functionally similar to pressing the accelerator on a car. More gasoline is needed to accelerate from 0 to 60 mph than is needed to cruise at 60. When several disk drives are present in a single computer, their combined power needs can overtax an otherwise adequate power supply, should they all spin up at once. The delayed start mechanism forces the drive to wait until a signal is provided by the host adapter before it powers up. The host adapter sends the signal to each drive on the bus in turn, thus preventing multiple drives from spinning up at once. If your drives are located in an external housing, then this feature is unnecessary, but if you have several drives within the main case of the computer, this feature is recommended.

Other SCSI Devices. One of the main advantages of using the SCSI interface in a network server is the wide array of devices that are supported. SCSI tape and CD-ROM drives have become commonplace features of the corporate network, providing the means for backing up the system and furnishing the wealth of data available on CD-ROM to users all over the network. Magneto-optical drives and other technologies provide mid-range storage options that allow for faster access times than tape, and lower costs per megabyte than hard drives. Jukeboxes and autochangers provide automated switching of tape and optical cartridges, yielding a potential for enormous amounts of data storage.

Hierarchical storage management systems are available to manage these multiple devices and automatically migrate network files to less expensive media based on their usage patterns. Writeable CD-ROM "burners" have dropped dramatically in price to the point where they have become a viable, albeit permanent, storage medium, and even printers are available that can use the SCSI interface. Detailed descriptions of some of these devices and the procedures for installing them on the network are covered in part VI, "Adding Network Services," of this book.

All these devices are SCSI and can communicate with the same host adapter that controls hard drives. One of the primary improvements of the SCSI-2 specification over the earlier revision was the inclusion of standardized commands for the control of devices such as these. The rules for connecting these devices to the SCSI bus are virtually identical to those for hard disk drives. Unique SCSI IDs must be assigned to each device, and termination provided where necessary. External devices usually have some sort of selector mechanism, either mechanical or electronic, for setting the ID, and have a separate resistor pack for termination. Internal devices use the same sort of DIP switches or jumpers as hard drive units. If anything, the hardware installation of these devices is simpler than that of a hard drive. Most other SCSI devices do not have the additional configuration settings that hard drives have. Once the ID and termination are considered, it is just a matter of connecting the data and power cables, and then turning on the unit. The area in which difficulty might arise is when you try to address the device with software. This is covered later, in the "SCSI Host Adapters" section.

One of the more prevalent tendencies in these products, now that the technology has matured and confidence in the interface is high, is the combination of multiple devices in a single unit. CD-ROM changers and multiple drives are available that can hold up to 7 or 10 CD-ROMs or more, providing either simultaneous access or automatic media switching. Tape autochangers and optical jukeboxes typically consist of one or more drives for actually reading the media, and a robotic mechanism that is responsible for shuttling the cartridges in and out of the drives. Some of these units are truly gigantic. Exabyte markets several autochangers that could easily be mistaken for refrigerators. One model has four 8mm drives and holds 120 tapes, for a total capacity of up to 1,200G. That's 1.2 terabytes of storage without changing media!

Logical Unit Numbers. When more than one SCSI device is found in a single unit, each one must be assigned its own ID. For example, a tape autochanger (one more modest than the Exabyte unit described in the preceding section) might contain a single 4mm DAT drive and slots for 12 tapes. The drive itself requires a SCSI ID, of course, but the mechanism that changes the tapes requires an ID, also. This may be accomplished in one of two ways. Normal SCSI IDs may be assigned to the two devices, utilizing separate selector switches of the usual types. Some units might preassign a specific SCSI ID to one of the devices and so have only one selector. Other units might utilize a technique by which a single SCSI ID is shared by both devices. In these cases, a single selector allows for ID selection, and the chosen ID is then broken into two or more logical unit numbers (LUNs). LUNs are simply a second level of unit addressing within the SCSI architecture. Every command sequence sent to a specific SCSI ID also contains an LUN specification. Up to eight LUNs, numbered 0 through 7, can be defined for each SCSI ID on the bus. A unit that contains a single device uses LUN 0, while units with multiple devices have LUNs preassigned to each device in the unit, ensuring that the correct commands always reach the proper device.

The use of LUNs, however, does not necessarily expand the overall device capacity of the SCSI bus. Depending on the drivers and other software used to address the devices, attachment of a device using multiple LUNs may limit the number of other devices that can be addressed. For example, a backup software package might be able to address a tape autochanger device that uses two LUNs, but it very likely will not be able to support seven such devices on the same SCSI bus. A maximum of seven total devices usually is imposed whether or not LUNs are used. In this case, up to three autochangers using two LUNs each, plus one other single device, are all that are addressable by the particular software package. This is a limitation imposed solely by the software. The SCSI standard calls for the inclusion of LUNs in all command language addressing, even when each device on the bus uses only a single LUN.

Compatibility. SCSI device compatibility has been a sore point ever since the early days of the SCSI interface. There was a time when many administrators routinely installed separate SCSI adapters for each of their devices, rather than even attempt to get all of them to function on one bus. Fortunately, this situation has changed considerably, but there are still circumstances when it is not advisable to mix devices of different types on the same SCSI bus.

If, for example, I was administering a network in a 24-hour shop and had to back up my servers while a full complement of users were logged on, I would avoid having the tape backup drive on the same bus as my hard drives. The data stream to the tape drive would undoubtedly flood the bus and slow down hard drive access significantly. In a case like this, a separate host adapter (or better yet, a dedicated backup server) is recommended. The same is true when running SCSI hard drives in a duplexed configuration. Attaching other devices to one of the host adapters could lessen the efficiency of the drive mirroring process. When backups are performed during off-hours, however, there should be no problem running a tape drive on the same bus as hard drives. Indeed, the data transfers from hard disk to tape on the same SCSI bus would be performed wholly within the confines of the bus, yielding the highest possible throughput that the devices are capable of.

SCSI Host Adapters. We finally come to the "brains" of the outfit, the SCSI host adapter. The host adapter is the communications nexus for the entire SCSI bus and provides the gateway to the rest of the computer through the expansion bus. Requests for access to a particular SCSI resource are sent to the host adapter, which prioritizes them, sends them off to the appropriate device, and then manages the responses, sending them back out through the expansion bus. Intelligent host adapters are able to manage incoming requests to maximize efficiency. While older SCSI-1 designs could only process one request at a time, remaining idle until the target device returned a result, today's adapters can send a command to a particular device, and then go about other business until the device signals its completion of the command. This is called SCSI disconnection and is especially useful when devices other than hard drives are on the bus.

As an example, consider that an adapter may send a command to a tape drive instructing it to rewind the tape. Depending on the type of drive, the rewind process could take from several seconds to several minutes. Rather than sit and wait for the acknowledgment from the tape drive that rewinding has been completed, the adapter disconnects from that device and is free to process other tasks. The tape drive later signals its desire to reestablish communications with the host adapter, at which time it reports its successful completion of the rewind command, and receives its next instruction. A host adapter can keep track of these communications for all the devices on the bus at once, allowing for concurrent I/O operations to occur. This is the primary advantage of SCSI in the server environment.

SCSI host adapters are available for all the bus types discussed earlier in this chapter. They range from simple ISA cards running with 5 M/sec transfer rates all the way to high-end Fast & Wide, bus mastering DMA controllers with on-board RISC processors supporting synchronous burst transfer rates of up to 20 M/sec. Most contain a built-in BIOS that allows the computer to be booted from a SCSI hard drive, and some also have optional floppy drive support, making them fully functional I/O cards. Twin channel controllers are also available; these provide host adapter functions for two completely separate SCSI buses on a single expansion card.

Many computers, such as servers available from Compaq and Hewlett Packard, have the SCSI host adapter integrated into the motherboard design. This can be very beneficial, as any components built into the motherboard cut down on signal noise and communications overhead, but be sure to determine the exact capabilities of these integrated adapters before purchasing them. Hewlett Packard, for example, uses Adaptec chipsets in most HP servers, utilizing drivers originally written by Adaptec for their AHA-2740 EISA adapter, which is very good indeed. Compaq utilizes its own proprietary drivers, but their embedded 710 EISA adapter (which is also available as a separate card) is another very good choice. Other vendors providing integrated SCSI on their motherboards, however, may opt for a less expensive, less intelligent adapter circuitry that would be better off in a workstation than a file server, so be sure to ask what chipset and system bus width is used in their designs, as well as whose drivers they use. As always, if they don't know, hang up!

When purchasing a SCSI host adapter on an expansion card, be sure that you purchase a card that can be fully supported by the expansion bus of your motherboard. For example, high-performance bus mastering cards are available for EISA, VLB, and PCI systems, but for all three of these bus types, a particular slot may or may not be capable of bus mastering. Be sure to purchase a card that will function properly in the slot you have available. If you are building a monstrous, high-performance server with multiple SCSI buses, make sure that you know how many bus mastering slots there are on your motherboard.

Host Adapter Configuration. SCSI host adapters can have a great number of configurable attributes, and several different ways of setting them. Older or lower-end cards might still use DIP switches or jumper blocks, but most of the better cards on the market have some sort of software configuration utility. This might be provided on a floppy disk or located in the adapter's ROM along with the BIOS. In any case, though, termination, SCSI ID selection, and the ability to enable or disable the card's BIOS are often provided through DIP switches as well, and these might need to be set before the card is actually installed. In some cases--as with most Adaptec cards--the BIOS needs to be enabled to access the software configuration program. Once the card has been configured, the BIOS can be disabled again if you will not be booting from a SCSI hard drive.

The added convenience of these software-configured cards should not be underestimated, especially when used with file servers. The ability to make adjustments on the fly to the basic expansion hardware configuration items--such as IRQ, memory address, and DMA channel--can save a great deal of work when you are trying to put together a heavily-loaded server with several such devices. Of course, the more advanced self- configuring bus types, such as EISA, MCA, and PCI eliminate the need for such adjustments, and their growing popularity has simplified this task considerably.

Most of the configuration options discussed earlier in the section on SCSI hard drives are also present on the host adapter. As explained earlier, it is necessary for these options to be configured in the same way on both devices for them to function properly. Settings for parity, delayed start, synchronous negotiation, and terminator power should all be present on most adapters. As is the case with hard drives, nearly all the host adapters on the market today support Fast SCSI, and most allow you to disable the synchronous transfers if you desire (usually as a troubleshooting measure). Most of the time, however, a well designed host adapter is configured to function to the fullest of its capabilities right out of the box. Default configurations almost always include SCSI ID 7, termination enabled, and the other parameters set for the best possible transfer speed. Changes need to be made only to disable termination when both internal and external SCSI devices are being used or to troubleshoot problems that occur with specific devices on the bus.

SCSI Drivers. It becomes necessary here to deviate from the hardware concentration of this chapter to discuss the software that a SCSI host adapter uses to communicate with the operating system of the computer in which it resides. While the SCSI bus is a self-contained unit completely capable of communicating among its own devices without software support, it is useless unless it is able to communicate with the rest of the computer. The host adapter is essentially a bilingual component of the system that "translates" the requests made by the operating system into its native command language and relays them to the integrated controller of the target device. (This controller then communicates with the device itself using still another language, but that doesn't concern us here.)

The NOS can receive requests for access to a SCSI device from three sources: applications running on client workstations throughout the network, applications running on the file server itself, or another server on the network. When a SCSI bus is populated only with hard drives, these procedures are relatively simple, as all the requests are fundamentally of the same format. A single driver, running on the file server, can control all the hard drives on the bus. When different types of devices are used on a single SCSI bus, however, things become more complicated, as each device type must have its own driver to facilitate the unique requests being furnished to that device.

Therefore, a backup software application may generate requests that are intended for execution by a tape drive, but the driver furnished by the backup software manufacturer communicates with the host adapter, not with the tape drive directly. Thus, the host adapter may have to contend with separate drivers for hard drives, CD-ROM drives, tape drives, and other devices all at once. Based upon the underlying technology for which the SCSI standard was designed, this is virtually impossible. A SCSI host adapter occupies one IRQ, one memory address, and perhaps one DMA channel. Once a single driver has established communications with the adapter, then subsequent drivers, attempting to address the same adapter, will find those resources already occupied and will fail to load.

ASPI. It was necessary, therefore, to create a software-based interface that could ac-commodate these multiple drivers and allow them unified access to a single SCSI host adapter. Actually, several such interfaces exist, but the de facto standard among them is known as the Advanced SCSI Programming Interface (ASPI). The "A" in ASPI at one time stood for Adaptec, the company that developed it, but Adaptec has since acknowledged this standard's role as an industry standard by opening the specification to outside developers and changing the name.

Most SCSI host adapters today are ASPI-compatible, meaning that they ship with basic board drivers that provide hard drive access and establish an ASPI layer in memory. This layer consists of API calls, public symbols, or hooks to which other drivers can attach themselves. ASPI is a protocol that is independent of the computer's host operating system. Host adapters nearly always ship with ASPI drivers for DOS and NetWare, and might support many other operating systems as well. Some of the more advanced operating systems such as OS/2 include native ASPI support and require no additional drivers. While many host adapters are capable of running hard drives by using only the card's BIOS, it is necessary to load the software drivers to provide ASPI support.

When an ASPI layer is established in a NOS's memory, this usually is done by spawning an additional process. On a NetWare file server, loading the disk driver for an Adaptec host adapter causes an additional NLM named ASPITRAN.NLM to be loaded. At this point, public symbols--API calls or "hooks"--are made available for the use of other drivers.

> CAUTION: It's important to note that the file name ASPITRAN.NLM is also used by some other host adapter manufacturers. Care should be taken to use the module furnished with the disk driver that you will be running, as they are not interchangeable.

The developer of a software package that requires access to a SCSI device (other than a hard drive) then furnishes a driver that addresses neither the SCSI device nor even the host adapter hardware but the ASPI layer hooks that have been provided. In this manner, all the different devices on the bus can be serviced by their respective software packages without conflict.

One of the important factors that must be considered when running multiple SCSI buses on a single NetWare file server is that there can be no more than one ASPI layer in memory on a single machine, and that the single ASPI layer can only address the devices of one SCSI bus. It might be necessary, then, to adjust the distribution of your SCSI devices between the buses to account for this limitation. Most software packages designed for use with SCSI devices have options that allow for a choice between ASPI operation or the use of a dedicated driver that addresses a host adapter directly and exclusively. Depending on the needs of your network, it might be necessary to group all hard drives onto one SCSI bus and reserve the use of ASPI for a second bus containing all the other device types. Or, to present another scenario, if you need several devices of the same type, such as multiple tape drives for a dedicated backup server, it might be better to group all those on a single bus, and use the dedicated board driver included with the backup software package--ASPI could then be used for the hard drives and other devices on a second host adapter.

Other multiple device driver interfaces do exist, but none has the wide-ranging hardware and software support that ASPI does. It is strongly recommended that any software you purchase for use with a SCSI device contains ASPI driver support, and that the SCSI host adapter you select includes drivers that can establish and support an ASPI layer in your NOS.

One Final Note. As a final aside on the subject of SCSI board drivers, be aware of the fact that hardware innovations very often outpace the progress of the driver support furnished with them. Particularly in the case of adapters for high-performance system buses such as EISA, VLB, and especially PCI, I have found that nearly all manufacturers go through several revisions of their drivers before they are adequately debugged. A great many SCSI performance and installation problems are ultimately attributable to drivers. It is strongly recommended that you obtain the latest drivers for a new model host adapter before even attempting to install it. Nearly all manufacturers are aware of these problems and offer frequent driver revisions over a variety of online services. BBSs, CompuServe forums, and the World Wide Web can be invaluable resources in this respect, offering not only software updates but also extensive troubleshooting, background, and pre-sales information.

## Network Interface Cards

Despite the presence of a NOS and the enormous resources that may be present in a PC, it is still an isolated machine with one or more NICs to connect it to the rest of the network. While we considered, in previous sections, the relative transfer rates of system buses, SCSI buses, and hard drives, the speed of the network itself must also be factored into the equation. The NIC provides the link with the network medium and can be another possible location of a bottleneck in the data path from server to workstation. It is important that the NIC provide sufficient bandwidth for data to be fed to the network at the highest speed it can accommodate. This usually means that a server's NICs should utilize at the very least an EISA bus connection, but it is also possible to purchase too much of a card--that is, one whose capabilities will be underused due to limitations of the network medium or the storage subsystem feeding it.

As with SCSI adapters, NICs are available now for all the latest bus types; for file server use, these should definitely be considered. However, any NIC that is suitable for workstation use can also be installed into a file server, and likewise, there are situations, such as high-color graphics and CAD work, in which even the highest-performance PCI net- work card may be used in a workstation. The sole exception to this is in the case of multichanneled NICs that are specifically designed for use in a file server that also functions as a router connecting two network segments. These cards, essentially two adapters in one, allow a file server access to multiple network segments without the difficulties of configuring server resources individually for each adapter.

Otherwise, the hardware for the two platforms is interchangeable, with software drivers providing support for the different operating systems. For this reason, NICs are considered in detail in chapter 6, "The Workstation Platform." Any observations made there are equally viable when considering devices for use in a file server, except where expressly noted. The selection of NICs for use in a file server is also a matter that must be considered in coordination with the overall design of the network. For this reason, chapter 7, "Major Network Types," should also be consulted before purchasing decisions are made.

## The Case and the Power Supply

This chapter has covered all the primary components of a basic file server configuration. All that is left is the power supply that feeds it all and a box to put it in. Many components of today's file servers utilize far more power than older technologies. Many large capacity hard drives designed for server use now have platters that rotate at speeds well in excess of the traditional 3,500rpm. The newer processors and expansion buses also require more power than their predecessors and generate more heat. The primary concerns in this respect--when building a computer to be used as a file server or adding additional components to an existing PC--are whether there is an adequate supply of power for all the contents and whether the system is being sufficiently cooled.

Most servers today use large tower cases because they have ample room for numerous expansion slots and drive bays. The temptation with these cases is to fill all those slots and bays with internally mounted devices--this can be done, as long as a little thought is put into the process. If the computer has been purchased in a server configuration, then the manufacturer should have anticipated this heavy usage and installed a power supply providing at least 250-300 watts or more. When purchasing a tower case for the construction of a server, you should check on the capabilities of the power supply if it is furnished as part of the package. Some power supplies also include surge protection and other power management features, which are an added plus.

Adequate cooling of the system is essential, and a well-designed PC is organized so that its fans draw air through the system in a manner intended to cool the components that generate the most heat. Many of today's better systems use two or more fans to ensure that this is done properly. This is why computers should always be run with their cases on. An open computer case dissipates the airflow provided by the fans, cooling only the area around the power supply, and leaving other vital components to be cooled by room air. This can shorten the life of the system and even cause complete failures! When adding components to an existing machine, it is wise to space them out as much as possible to allow space for air to circulate around them. For the same reason, it is advisable to stagger your populated expansion slots, if possible, leaving empty slots between filled ones.

486 and Pentium processors are well-known sources of additional heat. Most systems ship with a heat sink or fan (or both) already attached to the top of the processor chip. If you are assembling the machine yourself, aftermarket kits are available that allow the addition of either. A heat sink, to be effectively installed, requires the use of a special thermal conductive adhesive to attach it to the processor.

As an alternative to placing all the computer's components into one case, external housings for drive arrays are becoming increasingly popular, and since they contain their own dedicated power and cooling systems, both of the above problems are significantly reduced. If you plan on using four or more hard drives in a single server, for example, an arrangement such as this is preferable to cramming all those drives into even the largest tower case. Additional functionality can also be provided by these housings. Storage Dimensions makes drive arrays with a built-in SCSI backplane and power supply, allowing drives to be "hot-plugged" in and out of the system while it is running. When used in a RAID configuration, this could mean that a hard drive that has failed can be replaced with no downtime and no loss of data. In addition, these arrays can support tape drive and CD-ROM modules in the same housing, in any combination desired. This is a high-end solution, adding significant additional expense to your system, but simple external cases for the mounting of SCSI devices are also readily available.

## Pre-Built or "Roll-Your-Own"

When considering all the detailed characteristics of the various components in a PC being used as a file server, the question of whether it is preferable to purchase a server intact or assemble one yourself, or "roll-your-own," naturally arises. As I've said throughout this chapter, the file server is not a good place to experiment, and my general advice is not to do anything you haven't done before. The actual assembly of the machine's components is not the difficult part--virtually anyone could take an existing PC as a model and, given all the basic units, assemble a working computer. The difficulty is in selecting components that will all work together. This applies not just to cutting edge technology but also to mundane considerations like "Does the motherboard fit in the case?" and "Do the screw holes line up?" Trivial items like these often stand in the way of a successful job.

Assembling a server yourself can ensure that you have exactly the features you desire and can save you a great deal of money, but you essentially are left without a safety net. If you have a substantial MIS infrastructure to rely on, including uses for non-returnable parts that don't fit and in-house expert help, this can be the way to go, but large corporations are likely to already have a mechanism in place for supplying new servers, making this procedure either extremely easy or out of the question (except perhaps as a training exercise).

There are still several alternatives, however, when you decide to purchase a pre-built server, depending on what technical support will be needed after the purchase, and the extent of your budget. The cheapest way to go this route is direct purchase, through mail order. There are several gigantic, direct market PC vendors that can provide pre- configured systems at very good prices and customized ones for a little more. With these vendors, "customized" generally means that you select from a fairly limited range of components, unless you are planning a very large order (500 systems or more). If you are looking for a specific make of motherboard, for example, you are probably out of luck. Smaller firms might be more willing to work with you in putting together a server to your exact specifications, but their prices will generally be higher, and you risk the company dissolving before your warranty expires.

Another alternative is to find a consulting firm in your area that will build custom servers for you. This will cost more (perhaps a lot more), but you will be able to specify exactly what you want and will have someone nearby to provide support whenever it's needed. The trick when dealing with consultants is the same as when dealing with any salesman, which is to do your homework beforehand. If you approach them with a sheepish smile and say, "I'm not a technical guy, but..." then you're essentially hanging a sign around your neck that says "SUCKER" in blinking neon letters. If you go in with a list of exactly what you want, and can talk intelligently about alternatives, then you can probably wheedle a consultant down to a reasonably fair price. As with any major purchase, you generally get what you pay for, and you might end up having to pay a good deal more to have things exactly the way that you want them. If, however, you have ever been in the position where a primary server goes down and workers are sitting around drinking coffee while you wait for a $150/hour technician to arrive, you probably will consider some added upfront expense to be worthwhile.

## Summary

We have, in this chapter, examined many of the hardware components that are found in a typical network server. We have discussed the ways that a server differs from and is similar to an ordinary PC. It is due to these similarities that not all of the major server components are dealt with in this chapter. The line of Intel microprocessors used in the vast majority of network servers are the same as those used in most network workstations. To avoid repetition, these processors are examined in chapter 6, "The Workstation Platform."

For the same reason, the sections in this chapter on the various types of server memory and the procedures for upgrading them are equally applicable to workstations. It remains to be seen whether or not any of the new memory technologies discussed here will fall into common use in servers or workstations, but they serve to illustrate some of the ways that the industry is trying to overcome the limitations of current memory chip designs.

SCSI is still the dominant mass storage subsystem in the server world, but it too is frequently being used in workstations. As covered in chapter 6, new enhanced IDE technologies have taken a firm hold on the workstation platform, but these may also come to be popular in the lower-end servers of the near future.

Server technology is moving forward at an amazing pace, as is the rest of the computer industry, and the growing popularity of advanced concepts like symmetric multiprocessing all but ensures that servers will continue to increase in speed and capability. One of the most difficult aspects of network administration is achieving an appropriate medium between tried and true technology and the cutting edge. The only way to do this is to remain informed of new developments and yet to remain skeptical of them until they are definitely proven.

© Copyright, Macmillan Computer Publishing. All rights reserved.