Novell NetWare

NetWare is a network operating system developed by Novell, Inc. It initially used cooperative multitasking to run various services on a PC, and the network protocols were based on the archetypal Xerox XNS stack.

NetWare established the dominant position in the market in the early and middle 1990s by developing its XNS-derived IPX/SPX protocol as the local area network (LAN) standard. At the end of the 1990s, with Internet connectivity booming, the Internet's TCP/IP protocol became dominant on LANs also. Novell however took several years to adapt NetWare to operate natively over TCP/IP, opening a window for Microsoft to assert its market position and make Windows NT the dominant LAN network operating system.

NetWare evolved from a very simple concept: one or more dedicated servers were connected to the network, and disk space was shared in the form of volumes. Clients running MS-DOS would run a special Terminate and Stay Resident (TSR) program that allowed them to map a volume as if it were a local hard disk. Clients had to log in to a server in order to be allowed to map volumes, and access could be restricted according to the login name. Similarly, they could connect to shared printers on the dedicated server, and print as if the printer was connected locally.

While early NetWare systems did entirely trust all modules (any misbehaving module could bring the whole system down), it was very stable. There are reports of Netware servers running for years without any human intervention.

History

Early years

Netware was based on the consulting work by SuperSet Software, a group founded by the friends Drew Major, Dale Neibauer, Kyle Powell and later Mark Hurst. This work was based on their classwork at Brigham Young University in Provo, Utah, starting in October 1981.

In 1983 Raymond Noorda engaged the work by the SuperSet team. The team was originally assigned to create a CP/M disk sharing system to help network the CP/M hardware that Novell was selling at the time. The team was privately convinced that CP/M was a doomed platform and instead came up with a successful file sharing system for the newly introduced IBM-compatible PC. They also wrote an application called Snipes, a text-mode game and used it to test the new network and demonstrate its capabilities. Snipes was the first network application ever written for a commercial personal computer, and it is recognized as one of the precursors of many popular multi-player games such as Doom and Quake. ^*

This Network operating system (NOS) was later called Novell NetWare. NetWare was based on the NetWare Core Protocol (NCP), which is a packet-based protocol that enables a client to send requests to and receive replies from a NetWare server. Initially, NCP was directly tied to the IPX/SPX protocol, which meant that natively, NetWare could only communicate using IPX/SPX.

The first product to bear the NetWare name was released in 1983. It was called Netware 68 (aka S-Net); it ran on the Motorola 68000 processor and used a Star network topology. It was replaced in 1985 with NetWare 86 version 1.5, which was written for the Intel 8086. After the Intel 80286 processor came out, Novell released NetWare 286 in 1986. The same happened with the release of the Intel 80386; Novell released NetWare 386 in 1989. Later, Novell consolidated the numbering of their NetWare releases, with NetWare 286 becoming NetWare 2.x, and NetWare 386 becoming NetWare 3.x.

NetWare version 2 was notoriously difficult to configure; any change required a re-linking the kernel and a reboot of the system, requiring at least 20 diskette swaps. NetWare was administered using text-based utilities such as SYSCON. The file system used by NetWare 2 was NetWare File System 286, or NWFS 286, supporting volumes of up to 256 MB.

NetWare 2 already implemented a number of features inspired by mainframe and minicomputer systems that were not available in other operating systems of the day. The System Fault Tolerance (SFT) features included standard read-after-write verification (SFT-I) with on-the-fly bad block re-mapping (at the time, disks did not have that feature built in) and software RAID1 (disk mirroring, SFT-II). The Transaction Tracking System (TTS) optionally protected files against incomplete updates. For single files, this required only a file attribute to be set. Transactions over multiple files and controlled roll-backs were possible by programming to the TTS API.

NetWare 3.x

NetWare version 3 eased development and administration by modularization. Each functionality was controlled by a software module called a NetWare Loadable Module (NLM) loaded either at startup or when it was needed. It was then possible to add functionality such as anti-virus software, backup software, database and web servers, long name support (standard filenames were limited to 8 characters plus a three letter extension, matching MS-DOS) or Macintosh style files. NetWare continued to be administered using text-based utilities. The file system introduced by Netware 3.x and used by default until Netware 5.x was Netware File System 386, or NWFS 386, which significantly extended volume capacity (1 TB, 4 GB files) and could handle volume segments spanning multiple physical disk drives.

Initially, NetWare used Bindery services for authentication. This was a stand-alone database system where all user access and security data resided individually on each server. When an infrastructure contained more than one server, users had to log-in to each of them individually, and each server had to be configured with the list of all allowed users.

For a while, Novell also marketed an OEM version of NetWare 3, called Portable NetWare, together with OEMs such as Hewlett-Packard, DEC and Data General, who ported Novell source code to run on top of their Unix operating systems. Portable NetWare met only with marginal success.

In the 3.x era, Novell introduced its first high-availability clustering system, named NetWare SFT-III, which allowed a logical server to be completely mirrored to a separate physical machine. Implemented as a shared-nothing cluster, under SFT-III the OS was logically split into an interrupt-driven I/O engine and the event-driven OS core. The I/O engines serialized their interrupts (disk, network etc.) into a combined event stream that was fed to two identical copies of the system engine through a fast (typically 100 Mbit/s) inter-server link. Because of its non-preemptive nature, the OS core, stripped of non-deterministic I/O, behaves deterministically, like a large finite state machine. The outputs of the two system engines were compared to ensure proper operation, and two copies fed back to the I/O engines. Using the existing SFT-II software RAID funcionality present in the core, disks could be mirrored between the two machines without special hardware. The two machines could be separated as far as the server-to-server link would permit. In case of a server or disk failure, the surviving server could take over client sessions transparently after a short pause since it had full state information and did not, for example, have to re-mount the volumes - a process that NetWare was notoriously slow at. Incidentially, SFT-III was the first NetWare version able to make use of SMP hardware - the I/O engine could optionally be run on its own CPU. NetWare SFT-III, being ahead of its time in several ways, was a mixed success.

NetWare 4.x

Version 4 introduced the Novell Directory Services (NDS) which replaced the bindery with a global directory service where the infrastructure was described and managed in a single place. This allowed a single user authentication to NDS to govern access to any server in the directory tree structure. Users could therefore access network resources no matter on which server they resided, although user license counts were still tied to individual servers. (Large enterprises could opt for a license model giving them essentially unlimited per-server users if they let Novell audit their total user count.)

Version 4 also introduced a number of useful tools and features, such as transparent compression at file system level and RSA public/private encryption.

NetWare released version 4.11, also called intraNetWare, which included many enhancements that made the operating system easier to install, easier to operate, faster, and more stable. It also included the first fully 32-bit client for Microsoft Windows-based workstations, SMP support and the NetWare Administrator (NWADMIN or NWADMN32), a GUI-based administration tool for NetWare.

By this time, NetWare was still tied to IPX/SPX by the NCP reliance on that protocol, but Novell started to acknowledge the demand for TCP/IP with NetWare 4.11 by including tools and utilities that made it easier to create intranets and link networks to the Internet. Novell bundled handy tools, such as the IPX/IP gateway, to ease the connection between IPX workstations and IP networks. It also began integrating Internet technologies and support through features such as a natively hosted Web server.

During this time Novell also realized the importance of leveraging its directory service, NDS, by tying their other products into the directory. Their e-mail system, GroupWise, was integrated with NDS, and Novell released many other directory-enabled products such as ZENworks and BorderManager.

NetWare 5.x

With the release of NetWare 5 in October 1998, Novell finally acknowledged the prominence of the Internet by switching its primary NCP interface from the IPX/SPX network protocol to TCP/IP. IPX/SPX was still supported, but the emphasis shifted to TCP/IP. Other new features were:

Novell Storage Services (NSS), a new file system to replace the traditional NetWare file system
Java virtual machine for NetWare
Novell Distributed Print Services (NDPS)
ConsoleOne, a new Java-based GUI administration console
directory-enabled Public key infrastructure services (PKIS)
directory-enabled DNS and DHCP servers
support for Storage Area Networks (SANs) and clustered servers
5-user version of Oracle 8i

NetWare 5 was released during a time when NetWare market share dropped, as it was massively being replaced by Windows NT servers. Novell also released their last upgrade to the NetWare 4 operating system with the release of NetWare 4.2.

NetWare 5.1 was released in January 2000, shortly after its predecessor. It introduced a number of useful tools, such as:

IBM WebSphere Application Server
NetWare Management Portal (later renamed Novell Remote Manager), web-based management of the operating system
FTP, NNTP and streaming media servers
NetWare Web Search Server
WebDAV support

NetWare 6.x

NetWare 6 was released in October 2001.NetWare 6 has a simplified licensing scheme based on users, not servers. This reduces licensing costs and allows unlimited connections per user.

Other changes, new features and improvements included:

enhanced SMP support - up to 32 processors per server
iFolder - location- and platform-independent access to local files by automatic intelligent synchronization of the local iFolder directory with the iFolder server
NetStorage - access to personal files through a standard web browser
iPrint - ability to install printers from a web browser and submit print jobs over the Internet through the standard IPP protocol
iManager - web-based administration for NetWare and other Novell products
the Apache web server and the Tomcat servlet container
Native File Access Protocols - support for the SMB, AFP and NFS protocols to provide Windows, Macintosh and Unix/Linux clients with access to files on a NetWare server without a Novell client

NetWare 6.5 was released in August 2003. Some of the new features in this version were:

more open-source products such as PHP, MySQL and OpenSSH
a port of the Bash shell and a lot of traditional Unix utilities such as wget, grep, awk and sed to provide additional capabilities for scripting
iSCSI support (both target and initiator)
Virtual Office - an "out of the box" web portal for end users providing access to e-mail, personal file storage, company address book, etc.
Domain controller functionality
Universal password
DirXML Starter Pack - synchronization of user accounts with another eDirectory tree, a Windows NT domain or Active Directory.
exteNd Application Server - a J2EE 1.3-compatible application server
support for customized printer driver profiles and printer usage auditing
NX bit support
support for USB storage devices
support for encrypted volumes

Novell Open Enterprise Server

In 2003, Novell announced a port of NetWare applications to the Linux platform in a dual-kernel product called Novell Open Enterprise Server (OES). The first version was released in March 2005. Also, consequent to Novell's acquisitions of Ximian and SuSE, a German Linux distributor, it is widely observed that Novell may be moving away from NetWare and shifting its focus towards Linux. Officially though, Novell denies these claims and says it will focus on both NetWare and Linux.

Performance

NetWare dominated the network operating system (NOS) market from the mid-80s through the mid- to late-90s due to its extremely high performance relative to other NOS technologies. Most benchmarks during this period demonstrated a 5:1 to 10:1 performance advantage over products from Microsoft, Banyan, and others. One benchmark that was especially interesting compared NetWare 3.x running NFS services over TCP/IP (not NetWare's native IPX protocol) to an expensive, dedicated Auspex NFS server and a SCO Unix server running NFS service. NetWare NFS outperformed both 'native' NFS systems and clocked in with a 2:1 performance advantage over SCO Unix NFS on the same hardware!

There were several reasons for NetWare's performance.

File service instead of disk service

At the time NetWare was first developed, nearly all LAN storage was based on the disk server model. This meant that if a client computer wanted to read a particular block from a particular file it would have to issue the following requests across the relatively slow LAN:

Read first block of directory
Continue reading subsequent directory blocks until the directory block containing the information on the desired file was found, could be many directory blocks
Read through multiple file entry blocks until the block containing the location of the desired file block was found, could be many directory blocks
Read the desired data block

NetWare, since it was based on a file service model interacted with the client at the file API level:

Send file open request (if this hadn't already been done)
Send a request for the desired data from the file

All of the work of searching the directory to figure out where the desired data was physically located on the disk was performed at high speed locally on the server. By the mid-1980s, most NOS products had shifted from the disk service to the file service model. Today, the disk service model is making a comeback, see SAN.

Aggressive caching

From the start, NetWare was designed to be used on servers with copious amounts of RAM. The entire file allocation table (FAT) was read into RAM when a volume was mounted, thereby requiring a minimum amount of RAM proportional to online disk space; adding a disk to a server would often require a RAM upgrade as well. Unlike most competing network operating systems prior to Windows NT, NetWare automatically used all otherwise unused RAM for caching active files, employing delayed write-backs to facilitate re-ordering of disk requests (elevator seeks). An unexpected shutdown could therefore corrupt data, practically making an uninterruptible power supply a mandatory part of a server installation.

Efficiency of NetWare Core Protocol (NCP)

Most network protocols in use at the time NetWare was developed didn't trust the network to deliver messages. A typical client file read would work something like this:

Client sends read request to server
Server acknowledges request
Client acknowledges acknowledgement
Server sends requested data to client
Client acknowledges data
Server acknowledges acknowledgement

In contrast, NCP was based on the idea that networks worked perfectly most of the time, so the reply to a request served as the acknowledgement. Here is an example of a client read request using this model:

Client sends read request to server
Server sends requested data to client

All requests contained a sequence number, so if the client didn't receive a response within an appropriate amount of time it would re-send the request with the same sequence number. If the server had already processed the request it would resend the cached response, if it had not yet had time to process the request it would send a 'positive acknowledgement' which meant, "I received your request but I haven't gotten to it yet so don't bug me". The bottom line to this 'trust the network' approach was a 2/3 reduction in network traffic and the associated latency.

Non-preemptive OS designed for network services

One of the raging debates of the 90s was whether it was more appropriate for network file service to be performed by a software layer running on top of a general purpose operating system, or by a special purpose operating system. NetWare was a special purpose operating system, not a timesharing OS. It was written from the ground up as a platform for client/server processing services. Initially it focused on file and print services, but later demonstrated its flexibility by running database, email, web and other services as well. It also performed efficiently as a router, supporting IPX, TCP/IP, and Appletalk, though it never offered the flexibility of a 'hardware' router.

In 4.x and earlier versions, NetWare did not support preemption, virtual memory, graphical user interfaces etc. Processes and services running under the NetWare OS were expected to be cooperative, that is to process a request and return control to the OS in a timely fashion. On the down side, this trust of application processes to manage themselves could lead to a misbehaving application bringing down the server.

By comparison, general purpose operating systems such as Unix or Microsoft Windows were based on an interactive, time-sharing model where competing programs would consume all available resources if not held in check by the OS. Such environments operated by preemption, memory virtualization, etc., generating significant overhead because there were never enough resources to do everything every application desired. These systems improved over time as network services shed their “application” stigma and moved deeper into the kernel of the “general purpose” OS, but they never equaled the efficiency of NetWare.

Probably the single greatest reason for Novell's success during the 80's and 90's was the efficiency of NetWare compared to general purpose operating systems. However, as microprocessors increased in power, efficiency became less and less of an issue and with the introduction of the Pentium processor, NetWare's performance advantage began to be outweighed by the complexity of managing and developing applications for the NetWare environment.

External links

Netware | Operating systems

Gourt Home::Gourt :: Novell NetWare