BSD: The First Free UNIX Family
Foundations of Computation: Part 18 of 18
The distinctions between UNIX ancestry, standards, and compatibility give us a clearer way to follow the operating-system families that emerged from UNIX and its influence. An operating system could belong to the historical UNIX source lineage, implement standardized Unix interfaces, qualify to use the UNIX trademark, or independently implement the broader Unix operating model.
BSD is one of the clearest examples of the original UNIX lineage continuing forward. The Berkeley Software Distribution grew directly from licensed UNIX source code and gradually became independent from it. That ancestry continues through operating systems still developed today, making BSD a living branch of the UNIX family rather than an independent reimplementation of the Unix model.
The BSD family did not remain a single operating system. FreeBSD, NetBSD, OpenBSD, DragonFly BSD, and other descendants separated at different points for different reasons. Each developed its own technical priorities while retaining a common ancestry. BSD code and technologies also spread into firewalls, network appliances, storage systems, and operating systems that do not necessarily carry BSD in their names.
Following BSD means looking at more than a family tree. We need to understand why its major branches separated, where they went afterward, and how technology moved both out of and into the family. That helps explain how BSD could become less visible than some other operating-system families while remaining deeply embedded in modern computing.
From Berkeley UNIX to an Independent BSD
BSD did not begin as an independent operating system. The earliest Berkeley Software Distributions were collections of programs and modifications created at the University of California, Berkeley for systems already running licensed UNIX. As Berkeley’s work expanded, BSD became a recognizable branch of UNIX, but it still contained AT&T source code. Anyone receiving a complete BSD system therefore needed the appropriate UNIX source license.
BSD has direct UNIX ancestry because it really did begin with UNIX source. The later BSD systems did not recreate Unix behavior independently from published interfaces or standards. Their lineage runs backward through Berkeley to the original UNIX family.
What changed was how much of the operating system belonged to Berkeley. The Computer Systems Research Group continued replacing, extending, and developing parts of the system until BSD contained a substantial body of code Berkeley could distribute under its own terms. Eventually, enough of the proprietary UNIX source would be replaced for the BSD lineage to continue without it.
Berkeley Builds on UNIX
Berkeley’s contributions became substantial long before BSD could be distributed independently. BSD introduced or advanced technologies including virtual memory, job control, the Fast File System, and networking capabilities that would influence systems far outside Berkeley.
Networking is especially important to where we are in the Foundations of Computation series. Berkeley received funding from DARPA to integrate TCP/IP networking into BSD, and 4.2BSD included a TCP/IP stack and the sockets programming interface. BSD became an important platform for the protocols connecting previously independent computers into networks.
The sockets interface gave programs a practical way to use those networks. An application did not need to manipulate network hardware directly. It could create a socket, establish or accept a connection, and send or receive data through an interface provided by the operating system. Sockets spread well beyond BSD and became one of Berkeley’s most enduring contributions to networked computing.
BSD was developing an identity of its own while remaining part of the UNIX source family. Berkeley was not simply packaging AT&T’s work. It was changing the operating system, adding capabilities, and producing code that other Unix vendors and researchers wanted.
The remaining AT&T code still mattered. Berkeley could distribute the portions it had written, but it could not simply release AT&T’s proprietary source along with them. For BSD to become independently distributable, those dependencies had to go.
Removing the AT&T Source
Berkeley removed those dependencies incrementally. The Computer Systems Research Group replaced code still restricted by the UNIX license while separating Berkeley-developed components that could be released more freely.
Networking Release 1, commonly called Net/1, was an important step. Instead of distributing a complete BSD operating system containing restricted UNIX source, Berkeley released networking code it could distribute under its own license. This also helped establish the permissive licensing approach that would become closely associated with BSD.
Work continued on the remaining proprietary components. In 1991, Berkeley released Networking Release 2, or Net/2. Despite the name, Net/2 contained substantially more than a networking stack. Most of the components needed for a complete BSD system were now available without the traditional AT&T UNIX source license.
Net/2 was not quite complete. Important portions of the kernel still needed replacement before it could function as a standalone operating system. Developers outside Berkeley would soon supply that work, moving BSD development beyond the institution where it began.
There was also an unresolved legal problem. Berkeley believed it had removed the restricted UNIX code necessary to make Net/2 redistributable. UNIX System Laboratories disagreed.
The BSD Lawsuit and 4.4BSD-Lite
The disagreement became a lawsuit after Berkeley Software Design, Inc., commonly known as BSDi, began selling its own Intel-based operating system derived from Berkeley’s work. UNIX System Laboratories sued BSDi in 1992, and the dispute subsequently involved the University of California as well.
The legal fight reached farther than the two companies. If Net/2 still contained protected UNIX code, every project attempting to build a freely distributable operating system from it faced uncertainty over its foundation.
The dispute was eventually settled after Novell acquired UNIX System Laboratories. A small number of files from Net/2 were identified for removal or modification, and Berkeley produced 4.4BSD-Lite as the release projects could use going forward without the same UNIX source encumbrances.
4.4BSD-Lite gave the growing BSD family a cleaner source foundation. BSD development could continue without depending on the old AT&T licensing relationship.
Removing the proprietary portions did not erase BSD’s history. The lineage remained continuous.
The lineage remained continuous even as Berkeley and later BSD developers replaced the pieces that could not be freely carried forward.
BSD Moves to Commodity Hardware
While Berkeley was separating BSD from its remaining proprietary dependencies, the hardware underneath Unix systems was changing. The Intel 80386 provided a 32-bit processor with memory protection and other capabilities suitable for a Unix-like operating system, but it belonged to the rapidly expanding world of IBM PC-compatible hardware rather than the traditional workstation and minicomputer markets.
William and Lynne Jolitz used the Net/2 source as the foundation for 386BSD, filling in missing portions and porting the system to the Intel 386. The first public release appeared in 1992.
BSD could now move beyond university systems, proprietary workstations, and expensive Unix hardware. It could run on the same increasingly ordinary PC architecture that had already transformed personal computing.
386BSD attracted a community that wanted to improve and maintain the system. Patches accumulated, hardware support needed to expand, and users wanted development to move faster. Difficulties incorporating that work back into 386BSD eventually encouraged developers to organize projects of their own.
A commercial path was developing at roughly the same time. BSDi produced BSD/386, later renamed BSD/OS, as a commercial BSD operating system for Intel hardware. BSD’s move onto commodity computers produced both commercial and freely available descendants.
By 1993, developers from the 386BSD community had begun creating NetBSD and FreeBSD. Berkeley supplied the UNIX-derived foundation, the effort to remove proprietary dependencies made independent distribution possible, and 386BSD demonstrated that the system could live on commodity hardware.
BSD’s future no longer depended on Berkeley maintaining a single operating system. It was becoming a family.
The BSD Family Splits
NetBSD and FreeBSD marked the beginning of the modern BSD family, but the branching did not stop there. Each project developed its own priorities, community, and technical direction. Later disagreements and new goals produced additional descendants, while commercial and specialized branches developed alongside the better-known free BSDs.
The major branches look like this:
Berkeley BSD
│
├── Net/2
│ │
│ ├── 386BSD
│ │ │
│ │ ├── NetBSD
│ │ │ │
│ │ │ └── OpenBSD
│ │ │
│ │ └── FreeBSD
│ │ │
│ │ └── DragonFly BSD
│ │
│ └── BSD/386
│ │
│ └── BSD/OS
│
└── continued Berkeley development
│
└── 4.4BSD
│
└── 4.4BSD-Lite
│
└── source incorporated into the
continuing BSD projects
This is a simplified ancestry tree, not a complete map of source-code inheritance. Code continued to move between projects, and the modern BSDs incorporated later Berkeley code, particularly 4.4BSD-Lite. For ancestry, the important relationships are clear: NetBSD and FreeBSD emerged from the 386BSD environment, OpenBSD later separated from NetBSD, and DragonFly BSD later separated from FreeBSD.
NetBSD: Portability Across Architectures
NetBSD began in 1993 when Chris Demetriou, Theo de Raadt, Adam Glass, and Charles Hannum organized a new project around the freely available BSD work. Problems surrounding 386BSD development helped motivate the project. Patches were circulating outside the main source tree, developers had difficulty getting changes incorporated, and maintaining a coherent system was becoming increasingly difficult.
The new project also had a broader goal than maintaining BSD specifically for the Intel PC. NetBSD emphasized clean design and portability across hardware architectures. The operating system should not depend unnecessarily on assumptions specific to one processor, bus, or machine.
Portability became NetBSD’s defining characteristic. Over time, it was ported to an unusually broad collection of computers, including x86, ARM, SPARC, VAX, Motorola 68k, MIPS, PowerPC, and many others. The project’s long-standing slogan, “Of course it runs NetBSD,” grew from that emphasis.
Portability has value beyond keeping unusual or obsolete hardware alive. Moving the same operating system across substantially different machines exposes assumptions hidden in the code. Differences in byte ordering, memory layout, processor behavior, device access, and platform architecture force the boundaries between machine-dependent and machine-independent code to remain clearly defined.
NetBSD’s answer to what BSD should become after 386BSD was a portable operating system designed to follow the hardware rather than remain tied to one platform.
FreeBSD: BSD on the Growing PC Platform
FreeBSD also emerged from the 386BSD community in 1993, but with a different early emphasis. Instead of making support for many architectures its defining objective, FreeBSD initially concentrated on producing a practical BSD system for the Intel 386 PC platform.
That was not a small target. IBM-compatible hardware was becoming increasingly capable, inexpensive, and standardized. The same commodity-hardware transition that transformed personal computing was making PCs suitable for jobs that previously required workstations and specialized Unix systems.
FreeBSD concentrated heavily on making BSD useful on that hardware: installation, hardware support, networking, performance, and system administration. When the USL settlement made the Net/2 foundation problematic, the project moved to 4.4BSD-Lite and released FreeBSD 2.0 from the new foundation.
As commodity x86 hardware became more capable, FreeBSD grew with it. The system found substantial use in Internet servers, network infrastructure, storage, and later appliances. Its BSD networking heritage, integrated base system, Ports Collection, and permissive licensing made it useful both as an operating system and as a foundation for building other systems.
That second role became important. Many people who have never deliberately installed FreeBSD have still used systems built on it.
OpenBSD: Security, Correctness, and a New Branch
The next major branch came from NetBSD. Theo de Raadt had been one of NetBSD’s founders, but disagreements within the project eventually led to his removal from the NetBSD core team. In 1995, he began OpenBSD from the NetBSD source tree; its first official release followed in 1996.
The split explains where OpenBSD came from. What happened afterward defines the project. OpenBSD developed a strong emphasis on code correctness, auditing, security, documentation, and a coherent base system with carefully chosen defaults.
Security is treated as a property of the complete system rather than something added afterward. OpenBSD developers have repeatedly audited existing code, removed unsafe practices, introduced exploit mitigations, and replaced components when they believed they could build something better.
Some of that work spread far beyond OpenBSD. OpenSSH, developed by the OpenBSD project from the earlier SSH codebase, became one of the most widely deployed implementations of the Secure Shell protocol. The PF packet filter likewise became important within OpenBSD and other BSD-derived systems.
OpenBSD’s reach is larger than the number of computers running OpenBSD itself.
Software created because the project pursued its own goals became useful throughout the wider computing ecosystem.
DragonFly BSD: A Different Approach to Multiprocessing
The BSD family branched again in 2003, this time from FreeBSD. Matthew Dillon created DragonFly BSD from FreeBSD 4.8 after disagreeing with the direction FreeBSD was taking as it redesigned portions of the operating system for symmetric multiprocessing.
Multiprocessor hardware was becoming increasingly important, and operating systems had to determine how kernel work should be divided and synchronized across processors. FreeBSD was pursuing its approach during the transition toward the FreeBSD 5 series. Dillon believed a different architecture would provide a better long-term foundation.
DragonFly BSD provided a place to pursue that alternative without requiring FreeBSD to accommodate two competing kernel designs. The project developed its own approaches to concurrency, messaging, clustering, and other parts of operating-system architecture.
It also developed its own storage technology. HAMMER, followed by HAMMER2, provided DragonFly with a filesystem designed around features including snapshots, historical data retention, checksumming, and large storage systems. DragonFly remained much smaller than FreeBSD, but its continued development showed that BSD branching did not end in the 1990s.
Forks do not always happen because a project failed. Sometimes separate systems are the practical way to explore competing architectural ideas.
BSD/OS: The Commercial BSD Branch
Not every important BSD descendant followed the free community-project model. Berkeley Software Design developed BSD/386 as a commercial Unix operating system for Intel hardware. The product was later renamed BSD/OS.
BSD/OS shared the Berkeley lineage but occupied a different part of the market from the emerging free BSD projects. It provided a commercially supported BSD system at a time when Unix vendors commonly sold the operating system, support, and related products as commercial offerings.
The company and operating system changed ownership as the Unix market consolidated, and BSD/OS was eventually discontinued. Its technology and developers still crossed back into the wider BSD ecosystem. Wind River, which had acquired BSDi, later transferred BSD/OS technology to the FreeBSD Foundation, while people associated with the commercial BSD world contributed knowledge and code elsewhere.
BSD ancestry and open-source community development are not the same thing. The lineage produced both commercial and freely developed operating systems, just as the larger UNIX family had done before it.
The Family Continues to Branch
FreeBSD, NetBSD, OpenBSD, and DragonFly BSD form the major independently developed open BSD operating systems surviving from these branches, but they do not account for every system carrying BSD ancestry. Other projects have taken one of these operating systems and adapted it for security, desktops, storage, firewalls, embedded systems, and other specialized purposes.
These later descendants are somewhat different from the major forks. OpenBSD and DragonFly BSD separated to pursue independent operating-system development. Other projects begin with an existing BSD because it already provides a useful foundation for a more specialized goal.
The BSD family can therefore grow in two ways: independent branches can emerge over the direction of the operating system itself, while derived systems can build something more specialized on an existing BSD.
There is another difference we need before following those derivatives. FreeBSD, NetBSD, OpenBSD, and DragonFly BSD are not BSD equivalents of Linux distributions. Each develops an operating system as an integrated whole.
BSD as a Complete Operating System
The modern BSD projects share ancestry, exchange code, and provide many of the same familiar Unix tools, but each develops a complete operating system. FreeBSD, NetBSD, OpenBSD, and DragonFly BSD are not different collections of software placed around a common BSD kernel. Each project maintains its own kernel together with the core userspace that makes up its base system.
This differs from the model familiar to Linux users. A Linux distribution combines the Linux kernel with software from many independently maintained projects to create a complete operating environment. GNU utilities and libraries have historically supplied many of those components, while other projects provide init systems, networking tools, package managers, graphical environments, and applications. Different distributions can combine those pieces in substantially different ways while sharing the Linux kernel lineage.
BSD development starts with the operating system as an integrated project. The kernel, commands, system libraries, startup infrastructure, administrative tools, manual pages, and other components of the base operating system can be developed together instead of being assembled afterward from unrelated upstream projects.
BSD Tip: FreeBSD, NetBSD, OpenBSD, and DragonFly BSD share ancestry, not a common modern kernel. They are separate operating systems, not BSD versions of Linux distributions.
The Base System
The base system is the portion of a BSD maintained as part of the operating system itself. Its exact contents and implementation differ between projects, but the concept provides a useful boundary between the operating system and additional software installed on top of it.
In FreeBSD, for example, the base system includes the kernel and the fundamental userspace needed to operate and administer the machine. Its source is maintained together, and the project produces coordinated operating-system releases from it. NetBSD, OpenBSD, and DragonFly BSD follow the same broad integrated-system approach while maintaining their own implementations and release processes.
A change to a BSD does not have to stop at the kernel boundary. A project can modify the kernel, a system utility, a library, its documentation, and the interfaces connecting them as parts of the same operating system. This helps explain why the individual BSDs developed such distinct identities after their forks. Their differences are not limited to kernel patches or which software gets installed. Each project can make decisions across the complete base system.
The manual pages are a good example. Documentation for the base system can evolve with the code it documents and ship as part of the same project. The documentation installed on the machine is intended to describe that particular operating system rather than a generic collection of Unix-like tools.
Packages and Ports Build on the Base
An integrated base system does not mean BSD projects try to develop every program their users might need. Web servers, databases, programming languages, desktop environments, editors, and thousands of other applications remain separate software projects.
BSD systems provide ways to install this additional software without making it part of the base operating system. FreeBSD, for example, provides binary packages and the Ports Collection, which contains the information needed to build and install third-party software from source. The other BSDs provide their own package and ports systems.
The basic division is:
- Base system — the kernel and core operating-system components maintained by the BSD project.
- Packages and ports — additional software installed on top of that operating system.
BSD Tip: When reading BSD documentation, keep the base system and installed packages separate in your head. The operating system is more than its kernel, but it also does not include every application installed on the machine.
The base system can still contain software that originated elsewhere. BSD projects can import external projects into their source trees and integrate or maintain them as part of the operating system. What matters is responsibility: once something becomes part of the base system, its integration and release are managed within the operating-system project.
Different BSDs Are Different Operating Systems
FreeBSD, NetBSD, OpenBSD, and DragonFly BSD should not be described as BSD distributions. There is no single modern “BSD kernel” underneath all four systems. Their kernels and base systems have diverged, and each project makes its own decisions about interfaces, security, hardware support, networking, filesystems, utilities, and release engineering.
Their common ancestry remains real. Code still moves between them, and similarities inherited from Berkeley remain obvious. But installing OpenBSD instead of FreeBSD means choosing a different operating system, not another packaging of the same underlying BSD.
When BSD becomes a foundation for something else, the distinction changes. A project can begin with FreeBSD and build a firewall, storage appliance, desktop system, or other specialized environment around it. Then there really is an underlying BSD with another system built from it.
This is one reason BSD’s influence is much larger than the number of computers whose users say they are running BSD. Much of the modern BSD family is hiding underneath something else.
BSD Underneath Other Systems
The BSD family becomes harder to see once we move beyond operating systems carrying BSD in their names. FreeBSD, NetBSD, OpenBSD, and DragonFly BSD are identifiable operating systems, but BSD’s permissive licensing and integrated design also make the family useful as a foundation for other projects. A developer can begin with a complete BSD system, change it for a particular purpose, and distribute the result under another identity.
These descendants are not all additional branches equivalent to OpenBSD or DragonFly BSD. Some remain closely tied to their parent BSD and make it easier to use for a particular purpose. Others substantially change the system around a specialized workload. Apple’s Darwin combines BSD technology with components from entirely different operating-system traditions.
A person can use BSD code every day without ever deliberately choosing to install a BSD operating system.
Firewalls and Network Appliances
Networking is a natural place for BSD to appear beneath another system. The family already carried Berkeley’s long networking history, while a complete BSD operating system provided the routing, packet filtering, interfaces, storage, user management, and other facilities needed to turn ordinary hardware into a network appliance.
m0n0wall was an important example. Beginning in the early 2000s, it used FreeBSD as the foundation for a small firewall system managed primarily through a web interface. The administrator did not need to treat the machine as a general-purpose FreeBSD server. FreeBSD supplied the operating system while m0n0wall turned it into an appliance built for a specific job.
That model produced descendants of its own. pfSense began as a fork of m0n0wall in 2004 and continued using FreeBSD as its operating-system foundation. It expanded the appliance model into a broader firewall and routing platform while retaining FreeBSD’s networking capabilities.
OPNsense forked from pfSense in 2014 rather than starting again from scratch. It continued along the inherited FreeBSD-derived path, later incorporated work from HardenedBSD, and eventually moved its base more directly back toward FreeBSD.
This portion of the family has a genealogy of its own:
FreeBSD
│
└── m0n0wall
│
└── pfSense
│
└── OPNsense
The tree shows project ancestry rather than every source-code relationship between them. Several generations of firewall projects could concentrate on firewall and routing functionality because BSD already supplied the operating system underneath.
Someone installs OPNsense because they want a firewall, not because they decided to deploy a general-purpose BSD server. BSD remains underneath doing the work.
Security-Focused and Desktop Derivatives
FreeBSD has also provided a foundation for projects that retain more of a general-purpose operating system while changing its emphasis.
HardenedBSD began as a FreeBSD-derived project focused on additional security and exploit-mitigation technologies. Instead of creating a new kernel and userspace independently, the project could begin with FreeBSD and concentrate on its security-oriented changes.
GhostBSD takes the idea in another direction. It builds a desktop-oriented operating system from the FreeBSD family, providing a graphical environment and installation experience intended to make BSD more approachable on the desktop.
These differ from the major BSD branches discussed earlier. NetBSD and FreeBSD emerged as independent operating-system projects. OpenBSD and DragonFly BSD forked to pursue their own continuing development of the operating system itself. HardenedBSD and GhostBSD show how an existing BSD can instead become the platform for a more specialized system.
The boundary is not absolute. A derivative can develop substantial technology of its own, and its relationship with its parent can change over time. The distinction still helps explain why the BSD family contains both a handful of major independently developed operating systems and a much larger collection of systems built from them.
BSD as a Storage Foundation
Storage is another area where FreeBSD became useful underneath a specialized system. FreeNAS used FreeBSD to create a network-attached storage platform that could be administered as an appliance instead of a manually assembled general-purpose server.
FreeNAS eventually became part of the TrueNAS product family. TrueNAS later developed both FreeBSD-based CORE and Linux-based SCALE, with new feature development eventually shifting toward the Linux-based branch. FreeBSD nevertheless played an important role in TrueNAS history as a storage platform.
One technology became particularly important to that relationship: ZFS. Unlike the Berkeley networking stack and other technologies that originated within the BSD lineage, ZFS came from another branch of the UNIX family. Sun developed it for Solaris, yet it eventually became strongly associated with FreeBSD storage systems.
This time the technology was moving into BSD rather than out of it.
Darwin and macOS
Apple provides a different example because Darwin is not simply another FreeBSD derivative. Its ancestry combines technologies from several operating-system traditions.
Part of the path runs through NeXT. NeXTSTEP combined the Mach kernel work developed at Carnegie Mellon with BSD-derived Unix facilities and NeXT’s own software. When Apple acquired NeXT in 1997, that technology became the foundation for the operating-system architecture that ultimately became Mac OS X.
At the center of Darwin is XNU, whose name originally reflected the idea that “X is Not Unix.” XNU combines Mach with BSD-derived components and Apple’s own development. Its BSD portion provides important Unix facilities including process and user models, networking, sockets, permissions, and familiar Unix interfaces.
FreeBSD also became an important source for BSD userspace and kernel components incorporated into Darwin. Apple could use BSD code without turning Darwin into FreeBSD or making macOS a BSD distribution. BSD’s permissive licensing allowed the technology to become part of a larger proprietary product while open portions continued to exist in Darwin.
BSD Tip: macOS is built partly from BSD technology, but it is not FreeBSD with an Apple interface. Mach, XNU, NeXT technology, BSD components, and Apple’s continuing development all contribute to the system.
The Darwin/XNU foundation later spread across Apple’s other operating systems. BSD-derived technology consequently reached an enormous number of computers and mobile devices whose users would never describe them as BSD machines.
Influence Without the BSD Name
Installation counts provide an incomplete picture of BSD. The family can appear in several forms:
- a directly installed BSD such as FreeBSD, NetBSD, OpenBSD, or DragonFly BSD;
- a specialized system built from a BSD, such as a firewall or desktop derivative;
- an appliance whose BSD foundation is largely hidden behind its purpose;
- or BSD code and interfaces incorporated into a larger system with a different architecture and identity.
Permissive licensing helped that movement happen. BSD code could be adopted, modified, and incorporated into open or proprietary systems without requiring those systems to become BSD projects themselves. The same characteristic that helped BSD technology spread could also make its influence less visible.
BSD did not need every system using its technology to carry the BSD name.
ZFS shows technology moving in the other direction. BSD could absorb an important technology from another UNIX branch and provide it with another place to develop and survive.
ZFS Moves into BSD
Not every important technology associated with modern BSD originated at Berkeley or within one of the BSD projects. ZFS came from a different branch of the UNIX family entirely. Sun Microsystems developed it for Solaris as a new approach to managing storage, then released its source through OpenSolaris.
ZFS eventually became deeply integrated with FreeBSD and particularly important to BSD-based storage systems. Sun had created an unusually capable storage technology that could help distinguish Solaris and OpenSolaris from competing operating systems. Once the code was available, however, the technology proved more portable than the platform that created it.
ZFS left the boundaries of Solaris, found new homes, and continued developing after OpenSolaris itself disappeared. FreeBSD became one of those homes.
Rethinking the Filesystem and the Disk
Traditional storage systems commonly separate several responsibilities. Physical disks may be combined or divided by RAID and volume-management layers, while a filesystem is created on the resulting block device. Each layer performs its own job, but the filesystem does not necessarily know what is happening beneath it.
ZFS was designed differently. It combines the filesystem with storage-pool management, giving ZFS knowledge of both the logical organization of files and the storage holding them. Disks and other devices become part of a storage pool, and filesystems draw capacity from that pool instead of being permanently tied to individually sized disk partitions.
That integration lets ZFS treat storage integrity as a system-wide problem. ZFS uses copy-on-write instead of overwriting live filesystem blocks in place. New data is written elsewhere, and the filesystem’s metadata is updated to reference it only after the write is complete.
ZFS also checksums data. When data is read, its checksum can be compared with the value recorded when it was written. If redundant copies are available through a mirror, RAID-Z, or another supported form of redundancy, ZFS can use a correct copy to repair corrupted data instead of merely reporting the corruption.
The integrated design also provides features particularly useful for servers and storage appliances:
- storage pools combine available storage instead of requiring administrators to preallocate fixed filesystem partitions;
- copy-on-write preserves previous blocks until replacement data has been safely written;
- checksums allow ZFS to detect corruption instead of assuming a successful disk read returned the correct data;
- scrubs systematically read stored data and verify its integrity;
- mirrors and RAID-Z provide redundancy ZFS can use when repairing damaged data;
- snapshots and clones preserve filesystem states efficiently through copy-on-write;
- compression can be managed directly by the filesystem;
- replication allows snapshots to be transferred between ZFS systems.
No single feature makes ZFS important. The strength comes from these capabilities working with knowledge of the same storage system instead of being assembled entirely from independent layers.
From Solaris to FreeBSD
Sun introduced ZFS through Solaris and later made its source available as part of OpenSolaris. FreeBSD developers began porting ZFS to their operating system, and FreeBSD 7.0 shipped with ZFS support in 2008.
FreeBSD was already used for servers and network infrastructure, and its integrated operating-system model gave ZFS room to become more than an unusual third-party filesystem. Over time, FreeBSD gained the ability to use ZFS throughout the system, including for boot and root filesystems.
Storage appliances made the relationship especially visible. FreeNAS originally used UFS but moved to the FreeBSD ZFS port in 2010. Its later development as TrueNAS continued making ZFS central to the storage platform.
Berkeley networking technology had moved outward from BSD and into other systems. ZFS moved inward from Solaris and became an important part of the BSD ecosystem.
ZFS Outlives OpenSolaris
ZFS initially gave Solaris and OpenSolaris a substantial storage technology of their own. Sun controlled the operating system where ZFS originated, and Solaris could integrate the filesystem as a native part of its storage architecture.
Once ZFS was released as open-source software, other operating-system communities could bring that technology to their own platforms.
FreeBSD did exactly that.
ZFS did not cause the decline of OpenSolaris. Solaris and OpenSolaris existed in a much larger environment involving hardware support, applications, administration, commercial support, Sun’s business decisions, Linux’s rapid expansion, and eventually Oracle’s acquisition of Sun. Reducing that history to a filesystem would miss most of it.
What happened to ZFS is more specific. A valuable operating-system technology does not necessarily preserve the position of the operating system that created it. If the technology can move, some of its advantage can move with it.
When Oracle discontinued OpenSolaris in 2010, the open ZFS code did not disappear. The illumos project continued development from the OpenSolaris codebase, while ZFS development increasingly occurred across several operating systems. OpenZFS eventually became the common upstream project for continued open development.
FreeBSD later moved from maintaining its older illumos-derived ZFS implementation to the common OpenZFS codebase. TrueNAS made a corresponding transition with OpenZFS 2.0, bringing FreeBSD and Linux ZFS development around a shared upstream implementation.
ZFS had outgrown the operating system that created it.
Licensing Changes the Linux Story
There is a limit to how freely ZFS could move. Sun released ZFS under the Common Development and Distribution License, or CDDL. The Linux kernel is distributed under version 2 of the GNU General Public License, and code incorporated into the kernel source needs licensing compatible with GPLv2.
The CDDL and GPLv2 are generally considered incompatible for distributing ZFS as part of the Linux kernel itself. That does not mean ZFS cannot run on Linux. OpenZFS supports Linux, and Linux systems can use ZFS through separately distributed kernel modules. The licensing issue affects how ZFS can be combined and distributed with the kernel, not whether Linux can use it.
ZFS Tip: The CDDL/GPL issue is about combining and distributing ZFS with the Linux kernel. It does not mean Linux cannot use ZFS.
FreeBSD does not have the same conflict. Its permissive licensing model allowed CDDL-licensed ZFS to be incorporated into the operating system while ZFS retained the requirements of its own license. FreeBSD could integrate ZFS deeply without requiring it to be relicensed.
This puts the software-licensing differences discussed earlier in the series into practice. Licenses do more than determine whether source code can be read. They establish the conditions under which independently developed software can be combined, modified, and redistributed.
There is an interesting tension here. Open-source licensing helped ZFS escape Solaris and survive the end of OpenSolaris as a Sun project. The particular license Sun chose also created a boundary that kept ZFS from being incorporated into the Linux kernel as straightforwardly as it could be incorporated into FreeBSD.
ZFS connects several parts of the BSD story: FreeBSD as a storage platform, technology crossing between UNIX branches, BSD as the foundation for specialized appliances, and licensing having practical technical consequences.
The technology that survives is not always attached to the platform that invented it.
Summary
BSD began as additions to licensed UNIX, but Berkeley’s developers gradually replaced the proprietary dependencies while adding important technologies of their own. Networking, sockets, the Fast File System, and other Berkeley work gave BSD an identity beyond the UNIX source where it began. Net/2, 386BSD, and ultimately 4.4BSD-Lite provided foundations from which BSD could continue after Berkeley stopped developing the operating system.
That continuation did not produce a single successor. NetBSD emphasized portability across architectures. FreeBSD initially concentrated on the growing PC platform and became important in servers, networking, storage, and appliances. OpenBSD branched from NetBSD and developed its emphasis on security and correctness, while DragonFly BSD later branched from FreeBSD to pursue a different approach to multiprocessing and operating-system architecture. BSD/OS shows that commercial development existed alongside the freely developed BSDs.
BSD’s influence extends much farther than those operating systems. FreeBSD became a foundation for firewalls, storage appliances, desktop systems, and other specialized projects, while BSD technology became part of Darwin and Apple’s larger operating-system family. Permissive licensing allowed BSD code to move into systems that did not need to retain the BSD name or the development model of the project where the code originated.
Technology moved in the other direction as well. ZFS began in Solaris, migrated into FreeBSD, and continued through OpenZFS after OpenSolaris disappeared. Its history shows both how useful technology can cross operating-system families and how licensing affects where that technology can be integrated. BSD survives as a direct branch of the original UNIX lineage, but its reach extends far beyond the systems carrying the BSD name.
More from the "Foundations of Computation" Series
- Foundations of Computation: From Mechanical Systems to Early Electronic Computers
- Interfaces, Storage, and Early System Structure
- Mainframes, Minicomputers and Microcomputers
- Cray Supercomputers
- Computing History and Shared System Design
- 8-Bit Personal Computing and Fragmentation
- IBM PC Platform and Early Standardization
- The Technologies We Carry: Why Some Computers Become Part of Our Story
- BBS to Internet: Local to Global
- Experiencing DOS Today with FreeDOS and VirtualBox
- From Isolated Computers to Local Area Networks
- The Server Emerges as a Role
- When Hardware No Longer Defines the Computer
- UNIX Becomes a Family
- Software Distribution, Freedom, and Licensing
- UNIX: From Source to Standards
- UNIX Beyond the Original Source
- BSD: The First Free UNIX Family