Designing a New OS

What is an operating system?

An operating system (OS) is a piece of software that provides control and management of a computer system. An OS will deal with and arbitrate between requests from applications programs for the computer's resources. There are two primary ways to view an operating system:

  • A controller of computer hardware
  • An environment in which programs run

Both views are valid. An operating system does control and interface with computer hardware such as disks, keyboards, timers and the like. It also makes services available to programs by providing system calls that the programs can use.

What is the difference between an operating system and a kernel?

  • A kernel is a part of an operating system, usually that part which remains active constantly and which provides the core services. A kernel may provide fundamental operations such as task scheduling and memory management. A full OS, on the other hand, may include device drivers, editors, command shells, compilers, a GUI and system management.
  • A kernel which includes device drivers and possibly other functions is called monolithic.
  • A kernel which has a minimum number of functions built in is called a micro-kernel. In this case, device drivers and other functions are still needed but they will be loaded and run as separate modules.
  • A given kernel may have a combination of features and so be some way between a micro-kernel and a monolithic one.
  • The kernel will generally run with highest privilege and restrict other programs to lesser privilege so that it can maintain control and security.
  • A common model is for the kernel to run in privileged mode and provide system calls and for any shell (command interpreter) to run non-privileged. The shell can make calls to the kernel as would any other program.

Where do I start?

Operating system development is not for the fainthearted and is often carried out by teams of people. Whether many or one person is involved it helps to have a clear idea of what your OS is to do or how it is to be distinctive. You will need to know what you want to achieve and how to deal with the challenges that face an OS. Sources that may help you understand the requirements and to formulate your vision include

  • Modern Operating Systems. Andrew Tanenbaum
  • Operating Systems. Harvey Dietel
  • Concurrent Systems. Jean Bacon

What should I make first?

Opinions vary but there are two basic approaches:

  • Start with the boot loader, write and get the basic OS loaded, take control of the machine and add the bulk of the OS from there.
  • Start with the parts that you can build and test under an existing OS, then add the boot-up and core code later.

In either case there are pre-written components that you can use to avoid having to write all the OS yourself. These include boot loaders, shells, and compilers.

What hardware should I develop for?

  • It is usually easiest to carry out development on hardware or an emulator that you have available. Emulators can make the development cycle much easier.
  • Most OS developers target machines which, architecturally, are enhancements of the original IBM Personal Computer/AT.
  • The original PC/AT used a CPU which is now obsolete, the 80286, so the base CPU to target for OS development is normally at least an Intel 80386, also referred to as an i386. This CPU, while now very old, defined the IA-32 32-bit processor architecture called x86-32. Despite the age of the design it is still used even on high-end x86 CPUs today, even though they also support many enhancements. You can write your OS for the 80386 and have it run on a modern CPU. This makes the 80386 an appropriate base but there is also value in using later CPUs as a base. For example, the 80386 did not include a time stamp counter (TSC). If you want to rely on there being a TSC you may want to target the Pentium or later.
  • An IBM Personal Computer/AT but enhanced to include the 80386 or higher CPU, is often generically referred to simply as a "PC." Due to it being a common and well-documented base, if a machine type is not explicitly stated within this website the comments generally relate to a PC. (It should be noted that the term PC was previously applied specifically to the predecessor IBM Personal Computer but the term is now used generically.)
  • Although a PC is the most popular choice there are many alternatives such as later IBM-compatibles and Arm-based systems. The key requirement for any target hardware is documentation. Ultimately, whichever machine you choose to target, you will need to be able to control its hardware and so will need access to documentation which will tell you what hardware is available and how to control it so make sure that you have access to the required documentation before you commit to any given machine.
  • Although the PC is a common target this FAQ has some information on the Nintendo DS and the Raspberry Pi.
  • At a higher level, and considering an OS as a provider of services to programs, you may want to design your OS in such a way that it can run on a variety of CPUs and hardware. That would give the OS more independence from hardware but could add significantly to the work involved.

Next step

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License