White Paper on KGI, the Kernel Graphics Interface


This draft is about the Kernel Graphics Interface developed by the KGI Project. It is intended to give a short overview of the concepts behind, being kind of a white paper. It'll take about 20min to read, if you like to know more about it or comment on it, feel free to email the maintainer of this document. You can download a snapshot of the current sample implementation from ftp://kgi.sourceforge.net/pub/kgi. A sample implementation for Linux is currently under development, the first part - a reimplementation of the Linux console system to meet the requirements of the KGI architecture - being finished already. This includes an implementation of most of the concepts explained here. Any file references given here refer to files included in the snapshot. An online version of the source tree is available at http://kgi.sourceforge.net/source/kgi-0.9


The development of a Kernel Graphics Interface (KGI) is part of the GGI Projects effort to create a portable, fast and secure General Graphics Interface (GGI). While GGI itself is more a suite of application programming interfaces, most applications will not directly interface to KGI. Instead, KGI is intended to provide the neccessary kernel support (protection, virtualization and abstraction) in order to allow applications to benefit from graphics hardware with as little overhead as possible. Additionally, a certain amount of kernel level support for display hardware can ease the effort to meet the security and stability
requirements of modern multi-user multi-tasking operating systems.

Display Hardware Characteristics and KGI Desing implications

As mentioned, KGI deals primarily with display hardware. This section should give some fundamental properties of graphics hardware and their implication on KGI design. Some of these properties make it special compared to other computer subsystems, while others are just implied by the fact that it is hardware. Most of the KGI design goals and constraints can be summarized as:
Display Hardware is essential for human-machine interaction.
This very obvious property makes display hardware different from all other computer componentes and implies the following design requirements for KGI:
  1. Display ...

  2. Display hardware must become available very early during the boot process in order to display diagnostic messages. KGI must therefore provide means to output diagnostic messages from the kernel and do this as early as possible.
  3. ... hardware ...

  4. The primary intention of advanced display hardware (other than simple frame buffers) is to offload computing or memory bandwith intensive operations to specialized (co-)processing hardware, thereby freeing the main CPU for other tasks. In some aspects, display hardware therefore acts as a co-processor that needs to be virtualized -  just like the main (co-)processing unit(s) (e.g. FPU, ...). On the other hand, display hardware is also a system resource that needs to be protected from accidental or intentional misuse. Finally, KGI drivers - or firmware - are (as far as common features are concerned) considered part of the display hardware and must be designed to be independent of the OS and host architecture.
  5. ... is essential for ...

  6. Display hardware must be reliable. This implies that the user or operator should be able to get access to a 'rescue shell' even if graphical user interfaces are not accessible. (E.g. the graphical user interface sever crashed.) Therefore KGI must not rely on applications to free resources properly.
  7. ... human-machine...

  8. Display hardware is built for humans working with computers. This requires KGI neither to prescribe nor to prevent policies if/how to share display hardware, what to do in case of non-fatal errors etc.
  9. ... interaction.

  10. Display hardware must be responsive. This implies that even for highly loaded machines delays between user input and visible feedback due to this change must be short. With 3D graphics becoming more and more standard, this requires advanced hardware programming techniques, such as graphics processor virtualization, priority scheduling, "processes models" for graphics processors etc. KGI must therefore provide means to handle several processes sharing resources where this makes sense.
These requirements are accompanied by the fact that display hardware is not standardized and probably the fastest evolving computer subsystem. KGI should therefore be open for future developments and not be specific to a particular graphics application programming interface. And last but not least, KGI is intended for UNIX-like operating systems, which feature multi-user multi-tasking environments. Thus KGI should allow for simple setup of independent multihead machines, featuring several workplaces per machine.

The KGI Architecture

The KGI architecture can be structured into the following 'subsystems': Additionally the current KGI sample implementation for Linux has also a console subsystem and compatibility module, adopted to the changes neccessary to implement the KGI architecture.

The subsequent text will focus on the interaction and hardware models,OS environment, display driver and application interface. The input hardware driver, console and compatibility module will not be described in much detail.

The KGI Human-Machine Interaction Model

KGI is developed for multi-user multi-tasking operating systems that allow several users to run processes simultaneously, providing means to isolate (protect) processes from each other, but also to share limited resources. KGI is consequently designed to allow several users to work interactively with the same machine. It allows each user to use an arbitrary number of display and input hardware (often referred to as independent multi-head). Input and output devices are grouped to "workplaces" owned by a particular user (the one using it). In order to access resources of a particular workplace, processes must belong to the user or have explicitly been granted access to these resources.
A side note:
Actually all human-machine interaction hardware like sound, workplace local peripherals, etc. needs to be treated this way. However, KGI is currently limited to input and visual output.  Actually, Kernel User Interface describes better what most of the KGI sample implementation for Linux does, but originally KGI was intended to handle display hardware only. Also, it should be pointed out clearly, that this Kernel User Interface is restricted to hardware drivers, protection and virtualization. Especially it must not be misunderstood as running a Graphical User Interface at kernel level, which is a common misunderstanding about KGI.
Additionally, KGI allows for virtual (textual and graphical) consoles that are mapped on physical display and input hardware (thereby becoming visible/active).

Analysing the flow of information gives a very natural model of human-machine interaction. Information going into the machine is "digitized" by input hardware. This digitizing is dealt with by the Kernel Input Interface file:kgi-0.9/kgi/include/kii/kii.h Information going out of the machine is converted by output (display) hardware. This is dealt with by the Kernel Graphics Interface file:kgi-0.9/kgi/include/kgi/kgi.h The actual information processing is done in processes that interface to hardware through abstract hardware representations, referred to as devices. Devices virtualize limited hardware resources.

All input device drivers (event sources - kii_input_t) register with a central mangagement code ( file:kgi-0.9/kgi/src/Linux/kgi.c) that enumerates input devices and groups these into so-called focuses kii_focus_t. All input devices associated with a focus send events to the same focused object (e.g. a virtual console) represented by a device (event-sink) kii_device_t. The focused object then processes the event, the result being commands sent to the display hardware represented by a device kgi_device_t. These get handed to the display hardware driver kgi_display_t, which in turn sends commands to the hardware, usually resulting in some feedback to the user.

For both KGI and KII care has been taken to separate hardware device driver specific issues (kii_input_t, kgi_display_t) from logical device representation (kii_device_t, kgi_device_t).

The Display Hardware Model

Despite all differences between the various graphics hardware, ranging from simple text frame buffers (Monochrome Display Adapter, LCD panels for embedded systems, etc.) up to virtual reality hardware as SGI's "Infinite Reality(TM)", certain properties of graphics hardware can be modeled and abstracted. KGI's display hardware model is intended to give a functional representation of the underlying hardware, hiding implementation details from applications.

Display Drivers and Display Management

KGI display drivers bridge between the display hardware and the application interface. For the most common hardware design, a modular driver architecture has been developed to allow for maximum re-usage of coding efforts. The most common design of display hardware can be sketched like this:
Hardware Diagram [11kB JPEG]

Though recent developments allow the integration of some parts into one piece of silicon, the functional units can still be identified. Each functional unit (except memory) is represented by a driver module, using a well defined internal interface. A fully operational display hardware driver is obtained by linking the proper (binary) subsystem drivers together. For example, a graphics card based on a 3Dlabs PermediaNT, with a TI TVP 3026 ramdac with integrated clock chip and a multisync monitor may reuse the ramdac, clock and monitor driver written for a S3 Vision968 based board with TI TVP 3026 and multisync monitor. A proof-of-concept implementation for this concept has been released by the GGI Project in early 1998. Valuable feedback and conclusions have been drawn from this implementation, resulting in an improved version of the internal module interface. A sample driver for the architecture described in this document is being developed, but only semi-operational yet. However, it is developed far enough to give a impression how to use the internal module interface. As KGI display drivers may run in kernel mode, the use of floating point operations is not allowed. All drivers must conform to certain requirements, such as they must not call other than the KGI environment functions. Also, KGI drivers must not rely on certain advanced features such as interrupt handling, DMA support etc. being available. However, if present they may benefit from this. This allows KGI driver to be used as userspace drivers or in-kernel drivers without modification.
The intra-display driver interface is defined in kgi-0.9/drv/display/kgi/module.h

The Physical I/O and Environment OS Support

A main goal for the display driver modules was to be independent of the underlying Operating System Environment. This requires the usual abstraction of system-dependent data types, physical I/O methods, IRQ/DMA handling and environment. The system dependent data types and the environment functions (memset, etc.) are considered straightforward and will not be discussed in detail here.

Input drivers and management

KGI contains some input drivers modified to be aware of the multihead issues. This is for historical reasons as at the time of project start no unified input drivers were around.

The Application Interface

Applications do not interface to graphics hardware directly, unless approved by the display driver. Instead, an abstract representation of graphics hardware is defined and accessed by applications using a special device file and standard UNIX file operations (open, close, ioctl, mmap, read).
are used to maintain virtualization of display hardware resources
ioctl is used to control overall behaviour of display hardware and to send commands to the hardware (e.g. setting of look-up tables, control of video overlay scaling, etc.).
read is used to report asynchronous events (accelerator programming error, accelerator finished, request for shared mapping from other processes etc.)
mmap is used to gain direct access to accelerator command buffers, memory mapped I/O regions, such as video stream ports, accelerator register files, frame buffer regions etc. Applications that want to use direct access may be required to conform to certain restrictions, e.g. access only with certain widths, etc. This may be indicated by setting access permissions for the special file and using set-user-id features, or implementing some verification protocol with the session manager. KGI tries not to impose policy here, except that the driver's constraints have to be met.

Summary and State of Development

This article was intended to give an overview about the KGI architecture to handle display hardware. Currently the KGI Project is working on a improved proof-of-concept implementation (kgi-0.9) using Linux as a target OS, that will implement the enhanced concepts presented in this articles, such as graphics processor virtualization as part of a first version of a direct-rendering architecture. This will include an experimental X server (XKGI) adopted to this architecture.
The implementation of the KGI human-machine interface model is already finished and currently it beta-testing state. It already allows to setup true multi-head machines with XFree-4.0. However, most advanced features require native KGI drivers, which are still under development.