COP 3353: INTRODUCTION TO UNIX and Computing in 2017

• General purpose computers have three critical logical elements: CPUs, RAM, and persistent storage (also called "secondary storage", "non-volatile storage", or even the very old term "backing store".)
• A CPU keeps its internal state in what we call "registers". Taken with the state of the RAM, the aggregate of all of the CPU register state and all of the RAM state is called the state of a computer system. State is just a snapshot of all of the 0s and 1s (bits) in the registers and the RAM at a given instant.
• CPUs run off "clocks". Each clock cycle, each CPU uses the state of its registers and the state of RAM to compute a new state for the next clock cycle.
• Other devices in a computer system usually communicate state to the CPU via RAM or directly into CPU registers.
• Once you have hardware, you need software (aka as "initial state") to do something useful with the hardware. While some embedded devices have very simple software, most computing devices in 2017 have an "operating system". An "operating system" provides a mediator of hardware and software resources; programs now make requests to the operating system for access to hardware rather than just directly accessing the hardware. This mediation allows concurrency, and we can have the idea of individual "processes" which can exist simultaneously.

Introduction to Unix

• Unix began in 1969, and is now 48 years old
• See The Evolution of the Unix Time-sharing System by Dennis Ritchie.

Introduction to Unix

• Started at AT&T's Bell Labs, and it was derived from MULTICS.
• Initial hardware was a DEC PDP-7, and the filesystem was hierarchical but did not have pathnames.
• I.e., there was no equivalent to a pathname such as /etc/hosts, it would just be hosts. This was actually common with other comparable computer operating systems of the late 1960s.

Introduction to Unix

• Rather than a product from a manufacturer, Unix began collaboratively with these goals:
  • Simplicity
  • Multi-user support
  • Portability
  • Universities could get source code easily
  • Users shared ideas, programs, bug fixes

Introduction to Unix

• The development of early Unix was user-driven rather than corporate-driven
• Note that Linux and the BSDs (FreeBSD, OpenBSD, NetBSD) now flourish in similar "open source" environments (http://www.freebsd.org, http://www.openbsd.org, http://www.netbsd.org)

Introduction to Unix

The first meeting of the Unix User Group was in May, 1974; this group would late become the Usenix Association

You can watch the ever-varying open source world of Linux/BSD distributions at http://www.distrowatch.com

A History of Unix

In the beginning, processes were very different

Today the parent uses a fork/exec/exit/wait model:

• fork(2) (to create a new child process)
• exec*(2) (to have the child process start executing a new program)
• exit(2) (exit the process)
• wait*(2) (to wait on the child (or at least check on its status if non-blocking))

Linux: a complete Unix-compatible operating system

• Runs on huge array of hardware, from IBM's biggest machines down to commodity routers such as the Linksys WRT routers using OpenWRT, and on everything to airliners to the Raspberry Pi.
• Based on kernel begun by Linus Torvalds. (He is still in charge of kernel development (Linux 4.11) is the current stable release — see http://www.kernel.org), though now many people work on the kernel.)

Linux: a complete Unix-compatible operating system

• If you work in the majors lab, there are a number of Linux machines in the lab. Distributions vary, and the versions (or even existence) of programs might well vary from machine to machine.

Mechanics of using Unix/Linux: shells, editors, scripting, and compilation

• Shells: The Unix world grew up under the idea of a shell, a keyboard-based approach to computing.
• Editors: A lot of Unix/Linux tasks revolve around editing various files. The best two choices for editing are the programs vim and emacs, although there are many, many editors available, including the very easy to use pico and nano.
• Scripting: While scripting was not unique to Unix, its ability join programs together ad hoc using pipes was a very powerful paradigm when combined with the ability to create sequences of commands, test for conditions, and iterate over a sequence.
• Compilation: Unix and C are inextricably linked, with both created by the same people at the same time. In addition to C, Unix/Linux also supports virtually every other language imaginable, from ancient ones like Fortran, Basic, FORTH, and COBOL to more modern choices, such as Haskell and Racket.

Editors — looking at vi

• vi: It is almost always found somewhere on a Unix/Linux machine, often in the form of vim.
• vi advantage: It is simple to learn. It has two modes, ``motion'' and ``insert''. In the ``motion'' mode, the most important keys are just
  • h: Move the cursor left.
  • l: Move the cursor right.
  • j: Move the cursor down.
  • k: Move the cursor up.
  • i: Go into ``insert'' mode.
  • :: Execute an internal vi command of some sort. The most popular are w for "write", q for "quit", and quit! for "quit without saving anything".

Editors — still looking at vi

• vi disadvantage: While original vi is quite simple, it also is not very featureful. This isn't as much of a problem with the more recent incarnation vim, an updated version of vi. These extensions are not as logical as those emacs since they were created as an afterthought.

Editors — looking at emacs

emacs: Not always installed by default. In fact, in the last few years, default installation of Emacs in Unix/Linux distributions has become less common.

• emacs advantages: As computer scientists, emacs is quite intuitive: each sequence of keystrokes can be mapped to an arbitrary Emacs Lisp function. For instance, by default in TeX mode the key a is just mapped to a function which inserts an a, but the double quote key is mapped to insert first a pair of `` and then the second time to insert a pair of closing '' in accord with TeX's expectations.
• emacs disadvantage: It's not found on every machine.
• emacs disadvantage: It's big. Doing an install via repositories can occasionally entail many more packages than you might expect!

Shells in detail

• A few interrelated ideas have made the Unix shell powerful: "standard i/o", bytestreams, and pipes.
• The fundamental idea is that processes would assume that they had access to "standard i/o", usually abbreviated as "stdio"; the three components of stdio are "standard in" (stdin), "standard out" (stdout), and "standard error" (stderr). Bytestreams would enter a process via stdin and exit the process via stdout; any error reporting was to go out stderr (and not pollute the outgoing bytestream.)
• Finally, processes could have stdout hooked to another process's stdin via a pipe; the syntax for such piping of bytestreams is a fundamental construct of all Unix shells., and strong kernel support for pipe(2) has been a fundamental attribute of Unix since the early 1970s.

Shells and people

• Human interaction with a shell is keyboard-based: the shell displays a "prompt", and waits for the user to type something.
• The human might enter a "command" (i.e., request a program to start, or a series of programs feeding from the first to the last), or the user might ask for some type of iteration to start (a "for" loop, for instance) and then specify what command or commands might be executed, or even specify some type of alternation (an "if" construction, for instance).
• The shell then tries to do what the user requests, and as output is generated, it might end up coming out of "stdout" (the screen).
• Sequences of commands, iterations, and alternations can be saved into a file that we call a "script" so that these can be repeated later. (We can also save them into internal states of a shell process, such as by "aliases" or by "functions").

Scripting languages

• Scripting languages are a subset of what we call "interpreted languages". Interpreted languages require a interpreter program to run and don't go through a separate compilation/link step to create a separate executable file. Examples of interpreted languages are Bash, AWK, Perl, Python, Ruby, and Lua.
• Scripts in the Unix/Linux world are most commonly written in a shell, Perl, or Python. (We still occasionally find some AWK scripts, but it's been fading fast over the last few years.) Very occasionally we find Lua scripts — but it has turned to be more used to augment applications rather than as an independent scripting language.

Common scripting languages

• bash: The ``Bourne-Again SHell''. Introduced the ``readline'' library. A comprehensive overhaul of the original Bourne shell.
• csh: A decreasingly popular shell; designed to be more of a user interface than a scripting language.
• perl5/perl6: The ``Practical Extraction and Report Language''. An amazing language in its own right; originally, early versions of Perl programs could have be mistaken for line noise, much as the old TECO editor commands.
• python: An increasingly popular programming language.

Compiled languages and Make

• "Compilation" is the process of turning ordinary text files, called "source files", into code that is directly executable by hardware.
• Interpreted languages are slower than compiled languages, and often we want programs to be as responsive as possible. Languages like C are quite fast since they are generally quite close to the actual hardware and one line of C often is only a few lines of assembly code.
• Generally on Unix/Linux machines, gcc is a good choice for C compilation. While it is not perfect, and the various generations of gcc can be somewhat frustrating to deal with, it is hard to beat the price.
• In order to stitch individual gcc compilation units together, we use make. As you develop a program and recompile it, make only rebuilds the necessary components each time. So if you have 20 separate C source files in a program but edit only one, only that one file is re-run through your compiler to "re-make" your executable.