Introduction to Unix

• Unix is now 46 years old
• It began in 1969
• See The Evolution of the Unix Time-sharing System by Dennis Ritchie

Introduction to Unix

• Started at AT&T's Bell Labs, and it was derived from MULTICS.
• Initial hardware was a DEC PDP-7, and the filesystem was hierarchical but did not have pathnames
• I.e., there was no equivalent to a pathname such as /etc/hosts, it would just be hosts; directory information was kept in a special file called dd.

Introduction to Unix

• Rather than a product from a manufacturer, Unix began as collaboration with these goals:
  • Simplicity
  • Multi-user support
  • Portability
  • Universities could get source code easily
  • Users could and did share ideas, programs, bug fixes

Introduction to Unix

• The development of early Unix was user-driven rather than corporate-driven
• Note that Linux and the BSDs (FreeBSD, OpenBSD, NetBSD) now flourish in similar ``open source'' environments (http://www.freebsd.org, http://www.openbsd.org, http://www.netbsd.org)

Introduction to Unix

The first meeting of the Unix User Group was in May, 1974; this group would late become the Usenix Association

You can watch the ever-varying open source world of Linux/BSD distributions at http://www.distrowatch.com

A History of Unix

In the beginning, processes were very different

• Originally, each terminal only could have one active process
• When creating a ``child'' process, the parent first closed all of its open files
• Then the parent linked to the executable and opened it
• Then the parent copied a bootstrap to the top of memory and jumped into the bootstrap
• The bootstrap copied the code for the new process over the parent's code and then jumped into it
• When the child did an exit, it first copied in the parent process code into its code area, and then jumped back into the parent code at the beginning

Old Unix

Today the parent uses a fork/exec/wait model:

• fork(2) (to create a new child process)}
• exec*(2) (to have the child process start executing a new program)}
• wait*(2) (to wait on the child (or at least check on its status if non-blocking))}

Linux: a complete Unix-compatible operating system

• Runs on huge array of hardware, from IBM's biggest machines down to commodity routers such as Linksys routers using DD-WRT/OpenWRT
• Based on kernel begun by Linus Torvalds. (He is still in charge of kernel development (3.14.4 is the current stable release (2014/05/13) — see http://www.kernel.org), though now many people work on the kernel.)

Linux: a complete Unix-compatible operating system

• The Linux distribution(s) we will use include a full development environment, windowing interface, office environment products (word processors, spreadsheets, etc), C, C++, Fortran, several mail systems (exim, postfix, sendmail, others) and whole lot more (a full install is around 5 to 15 gigabytes). For BSD, we will use FreeBSD; for Solaris, we will use OpenIndiana.
• Linux is mostly POSIX.1 compliant. FAQ on POSIX is at http://www.opengroup.org/austin/papers/posix_faq.html

System administration duties

• Installing new hardware and software
• Updating and upgrading hardware and software
• Monitoring for problems
• Resolving problems
• Planning for growth and obsolence
• Making backups
• Recovering data from backups

System administration responsibilities (AIC)

• Availability of services and data
• Integrity of services and data
• Confidentiality

System administration responsibilities: Availability

• Lack of availability can have a large dollar impact on many businesses
• The most common dangers to availability are SPOFs (single points of failure)
• SPOFs are generally cured by redundancy. Redundancy models:
  • Cold → Good points: ``guarantees'' availability, but (1) typically most expensive (2) tends to require frequent testing (3) typically slow to bring live
  • Warm dual-use hardware that is powered up and used for related but non-production purposes such as development or q/a) → Typically good availability and less expensive than cold since the hardware has day-to-day use, but (1) tends to be one-offish and (2) tends to require frequent testing of switchover
  • Hot (redundant hardware in production) typically least expensive since it is in active production use and testing requirements tend to be less but more likely to have capacity problems in the event of partial failure

System administration responsibilities: Integrity of data

• For system administration purposes, often data integrity is implicitly enforced both at the hardware and the operating system level, such as CRCs for disk reads, parity in RAID-5 controllers, and so forth.

System administration responsibilities: Data security

• This is a big topic, even from a pure system administration point of view.
• Passive security: design your systems with security in mind. (Fail2ban is one of the better intrusion prevention packages available.)
• Active security: (1) Searching for problems with tools such as Snort (2) Forensically researching problems with tools such as with Sleuth Kit. Digital forensics is a separate science on its own.

Mechanics of Unix/Linux administration: editors, scripting, and compilation

• Editors: A lot of Unix/Linux system administration tasks revolve around editing various files. The best two choices for editing are the programs vi and emacs, although there are many, many editors available.
• Scripting: In addition to simple configuration files, another important subset of tasks that require editors are scripts. These scripts are used for various purposes, and often automating routine tasks such as running programs such as updatedb. Popular scripting languages are Bash, Python, and Perl.
• Compilation: Generally on Unix/Linux machines, gcc has been the standard choice for compilation.

Unix/Linux system administration: editors &mdash looking at vi

• vi: It is almost always found somewhere on a Unix/Linux machine, often in the form of vim.
• vi advantage: It is simple to learn. It has two modes, ``motion'' and ``insert''. In the ``motion'' mode, the most important keys are just
  • h: Move the cursor left.
  • l: Move the cursor right.
  • j: Move the cursor down.
  • k: Move the cursor up.
  • i: Go into ``insert'' mode.
  • :: Execute an internal vi command of some sort. The most popular are w for "write", q for "quit", and quit! for "quit without saving anything".

Unix/Linux system administration: editors &mdash still looking at vi

• vi disadvantage: While original vi is quite simple, it also was not very featureful. This isn't as much of a problem with vim, an updated version of vi. The vim extensions are not as logical as those emacs since they were created as an afterthought.

Unix/Linux system administration: editors &mdash looking at emacs

emacs: Not always installed by default. In fact, in the last few years, default installation of Emacs in Unix/Linux distributions has become less common.

• emacs advantages: As computer scientists, emacs is quite intuitive: each sequence of keystrokes can be mapped to an arbitrary Emacs Lisp function. For instance, by default in TeX mode, the key a is just mapped to a function which inserts an a, but the double quote key is mapped to insert first a pair of `` and then the second time to insert a pair of closing '' in accord with TeX's expectations.
• emacs disadvantage: It's not found on every machine.
• emacs disadvantage: It's big. Doing an install via repositories can occasionally entail many more packages than you might expect!

Unix/Linux system administration: scripting languages

• bash: The ``Bourne-Again SHell''. Introduced the ``readline'' library. A comprehensive overhaul of the original Bourne shell.
• perl5: The ``Practical Extraction and Report Language''. An amazing language in its own right; originally, its syntax could be mistaken for "line noise", much as the old TECO editor commands.
• perl6: Currently, version 5 is very useful, though version 6 still hasn't jelled. There was a recent development release of Rakudo, called "Rakudo Star"; look at http://rakudo.org/.
• python: An increasingly popular programming language for system administration tasks. Systems as rdiff-backup and even music players like Exaile are written in python.
• lua: Another language that is recently getting attention is lua. It's fast and it's expressive; for some people, Perl's syntax is too cryptic, Python's syntax is too inexpressive, but they find Lua more acceptable. It's hard to guess just how popular any language will become, and lua hasn't gained much in the last two years or so. Looking at a Fedora 16 box, I don't see any lua scripts in /bin, /sbin, /usr/bin/, or /usr/sbin. In contrast, there are 138 Perl scripts, 133 Python scripts, 3 AWK scripts, and 667 shell scripts. On a random Linux Mint 13 box, I find no lua scripts, but 247 Perl and 120 Python scripts.

Unix/Linux system administration: scripting languages

• Scripts in the Unix/Linux world are most commonly written in shell, Perl, and Python. (We still occasionally find some AWK scripts, but it's been fading fast over the last few years.)