# Unix

*"Those who don't know Unix are [doomed](doom.md) to reinvent it, poorly."* --obligatory quote by Henry Spencer

Unix (plurar *Unixes* or *Unices*) is an [old](old.md) [operating system](operating_system.md) developed since 1960s as a research project of [Bell Labs](bell_labs.md), which has become one of the most influential pieces of software in [history](history.md) and whose principles (e.g. the [Unix philosophy](unix_philosophy.md), [everything is a file](everything_is_a_file.md), ...) live on in many so called  Unix-like operating systems such as [GNU](gnu.md)/[Linux](linux.md) and [BSD](bsd.md) (at least to some degree). The original system itself is no longer in wide use (it was later followed by [plan9](plan9.md), a [project](projects.md) which by now is itself also pretty old), the name UNIX is currently a [trademark](trademark.md) and a certification. Nonetheless Unix is not significant for being a fossil operating system in a museum but rather as a concept, for as someone once said: *Unix is not so much an operating system as a way of thinking*.

In one aspect Unix has achieved the highest status a piece of software can strive for: it has transcended its implementation and became a [de facto standard](de_facto_standard.md). That is to say it became a set of interface conventions, programming principles, "[paradigms](paradigm.md)", [cultural](culture.md) and philosophical ideas rather than being a single system, a blob of 1s and 0s, it lives on as a concept that's being reimplemented, imitated and adopted to various degrees. This is remarkably important and puts Unix among the most significant terms in technological world, as we now don't depend on any single Unix implementation but instead have a choice of great variety of Unix systems which we can switch between without too much trouble, just like for example the [C](c.md) language (which was developed as part of Unix) is nowadays an abstract language enjoying many different implementations. This is invaluable as prerequisite for true technological [freedom](freedom.md), as freedom of choice prevents monopolization, and as a consequence stands as yet another argument for using Unix systems more.

The main highlights of Unix are possibly these:

- **[Unix philosophy](unix_philosophy.md)**: a kind of general mindset of software development, based on [minimalism](minimalism.md), frequently summed up as **"do one things well"** (rather than ["do everything but poorly"](windows_philosophy.md)) and "make programs work in collaboration with other programs", advising on using universal text interfaces for communication etc. In practice this manifests as so called **[pipelines](pipe.md)**, a way of chaining programs (typically using the pipe `|` operator, hence the name) by sending one program's output to other program's input. This philosophy is in contrast with so called "[Windows](windows.md) philosophy": that of creating big, [bloated](bloat.md) "monolithic" programs.
- **[everything is a file](everything_is_a_file.md)**: Unix chose to use the [file](file.md) [abstraction](abstraction.md) to enable universal communication of programs with hardware and among themselves, i.e. on unices most things such as printing, reading keyboard, networking etc. will be likely implemented as reading or writing to/from some special (sometimes just virtual) file. This has the advantage of being able to just use some file reading library or syscall, not having to access physical memory bits in memory, which may be difficult, unsafe etc.
- Text centrism, preference of working with the [command line](cli.md) (avoiding [GUI](gui.md)), preference of text formats, value on [portability](portability.md) (even before performance), sharing of source code, freedom of [information](information.md) and openness, connection to [hacker culture](hacking.md), valuing human time over machine time, ...
- ...

Unix is significantly connected to software [minimalism](minimalism.md), however most unices are still not minimalist to absolute extreme and many unix forks (e.g. [GNU](gnu.md)/[Linux](linux.md)) just abandon minimalism as a top priority. So the question stands: **is Unix [LRS](lrs.md) or is it too [bloated](bloat.md)?** The answer to this will be similar to our stance towards the [C](c.md) language (which itself was developed alongside Unix); from our point of view Unix -- i.e. its concepts and some of their existing implementations -- is relatively [good](good.md), there is a lot of wisdom to take away (e.g. "do one thing well", modularity, "use text interfaces", ...), however these are intermixed with things which under more strict minimalism we may want to abandon (e.g. multiple users, file permissions and ownership, also "everything is a file" requires we buy into the file [abstraction](abstraction.md) and will often also imply existence of a file system etc., which may be unnecessary, even multitasking could be dropped), so in some ways we see Unix as a temporary "[least evil](least_evil.md)" tool on our way to truly good, extremely minimalist technology. [DuskOS](duskos.md) is an example of operating system more close to the final idea of LRS. But for now Unix is very cool, some Unix-like systems are definitely a good choice nowadays.

There is a semi humorous group called the *UNIX HATERS* that has a mailing list and a whole book that criticizes Unix, arguing that the systems that came before it were much better -- though it's mostly just [joking](jokes.md), they give some good points sometimes. It's like they are the biggest boomers for whom the Unix is what [Windows](windows.md) is to the Unix people.

## History

In the 1960s, Bell Labs along with other groups were developing [Multics](multics.md), a kind of [operating system](os.md) -- however the project failed and was abandoned for its complexity and expensive development. In 1969 two Multics developers, [Ken Thompson](key_thompson.md) and [Dennis Ritchie](dennis_ritchie.md), then started to create their own system, this time with a different approach; that of [simplicity](minimalism.md) (see [Unix philosophy](unix_philosophy.md)). They weren't alone in developing the system, a number of other hackers helped program certain parts such as a file system, [shell](shell.md) and simple utility programs. At VCF East 2019 Thompson said that they developed Unix as a working system in three weeks. At this point Unix was written in [assembly](assembly.md).

In the early 1970s the system got funding as well as its name Unix (a pun on Multix). By now Thompson and Richie were developing a new language for Unix which would eventually become the [C](c.md) language. In version 4 (1973) Unix was rewritten in C.

Unix then started being sold commercially. This led to its fragmentation into different versions such as the [BSD](bsd.md) or [Solaris](solaris.md). In 1983 a version called System V was released which would become one of the most successful. The fragmentation and a lack of a unified standard led to so called [Unix Wars](unix_wars.md) in the late 1980s, which led to a few Unix standards such as [POSIX](posix.md) and Single Unix Specification.

For [zoomers](genz.md) and other noobs: Unix wasn't like [Windows](windows.md), it was more like [DOS](dos.md), things were done in [text interface](cli.md) only (even a [TUI](tui.md) or just colorful text was a luxury) -- if you use the command line in "[Linux](linux.md)" nowadays, you'll get an idea of what it was like, except it was all even more primitive. Things we take for granted such as a [mouse](mouse.md), [copy-pastes](copy_paste.md), interactive text editors, having multiple user accounts or [running multiple programs at once](multitasking.md) were either non-existent or advanced features in the early days. There weren't even personal computers back then, people accessed share computers over terminals. Anything these guys did you have to see as done with stone tools -- they didn't have GPUs, gigaherts CPUs, gigabytes of RAM, scripting languages like Python or JavaScript, Google, stack overflow, wifi, mice, IDEs, multiple HD screens all around, none of that -- and yet they programmed faster, less buggy software that was much more efficient. If this doesn't make you think, then probably nothing will.

## How To For Noobs

UNDER CONSTRUCTION

*Note: here by "Unix" we will more or less assume a system conforming to some version of the POSIX standard.*

This should help complete noobs kickstart their journey with a Unix-like system such as [GNU](gnu.md)/[Linux](linux.md) or [BSD](bsd.md). Please be aware that each system has its additional specifics, for example [package managers](package_manager.md), init systems, [GUI](gui.md) and so on -- these you must learn about elsewhere as here we may only cover the core parts those systems inherited from the original Unix. Having learned this though you should be able to somewhat fly any Unix like system. Obviously we'll be making some simplifications here too, don't be too pedantic if you're a pro Unix guru please.

Also a NOTE: terms such as *command line*, *terminal* or *shell* have different meanings, but for simplicity we'll be treating them more or less as synonyms here.

Learning to use Unix in practical terms firstly means **learning the [command line](cli.md)** and then a few extra things (various concepts, philosophies, conventions, file system structure etc.). Your system will have a way for you to enter the command line that allows you to interact with it only through textual commands (i.e. without [GUI](gui.md)). Sometimes the system boots up to command line, other time you must click an icon somewhere (called *terminal*, *term*, *shell*, *command line* etc.), sometimes you can switch [TTY](tty.md)s with *CTRL+ALT+Fkeys* etc. To command line virgins this will seem a little intimidating but it's absolutely necessary to know at least the basics, on Unices the command line is extremely powerful, efficient and much can only ever be achieved through the command line.

**The gist**: unsurprisingly in command line you write commands -- many of these are actually tiny programs called **Unix utilities** (or just "utils"). These are installed by default and, they're tools for you to do whatever you want (including stuff that on normie systems you usually do by clicking with a [mouse](mouse.md)). For example `ls` is a program that writes out list of files in the working directory, `cd` is a program that changes working directory etc. There are many more such programs and you must learn at least the most commonly used ones. Good news is that the programs are more or less the same on every Unix system so you just learn this once. There also exist other kinds of commands -- those defined by the shell language ([shell](shell.md) is basically a fancy word for the textual interface), which allow us to combine the utilities together and even [program](programming.md) the shell (we call this [scripting](script.md)). First learn the utils (see the list below).

PRO TIP: convenient features are often implemented, most useful ones include going through the history of previously typed commands with UP/DOWN keys and completing commands with the TAB key, which you'll find yourself using very frequently. Try it. It's enough to type just first few letters and then press tab, the command will be completed (at least as much as can be guessed).

You run a utility simply by typing its name, for example writing `ls` will show you a list of files in your current directory. Very important is the `man` command that shows you a **manual page** for another command, e.g. typing `man ls` should display a page explaining the `ls` utility in detail. Short help for a utility can also usually be obtained by writing `-h` after it, for example `grep -h`.

Unix utilities (and other programs) can also be invoked with **arguments** that specify more detail about what should be done. Arguments go after the utility name and are separated by spaces (if the argument itself should contain a space, it must be enclosed between double quotes, e.g.: `"abc def"` is a single arguments containing space, but `abc def` are two arguments). For example the `cd` (change directory) utility must be given the name of a directory to go to, e.g. `cd mydirectory`.

Some arguments start with one or two minus characters (`-`), for example `-h` or `--help`. These are usually called **flags** and serve either to turn something on/off or to name other parameters. For example many utilities accept a `-s` flag which means "silent" and tells the utility to shut up and not write anything out. A flag oftentimes has a short and long form (the long one starting with two minus characters), so `-s` and `--silent` are the same thing. The other type of flag says what kind of argument the following argument is going to be -- for example a common one is `--output` (or `-o`)  with which we specify the name of the output file, so for instance running a C compiler may look like `c99 mysourcecode.c --output myprogram` (we tell the compiler to name the final program "myprogram"). Short flags can usually be combined like so: instead of `-a -b -c` we can write just `-abc`. Flags accepted by utilities along with their meaning are documented in the manual pages (see above).

To **run a program** that's present in the current directory as a file you can't just write its name (like you could e.g. in [DOS](dos.md)), it MUST be prefixed it with `./` (shorthand for current directory), otherwise the shell thinks you're trying to run an INSTALLED program, i.e. it will be looking for the program in a directory where programs are installed. For example having a program named "myprogram" in current directory it will be run with `./myprogram`. Also note that to be able to run a file as a program it must have the executable mode set, which is done with `chmod +x myprogram` (you may have to do this if you e.g. download the program from the Internet). **Programs can also take arguments** just like we saw with the built-in utilities, so you can run a program like `./myprogram abc def --myflag`.

Now to the very basic stuff: **browsing directories, moving and deleting files etc.** This is done with the following utils: `ls` (prints files in current directory), `pwd` (prints path to current directory), `cd` (travels to given directory, `cd ..` travels back), `cat` (outputs content of given file), `mkdir` (creates directory), `rm` (removes given file; to remove a directory `-rf` flag must be present), `cp` (copies file), `mv` (moves file, including directory -- note that moving also serves for renaming). As an exercise try these out (careful with `rm -rf`) and read manual pages of the commands (you'll find that `ls` can also tell you for example the file sizes and so on).

**Files and file system**: On Unices the whole filesystem hierarchy starts with a directory called just `/` (the root directory), i.e. every absolute (full) path will always start with slash (don't confuse `/` with `\`). For example pictures belonging to the user *john* may live under `/home/john/pictures`. It's also possible to use relative paths, i.e. ones that are considered to start in the current (working) directory. A dot (`.`) stands for current directory and two dots (`..`) for the directory "above" the current one. I.e. if our current directory is `/home/john`, we can list the pictures with `ls pictures` as well as `ls /home/john/pictures` or `ls ./pictures`. Absolute and relative paths are distinguished by the fact the absolute one always starts with `/` while relative don't. There are several **types of files**, most importantly *regular files* (the "normal" files) and *directories* (there are more such *symbolic links*, *sockets*, *block special files* etc., but for now we'll be ignoring these). Unix has a [paradigm](paradigm.md) stating that **everything's a [file](file.md)**, so notably accessing e.g. hardware devices is done by accessing special device files (placed in `/dev`). Just remember this concept, you'll hear about it often.

NOTE: On Unices files often don't have extensions as it's often relied on so called [magic number](magic_numbers.md) (first few bytes in the file) to decide what kind of file we're dealing with. You will see files with extension (`.sh`, `.txt`, ...) but notably for example compiled programs typically don't have any (unlike for example on [Windows](windows.md)).

Files additionally have attributes, importantly so called **permissions** -- unfortunately these are a bit complicated, but as a mere user working with your own files you won't have to deal too much with them, only remember if you encounter issues with accessing files, it's likely due to this. In short: each file has an owner and then also a set of permissions that say who's allowed to do what with the file. There are three kind of permissions: *read* (`r`), *write* (`w`) and *execute* (`x`), and ALL THREE are defined for the file's owner, for the file's group and for everyone else, plus there is a magical value suid/sgid/sticky we won't delve into. All of this is then usually written either as a 4 digit [octal](octal.md) number (each digit expresses the three permission [bits](bit.md)) or as a 12 character string (containing the `r`/`w`/`x`/`-` characters). Well, let's not dig much deeper now.

PRO TIP: there is a very useful feature called **wildcard characters** that help us handle many files at once. Most commonly used are `*` and `?` wildcards -- if we use these in a program argument, the arguments will be expanded so that we get a list of files matching certain pattern. This sounds complicated but let's see an example. If we write let's say `rm *.jpg`, we are going to remove all files in current directory whose name ends with `.jpg`. This is because `*` is a wildcard character that matches any string and when we execute the command, the shell actually replaces our argument with all files that match our pattern, so the command may actually internally look like `rm picture1.jpg picture2.jpg picture3.jpg`. `?` character is similar but matches exactly one character (whatever it is), so to list for example all files whose name is exactly three characters we can write `ls ???`.

Here is a quick cheatsheet of the most common Unix utilities:

| name          | function                                                                 | possible arguments (just some)              |
| ------------- | ------------------------------------------------------------------------ | ------------------------------------------- |
| alias         | create or display alias (nickname for another command)                   | alias=command                               |
| [awk](awk.md) | text processing language (advanced)                                      |                                             |
| bc            | interactive calculator                                                   |                                             |
| c99           | [C](c.md) language compiler (advanced)                                   | file, -o (output file)                      |
| cd            | change directory                                                         | directory name (`..` means back)            |
| chmod         | change file mode                                                         | +x (execute), +w (write), +r (read), file   |
| cmp           | compare files                                                            | -s (silent), file1, file2                   |
| cp            | copy files                                                               | -r (recursive, for dirs), file, newfile     |
| date          | write date and/or time                                                   | format                                      |
| df            | report free space on disk                                                | -k (use KiB units)                          |
| du            | estimate size of file (useful for directories)                           | -k (use KiB units), -s (only total), file   |
| echo          | write out string (usually for scripts)                                   |                                             |
| [ed](ed.md)   | ed is the standard text editor                                           |                                             |
| expr          | evaluate expression (simple calculator)                                  | expression (as separate arguments)          |
| false         | return false value                                                       |                                             |
|[grep](grep.md)| search for pattern in file                                               | pattern, file, -i (case insensitive)        |
| head          | show first N lines of a file                                             | -n (count), file                            |
| kill          | terminate process or send a signal to it                                 | processid, -9 (kill), -15 (terminate)       |
| ls            | list directory (shows files in current dir.)                             | -s (show file sizes in block)               |
| [man](man.md) | show manual page for topic                                               | topic                                       |
| mkdir         | make directory                                                           | name                                        |
| mv            | move (rename) file                                                       | -i (ask for rewrite), file, newfile         |
| pwd           | print working directory                                                  |                                             |
| rm            | remove files                                                             | -r (recursive, for dirs), -f (force)        |
| [sed](sed.md) | stream editing util (replacing text etc.), see also [regex](regex.md)    | script, file                                |
| [sh](sh.md)   | shell (the command line interpreter, usually for scripting)              | -c (command string)                         |
| sort          | sort lines in file                                                       | -r (reverse), -u (unique), file             |
| tail          | show last N lines of a file                                              | -n (count), file                            |
| true          | return true value                                                        |                                             |
| uname         | output system name and info                                              | -a (all, output everything)                 |
| [vi](vi.md)   | advanced text editor                                                     |                                             |
| wc            | word count (count characters or lines in file, can tell exact file size) | -c (character), -l (lines), file            |

NOTES on the above table:

- Typically there are two ways of feeding input data to a utility: either by specifying a file to read from or by feeding the input on to the utility's standard input. This also applies to the output. Using standard input/output is a more "Unix" way as it allows us to chain the utlities with pipes, make one program feed its output to another as input.
- Utilities try to follow common conventions so that it's easier to guess and remember what flags mean etc., for example `-h` is commonly a flag for getting help, `-o` is one for specifying output file etc.
- Specific Unix systems will normally have more feature rich utilities, supporting additional flags and even adding new utilities. Check out manual pages on your system. You'll probably have to learn about common utils that aren't part of POSIX, e.g. *[wget](wget.md)*, *history*, *[ssh](ssh.md)*, *[curl](curl.md)*, *[sudo](sudo.md)*, *[apt](apt.md)* and more. And of course there are thousands and thousands of additional utilities/programs you can download, program or otherwise install on your system.

Now on to a key feature of Unix: **pipelines and redirects**. [Processes](process.md) (running programs) on Unix have so called **standard input** (*stdin*) and **standard output** (*stdout*) -- these are streams of data (often textual but also binary) that the process takes on input and output respectively. There may also exist more streams (notably e.g. *standard error output*) but again, we'll ignore this now. When you run a program (utility etc.) in the command line, standard input will normally come from your keyboard and standard output will be connected to the terminal (i.e. you'll see it being written out in the command line). However sometimes you may want the program to take input from a file and/or to write its output to a file (imagine e.g. keeping [logs](logging.md)), or you may even want one program to feed its output as an input to another program! This is very powerful as you may combine the many small utilities into more powerful units. See also [Unix philosophy](unix_philosophy.md).

Most commonly used redirections are done like this:

- `command > file`: redirects output of *command* to file *file* (rewriting its content if there is any).
- `command < file`: redirects input of *command* to come from *file*.
- `command >> file`: output of *command* will be appended to *file* (i.e. added at its end).

Pipelines are similar: they are chains of several programs separated by a "pipe" character: `|`. This makes a program feed its output to the input of the next program. For example `ls | grep \.html` will run the `ls` command and pass its output (list of files in current directory) to `grep`, which will only filter out those that contain the ".html" string.

Several commands can also be written on a single line, they just have to be separated with `;`.

**Example** of doing stuff in a Unix terminal (`#` character starts a [comment](comment.md) -- these are here only to describe what's happening in the example):

```
> pwd
/home/drummyfish
> ls
Pictures    Documents    Downloads
git         Videos
> cd Downloads
> ls
free_software_song.midi  hentai_porn.mp4
lrs_wiki.txt
> rm hentai_porn.mp4     # oh noes, quickly delete this
> cd ../git; ls
Anarch      comun       Licar
raycastlib  small3dlib
> cd Anarch
> wc -l *.h *.c | tail -n 1    # count lines of code in .h and .c files
14711 total
> cat *.h *.c | grep "TODO"    # show all TODOs in code
      (SFG_game.backgroundScaleMap[(pixel->position.y          // ^ TODO: get rid of mod?
  RCL_Unit     direction;  // TODO: rename to "angle" to keep consistency
  TODO: maybe array functions should be replaced by defines of funtion names
  /* FIXME/TODO: The adjusted (=orthogonal, camera-space) distance could
  RCL_Unit limit1, // TODO: int16_t?
  RCL_Unit depth = 0; /* TODO: this is for clamping depth to 0 so that we don't
         increment == -1 ? i >= limit : i <= limit; /* TODO: is efficient? */\
  RCL_Unit limit1, // TODO: int16_t?
       increment == -1 ? i >= limit : i <= limit; // TODO: is efficient?
  // TODO: probably doesn't work
#if 1 // TODO: add other options for input handling (SDL, xinput, ...)
> echo "Remember to fix the TODOs in code!" >> TODO.txt  # add note to TODO file
```

## See Also

- [unix philosophy](unix_philosophy.md)
- [Linux](linux.md)
- [GNU](gnu.md)
