Update

unix.md

@@ -31,7 +31,7 @@ For [zoomers](genz.md) and other noobs: Unix wasn't like [Windows](windows.md),
 
 UNDER CONSTRUCTION
 
-*Note: here by "Unix" we will assume a system conforming to the POSIX standard from 2001.*
+*Note: here by "Unix" we will more or less assume a system conforming to some version of the POSIX standard.*
 
 This section will help complete noobs kickstart their journey with a Unix-like system such as [GNU](gnu.md)/[Linux](linux.md) or [BSD](bsd.md). Please be aware that each system has its additional specifics, for example [package managers](package_manager.md), init systems and so on -- these you must learn about elsewhere as here we may only cover the core parts those systems inherited from the original Unix. Having learned this though you should be able to somewhat fly any Unix like system. Obviously we'll be making some simplifications here too.
 
@@ -43,13 +43,23 @@ PRO TIP: convenient features are often implemented, most useful ones include goi
 
 You run a utility simply by writing its name, for example typing `ls` will show you a list of files in your current directory. Very important is the `man` command that shows you a **manual page** for another command, e.g. typing `man ls` should display a page explaining the `ls` utility in detail. Short help for a utility can also usually be obtained by writing `-h` after it, for example `grep -h`.
 
-Unix utilities (and other programs) can also be invoked with **arguments** that specify more detail about what should be done. Arguments are written after the utility name and are separated by spaces (if the argument itself should contain space, it must be enclosed between double quotes, e.g. `"abc def"`). For example the `cd` (change directory) utility must be given the name of a directory to go to, e.g. `cd mydirectory`.
+Unix utilities (and other programs) can also be invoked with **arguments** that specify more detail about what should be done. Arguments are written after the utility name and are separated by spaces (if the argument itself should contain space, it must be enclosed between double quotes, e.g.: `"abc def"` is a single arguments containing space, but `abc def` are two arguments). For example the `cd` (change directory) utility must be given the name of a directory to go to, e.g. `cd mydirectory`.
 
 Some arguments start with one or two minus characters (`-`), for example `-h` or `--help`. These are usually called **flags** and serve either to turn something on/off or to name other parameters. For example many utilities accept a `-s` flag which means "silent" and tells the utility to shut up and not write anything out. A flag oftentimes has a short and long form (the long one starting with two minus characters), so `-s` and `--silent` are the same thing. The other type of flag says what kind of argument the following argument is going to be -- for example a common one is `--output` (or `-o`)  with which we specify the name of the output file, so for instance running a C compiler may look like `c99 mysourcecode.c --output myprogram` (we tell the compiler to name the final program "myprogram"). Short flags can usually be combined like so: instead of `-a -b -c` we can write just `-abc`. Flags accepted by utilities along with their meaning are documented in the manual pages (see above).
 
+To **run a program** that's present in the current directory as a file you can't just write its name (like you could e.g. in [DOS](dos.md)), it MUST be prefixed it with `./` (shorthand for current directory), otherwise the shell thinks you're trying to run an INSTALLED program, i.e. it will be looking for the program in a directory where programs are installed. For example having a program named "myprogram" in current directory it will be run with `./myprogram`. Also note that to be able to run a file as a program it must have the executable mode set, which is done with `chmod +x myprogram` (you may have to do this if you e.g. download the program from the Internet). **Programs can also take arguments** just like we saw with the built-in utilities, so you can run a program like `./myprogram abc def --myflag`.
+
 Now to the very basic stuff: **browsing directories, moving and deleting files etc.** This is done with the following utils: `ls` (prints files in current directory), `pwd` (prints path to current directory), `cd` (travels to given directory, `cd ..` travels back), `cat` (outputs content of given file), `mkdir` (creates directory), `rm` (removes given file; to remove a directory `-rf` flag must be present), `cp` (copies file), `mv` (moves file, including directory -- note that moving also serves for renaming). As an exercise try these out (careful with `rm -rf`) and read manual pages of the commands (you'll find that `ls` can also tell you for example the file sizes and so on).
 
-To **run a program** that's present in the current directory as a file you can't just write its name (like you could e.g. in [DOS](dos.md)), it MUST be prefixed it with `./` (shorthand for current directory), otherwise the shell thinks you're trying to run an INSTALLED program, i.e. it will be looking for the program in a directory where programs are installed. For example having a program named "myprogram" in current directory it will be run with `./myprogram`. Also note that to be able to run a file as a program it must have the executable mode set, which is done with `chmod +x myprogram` (you may have to do this if you e.g. download the program from the Internet). **Programs can also take arguments** just like we saw with the built-in utilities, so you can run a program like `./myprogram abc def --myflag`.
+**Files and file system**: On Unices the whole filesystem hierarchy starts with a directory called just `/` (the root directory), i.e. every absolute (full) path will always start with slash. For example pictures belonging to the user *john* may live under `/home/john/pictures`. It's also possible to use relative paths, i.e. ones that are considered to start in the current (working) directory. A dot (`.`) stands for current directory and two dots (`..`) for the directory "above" the current one. I.e. if our current directory is `/home/john`, we can list the pictures with `ls pictures` as well as `ls /home/john/pictures` or `ls ./pictures`. Absolute and relative paths are distinguished by the fact the absolute one always starts with `/` while relative don't. There are several **types of files**, most importantly *regular files* (the "normal" files) and *directories* (there are more such *symbolic links*, *sockets*, *block special files* etc., but for now we'll be ignoring these). Unix has a [paradigm](paradigm.md) stating that **everything's a [file](file.md)**, so notably accessing e.g. hardware devices is done by accessing special device files (placed in `/dev`).
+
+Files additionally have attributes, importantly so called **permissions** -- unfortunately these are a bit complicated, but as a mere user working with your own files you won't have to deal too much with them, only remember if you encounter issues with accessing files, it's likely due to this. In short: each file has an owner and then also a set of permissions that say who's allowed to do what with the file. There are three kind of permissions: *read* (`r`), *write* (`w`) and *execute* (`x`), and ALL THREE are defined for the file's owner, for the file's group and for everyone else, plus there is a magical value suid/sgid/sticky we won't delve into. All of this is then usually written either as a 4 digit [octal](octal.md) number (each digit expresses the three permission [bits](bit.md)) or as a 12 character string (containing the `r`/`w`/`x`/`-` characters). Well, let's not dig much deeper now.
+
+
+
+
+
+
 
 TODO: more more more
 
@@ -58,8 +68,9 @@ Here is a quick cheatsheet of the most common Unix utilities:
 | name          | function                                                                 | possible arguments (just some)              |
 | ------------- | ------------------------------------------------------------------------ | ------------------------------------------- |
 | alias         | create or display alias (nickname for another command)                   | alias=command                               |
+| [awk](awk.md) | text processing language (advanced)                                      |                                             |
 | bc            | interactive calculator                                                   |                                             |
-| c99           | C language compiler                                                      | file, -o (output file)                      |
+| c99           | [C](c.md) language compiler (advanced)                                   | file, -o (output file)                      |
 | cd            | change directory                                                         | directory name (`..` means back)            |
 | chmod         | change file mode                                                         | +x (execute), +w (write), +r (read), file   |
 | cmp           | compare files                                                            | -s (silent), file1, file2                   |
@@ -68,27 +79,47 @@ Here is a quick cheatsheet of the most common Unix utilities:
 | df            | report free space on disk                                                | -k (use KiB units)                          |
 | du            | estimate size of file (useful for directories)                           | -k (use KiB units), -s (only total), file   |
 | echo          | write out string (usually for scripts)                                   |                                             |
-| ed            | simple text editor                                                       |                                             |
+| [ed](ed.md)   | ed is the standard text editor                                           |                                             |
 | expr          | evaluate expression (simple calculator)                                  | expression (as separate arguments)          |
 | false         | return false value                                                       |                                             |
-| grep          | search for pattern in file                                               | pattern, file, -i (case insensitive)        |
+|[grep](grep.md)| search for pattern in file                                               | pattern, file, -i (case insensitive)        |
 | head          | show first N lines of a file                                             | -n (count), file                            |
 | kill          | terminate process or send a signal to it                                 | processid, -9 (kill), -15 (terminate)       |
 | ls            | list directory (shows files in current dir.)                             | -s (show file sizes in block)               |
-| man           | show manual page                                                         | topic                                       |
+| [man](man.md) | show manual page for topic                                               | topic                                       |
 | mkdir         | make directory                                                           | name                                        |
 | mv            | move (rename) file                                                       | -i (ask for rewrite), file, newfile         |
 | pwd           | print working directory                                                  |                                             |
 | rm            | remove files                                                             | -r (recursive, for dirs), -f (force)        |
-| sed           | stream editing utility (for replacing text etc.)                         | script, file                                |
-| sh            | shell (the command line interpreter, usually for scripting)              | -c (command string)                         |
+| [sed](sed.md) | stream editing util (replacing text etc.), see also [regex](regex.md)    | script, file                                |
+| [sh](sh.md)   | shell (the command line interpreter, usually for scripting)              | -c (command string)                         |
 | sort          | sort lines in file                                                       | -r (reverse), -u (unique), file             |
 | tail          | show last N lines of a file                                              | -n (count), file                            |
 | true          | return true value                                                        |                                             |
 | uname         | output system name and info                                              | -a (all, output everything)                 |
+| [vi](vi.md)   | advanced text editor                                                     |                                             |
 | wc            | word count (count characters or lines in file, can tell exact file size) | -c (character), -l (lines), file            |
 
-TODO: stdin/out/err, utils, shell, sh (running programs, ...), usual "workflows" (man pages, history, arrows, tab-completion, ...), often used commands, examples, permissions, variables and exit codes
+NOTES on the utilities:
+
+- Typically there are two ways of feeding input data to a utility: either by specifying a file to read from or by feeding the input on to the utility's standard input. This also applies to the output. Using standard input/output is a more "Unix" way as it allows us to chain the utlities with pipes, make one program feed its output to another an input.
+- Utilities try to follow common conventions so that it's easier to guess and remember what flags mean etc., for example `-h` is commonly a flag for getting help, `-o` is one for specifying output file etc.
+- Specific Unix systems will normally have more feature rich utilities, supporting additional flags and even adding new utilities. Check out manual pages on your system. You'll have to learn about common utils that aren't part of POSIX, e.g. [wget](wget.md), [ssh](ssh.md), [curl](curl.md), [sudo](sudo.md), [apt](apt.md) and more.
+
+Now on to a key feature of Unix: **pipelines and redirects**. [Processes](process.md) (running programs) on Unix have so called **standard input** (*stdin*) and **standard output** (*stdout*) -- these are streams of data (often textual but also binary) that the process takes on input and output respectively. There may also exist more streams (notably e.g. *standard error output*) but again, we'll ignore this now. When you run a program (utility etc.) in the command line, standard input will normally come from your keyboard and standard output will be connected to the terminal (i.e. you'll see it being written out in the command line). However sometimes you may want the program to take input from a file and/or to write its output to a file (imagine e.g. keeping [logs](logging.md)), or you may even want one program to feed its output as an input to another program! This is very powerful as you may combine the many small utilities into more powerful units. See also [Unix philosophy](unix_philosophy.md).
+
+Most commonly used redirections are done like this:
+
+- `command > file`: redirects output of *command* to file *file* (rewriting its content if there is any).
+- `command < file`: redirects input of *command* to come from *file*.
+- `command >> file`: output of *command* will be appended (added to the end) to *file*.
+
+
+
+**Example**:
+
+
+TODO: stdin/out/err, utils, shell, sh (running programs, ...), usual "workflows" (man pages, history, arrows, tab-completion, ...), often used commands, examples, permissions, variables and exit codes, wildcards
 
 ## See Also