Update

turing_machine.md

@@ -1,12 +1,14 @@
 # Turing Machine
 
-Turing machine is a [mathematical](math.md) [model](model.md) of a [computer](computer.md) which works in a quite simple way but has nevertheless the full computational power that's possible to be achieved. Turing machine is one of the most important tools of theoretical [computer science](compsci.md) as it presents a basic [model of computation](model_of_computation.md) (i.e. a mathematical system capable of performing general mathematical calculations) for studying computers and [algorithms](algorithm.md) -- in fact it stood at the beginning of theoretical computer science when [Alan Turing](alan_turing.md) invented it in 1936 and used it to mathematically [prove](proof.md) essential things about computers; for example that their computational power is inevitably limited (see [computability](computability.md)) -- he showed that even though Turing machine has the full computational power we can hope for, there exist problems it is incapable of solving (and so will be any other computer equivalent to Turing machine, even human [brain](brain.md)). Since then many other so called **[Turing complete](turing_completeness.md)** systems (systems with the exact same computational power as a Turing machine) have been invented and discovered, such as [lambda calculus](lambda_calculus.md) or [Petri nets](petri_net.md), however Turing machine still remains not just relevant, but probably of greatest importance, not only historically, but also because it is similar to physical computers in the way it works.
+Turing machine is a [mathematical](math.md) [model](model.md) of a [computer](computer.md) which works in a quite simple way but has nevertheless the full computational power, i.e. it is able to perform any possible computation which could be achieved by any other means. Turing machine is one of the most important tools of theoretical [computer science](compsci.md) as it presents a basic [model of computation](model_of_computation.md) (i.e. a mathematical system capable of performing general mathematical calculations) for studying computers and [algorithms](algorithm.md) -- in fact it stood at the beginning of theoretical computer science when [Alan Turing](alan_turing.md) invented it in 1936 and used it to mathematically [prove](proof.md) essential things about computers; for example that their computational power is inevitably limited (see [computability](computability.md)) -- he showed that even though Turing machine has the full computational power we can hope for, there exist problems it is incapable of solving (and so will be any other computer equivalent to Turing machine, even human [brain](brain.md)). Since then many other so called **[Turing complete](turing_completeness.md)** systems (systems with the exact same computational power as a Turing machine) have been invented and discovered, such as [lambda calculus](lambda_calculus.md) or [Petri nets](petri_net.md), however Turing machine still remains not just relevant, but probably of greatest importance, not only historically, but also because it is similar to physical computers in the way it works.
+
+NOTE: It seems the term "computational power" is sometimes used as a measure of the speed of computation, but here we mean something different -- computational power to us is just the pure ABILITY to compute something in finite time. Turing machines can also be used to theoretically study the "speed" of computation (see [computational complexity](computational_complexity.md)), but in this introduction we are merely interested in what CAN be computed, no matter in how many steps or how much memory it will require (as long as it's not [infinitely](infinity.md) many). I.e. when we say a Turing machine can in theory run for example the latest GTA game, we don't mean it would be efficient, fast etc. We are talking pure theoretical possibility.
 
 The advantage of a Turing machine is that it's firstly very [simple](minimalism.md) (it's basically a finite state automaton operating on a [memory](memory.md) tape), so it can be mathematically grasped very easily, and secondly it is, unlike many other systems of computations, actually similar to real computers in principle, mainly by its sequential instruction execution and possession of an explicit memory tape it operates on (equivalent to [RAM](ram.md) in traditional computers). However note that a **pure Turing machine cannot exist in reality** because there can never exist a computer with infinite amount of memory which Turing machine possesses; computers that can physically exist are really equivalent to [finite state automata](finite_state_automaton.md), i.e. the "weakest" kind of systems of computation. However we can see our physical computers as [approximations](approximation.md) of a Turing machine that in most relevant cases behave the same, so we do tend to theoretically view computers as "Turing machines with limited memory".
 
 { Although purely hypothetically we could entertain an idea of a computer that's capable of manufacturing itself a new tape cell whenever one is needed, which could then have something like unbounded memory tape, but still it would be limited at least by the amount of matter in observable universe. ~drummyfish }
 
-In Turing machine data and program are separated (data is stored on the tape, program is represented by the control unit), i.e. it is closer to [Harvard architecture](harvard_architecture.md) than [von Neumann architecture](von_neumann_architecture.md).
+In the "vanilla" Turing machine data and program are separated (data is stored on the tape, program is represented by the control unit), i.e. it is closer to [Harvard architecture](harvard_architecture.md) than [von Neumann architecture](von_neumann_architecture.md); however (since with Turing machine we can program anything we could on any other computer) using this basic concept of a Turing machine we can construct so called *universal Turing machine*, i.e. basically an [interpreter](interpreter.md) of Turing machines, which runs a program (the Turing machine being interpreted) that's stored in the same memory as the data.
 
 **Is there anything computationally more powerful than a Turing machine?** Well, yes, but it's just kind of "mathematical fantasy". See e.g. [oracle machine](oracle_machine.md) which adds a special "oracle" device to a Turing machine to make it [magically](magic.md) solve undecidable problems.
 
@@ -201,6 +203,16 @@ halted
 
 Turing machines can be used to define computable [formal languages](formal_language.md). Let's say we want to define language *L* (which may be anything such as a programming language) -- we may do it by programming a Turing machine that takes on its input a string (a word) and outputs "yes" if that string belongs to the language, or "no" if it doesn't. This is again useful for the theory of [decidability](decidability.md)/[computability](computability.md).
 
+### How Can Something So Simple Have The Maximum Computational Power And How Do We Know It Is So?
+
+Well, at first people are usually surprised that Turing machine can "compute anything that any other computer", but given that we don't care about efficiency at all it can be shown with a Turing machine we can emulate all that's needed to perform any [algorithm](algorithm.md): we can have a sequence of instructions (just states transitioning to other states unconditionally), branching (conditional state transitions) and loops (transitioning to previous states) -- that's all we really need. Curiously it turns out that many systems, such as some card games for example, have this property without even having been intended to, i.e. it's not rare for a randomly encountered system to be Turing complete (and it's usually proven to be Turing complete by showing how it can emulate a Turing machine). Even if it would be a challenge to make a big program with Turing machine, we can just start with simple things like basic arithmetic operations, data types, etc. and eventually we reimplement any fancy programming language. If we accept that let's say the [C](c.md) programming language (or any other language we can imagine) can "program anything", we can show (in a long and boring way, but you can try it) that C can really be implemented with a Turing machine and so it can compute anything that can be computed with C. All the fancy things like data types, control structures and preprocessors are just [sugar](sugar.md) made from very simple basics.
+
+There is so called *Church–Turing thesis* which is basically a claim -- unprovable but taken as truth, or maybe rather a definition -- that anything we would intuitively call an "[algorithm](algorithm.md)" (a finite series of exact, simple steps leading to computing some result) can always be performed by some Turing machine, or maybe it's better to say in reverse: we say that Turing machine DEFINES what an algorithm is and kind of dare anyone to come up with something that would shake this claim, like finding out that something can be computed only in a way that's fundamentally impossible to be recorded as a series of exact steps. In fact Turing machine was created because we didn't have a rigorous, mathematically precise definition of what an algorithm is, before this we could only intuitively talk about the "series of simple steps" without knowing what a "simple" or "step" can really mean, what kind of resources we can use and so on, so Alan Turing created the machine, and indeed when we examine it in detail, the claim that it embodies the definition of an algorithm becomes "obviously true", so people basically agree that *algorithm* equals a *Turing machine* and that nothing more computationally powerful can exist, save for maybe a few religious fanatics who say there is something [magical](free_will.md) in human brain that's more than pure computation and which cannot be imitated by algorithms, but these are absolutely irrational beliefs. Holding on to [mathematics](math.md) we have to accept that any computable [function](function.md) must by definition be computable by performing a number of calculation steps and this can always be done by a Turing machine. If a human with pen and pencil can compute it, a Turing machine can too.
+
+### How Does It Relate To A Practical Computer?
+
+Quite a lot actually -- Turing machines, unlike some equivalent models of computation, are the basis of practically every electronic [digital](digital.md) computer you will encounter; we might even say today's computers are just very pimped up, greatly [optimized](optimization.md) fancy Turing machines with added [input/output](io.md) devices, also having the practical limitation of only having finite memory. We may see the computer's [CPU](cpu.md) as the finite state machine -- the control unit -- i.e. the hardcoded basic program that's burned in the [hardware](hardware.md); this control unit is actually hardwired to implement the universal Turing machine, i.e. to interpret a program that's stored in memory, which achieves the [programmability](programming.md) of the computer. How to effectively encode programs so that they are small and fast to interpret is a matter of designing [instruction set architectures](isa.md). Everything around this, like multiple CPU cores, [GPUs](gpu.md), [caches](cache.md), [buses](bus.md) etc. are just things to make it all faster and more effective.
+
 ## See Also
 
 - [brainfuck](brainfuck.md)