Update

optimization.md

@@ -1,6 +1,6 @@
 # Optimization
 
-Optimization means making a program more efficient in terms of some computing resource usage or by any similar metric, commonly aiming for higher execution speed or lower memory usage (but also e.g. lower power consumption, lower [network](network.md) speed demand etc.) while preserving how the program functions externally; this can be done manually (by rewriting parts of your program) or automatically (typically by [compiler](compiler.md) when it's translating your program). Unlike [refactoring](refactoring.md), which aims primarily for a better readability of source code, optimization changes the inner behavior of the executed program to a more optimal one. Apart from optimizing programs/[algorithms](algorithm.md) we may also more widely talk about optimizing e.g. [data structures](data_structure.md), file formats, [hardware](hardware.md), [protocol](protocol.md) and so on.
+Optimization means making a [program](program.md) more efficient in terms of some computing resource usage or by any similar metric, commonly aiming for higher execution speed or lower memory usage (but also e.g. lower power consumption, lower [network](network.md) speed demand etc.) while preserving how the program functions externally; this can be done manually (by rewriting parts of your program) or automatically (typically by [compiler](compiler.md) when it's translating your program). Unlike [refactoring](refactoring.md), which aims primarily for a better readability of source code, optimization changes the inner behavior of the executed program to a more optimal one. Apart from optimizing programs/[algorithms](algorithm.md) we may also more widely talk about optimizing e.g. [data structures](data_structure.md), file formats, [hardware](hardware.md), [protocol](protocol.md) and so on.
 
 ## Manual Optimization
 
@@ -40,6 +40,7 @@ These are mainly for [C](c.md), but may be usable in other languages as well.
 - **[Single compilation unit](single_compilation_unit.md) (one big program without [linking](linking.md)) can help compiler optimize better** because it can see the whole code at once, not just its parts. It will also make your program compile faster.
 - Search literature for **algorithms with better [complexity class](complexity_class.md)** ([sorts](sorting.md) are a nice example).
 - For the sake of simple computers such as [embedded](embedded.md) platforms **avoid [floating point](floating_point.md)** as that is often painfully slowly emulated in software (and also inserts additional code, making the executable bigger). Use [fixed point](fixed_point.md), or at least offer it as a [fallback](fallback.md). This also applies to other hardware requirements such as [GPU](gpu.md) or sound cards: while such hardware accelerates your program on computers that have the hardware, making use of it may lead to your program being slower on computers that lack it.
+- **Consider various [overheads](overhead.md) in critical parts of code**. Overhead is an extra resource price for some kind of feature or mechanism you're using, for example a [function](function.md) call is a bit slower than directly embedding code that's not inside a function, an [OOP](oop.md) program will use more memory than a non-OOP program just because it uses OOP etc. That's why critical parts of code are often written in [assembly](assembly.md) -- to avoid overhead of higher level languages. However, you can (and should) minimize overhead in other ways also: for example in [3D graphics](3d_rendering.md) rendering a single 3D object has a certain overhead, so instead of rendering a scene with many separate 3D objects it's better to merge then all into a single big 3D object.
 - **Factoring out invariants from loops and early branching can create a speed up**: it's sometimes possible to factor things out of loops (or even long non-looping code that just repeats some things), i.e. instead of branching inside the loop create two versions of the loop and branch in front of them. This is a kind of space-time tradeoff. Consider e.g. `while (a) if (b) func1(); else func2();` -- if *b* doesn't change inside the loop, you can rewrite this as `if (b) while (a) func1(); else while (a) func2();`. Or in `while (a) b += c * d;` if *c* and *d* don't change (are invariant), we can rewrite to `cd = c * d; while (a) b += cd;`. And so on.
 - **Division can be replaced by multiplication by [reciprocal](reciprocal.md)**, i.e. *x / y = x * 1/y*. The point is that multiplication is usually faster than division. This may not help us when performing a single division by variable value (as we still have to divide 1 by *y*) but it does help when we need to divide many numbers by the same variable number OR when we know the divisor at compile time; we save time by precomputing the reciprocal before a loop or at compile time. Of course this can also easily be done with [fixed point](fixed_point.md) and integers!
 - **Consider the difference between logical and bitwise operators!** For example [AND](and.md) and [OR](or.md) boolean functions in C have two variants, one bitwise (`&` and `|`) and one logical (`&&` and `||`) -- they behave a bit differently but sometimes you may have a choice which one to use, then consider this: bitwise operators usually translate to only a single fast (and small) instruction while the logical ones usually translate to a branch (i.e. multiple instructions with potentially slow jumps), however logical operators may be faster because they are evaluated as [short circuit](short_circuit_eval.md) (e.g. if first operand of OR is true, second operand is not evaluated at all) while bitwise operators will evaluate all operands.