This commit is contained in:
Miloslav Ciz 2024-08-19 21:04:41 +02:00
parent ac9725b356
commit 3a465aea74
21 changed files with 1985 additions and 1866 deletions

View file

@ -1,14 +1,14 @@
# Dynamic Programming
Dynamic programming is a [programming](programming.md) technique that can be used to make many [algorithms](algorithm.md) more efficient (usually meaning faster). It can be seen as an [optimization](optimization.md) technique that works on the principle of repeatedly breaking given problem down into smaller subproblems and then solving one by one from the simplest and remembering already calculated results that can be reused later.
Dynamic programming is a [programming](programming.md) technique that allows us to increase efficiency of certain types of [algorithms](algorithm.md) (efficiency usually meaning faster execution). It can be seen as an [optimization](optimization.md) technique that works on the principle of repeatedly breaking given problem down into smaller subproblems and then solving one by one from the simplest and remembering already calculated results that can be reused later.
It is frequently contrasted to the *[divide and conquer](divide_and_conquer.md)* (DAC) technique which at the first sight looks similar but is in fact quite different. DAC also subdivides the main problem into subproblems, but then solves them [recursively](recursion.md), i.e. it is a top-down method. DAC also doesn't remember already solved subproblem and may end up solving the same problem multiple times, wasting computational time. Dynamic programming on the other hand starts solving the subproblems from the simplest ones -- i.e. it is a **bottom-up** method -- and remembers solutions to already solved subproblems in some kind of a [table](lut.md) which makes it possible to quickly reuse the results if such subproblem is encountered again. The order of solving the subproblems should be made such as to maximize the efficiency of the algorithm.
It is frequently contrasted with the *[divide and conquer](divide_and_conquer.md)* (DAC) method which at the first sight looks similar but is in fact quite different. DAC also subdivides the main problem into subproblems, but then solves them [recursively](recursion.md) and separately, i.e. it is a top-down method. DAC also doesn't remember already solved subproblem and may end up solving the same problem multiple times, wasting computation. Dynamic programming on the other hand starts solving the subproblems from the simplest ones -- i.e. it is a **bottom-up** method -- and remembers solutions to already solved subproblems in some kind of a [table](lut.md) which enables quick reusing of the results should the same subproblem be encountered again. The order of solving the subproblems should be chosen so as to maximize the efficiency of this approach.
It's not the case that dynamic programming is always better than DAC, it depends on the situation. Dynamic programming is effective **when the subproblems overlap** and so the same subproblems WILL be encountered multiple times. But if this is not the case, DAC can easily be used and memory for the look up tables will be saved.
It is NOT the case that dynamic programming would always beat DAC, all depends on the situation. Dynamic programming is effective **when the subproblems overlap** and thus the same subproblems WILL be encountered multiple times -- this is the fact that dynamic programming exploits. Should this not be the case -- i.e. if we are solving a problem that doesn't exhibit this property -- DAC should be used instead.
## Example
Let's firstly take a look at the case when divide and conquer is preferable. This is for instance the case with many [sorting](sorting.md) algorithms such as [quicksort](quicksort.md). Quicksort recursively divides parts of the array into halves and sorts each of those parts: sorting each of these parts is a different subproblem as these parts (at least mostly) differ in size, elements and their order. The subproblems therefore don't overlap and applying dynamic programming makes little sense.
For starters let's view a case when divide and conquer would be preferable: this is true for instance about many [sorting](sorting.md) algorithms including [quicksort](quicksort.md) and others. Quicksort [recursively](recursion.md) splits parts of the array into halves and sorts each one separately: sorting each part is a different subproblem given the parts (at least generally) differ in size, elements and their order. The subproblems therefore don't overlap and applying dynamic programming makes little sense.
But if we tackle a problem such as computing *N*th [Fibonacci number](fibonacci_number.md), the situation changes. Considering the definition of *N*th Fibonacci number as a *"sum of N-1th and N-2th Fibonacci numbers"*, we might naively try to apply the divide and conquer method:
@ -21,7 +21,7 @@ int fib(int n)
}
```
But we can see this is painfully slow as calling `fib(n - 2)` computes all values already computed by calling `fib(n - 1)` all over again, and this inefficiency additionally appears inside these functions recursively. Applying dynamic programming we get a better code:
However we make the observation that this is painfully slow due to the fact that calling `fib(n - 2)` computes all values already computed by calling `fib(n - 1)` all over again, and this inefficiency additionally appears inside these functions recursively. Applying dynamic programming we get a better code:
```
int fib(int n)
@ -42,4 +42,4 @@ int fib(int n)
}
```
We can see the code is longer, but it is faster. In this case we only need to remember the previously computed Fibonacci number (in practice we may need much more memory for remembering the partial results).
Now the code is longer, but it is faster. In this specific case we only need to remember the previously computed Fibonacci number (in practice we may need much more memory for remembering the partial results).