drummyfish/less_retarded_wiki
5
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update

Browse source
This commit is contained in:
Miloslav Ciz 2024-08-01 02:31:53 +02:00
parent df80221a15
commit 9fc5ae8d5b
16 changed files with 2136 additions and 1807 deletions
Download patch file Download diff file Expand all files Collapse all files

306
3d_rendering.md
View file

@ -275,9 +275,311 @@ Let's see what a typical pipeline might look like, similarly to something we mig
4. **Stage: vertex post processing**: Usually not programmable (no shaders here). Here the GPU does things such as [clipping](clipping.md) (handling vertices outside the screen space), primitive assembly and [perspective divide](perspective_divide.md) (transforming from [homogeneous coordinates](homogeneous coordinates.md) to traditional [cartesian coordinates](cartesian_coordinates.md)).
5. **Stage: [rasterization](rasterization.md)**: Usually not programmable, the GPU here turns triangles into actual [pixels](pixel.md) (or [fragments](fragment.md)), possibly applying [backface culling](backface_culling.md), [perspective correction](perspective_correction.md) and things like [stencil test](stencil_test.md) and [depth test](dept_test.md) (even though if fragment shaders are allowed to modify depth, this may be postpones to later).
6. **Stage: pixel/fragment processing**: Each pixel (fragment) produced by rasterization is processed here by a [pixel/fragment shader](fragment_shader.md). The shader is passed the pixel/fragment along with its coordinates, depth and possibly other attributes, and outputs a processed pixel/fragment with a specific color. Typically here we perform [shading](shading.md) and [texturing](texture.md) (pixel/fragment shaders can access texture data which are again stored in texture buffers on the GPU).
7. Now the pixels are written to the output buffer which will be shown on screen. This can potentially be preceded by other operations such as depth tests, as mentioned above.
7. Now the pixels are written to the output buffer which will be shown on screen. This can potentially be preceded by other operations such as depth tests, as mentioned above.
TODO: example of specific data going through the pipeline
### Complete Fucking Detailed Example Of Rendering A 3D Model By Hand
WORK IN PROGRESS
{ This turned out to be long as hell, sowwy. ~drummyfish }
This is an example of how two very simple 3D models would be rendered using the traditional triangle rasterization pipeline. Note that this is VERY simplified, it's just to give you an idea of the whole process, BUT if you understand this you will basically get an understanding of it all.
Keep in mind this all can be done just with [fixed point](fixed_point.md), [floating point](float.md) is NOT required.
First we need to say what conventions we'll stick to:
- We'll be using ROW VECTORS, i.e. we'll be writing vectors like [x,y,z]. Some people rather use column vectors, which then means their matrices are also transposed and they do multiplication in opposite direction etcetc. Watch out about this, it's quite important to know which convention you're using, because e.g. matrix multiplication is non-commutative (i.e. with matrices A * B does NOT generally equal B * A) and the order you need to multiply in depends on this convention, so be careful.
- Counterclockwise triangles are front facing, clockwise ones are back facing (invisible).
- We'll be using LEFT HANDED coordinate systems, i.e X axis goes right, Y goes up, Z goes forward (right handed system would be the same except Z would go the opposite way -- backwards). Watch out: some systems (for example OpenGL) use the other one. I.e. our coordinate system looks like this:
Y ^ _
| _/| Z
| _/
|_/
'-------> X
Now let's have a simple **3D model data** of a quad. Quad is basically just a square made of four vertices and two triangles, it will look like this:
*quadModel*:
v3________v2
| _/ |
| _/ |
| _/ |
|/_______|
v0 v1
In a computer this is represented with two arrays: vertices and triangles. Our vertices here are (notices all Z coordinates are zero, i.e. it is a 3D model but it's flat):
*quadVertices*:
```
v0 = [-1, -1, 0]
v1 = [ 1, -1, 0]
v2 = [ 1, 1, 0]
v3 = [-1, 1, 0]
```
And our triangles are (they are indices to the vertex array, i.e. each triangle says which three vertices from the above array to connect to get the triangle):
*quadTriangles*:
```
t0 = [0,1,2]
t1 = [0,2,3]
```
Note the triangles here (from our point of view) go counterclockwise -- this is called *winding* and is usually important because of so called **backface culling** -- the order of vertices basically determines which is the front side of the triangle and which is the back side, and rendering systems often just draw the front sides for efficiency (back faces are understood to be on the inside of objects and invisible).
Now the vertex coordinates of the model above are in so called **model space** -- these are the coordinates that are stored in the 3D model's file, it's the model's "default" state of coordinates. The concept of different spaces is also important because 3D rendering is a lot about just transforming coordinates between different spaces ("frames of reference"). We'll see that the coordinates in model space will later on be transformed to world space, view space, clip space, screen space etc.
OK, so next let's have 2 actual **3D model instances** that use the above defined 3D model data. Notice the difference between 3D model DATA and 3D model INSTANCE: instance is simply one concrete, specific model that has its own place in the global 3D world (world space) and will be rendered, while data is just numbers in memory representing some 3D geometry etc. There can be several instances of the same 3D data (just like in [OOP](oop.md) there can be multiple instances/objects of a class); this is very efficient because there can be just one 3D data (like a model of a car) and then many instances of it (like many cars in a virtual city). Our two model instances will be called *quad0* and *quad1*.
Each model instance has its own **transformation**. Transformation says where the model is placed, how it's rotated, how it's scaled and so on -- different 3D engines may offer different kind of transformations, some may support things like flips, skews, non-uniform scaling, but usually at least three basic transforms are supported: translation (AKA offset, shift, position), rotation and scale. The transforms can be combined, e.g. a model can be shifted, rotated and scaled at the same time. Here we'll just rotate *quad0* by 45 degrees (pi/4 radians) around Y (vertical) axis and translate *quad1* one unit to the right and 2 back:
*quad0*:
- translation = [0,0,0]
- rotation = [0,pi/4,0]
*quad1*:
- translation = [1,0,2]
- rotation = [0,0,0]
So now we have two model instances in our world. For rendering we'll also need a **camera** -- the virtual window to our world. Camera is also a 3D object: it doesn't have 3D model data associated but it does have a transformation so that we can, naturally, adjust the view we want to render. Camera also has additional properties like field of view (FOV), aspect ratio (we'll just consider 1:1 here), near and far distances and so on. Our camera will just be shifted two units back (so that it can see the first quad that stays at position [0,0,0]):
*camera*:
- translation = [0,0,-2]
- rotation = [0,0,0]
It is important to mention the **near and far planes**. Imagine a camera placed at some point in space, looking in certain direction: the volume of space that it sees creates a kind of infinitely long pyramid (whose "steepness" depends on the camera field of view) with its tip at the point where the camera resides. Now for the purpose of rendering we define two planes, perpendicular to the camera's viewing direction, that are defined by the distance from the camera: the near plane (not surprisingly the one that's the nearer of the two) and the far plane. For example let's say our camera will have the near plane 1 unit in front of it and the far plane 5 units in front of it. These planes will CUT OFF anything that's in front and beyond them, so that only things that are BETWEEN the two planes will be rendered (you can notice this in games with render distance -- if this is not cleverly masked, you'll see things in the distance suddenly cut off from the view). These two planes will therefore CUT OFF the viewing pyramid so that now it's a six sided, finite-volume shape that looks like a box with the front side smaller than the back side. This is called the **view frustum**. Nothing outside this frustum will be rendered -- things will basically be sliced by the sides of this frustum.
You may ask WHY do we establish this frustum? Can't we just leave the near and far planes out and render "everything"? Well, firstly it's obvious that having a far cutoff view distance can help performance if you have a very complex model, but this is not the main reason why we have near and far planes. We basically have them for mathematical convenience -- as we'll see for example, perspective mapping means roughly "dividing by distance from camera" and if something was to be exactly where the camera is, we'd be dividing by zero! Attempting to render things that are just very near or on the back side of the camera would also do very nasty stuff. So that's why we have the near plane. In theory we might kind of get away with not having a strict far plane but it basically creates a nice finite-volume that will e.g. allow us to nicely map depth values for the z-buffer. Don't worry if this doesn't make much sense, this is just to say there ARE good reasons for this stuff.
Now let's summarize what we have with this top-down view of our world (the coordinates here are now *world space*):
```
Z
:
- - - - - - - 3 + - - - - - - - - far plane
:
:
2 +-----*------ quad2
:
:
1 +
: quad1
0 : _/
.|.....|.....*/....|.....|. X
-2 -1 _/: 1 2
. / : .
- - - - -'.=====:=====.'- - - - - near plane
'. : .'
'. : .'
-2 '*' camera
:
:
```
NOW actually let's see how to in fact render this. The big picture overview is this:
1. Get the model instances from model space to world space, i.e. transform their vertex coordinates according to each instance's transformation.
2. Get the model instances from world space to view space (AKA camera space). View space is the coordinate system of the camera in which the camera sits in the origin (poinr [0,0,0]) and is looking straight forward along the positive Z axis direction.
3. Get the model instances from view space to clip space. This applies perspective (deforms objects so that the further away ones become smaller as by distance) and transform everything in the view frustum to a nice cube of fixed size and with walls aligned with principal axes (unlike view frustum itself).
4. Clip everything outside the clip space, i.e. throw away everything outside the box we've got. If some things (triangles) are partially in and partially out, we CLIP them, i.e. we literally cut them into several parts and throw away the parts that aren't in (some simpler renderers just do simpler stuff like throw away anything that sticks outside or just force push the vertices inside but it will look a bit awkward).
5. Get everything from clip space into screen space, i.e. to actual pixel coordinates of the screen we are rendering to.
6. Rasterize the triangles of the models between the points we have mapped to the screen now, i.e. literally fill all the triangle pixels so that we get the final image.
As seen this involves doing many transformations between different spaces. We do these using **[linear algebra](linear_algebra.md)**, i.e. with **[vectors](vector.md)** and **[matrices](matrix.md)**. Key things here are:
- We can represent all the transformations that we need using matrices.
- Every such transformation (translation, rotation, scale, ...) can be represented with one 4x4 matrix.
- To apply a transformation to a model we simply multiply each of its vertices with the transformation matrix. So if we want to rotate a model by 30 degrees, we make a matrix that represents this rotation and just multiply all the model's vertices and it's done. Pretty elegant!
- AMAZING stuff: any number of these transformations combined in ANY order can still be represented just by a single 4x4 matrix! You just take the matrix of each of the transformations, multiply them together and you get a matrix that just does all these transformation at once. Duuuude what? Yeah that's right, this is extremely awesome, isn't it? We can basically create a single matrix that combines in it all the above mentioned rendering steps and it will just do everything. This is not only elegant but also very efficient (instead of just moving, rotating and scaling points there and back many times we simply perform ONE matrix multiplication for each vertex and that's it).
You HAVE TO learn how to multiply vector with matrix and matrix with matrix (it's not hard) else you will understand nothing now.
BIG BRAIN MOMENT: **[homogeneous coordinates](homogeneous_coordinates.md)**. Please DO NOT ragequit, it looks complicated as hell (it is a little bit) but it makes sense in the end, OK? We have to learn what homogeneous coordinates are because we need them to be able to do all the awesome matrix stuff described above. In essence: in 3D space we can perform linear transformations with 3x3 matrices -- linear operations are for example scaling and rotation, BUT some, most importantly translation (shifting and object, which we absolutely NEED), are not linear (but rather [affine](affine.md)) so they cannot be performed by a 3x3 matrix. But it turns out that if we use special kind of coordinates, we CAN do affine 3D transformations with 4x4 matrices, OK? These special coordinates are homogeneous coordinates, and they simply add one extra coordinate, *w*, to the original *x*, *y* and *z*, while it holds that that multiplying all the *x*, *y*, *z* and *w* components by the same number does nothing with the point they represent. Let's show it like this:
If we have a 3D point [1,2,3], in homogeneous coordinates we can represent it as [1,2,3,1] or [2,4,6,2] or [3,6,9,3] and so on. That's easy no? So we will ONLY add an additional 1 at the end of our vertex coordinates and that's basically it.
Let's start doing this now!
Firstly let us transform *quad0* from model space to world space. For this we construct so called **model matrix** based on the transformation that the model instance has. Our *quad0* is just rotated by pi/4 radians and for this the matrix will look like this (you don't have to know why, you usually just look up the format of the matrix somewhere, but you can derive it, it's EZ):
*quad0* model matrix:
```
| cos(A) 0 sin(A) 0| |0.7 0 0.7 0|
Mm0 = | 0 1 0 0| ~= |0 1 0 0|
|-sin(A) 0 cos(A) 0| |-0.7 0 0.7 0|
| 0 0 0 1| |0 0 0 1|
```
Let's see if this works, we'll try to multiply the first model vertex with this matrix (notice we add 1 at the end of the vertex, to convert it to homogeneous coordinates):
```
|0.7 0 0.7 0|
|0 1 0 0|
|-0.7 0 0.7 0|
|0 0 0 1|
v0 * Mm0 = [-1, -1, 0, 1] [-0.7 -1 -0.7 1] <-- result
```
So from [-1,-1,0] we got [-0.7,-1,-0.7] -- looking at the top-down view picture above this seem pretty correct (look at the coordinates of the first vertex). Try it also for the rest of the vertices. Now for the model matrix of *quad1* (again, just look up what translation matrix looks like):
*quad1* model matrix:
```
|1 0 0 0|
Mm1 = |0 1 0 0|
|0 0 1 0|
|1 0 2 1|
```
Here you can even see that multiplying a vector by this will just add 1 to *x* and 2 to *z*, right? Again, try it.
NEXT, the **view matrix** (matrix that will transform everything so that it's "in front of the camera") will basically just do the opposite transformation of that which the camera has. Imagine if you shift a camera 1 unit to the right -- that's as if the camera stands still and everything shifts 1 unit to the left. Pretty logical. So our view matrix looks like this (notice it just pushes everything by 2 to the front):
*view matrix*:
```
|1 0 0 0|
Mv = |0 1 0 0|
|0 0 1 0|
|0 0 2 1|
```
Then we'll need to apply perspective and get everything to the clip space. This will be done with so called **projection matrix** which will in essence make the *x* and *y* distances be divided by the *z* distance so that further away things will shrink and appear smaller. You can derive the view matrix, its values depend on the field of view, near and far plane etc., here we'll just copy paste numbers into a "template" for the projection matrix, so here it is:
*projection matrix* (*n* = near plane distance, *f* = far plane distance, *r* = right projection plane distance = 1, *t* = top projection plane distance = 1):
```
|n/r 0 0 0| |1 0 0 0|
Mp = |0 n/t 0 0| = |0 1 0 0|
|0 0 (f+n)/(f-n) 1| |0 0 3/2 1|
|0 0 -2*f*n/(f-n) 0| |0 0 -5/2 0|
```
This matrix will basically make the points so that their *w* coordinates will be such that when in the end we divide all components by it (to convert them back from [homo](gay.md)geneous coordinates), we'll get the effect of the perspective (it's basically the "dividing by distance from the camera" that perspective does). That is what the homogeneous coordinates allow us to do. To visually demonstrate this, here is a small picture of how it reshapes the view frustum into the clip space box (it kind of "squishes" all in the back of the frustum pyramid and also squeezes everything to shape it into that little box of the clipping space, notice how the further away objects became closer together -- that's the perspective):
```
___________________ far plane _____________
\ A B C / | ABC |
\ / | |
\ D E F / | D E F |
\ / | |
\G H I/ |G H I|
\_______/ near plane |_____________|
: : : :
: : : screen :
: : : :
* camera
```
At this point we have the matrices of the individual transforms, but as we've said, we can combine them into a single matrix. First let's combine the view matrix and projection matrix into a single view-projection matrix by multiplying the two matrices (WATCH OUT: the order of multiplication matters here! It defines in which order the transformations are applied):
*view-projection matrix*:
```
|1 0 0 0|
Mvp = Mv * Mp = |0 1 0 0|
|0 0 3/2 1|
|0 0 1/2 2|
```
The rendering will begin with *quad0*, we'll combine its model matrix and the view-projection matrix into a single uber matrix that will just do the whole transformation for this model instance:
*quad0 model-view-projection matrix*:
```
|0.7 0 21/20 0.7|
Mm0vp = Mm0 * Mvp = |0 1 0 0 |
|-0.7 0 21/20 0.7|
|0 0 1/2 2 |
```
Now we'll just transform all of the model's vertices by multiplying with this matrix, and then we'll convert back from the homogeneous coordinates to "normal" coordinates by dividing all components by *w* (AKA "perspective divide") like this:
```
v0: [-1, -1, 0, 1] (matrix multiplication) => [-0.7, -1, -0.55, 1.3] (w divide) => [-0.53, -0.76, -0.43]
v1: [ 1, -1, 0, 1] (matrix multiplication) => [ 0.7, -1, 1.55, 2.7] (w divide) => [ 0.26, -0.37, 0.57]
v2: [ 1, 1, 0, 1] (matrix multiplication) => [ 0.7, 1, 1.55, 2.7] (w divide) => [ 0.26, 0.37, 0.57]
v3: [-1, 1, 0, 1] (matrix multiplication) => [-0.7, 1, -0.55, 1.3] (w divide) => [-0.53, 0.76, -0.43]
```
And let's also do this for *quad1*.
*quad1 model-view-projection matrix*:
```
|1 0 0 0|
Mm1vp = Mm1 * Mvp = |0 1 0 0|
|0 0 3/2 1|
|0 0 7/2 4|
```
and
```
v0: [-1, -1, 0, 1] (matrix multiplication) => [-1, -1, 3.5, 4] (w divide) => [-0.25, -0.25, 0.87]
v1: [ 1, -1, 0, 1] (matrix multiplication) => [ 1, -1, 3.5, 4] (w divide) => [ 0.25, -0.25, 0.87]
v2: [ 1, 1, 0, 1] (matrix multiplication) => [ 1, 1, 3.5, 4] (w divide) => [ 0.25, 0.25, 0.87]
v3: [-1, 1, 0, 1] (matrix multiplication) => [-1, 1, 3.5, 4] (w divide) => [-0.25, 0.25, 0.87]
```
Hmmm mkay let's draw the transformed points to an X/Y grid:
```
Y
|
[-1,1]|_______________:_______________[1,-1]
| |
| v3 +--.... |
| | '''---+ v2 |
| | .'| |
| |v3 +-----:--|+ v2 |
| | | .' ..|| |
--| | | :..' || |--
| | | '' || |
| |v0 +'-------|+ v1 |
| | : | |
| |.' ...---+ v1 |
| v0 +--'''' |
|_______________________________|___X
[-1,-1] : [1,-1]
```
HOLY SHIT IT'S 3D!!1! [Magic](magic.md)! In the front we see *quad0*, rotated slightly around the vertical (Y) axis, behind it is *quad1*, non-rotated but smaller because it's further away. This looks very, very good! We're almost there.
Also notice that the points -- now nicely projected onto a 2D X/Y plane -- still have 3 coordinates, i.e. they retain the Z coordinate which now holds their depth, or distance from the camera projection plane kind of. This depth is now in range from -1 (near plane) to 1 (far plane). The depth will be important in actually drawing pixels, to decide which are more in the front and therefore visible (this is the problem of *visibility*). The depth value can also be used for cool effects like the distance fog and so on.
The work up until now -- i.e. transforming the vertices with matrices -- is what **vertex shaders** do. Now comes the **rasterization** part -- here we literally draw triangles (as in individual pixels) between the points we have now mapped on the screen. In systems such as OpenGL This is usually done automatically, you don't have to care about rasterization, however you will have to write the algorithm if you're writing e.g. your own [software renderer](sw_rendering.md). Triangle rasterization isn't trivial, it has to be not only efficient but also deal with things such as not leaving holes between adjacent triangles, interpolating triangle attributes and so on. We won't dive deeper into this, let's just suppose we have a rasterization algorithm now. For example rasterizing the first triangle of *quad0* may look like this:
```
_______________________________
| |
| +--.... |
| | '''---# |
| | .## |
| | +-----####+ |
| | | .#####| |
| | | :######| |
| | | ########| |
| | ##########+ |
| | ########### |
| | ############ |
| ####### |
|_______________________________|
```
During this rasterization process the Z coordinates of the mapped vertices are important -- the rasterizer [interpolates](interpolation.md) the depth at the vertices, so it knows the depth of each pixel it creates. This depth is written into so called **z-buffer** (AKA depth buffer) -- basically an off-screen buffer that stores one numeric value (the depth) for each pixel. Now when let's say the first triangle of *quad1* starts to be rasterized, the algorithm compares the rasterized pixel's depth to that stored on the same position in the z-buffer -- if the z-buffer value is small, the new pixel mustn't be drawn because it's covered by a previous drawn pixel already (here probably that of the triangle shown in the picture).
So the rasterization algorithm just shits out individual pixels and hands them over to the **fragment shader** (AKA pixel shader). Fragment is a program that just takes a pixel and says what color it should have (basically) -- this is called **[shading](shading.md)**. For this the rasterizer also hands over additional info to the fragment shader which may include: the X/Y coordinates of the pixel, its interpolated depth (that used in z-buffer), vertex normals, ID of the model and triangle and, very importantly, the **[barycentric coordinates](barycentric.md)**. These are three-component coordinates that say where exactly the pixel is in the triangle. These are used mainly for **texturing**, i.e. if the model we're rendering has a texture map (so called UV map) and a bitmap image (texture), the fragment shader will use the UV map and barycentric coords to compute the exact pixel of the texture that the rasterized pixel falls onto AND this will be the pixel's color. Well, not yet actually, there are more things such as **lighting**, i.e. determining what brightness the pixel should have depending on how the triangle is angled towards scene lights (for which we need the normals), how far away from them it is, what colors the lights have etcetc. And this is not nearly all, there are TONS and tons of other things, for example the interpolation done by rasterizer has to do **perspective correction** (linearly interpolating in screen space looks awkward), then there is texture filtering to prevent [aliasing](aliasing.md) (see e.g. [mipmapping](mipmap.md), transparency, effects like bump mapping, environment mapping, screen space effects, stencil buffer etcetc. -- you can read whole books about this. That's beyond the scope of this humble tutorial -- in simple renderers you can get away with ignoring a lot of this stuff, you can just draw triangles filled with constant color, or even just draw lines to get a wireframe renderer, all is up to you. But you can see it is a bit [bloated](bloat.md) if everything is to be done correctly -- don't forget there also exist other ways of rendering, see for example [raytracing](raytracing.md) which is kind of easier.
## See Also