Lecture 13: 3D graphics

# Creación de Videojuegos

### 3D Graphics

---

# Triangles

---

# Some definitions

* Vertex (plural Vertices): A point in 3D space
  
  * Triangle: Three vertices
  
  * Face: The "front" side of a triangle
  
  * Normal: A vector that points "away" from the face

---

# Why triangles?

* Triangles are always flat
  
  * Flat makes it easier to calculate lighting, color, etc.
  
  * Triangles are convex
  
  * Convex makes it easier to interpolate between the vertices
  
  * Any polygon can be divided into triangles 
  
---

# An Example

Consider the triangle 
$$
a = (2,2,3), 
b = (3,3,4), 
c = (3,1,3)
$$
with the normal vector 
$$
\vec{n} = \begin{pmatrix} 
\frac{-1}{\sqrt{6}} \\\\ 
\frac{-1}{\sqrt{6}} \\\\
\frac{2}{\sqrt{6}}
\end{pmatrix}
$$
and a point 
$$
p = (3\sqrt{6},0, 2\sqrt{6})
$$

## Does the point p lie in front of or behind the triangle?
  
---

# Color

* We can assign a color to a triangle
  
  * But what if we want to have different colors?
  
  * Ideally, we want a gradual change
  
  * Idea: Assign color to the vertices and interpolate
  
  * It's such a good idea, 3D libraries have it built-in

---

# Interpolation
  
Halfway between two values:

$$ p = \frac{r + s}{2} = \frac{1}{2} r + \frac{1}{2}s$$

Closer to r than to s

$$ p = \frac{2}{3} r + \frac{1}{3} s$$

or, generally:

$$ p = \theta \cdot r + (1- \theta) \cdot s$$

---

# Interpolation
  
What if we have more than two values?

$$ p = \theta \cdot r + \phi \cdot s + \psi \cdot t$$

with

$$\theta + \phi + \psi = 1$$

Note: r, s, and t don't have to be numbers, they could also be vectors! In particular, they can be the vertices of a triangle. Every point in the triangle can then be represented as

$$p = (\theta,\phi,\psi)$$

---

# Barycentric coordinates

<svg preserveAspectRatio="xMidYMid meet" version="1.0" id="svg2" docname="TriangleBarycentricCoordinates.svg" width="1344.0642" height="2011.8257" font-size="23">
  <metadata id="metadata65">
    <RDF id="RDF4">
      <Work about="" id="Work6">
        <format id="format8">image/svg+xml</format>
        <type resource="http://purl.org/dc/dcmitype/StillImage" id="type10"/>
      </Work>
    </RDF>
  </metadata>
  <desc id="desc4">Barycentric Coordinates in a triangle</desc>
  <g id="g2526" transform="matrix(0.9,0,0,0.9,0,0)">
    <path id="path6" d="M 100,50 L 600,50 L 350,483.0127 L 100,50 z" style="fill:none;stroke:#000000;stroke-width:3"/>
    <circle id="circle8" r="4" cy="194.33757" cx="350" rx="4" ry="4"/>
    <circle id="circle10" r="4" cy="50" cx="100" rx="4" ry="4"/>
    <circle id="circle12" r="4" cy="50" cx="600" rx="4" ry="4"/>
    <circle id="circle14" r="4" cy="483.0127" cx="350" rx="4" ry="4"/>
    <circle id="circle16" r="4" cy="50" cx="350" rx="4" ry="4"/>
    <circle id="circle18" r="4" cy="266.50635" cx="475" rx="4" ry="4"/>
    <circle id="circle20" r="4" cy="266.50635" cx="225" rx="4" ry="4"/>
    <circle id="circle22" r="4" cy="266.50635" cx="350" rx="4" ry="4"/>
    <circle id="circle24" r="4" cy="158.25317" cx="412.5" rx="4" ry="4"/>
    <circle id="circle26" r="4" cy="158.25317" cx="287.5" rx="4" ry="4"/>
    <g id="g28" style="stroke:#0000ff;stroke-width:0.5;stroke-dasharray:3, 3">
      <path id="path30" d="M 100,50 L 475,266.50635"/>
      <path id="path32" d="M 600,50 L 225,266.50635"/>
      <path id="path34" d="M 350,483.0127 L 350,50"/>
      <path id="path36" d="M 475,266.50635 L 225,266.50635"/>
      <path id="path38" d="M 225,266.50635 L 350,50"/>
      <path id="path40" d="M 350,50 L 475,266.50635"/>
    </g>
    <text id="text42" dy="-5" dx="-5" y="50" x="100" style="text-anchor:end">(1,0,0)</text>
    <text id="text44" dy="-5" dx="5" y="50" x="600">(0,1,0)</text>
    <text id="text46" dy="20" y="485.9617" x="350" style="text-anchor:middle">(0,0,1)</text>
    <text id="text48" dy="-5" y="49.016991" x="350" style="text-anchor:middle">(1/2,1/2,0)</text>
    <text id="text50" dy="20" y="266.50635" x="225" style="text-anchor:end">(1/2,0,1/2)</text>
    <text id="text52" dy="20" y="266.50635" x="475">(0,1/2,1/2)</text>
    <text id="text54" dy="20" y="269.45535" x="350" style="text-anchor:middle">(1/4,1/4,1/2)</text>
    <text id="text56" dy="-5" y="155.30417" x="434.12604" style="text-anchor:middle">(1/4,1/2,1/4)</text>
    <text id="text58" dy="-5" y="155.30417" x="264.89096" style="text-anchor:middle">(1/2,1/4,1/4)</text>
    <text id="text60" dy="20" y="197.28658" x="350" style="text-anchor:middle">(1/3,1/3,1/3)</text>
  </g>
</svg>

---

# Interpolation
  
What can we do with this representation?

$$p = (\theta,\phi,\psi)$$

Calculate color values!

$$c(p) = \theta \cdot c(r) + \phi \cdot c(s) + \psi \cdot c(t)$$

---

# A colored triangle

---

# An example

Consider the triangle 
$$
a = (1,0,0), 
b = (1,1,0), 
c = (0,1,0)
$$

## If a has the color (99,0,0), b has the color (0,39,0), and c has the color (0,81,81), what color does the barycenter of the triangle have using linear interpolation? Which color does the point that is twice as close to a as to b and c (in the barycentric sense) have?

---

# Normals

* Normals are (unit) vectors that point away from a face
  
  * They are used for lighting (we'll discuss how next week)
  
  * What does "away" mean?
  
  * Simplest case: perpendicular to the triangle 
  
  * What if we have adjacent triangles?
  
  * We can assign normals to each vertex and interpolate, just as with colors
 
---

# Textures

* Colored triangles are boring!
  
  * What if we want to show something like wood?
  
  * Use an image as the texture 
  
  * Assign texture (uv) coordinates to each vertex 
  
  * *Interpolate* between the vertices to get the uv coordinate for each pixel!
  
---

# Textured Triangle

Texture coordinates (uv): (0,0), (1,1), (1,0)

---

# Textured Triangle

Let's use (0.5,0.5) as the UV value for one vertex.

---

# An example

Consider the triangle 
$$
a = (2,1,3), 
b = (4,3,4), 
c = (3.5,2.5,4.5) 
$$
The texture coordinates are 
$$
t(a) = (\frac{3}{5}, \frac{3}{8}), t(b) = (\frac{3}{10}, \frac{3}{16}), t(c) = (\frac{3}{7}, \frac{3}{10})
$$
## The point p is the closest point to c on the line from a to b. What texture coordinates does p have?

---

# Meshes

* Triangles are boring!
  
  * A *Mesh* is a collection of triangles (and normals, uv coordinates, colors)
  
  * How about something more exciting, like a cube?
  
  * We can assemble a cube out of triangles!
  
  * Or a bunny!
  
---

# Stanford Bunny

---

# Back to the cube

* Cubes have 8 vertices and 6 faces (12 triangles!) 
  * Vertices:
     - v1 = (0,0,0)
     - v2 = (1,0,0)
     - v3 = (0,1,0)
     - v4 = (1,1,0)
     - v5 = (0,0,1)
     - v6 = (1,0,1)
     - v7 = (0,1,1)
     - v8 = (1,1,1)
  * Faces:
     - f1: (v1,v2,v3)
     - f2: (v3,v2,v4)
     - etc.
     
---

# The .obj file format

</td><td width="50%">
<span style="font-family: monospace;">
f 2 1 3<br/>
f 2 3 4<br/>
<br/>
f 8 5 6<br/>
f 5 8 7<br/>
<br/>
f 6 1 2<br/>
f 1 6 5<br/>
<br/>
f 3 8 4<br/>
f 8 3 7<br/>
<br/>
f 1 7 3<br/>
f 7 1 5<br/>
<br/>
f 8 2 4<br/>
f 2 8 6<br/>
</span>
</td></tr></table>

Download file <a href="https://github.com/yawgmoth/CI-2700/blob/master/assets/unity/cube.obj">here</a>

---

# .obj files

* List of vertices, faces, uv coordinates, normals, etc.
  
  * You could hand-write one (like for the cube!)
  
  * Or you write code to generate the file
  
  * Unity (and most other 3D software) reads them!

---

# Cube (Controls: w, a, s, d)

---

# Level of Detail

* Many modern meshes have a lot of detail 
  
  * But what if it is far away?
  
  * Idea: Use a lower-resolution (fewer triangles) version of the same mesh!
  
  * Can often be precomputed automatically
  
  * Limitation: Visible "jumping" when the low-res mesh is replaced with the higher-res one
  
---

# Stanford Bunny and Level of Detail

---

# Billboards

* Many background objects like trees, grass, rocks only exist as decoration
  
  * It may not be important for them to have a lot of detail 
  
  * Actually, as long as it looks like a tree from all sides, it's probably fine
  
  * Just show a picture of a tree, and turn it so it always faces the camera!
  
  * Also useful for health bars or similar information
  
  * `transform.LookAt(2 * transform.position - Camera.main.transform.position);`

---

# Rendering Pipeline

## Or: How do we actually get something onto the screen

---

# Rendering

* Model coordinates need to be translated to world coordinates (where is the mesh)
  
  * World coordinates are then translated to Camera coordinates (where in the field of view of the camera is the mesh?)
  
  * The Camera coordinates need to be projected onto a Viewport (objects that are further away become smaller)
  
  * The Viewport is then mapped onto a window on the screen
  
  * We will discuss the math behind this next week
  
---

# Camera

---

# Clipping and Culling

* Many triangles do not need to be rendered
  
  * Anything outside the frustum or not facing the camera can be discarded (Frustum culling and backface culling)
  
  * Triangles that are behind other triangles can also be culled (Occlusion culling)
  
  * Triangles that are partially inside the frustum have to be clipped (cut into pieces)
  
---

# Backface culling

* How do we recognize what is facing away from the camera?
  
  * Assume a and b are unit vectors
  
$$ \vec{a} \cdot \vec{b} = \cos(\theta) $$

* The cosine is negative iff the vectors point in opposite directions

* Discard any triangle where the face normal points in the same direction as the (unit) vector pointing from the camera to the triangle
  
---

# Painter's algorithm

* How do we avoid showing triangles that are occluded?

* When you paint, whatever you draw last is visible on the canvas 
  
  * Let's use the same idea with triangles!
  
  * Draw triangles starting with the one furthest from the camera 
  
  * The closer a triangle is to the viewer, the later it is drawn
  
  * If a triangle is partially occluded, you will still see the rest
  
---

# Which order?

<img src="/CI-2700/assets/img/painters.png" width="50%"/>
  
---

# z-Buffer

* Instead of deciding per triangle, decide per pixel 
  
  * Store z-coordinate (depth) in a special buffer
  
  * When something would be drawn on a pixel:
       
       - If the z-coordinate of the new object is greater than the buffer, don't draw
       - Otherwise draw, and update the buffer 
       
  * The z-Buffer usually exists in hardware
  
---

# Occlusion culling

* Both of these approaches still attempt to draw everything
  
  * In large scenes this can be very expensive
  
  * Specialized algorithms exist to determine objects that are occluded
  
  * Such objects can be culled entirely
  
  * Unity has an Occlusion culling algorithm, but it requires some setup
  
---

# Shaders

---

# Shaders

* Shaders are little programs that can change draw operations
  
  * Vertex shaders operate on vertices 
  
  * Geometry shaders operate on primitives (triangles)
  
  * Fragment/pixel shaders operate on fragments (drawable parts)
  
  * Compute shaders can be used to do calculations on the GPU

* Written in a Shader Language
  
---

# Shader languages

* ARB assembly (legacy)
  
  * Cg (deprecated)
  
  * GLSL (OpenGL)
  
  * HLSL (DirectX, used by Unity)
  
  * Tool: HLSL2GLSL
  
---

# HLSL Shaders

```HLSL
Shader "ShaderName"
{
    Properties
    {
       // Properties you can set in Unity
    }

SubShader
    {
        // Shader code
    }   
}
```

---

# HLSL Shader properties

```HLSL
Properties
{
    _MyTexture ("My texture", 2D) = "white" {}

_MyInt ("My integer", Int) = 2
    _MyFloat ("My float", Float) = 1.5
    _MyRange ("My range", Range(0.0, 1.0)) = 0.5

_MyColor ("My color", Color) = (1, 0, 0, 1)    // (R, G, B, A)
    _MyVector ("My Vector4", Vector) = (0, 0, 0, 0) // (x, y, z, w)
}
```
  
---

# HLSL Shader properties as

```HLSL
SubShader
{
    sampler2D _MyTexture; // 2D = sampler2D

int _MyInt;
    float _MyFloat;
    float _MyRange;
    half4 _MyColor; // Color = half4
    float4 _MyVector; // Vector = float4

// Code
}
```

---

# Vertex and Pixel Shader

```HLSL
Pass {
    CGPROGRAM
    #pragma vertex vert             
    #pragma fragment frag
    struct vertInput { float4 pos : POSITION; };  
    struct vertOutput { float4 pos : SV_POSITION; };
    vertOutput vert(vertInput input) { 
        vertOutput o;
        o.pos = mul(UNITY_MATRIX_MVP, input.pos);
        return o;
    }
    half4 frag(vertOutput output) : COLOR {
        return half4(1.0, 0.0, 0.0, 1.0); 
    }
    ENDCG
}
```

---

# What can you pass in and out of these shaders?

* Possible inputs:
      - `POSITION`
      - `NORMAL`
      - `COLOR`
      - `TEXCOORD0`, `TEXCOORD1`, ...
      - etc.
      
  * Possible outputs:
      - `SV_POSITION`
      - `COLOR`
      - `TEXCOORD0`, `TEXCOORD1`, ...
      - etc.
      
---

# What can you do?

* Vertex Shader:
     - Change positions of vertices
     - Modify texture coordinates
     - Store information for the pixel shader 
   
  * Pixel Shader:
     - Change how a pixel is drawn 
     - Add noise or distortion effects (water, glass)
     
---

# How do you use a Shader in Unity?

* Create a new Asset:
      - Surface Shader 
      - Unlit Shader for a Vertex *and* Pixel Shader 
      
  * On your material, select your Shader from the dropdown box 
  
<img src="/CI-2700/assets/img/unity_shader.png" width="50%"/>
     
---

# Wobbling

In the Vertex Shader:

```HLSL
fixed4 col = tex2Dlod(_NoiseTex, float4(v.uv, 0, 0));
float d = 1 - (v.uv[1]*v.uv[1] + v.uv[0]*v.uv[0]) + col[0];

o.vertex = UnityObjectToClipPos(v.vertex + 
              float4(0.0, 0.5*(sin(_Time[2] + d*8) / 2 + 0.5), 
                                0.0, 0.0));
o.tex = float4(d,0,0,0);
```

In the Pixel Shader:
```HLSL
fixed4 col = float4(0, 0, 
         (sin(_Time[2] + i.tex[0]*8)/2 + 0.5),1);
```

---

# Wobbling: Result

<a href="/CI-2700/assets/unity/WobblyShader.shader">Shader source code</a>

---

# References

* Book: *Mathematics for Game Developers*, by Christopher Tremblay

* [Barycentric coordinate system](https://en.wikipedia.org/wiki/Barycentric_coordinate_system)
   
   * [NeHe OpenGL tutorials](http://nehe.gamedev.net/) (slightly outdated, but explains the basics well)
   
   * [OpenGL Billboard tutorial](http://www.opengl-tutorial.org/intermediate-tutorials/billboards-particles/billboards/)

* [three.js](https://threejs.org/) (JavaScript library for easy 3D rendering)
   
   * [OpenGL Render Pipeline](https://www.khronos.org/opengl/wiki/Rendering_Pipeline_Overview)
   
   * [Vertex and Fragment shaders in Unity](https://www.alanzucconi.com/2015/07/01/vertex-and-fragment-shaders-in-unity3d/)