Implementation

The project was implemented using WebGL in JavaScript, after a brief and frustrating venture into TypeScript. Recreating the project scaffolding of CPE 471 from scratch in a different language proved to be more challenging than anticipated. The tutorials at webglfundamentals.org were very helpful, along with the WebGL tutorial on MDN. MDN's documentation was also vital.

After finally getting a simple "Hello World" of displaying an image in WebGL, one of the challenges was dealing with a flexible aspect ratio. Previous CPE 471 assignments had included code in the perspective setup that scaled the window according to the correct aspect ratio. For this project, however, the goal was a background image that always filled the screen without cutting off any part of the image. To this purpose, I attempted to dynamically scale the image's model-space size according to the correct viewport & image dimensions & aspects. I eventually a version of this working for the background alone (not the sprite), but I ended up forgoing the above for a hard-coded viewport size for the sake of time.

Another challenge in the project was indexing into the sprite. I initially picked a spritesheet of a dog whose individual sprites were not uniformly spaced across the page. It took me a while to figure this out and to switch to a different spritesheet whose sprites were all the same size and uniformly laid out in a grid. This made it simpler to find the sprite frames:

Switched from non-uniform layout to grid layout

One of the nice things about drawing images in WebGL in comparison to OpenGL is that the Browser DOM already has the ability to load images in a variety of formats and WebGL's texImage2D() takes a JavaScript Image object as an argument. However, the resolution of urls into images is not synchronous in JavaScript, so that had to be handled asynchronously.

One thing I learned is that the rate (in frames per second) of sprite motion does not necessarily match the sprite's animation. With only 4 frames in each of my sprite's animations, I had to define a FPS rate for the sprite's animation itself, while allowing the sprite's motion to happen at the global redraw FPS.

After getting the sprite to animate, and placing it in the 3d world, another hurdle I faced was displaying the sprite in the 3d world without skewing the image. I tried a variety of different things with varying success, but I finally succeeded to by passing to the vertex shader the sprite's centerpoint in model space (in my case it was always (0, 0, 0)), and using that in the perpective matrix multiplication so that all perspectivized X & Y values are based on the same depth. Excerpt from my sprite's fragment shader:

vec4 cameraSpriteCenter = uV * uM * vec4(uSpriteCenter, 1.0);

vec4 cameraPos = uV * uM * vec4(aVertPos, 1.0);

//When calculating the "perspectivized" x & y values, use the same 
//z values for all points on the sprite so that the spite doesn't skew
vec4 perspectivePos = uP * vec4(cameraPos.xy, cameraSpriteCenter.z, 1.0);

gl_Position = vec4(perspectivePos.xy / perspectivePos.w, cameraPos.z, 1.0);

I later realized that this was called "billboarding", and I probably could have looked it up if I had looked through the notes of topics Professor Wood gave me to google.

I added the triangle of light to make the image look nicer and because it wasn't difficult. I used the barycentric coordinate calculation from program 1 to determine if points were in the triangle

The final challenge in this project was "inferring" the correct 3d world to match the image. I decided to start with something easy and do a flat terrain of y=0, setting the view matrix (eye position, lookAt position, and up vector) in such a way as to make the plane align with the floor in the picture. I also defined minimum and maximum x & z values for this area. However, as seen below, I was never able to perfect the alignment with the image. I had chosen NOT to try and draw the plane (as a transparent grid) in order to make my goals more acheivable and to save time, but after struggling to get the motion of the sprite to match the image, I finally realized that trying to position the plane of the floor without seeing what I was doing wasn't working. By that point I was out of time. While I do think I got it fairly close, it is still not correct. In particular, the walkable area at the far end of the hallway fills the hallway in the image, but at the near end of the image the walkable area is only about 2/3 of the hallway's width:

Walkable area is of uniform width, yet seems to take up greater percentage of hall width as you move away from the camera

While walking along the Z-axis, the motion seems to align fairly well, though not perfectly, with the hall tiles:

Same x values

However, walking along the X-axis does not and seems to slant too far down and to the right:

Same z values

I expect this has to do with the perspective matrix, and I experimented with changing my perspective matrix settings, but I was never able to get it to work how I wanted. While I was able to follow along with the derivation of the perspective matrix in class, I still feel like my understanding is incomplete, especially with relation to the angle passed into makePerspective and the frustum. I think drawing the terrain to screen as a grid, as well as being able to change the perspective & view matrix values "live" without having to switch back and forth from code to browser would make it easier to visualize and see what was happening—and thus easier to get an accurate "inferred world" setup.

I have considered continuing on this project for fun, and I think—as I mentioned above—that the next step would be to draw out the flat terrain as a grid of semi-transparent lines that could be toggled on and off. This would allow me to better see how the geometry was lining up. I believe that after defining a grid like this, extending the code to support non-planar terrains would not be too much more effort. Another helpful addition would be some buttons or keys on the page to change some of the world setup parameters like the eye position, look-at position, up vector, terrain boundaries, etc.

Description

Implementation

Attribution

Helpful Tutorials

Graphics libraries

Art

Other libraries