Output

See the live version with webcam in this link.

Ideation

For ICM Week 9, our challenge was to create a 1-minute alternate visual reality that manipulates pixels in p5.js. We brainstormed a list of pixel visual properties we could focus on. After ideation, we decided to focus on three properties - Cubes, fake 3d and shadows. While researching, we got some inspiration from artists such as Bees and Bombs and Zach Lieberman.​ Based on these keywords, En Tung brought up a wonderful idea of projecting webcam images in one side of iterating cubes.

Process

Drawing multiple quad()s to make fake 3d cubes

Using the quad() function, I drew a pattern of fake 3d looking cubes. It was also pretty interesting when we had weird looking patterns due to mistakes.

Projecting skewed webcam images to the fake 3d cubes

This was the most challenging part in this assignment. I used a strategy as splitting the webcam image into small squares and skewing them one by one with shearY(). However, the position of the skewed image piece kept being placed in the canvas as if the original image as a whole was skewed.

I spent a lot of time since push(), pop() and translate() didn't act as what we had expected. However thanks to Mimi Yin, I could figure out that that I was using the translate() function in the wrong way.

We also got a lot of help from Adam Krebs & Daniel Shiffman. Especially, Shiffman suggested we use the WEBGL canvas and applying a texture to the shape. We learned that there are multiple solutions to the same problem.

Applying the video pattern to the cubes

En Tung also made a great looking monochrome color change interaction which made the sketch look as if it had different color weights. We combined our sketches into the final version.

Takeaways

One issue we didn't solve yet is that the skewed webcam pattern had different sizes from the quad() pieces. This resulted in a slight mismatch of the pixels.

Also, due to time constraints, we couldn't try the WEBGL version of the solution yet. We will try developing the sketch further with the WEBGL version.