AI PERSONAL ROADMAP

Topaz Gigapixel AI - January 2019

My background is in filmmaking and video post-production, so naturally my first entry into AI was in finding ways to enhance footage. Especially a decade ago and more, there were many limitations to shooting in digital. A lot of the things that we produced prior to the introduction of HD cameras did not age well, and I always wanted to remaster a few of my favorite edits. One of them was a short film my friend and now commercial director King Palisoc’s final student film for college, Mahal Ko Si Direk! (I Love My Director!)

This movie was shot on a Canon XL2. While that was the top of the line for MiniDV cameras at the time (2005!), ultimately the image resolution was a measly 720x480. I was able to recapture the original footage from tape, upscaled the footage (using software that I’ve unfortunately been unable to recall) and reedited/colored in 1920x1080.

Nowadays it’s easier to upscale footage, with many options online. I would recommend Topaz Video Enhance, but there are free options as well.

DAIN and RIFE - 2020

Another AI video enhancement technique I looked into was frame interpolation, i.e. creating intermediate frames in between existing frames to provide more motion data, aka synthetic slow motion.

Fast Style transfer via DeepAI.org - February 2021

Deep Dream AI by Google - February 2021

I also experimented with AI solutions to applying an image’s style onto another. As interesting as this approach was, I was looking for a video-centric solution, which brought me to a more dependable, widely used (albeit non-AI) option - EbSynth - Transform Video by Painting Over a Single Frame. Still, I wanted something that had I could run on my own hardware and give me more control.

Text to Image model @ RunwayML.com - February 2021

unknown_152885918_855418341676310_4923269901439104940_n.jpg

unknown_152613106_182210936670645_6873797825063801873_n.jpg

unknown_153638206_124818276152867_7993552723326759988_n.jpg

unknown_152912519_457809905370059_5334809200877057696_n.jpg

unknown_151990629_302665531191152_8406361426345756210_n.jpg

unknown_152148804_249111323481100_8854210963858912685_n.jpg

unknown_152815884_461315778573036_5414701802962356096_n.jpg

unknown_152618551_238996091206112_5026338397779721943_n.jpg

Just over a year ago I started playing around with Runway.ml’s text2image model. It’s interesting to see how far we’ve come in such a short span of time.

Wombo - January 2022

unknown_271868958_666020151070816_376864641943062084_n.jpeg

unknown_271550555_103609248836401_659088878229179240_n.jpeg

unknown_271791043_350984846499031_534179018135181321_n.jpeg

unknown_271432937_370537861506560_9072674870159564556_n.jpg

unknown_271417541_715133846123588_7692794606867627962_n.jpg

unknown_271502136_457752202715860_1876440455892431966_n.jpg

unknown_271298882_887414381940802_4064895503631643798_n.jpg

unknown_271261148_215119634154681_1489034708425722088_n.jpg

After a year we were getting even closer, with Wombo being able to generate more coherent images.

Midjourney v.1-2 - July 2022

When Midjourney first came out, my immediate objective was to see how far one could push the algorithm, prompting a variety of scenes and compositions. True enough, especially compared to the images above, Midjourney certainly produced images with a level of coherence we’ve never seen before. Of course apps from other companies have already been operating since before, but there was a certain unmatchable ease one got with Midjourney.

unknown_294448811_504652044765832_6120118734192428774_n.jpeg

unknown_294361575_782158986137471_2802478106692251306_n.jpeg

unknown_294250610_1444400485986525_8764966744314291968_n.jpeg

unknown_280392317_1266167827459966_739572722262866828_n.jpeg

unknown_295107287_416157750539341_8316490490928465931_n.jpeg

unknown_294166521_957886251841524_8214540352175271028_n.jpeg

unknown_293844066_461462922011395_2796188862389126251_n.jpeg

unknown_290286873_570170084818069_3626190829320425335_n.jpeg

Using repeated iterations of midjourney same-seed renders in animation

Midjourney v.3 and beta - August 2022

unknown_297091329_790477508970356_6182810942043552086_n.jpeg

unknown_296273096_616875486527856_1876015927722477739_n.jpeg

unknown_295702837_1716961145323387_5225586788438341528_n.jpeg

unknown_300860003_1424184854758608_8675632028536295676_n.jpeg

unknown_300866222_1063054984377032_2808408122775482502_n.jpeg

unknown_297150667_125220306906889_7424329995707120821_n.jpeg

unknown_298169605_166233799285293_225301026029621706_n.jpeg

unknown_298361929_776433550267830_4172995773072565370_n.jpeg

Using AI generated assets in traditional animation

Midjourney —test and —testp - August 29, 2022

unknown_305527627_794383448429174_5261569402238183702_n.jpeg

unknown_300984585_108288528672210_6128587750778321833_n.jpeg

unknown_301046924_3253516484931627_7693940149016726650_n.jpeg

unknown_305377449_199130059132555_6996339392028329860_n.jpeg

unknown_302065390_1242206986561499_4054692682541986555_n.jpeg

unknown_301666693_483392696541802_3055106756996140435_n.jpeg

unknown_304051221_601105611521450_7540121362098310502_n.jpeg

unknown_301927967_1121603035107128_5656567382395382537_n.jpeg

After a month of using the platform however I noticed that while Midjourney’s images were getting better and better every week, Midjourney seemed to heavily prefer certain styles and subjects. And of course, Midjourney didn’t do animation, so it wasn’t a viable long term solution for me.

Things changed when Stability.ai released stable diffusion to the public. Suddenly anyone could make applications to run on their open, publicly available model. In a few short days several options opened up to everyone, including me.

Stable Diffusion via Google Colab and VoC - September 2022