Single Image Animation

Research on how to create animations from a single image.
Most existing work here is focused on transfering an animation from a video or collection of videos onto an image.

Fluids

Animating Fluids

Relevent People

• Makoto Okabe Google Scholar

Animating Pictures of Fluid using Video Examples

• Published in Computer Graphics Forum 2009
• Authors: Makoto Okabe Ken Anjyo Takeo Igarashi Hans‐Peter Seidel
• https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-8659.2009.01408.x
• https://www.youtube.com/watch?v=RBK6UGz3GPc
• Basic Idea
• Given an image and a video
  • Decompose video into average image, flow field, residual
  • User draws desired flow field for the image and transfers (?) the residual from the video.
  • New animation is average of image + drawn flow field + transfered residual

Creating Fluid Animation from a Single Image using Video Database

• Published in Computer Graphics Forum 2011
• Authors: Makoto Okabe, Ken Anjyo, Rikio Onai
• https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-8659.2011.02062.x
• https://www.youtube.com/watch?v=S5iqKa7FRy0
• Basic Idea
  • Create database of one type of fluid (fire, smoke, water)
  • Database is composed of (average image + differences) of video patches and indexed with visual words (SIFT).
  • Given a new image of a fluid, cut it up into pieces.
  • For each piece, find a video from the database. Combine the differences from the database with the image to form an animation. Lots of small details in the paper...
  • User can draw a flow field and apply a mask. The flow field influences which video is selected from the database.

Animating pictures of water scenes using video retrieval

• Published in The Visual Computer 2018
  • First online in 2016
• Authors: Makoto Okabe, Yoshinori Dobashi, Ken Anjyo
• https://link.springer.com/content/pdf/10.1007%2Fs00371-016-1337-6.pdf
• https://download-tls-cdn.edge-cdn.net/videodb/5501/videodb_5501_53500_9324610_16x9_hq.mp4
• Basic Idea
  • Given an input image
  • User draws regions of interests
  • For each ROI, find a video with a similar structure to the ROI
  • Merge patches and video by coping residuals
  • Residuals are original frame - gaussian blurred frame

Using Neural Networks

• Animating Pictures with Eulerian Motion Fields (2020)
  • https://eulerian.cs.washington.edu/
  • Warps deep features using a predicted motion field.
  • Uses a symmetric splatting and blending technique to fill in holes in the feature maps.

Animals

Animating Animal Motion from Still

• Siggraph Asia 2008
• Authors: Xuemiao Xu, Liang Wan, Xiaopei Liu, Tien-Tsin Wong, Liansheng Wang, Chi-Sing Leung
• https://www.cse.cuhk.edu.hk/~ttwong/papers/flock/flock.html
• Basic Idea
  • Given a photo with multiple of the same animal
  • Crop each animal to get a set of "motion snapshots"
  • Using some structural similarity measures, order the motion snapshots to create a motion cycle
  • Do consistency refinement to morph between snapshots.