Core Idea
Is it possible to automate the production of a Hollywood movie trailer? The movie industry has long perfected advertising movies to attract audiences. Knowing that every trailer is a piece of art produced by creative humans, we were curious to see how close algorithms could get to this goal. For our one-year project we limited the scope to Hollywood action movies.

Research
Beside film theory and trailer documentation we built our own research pool: an online database we filled with detailed shot-by-shot trailer information. We analyzed 11 action trailers for several aspects such as shot duration, actor appearance, camera position or sound volume. From the resulting pool of data sets we extracted typical trailer patterns. This research formed the basis for modeling our systems trailer knowledge.

Movie Analysis
To automatically create a trailer it is necessary to extract the defining features of a movie. Analyzing operations gather knowledge about features such as actor appearance, speech, text, sound events and
many more. The feature data forms the basis for footage classification and automatic generation.

Image Features

• We find faces in the movie and try to distinguish the main actor.
• The image motion is calculated. High image motion indicates action.
• Overlay text is detected automatically. Start or end credits are not usable for a trailer.

Audio Features

• We trained our software to find explosions, gunshots and screams as well as other dramatic sound events.
• We distinguish soundtrack music.
• Our speech analysis can detect speaking actors and finds important quotes within the movie that are registered in the international movie database imdb.

 

Composing a Trainler Automatically
Based on extracted movie features our generation software creates a fully fledged movie trailer.
An underlying trailer knowledge base we fed with results of our research dynamically generates a trailer structure based on semantics. This sequence is enhanced with all kinds of effects to
create “real trailer feeling”.

• Categorizing Footage
  With our analysis data the system breaks down movie footage into small units and categorizes them based on extracted features.
• Music and Sound Effects
  We use separate music and sound effect media to produce a unique trailer soundtrack during generation.
• 3D Animations
  The system creates 3D text animations at runtime for titles or actors names.
• Special Transitions
  White flashes, fades or hard cuts are used according to trailer semantics.
• Playback Speed, Volume, Location
  The software manages several footage properties for fine-tuned trailer results.

Tools and Techniques
In the context of the project there is already a great number of previous research results and applications available which was very helpful to use in different ways. For example, the AGKI group of the TZI Bremen developed image and video processing tools (such as for shot or face detection). Other anaylsis application were self developed using several programming language and libraries (for example C++ and OpenCV) that are capable of sophisticated image and video processing. The Generator software module uses C++ and the free expert system CLIPS to model our trailer ontology. As well it uses the open source software virtualDub, aviSynth and Blender controlled by scripts for the automatic generation process.