Microsoft's Story Remix puts a Hollywood studio in your PC
The app is arriving with the Fall Creators Update.
For the Fall Creators Update, Microsoft has unveiled a new app that can transform your photos and videos into a cinematic spectacle, complete with CG effects, titles and a soundtrack. Called Story Remix, it's a Universal Windows App that uses deep learning and Microsoft's Graph, letting you pull media in from colleagues and friends. It's a powerful yet simple app that uses image analysis and AI to do clip discovery, effects integration and the entire edit, complete with music.
It's like a fancier, smarter version of Apple Clips, Google's Photo Assistant or the Magisto app (remember that?) but geared toward Windows 10 content. As a nod to the latest Creators Update, it can also pull in 3D objects from Paint 3D and other programs or let you draw on your videos and photos using Windows Ink.
The interface features an asset pane on the right; "explore," "create" and "folders" tabs up top; and a playback window below -- like a simplified version of Final Cut Pro X or Adobe Premiere. However, rather than using frames, video clips and photos are shown as thumbnails with the length in seconds and frames superimposed.
From there, you just drag clips around to reorder them or grab the head and tail to change the length. Various tools let you add text, filters, soundtracks and more. Microsoft Graph comes into play by letting you pull in content from friends or colleagues, contribute clips or marry your own videos with others. Thanks to deep learning, it can remember your preferences and prebuild videos that are creatively in tune with your past projects.
While relatively simple, Story Remix goes beyond other editing systems in many ways. Thanks to machine learning, it can analyze your photos and videos, then figure out when and where your media was shot and who's in it, and it can even pick out the ground plane and other objects. That makes it much easier to find specific shots containing dogs, trains or people, saving you a whole bunch of searching and scrolling through clips.
From there, it can automatically edit media, add a soundtrack, lay in transitions and give you a final cut with hardly any effort on your part. If you don't like it the first time, you simply hit the "remix" button and it will completely change the style and tone of your piece.
You can add titles from Microsoft's Pen and Ink or insert 3D models, explosions and other mixed-reality content from Microsoft's recently launched 3D Remix asset site. Thanks to its AI, Story Remix can do advanced 3D motion tracking and figure out the 3D perspective and camera position of a photo or video.
In one example (see our lead image), a boy with a magic wand shot on an iPhone conjures up a CG dragon from 3D Remix, with lightning bolts from the boy's wand (again from 3D Remix) used to complete the effect. All the user has to do is drop in the model and everything will automatically line up in perspective.
Story Remix will add royalty-free theme music from Groove that readjusts for each video by changing the tempo and lining up beats with the video. Hitting the "remix" button will select different music, or you can adjust it yourself using a slider that varies from "chill" to "energetic."
Story Remix is a pretty canny app: It takes the drudgery out of editing, helping inexperienced users create full videos. At the same time, it enables some pretty advanced effects, again without a lot of complex knowledge required. That makes it ideal as an educational tool, letting kids collaborate to build fun videos while learning the art of cinema.
The full Story Remix app, with automatic editing, 3D Remix support and more, will arrive in September with the final release of the Fall Creators Update. However, a basic version called Photos will arrive on Thursday with the first Windows Insider build.
Click here to catch up on the latest news from Microsoft Build 2017.