Synote Media Fragment Player: a polyfill of to play the temporal and spatial fragment in a media fragment URI

Introduction

A very long time ago, I intended to extract the media fragment player we developed for Synote as a individual easy-to-use HTML5 player. Finally, I got the time recently to develop the Synote Media Fragment Player (smfplayer). In general, smfplayer is a jQuery plugin and polyfill for web developers to easily playback the temporal and spatial fragment of different video/audio resources. In Synote, smfplayer only replays the temporal fragment. But, inspired by two great work from Ninsuna and Chris Coyier, I think it's also possible to create a polyfill to highlight the spatial dimension.

The Rational

Nowadays, there are two kinds of multimedia resources we usually want to play on the Web if not considering the live streaming of video/audio: the video/audio files hosted in some web server (Apache, Amazon S3, etc), and multimedia sharing websites (YouTube, Dailymotion and Vimeo). As to the support for media fragment, the Media Fragment Working Group showcase page includes some browsers' latest support for media fragment. Meanwhile, YouTube, Dailymotion and Vimeo all have "copy current time URL" or similar functions, but they are based on the Flash player, not HTML5. So it is still impossible to use a <video> or <audio> tag and the "src" attribute to enable media fragments for all browsers and for all those multimedia resources.

There are zillions of HTML5 video player out there and some of them offers "fallback" compatibility for Flash (or even silverlight) like JW player. Theoretically (I mean forget about the legal issues), any HTML5 player with flash fallback can embed chromeless Flash player from online video sharing websites and provide us a unique UI for all resources. By searching around, I only find MediaElement.js player (Mejs) best suits my requirement. What amazed me most about Mejs is that it has "fallforward" Flash and Silverlight plugins. Based on the browsers' compatibility for a certain file type, Mejs can automatically choose the best way to play the video: HTML5, Flash or Silverlight. More surprisingly, Mejs can directly play YouTube and Vimeo (as well as Dailymotion with a pull request). Everything in Mejs has a unique interface to control all the players (HTML5, Flash and Sliverlight).

So next, using Mejs as the background, we can start to build the polyfill to control the player highlighting media fragments. First thing first, we need a media fragment parser. Fortunately, this library has been developed by Thomas Steiner. After obtaining the start time, endtime and xywh defined in the media fragment URI, we can start highlighting both temporal and spatial fragments.

Highlighting temporal fragment is easy. When Mejs autostart playing or the user click the play button, smfplayer will jump to the start time if it is defined. Smfplayer will pause the video if the currentTime is bigger than the end time. For spatial highlighting, smfplayer create an overlay <div> on top of the video based on the xywh value in media fragment URI. You can define xywh in two ways, pixel and percent, so we need to ask the user what is the original width and height of the video if percent is used. In Ninsuna example, only the spatial fragment area is visible, while smfplayer just circle around the area with some highlight colour (default is yellow). One issue of this solution is that when the video goes to fullscreen mode, the overlay <div> will disappear.

Demo Time!

The following YouTube video is an introduction of Web Platform. Well, Web Platform is exciting for me, but what's more exciting is that the creator of this video somehow decided to call Sir Tim-Berners Lee a WEB DEVELOPER!

I want to share this interesting finding via Twitter. Tim only appears in this video for 4 seconds from 1:30, so I can send this link via twitter:

http://youtu.be/Ug6XAw6hzaw?t=1m30s

This is not enough. What I really want my followers to see is the funny title of Tim-Berners Lee on the bottom-left corner. So what I really want to share is this following media fragment URI given the video resolution is 480x320:

http://www.youtube.com/watch?v=Ug6XAw6hzaw#t=00:01:30,00:01:33&xywh=40,220,200,50

And using smfplayer, you can easily write some programmes to playback this media fragment:

This demo is similar one of the features of twitter: the in-page embedded playback of the videos from tweet list page. As a developer, you need to write different code to embed different players from YouTube, Dailymotion, Vimeo and files from web servers. With smfplayer, you can implement this function easily. You can try to play different resources at smfplayer demo page.

Conclusions

Followings are some useful links in case you want to know more about smfplayer:

• smfplayer home page
• smfplayer on Github
• smfplayer documentation
• smfplayer test results on different browsers. Smfplayer currently is still at alpha test stage and there are still some problems in different browsers and devices.

All in all, smfplayer is a polyfill and temporal solutions for the developers who want to playback media fragments. I really hope some day different browsers can implement those functions natively.