Thursday, 17 October 2013

Are the Videos We are Watching Online Media Fragment Ready?

The Media Fragments URI (basic) has been published as a W3C recommendation since Sep, 2012. It's pretty excited to see that many use cases have already come about to use media fragments for better linking and indexing of multimedia resources. A recent webniar given by one of the chairs of W3C Media Fragments Working Group, Raphael Troncy, has pointed out that YouTube and Dailymotion has partially implemented the specification (See the slices below).What really interest me now is how many "big players" in this area have actually implemented or partially implemented this specification, i.e. are they "media fragment ready"? So I spent a couple of days to investigate the video sharing applications online. If you are not interested in how I did the experiment, you can jump directly to see the result.

The Methdology

First things first, I needed to find out a list of websites I want to take a look at. Fortunately, I found a wikipedia page with a list of major video sharing platforms in different countries. In my opinion, some "major" players are not listed there, such as and So I add them into the list and removed a couple of them as they are not public video sharing services or they are severing adult content. You can see all the 59 websites I investigated in the final result.

Honestly, what I did next was not contacting all the websites one by one and asking them "can you give me a URI that points to a certain time or spatial area of the video resources hosting on your website?". I just went to each website and tried the following steps to find out the answer:

  1. Open a desktop browser. I used Google Chrome for this investigation.
  2. Go to the landing page of a random video. Login if necessary.
  3. Right click on the player, of course it must be a flash player, and see if there is an selection called something like "Get video url at the current time".
  4. On the landing page, find out whether there is any social sharing button (including the buttons emerged after you pause the video player) allowing you to share the video at a certain time point.
  5. Go to Twitter and search whether any video fragment has been shared recently.
  6. If none of the above works, I would make the conclusion that this website doesn't support media fragments, at least doesn't support W3C Media Fragment (basic) Specification.

The methodology is not flawless and I may have missed something. So if you find anything that is not correct in the result table, please feel free to make comments and I will mend it.

The Result Table

Table 1 and 2 are my investigation results. Table 1 shows the name of the website, their implementation status of temporal and spatial fragments and the page views per day. The page views per day is obtained by myself from manually searching . I am not 100% sure if they have provided the accurate data. But compared it with the site traffic data from wikipedia, they seem more or less match. So currently I just trust All the site views data is valid in Oct,2013.

Table 1. Media Fragment Compatibility on Video Hosting Services
Hoster t xywh PageViews/Day Notes Partially No 7,142,857 Chinese→ No No 5,978,260
AfreecaTV Unknown Unknown 91,674 Korean→ No No 214,174
BlogTV No No 33,475 Now at No No 804,681
Buzznet No No 120,733 No No 3,520
Crackle No No 344,611
DaCast No No 2,897 Online Video Streaming. Haven't tried.
Dailymotion Partially No 11,702,127
EngageMedia No No 3,426
ExpoTV No No 34,042
Facebook No No 18,600,000 video since 2007. Estimated based on this report. No No 506,678
Funshion No No 601,300 Chinese→ Page views according to
Fotki No No 139,611
GodTube No No 68,909 formerly
Hulu Partially No 3,142,857 Area restriction→
Lafango No No 11,620
LeTV No No 3,459,119 Chinese→
Liveleak No No 1,929,824 No No 0 Page views unknown as the video section is a subset of the whole website, and there is no relevant data about video views online.
Mefeedia No No 173,803 Area Restriction→
Metacafe No No 1,127,049
Mevio No No 52,276
Mobento No No 1,014 Focus on video search
Myspace No No 0 No data about the video views.
MyVideo No No 553,319
MUZU.TV No No 13,801
Nico Nico Douga No No 7,746,478 Japanese
Openfilm No No 8,810
Photobucket No No 0 Mainly sharing photoes (5,263,157 views per day). Not sure about video views.
RuTube No No 601,750 Russian
Sapo Videos No No 0 Portuguese. is a portal website, not sure about the video views everyday.
SchoolTube No No 9,893
ScienceStage No No 10,314
Sevenload No No 7,291
SmugMug No No 542,138 Unknown Unknown 41,323 Area Restrictions→ No No 722,733 only music videos
Trilulilu No No 68,293 Romanian
Tudou Partially No 6,010,928 Chinese→
Vbox7 Partially No 319,303 Bulgarian→
Veoh No No 359,359
Vevo No No 685,358
Viddler No No 122,073
Videojug No No 86,901
Videolog No No 79,687 Portuguese No No 6,697 So few views? I thought it should be more.
Vidoosh No No 3,786 No No 203,628 Allow timed comments
Vimeo Partially No 19,680,000
Vuze No No 54,380 No No 987
Wistia No No 153,331
Yahoo! Video No No 10,000,000 video since 2008. Estimated based on this report.
Youku Partially No 15,277,777 Chinese→
YouTube Partially No 366,666,666

Table 2. The Supported Media Fragment Syntax in Different Video Hosting Services
Hoster Example url Fragment variable
Dailymotion (is this a bug?)
"start" query in seconds
Hulu st query as start time and et query as end time.
Viddler "offset" query in seconds
"t" query or hash in seconds
Tudou "lvt" query in seconds
Youku "firsttime" query in seconds
"t" query or hash in seconds or DDhDDmDDs format


I have examined totally 59 websites, but only 8 of them have implemented some notion of media fragments in their system. The syntax (at least the variable) in each website are different and most of them represent the time fragments in seconds. In addition, none of them actually expose the spatial fragment. From this point of view, the result might be a little bit disappointed (see Figure 1).

Then, let's consider more about the page views per day. In my opinion, this is a very important reference. It shows how many videos, which users actually watch, can be further exposed by media fragments and furthermore, could be shared via social media, indexed by search engines and even linked to named entities on fragment level.

Clearly, in Figure 2. we can see that only 12.2% of the views of the video are not related to media fragments. It's really excited to know that most videos we watched are already (at least partially) media fragment ready. This information can be interpreted in several ways. If I am a user and I want to share only part of the video with my friends, nearly 9 out of 10 chances, I can do it. More importantly, most videos that we watched, can be further indexed on fragment level. This new SEO possibility will definitely bring more traffic to websites whoever implement it. Please keep in mind that, there are still "big players" like Hulu that I haven't investigated yet. So the potential media fragment ready videos could be even larger.

Language Barrier and Area Restrictions

I cannot access some of the websites as they are limited to a certain countries and regions. And some of them are in another language other than English and Chinese. So I don't know how to proceed the methodology to find out the media fragments. Please drop me a comment if you have any update about those websites.

Media Fragment in China

I am glad to see that the largest three video sharing or delivering services in China, Tudou (土豆网), Youku(优酷) and LeTV (乐视) have already partially supported media fragments. Unlike YouTube or Dailymotion, Chinese guys give media fragments a very fashionable name "Chuanyue (穿越)", which means you can magically move something from one place to another. This calling seems confused, but what they (actually "we") want to say is that you can watch the video on any of your device, pause at any time, and start to watch from that time on any other device. So it looks like the video has been moved from one device to another and the status of the video is kept through.

Conclusions and Future Work

In this post, I presented some media fragments URI compatibility investigation for video sharing applications online. The result shows that even though the number of websites who implemented media fragments specification is not large, the major players in this area has already developed the notion of media fragments into their systems, even though they follow different syntax. It's very encouraging to see that nearly 90% of our daily watched videos are "media fragment ready".

Of course, there are many things we need to improve for this investigation, especially collecting more relevant and accurate data:

  1. It's better if we can get the video viewing data instead of page viewing data. After all, you can browser channels on YouTube without actually watching the video.
  2. Find out how many videos each site host. That will be the potential videos that could be exposed by media fragments and indexed on fragment level. Well, not all of them will be treated equally because many of them may not be watched as frequently as others.
  3. It will be also interesting to investigate which website can make timed comments, i.e. comments aligned with the timeline of the video. I know we can do that on YouTube and Viki. So theoretically, if they have timed comments enabled, they should have implemented the player api of jumping to a certain time point of the video. But they may not have exposed it as part of the video landing page url.

This is only a kick start of a more comprehensive and maybe scientific investigation on the media fragment readiness for the video sharing services online. Now, take a wide guess of how we can use the data presented in Table 1 and Table 2. Maybe you have already got some thoughts? Leave me a comment then.

P.S. I again need to mention my work about Synote and the Media Fragment Enricher. They are complete compatible with Media Fragment URI Specification for the video landing page :=)

Tuesday, 8 October 2013

Synote Media Fragment Player v1.0: A hack of sharing media fragments on Tumblr

We have just released a new version of Synote Media Fragment Player (smfplayer) v1.0. Compared with the v1.0-alpha, we fixed a few bugs and did more tests on different platforms. In this blog, I am trying to explain how to use smfplayer in Tumblr, and make it possible to share and highlight media fragments in your Tumblr blog.

The problems of sharing media fragments on social networks

See the following twitter message I have sent last year. I only shared a media fragment in the tweet.

In this case, the video server must be able to handle the media fragment, i.e. parsing the fragment URI and highlight the video fragment. YouTube and Dailymotion has partially implemented this function. If you write click on the YouTube Flash player, you will see a link "copy URL at current time". Unfortunately, the string format of those URL are not compatible with the W3C Media Fragment Specification (basic) and they cannot highlight spatial fragments.

That is the reason we designed Synote Media Fragment Player (smfplayer). In the previous blog, I explained how to use smfplayer in Theoretically, this method could be extend to other blogging or CMS systems such as WordPress, where you have a fully control of the web-page template and the rendering of the template.

Well, I have to admit that smfplayer doesn't work on Twitter and Facebook as the user doesn't have any control of how Twitter and Facebook rendering the embedded video code at all. However, Tumblr is a kind of exception.

Some words about Tumblr

For anyone who doesn't know Tumblr very well, Tumblr is a new microblogging and social network platform. It's sort of like Twitter+Facebook, where you can easily share multimedia posts (images, audio and video) around your followers. You have a blog page, where all your blogs can be viewed, and you can get updates about other users' new sharings on the Dashborad. The blogs on Tumblr are usually short (one sentence, one image, etc) compared with Blogger.

Video sharing is very straight forward on Tumblr. You can embed video code or upload a video yourself. The challenge for me is that is it possible to use smfplayer in Tumblr?

How To?

It's not difficult, but you need to know some programming skills. Firstly, you need to customize your theme and add the smfplayer css and javascript files into the header:

<link href="" rel="stylesheet" type="text/css"></link>
<link href="" rel="stylesheet" type="text/css"></link>
<script src="" type="text/javascript"></script>
<script src="" type="text/javascript"></script>
<script src="" type="text/javascript"></script>

Then the next question will be how can I initialise smfplayer, in the template. This is the tricky part! If you look at the html code of the template, you will see that all the posts, such as text, image, audio, chat and audio, they are placed in {block:something} tag. For example, this is the code for Photo block:

<div class="media">{LinkOpenTag}<img src="{PhotoURL-500}" alt="{PhotoAlt}" />{LinkCloseTag}</div>
    {block:Caption}<div class="copy">{Caption}</div>{/block:Caption}

The code is similar to Video block. So one straight forward thinking will be that we create a video/photo post first, and then replace the div media with smfplayer. But how can we locate the correct div with media class for the video as there might be many such divs? And how can we pass the media fragment URI to initialise smfplayer? To solve these two problems, we need to make full use of the div class="copy" and the HTML editor provided by Tumblr.

Whenever you publish a photo or video post, you can add "caption" for the post and you are allowed to embed HTML into the caption. So I can create a post like this:

The div class="smf-placehoder" specifies that the media embedded in this post needs to be replaced by smfplayer. We also use HTML5 data api to define the actual "mfuri" that will be used to replace the media div. This is because embedding video in Tumblr from YouTube or other resources will result in an iframe, which will depend on the video providers. So it's difficult to traverse the DOM node and find the exact URI of the video. Then, we can add the following code into the html head:

    var placeholders = $(".smf-placeholder");
    if(placeholders.size() >0)
              var smf_div = $(placeholder).closest(".copy").prev(".media");
              var mfuri = $(placeholder).data("mfuri");
                 var player = smf_div.smfplayer({

The code will find the nearest div media and initialise the smfplayer. The following image is a screenshot of the blogpost on tumblr. You can also goto example1 and example2 to see it yourself.

Known Issue

One serious problem of this solution is that Tumblr actually depends much on "Dashboard", which has nothing to do with the template. So your followers won't be able to see any video playing on your blogpost from their Dashboard. What they can see are only captions :( As a temporary solution, we can upload a thumbnail image as a photo post and add a link "View Media Fragment" to lead users to open the actual blog page. This will also be a problem for users on tablets and mobile phones. 

The template of Tumblr may vary from users to users. So the code that is working for my template doesn't necessarily work in your template. However, the basic idea is that you use caption and html data api to save the media fragment uri and initialise smfplayer from javascript.


In this blog, I introduced a way to share media fragments on Tumblr. This solution is still a hack, but should work well as a demo to explore new ways of sharing videos. Again, a more sophisticated solution is that the social network platforms can support the sharing of media fragments natively following the W3C Media Fragment specification.

Tuesday, 2 April 2013

Synote Media Fragment Player: a polyfill of to play the temporal and spatial fragment in a media fragment URI


A very long time ago, I intended to extract the media fragment player we developed for Synote as a individual easy-to-use HTML5 player. Finally, I got the time recently to develop the Synote Media Fragment Player (smfplayer). In general, smfplayer is a jQuery plugin and polyfill for web developers to easily playback the temporal and spatial fragment of different video/audio resources. In Synote, smfplayer only replays the temporal fragment. But, inspired by two great work from Ninsuna and Chris Coyier, I think it's also possible to create a polyfill to highlight the spatial dimension.

The Rational

Nowadays, there are two kinds of multimedia resources we usually want to play on the Web if not considering the live streaming of video/audio: the video/audio files hosted in some web server (Apache, Amazon S3, etc), and multimedia sharing websites (YouTube, Dailymotion and Vimeo). As to the support for media fragment, the Media Fragment Working Group showcase page includes some browsers' latest support for media fragment. Meanwhile, YouTube, Dailymotion and Vimeo all have "copy current time URL" or similar functions, but they are based on the Flash player, not HTML5. So it is still impossible to use a <video> or <audio> tag and the "src" attribute to enable media fragments for all browsers and for all those multimedia resources.

There are zillions of HTML5 video player out there and some of them offers "fallback" compatibility for Flash (or even silverlight) like JW player. Theoretically (I mean forget about the legal issues), any HTML5 player with flash fallback can embed chromeless Flash player from online video sharing websites and provide us a unique UI for all resources. By searching around, I only find MediaElement.js player (Mejs) best suits my requirement. What amazed me most about Mejs is that it has "fallforward" Flash and Silverlight plugins. Based on the browsers' compatibility for a certain file type, Mejs can automatically choose the best way to play the video: HTML5, Flash or Silverlight. More surprisingly, Mejs can directly play YouTube and Vimeo (as well as Dailymotion with a pull request). Everything in Mejs has a unique interface to control all the players (HTML5, Flash and Sliverlight).

So next, using Mejs as the background, we can start to build the polyfill to control the player highlighting media fragments. First thing first, we need a media fragment parser. Fortunately, this library has been developed by Thomas Steiner. After obtaining the start time, endtime and xywh defined in the media fragment URI, we can start highlighting both temporal and spatial fragments.

Highlighting temporal fragment is easy. When Mejs autostart playing or the user click the play button, smfplayer will jump to the start time if it is defined. Smfplayer will pause the video if the currentTime is bigger than the end time. For spatial highlighting, smfplayer create an overlay <div> on top of the video based on the xywh value in media fragment URI. You can define xywh in two ways, pixel and percent, so we need to ask the user what is the original width and height of the video if percent is used. In Ninsuna example, only the spatial fragment area is visible, while smfplayer just circle around the area with some highlight colour (default is yellow). One issue of this solution is that when the video goes to fullscreen mode, the overlay <div> will disappear.

Demo Time!

The following YouTube video is an introduction of Web Platform. Well, Web Platform is exciting for me, but what's more exciting is that the creator of this video somehow decided to call Sir Tim-Berners Lee a WEB DEVELOPER!

I want to share this interesting finding via Twitter. Tim only appears in this video for 4 seconds from 1:30, so I can send this link via twitter:

This is not enough. What I really want my followers to see is the funny title of Tim-Berners Lee on the bottom-left corner. So what I really want to share is this following media fragment URI given the video resolution is 480x320:,00:01:33&xywh=40,220,200,50

And using smfplayer, you can easily write some programmes to playback this media fragment:

This demo is similar one of the features of twitter: the in-page embedded playback of the videos from tweet list page. As a developer, you need to write different code to embed different players from YouTube, Dailymotion, Vimeo and files from web servers. With smfplayer, you can implement this function easily. You can try to play different resources at smfplayer demo page.


Followings are some useful links in case you want to know more about smfplayer:

All in all, smfplayer is a polyfill and temporal solutions for the developers who want to playback media fragments. I really hope some day different browsers can implement those functions natively.