HTML5 Video and WebVTT Support in Mobile Browsers

Since proposed by Opera Software in 2007, the HTML5 video tag has been around for a pretty long time. Video tag is a great step forward to make video the first class citizen of the Web. Unfortunately, different browsers understand the citizenships in different ways. The citizenship of AVI is terminated. WMV needs to hold Sliverlight Visa in order to get the entry clearance to browsers other than IE. FLV's visa application for iOS has been Denied by Apple Empire and Adobe has declared that he's not going to help FLV to re-apply the visa again. H.264 is like a mafia, who asks for protection fees from browsers. You see, there is no Federal Government in HTML5.

Since the smart phone joined the Web, things are getting worse again. It seems to me that there is no systemetic tests for the video compitablity on different mobile browsers. So the goal of our test is to find out which video format could be played in which browsers, or which major versions of the browsers.

Our Test

We need to emphasis something before going into the test.

1. All the browsers in the test are on mobile phones. They are not desktop versions of the browsers.
2. We only tested on android and iphones. Browsers for Windows Phone, Blackberry and Palm WebOS are not tested.
3. All the tests are carried out in mobile phones. Well, yes, you can make a phone call using 7 inch Samsung Galaxy tablets if you don't feel it's too big. But the mobile phones used in this test are 3.0 to 4.8 inches for the screensize.
4. Not all browsers, as well as a specific version of the browser, could be used in all different mobile devices. For example, there is no Opera Mobile browser on iPhone iOS 5. It is the same case for Firefox. But at the time of testing, we chose the latest version of the browser in that OS.
5. We suppose that the server which hosts the video file is properly configured in order that the file could be properly delivered.

Finally, we have tried our best to keep the data correct on the phones we have tested. But it's quite likely that some of the test results are not accurate. If you find anything that is not correct, please feel free to let me know. I would be glad to test it again. OK. Let's go!

Test 1, Playable

This test shows if a video file with a certain codec could be played in a certain browser on a mobile phone. We test mp4, ogv and webm on Android 2.3, 4.0 and iOS 5. We designed a test page, which embeds mp4, ogg and Webm videos in one page. Then we open this page in different mobile browsers on different mobile OSs. Table 1,2 and 3 shows the results.

---

Table 1.HTM5 Video Compatibility in Android 2.3

Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Opera Mobile 12.0	Yes	No	No
Opera Mini 7.0.29	No	No	No
Firefox 14.0.1	No	Yes*	Yes*
Android Native Browser	Yes	No	No

*Unlike other browsers, the video will not enter full screen when playing, and there is no full screen control in the player in Firefox 14.0.1.

Test Phones for Table 1:

1. HTC Design S, Android 2.3.5, 3.7-incn touch screen
2. Sony SK17i, Android 2.3.4, 3.0-inch touch screen
3. Samsung Galaxy S II, Android version 2.3.3, 4.3-inch touch screen
4. HTC G10, Android 2.3.5, 4.3-incn touch screen

P.S. Google Chrome is not compatible with Android 2.3.

---

Table 2.HTM5 Video Compatibility in Android 4.0.4

Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Opera Mobile 12.0	Yes	No	No
Opera Mini 7.0.29	No	No	No
Chrome 18.0	Yes	No	Yes
Firefox 14.0.1	No	Yes	Yes
Android Native Browser	Yes	No	No

Test Phones for Table 2:

1. Samsung Galaxy S III, 4.8-inch touch screen

---

Table 3.HTM5 Video Compatibility in iOS 5.1.1

Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Safari 5	Yes	No	No
Chrome 21.0	Yes	No	No
Opera Mini 7.0.29	No	No	No

Test Phones for Table 3:

1. iPhone 4S, 3.5 inches screen
2. iPhone 4, 3.5 inches screen

As a summary, here are some interesting findings:

1. There is no "killer" video codecing format for mobile browsers currently. But MP4 seems compatible with most "default" browsers, i.e. Android Native browser in Android phons and Safari in iPhones.
2. Opera Mini is pretty disappointing as no video could be played.
3. Different from desktop browsers (see this table), WebM is not well supported by the native players on mobile browsers. Interestingly, WebM is developed (or more actually, sponsored) by Google, but the Android Native Browser, which is also developed by Google, can't play WebM
4. Most mobile phones play the video in full screen mode by default and users cannot quit the full screen unless stopping the video. But Firefox is an exception. What's more, if the screen size of the mobile phone is larger than 4 inches, The full screen, if screensize is bigger than 4.3 inches, sometimes, video won't be played in full screen mode automatically.

WebVTT support in Mobile Browsers

HTML5 video tag not only brings video playing natively, but also adds some cool features alongside the video. One of them is the track tag for subtitles. Subtitles are very important for the accessiblity of videos on the Web. Many standards and de facto standards have been around for a long time. SubRip (SRT), Timed Text Markup Language and .sub, to name a few. I personally think the most promising one is Web Video Text Track (WebVTT). WebVTT is specially tailored for videos on the Web and you can also use it for audio description, chapter navigation, etc. Silvia Pfeiffer has given an very good presentation about WebVTT.

WebVTT is still a moving target and there are many things which need to be nailed before the first release. Except for the WebVTT specification itself, as developers, we are really care about the technical support of WebVTT in different browsers and are there any tools out there we can easily use? Silvia summarised the current situation of WebVTT support in different desktop browsers.

A further question we ask is that is there any mobile browsers which support WebVTT natively? After our test, the answer is NO! Table 4 is our test result. "N/A" means a certain format of the video file cannot be played in that browser, so it is meaning less to say if the WebVTT can be displayed or not.

Table 4.WebVTT Support in Different Mobile Browsers

OS	Browser	MP4, H.264	Ogg, Theora	WebM, VP8
Android 2.3	Opera Mobile 12.0	No	N/A	N/A
	Opera Mini 7.0.29	N/A	N/A	N/A
	Firefox 14.0.1	N/A	No	No
	Android Native Browser	No	N/A	N/A
Android 4.0.4	Opera Mobile 12.0	No	N/A	N/A
	Opera Mini 7.0.29	N/A	N/A	N/A
	Chrome 18.0	No	N/A	No
	Firefox 14.0.1	N/A	No	No
	Android Native Browser	No	N/A	N/A
iPhone iOS 5	Safari 5	No	N/A	N/A
	Chrome 21.0	No	N/A	N/A
	Opera Mini 7.0.29	N/A	N/A	N/A

Of course, there are tons of HTML5 video players on the Web, VideoJS, PopcornJS, MediaEleemntJS, JWPlayer, Kaltura Player, etc. Some of indeed support WebVTT on desktop browsers. But they are all "polyfills", which means the support is not native and they use javascript and CSS to somehow present WebVTT together with the video. We did test MediaElementJS on iphones, but unfortunately, it still failed to display the subtitle. However, MediaElementJS can sucessfuly display WebVTT on iPad.

Conclusions and Future Directions

The test for HTML5 video tag and WebVTT on mobile browsers is important. Developers should be clear what video could be played in the target device. On the server side, the web applications need to host the best format that compitable with the target device. Video sharing services, such as YouTube, must be adaptive enough to deliver videos to different devices and browsers. Subtitles are always important for videos, especially on mobile platforms, where the support for subtitle is not thorough yet.

With each updates or release of new versions of browsers, new codecs might be supported and old formats might be abandoned. WebVTT is currently is not well supported on mobile browsers and we are still waiting for some break-through.

The tests tell us the HTML5 video support is on the way. In the tests above, we are trying to keep things easy, i.e. we didn't bring in a lot of variables and we just used the most popular phones and most popular browsers. However, the real world is not that simple. Between mobile phones and desktop machine, we have got tablets, 7 inches to 10 inches. The screensize of mobile phones are getting bigger and bigger. I am very curious on how different browser kernels choose the display mode for these screensizes. Is there a borderline or threshold? So in the future, it might be necessary to compare the HTML5 video tag in a single browser across different size of devices (phones and tablets). We are also expecting more documents could be released or discussed regarding this aspect.

Again, if you find anything wrong or anything new, please feel free to leave your comments.

The Chinese Copy of Web

Introduction

I have been thinking to write something about the "Chinese Copy" of Web for a long time. Why? Because I think most people outside of China wouldn't know how the Chinese people spend their time on the Web. Of course, language is a problem, but more importantly, the Web in China is a silo isolated  from outside of the world. China has 538 million internet users in July 2012 and it's going to hit 800 million in 2015! (if the Chinese government is not bluffing, I believe they are not). Many of my friends are pretty interested in what a quarter of the internet populations in the whole world is doing on the Web. They ask me questions such as do they go to Facebook, view videos on YouTube? My answer is simple: we have counterpart services in China. Finally, I put my hands on the keyboard and start listing the counterpart services in China. I think this kind of information will be helpful for a lot of people in other countries to know China better.

Let's List!

I will list some of the major counterpart services on the Web in China in different aspects. I won't list many statistical comparisons as there are plenty of them which could be found on the Web.

Social Networks

No one will deny that Facebook is the largest social network website in the world. We have two major applications similar to Facebook. Renren (人人网, means "everybody"), which is formerly known as Xiaonei (校内网, which means "network for students in schools or colleges), and Kaixin (开心网, means "happy"). Facebook has claimed 500 million users in 2011, while Renren has around 137 million users in September 2011 and it's increasing by 24 million to 38 million every month.

Well, if you go to Renren or Kaixin, you will notice that, the user interface looks very similar to Facebook. If you happen to know Chinese and have an account, you will find that the personal pages are similar too. The only difference is the colour.

Search Engine

You know Google? Good! But in China, Google only takes around 16% of the search engine market. The largest search engine in China is Baidu (百度). What Baidu good at is searching Chinese documents on the Web. If you happen to know some Chinese friends, you can ask him or her to explain the video above, which tells the different between Baidu's search and Google's search, even though it explains it in a funny way.

One of Baidu's founders is Robin Li (李彦宏), who has experience to study and work in U.S. He is a kind of legend in China. Using semantic Web, I would say "Robin Li owl:sameAs Bill Gates". Now Baidu has successfully completed IPO on NASDAQ and it has extended its market to Korean and Japan, where the languages are quite similar to Chinese.

Video Sharing 

We all know YouTube, Vimeo, Dailymotion, etc. In China, there are many video uploading and sharing websites. Two famous ones are Tudou (土豆, which means "potato") and youku (优酷). Tudou and youku has merger plan recently, but it's still in progress. After the merger, "Youku Tudou" will become the lagest video site. I am not a business man, so I think both Youku and Tudou serve similar content and they are also similar to YouTube.

Online Shopping

Please gimme the first two websites in your mind that you will go when you want to buy something. Yes, they are eBay and Amazon. In UK, eBay is the best place to sell or buy something quickly in a reasonable price for individuals or small businesses. In China, we have an online shopping website called Taobao (淘宝网, which means "looking for treasures"), owned by Alibaba Group. For Taobao, you only need one sentence to describe it: "you can buy anything there". Jack Ma (马云), who owns Taobao and Alibaba Group, is one of the first entrepreneurs, who brought e-commerce into China. Interestingly, he claims that he doesn't know anything about computer or internet, I mean the technology side. He even find himself hard to send an email!

One of the most widely used products of eBay is Paypal. The counterpart in China is called Alipay (支付宝, which means "a tool can pay for anything any ways"). No surprising, Alipay is developed by Alibaba Group (God! We just copied EVERYTHING, even the way it exists!). By 2010, it is said there are 550 million registers in Alipay, not only in China but around all Chinese communities all over the world.

Next to eBay, we are also familiar with Amazon. But Amazon is not very successful in China, because of the existence of Dangdang (当当网), which was founded by a couple, Peggy Yu and Li Guoqing. Similar, Dangdang started by selling books online and then it extends its market to household merchandise, digital devices, etc.

Instant Messaging

Anybody remembers what is ICQ? Maybe you know MSN Messenger, Yahoo Messenger or Skype. As far as I can recall, ICQ is the first instant messaging application I have ever use. When we no longer (maybe you never did) pay any attention to ICQ, the counterpart of ICQ in China is growing tremendously out of anybody's imagination. The application is called Tencent QQ (腾讯QQ). I personally think it is a copy of ICQ as the original name of Tencent QQ is OICQ, Open ICQ.

Instant messaging application may not be totally categorised as Web-based application as it is originally a desktop application. But I need to mention Tencent, the company, and QQ here, because it's another legend in China. Even though  the CEO of Tencent Ma Huateng (马化腾) has been severely accused for the act of plagiarism, he is still ranked as the second richest person in IT in mainland China, next to Robin Li, in 2010. 

Which application in the world has the largest population of registration? Facebook, YouTube, Twitter, Google plus? No! I wouldn't be surprise to know that in September 2011, there were more than 710 million registered users in Tencent QQ! That is nearly the population of the Europe population, more than twice as much as the US population. Even though it is a copy, I would say it's a very successful copy. In China, you can safely say you don't know who is the president of US, but it would be a shame to tell somebody you don't have a QQ membership. 

Blog

The blog application are not dominated by one or two company now in Europe or U.S. After the downfall of MySpace, MSN spaces, the blog applications have been largely replaced by or further integrated with Social networks. In China, the situation is quite similar. Major portal websites all have their blog services, such as Sina blog (新浪博客), Tencent QZone (QQ空间). 

Micro Blog

Twitter makes the information spread quicker than you can ever imagine and that's why twitter is strictly blocked in China. However, entrepreneurs would never be stopped as long as there are potentials of making money. In China, there are several counterparts of Twitter and they are called Weibo (微博). The two major Weibo in China are Sina Weibo (新浪微博) and Tencent Weibo (腾讯微博). Of course, they are monitored and controlled by the government, as well as Baidu I mentioned earlier. Any "sensitive words" will be blocked from searching or publishing. But as a Chinese, I should proudly say that 140 characters in Chinese Characters are far more expressive than English :)

I have just checked the followers (27th of August 2012),the most popular person on Twitter is Lady Gaga, around 28 million followers (I didn't follow her), Meanwhile, the most popular Chinese person on Weibo is Yao Chen, a Chinese actress, who has around 23 million followers, but nearly all of them are Chinese! Yao Chen even has more followers than Britney Spears and Barack Obama. 

Wiki

Everybody is impressed by Jimmy Wales and his Wikipedia. Wikipedia is now the de facto knowledge hub of, at least, English. You can probably find any facts you want to know on Wikipedia. However, wikipedia is blocked in China, as there are many anti-government content. Well, I don't care if the content is anti-government or not, but we should find a way to manage our own knowledge, right? So our counterpart services are Baidu Baike (百度百科) and Hudong Baike (互动百科). Baike in Chinese means "encyclopaedia". 

Different from Wikipedia, which funded by non-profit organization, Baidu Baike and Hudong Baike are run by for-profit companies, Baidu Inc. and Hudong Inc respectively. The CEO of Hudong is Pan Haidong who got his PhD degreen in Boston University in US.  

Everybody in the semantic Web area knows DBpedia. In the ISWC 2011 conference in Bonn, Germany, I met a few guys from Shanghai Jiao Tong University, who was start creating the Chinese version of DBpedia: zhishi.me (zhishi in Chinese means "knowledge"). It's a very impressive work and hopefully, there will be more progress coming out. Unfortunately, I don't think will be possible to merge zhishi.me with DBpedia, as the data in zhishi.me is not freely available. The 4.1 and 4.2 in terms and conditions in Hudong Haike clearly states that Hudong Baike reserves all the rights for anything uploaded to the website. 

Downloading

"Intellectual property" is not a very popular phrase until the trademark war launched by Wei Guan Corp. China. I have been to UK for 5 years, and I still cannot find any place to download movies, software, music for free, or even watch freely online. Pirate Bay is an exception. There are some file sharing systems based on Cloud, but unfortunately, none of them lives longer than a few months as they are shutdown quickly. 

In China, this is totally a different story. Xunlei (迅雷, which means "as fast as thunder") and VeryCD (电驴, which means "electronic donkey") host millions of movies, music records, TV programmes, cartoons, even pirate software. Everybody with an account can download whatever there. The only difference between paid users and free users is that paid users can download really fast. Xunlei and VeryCD has recently got sued for intellectual properties issues and many content are no longer available for downloading. What's more, they have somehow limit the downloading service to users from mainland China. 

If you have time and know Chinese, take a look around in Xunlei and VeryCD, maybe you can find Total Recall, The Expendables 2 and other latest HD movies there. 

Integration among these Applications

Like the integration among Facebook, twitter, YouTube, Google Plus, the counterpart services in China also implement the integration functions. If you find a good video in Tudou, you can click the "share button" (in Chinese its called 分享 fenxiang) to share it in Renren or send it via Weibo. When you take a picture using your smartphone, you can put it immediately on Renren or Weibo. 

Summary

As a summary, I list all the counterpart services:

• Social Networks: Renren (人人网) and Kaixin (开心网)
• Search Engine: Baidu (百度)
• Video Sharing: Tudou (土豆) and youku (优酷)
• Online Shopping: Taobao (淘宝网) Dangdang (当当网)
• Online Payment: Alipay (支付宝)
• Instant Messaging: Tencent QQ (腾讯QQ)
• Blog: Sina blog (新浪博客) and Tencent QZone (QQ空间)
• Mico Blog: Sina Weibo (新浪微博) and Tencent Weibo (腾讯微博)
• Wikipedia: Baidu Baike (百度百科) and Hudong Baike (互动百科)
• Downloading: Xunlei (迅雷) and VeryCD (电驴) 

For "security" reasons, Facebook, YouTube, Twitter, Wikipedia have been totally blocked in main land China. Google has been moved to Hong Kong because they cannot reach the agreement with the government for the email and sensitive words filtering. eBay and Amazon currently are not in good position in the competition in China. Considering the 710 million registered users of QQ, MSN Messenger has no advantage. Dear me, only Microsoft survives!

Limitations

OK. Since we have counterpart services in every aspect of Web applications, what are the limitations for these services in China? Let's list a few of them.

The first problem is the openness. The Chinese Web is totally contradict to Tim Berners Lee's vision for the openness of the Web. With in the internet in China, you cannot view YouTube or use twitter. On the other hand, users in outside of China cannot view some of the video on Tudou or Youku as they can be only viewed by users within mainland China. 

Secondly, most the web applications have accessibility problems. They have ads and popping up windows everywhere. To make things worse, many applications still only display well in IE! Thanks to Steve Jobs, more Chinese buy Macbook, more application owners will consider to make them compatible with Safari, Firefox and Google Chrome. I have to say, Facebook, YouTube and Twitter they are very accessible for keyboard and screen readers. But Renren, Kaixin and Tudou are too dazzling for me. I really wonder how a blind user could navigate through the screen. 

Another problem is the security. Many applications, such as Tencent QQ have been accused to illegally collect users' data. For Alipay, you need to install a plugin for the browser in order to use it. I've never heard that Paypal has some browser plugin.

Conclusion

You see, Chinese people are living very well inside the silo. We have millions of users, we have a huge market and many entrepreneurs have devote their passions into this market for decades. The Web eco-system has started to grow in China, even though monitored by the government. It's very good to know that many entrepreneurs in IT had experience of studying and working in US or other developed countries. So they bring back the advanced ideas and start their own career in China. Chinesizing the ideas is good at first, but we should feel the hazard because a market without creativity will not live long. So does the Web in China. Are there any Web applications that origins from Chinese people and have been widely adapted around the world? I don't see any to be honest.

In my own opinion, nothing would develop well without openness, which is the original essence of Web and Internet. Chinese people have been well taught by the proverb "behind an able man, there are always other able men". So "behind an isolated Web,  there are always better Web", we will finally know one day there is a bigger Web outside the silo.