2009年12月11日星期五

Some Twist on the Blog

Creative Commons License
本作品采用知识共享署名-非商业性使用 2.5 中国大陆许可协议进行许可。

I spent some time on twist my blog(grissiom.blogspot.com) today. The major changes are direct RSS feed to feedburner( http://feeds.feedburner.com/Grissiom ) and add a CC icon on the blog. I choose NOT allow commercial uses of your work and Allow modifications of your work(not require as long as others share alike). So the articles on this blog will be covered by that license. Take care of this.

Also, I found a script named rst2wp.py from Using ReStructured Text with WordPress which is wonderful work. I adapted it to meet my blogspot's need. The script can be found on my MLES repository along with my other Make Life Easier Scripts ;-)

Hope you could enjoy them :-)

2009年12月10日星期四

Extract Audio to Ogg with mencoder and oggenc

If you got a video of a though-provoking speech, you may want to convert it to a audio file which could copy into your mp3 player and be with you all the time. You can listen to it without boot up the computers. In some situations listen is much easier than seeing.

mp3 is one of the choice of the audio file formats. But for patent reason, my distribution doesn't ship lame. And I know there is ogg format [Ref1] which is more open and free. So I decided to encode all the audio into ogg format.

mencoder is part of mplayer which is very very powerful multi-media player. With mencoder/mplayer, you can play nearly all kind of video/audio files. We only care about the audio here. Let's start to work!

Basicly we need to extract the audio from the multi-media stream first:

mencoder video.mp4 -of rawaudio -ovc copy -oac pcm -o audio.pcm

-of rawaudio tells the mencoder to output audio stream only. Although the man page told me the -of option is still BETA but it works like a charm here. -ovc copy tells mencoder not to decode the video stream, which would consume less resources on your computer. -oac pcm tells mencoder to encode the audio to uncompressed PCM. Now, we can use ogg --raw audio.pcm to convert the raw pcm file to an ogg file.

But it's far from perfect. One big problem is this way of converting will last long time. When mencoder is extracting, oggenc doesn't convert simultaneously. So the time will be doubled. The other problem is you have to provide a relatively large temperary file to contain the pcm data. The method that could kill two birds with one stone is using a pipe. So the second version will be:

mencoder video.mp4 -of rawaudio -ovc copy -oac pcm -o - | \
    oggenc --raw - -o audio.ogg                            

But this version have bugs too. mencoder will print out something like:

MEncoder r29789-4.3.3 (C) 2000-2009 MPlayer Team
success: format: 0  data: 0x0 - 0x34468059      
libavformat file format detected.               
[lavf] Audio stream found,  -aid 0              
[lavf] Video stream found,  -vid 1              
VIDEO:  [H264]  1280x720  24bpp  29.970 fps    0.0 kbps ( 0.0 kbyte/s)
[V] filefmt:44  fourcc:0x34363248  size:1280x720  fps:29.970  ftime:=0.0334
========================================================================== 
Opening audio decoder: [faad] AAC (MPEG2/4 Advanced Audio Coding)          
FAAD: compressed input bitrate missing,  assuming 128kbit/s!               
AUDIO: 44100 Hz,  2 ch,  s16le,  128.0 kbit/9.07% (ratio: 16000->176400)
Selected audio codec: [faad] afm: faad (FAAD AAC (MPEG-2/MPEG-4 Audio))    
========================================================================== 
Ignoring video stream!                                                     
videocodec: framecopy (1280x720 24bpp fourcc=34363248)                     
Pos:   0.7s     22f ( 0%)  0.00fps Trem:   0min 141mb  A-V:0.068 [0:1411]  
Skipping frame!                                                            

which will corrupt the pcm stream to stdout and cause a corrupted ogg file eventually. In man page, it said -quiet will Make console output less verbose and -really-quiet will Display even less output and status messages than with -quiet but there is no guarantee that There won't be any message to stdout. That's unbearable for our condition.

So I come up with the idea that shell can redirect file-descriptor 3-10 to stdout, stdin or stderr. So we can write the pcm stream to file descriptor 3, and redirect the fd 3 to stdout. Then, redirect the origin stdout where the messages going to /dev/null. But how to tell mencoder write the pcm stream to fd 3? The trick is simple. Actually, fd n is /proc/self/fd/n for the process. What a great design of Linux system! Now, the pipe should be clean. Here is the final version:

mencoder video.mp4 -of rawaudio -ovc copy -oac pcm \
    -o /proc/self/fd/3 3>&1 1>/dev/null | \
    oggenc --raw - -o audio.ogg

There are two things you should care:

  1. You must redirect fd 3 when dumping things into it, otherwise it will break.
  2. The order in redirection is important. 3>&1 means redirect 3 to what 1 currently is. If you 1>/dev/null and then 3>&1, fd 3 will be redirected to /dev/null as well.

Now, you can extract the audio to ogg file flawlessly. Big thanks to http://www.commandlinefu.com where tell me the usage of mencoder.

[Ref1]http://www.vorbis.com/

2009年12月8日星期二

Gappproxy plus pac is your friend to break the wall

Nearly everyone on the earth know that China has a Great Fire Wall to monitor and control connections abroad. It block some web pages that should not be seen by teenagers or have politic issues. But it blocks some useful webs too. For example, *.blogspot.com where holds many technical blogs, http://chrome.google.com/extensions where you can extent you Chromium... It's unbearable for someone(like me, although I tolerated it for quite a long time) got "Connection Reset" now and then. So I decided and managed to get through it with Gappproxy.

Why Gappproxy?

It based on GAE(Google App Engine), which is free(within quotas) and powerful and 7x24 on line. Someone may found GAE has been blocked too. I will talk on it later. There are many ways to break walls such as buy a VPS abroad and use ssh tunnel through it or use VPN. But neither I don't have the money and time to buy a VPS nor many free VPN sites are not accessible, I choose Gappproxy as my way to embrace the free world.

There are some tutorials that could teach you how to setup a Gappproxy for yourself, search it on baidu.com will give you many results. (I nearly never search sensitive topic on Google because it would bring it into trouble) So I won't reinvent the wheel. One thing I want to mention is the Gappproxy main site was blocked too(not surprisingly). So you could not retrieve the source files unless you pass the wall... Here is a link for the tarballs and you may find it accessible behind the wall:

http://gappproxy.googlecode.com/files/gappproxy-1.0.0beta.tar.gz

In this tarball, fetchserver folder have the files that should be deployed to GAE, which is equivalent to the fetchserver tarball in most tutorials; in localproxy folder there is the client need to be run on your box. Don't forget to adjust them before you deploy/run :-)

After it was fully setup, set your browser's proxy to localhost:8000. I prefer to use a pac file to do the trick.(See below) After got it run successfully, you may need to update your Gappproxy to the latest version.

Access to GAE

Actually it's quite easy to do the trick. Following this link http://ftofficer.spaces.live.com/blog/cns!423B72634E2F6B7E!1073.entry (which is also blocked, but you can search "appspot www.google.cn" in baidu.com. It will be the third entry. Take snapshot with baidu you can know what he/she is saying about, in Chinese), you can edit your /etc/hosts to let your appspot point to a www.google.cn's IP. Now your GAE will be accessible. I hope GFW won't block www.google.cn ;-)

Use a pac file to setup proxy for your web browser

The last blog I mentioned above have taught me how to use a pac file to set up proxy for web browser. Why use pac file?

  1. It's plain text file with JavaScript syntax. So it will be editable with any of your favorite text editor. I know nothing about JavaScript but a pac file is rather easy to write and understand and thus, easy to extend.
  2. It's flexible and powerful. You can set different proxies for different urls or hosts. You can even tell the browser to connect to the web directly for some urls.
  3. It's browser-independent. It's not a fancy Firefox plugin which could not run on Chromium. It's a standard which is supported by most of the browsers, Firefox, Chromium, Opera... Maybe if you find a browser which does not support it, you should not use that browser ;-) (I haven't tested it in IE though.. ;)

There are tutorials teaching you what a pac file is and how to write a pac file [Ref1], [Ref2] , so I won't reinvent the wheel again ;)

Let me use one sentence to finish this article:

Standing on the Cloud, where is the Wall?
[Ref1]http://en.wikipedia.org/wiki/Proxy_auto-config
[Ref2]http://code.google.com/p/gappproxy/wiki/UsingProxy_StepByStep# _2._使用_PAC_文件,强大、智能