WIPVideo

Version 34 (Saúl Ibarra Corretgé, 08/27/2014 05:22 pm)

1 1 Saúl Ibarra Corretgé
h1. WIPVideo
2 1 Saúl Ibarra Corretgé
3 5 Saúl Ibarra Corretgé
Notes while video is a work in progress. Repository: http://devel.ag-projects.com/cgi-bin/darcsweb.cgi?r=saul/python-sipsimple-video;a=summary
4 1 Saúl Ibarra Corretgé
5 1 Saúl Ibarra Corretgé
h2. Dependencies
6 1 Saúl Ibarra Corretgé
7 1 Saúl Ibarra Corretgé
The following dependencies are required to build PJSIP with video support (including H264)
8 1 Saúl Ibarra Corretgé
9 1 Saúl Ibarra Corretgé
* ffmpeg (libavformat, libswscale, libavcodec, libavutil)
10 1 Saúl Ibarra Corretgé
* libx264
11 1 Saúl Ibarra Corretgé
12 1 Saúl Ibarra Corretgé
Versions I have tried:
13 1 Saúl Ibarra Corretgé
14 1 Saúl Ibarra Corretgé
* ffmpeg (2.0 release)
15 1 Saúl Ibarra Corretgé
* libx264 (snapshot-20130806-2245-stable)
16 1 Saúl Ibarra Corretgé
17 20 Saúl Ibarra Corretgé
h2. Installing dependencies (Debian / Ubuntu systems)
18 20 Saúl Ibarra Corretgé
19 20 Saúl Ibarra Corretgé
The situation here is a bit sad. Both Debian and Ubuntu ship with libav instead of FFmpeg, but libraries are called the same. PJSIP had to be patched in order to properly work with libav, and the patch as not yet been included upstream.
20 1 Saúl Ibarra Corretgé
21 20 Saúl Ibarra Corretgé
On Debian, when the Debian-Multimedia repositories are used (quite common) you get FFmpeg and not libav. Oh the joy!
22 20 Saúl Ibarra Corretgé
23 20 Saúl Ibarra Corretgé
Installing dependencies on Debian:
24 20 Saúl Ibarra Corretgé
25 20 Saúl Ibarra Corretgé
<pre>
26 34 Saúl Ibarra Corretgé
apt-get install libv4l-dev libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libswresample-dev libx264-dev libavcodec-extra
27 1 Saúl Ibarra Corretgé
</pre>
28 20 Saúl Ibarra Corretgé
29 20 Saúl Ibarra Corretgé
If using the Debian-Multimedia repositories, do not install libavcodec-extra.
30 20 Saúl Ibarra Corretgé
31 20 Saúl Ibarra Corretgé
Installing dependencies on Ubuntu:
32 20 Saúl Ibarra Corretgé
33 20 Saúl Ibarra Corretgé
<pre>
34 34 Saúl Ibarra Corretgé
apt-get install libv4l-dev libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libswresample-dev libx264-dev libavcodec-extra53
35 20 Saúl Ibarra Corretgé
</pre>
36 1 Saúl Ibarra Corretgé
37 20 Saúl Ibarra Corretgé
*Note on H.264 support*: In order to have H.264 support, FFmpeg (or libav) need to be compiled with support for it. The standard packages don't, hence the need for installing the libavcodec-extra packages.
38 21 Saúl Ibarra Corretgé
39 20 Saúl Ibarra Corretgé
h2. Compiling dependencies
40 3 Saúl Ibarra Corretgé
41 3 Saúl Ibarra Corretgé
All dependencies will be compiled to a directory in the user's HOME directory:
42 3 Saúl Ibarra Corretgé
43 3 Saúl Ibarra Corretgé
<pre>
44 34 Saúl Ibarra Corretgé
export MY_FFMPEG_LIBS=$HOME/work/ag-projects/video/local
45 3 Saúl Ibarra Corretgé
</pre>
46 3 Saúl Ibarra Corretgé
47 3 Saúl Ibarra Corretgé
h3. libx264
48 3 Saúl Ibarra Corretgé
49 3 Saúl Ibarra Corretgé
<pre>
50 3 Saúl Ibarra Corretgé
./configure --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
51 3 Saúl Ibarra Corretgé
make
52 3 Saúl Ibarra Corretgé
make install
53 8 Saúl Ibarra Corretgé
54 8 Saúl Ibarra Corretgé
# If a 32bit build is wanted, then run this configure instead:
55 8 Saúl Ibarra Corretgé
./configure --host=i386-apple-darwin --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
56 3 Saúl Ibarra Corretgé
</pre>
57 3 Saúl Ibarra Corretgé
58 3 Saúl Ibarra Corretgé
h3. ffmpeg
59 3 Saúl Ibarra Corretgé
60 3 Saúl Ibarra Corretgé
<pre>
61 3 Saúl Ibarra Corretgé
# Some exports
62 3 Saúl Ibarra Corretgé
export PKG_CONFIG_PATH=$MY_VIDEO_LIBS/lib/pkgconfig
63 1 Saúl Ibarra Corretgé
64 7 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`"
65 3 Saúl Ibarra Corretgé
make
66 3 Saúl Ibarra Corretgé
make install
67 1 Saúl Ibarra Corretgé
68 1 Saúl Ibarra Corretgé
# If a 32bit build is wanted do:
69 9 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`" --cc="gcc -m32" --disable-asm
70 9 Saúl Ibarra Corretgé
# TODO: I WANT MY ASM
71 4 Saúl Ibarra Corretgé
</pre>
72 4 Saúl Ibarra Corretgé
73 4 Saúl Ibarra Corretgé
74 4 Saúl Ibarra Corretgé
h2. Proposed API
75 12 Saúl Ibarra Corretgé
76 34 Saúl Ibarra Corretgé
TODO: API changed, update this.
77 34 Saúl Ibarra Corretgé
78 13 Saúl Ibarra Corretgé
API for video components is based on 2 different types of video capable entities:
79 13 Saúl Ibarra Corretgé
80 13 Saúl Ibarra Corretgé
* VideoProducer: a source for video data, for example a video camera or a remote video stream
81 13 Saúl Ibarra Corretgé
* VideoConsumer: a sink or destination for video data, for example a video rendering window
82 13 Saúl Ibarra Corretgé
83 13 Saúl Ibarra Corretgé
h3. Data flow
84 13 Saúl Ibarra Corretgé
85 13 Saúl Ibarra Corretgé
Data flow works in _pull_ fashion, that is, a producer doesn't start to produce data until there is a consumer which will consume it.
86 13 Saúl Ibarra Corretgé
87 13 Saúl Ibarra Corretgé
h3. VideoProducer
88 13 Saúl Ibarra Corretgé
89 13 Saúl Ibarra Corretgé
Produces video data. 
90 13 Saúl Ibarra Corretgé
91 13 Saúl Ibarra Corretgé
Internal API:
92 13 Saúl Ibarra Corretgé
93 1 Saúl Ibarra Corretgé
* _add_consumer: attach a consumer, called by the consumer
94 13 Saúl Ibarra Corretgé
* _remove_consumer: detach a consumer from a producer, called by the consumer
95 13 Saúl Ibarra Corretgé
96 13 Saúl Ibarra Corretgé
97 18 Saúl Ibarra Corretgé
Public API:
98 1 Saúl Ibarra Corretgé
99 1 Saúl Ibarra Corretgé
* start: start producing video as soon as a consumer is attached
100 18 Saúl Ibarra Corretgé
* stop: immediately stop producing data
101 18 Saúl Ibarra Corretgé
* close: remove all consumers and stop producing video data (also deallocate all C structures)
102 29 Saúl Ibarra Corretgé
* producer_port: pointer to the pjmedia_port object
103 14 Saúl Ibarra Corretgé
104 14 Saúl Ibarra Corretgé
h3. VideoConsumer
105 14 Saúl Ibarra Corretgé
106 14 Saúl Ibarra Corretgé
Consumes video data.
107 14 Saúl Ibarra Corretgé
108 14 Saúl Ibarra Corretgé
Public API:
109 14 Saúl Ibarra Corretgé
110 14 Saúl Ibarra Corretgé
* producer: (r/w property) attach this consumer to a producer, in order to render the video data generated by the producer. If set to None, it's detached
111 24 Saúl Ibarra Corretgé
* consumer_port: pointer to the pjmedia_port object
112 14 Saúl Ibarra Corretgé
* close: detach from producer and free all resources (also deallocate all C structures)
113 30 Saúl Ibarra Corretgé
114 15 Saúl Ibarra Corretgé
h3. Producer and consumer objects
115 15 Saúl Ibarra Corretgé
116 15 Saúl Ibarra Corretgé
* VideoDevice: Producer, acquires video from a user camera.
117 15 Saúl Ibarra Corretgé
* VideoWindow: Consumer, renders video in an SDL window. Extra methods: show/hide. Properties: native_handle, size.
118 15 Saúl Ibarra Corretgé
119 15 Saúl Ibarra Corretgé
* LocalVideoStream: Consumer, takes video from a VideoDevice and sends it to the remote party.
120 15 Saúl Ibarra Corretgé
* RemoteVideoStream: Producer, produces video sent by the remote party.
121 15 Saúl Ibarra Corretgé
122 15 Saúl Ibarra Corretgé
These are just theoretical objects, won't be implemented in the first go.
123 15 Saúl Ibarra Corretgé
124 17 Saúl Ibarra Corretgé
* VideoFileWriter: Consumer, saves incoming video data to a video file.
125 15 Saúl Ibarra Corretgé
* VideoFilePlayer: Producer, produces video data out of a video file.
126 15 Saúl Ibarra Corretgé
127 15 Saúl Ibarra Corretgé
* VideoMixer: Producer/Consumer, consumes video from multiple sources and produces aggregated video data.
128 16 Saúl Ibarra Corretgé
129 34 Saúl Ibarra Corretgé
NOTE: pjsip does have a AVI file player, which also seems to support audio. (this could be used to stream a movie, for example)
130 23 Saúl Ibarra Corretgé
131 23 Saúl Ibarra Corretgé
h2. H264
132 23 Saúl Ibarra Corretgé
133 23 Saúl Ibarra Corretgé
Information about H264 profiles:
134 23 Saúl Ibarra Corretgé
135 23 Saúl Ibarra Corretgé
* http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels
136 23 Saúl Ibarra Corretgé
* https://supportforums.cisco.com/blog/149561/video-telepresence-sip-h264-profile-level-id
137 28 Saúl Ibarra Corretgé
138 28 Saúl Ibarra Corretgé
h2. OpenH264 implementation
139 28 Saúl Ibarra Corretgé
140 28 Saúl Ibarra Corretgé
PJSIP has an initial version of a wrapper for Cisco's OpenH264 implementation (http://www.openh264.org/). http://trac.pjsip.org/repos/changeset?reponame=&old=4815%40%2F&new=4815%40%2F
141 28 Saúl Ibarra Corretgé
142 28 Saúl Ibarra Corretgé
OpenH264 seems to implement SVC, which is better than AVC.