WIPVideo

Version 20 (Saúl Ibarra Corretgé, 03/28/2014 10:50 am)

1 1 Saúl Ibarra Corretgé
h1. WIPVideo
2 1 Saúl Ibarra Corretgé
3 5 Saúl Ibarra Corretgé
Notes while video is a work in progress. Repository: http://devel.ag-projects.com/cgi-bin/darcsweb.cgi?r=saul/python-sipsimple-video;a=summary
4 1 Saúl Ibarra Corretgé
5 1 Saúl Ibarra Corretgé
h2. Dependencies
6 1 Saúl Ibarra Corretgé
7 1 Saúl Ibarra Corretgé
The following dependencies are required to build PJSIP with video support (including H264)
8 1 Saúl Ibarra Corretgé
9 1 Saúl Ibarra Corretgé
* SDL 2
10 1 Saúl Ibarra Corretgé
* ffmpeg (libavformat, libswscale, libavcodec, libavutil)
11 1 Saúl Ibarra Corretgé
* libx264
12 1 Saúl Ibarra Corretgé
13 1 Saúl Ibarra Corretgé
Versions I have tried:
14 1 Saúl Ibarra Corretgé
15 1 Saúl Ibarra Corretgé
* SDL (2.0.0-7655)
16 1 Saúl Ibarra Corretgé
* ffmpeg (2.0 release)
17 1 Saúl Ibarra Corretgé
* libx264 (snapshot-20130806-2245-stable)
18 1 Saúl Ibarra Corretgé
19 1 Saúl Ibarra Corretgé
h2. Patches
20 1 Saúl Ibarra Corretgé
21 11 Saúl Ibarra Corretgé
-If the above versions are used, PJSIP needs to be patched with the attached patch (avcodec.diff) or it won't compile. This does not occur when compiling it against the latest library versions on Debian unstable.- No longer needed.
22 3 Saúl Ibarra Corretgé
23 20 Saúl Ibarra Corretgé
h2. Installing dependencies (Debian / Ubuntu systems)
24 20 Saúl Ibarra Corretgé
25 20 Saúl Ibarra Corretgé
The situation here is a bit sad. Both Debian and Ubuntu ship with libav instead of FFmpeg, but libraries are called the same. PJSIP had to be patched in order to properly work with libav, and the patch as not yet been included upstream.
26 20 Saúl Ibarra Corretgé
27 20 Saúl Ibarra Corretgé
On Debian, when the Debian-Multimedia repositories are used (quite common) you get FFmpeg and not libav. Oh the joy!
28 20 Saúl Ibarra Corretgé
29 20 Saúl Ibarra Corretgé
Installing dependencies on Debian:
30 20 Saúl Ibarra Corretgé
31 20 Saúl Ibarra Corretgé
<pre>
32 20 Saúl Ibarra Corretgé
apt-get install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libsdl2-dev libx264-dev libavcodec-extra
33 20 Saúl Ibarra Corretgé
</pre>
34 20 Saúl Ibarra Corretgé
35 20 Saúl Ibarra Corretgé
If using the Debian-Multimedia repositories, do not install libavcodec-extra.
36 20 Saúl Ibarra Corretgé
37 20 Saúl Ibarra Corretgé
Installing dependencies on Ubuntu:
38 20 Saúl Ibarra Corretgé
39 20 Saúl Ibarra Corretgé
<pre>
40 20 Saúl Ibarra Corretgé
apt-get install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libsdl2-dev libx264-dev libavcodec-extra53
41 20 Saúl Ibarra Corretgé
</pre>
42 20 Saúl Ibarra Corretgé
43 20 Saúl Ibarra Corretgé
*Note on H.264 support*: In order to have H.264 support, FFmpeg (or libav) need to be compiled with support for it. The standard packages don't, hence the need for installing the `libavcodec-extra` packages.
44 20 Saúl Ibarra Corretgé
45 3 Saúl Ibarra Corretgé
h2. Compiling dependencies
46 3 Saúl Ibarra Corretgé
47 3 Saúl Ibarra Corretgé
All dependencies will be compiled to a directory in the user's HOME directory:
48 3 Saúl Ibarra Corretgé
49 3 Saúl Ibarra Corretgé
<pre>
50 3 Saúl Ibarra Corretgé
export MY_VIDEO_LIBS=$HOME/work/ag-projects/video/local
51 3 Saúl Ibarra Corretgé
</pre>
52 3 Saúl Ibarra Corretgé
53 3 Saúl Ibarra Corretgé
h3. libx264
54 3 Saúl Ibarra Corretgé
55 3 Saúl Ibarra Corretgé
<pre>
56 3 Saúl Ibarra Corretgé
./configure --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
57 3 Saúl Ibarra Corretgé
make
58 3 Saúl Ibarra Corretgé
make install
59 8 Saúl Ibarra Corretgé
60 8 Saúl Ibarra Corretgé
# If a 32bit build is wanted, then run this configure instead:
61 8 Saúl Ibarra Corretgé
./configure --host=i386-apple-darwin --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
62 3 Saúl Ibarra Corretgé
</pre>
63 3 Saúl Ibarra Corretgé
64 3 Saúl Ibarra Corretgé
h3. ffmpeg
65 3 Saúl Ibarra Corretgé
66 3 Saúl Ibarra Corretgé
<pre>
67 3 Saúl Ibarra Corretgé
# Some exports
68 3 Saúl Ibarra Corretgé
export PKG_CONFIG_PATH=$MY_VIDEO_LIBS/lib/pkgconfig
69 1 Saúl Ibarra Corretgé
70 7 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`"
71 3 Saúl Ibarra Corretgé
make
72 3 Saúl Ibarra Corretgé
make install
73 9 Saúl Ibarra Corretgé
74 9 Saúl Ibarra Corretgé
# If a 32bit build is wanted do:
75 9 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`" --cc="gcc -m32" --disable-asm
76 9 Saúl Ibarra Corretgé
# TODO: I WANT MY ASM
77 3 Saúl Ibarra Corretgé
</pre>
78 3 Saúl Ibarra Corretgé
79 3 Saúl Ibarra Corretgé
h3. SDL
80 3 Saúl Ibarra Corretgé
81 3 Saúl Ibarra Corretgé
<pre>
82 3 Saúl Ibarra Corretgé
./configure --disable-audio --prefix=$MY_VIDEO_LIBS
83 3 Saúl Ibarra Corretgé
make
84 3 Saúl Ibarra Corretgé
make install
85 10 Saúl Ibarra Corretgé
86 10 Saúl Ibarra Corretgé
# If a 32bit build is wanted:
87 10 Saúl Ibarra Corretgé
CFLAGS="-arch i386" CXXFLAGS="-arch i386" LDFLAGS="-arch i386" ./configure --disable-audio --prefix=$MY_VIDEO_LIBS
88 3 Saúl Ibarra Corretgé
</pre>
89 4 Saúl Ibarra Corretgé
90 4 Saúl Ibarra Corretgé
h2. Compiling PJSIP (pjsua, for testing)
91 4 Saúl Ibarra Corretgé
92 4 Saúl Ibarra Corretgé
<pre>
93 4 Saúl Ibarra Corretgé
svn co http://svn.pjsip.org/repos/pjproject/trunk pjsip
94 4 Saúl Ibarra Corretgé
cd pjsip
95 4 Saúl Ibarra Corretgé
echo "#define PJMEDIA_HAS_VIDEO 1" > pjlib/include/pj/config_site.h
96 4 Saúl Ibarra Corretgé
./configure --with-sdl=$MY_VIDEO_LIBS --with-ffmpeg=$MY_VIDEO_LIBS
97 4 Saúl Ibarra Corretgé
make dep
98 4 Saúl Ibarra Corretgé
make
99 4 Saúl Ibarra Corretgé
# pjsua will be located in pjsip-apps/bin/
100 4 Saúl Ibarra Corretgé
</pre>
101 12 Saúl Ibarra Corretgé
102 12 Saúl Ibarra Corretgé
h2. Proposed API
103 12 Saúl Ibarra Corretgé
104 13 Saúl Ibarra Corretgé
API for video components is based on 2 different types of video capable entities:
105 13 Saúl Ibarra Corretgé
106 13 Saúl Ibarra Corretgé
* VideoProducer: a source for video data, for example a video camera or a remote video stream
107 13 Saúl Ibarra Corretgé
* VideoConsumer: a sink or destination for video data, for example a video rendering window
108 13 Saúl Ibarra Corretgé
109 13 Saúl Ibarra Corretgé
h3. Data flow
110 13 Saúl Ibarra Corretgé
111 13 Saúl Ibarra Corretgé
Data flow works in _pull_ fashion, that is, a producer doesn't start to produce data until there is a consumer which will consume it.
112 13 Saúl Ibarra Corretgé
113 13 Saúl Ibarra Corretgé
h3. VideoProducer
114 13 Saúl Ibarra Corretgé
115 13 Saúl Ibarra Corretgé
Produces video data. 
116 13 Saúl Ibarra Corretgé
117 13 Saúl Ibarra Corretgé
Internal API:
118 13 Saúl Ibarra Corretgé
119 13 Saúl Ibarra Corretgé
* _add_consumer: attach a consumer, called by the consumer
120 13 Saúl Ibarra Corretgé
* _remove_consumer: detach a consumer from a producer, called by the consumer
121 13 Saúl Ibarra Corretgé
122 18 Saúl Ibarra Corretgé
123 1 Saúl Ibarra Corretgé
Public API:
124 1 Saúl Ibarra Corretgé
125 18 Saúl Ibarra Corretgé
* start: start producing video as soon as a consumer is attached
126 18 Saúl Ibarra Corretgé
* stop: immediately stop producing data
127 13 Saúl Ibarra Corretgé
* close: remove all consumers and stop producing video data
128 14 Saúl Ibarra Corretgé
* producer_port: pointer to the pjmedia_port object
129 14 Saúl Ibarra Corretgé
130 14 Saúl Ibarra Corretgé
h3. VideoConsumer
131 14 Saúl Ibarra Corretgé
132 14 Saúl Ibarra Corretgé
Consumes video data.
133 14 Saúl Ibarra Corretgé
134 14 Saúl Ibarra Corretgé
Public API:
135 14 Saúl Ibarra Corretgé
136 14 Saúl Ibarra Corretgé
* attach: tie this consumer to a producer, in order to render the video data generated by the producer
137 14 Saúl Ibarra Corretgé
* detach: untie this consumer from a producer
138 14 Saúl Ibarra Corretgé
* consumer_port: pointer to the pjmedia_port object
139 15 Saúl Ibarra Corretgé
140 15 Saúl Ibarra Corretgé
h3. Producer and consumer objects
141 15 Saúl Ibarra Corretgé
142 15 Saúl Ibarra Corretgé
* VideoDevice: Producer, acquires video from a user camera.
143 15 Saúl Ibarra Corretgé
* VideoWindow: Consumer, renders video in an SDL window. Extra methods: show/hide. Properties: native_handle, size.
144 15 Saúl Ibarra Corretgé
145 15 Saúl Ibarra Corretgé
* LocalVideoStream: Consumer, takes video from a VideoDevice and sends it to the remote party.
146 15 Saúl Ibarra Corretgé
* RemoteVideoStream: Producer, produces video sent by the remote party.
147 15 Saúl Ibarra Corretgé
148 15 Saúl Ibarra Corretgé
These are just theoretical objects, won't be implemented in the first go.
149 15 Saúl Ibarra Corretgé
150 17 Saúl Ibarra Corretgé
* VideoFileWriter: Consumer, saves incoming video data to a video file.
151 15 Saúl Ibarra Corretgé
* VideoFilePlayer: Producer, produces video data out of a video file.
152 15 Saúl Ibarra Corretgé
153 15 Saúl Ibarra Corretgé
* VideoMixer: Producer/Consumer, consumes video from multiple sources and produces aggregated video data.
154 16 Saúl Ibarra Corretgé
155 16 Saúl Ibarra Corretgé
NOTE: pjsip does have a AVI file player, which also seems to support audio.