WIPVideo

Version 23 (Saúl Ibarra Corretgé, 04/01/2014 11:40 am)

1 1 Saúl Ibarra Corretgé
h1. WIPVideo
2 1 Saúl Ibarra Corretgé
3 5 Saúl Ibarra Corretgé
Notes while video is a work in progress. Repository: http://devel.ag-projects.com/cgi-bin/darcsweb.cgi?r=saul/python-sipsimple-video;a=summary
4 1 Saúl Ibarra Corretgé
5 1 Saúl Ibarra Corretgé
h2. Dependencies
6 1 Saúl Ibarra Corretgé
7 1 Saúl Ibarra Corretgé
The following dependencies are required to build PJSIP with video support (including H264)
8 1 Saúl Ibarra Corretgé
9 1 Saúl Ibarra Corretgé
* SDL 2
10 1 Saúl Ibarra Corretgé
* ffmpeg (libavformat, libswscale, libavcodec, libavutil)
11 1 Saúl Ibarra Corretgé
* libx264
12 1 Saúl Ibarra Corretgé
13 1 Saúl Ibarra Corretgé
Versions I have tried:
14 1 Saúl Ibarra Corretgé
15 1 Saúl Ibarra Corretgé
* SDL (2.0.0-7655)
16 1 Saúl Ibarra Corretgé
* ffmpeg (2.0 release)
17 1 Saúl Ibarra Corretgé
* libx264 (snapshot-20130806-2245-stable)
18 1 Saúl Ibarra Corretgé
19 1 Saúl Ibarra Corretgé
h2. Patches
20 1 Saúl Ibarra Corretgé
21 11 Saúl Ibarra Corretgé
-If the above versions are used, PJSIP needs to be patched with the attached patch (avcodec.diff) or it won't compile. This does not occur when compiling it against the latest library versions on Debian unstable.- No longer needed.
22 3 Saúl Ibarra Corretgé
23 20 Saúl Ibarra Corretgé
h2. Installing dependencies (Debian / Ubuntu systems)
24 20 Saúl Ibarra Corretgé
25 20 Saúl Ibarra Corretgé
The situation here is a bit sad. Both Debian and Ubuntu ship with libav instead of FFmpeg, but libraries are called the same. PJSIP had to be patched in order to properly work with libav, and the patch as not yet been included upstream.
26 20 Saúl Ibarra Corretgé
27 20 Saúl Ibarra Corretgé
On Debian, when the Debian-Multimedia repositories are used (quite common) you get FFmpeg and not libav. Oh the joy!
28 20 Saúl Ibarra Corretgé
29 20 Saúl Ibarra Corretgé
Installing dependencies on Debian:
30 20 Saúl Ibarra Corretgé
31 20 Saúl Ibarra Corretgé
<pre>
32 20 Saúl Ibarra Corretgé
apt-get install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libsdl2-dev libx264-dev libavcodec-extra
33 20 Saúl Ibarra Corretgé
</pre>
34 20 Saúl Ibarra Corretgé
35 20 Saúl Ibarra Corretgé
If using the Debian-Multimedia repositories, do not install libavcodec-extra.
36 20 Saúl Ibarra Corretgé
37 20 Saúl Ibarra Corretgé
Installing dependencies on Ubuntu:
38 20 Saúl Ibarra Corretgé
39 20 Saúl Ibarra Corretgé
<pre>
40 20 Saúl Ibarra Corretgé
apt-get install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libsdl2-dev libx264-dev libavcodec-extra53
41 20 Saúl Ibarra Corretgé
</pre>
42 20 Saúl Ibarra Corretgé
43 21 Saúl Ibarra Corretgé
*Note on H.264 support*: In order to have H.264 support, FFmpeg (or libav) need to be compiled with support for it. The standard packages don't, hence the need for installing the libavcodec-extra packages.
44 20 Saúl Ibarra Corretgé
45 3 Saúl Ibarra Corretgé
h2. Compiling dependencies
46 3 Saúl Ibarra Corretgé
47 3 Saúl Ibarra Corretgé
All dependencies will be compiled to a directory in the user's HOME directory:
48 3 Saúl Ibarra Corretgé
49 3 Saúl Ibarra Corretgé
<pre>
50 3 Saúl Ibarra Corretgé
export MY_VIDEO_LIBS=$HOME/work/ag-projects/video/local
51 3 Saúl Ibarra Corretgé
</pre>
52 3 Saúl Ibarra Corretgé
53 3 Saúl Ibarra Corretgé
h3. libx264
54 3 Saúl Ibarra Corretgé
55 3 Saúl Ibarra Corretgé
<pre>
56 3 Saúl Ibarra Corretgé
./configure --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
57 3 Saúl Ibarra Corretgé
make
58 3 Saúl Ibarra Corretgé
make install
59 8 Saúl Ibarra Corretgé
60 8 Saúl Ibarra Corretgé
# If a 32bit build is wanted, then run this configure instead:
61 8 Saúl Ibarra Corretgé
./configure --host=i386-apple-darwin --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
62 3 Saúl Ibarra Corretgé
</pre>
63 3 Saúl Ibarra Corretgé
64 3 Saúl Ibarra Corretgé
h3. ffmpeg
65 3 Saúl Ibarra Corretgé
66 3 Saúl Ibarra Corretgé
<pre>
67 3 Saúl Ibarra Corretgé
# Some exports
68 3 Saúl Ibarra Corretgé
export PKG_CONFIG_PATH=$MY_VIDEO_LIBS/lib/pkgconfig
69 1 Saúl Ibarra Corretgé
70 7 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`"
71 3 Saúl Ibarra Corretgé
make
72 3 Saúl Ibarra Corretgé
make install
73 9 Saúl Ibarra Corretgé
74 9 Saúl Ibarra Corretgé
# If a 32bit build is wanted do:
75 9 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`" --cc="gcc -m32" --disable-asm
76 9 Saúl Ibarra Corretgé
# TODO: I WANT MY ASM
77 3 Saúl Ibarra Corretgé
</pre>
78 3 Saúl Ibarra Corretgé
79 3 Saúl Ibarra Corretgé
h3. SDL
80 3 Saúl Ibarra Corretgé
81 3 Saúl Ibarra Corretgé
<pre>
82 3 Saúl Ibarra Corretgé
./configure --disable-audio --prefix=$MY_VIDEO_LIBS
83 3 Saúl Ibarra Corretgé
make
84 3 Saúl Ibarra Corretgé
make install
85 10 Saúl Ibarra Corretgé
86 10 Saúl Ibarra Corretgé
# If a 32bit build is wanted:
87 10 Saúl Ibarra Corretgé
CFLAGS="-arch i386" CXXFLAGS="-arch i386" LDFLAGS="-arch i386" ./configure --disable-audio --prefix=$MY_VIDEO_LIBS
88 3 Saúl Ibarra Corretgé
</pre>
89 4 Saúl Ibarra Corretgé
90 4 Saúl Ibarra Corretgé
h2. Compiling PJSIP (pjsua, for testing)
91 4 Saúl Ibarra Corretgé
92 4 Saúl Ibarra Corretgé
<pre>
93 4 Saúl Ibarra Corretgé
svn co http://svn.pjsip.org/repos/pjproject/trunk pjsip
94 4 Saúl Ibarra Corretgé
cd pjsip
95 4 Saúl Ibarra Corretgé
echo "#define PJMEDIA_HAS_VIDEO 1" > pjlib/include/pj/config_site.h
96 4 Saúl Ibarra Corretgé
./configure --with-sdl=$MY_VIDEO_LIBS --with-ffmpeg=$MY_VIDEO_LIBS
97 22 Saúl Ibarra Corretgé
# if a 32bit build is wanted:
98 22 Saúl Ibarra Corretgé
# CFLAGS="-arch i386" CXXFLAGS="-arch i386" LDFLAGS="-arch i386" ./configure --with-sdl=$MY_VIDEO_LIBS --with-ffmpeg=$MY_VIDEO_LIBS
99 4 Saúl Ibarra Corretgé
make dep
100 4 Saúl Ibarra Corretgé
make
101 4 Saúl Ibarra Corretgé
# pjsua will be located in pjsip-apps/bin/
102 4 Saúl Ibarra Corretgé
</pre>
103 12 Saúl Ibarra Corretgé
104 12 Saúl Ibarra Corretgé
h2. Proposed API
105 12 Saúl Ibarra Corretgé
106 13 Saúl Ibarra Corretgé
API for video components is based on 2 different types of video capable entities:
107 13 Saúl Ibarra Corretgé
108 13 Saúl Ibarra Corretgé
* VideoProducer: a source for video data, for example a video camera or a remote video stream
109 13 Saúl Ibarra Corretgé
* VideoConsumer: a sink or destination for video data, for example a video rendering window
110 13 Saúl Ibarra Corretgé
111 13 Saúl Ibarra Corretgé
h3. Data flow
112 13 Saúl Ibarra Corretgé
113 13 Saúl Ibarra Corretgé
Data flow works in _pull_ fashion, that is, a producer doesn't start to produce data until there is a consumer which will consume it.
114 13 Saúl Ibarra Corretgé
115 13 Saúl Ibarra Corretgé
h3. VideoProducer
116 13 Saúl Ibarra Corretgé
117 13 Saúl Ibarra Corretgé
Produces video data. 
118 13 Saúl Ibarra Corretgé
119 13 Saúl Ibarra Corretgé
Internal API:
120 13 Saúl Ibarra Corretgé
121 13 Saúl Ibarra Corretgé
* _add_consumer: attach a consumer, called by the consumer
122 13 Saúl Ibarra Corretgé
* _remove_consumer: detach a consumer from a producer, called by the consumer
123 13 Saúl Ibarra Corretgé
124 18 Saúl Ibarra Corretgé
125 1 Saúl Ibarra Corretgé
Public API:
126 1 Saúl Ibarra Corretgé
127 18 Saúl Ibarra Corretgé
* start: start producing video as soon as a consumer is attached
128 18 Saúl Ibarra Corretgé
* stop: immediately stop producing data
129 13 Saúl Ibarra Corretgé
* close: remove all consumers and stop producing video data
130 14 Saúl Ibarra Corretgé
* producer_port: pointer to the pjmedia_port object
131 14 Saúl Ibarra Corretgé
132 14 Saúl Ibarra Corretgé
h3. VideoConsumer
133 14 Saúl Ibarra Corretgé
134 14 Saúl Ibarra Corretgé
Consumes video data.
135 14 Saúl Ibarra Corretgé
136 14 Saúl Ibarra Corretgé
Public API:
137 14 Saúl Ibarra Corretgé
138 14 Saúl Ibarra Corretgé
* attach: tie this consumer to a producer, in order to render the video data generated by the producer
139 14 Saúl Ibarra Corretgé
* detach: untie this consumer from a producer
140 14 Saúl Ibarra Corretgé
* consumer_port: pointer to the pjmedia_port object
141 15 Saúl Ibarra Corretgé
142 15 Saúl Ibarra Corretgé
h3. Producer and consumer objects
143 15 Saúl Ibarra Corretgé
144 15 Saúl Ibarra Corretgé
* VideoDevice: Producer, acquires video from a user camera.
145 15 Saúl Ibarra Corretgé
* VideoWindow: Consumer, renders video in an SDL window. Extra methods: show/hide. Properties: native_handle, size.
146 15 Saúl Ibarra Corretgé
147 15 Saúl Ibarra Corretgé
* LocalVideoStream: Consumer, takes video from a VideoDevice and sends it to the remote party.
148 15 Saúl Ibarra Corretgé
* RemoteVideoStream: Producer, produces video sent by the remote party.
149 15 Saúl Ibarra Corretgé
150 15 Saúl Ibarra Corretgé
These are just theoretical objects, won't be implemented in the first go.
151 15 Saúl Ibarra Corretgé
152 17 Saúl Ibarra Corretgé
* VideoFileWriter: Consumer, saves incoming video data to a video file.
153 15 Saúl Ibarra Corretgé
* VideoFilePlayer: Producer, produces video data out of a video file.
154 15 Saúl Ibarra Corretgé
155 15 Saúl Ibarra Corretgé
* VideoMixer: Producer/Consumer, consumes video from multiple sources and produces aggregated video data.
156 16 Saúl Ibarra Corretgé
157 16 Saúl Ibarra Corretgé
NOTE: pjsip does have a AVI file player, which also seems to support audio.
158 23 Saúl Ibarra Corretgé
159 23 Saúl Ibarra Corretgé
h2. H264
160 23 Saúl Ibarra Corretgé
161 23 Saúl Ibarra Corretgé
Information about H264 profiles:
162 23 Saúl Ibarra Corretgé
163 23 Saúl Ibarra Corretgé
* http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels
164 23 Saúl Ibarra Corretgé
* https://supportforums.cisco.com/blog/149561/video-telepresence-sip-h264-profile-level-id