WIPVideo

Version 18 (Saúl Ibarra Corretgé, 09/10/2013 04:01 pm)

1 1 Saúl Ibarra Corretgé
h1. WIPVideo
2 1 Saúl Ibarra Corretgé
3 5 Saúl Ibarra Corretgé
Notes while video is a work in progress. Repository: http://devel.ag-projects.com/cgi-bin/darcsweb.cgi?r=saul/python-sipsimple-video;a=summary
4 1 Saúl Ibarra Corretgé
5 1 Saúl Ibarra Corretgé
h2. Dependencies
6 1 Saúl Ibarra Corretgé
7 1 Saúl Ibarra Corretgé
The following dependencies are required to build PJSIP with video support (including H264)
8 1 Saúl Ibarra Corretgé
9 1 Saúl Ibarra Corretgé
* SDL 2
10 1 Saúl Ibarra Corretgé
* ffmpeg (libavformat, libswscale, libavcodec, libavutil)
11 1 Saúl Ibarra Corretgé
* libx264
12 1 Saúl Ibarra Corretgé
13 1 Saúl Ibarra Corretgé
Versions I have tried:
14 1 Saúl Ibarra Corretgé
15 1 Saúl Ibarra Corretgé
* SDL (2.0.0-7655)
16 1 Saúl Ibarra Corretgé
* ffmpeg (2.0 release)
17 1 Saúl Ibarra Corretgé
* libx264 (snapshot-20130806-2245-stable)
18 1 Saúl Ibarra Corretgé
19 1 Saúl Ibarra Corretgé
h2. Patches
20 1 Saúl Ibarra Corretgé
21 11 Saúl Ibarra Corretgé
-If the above versions are used, PJSIP needs to be patched with the attached patch (avcodec.diff) or it won't compile. This does not occur when compiling it against the latest library versions on Debian unstable.- No longer needed.
22 3 Saúl Ibarra Corretgé
23 3 Saúl Ibarra Corretgé
h2. Compiling dependencies
24 3 Saúl Ibarra Corretgé
25 3 Saúl Ibarra Corretgé
All dependencies will be compiled to a directory in the user's HOME directory:
26 3 Saúl Ibarra Corretgé
27 3 Saúl Ibarra Corretgé
<pre>
28 3 Saúl Ibarra Corretgé
export MY_VIDEO_LIBS=$HOME/work/ag-projects/video/local
29 3 Saúl Ibarra Corretgé
</pre>
30 3 Saúl Ibarra Corretgé
31 3 Saúl Ibarra Corretgé
h3. libx264
32 3 Saúl Ibarra Corretgé
33 3 Saúl Ibarra Corretgé
<pre>
34 3 Saúl Ibarra Corretgé
./configure --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
35 3 Saúl Ibarra Corretgé
make
36 3 Saúl Ibarra Corretgé
make install
37 8 Saúl Ibarra Corretgé
38 8 Saúl Ibarra Corretgé
# If a 32bit build is wanted, then run this configure instead:
39 8 Saúl Ibarra Corretgé
./configure --host=i386-apple-darwin --enable-shared --disable-avs --disable-swscale --disable-lavf --disable-ffms --disable-gpac --prefix=$MY_VIDEO_LIBS
40 3 Saúl Ibarra Corretgé
</pre>
41 3 Saúl Ibarra Corretgé
42 3 Saúl Ibarra Corretgé
h3. ffmpeg
43 3 Saúl Ibarra Corretgé
44 3 Saúl Ibarra Corretgé
<pre>
45 3 Saúl Ibarra Corretgé
# Some exports
46 3 Saúl Ibarra Corretgé
export PKG_CONFIG_PATH=$MY_VIDEO_LIBS/lib/pkgconfig
47 1 Saúl Ibarra Corretgé
48 7 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`"
49 3 Saúl Ibarra Corretgé
make
50 3 Saúl Ibarra Corretgé
make install
51 9 Saúl Ibarra Corretgé
52 9 Saúl Ibarra Corretgé
# If a 32bit build is wanted do:
53 9 Saúl Ibarra Corretgé
./configure --enable-shared --disable-static --enable-memalign-hack --enable-gpl --enable-libx264 --prefix=$MY_VIDEO_LIBS --extra-cflags="`pkg-config --cflags x264`" --extra-ldflags="`pkg-config --libs x264`" --cc="gcc -m32" --disable-asm
54 9 Saúl Ibarra Corretgé
# TODO: I WANT MY ASM
55 3 Saúl Ibarra Corretgé
</pre>
56 3 Saúl Ibarra Corretgé
57 3 Saúl Ibarra Corretgé
h3. SDL
58 3 Saúl Ibarra Corretgé
59 3 Saúl Ibarra Corretgé
<pre>
60 3 Saúl Ibarra Corretgé
./configure --disable-audio --prefix=$MY_VIDEO_LIBS
61 3 Saúl Ibarra Corretgé
make
62 3 Saúl Ibarra Corretgé
make install
63 10 Saúl Ibarra Corretgé
64 10 Saúl Ibarra Corretgé
# If a 32bit build is wanted:
65 10 Saúl Ibarra Corretgé
CFLAGS="-arch i386" CXXFLAGS="-arch i386" LDFLAGS="-arch i386" ./configure --disable-audio --prefix=$MY_VIDEO_LIBS
66 3 Saúl Ibarra Corretgé
</pre>
67 4 Saúl Ibarra Corretgé
68 4 Saúl Ibarra Corretgé
h2. Compiling PJSIP (pjsua, for testing)
69 4 Saúl Ibarra Corretgé
70 4 Saúl Ibarra Corretgé
<pre>
71 4 Saúl Ibarra Corretgé
svn co http://svn.pjsip.org/repos/pjproject/trunk pjsip
72 4 Saúl Ibarra Corretgé
cd pjsip
73 4 Saúl Ibarra Corretgé
echo "#define PJMEDIA_HAS_VIDEO 1" > pjlib/include/pj/config_site.h
74 4 Saúl Ibarra Corretgé
./configure --with-sdl=$MY_VIDEO_LIBS --with-ffmpeg=$MY_VIDEO_LIBS
75 4 Saúl Ibarra Corretgé
make dep
76 4 Saúl Ibarra Corretgé
make
77 4 Saúl Ibarra Corretgé
# pjsua will be located in pjsip-apps/bin/
78 4 Saúl Ibarra Corretgé
</pre>
79 12 Saúl Ibarra Corretgé
80 12 Saúl Ibarra Corretgé
h2. Proposed API
81 12 Saúl Ibarra Corretgé
82 13 Saúl Ibarra Corretgé
API for video components is based on 2 different types of video capable entities:
83 13 Saúl Ibarra Corretgé
84 13 Saúl Ibarra Corretgé
* VideoProducer: a source for video data, for example a video camera or a remote video stream
85 13 Saúl Ibarra Corretgé
* VideoConsumer: a sink or destination for video data, for example a video rendering window
86 13 Saúl Ibarra Corretgé
87 13 Saúl Ibarra Corretgé
h3. Data flow
88 13 Saúl Ibarra Corretgé
89 13 Saúl Ibarra Corretgé
Data flow works in _pull_ fashion, that is, a producer doesn't start to produce data until there is a consumer which will consume it.
90 13 Saúl Ibarra Corretgé
91 13 Saúl Ibarra Corretgé
h3. VideoProducer
92 13 Saúl Ibarra Corretgé
93 13 Saúl Ibarra Corretgé
Produces video data. 
94 13 Saúl Ibarra Corretgé
95 13 Saúl Ibarra Corretgé
Internal API:
96 13 Saúl Ibarra Corretgé
97 13 Saúl Ibarra Corretgé
* _add_consumer: attach a consumer, called by the consumer
98 13 Saúl Ibarra Corretgé
* _remove_consumer: detach a consumer from a producer, called by the consumer
99 13 Saúl Ibarra Corretgé
100 18 Saúl Ibarra Corretgé
101 1 Saúl Ibarra Corretgé
Public API:
102 1 Saúl Ibarra Corretgé
103 18 Saúl Ibarra Corretgé
* start: start producing video as soon as a consumer is attached
104 18 Saúl Ibarra Corretgé
* stop: immediately stop producing data
105 13 Saúl Ibarra Corretgé
* close: remove all consumers and stop producing video data
106 14 Saúl Ibarra Corretgé
* producer_port: pointer to the pjmedia_port object
107 14 Saúl Ibarra Corretgé
108 14 Saúl Ibarra Corretgé
h3. VideoConsumer
109 14 Saúl Ibarra Corretgé
110 14 Saúl Ibarra Corretgé
Consumes video data.
111 14 Saúl Ibarra Corretgé
112 14 Saúl Ibarra Corretgé
Internal API:
113 14 Saúl Ibarra Corretgé
114 14 Saúl Ibarra Corretgé
* _start: start consuming video data
115 14 Saúl Ibarra Corretgé
* _stop: stop consuming video data
116 14 Saúl Ibarra Corretgé
117 14 Saúl Ibarra Corretgé
Public API:
118 14 Saúl Ibarra Corretgé
119 14 Saúl Ibarra Corretgé
* attach: tie this consumer to a producer, in order to render the video data generated by the producer
120 14 Saúl Ibarra Corretgé
* detach: untie this consumer from a producer
121 14 Saúl Ibarra Corretgé
* consumer_port: pointer to the pjmedia_port object
122 15 Saúl Ibarra Corretgé
123 15 Saúl Ibarra Corretgé
h3. Producer and consumer objects
124 15 Saúl Ibarra Corretgé
125 15 Saúl Ibarra Corretgé
* VideoDevice: Producer, acquires video from a user camera.
126 15 Saúl Ibarra Corretgé
* VideoWindow: Consumer, renders video in an SDL window. Extra methods: show/hide. Properties: native_handle, size.
127 15 Saúl Ibarra Corretgé
128 15 Saúl Ibarra Corretgé
* LocalVideoStream: Consumer, takes video from a VideoDevice and sends it to the remote party.
129 15 Saúl Ibarra Corretgé
* RemoteVideoStream: Producer, produces video sent by the remote party.
130 15 Saúl Ibarra Corretgé
131 15 Saúl Ibarra Corretgé
These are just theoretical objects, won't be implemented in the first go.
132 15 Saúl Ibarra Corretgé
133 17 Saúl Ibarra Corretgé
* VideoFileWriter: Consumer, saves incoming video data to a video file.
134 15 Saúl Ibarra Corretgé
* VideoFilePlayer: Producer, produces video data out of a video file.
135 15 Saúl Ibarra Corretgé
136 15 Saúl Ibarra Corretgé
* VideoMixer: Producer/Consumer, consumes video from multiple sources and produces aggregated video data.
137 16 Saúl Ibarra Corretgé
138 16 Saúl Ibarra Corretgé
NOTE: pjsip does have a AVI file player, which also seems to support audio.