DesignAEC
Version 17 (Adrian Georgescu, 11/16/2009 06:23 pm) → Version 18/27 (Adrian Georgescu, 11/16/2009 06:26 pm)
= Acoustic Echo cancellation =
[[TOC(Design*, depth=1)]]
SIP SIMPLE client is a Python software library that allows for easy development of Internet communications end-points based on SIP and related protocols for voice, rich presence, session based instant messaging (IM), file transfers and desktop sharing. Other media types can be easily added by using an extensible high-level API. SIP SIMPLE client uses [http://www.pjsip.org/pjmedia/docs/html/index.htm PJSIP media library] for audio processing (sound card abstraction, audio codec and acoustic echo cancellation).
For more information see http://sipsimpleclient.com. http://sipsimpleclient.com
The project is maintained by AG Projects that can provide help to the developer to integrate its AEC implementation.
== Background ==
The present implementation of the Acoustic Echo Canceller echo canceller from PJSIP media library is not performing to the desired level of quality. Practically, the speakerphone function of SIP SIMPLE client is not reliable and not performing satisfactory. The result is that the software phone cannot be used without a head-set, which make people use proprietary applications like Skype or Google Talk. head-set.
[[Image(http://www.pjsip.org/images/media-flow.jpg)]]
== Project goal ==
Replace the existing AEC from Pjmedia library (what is displayed as echo.h in the above diagram whihc employes the AEC of the speex project) with an alternative to be developed solution that provides a high quality speakerphone user experience. The acoustic echo cancelation should perform comparable with any other proven VoIP commercial solution like Skype or iChat making the need of a headset un-necessary.
The developed software will be released under an Open Source open source licence and distributed with SIP SIMPLE client library.
The deliverable is C written program that can be applied to the pjsip cvs trunk 1.0 with the '''patch''' command.
== Resources ==
General
* Digital Signal Processing http://www.dsptutor.freeuk.com/
* Graduation thesis about AEC http://innovexpo.itee.uq.edu.au/2003/exhibits/s365914/
* DSP concepts: http://www.bores.com/courses/intro/basics/index.htm
== PJMEDIA ==
This is the actual audio PJSIP media library used by SIP SIMPLE client. It is part of PJSIP, a complete framework for building SIP clients, having an open source license.
* PJSIP media library http://pjsip.org, used by this project, AEC is implemented based on the Speex library, the code in pjsip/pjmedia/src/pjmedia/echo_speex.c
* Media flow - http://trac.pjsip.org/repos/wiki/media-flow
* PJSIP AEC API http://www.pjsip.org/pjmedia/docs/html/group__PJMEDIA__Echo__Cancel.htm
=== Speex AEC ===
This is the standard used AEC algorithm used by PJMEDIA in PJSIP that needs to be replaced by a better solution. The actual code is maintained by http://speex.org project.
* Speex project http://speex.org, the echo canceller implements AEC based on the MDF algorithm, see speex/libspeex/mdf.c. A test client is available in speex/speexclient, this has been used for developing pjmedia equivalent
* http://speex.org/docs/api/speex-api-reference/group__SpeexEchoState.html
=== Andree Adrian AEC ===
Research for other implementations revealed a well documented algorithm together with source code as C++ implementation.
* Draft AEC http://www.andreadrian.de/echo_cancel/draft-aec-03.txt
The white-paper gave enough insight and trust into its author understanding and capability of implementation for considering its blueprint as an alternative. Unfortunately, the author's claims about its high quality could not be tested, due to the fact that his C++ code dates back from 2004 and the application that used it could not be compiled on todays newer systems.
* Kphone AEC implemented using above AEC http://www.andreadrian.de/echo_cancel/ (not tested)
* http://www.andreadrian.de/intercom/ implements AEC based on NLMS algorithm (not tested yet)
Finally, this AEC has been implemented by a student in plain C and integrated to the SIP SIMPLE project. It does not work properly, audio artifacts being present and further debugging requires DSP knowledge which the developer did not have.
The actual implementation can be used as working example for how a third party AEC algorithm can be integrated with the SIP SIMPLE project. See attached file for more information.
== Next steps ==
At this stage the problem has not been solved due to the lack of knowledge of the developers in this specialized DSP area.
There are two choices for going further:
1. Implement an AEC from scratch and integrated with PJSIP based on its API
2. Debug and fix Andree Adrian AEC, an analysis of the code can reveal if the concept is valid but poorly implemented
[[TOC(Design*, depth=1)]]
SIP SIMPLE client is a Python software library that allows for easy development of Internet communications end-points based on SIP and related protocols for voice, rich presence, session based instant messaging (IM), file transfers and desktop sharing. Other media types can be easily added by using an extensible high-level API. SIP SIMPLE client uses [http://www.pjsip.org/pjmedia/docs/html/index.htm PJSIP media library] for audio processing (sound card abstraction, audio codec and acoustic echo cancellation).
For more information see http://sipsimpleclient.com. http://sipsimpleclient.com
The project is maintained by AG Projects that can provide help to the developer to integrate its AEC implementation.
== Background ==
The present implementation of the Acoustic Echo Canceller echo canceller from PJSIP media library is not performing to the desired level of quality. Practically, the speakerphone function of SIP SIMPLE client is not reliable and not performing satisfactory. The result is that the software phone cannot be used without a head-set, which make people use proprietary applications like Skype or Google Talk. head-set.
[[Image(http://www.pjsip.org/images/media-flow.jpg)]]
== Project goal ==
Replace the existing AEC from Pjmedia library (what is displayed as echo.h in the above diagram whihc employes the AEC of the speex project) with an alternative to be developed solution that provides a high quality speakerphone user experience. The acoustic echo cancelation should perform comparable with any other proven VoIP commercial solution like Skype or iChat making the need of a headset un-necessary.
The developed software will be released under an Open Source open source licence and distributed with SIP SIMPLE client library.
The deliverable is C written program that can be applied to the pjsip cvs trunk 1.0 with the '''patch''' command.
== Resources ==
General
* Digital Signal Processing http://www.dsptutor.freeuk.com/
* Graduation thesis about AEC http://innovexpo.itee.uq.edu.au/2003/exhibits/s365914/
* DSP concepts: http://www.bores.com/courses/intro/basics/index.htm
== PJMEDIA ==
This is the actual audio PJSIP media library used by SIP SIMPLE client. It is part of PJSIP, a complete framework for building SIP clients, having an open source license.
* PJSIP media library http://pjsip.org, used by this project, AEC is implemented based on the Speex library, the code in pjsip/pjmedia/src/pjmedia/echo_speex.c
* Media flow - http://trac.pjsip.org/repos/wiki/media-flow
* PJSIP AEC API http://www.pjsip.org/pjmedia/docs/html/group__PJMEDIA__Echo__Cancel.htm
=== Speex AEC ===
This is the standard used AEC algorithm used by PJMEDIA in PJSIP that needs to be replaced by a better solution. The actual code is maintained by http://speex.org project.
* Speex project http://speex.org, the echo canceller implements AEC based on the MDF algorithm, see speex/libspeex/mdf.c. A test client is available in speex/speexclient, this has been used for developing pjmedia equivalent
* http://speex.org/docs/api/speex-api-reference/group__SpeexEchoState.html
=== Andree Adrian AEC ===
Research for other implementations revealed a well documented algorithm together with source code as C++ implementation.
* Draft AEC http://www.andreadrian.de/echo_cancel/draft-aec-03.txt
The white-paper gave enough insight and trust into its author understanding and capability of implementation for considering its blueprint as an alternative. Unfortunately, the author's claims about its high quality could not be tested, due to the fact that his C++ code dates back from 2004 and the application that used it could not be compiled on todays newer systems.
* Kphone AEC implemented using above AEC http://www.andreadrian.de/echo_cancel/ (not tested)
* http://www.andreadrian.de/intercom/ implements AEC based on NLMS algorithm (not tested yet)
Finally, this AEC has been implemented by a student in plain C and integrated to the SIP SIMPLE project. It does not work properly, audio artifacts being present and further debugging requires DSP knowledge which the developer did not have.
The actual implementation can be used as working example for how a third party AEC algorithm can be integrated with the SIP SIMPLE project. See attached file for more information.
== Next steps ==
At this stage the problem has not been solved due to the lack of knowledge of the developers in this specialized DSP area.
There are two choices for going further:
1. Implement an AEC from scratch and integrated with PJSIP based on its API
2. Debug and fix Andree Adrian AEC, an analysis of the code can reveal if the concept is valid but poorly implemented