SIP SIMPLE Client SDK

[[TOC(gui_progress, depth=3)]]

= Audio sessions =

 1. '''Mute''' microphone when pressing mute button

 1. Add  a '''record button''' left to Hold button. When pressed toggle recording the audio session

 1. Display RTP '''packet loss''' with red above the session buttons when > 0.5 %, sample every 10 seconds

 1. Display '''Ringing...''' when get ringing indication from remote party

 1. Closing many active sessions cause many tones to be played back. Throttle tones playback

 1. Call '''set_audio_devices()''' when changing the audio device or tail_length in global Preferences

 1. Calls made with bonjour account that do not resolve in DNS do not end, try dial abcd

 1. After starting many calls some remain in '''Initiating session''' state and no button can be pressed and remain in the drawer

 1. ~~Play '''ringtones'''~~ 

 1. ~~'''Cancel''' is not handled for incoming sessions, the pop-up remains active~~

  1. ~~With every new incoming session the window shrinks in size~~

  1. ~~After the second call is canceled the window does not auto close anymore and one previous canceled call remains on top~~

 1. ~~ Incoming sessions do not display '''session information''' and the control buttons are missing~~

 1. ~~ When '''rejecting a session''' it creates an entry in the drawer~~

 1. ~~'''Maximize drawer''' when it opens the first time, it seem to open only 80% now ~~

 1. ~~Playback DTMF tones~~

 1. ~~ A terminated session must '''disappear from the drawer''' automatically after 5 seconds ~~

 1. ~~ Remaining sessions must '''shuffle to the top''' when a session ends~~

 1. ~~ Toggle hold does not change hold '''button color''' ~~

 1. ~~ Update session audio information with '''Hold by remote''' when held by remote party ~~

 1. ~~ Only one audio session can be '''active at a time''', all other existing audio sessions must be put on hold~~

 1. ~~ Highlight the '''active session''' with a bold border, click on a session to select it as the active session~~

 1. ~~ When the active session is selected, what is typed on the keyboard must be transmitted as '''DTMF tones'''~~

 1. ~~ When a call is connected move keyboard focus to it ~~

 1. ~~ When a session has ended, display it for '''5 more seconds''' then hide it and shuffle the deck  ~~

 1. ~~ Add an item '''Show audio sessions''' to the '''Session menu''' to show the drawer, if it was closed  (activated the View -> Toggle Sessions Drawer) ~~

 1. ~~ '''Scroll bar''' appears as result of multiple sessions, but it does not disappear when the number of session decreases~~

 1. ~~ '''Hangup All''' is not implemented~~

 1. ~~Display icon with lock when sRTP is active~~

 1. ~~ Disable Hangup All button if no session is active~~

 1. ~~ Disable Conference button if less than two sessions active~~

 1. ~~Allow the session end button for incoming sessions, which have not yet been established  (waiting for the ACK message) ~~

= Chat sessions =

 1. When using a relay during stream initialization print system message: '''MSRP session reserved at relay host:port'''

 1. Use the '''same look and feel''' for the toolbar buttons as the main interface (greyish buttons) 

 1. If an audio session exists to or from the chat recipient, replace or append after the Audio button '''Record|Hold|Terminate''' buttons

 1. When click on the Audio button, create a new '''audio only session''' from scratch (as opposed to add_stream)

 1. Use same '''+ button''' from the main interface for add contact ''(not sure about this, having add an arbitrary contact in multiple places is redundant and can be confusing)''

 1. Display the number of '''un-read messages''' in non-active tabs within a red circle over the tab name

 1. To close a tab add a '''small x button''' to it

 1. When dragging a recipient tab outside of the window '''spawn a new chat window''' and move the session into it

 1. When start typing send a message with '''is-composing''' payload. This requires implementing the is-composing payload parser, see http://sipsimpleclient.com/ticket/40. If user has deleted the typed text before sending it update is-composing.     

 1. Display is-composing information received from the other end

 1. '''Auto-accept''' chat only sessions when caller is in the contacts list  (tricky because of aliases). Use the SIP URI user@domain from the From header of the INVITE to match the contact

 1. ~~Rename '''Close chat''' button to '''Close'''~~

 1. ~~Rename Call button text with '''Audio'''~~

 1. ~~If not connected, change Close button to '''Connect'''~~ better keep Connect and Close separate

 1. ~~When connecting, grey the action button and display '''Connecting''' until the session is in either connected or terminated state~~

 1. ~~ Add timestamp for system messages~~

 1. ~~When connected print system message: '''MSRP chat session established to MSRP_URI'''~~

 1. ~~When disconnected print system message: '''MSRP chat session terminated (reason)'''~~

 1. ~~When pressing the IM button in the main interface just '''open the chat window''' and display the last conversation from history, do not start a session automatically~~

 1. ~~Start session when user '''starts typing''' or by pressing the Connect button~~

 1. ~~Do not close the chat tab/window after the session ends~~

 1. ~~Read history directory from '''global.chat.history_directory''' setting~~

 1. ~~Remove confirmation window with: "There is an active chat session, would you like to terminate and close it?"~~ ''(left confirmation for more than 1 sessions)''

 1. ~~Do not play an '''audible tone when terminating''' a chat session, the stop tone is for audio session only~~

 1. ~~Display session information inline with a different border color~~

 1. ~~CPIM parser cannot pase Display names equal to one space~~

 1. ~~Add a '''+''' button (to add contact) in the toolbar~~

 1. ~~Display the full From and To headers~~

 1. ~~Use secondary ring tone on output_device  when incoming session with chat media only~~

 1. ~~When close the window or click on close button call session.end() if the session has no other stream or session.remove_stream() otherwise~~

 1. ~~When click on close, close the tab or the whole window if no tab left~~

 1. ~~'''Throttle playback''' of audible notifications to maximum one every 3 seconds ~~

 1. ~~ Display timestamp in HH:MM:ss format to the right side of the window, same lin as From/To header ~~ 

 1. ~~Bug: Cannot get rid of existing tabs, they keep growing~~

 1. ~~Bug: opening and closing chat windows bring up different number of former chat tabs back~~

 1. ~~Append chat messages to '''chat.history_directory/sip_account/YYYYMMDD-recipient.txt'''~~

 1. ~~Play an audible message when a message is received, based on '''silent''' and '''general.message_received_sound''' settings~~

 1. ~~Play an audible message when a message is sent, based on silent and '''general.message_sent_sound''' settings~~

 1. ~~Add '''File transfer''' button on toolbar~~

 1. ~~Add '''History''' button on toolbar~~

= Multi-party chat =

 1.  How to display participants in a multy-party  (drawer?)

 1. Starting a chat to a multiparty server must prompt the user for the Nickname to use for the session. Add a a new dialog window for such sessions:

  1. Room: use last used room

  1. Server: default to '''account.chat.server'''

  1. Nickname

= Main interface =

 1. Highlight mute button content with red when pressed

 1. ~~Use '''green + title button''' to collapse/expand the buddy list~~

 1. ~~Hide + (add contact) button when the interface is collapsed~~

 1. ~~Add a status bar on the bottom of the window ~~

 1. ~~Display the NAT type on the status bar~~

 1. ~~'''Bonjour account''' does not appear when deactivate/activate in preferences~~

 1. ~~An account marked  as active appears immediately in the account list. If registration for the account is active, display text with grey if not registered yet and black if registered~~ 

 1. ~~At least one account must be active, use the bonjour acocunt if no accounts are defined ~~

 1. ~~ The list of account is built based the active attribute of each account alone, registration status has no role~~

 1. ~~When selecting a different account, mark it as '''default''' using the settings api~~

 1. ~~Bonjour account does not show up in account list~~

 1. ~~'''Remove account''' from list when is disabled in preferences  ~~

 1. ~~On startup, '''select the default account''' in the account list~~

 1. ~~If red Close button is pressed, the main window disappears and is no option to bring it back~~

 1. ~~No dock icon available~~

= Log drawer =

 1. Lower the font size and use red font in case of errors

 1. Display more useful messages (e.g. DNS lookups, session status)

= Contacts =

 1. When add a contact without domain part, always append the '''curent domain'''

 1. Type a '''new group''' in add contact: (NameError: global name 'ContactsGroup' is not defined)

 1. '''Delete the contact''' when pressing delete

 1. Rename '''Add as a contact''' to Add Contact

 1. Hide '''Add contact''' if contact exists

 1. After dialing a found contact, '''display back''' the buddy list

 1. '''Cannot press''' start session buttons for input in search bar if no match found

 1. Show search results from system '''address book'''

= Preferences =

 1. Properly '''align''' horizontally the Advanced settings

 1. Audio device must also list 'Default system input ' and 'Default system output'

 1. ~~Make all text input fields equal in size~~

 1. '''bonjour account''' has only Display name as main property

 1. Hide the minus button for Bonjour, it cannot be deleted

 1. When password field is changed, re-Register the account 

 1. When Register flag is toggled for account, (de)Register the account accordingly

 1. Codec lists must be an ordered list with at least one active codec

 1. '''Stun server''' addresses cannot be set, they seem to inherit the results found in DNS  

 1. '''general.rtp.local_ip''': Can't set option local_ip illegal local IP address value: auto

 1. '''general.rtp.port_range''': (AttributeError: 'PortRangeOption' object has no attribute 'save')

 1. '''general.sip.transports''' cannot be saved

 1. '''general.audio.codec_list''' cannot be saved and displays duplicate codecs

 1. '''account.audio.codec_list''' cannot be saved and displays duplicate codecs

 1. '''account.msrp.relay''' cannot be set: global name 'MSRPRelayAddres' is not defined

 1. Display text to the '''left of the checkbox''' instead of the right

 1. ~~'''Delete account Enabled checkbox''', leave only the checkbox in the account list~~

 1. When '''enable an account''', select it in the main interface account list

 1. ~~ Display the '''bonjour account''' always at the end of the list ~~

 1. For '''account.ringtone''' advanced setting, the default setting must be inherited from '''general.ringtone.inbound'''

 1. '''general.message_received_sound''' does not fit in window

 1. While clicking on account I got this (cannot reproduce): (TypeError: 'NSAutoreleasePool' object is not iterable)

 1. Check if the wav file is in the right format before saving the settings related to ringtones

 1. Check if the TLS options set by the user are valid before saving them (by loading them outside the engine using gnutls library)

Display SIP registration state in '''account.advanced.registration''' section.

= Conferencing =

Only one conference is possible to make the interface easy to use.

Start conference as a mixer

 1. Set the conference bridge to mix audio between all parties

 1. Send a re-INVITE to take each call off hold and set the is-focus attribute of the Contact header

 1. Disable the hold buttons of the conferenced sessions

 1. Draw the active session selection rectangle around the sessions part of the conference

Stop  conference as a mixer

 1. Set the conference bridge to stop mixing audio

 1. Send a re-INVITE to disable is-focus attribute of the Contact header

 1. Enable the hold buttons

 1. Draw the selection rectangle around the selected session

New sessions as a mixer

 1. New sessions are added bellow the conference

 1. A session can be dragged in and out of the conference

 1. When switching from a conference to another single session, do not use hold

Conference participant

 1. SUBSCRIBE to the conference event package when receiving a re-INVITE with isfocus true

 1. Expand vertically the session frame to display the names of the participants received in subsequent NOTIFY that contain the list of the participants in the conference

= Audio history =

 1. Use the same session drawer to display previous sessions

 1. Keep same layout as active sessions with the following changes:

   1. Missed calls (incoming un-answered sessions must display the SIP address in red) 

   1. Duration line must display also the Start time: HH:MM:SS (YYYY-MM-DD HH:MM )

   1. Instead of audio session information display: Incoming|Outgoing

   1. Replace the current session buttons with a callback button

 1. Add a Session menu item to toggle the drawer content between active session and history sessions

 1. Show most recent sessions on top

= Engine =

 1. If '''engine is dead''', the message  '''no sip account is active''' is displayed but is unclear what the real reason is. An engine stop must be logged to general debug window. Not clear how to restart the engine, maybe we should restart the application until a good solution is found

 1. If the engine has stopped (because of some crash) the preferences do not show anymore:

{{{

Blink[26992:10b] <type 'exceptions.AttributeError'>: 'Engine' object has no attribute 'output_devices'

}}}

By setting the wrong TLS certificate files, the middleware does not start anymore:

{{{

[Session started at 2009-07-19 10:37:39 +0200.]

Traceback (most recent call last):

  File "/Users/adigeo/work/blink/build/Debug/Blink.app/Contents/Resources/BlinkAppDelegate.py", line 40,

   in applicationDidFinishLaunching_

    self.backend.init(options)

  File "/Users/adigeo/work/blink/build/Debug/Blink.app/Contents/Resources/SIPManager.py", line 108, in init

    self._app.start(config_backend=config_be)

  File "/Library/Python/2.5/site-packages/sipsimple/api.py", line 85, in start

    trace_sip=settings.logging.trace_sip,

  File "/Library/Python/2.5/site-packages/sipsimple/engine.py", line 75, in start

    self._ua = PJSIPUA(self._handle_event, 1.1.init_options)

  File "core.ua.pxi", line 66, in sipsimple.core.PJSIPUA.__init__ (sipsimple/core.c:23189)

  File "core.lib.pxi", line 106, in sipsimple.core.PJSIPEndpoint.__cinit__ (sipsimple/core.c:6231)

  File "core.lib.pxi", line 167, in sipsimple.core.PJSIPEndpoint._start_tls_transport (sipsimple/core.c:6884)

PJSIPError: Could not create TLS transport: Error loading/verifying SSL CA list file (PJSIP_TLS_ECACERT)

None

}}}

= Menu structure =

 1. File

 1. Edit

 1. View

  1. Audio sessions drawer 

  1. Previous audio sessions

  1. Previous chat sessions

 1. Audio

  1. Output device selection

  1. Input device selection

  1. Alert device selection

  1. Mute (mute mic input)

  1. Silent (mute output)

 1. Presence

  1. Activity

   1. Available

   1. Phone call

   1. Meeting

   1. Lunch

   1. Dinner

   1. Do not disturb (when enabled reject automatically all audio calls)

   1. Travel

   1. Offline

   1. Holiday

 1. Contacts

  1. Add contact

  1. Delete contact

  1. Edit contact

  1. Add group

  1. Delete group

  1. Edit group

 1. Sessions

  1. New audio session ...

  1. New chat session ...

  1. New multi party chat session ...

  1. Close all audio sessions

  1. Conference audio sessions

= Presence bar =

Make it as high as the contacts (two rows high).

 1. ~~ Show '''own photo''' to the left ~~

 1. Acquire photo from webcam and cache it

 1. Show '''Display name''' next to the photo on top row

 1. Show '''Presence activity note''' on second row, editable text

 1. Show '''Presence activity''' next to name on top row. Activities:

   1. Available

   1. Phone call

   1. Meeting

   1. Lunch

   1. Dinner

   1. Do not disturb

   1. Travel

   1. Offline

   1. Holiday

Add a Presence menu item before Session.

= Debug window =

 1. Display all traces in a '''debug window''' with tabs for each debug type

 1. Add a '''General messages''' window to the Debug menu, first item

= GUI settings =

Remember the following settings between restarts:

 1. '''Debug window''' and the selected tab

 1. Status of '''contact groups''' - colapsed/expanded and their order

= Middleware =

 1. On startup, check if the audio device from the preferences '''is available''' in the detected devices list. If not, set the device to the system default device and display a warning to the user to set the devices in the preferences

 1. Enable logging of notifications to file

 1. Check the validity of the '''TLS''' settings/files before starting the engine

 1. It seems that the '''stun servers''' are learned from the DNS and cannot be overwritten in settings, they show up automatically

 1. If I set the relay to abcd.com standard '''port 2855''' must be used, now is set to zero: relay = abcd.com:0;transport=tls

 1. We need a solution to accept a session without a 180- ringing. For instance we can have automatic answer based on various preferences.

 1. Move '''general.audio.codec_list''' to '''general.rtp.audio_codecs'''

 1. Move '''account.audio.codec_list''' to '''account.rtp.audio_codecs'''

 1. Move '''account.audio.srtp_encryption''' to '''account.rtp.srtp_encryption'''

 1. Delete '''general.audio.playback_dtmf''' setting

 1. Add '''general.audio.mute''' setting

 1. Delete '''account.audio''' section

 1. Add '''account.sip''' section

 1. Move '''account.outbound_proxy''' to '''account.sip.outbound_proxy'''

 1. Move '''account.presence.publish_interval''' to '''account.sip.publish_interval'''

 1. Move '''account.presence.subscribe_interval''' to '''account.sip.subscribe_interval'''

 1. Move '''account.registration.interval''' to '''account.sip.register_interval'''

 1. Delete '''account.ice.use_stun''', it is always true

 1. Rename '''account.ice''' to '''account.nat_traversal'''

 1. Rename '''account.ice.enabled''' to '''account.nat_traversal.enable_ice'''

 1. Delete global.chat.accept_types

 1. Delete global.chat.accept_wrapped_types

 1.  Add a new setting '''account.chat.server''' default = chatserver.domain

= General =

 1. Set SIP User Agent name to '''blink-<version>'''

 1. ~~Enable logging based on log settings~~

 1. ~~'''Set global.user_data_directory''' to ~/Library/Application\ Support/Blink/~~

 1. Scratchy noises when playing tone generator tones

 1. Crash when more than 32 tones in the tone generator queue

 1. Random crashes during functioning always end up in debugger at:

{{{

[Switching to process 59222 thread 0x9b73]

(gdb) bt

#0  0x937691c6 in mach_msg_trap ()

#1  0x937709bc in mach_msg ()

#2  0x9351104e in CFRunLoopRunSpecific ()

#3  0x93511c78 in CFRunLoopRunInMode ()

#4  0x9411128c in RunCurrentEventLoopInMode ()

#5  0x941110a5 in ReceiveNextEventCommon ()

#6  0x94110f19 in BlockUntilNextEventMatchingListInMode ()

#7  0x96863d0d in _DPSNextEvent ()

#8  0x968635c0 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()

#9  0x9685c5fb in -[NSApplication run] ()

#10 0x00233855 in ffi_call_SYSV ()

#11 0x00233d6c in ffi_call ()

#12 0x0024a825 in PyObjCFFI_Caller ()

#13 0x00261b63 in PyObjCSelector_GetMetadata ()

#14 0x00121505 in PyObject_Call ()

#15 0x0018f832 in PyEval_EvalFrameEx ()

#16 0x00191173 in PyEval_EvalCodeEx ()

#17 0x0018f79d in PyEval_EvalFrameEx ()

#18 0x00191173 in PyEval_EvalCodeEx ()

#19 0x00191260 in PyEval_EvalCode ()

#20 0x001a883c in PyErr_Display ()

#21 0x001a8e66 in PyRun_FileExFlags ()

#22 0x001aa7d2 in PyRun_SimpleFileExFlags ()

#23 0x00001d4e in main (argc=1, argv=0xbfffeef4) at /Users/adigeo/work/blink/main.m:33

kill

}}}

= IP connectivity checks =

If the Internet connectivity goes up and down the client must adjust to this

by taling appropiate actions. It is not enough to detect only what kind of

NAT we are behind but also what kind of IP connectivity we have.

== No IP network ==

Posible checks:

 * No real IP

 * DNS is down

 * No default route

== LAN without Internet ==

(e.g. wifi hotspot before login)

Posible checks:

 * Ethernet is up

 * DNS is up

== Internet access ==

Posible checks:

 * Ethernet is up

 * DNS is up

 * STUN is up

 * NAT type