Silence isn’t golden – VOIP and Unity3d

Once again I am back on the trail of a good voice solution to use in Unity3d to use in multiple users situations. Having done a good few projects using a variety of solutions, mainly with an adjusted version of the now defunct Uspeak (as in removed from the assets store but replaced with Daiken Forge voice) combined with self hosted Photon server I think it must be time to find a more scalable solution. Very often the pressures of voice lead to a swamping of any server that is also dealing with position and data model synchronisation across the network. This clearly makes sense as there are only a few bytes of message for something moving a long distance, yet an audio stream is a constant collection of data flowing in and out of a server.

Whilst there is huge focus on VR for the eyes, being able to communicate needs more than one channel but voice is one of those.
A voice connection can be 100Kbps per person so without a different tuning or set of priorities the regular multiplayer data messages, even if they are reliable, will get lost on the noise of an open mic.
My projects up to now have ended up with a push to talk or even my scarily complicated dial, ring answer/engaged phone simulator in order to cut down on the amount of voice. Raising a hand to speak works in many formal situations but not in social collaboration.
So whilst we can squeeze voice packets onto the same socket server as the rest of the “stuff” it feels like it should be a separate service and thread.
Oddly there is not a solution just sitting there for Unity3d? It seems that many of the existing voice clients for games and related services just don’t play well in the Unity3d/c# environment.
Unfortunately this seems to be down to the generic nature of Unity3d (which is a major strength) compared with the specifics of PC architecture and also in how to deal with shared memory and pipes.
There are a lot of voice servers out there for games. Mumble/Murmur is a major one. it is open source and appears to bolt on top of many games even supporting a 3d positional audio.
Many games are of course closed code bases. So Mumble has to piggy back on top of the game or have a little bit of help form the game developers to drop some hooks in. The main problem seems to be the use of shared memory versus a more closed boundary of applications. Obviously a shared memory space to communicate can bring problems and hacks. Strangely it was shared memory and various uses of it that let us do the early client server work back in the 90’s scraping from a terminal screen to make a legacy CICs application work with a fancy GUI!. Initially I figured that there would be a c# client for mumble that any of us could drop into Unity3d and away we go. However there are a few unfinished or implementations being attempted. Connecting to mumble and dealing with text is fine but voice and the various implementations of encoding seems to be a sticking point. Mumble also has a web interface implemented through something called ICE which is another Remote Procedure Call set of interfaces. It seems to be again focussed on building with C++ or PHP. Whilst ICE for .Net exists it does not work on all platforms. I am still looking at this though as surely if a web interface works we can get it to work on Unity3d. Of course the open source world is wide and diverse so there is always a chance I am missing something?
If a Mumble, or similar client can exists in a Unity3d context then we have our solution. It is nice to have the entire interface you need inside Unity3d not side loaded or overlayed. However it may be that a solution is to have implement the web interfaces next door to a unity interface and use plugin to webpage communication as the control. This is fraught with errors, browsers, DOM’s, script fails etc. Just a nice drop in mumble client hooked u to the shiny new UI interfaces.
I looked at the others in the area, Teamspeak is one of the biggest but incurs licensing charges so is not very open source minded. Ventrillo and Vivox seem to have fallen away.
Then of course there are the actual phone voip packages and standards. Maybe they offer the answer. I used Freeswitch on Opensim to provide a voice channel. Maybe I need to look at those again.
Or another solution could be trying to persuade the socket server applications like Photon than they need to talk to more than one server. At the moment the assumption is one server connection (though the load balancing may occur further on the server itself). If the client knew to talk to a different configure machine and network that was happy to deal with the firehose of audio but the normal movements talked to the server configure to deal with normal movements then we might have a solution.
Either way, its not straight forward and hasn’t been totally solved yet, or if it has everyone is keep quiet (Yes thats ironic when talking about VOIP!)
Let me know if you do have some great opensource or open sourceable solution.