Low-latency audio on Windows Phone 8 (hint: forget it for now...)

Question

My business specializes in audio and music apps for the Windows ecosystem. For this new project that I’m considering (a virtual instrument of sorts), I need to achieve the lowest possible audio latency from capture to render.
A measured latency below 3 ms would be ideal, 20 ms would be ok, and anything above 30 ms would be a deal breaker. And just to be precise about my definition of latency, I’m strictly talking about “mic-to-speaker” latency (includes the processing in-between).
I’m not talking about “glass-to-speaker” latency, although this topic is also of interest to me.
I figured that I should be using WASAPI, as this is apparently the API that sits at the bottommost of the audio stack in WinRT / WinPRT. So I spent a couple of days re-familiarizing with C++ (happy to meet an old friend)
and I coded a prototype capturing, processing and rendering audio using WASAPI having in mind the goal of achieving the lowest possible latency. After trying and comparing various methods, workflows, threading mechanisms, buffer values and mix formats, the
best I could achieve using WASAPI on Windows Phone was ~140 ms of latency. This is using 3 ms buffers. And although the audio is glitch-free, I could have been using 30 ms buffers and it wouldn’t have made the slightest difference! The latency would still
be around 140 ms. As I found out, the API/driver is extremely defensive in protecting against audio starvation, so it adds quite a bit of buffering at both ends. This is very unfortunate, because it basically disqualifies real-time audio/musical
applications.
I’d love to be able to provide quality audio/musical apps for the platform (both Windows Phone and Windows 8), but right now, this latency issue is kind of a deal breaker.
I've been pointing out the importance of low latency audio to Microsoft for quite a while, and I know I'm not the only one, and I know a lot of people at Microsoft realize this is important. But in its execution,
it seems Microsoft constantly fails to deliver a truly low-latency audio stack. In the pre-XP days, I've had talks with the sysaudio devs about this, and I was told, "yeah, we're working on a new architecture that will come out after XP and
it will solve the latency problem for all audio and musical applications." Fast forward to mid-2010 (pre-Windows Phone 7), and I was still there pointing out the horrible latency figures one would get from the APIs that were about to ship. And
now that WASAPI is available on WP8 (our best hope yet for low-latency audio), I discover the overly defensive and buffer-happy architecture of WASAPI (even though one of its promises was precisely low-latency audio).
So, the question is…
Is Microsoft aware of this issue? If so, is Microsoft giving up and simply conceding the pro-audio territory to iOS? If not, I’d be glad to discuss this issue with an engineer at Microsoft. I’m serious about bringing audio apps
to the platform. I just need some assurance that Microsoft is taking action on its end, so that I can sync my development with the next product cycle.
Thanks in advance for any help, advice, or insights!
/Antoine

Answer 1

Any update on this issue? Is there a roadmap for a fix?
I ported a realtime sound analyzing application (requiring both audio input and output) to WinRT and was quite surprised at the high latency from WASAPI. I then searched around and posted some questions regarding this on various MS forums and got to the
following conclusions based on feedback from people including MVPs and MS employees:
A lot of those people are under the impression that WASAPI qualifies as a low-latency API.
Apparently WinRT apps are not supposed to have high CPU usage, and this is purposefully baked into the framework (minimal thread priority control, async/await everywhere which results in thread priority inversion, and other issues).
Why would anyone ever need high-priority threads? They must use a lot of CPU, right?
A lot of people think low-latency audio means high CPU usage. You can see where this is going when you look at the previous point.
Async/Await is being forced down to all levels, even though it should only be used at the UI level. What some people are actually now calling "old-school" multithreading is now being pushed out (lock, etc). Async/Await has horrible overhead and
results in thread priority inversion amongst other issues. For example, witness that there is no StorageFile.Read, just StorageFile.ReadAsync. Do some IO benchmarks with some of these async methods and you will see some horrible performance compared to desktop
file IO.
To get an understanding of what low latency audio means and why it is important, see this video. It compares the latency of Android to iOS using the exact same music app. Ever wondered
why there are no quality music apps for Android? Well now you know. And then realize that WinRT has
twice as much latency as Android.
And if anyone thinks this is a niche use case, consider that Apple created adds showcasing "musicians" playing their iPad "instruments". A use case that is essentially unavailable for WinRT apps. Why would Apple create
adds for a niche use case?
This should have been one of the high priority issues solved from the start in WinRT. MS solved this issue in Windows (desktop) long ago, with the ability to get insanely low latency there (0.3 ms in some stress tests,
see here), even beating out OSX. It is as if there is a new generation of architects at MS that know nothing about this previous work and are doomed to make the same mistakes over again. I really don't
understand why these pre-existing APIs can't be exposed in WinRT. No need to re-invent the wheel. But I guess it just isn't important enough.
I find this situation really sad since otherwise it could be a great and powerful platform.