For a long time before joining Google, I did many multi-platform projects including iOS/Android. One of those required to estimate ambient Loudness Level. For iOS devices this was somewhat straightforward, mostly because of the consistency between devices, but it was a bigger challenge for Android based devices: as a developer you are working with many (many!) manufacturers and many models within those manufacturers.
At the time I joined Google, among the projects I embarked, was one related to automatic and semi-automatic assessment of audio quality (related to my CTS/CDD changes project), and revisited the challenge of using the built-in microphones to somewhat accurately measure loudness level.
I took this one step further and focused on solving a bigger question: how could built-in microphones in Android be used as high quality sensors?
My proposed solution had two important modules: First was to define a new Android microphone recording source which will not have any DSP or processing that could add non-linearities to the audio being recorded, and second, a way to report to the user what the expected frequency response and characteristics of these microphones would be.
The first part became the UNPROCESSED microphone source and feature in Android devices.
https://developer.android.com/reference/android/media/MediaRecorder.AudioSource#UNPROCESSED
UNPROCESSED recording source instructs the device to give the app the RAW audio from the microphones, with the highest available bit-depth (e.g float point) and in all the channels available, without any DSP or other processing in the way. This not only gives raw audio data, but also minimizes any algorithmic/pipeline latency.
I led the creation of this new audio source, did the code and lobbied to include the requirements in the Android’s CDD (Compatibility Definition Document).
https://source.android.com/docs/compatibility/11/android-11-cdd#5_11_capture_for_unprocessed
AOSP contributions: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19
Then to tackle the second part. For a developer to make full use of the RAW recording data, it is important to have a mechanism to also know what is the expected response/performance of the microphones. For this, I led the research and design of an api for apps to be able to query the system for the Microphone Info. With this api it is possible to learn about the theoretical frequency response of each microphone, sensitivity, orientation, even geometric position between microphones, polar pattern, etc. I followed a similar pattern by involving some internal/external stakeholders (e.g. audio developers) and suggested an api and information structure that will make it possible for a developer to compensate for the microphone info and properly process/analyze the audio signal.
https://developer.android.com/reference/android/media/MicrophoneInfo
My direct code contributions to AOSP were less in this project.
AOSP contributions: 1, 2, 3
Thanks for reading!
-Ricardo (rago)