Multi-modal Blind Source Separation with Microphones and Blinkies

Robin Scheibler, Nobutaka Ono
4/4/2019
cs.SDeess.AS

Official code: onolab-tmu/blinky-iva (12 stars, MIT, last commit 2019-05-23).

Abstract

We propose a blind source separation algorithm that jointly exploits measurements by a conventional microphone array and an ad hoc array of low-rate sound power sensors called blinkies. While providing less information than microphones, blinkies circumvent some difficulties of microphone arrays in terms of manufacturing, synchronization, and deployment. The algorithm is derived from a joint probabilistic model of the microphone and sound power measurements. We assume the separated sources to follow a time-varying spherical Gaussian distribution, and the non-negative power measurement space-time matrix to have a low-rank structure. We show that alternating updates similar to those of independent vector analysis and Itakura-Saito non-negative matrix factorization decrease the negative log-likelihood of the joint distribution. The proposed algorithm is validated via numerical experiments. Its median separation performance is found to be up to 8 dB more than that of independent vector analysis, with significantly reduced variability.

View on arXivView PDF

Code Implementations(1)

onolab-tmu/blinky-ivaOfficial100%
MIT

Multimodal formulation of IVA using conventional microphones and power sensing blinkies.

126PythonOct 16, 20187 years agoMIT
blind-source-separationblinkyindependent-vector-analysismicrophone-arraymulti-modal+1 more

Similar Papers

ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior

79% match1 repo412025

Modality-Specific Speech Enhancement and Noise-Adaptive Fusion for Acoustic and Body-Conduction Microphone Framework

78% match2025

A joint separation-classification model for sound event detection of weakly labelled data

77% match1 repo152017

Blind Source Separation of Single-Channel Mixtures via Multi-Encoder Autoencoders

77% match1 repo192023

Adversarial Semi-Supervised Audio Source Separation applied to Singing Voice Extraction

76% match1 repo832017

Study of the Performance of CEEMDAN in Underdetermined Speech Separation

76% match2024

Cite this paper

@article{scheibler2019multimodal,
  title  = {Multi-modal Blind Source Separation with Microphones and Blinkies},
  author = {Robin Scheibler and Nobutaka Ono},
  year   = {2019},
  eprint = {1904.02334},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1904.02334}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1904.02334.svg)](https://codeofpaper.com/paper/1904.02334)

Discussion