A Submodularity-based Agglomerative Clustering Algorithm for the Privacy Funnel

For the privacy funnel (PF) problem, we propose an efficient iterative agglomerative clustering algorithm based on the minimization of the difference of submodular functions (IAC-MDSF). For a data curator that wants to share the data $X$ correlated with the sensitive information $S$, the PF problem is to generate the sanitized data $\widehat{X}$ that maintains a specified utility/fidelity threshold on $I(X;\widehat{X})$ while minimizing the privacy leakage $I(S;\widehat{X})$. Our IAC-MDSF algorithm starts with the original alphabet $\widehat{\mathcal{X}}:=\mathcal{X}$ and iteratively merges the elements in the current alphabet $\widehat{\mathcal{X}}$ that minimizes the Lagrangian function $I(S;\widehat{X})-\lambda I(X;\widehat{X})$. We prove that the best merge in each iteration of IAC-MDSF can be searched efficiently over all subsets of $\widehat{\mathcal{X}}$ by the existing MDSF algorithms. We show that the IAC-MDSF algorithm also applies to the information bottleneck (IB), a dual problem to PF. By varying the value of the Lagrangian multiplier $\lambda$, we obtain the experimental results on a heart disease data set in terms of the Pareto frontier: $I(S;\widehat{X})$ vs. $-I(X;\widehat{X})$. We show that our IAC-MDSF algorithm outperforms the existing iterative pairwise merge approaches for both PF and IB and is computationally much less complex.