\(\newcommand{\Cov}{\mathrm{Cov}}\) \(\newcommand{\Corr}{\mathrm{Corr}}\) \(\newcommand{\Sample}{X_{1},\dots,X_{n}}\)
A good estimate for \(\mu\) for a symmetric distribution: The Hodges-Lehmann. For each \(i\le j\), compute \(\frac{1}{2}(x_{i}+x_{j})\). Then take the median of all the values.
Put another way, construct the set of all means of all pairs of the sample. Add to the set the sample itself (i.e. the means of single elements). Take the median.
This estimator is robust, and does not require knowledge of the distribution.