%last updated on 14.01.2000 by A. Endemann

\documentclass[runningheads]{llncs}
%In order to omit page numbers and running heads
%please use the following line instead of the first command line:
%\documentclass{llncs}.
%Furthermore change the line \pagestyle{headings} to
%\pagestyle{empty}.

\usepackage{epsfig}

\begin{document}

\pagestyle{headings}
%In order to omit page numbers and running heads
%please change this line to
%\pagestyle{empty}
%and change the first command line too, see above.

\mainmatter

\title{A Speaker Authentication Module in TelCorreo}

\titlerunning{A Speaker Authentication Module in TelCorreo}

\author{Leandro Rodr\'{\i}guez Li\~{n}ares\inst{1} \and
            Carmen Garc\'{\i}a Mateo\inst{2}}

\authorrunning{Leandro Rodr\'{\i}guez-Li\~{n}ares and Carmen Garc\'{\i}a-Mateo}

\institute{E.T.S.E. Inform\'{a}tica\\
    Campus As Lagoas\\
    32004 - Ourense, Spain\\
\email{leandro@tsc.uvigo.es}
\and
E.T.S.E. Telecomunicaci\'{o}n\\
    Campus Universitario de Vigo\\
    36200 - Vigo (Pontevedra), Spain\\
\email{carmen@tsc.uvigo.es}}

\maketitle

\begin{abstract}
Our experiments in Speaker Recognition showed that the combination
of Speaker Verification and Utterance Verification techniques is
an efficient way of improving the performance of a Speaker
Authentication System.

This technique is know implemented in the speaker authentication
module of TelCorreo, an e-mail client that allows
Internet users to read their e-mail using speech through the
telephone.
\end{abstract}


\section{Introduction}

TelCorreo \cite{telcorreo,telcorreowww} is a real-world
application designed to give its users the ability to access their
electronic mailbox and handle their electronic messages through a
simple telephone call. It can be viewed as a bilingual
(Spanish/Galician) e-mail client application which interacts with
the user through a speech interface.

Traditionally, the security of systems like TelCorreo relied on
the use of some private code or password. Given the fact that it
has been designed as a dialog system, the basic technology it
uses allowed us to implement a voice-based user authentication
module.

Presently, there are very few speaker recognition systems to be
deployed in real-world applications. The performance achieved by
most of the techniques proposed in the literature dramatically
drops for out-of-lab conditions. Scanning the literature, we can
find two generic approaches for designing a speaker authentication
system:
\begin{itemize}
\item Verifying the actual content of the speech
as a mean of verifying the speaker identity. We call these systems
``Utterance verifiers'' \cite{li}.
\item Verifying the speaker voice by itself as a way of verifying
the identity. This approach is by nature speech content
independent. We call these systems ``Speaker verifiers''
\cite{reynolds}, and although these systems are not speech
recognizers, they usually share with speech recognizers most of
their technology, namely, most of them are based on a HMM (Hidden
Markov Model) formulation with a front-end based on cepstral
analysis.
\end{itemize}

Our experiments in Speaker Recognition showed that the combination
of Speaker Verification and Utterance Verification techniques is
an efficient way of improving the performance of a Speaker
Authentication System. In \cite{tesis} and \cite{icslp} several
ways of combining such systems are explored using our database
called ``TelVoice''.

Among the explored combination techniques, the neural network
combination outperforms the others due to its ability to learn the
optimal operation point from the data. However, the real-world
implementation of this scheme is only possible when a certain
amount of experimental data has been collected.

For TelCorreo, we decided to use what we called the ``restrictive
criterion''. This is equivalent to demand that both tests have to
be passed simultaneously to accept the speaker.

The rest of this paper is structured as follows. Section 2
explains the module design from the users' point of view, while
section 3 presents TelCorreoDB: an experimental frame adequate for
testing the speaker authentication technology used in TelCorreo.
Section 4 presents the implementation details. Finally, some
conclusions are included.

\section{Speaker Authentication Module Design}

In this subsection, the speaker authentication module is presented
from the user's point of view. When the user access to the system,
he/she is asked to introduce the personal identification number
using the telephone keypad. From this moment, the goal of the
speaker authentication module is to decide if the speaker is
actually who he/she claims to be.

The first thing the system does is to verify if the voice speaker
authentication option is activated or not for this presumed
speaker:
\begin{enumerate}
\item If not, the user is asked to introduce his personal user's
password using the telephone keypad. If the password is correct,
the rest of the call is managed by the system's kernel.
\item If the option is activated, there are two possible
situations:
\begin{enumerate}
\item If the user has trained an acoustic model previously, he
is asked to pronounce the personal user's password. These
utterance is used by the speaker authentication module to
determine if the accessing attempt has to be accepted or not.
\item If there is no model for this user, he is asked to
introduce the personal user's password with the telephone keypad.
After the password is verified, the system asks if the user wants
to train his model or not in this moment. If he does, the user
is asked to repeat a set of phrases for training a model that will
be used in the following accessing attempts. If the user does not
want to train a model in this moment, the control of the call is
passed to the system's kernel.
\end{enumerate}
\end{enumerate}


\section{TelCorreoDB}

When transferring technology from laboratory to real-world, an
important point is how to test and adjust the systems before their
field deployment. This process must be made in conditions close to
the ones the system is going to have to deal with in real-world.

With this purpose in mind, we recorded TelCorreoDB: a database
designed as an experimental frame adequate for testing the speaker
authentication technology used in TelCorreo. TelCorreoDB includes
voice of 15 speakers recorded in the same conditions in which
these speakers would access through the speaker authentication
module. Each speaker recorded one training session, which consists
of 8 fixed phrases up to a total of approximately 50 seconds and
three test sessions, each one with a pronunciation of the personal
user's password.

We used TelCorreoDB for the following tasks:
\begin{enumerate}
\item Testing the components of the speaker authentication module:
parametrization, training, users' information mantainement...
\item Estimating thresholds for the speaker verification sub-module.
\item Training the world model.
\end{enumerate}

\section{Implementation Details}

\subsection{Front-end Block}

The modules should share technology as much as possible due to
computational reasons. This includes a front-end block that
performs the parametrization of the input utterance. The used
parametrization consist of the energy and 12 mel-cepstra
coefficients with their first and second derivatives. This makes a
total of 39 parameters per vector.

\subsection{Utterance Verification Sub-module}

The Utterance Verifier in TelCorreo is the same speech recognizer
used for the dialog management. As the accessing codes consist of
four digits, a simple grammar that allows all the possible
combination is used. In \cite{telcorreo}, it is stated that
TelCorreo is a bilingual system that can be used in Spanish or
Galician indistinctly. Thus, the grammar must allow the presence
of digits in both languages. The output of the utterance verifier
consists in the sequence of five recognized digits.

\subsection{Speaker Verification Sub-module}

We decided to use 16-mixtures covariance-tied GMM's (Gaussian
Mixture Models), which are a special case of continuous HMM's
where the number of states is one. This type of model has proved
to be effective for modeling the speaker identity in
text-independent speaker recognition applications \cite{reynolds}.

In a speaker verification problem the goal is to determine whether
a person is who he or she claims to be. The most straightforward
approximation would be to use the log-likelihood of the sequence
of parameters given the supposed speaker's model $\log ( P({\bf O}
/ \lambda_k))$. This is what it is called {\em unnormalized
log-likelihood score}. The speaker is accepted when the
unnormalized score $S_{\mathrm{unnor}}({\bf O},k)$ is above a
certain threshold $\tau_k$:
\[
S_{\mathrm{unnor}}({\bf O},k) =  \log ( P({\bf O} / \lambda_k)) \;
\geq \; \tau_k
\]
In \cite{reynolds}, it is stated that to avoid taking the verification
decision on a relative score dependent on non-speaker utterance
variations some normalization procedure must be used. This
normalization alleviates effects like voice quality or speaker's
vocal tract variations.
One possible solution to these problems is to use another
probability $P({\bf O} / \widehat{\lambda_{k}})$ as a
normalization factor to calculate the a normalized score:
\[
S_{\mathrm{nor}}({\bf O},k) = \frac{\log ( P({\bf O} / \lambda_k))}
{\log ( P({\bf O} / \widehat{\lambda_k}))}\;
\geq \; \tau_k
\]
$\widehat{\lambda_{k}}$ is called the {\em antimodel} and
represents the universe of possible speakers but the presumed one.
There are typically two strategies to build it up: to train a
world model shared by all the speakers or to use a set of
speaker-dependent models called cohort or background models. It is
normally accepted that the use of cohort models give better
performance than using a world model.

When transferring technology from laboratory conditions to
real-world applications, several restrictions must be applied.
This is particularly true in the Speaker Verification Sub-module,
where the data avalaible to training or estimation procedures is
limited. Thus, we have applied several restrictions:
\begin{itemize}
\item The number of registered users in TelCorreo is not constant.
Then, the use of cohort models for normalization purposes is not
practical, as each new registration would imply to recalculate the
cohorts for each registered speaker. We decided to use a {\em
world model}, trained using TelCorreoDB. This model consists of a
variance-tied 48-mixtures GMM.
\item As previously stated, a set of thresholds must be estimated
in order to decide if a utterance belongs to a user or not.
Theoretically, there are two possibilities: to use one threshold
shared by all the speakers or to use speaker-dependent thresholds.
The latter option give better results in laboratory conditions.
However, threshold estimation needs to be performed in the
conditions where the verification process is going to be
performed, and in a real world system this is not a practical
approach. Then, shared threshold must be used, and these
thresholds are a priori calculated using TelCorreoDB.
\end{itemize}


\subsection{Sub-modules Combination}

In summary, the system has to decide if a utterance belongs to a
speaker or not. This decision has to be made based on a string of
recognized digits and an acoustic score calculated by the Speaker
Verification Sub-module.

It was previously said that we decided to use the ``restrictive
criterion'', that is, to demand that both tests have to be passed
simultaneously to accept the speaker. To take into account
variations of channel or voice quality that would degrade the
verification performance, a two-level verification is performed
based on the use of two thresholds in the Speaker Verification
Sub-module. We call these thresholds $\tau_{low}$ and
$\tau_{high}$, where $\tau_{low} < \tau_{high}$.

The verification procedure is as follows. First, the recognized
sequence of digits is studied. This can lead the system to three
distinct situations:
\begin{enumerate}
\item If the sequence of digits is totally correct,
$S_{\mathrm{nor}}({\bf O},k)$ is compared with $\tau_{low}$. If
$S_{\mathrm{nor}}({\bf O},k) > \tau_{low}$ the speaker is
accepted, otherwise is rejected.
\item If there is only one error in the sequence of digits,
$S_{\mathrm{nor}}({\bf O},k)$ is compared with $\tau_{high}$. As
before, if $S_{\mathrm{nor}}({\bf O},k) > \tau_{high}$ the speaker
is accepted, otherwise is rejected. By one error in the sequence
of digits we mean one substitution, one insertion or one deletion.
\item If there is more than one error in the sequence of digits
the accessing attempt is rejected.
\end{enumerate}

This can be expressed as follows:
\[
\begin{array}{ccc}
\left. \begin{array}{c} N_{errors} = 0 \\ S_{\mathrm{nor}}({\bf O},k) > \tau_{low} \end{array} \right\}
    & \Rightarrow & Accepted\\
& & \\
\left. \begin{array}{c} N_{errors} = 1 \\ S_{\mathrm{nor}}({\bf O},k) > \tau_{high} \end{array} \right\}
    & \Rightarrow & Accepted\\
& & \\
{\mathrm Otherwise} & \Rightarrow & Rejected\\
\end{array}
\]

The estimation of thresholds $\tau_{low}$ and $\tau_{high}$ is
performed using TelCorreoDB. One ROC (Receiver Operating
Characteristic) is built representing the false acceptance rate
percentage (\%FA) versus the false rejection rate percentage
(\%FR) when the value of the threshold is varied. We decided to
use $\tau_{low}$ and $\tau_{high}$ as shown in table
\ref{thresholds}. It can be argued that the used database is
rather small and that these results may have low statistical
significance. However, these thresholds are a first estimation and
will probably have to be modified.

\begin{table}
\begin{center}
\begin{tabular}{cccc}
& Value & \%FA & \%FR \\
\hline
$\tau_{low}$ & -1 & $\approx$5 & $\approx$10 \\
$\tau_{high}$ & -0.5 & $\approx$3 & $\approx$18 \\
\hline
\end{tabular}
\end{center}
\caption{Estimated values for thresholds $\tau_{low}$ and $\tau_{high}$}
\label{thresholds}
\end{table}

\section{Conclusions and Further Work} \label{SecConclusions}

TelCorreo is a bilingual e-mail client over the telephone line
that integrates state-of-the-art speech technology
\cite{telcorreo}. One of the main and novel components of
TelCorreo is the User Authentication Module which is described in
depth.

TelCorreo is still under development. A few days before writing
this article, TelCorreo was deployed in real-world with real
users. The real-world working of the system will raise new issues
that will have to be solved. Examples of such issues are the
refinement of the thresholds or the adaptation of the users' models.


\begin{thebibliography}{5}
\bibitem{telcorreo}
L. Rodr\'{\i}guez-Li\~{n}ares et al.: TelCorreo: A bilingual e-mail client
over the telephone. TSD'2000.
%
\bibitem{telcorreowww}
In \verb+http://www.gts.tsc.uvigo.es/telcorreo+ a description of
TelCorreo can be found
%
\bibitem{li}
Q. Li et al.: Verbal Information Verification. EuroSpeech'97,
Rhodes, Greece, (1997).
%
\bibitem{reynolds}
D. Reynolds: Comparison of Background Normalization Methods for
Text-Independent Speaker Verification. Speech Communication, vol.
2, pp. 963-966, (1997).
%
\bibitem{tesis}
L. Rodr\'{\i}guez-Li\~{n}ares: Estudio y Mejora de Sistemas de
Reconocimiento de Locutores mediante el Uso de Informaci\'{o}n Verbal
y Ac\'{u}stica en un Nuevo Marco Experimental. Ph. D. Thesis,
Universidade de Vigo, Spain, (1999).
\verb+http://www.tsc.uvigo.es/~leandro/archivos/Tesis.ps.gz+
%
\bibitem{icslp}
L. Rodr\'{\i}guez-Li\~{n}ares and C. Garc\'{\i}a-Mateo: A Novel Technique for
the Combination of Utterance and Speaker Verification Systems in a
Text-dependent Speaker Verification Task. ICSLP'98, Sydney,
Australia, (1998).
%

%
\end{thebibliography}

\end{document}
