Cascaded Feedforward Neural Networks for speaker identification using Perceptual Wavelet based Cepstral Coefficients

The system performance during training and normal operation stages is improved by performing preprocessing. It is usually done in a series of steps that standardize the characteristics of the speech signal. The different stages in this phase involve i) pre-emphasizing ii) detection of starting and final speech signal point and iii) normalization of the signal amplitude. Initially, the speech signal is pre-emphasized by a first order high pass filter in order to accomplish all the formants of the signal having the same amplitude. The high pass filter recovers the effectiveness of the stages used for calculating the speech spectrum.

The beginning and final point of the word is detected by considering the energy and zero crossing of the speech signal. Let us consider that the signal x _i  (n) , n = 1, 2, …… , N is divided into non-overlapping frames of 50ms length, where N is the length of the frame. The signal energy for each frame for detecting silent periods in the audio signals is obtained as E i = 1 N ∑ n = 1 N | x i ( n ) | 2(1)

Next, the spectral position of the signal is located by computing the spectral centroid of the i ^th frame. Alternative frequency modulated (FM) features convey similar information as the spectral centroid features. But, Spectral centroid frequency carries formant-related information which are robust to additive noise than the other frame-averaged FM components [37] and is given by C i = ∑ k = 1 N X i ( k + 1 ) X i ( k ) ∑ k = 1 N X i ( k )(2) where X _i  (k) , k = 1, 2, …, N is the Discrete Fourier Transform (DFT) of the i ^th frame. The speech segments of the signal are extracted by finding the histogram of the DFT sequence of each frame followed by a smoothing filter, and then threshold T = wM 1 + M 2 w + 1 is applied to the feature sequences for detecting the local maxima of the histogram, where w is a user-defined parameter, M₁ and M₂ are the positions of the first and the second local maxima respectively. The frames which have values of energy, spectral centroid and histogram lesser than the threshold are discarded, and the remaining segments are merged. Finally, amplitude normalization is done to obtain greater similarity between the speech signals, thus avoiding the variation of amplitude during the recording process due to speaker position, fatigue or distortion.

The input signal is decomposed commensurate with the perceptual frequency scale of the human auditory system. Perceptual Wavelet Packet Transform (PWPT) is used to decompose the signal into several frequency subbands that approximate the critical bands. The sampling rate is 16 KHz, yielding a speech bandwidth of 8 KHz. Within this bandwidth, there are approximately 24 critical bands, and the decomposition is effected by a competent 7-level tree structure. The corresponding PWPT decomposition tree can be constructed as depicted in Fig. 2.

OPEN IN VIEWER

Fig.2

Tree structure of PWPT.

Wavelets help in analysing a signal according to a scale. Successive division in time and frequency scale, decomposes the signal into sets of smaller signals at multiple levels. Good time resolution with poor frequency resolution at high frequencies are suitable for short-duration high-frequency components and high-frequency resolution with poor time resolution at low frequencies for long-duration low-frequency components. They help in approximating data which has sharp discontinuities.

The WP utilizes a recursive binary tree for the recursive disintegration of data. A pair of filters represented by h [k] and g [k] is used for the generation of two sequences to obtain different frequency subband features of the original signal. These are the impulse response of lowpass and highpass paraunitary Quadrature mirror filters (QMF). Let { s n ( t ) n = 0 ∞ denote a sequence of functions, which can be divided into two sets of frequencies as follows:

s 2 n ( t ) = 2 ∑ k = ∞ ∞ h [ k ] s n ( 2 t - k )(3)

s 2 n + 1 ( t ) = 2 ∑ k = ∞ ∞ g [ k ] s n ( 2 t - k )(4) where s₀ (t) = φ (t) is the scaling function and S₁ (t) = ψ (t) is the wavelet function. (3) and (4) represents dividing the signal S _n  (t) into a component containing its approximated version (low-frequency part) and a component with the residual details (high-frequency part) respectively. The decomposition consists of the following steps:

Initially, divide the entire band of the speech signal by performing 3-level wavelet packet decomposition to get 8 subbands each of 1KHz.

The nodes of the tree corresponding to the frequency bands that mimic the Human Auditory Perception (HAP) system are selected. They are sequentially added and appraised, resulting in a non-uniform perceptual filter bank, with 24 terminating nodes. The center frequency f _c in wavelet scale is given by f c = f 1 f 2 , where f₁ and f₂ are the upper and the lower frequencies of each band. Table 1 shows the details of the selected nodes, frequency bands and their center frequencies in wavelet scale. Columns 8 and 9 depict energy and variance details of a particular speaker (S₁) of TIMIT database. After performing decomposition and node selection, energy (E _i ) and variance (V _i ) of each of the subbands (S _i ) i = 1, 2, 3, . . .24 has been found out by

< E i > j = ∑ [ W φ ( S ij ) i ] 2 N i(5) where Wφ (S) is S _i WPT of the speech signal, i is the subband frequency index, j is the temporal frame, and N _i is the number of coefficients in the i^th subband.

< V i > j = ∑ [ W φ ( S ij ) i - M i ] 2(6) where M _i is the mean value of the signal. The log of equal loudness energy and variance has been applied subsequently obtaining 24 coefficients. Finally, Discrete Cosine Transform (DCT) has been applied to these coefficients to de-correlate the energies and the variances of the filter banks. It is noted from Table 1 that, energy and variance for the subbands 14 to 24 are either zero or very less compared to the values of other subbands. The amplitude variation of the selected frequency band can also be analyzed by calculating Power Spectral Density (PSD) of the bands [42]. The PSD for 24 wavelet subbands for speaker (S₁) is shown in Fig. 3.

OPEN IN VIEWER

Fig.3

Power spectral density (PSD) of the bands.

The variation of PSD for 14 to 24 subbands is negligibly less compared to other frequency bands. Fig. 4 shows energy and variance distribution chart of speech belonging to 4 different speakers (S₁, S₂, S₃, S₄) in TIMIT database.

OPEN IN VIEWER

Fig.4

Energy/variance distribution chart for 4 different speakers.

It is investigated that, the cepstral coefficients for all the four speakers (S₁, S₂, S₃, S₄) are high for the levels 1,2,3,4 and 13 and beyond the 13^th level, i.e. 14 to 24, the energy and variance is approximately zero. From level 5 to 12, its value is very less. Since the variance distribution is approximately zero after the 13^th level of wavelet decomposition, 14^th to 24^th levels are discarded, only the first 13 levels are considered, which, in turn, reduces the computation thereby increasing the performance of the speaker identification system.

The first 13 cepstral coefficients, i.e. energy and variance corresponding to the highest 13 values are selected as feature vectors {F _ei } called Energy-based Static PWPCC (En-SPWPCC), and {F _vi } called Variance-based Static PWPCC (Var-SPWPCC). However, the static PWPCC describes only the frequency component of a single frame. To incorporate the dynamic changes over multiple frames, time derivatives of the features, i.e. the First Order Linear Regression (FOLR) coefficients (referred to as delta coefficients) are calculated using the recursion formula Δ F e = grad ( F e ) = { Δ F e } = ∑ θ = 1 Θ ( F et + θ - F et - θ ) 2 ∑ θ = 1 Θ θ 2(7) Δ F v = grad ( F v ) = { Δ F v } = ∑ θ = 1 Θ ( F v + θ - F v - θ ) 2 ∑ θ = 1 Θ θ 2(8) where ΔF _e is the energy delta coefficient computed in terms of the corresponding static coefficients F_et+θ and F_et-θ and △F _v is the variance delta coefficient computed in terms of the corresponding static coefficients F_vt+θ and F_vt-θ. The Second Order Linear Regression (SOLR) coefficients (referred to as acceleration coefficients), are given by Δ 2 F e = grad ( F e ) = { Δ 2 F ei } = ∑ θ = 1 Θ ( F et + θ - F e - θ ) 2 ∑ θ = 1 Θ θ 2(9) Δ 2 F v = grad ( F v ) = { Δ 2 F v i } = ∑ θ = 1 Θ ( F vt + θ - F vt - θ ) 2 ∑ θ = 1 Θ θ 2(10) where Δ²F _e is the energy acceleration coefficient and Δ²F _v is the variance acceleration coefficient. By computing these, we get the dynamic features, En-DPWPCC and Var-DPWPCC. The fused PWPCC is obtained by appending the dynamic features along with the static features, thus obtaining a new feature set F = {F _vi , ΔF _vi , Δ²F _vi } and F = {F _ei , ΔF _ei , Δ²F _ei } resulting, 39–element fused Var-PWPCC and fused En-PWPCC.

3. Phase III: CFNN-based speaker identification

The Cascaded Feedforward Neural Network with backpropagation algorithm shown in Fig. 5 is employed for classification [36]. The architecture consists of input layer, hidden layer and output layer. The input neuron is connected to all the hidden neurons and the hidden neuron is connected to all the input neurons as well as the output neurons. The network is built by adding hidden units at a time, until the desired mapping is obtained.

OPEN IN VIEWER

Fig.5

Architecture of cascade feed forward neural network.

CFNN starts learning with only one neuron. Then, the algorithm adds and trains new neurons forming a multilayer structure. The internal connections of the network are built up accordingly, thereby achieving the best performance in each new layer. These networks are similar to FNN except that they have a weight connection from the input and every previous layer to the following layers.

The network is trained and validated with Levenberg-Marquardt backpropagation (LMBP) algorithm. LMBP uses a training function that updates weight and bias values according to Levenberg-Marquardt optimization. It is one of the fastest backpropagation algorithms and a good supervised algorithm, although it does require more memory than other algorithms. This algorithm is an iterative technique that locates a local minimum of a multivariate function that is expressed as the sum of squares of several non-linear, real-valued functions. Also, it is a standard technique for non-linear least-square problems, widely adopted in various disciplines for dealing with classification problems [38]. During the training phase, feature vectors are presented to CFNN for training. For testing, the extracted feature vectors from the test signal are fed to the network for further classification. The procedure used in the test phase is almost the same as the one used in the training phase. The test speaker’s voice is preprocessed, and features are extracted and further classified individually by the trained CFNN. The additional connections in CFNN help to improve the speed of training and learning. The main steps of the learning CFNN algorithm are described below.

Initialize the weights with small random values say W _k , k = 1, 2, … M where M is the number of neural network layers.

The k-fold cross-validation method is used for the training phase, which is normally used to circumvent classification problem for ANN models [38–41]. In this scheme, the dataset is split into k equal-sized parts and randomly nominated one part is used as the test set, and the residual k–1 parts are used as the training dataset.

4.1 Experimental setup

In this section, we discuss the outcomes of the comprehensive testing performed on two different speech databases: one of them is a benchmark database –TIMIT corpus [3] and the other, a real voice database –STVOICE database [41]. The TIMIT database which is a result of intense research for more than two decades was designed to be task and speaker-independent, and is suitable for general acoustic-phonetic research. Moreover, TIMIT is a corpus of phonemically and lexically transcribed speech of 24 speakers of both gender and different dialects with each transcribed element being delineated in time. The database is recorded with loud and whisper voices of speaker uttering 450 phonetically balanced sentences of the TIMIT prompt set. It is widely used in speech analysis and speaker recognition studies, since it is one of the few databases with a relatively large number of speakers. It is a single-session database recorded in a sound booth with fixed wideband headset. TIMIT corpus includes time-aligned orthographic, phonetic and word transcriptions as well as a 16-bit, 16 kHz speech waveform file for each utterance. Each speaker speaks 26 sentences with approximately 8 words in each. Real voice database, STVOICE [41] has also been created to test the proposed methodology for its efficacy in speaker identification. The voice group consists of 8 volunteers from student community in the age group of 18-22 years. The recording of the voice has been carried out in a laboratory environment. The recording setup consists of a sensor, a recorder and a computer. The voice input was recorded in wav format with the sampling rate of 44.1 kHz and all the subjects spoke sentences in English language. For simplicity, a set of word samples was framed and recorded for each of the subjects.

4.2 Data preparation

First, the speech signal is resampled with the rate of 16 KHz and preprocessed. Then, the features En-SWPCC, Fused En-SWPCC, Var-SPWPCC and Fused Var-PWPCC are computed as discussed in Sections 2 and 3. These features are presented to CFNN for further training and testing. The presented data are automatically mapped onto the range -1 to+1. The input data are randomly divided into 60% for training, 20% for testing and 20% for validation. CFNN architecture used in this work has an input layer with 13 neurons for SPWPCC features and 39 neurons for fused PWPCC features, one hidden layer with 20 neurons with sigmoid activation function. For output layer, linear activation function is used. The network is trained with Levenberg-Marquard backpropagation algorithm

4.3 Speaker identification rate (SIR)

The performance of the ANN classifier is measured by finding the SIR which is given by the equation SIR = Correct classification Corerect classification + misclassification(17)

Table 2 summarizes the Speaker Identification Rate (SIR) for the proposed PWPCC with CFNN, by considering different wavelet families such as db, haar, sym, coif and bior.

The selection of the proper mother wavelet is one of the important issues in finding PWPCC. A good mother wavelet could improve the performance of the speaker identification system. The SIR using ‘db4’ wavelet is high, i.e. more than 96% with the maximum of 99.5% compared to the other mother wavelets as observed in Table 2. Thus, PWPCC using ‘db4’ can capture key information of the speech signal. Hence ‘db4’ wavelet is selected for further analysis purposes. Error histogram helps to verify the network performance. It indicates the outliers. Error histogram with 20 bins is shown in Fig. 6. The blue, green and red bars show training data, validation data and testing data respectively. Most data fall on zero outlier, which offers an idea to ensure outliers to govern whether the data is good or not. If the outliers are valid data points, but dissimilar from rest of the data, then the network is extrapolating for these points.

The magnitude of gradient and the number of validation checks are used to terminate training. The gradient will become very small as the training reaches a minimum of the performance. If the magnitude of the gradient is less than 1e-5, training will stop. If the validation performance fails to decrease in the successive iterations, it leads to a validation check.

OPEN IN VIEWER

Fig.6

Error histogram.

If this check reaches number 6, training will stop. The gradient and the validation check obtained for training CFNN are shown in Fig. 7.

OPEN IN VIEWER

Fig.7

Gradient and validation check.

The performance of the CFNN is also evaluated by comparing Levenberg-Marquardt back Propagation (LMBP) training algorithm with other Gradient Descent based training algorithms such as Gradient Descent Back Propagation (GDBP), Gradient Descent with Adaptive Learning rate Back Propagation (GDABP), Gradient Descent with Momentum Back Propagation (GDMBP) and Gradient Descent with Momentum and Adaptive Learning rate Back Propagation (GDMABP) by considering the same set of features using ‘db4’ wavelet, and the SIRs are shown in Fig. 8.

OPEN IN VIEWER

Fig.8

Comparison of SIR for different training algorithms.

The training algorithms GDBP and GDMBP bestow lower SIR than the other training algorithms, whereas GDABP and GDMABP produce better values.

Since, training by means of LMBP is fast and has stable convergence property, it produces high SIR for both the databases as shown in Fig. 8. SIR for the proposed speaker identification system is compared with other state-of-the-art methods, and the results are shown in Table 3.

We find from Table 3 that Fourier transform-based feature extraction techniques, i.e. MFCC produced less SIR, i.e. ∼65 to 80%, since it fails to detect the non-stationary properties of the speech signal. But by introducing wavelet-based features, i.e. WCC gives more than 87%. But still the performance is less because only low frequency components are considered for decomposition in WCC-based features. By the application of WPCC, SIR is increased to ∼1-3%. The proposed PWPT-based features, En-SPWPCC and CFNN, Var-SPWPCC and CFNN, Fused En-PWPCC and CFNN, Fused Var-PWPCC and CFNN give more promising results compared to the other methods. It is also noticed that CFNN produces improved SIR than other classifiers such as FNN, PNN and VQ.

The proposed methodology is also implemented and tested under different noise conditions, by adding Additive White Gaussian Noise (AWGN) of different noise levels along with the speech data, and the features are extracted using the same procedure discussed in Section 3 and SIR for different noise levels (20 dB, 15 dB, 10 dB,5 dB, 1 dB and -1 dB) are shown in Fig. 9.

As it is seen in Fig. 9, all the PWPCC methods produce better results, i.e. more than 80% under different noise levels. Even for low SNR, i.e. for -1 dB, SIR is ∼82% and for high SNR, i.e. for 20 dB, SIR is high. The above methods are compared with other conventional methods and the results are tabulated in Table 4. It is found that the performance in terms of SIR is greatly improved, for both the features SPWPCC and fused PWPCC. The use of PWPT gives at least 20% improvement. At the same time, the improvement from the existing MFCC to WPCC is very less, when noise disturbance is more.

OPEN IN VIEWER

Fig.9

Comparison of the proposed methods with different noise levels.

Thus it is clear that the proposed Perceptual WPCC and CFNN-based approaches prove their ability to identify the speakers, even in the noisy environment with low SNRs without any additional denoising techniques, thereby reducing the complexity and increasing the speed.

The proposed Perceptual Wavelet Cepstral Coefficient based feature extraction method can efficiently detect the non-stationary characteristics of the signal, but the state-of-art Mel-frequency Cepstral Coefficients (MFCC) based feature extraction methods produced less Speaker Identification Rate (SIR), since it fails to detect the non-stationary attributes of the signal. The proposed features also provide two dimensional information of the signal which captures both energy and variance along with the energy distribution within each subband i.e., we obtain a vector of approximation coefficients and detail coefficients, both at various scales, whereas the MFCC features are only based on vocal tract acoustics, which averages the energy of the signal as a single dimension. Another important aspect of the proposed approach is the wavelet decomposition tree. In the proposed methodology, we have adopted the ERB based Perceptual Wavelet Packet decomposition, which closely approximates the Human auditory system unlike the Mel filter bank used in MFCC. This ERB scale was originally designed such that the frequency axis models the human cochlear filtering. Further, the choice of classifier plays an important role for producing better Speaker Identification Rate (SIR). This speaker recognition approach uses the Cascaded Feeedforward Neural Network Classifier (CFNN). The CFNN is iterative and is one of the fastest backpropagation algorithms that uses a good supervised algorithm, than the conventional feedforward network.