SciELO - Scientific Electronic Library Online

vol.36 número2On the use of Lee's protocol for speckle-reduncing techniquesOn-line costate integration for nonlinear control índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados



  • No hay articulos citadosCitado por SciELO

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google


Latin American applied research

versión impresa ISSN 0327-0793

Lat. Am. appl. res. v.36 n.2 Bahía Blanca abr./jun. 2006


Adaptive filtering using projection onto convex sets

L. Rey Vega1, S. Tressens1, and H. Rey1

1 Departamento de Electronica, Facultad de Ingeniera, Universidad de Buenos Aires,
Buenos Aires, Paseo Coln 850, Argentina,,

Abstract — In this paper we propose a novel adaptive filtering algorithm. The algorithm exploits the information given by the power spectral density of the noise extracted from the pe-riodogram of filtering error. The goal is try to match the spectral properties of the error filtering with the spectral properties of the measurement noise. With this in mind appropriate convex and closed sets are built and projections onto them are computed. The simulation results show that the algorithm has excellent convergence properties with a reduced number of updates. This could be exploited to obtain a lower computational load.

Keywords — Adaptive Filtering. Projections. Convex Sets. Periodogram. Power Spectral Density.


The problem of adaptive filtering can be interpreted as one in which an unknown system has to be estimated. Adaptive filtering has a great number of applications such as channel equalization, noise cancellation, echo cancellation, etc. (Haykin, 2002).

Set Theoretic Estimation has received considerable attention for the last 20 years (Combettes, 1993). It has been applied to a considerable number of problems like image processing (Combettes, 1997), signal restoration (Trussell and Civanlar, 1984), etc. The idea behind this approach is to use certain a priori information about the object to be estimated. The solution is required to be consistent with this information. This is the only requirement to be fulfilled.

The a priori information is used to build sets (property sets), in such a way that they contain the true object with a high degree of confidence. A solution to the problem can be stated in the following manner: find one element in the intersection of the sets. This task could be very difficult to implement in practice (Combettes, 1993).

The application of this framework to adaptive filtering has been reported too. Dasgupta and Huang (1987), Gollamudi et al. (1998), Huang (1986) and Nagaraj et al. (1999) proposed to bound the feasibility set (the intersection set built with the sets representing the pieces of a priori information) with hyperellipsoids at each time instant. Yamada et al. (2002) utilized a method based on parallel subgradient projection (PSP) techniques onto convex sets for recursive estimation of the true system. Yukawa and Yamada (2004) proposed an interesting modification to the PSP algorithm, which improves its performance. In those previous works, information about additive noise is used for the construction of the property sets. The algorithms derived in those works show excellent convergence properties for highly-colored inputs and reduced number of updates.

This paper proposes a novel adaptive algorithm following the ideas given by Yamada et al. (2002). It uses information about the power spectral density of the noise. The periodogram of the filtering error plays a fundamental role in the algorithm for testing the consistency of the successive estimations with the information about the power spectral density of the noise.

Throughout the paper, the following notations are used: N and N are real and complex Hilbert spaces with inner products = xTy and = xHy respectively, where the superscripts T and H denote transposition and complex conjugate transposition. For any nonempty closed convex set in a Hilbert space , the projection operator is defined by .


Let N be an unknown linear FIR system. This is a common assumption in system identification because FIR systems constitute a simple and effective approximation in many practical problems. The associated adaptive filtering problem is shown in Fig. 1. The input signal at time n, x(n) = [x(n) x(n - 1) ... x(n - N + 1)]TN pass through the system giving an output . This output is observed but in this process usually appears a noise v(n) ∈ which will be considered additive. Thus, each successive input x(n) gives an output . The idea is to find to estimate w0. This filter receives the same input x(n), leading to an output estimation error .

Figure 1: An adaptive filtering problem

In the sequel we define the M × 1 output data vector y(n) = [y(n - M + 1) y(n - M + 2) ... y(n)]T and the N × M input data matrix X(n) = [x(n - M + 1) x(n - M + 2) ... x(n)]. It can be defined the M × 1 error vector .


A. Constructing the property sets

In the Set Theoretic Estimation framework the solution has to be consistent with the available a priori information. In this paper it is assumed that there is some a priori information about the additive noise. In fact it is assumed that its spectral density power is known. If a perfect estimation of w0 is available, , then . It can be proved that v(n) and have the same properties in their probability distributions (Combettes and Trussell, 1991). Defining the following set:


where , it could be known with a probability 0 < Pk < 1 that the true system w0 is in . The probability Pk depends on the distribution of the noise and on the parameter ξk. It is easy to see that is built by taking the periodogram at frequency bin k of the vector . It is known that the periodogram is a simple statistic for the spectral density power of a stationary stochastic process. The set is known as a property set (Combettes, 1993). In the Set Theoretic framework it is reasonable to seek the solution in this set provided that Pk is close to 1.

B. Determining ξk

Strictly, to determine ξk to guarantee that Pk is close to 1 the noise probability distribution has to be known. This kind of knowledge can be difficult to have. But if the noise v(n) is white and gaussian with variance σ2, it can be shown that I02 and IM/22 have a distribution, and 2I1σ2, ... 2IM/2-12 have a distribution, where:


and v(n) = [v(n - M + 1) v(n - M + 2) ... v(n)]T. For k = M/2 + 1, ..., M - 1, the results are the same due to the even symmetry of the periodogram of real signals. The determination of ξk for a required probability Pk can be accomplished using chi-squared tables. Moreover, if v(n) is not gaussian or white but it is a strongly mixing process (Combettes and Trussell, 1991) with summable second- and fourth-order cummulant functions and spectral density g(fk) with 0 ≤ fk = k/M ≤ 1/2 k = 0, 1, ..., M/2, it can be shown that I0/g(0) and IM/2/g(1/2) are asymptotically distributed as , and are asymptotically distributed as . As a result, in the general case, the sets can be built having knowledge of the spectral density provided that v(n) satisfies the above mentioned hypothesis.

C. Solving the problem

It is required to find a point in because this is the consistency condition that any valid solution has to fulfill. Actually, we need to find a point in:


to be consistent with all spectrum information. are closed and convex sets in a Hilbert space. It can be proved easily that is also a closed and convex set. Then, the concept of a projection in Hilbert space can be applied to find a point in given an arbitrarily point in the total space (Luenberger, 1969). However, the computation of the projection over can be a formidable task, while the projections over each can be more easily obtained. The POCS (Projections onto Convex Sets) method can be utilized to find a point in the intersection of a family of closed and convex sets using the individual projections (Combettes, 1993). However, its application in a real time problem which is the nature of adaptive filtering problem can be difficult or even impossible.

Yamada et al. (2002) proposed a general algorithm of potential application to a real time problem using the individual projections. It was used with other property sets, but it can be used with the sets defined in (1). Using these sets, the algorithm can be expressed in the following manner:


where is the projector onto , , k and . The parameter Ln ∈ (0,2Mn) is a relaxation parameter (Combettes, 1997) and Mn is:


It can be proved that Mn ≥ 1. Yamada et al. (2002) proved that the algorithm has the Fejér-monotonicity property: for every :


If we assume that ,the property is true for w0. These results are still valid taking the projections onto closed and convex sets that satisfy:


This last result allows the use of computable projections, if the ones onto the property sets are difficult to obtain. In view of this last result, the projections are computed using subgradients of convex functions.


It can be shown that the projections onto the sets defined in (1) are very difficult to obtain if we assume that all the quantities are real. For this reason it is necessary to find a way to circumvent this problem. It can be possible to follow the same steps that those carried on by Yamada et al. (2002) using subgradients. However another approach is possible. In this paper the following sets are considered:


These sets are built in N and have the property (7) assuming that X(n), y(n) and are real quantities. The projections onto the sets for each k can be computed more easily using the Lagrange multipliers (Luenberger, 1969):




Replacing these results in (4), the algorithm is obtained. For the calculation of it is necessary to check if belongs to . It is not difficult to show that the following rule applies:


The equations (10) and (12) show that the periodo-gram of the filtering error has to be evaluated for checking the membership of to (and because of (7), to ). Then the periodogram of the filtering error evaluates the degree of consistency of with the information about the power of the noise at frequency bin k. If this degree of consistency is high enough there is no need of update at this frequency bin.

The parameter controls the update in each frequency k. If = 0 ∀k at a given n, it is not difficult to see that . This possible absence of updates has been reported in the literature in others adaptive algorithms derived according to the Set Theoretic Estimation ideas (Dasgupta and Huang, 1987; Gollamudi et al., 1998; Huang, 1986; Nagaraj et al., 1999). Significant saving of computations can be achieved due to this feature of this adaptive algorithm.

It can be shown that the result in (9) is a complex vector. This can be a problematic situation since the final vector must be real because the true system is assumed to be real. In order to handle with this situation we have the following proposition:

Proposition 1 Given (4) and (5) where each projection is given by (9) and (10) and assuming that X(n), y(n) and are real quantities and k = 1, 2, ..., M/2 - 1 with M even, it can be proved that is a real vector.

Proof. Since we assumed that is a real vector it follows from (4) that we have to analyze the term:


It is clear that if then k = 1, 2, ..., M/2 - 1. The reason for this is that all the quantities in the algorithm are real and the periodogram of real quantities has even symmetry (we also assumed that ξk-M/2 = ξk+M/2 which is true if all the quantities are real and we have the same confidence levels for all the frequencies in the periodogram). It is not difficult to see that:


where {} denotes the imaginary part. The term:


has even symmetry and it can be proved that:


With these results it is not difficult to see that:


It should be clear that:



To verify the efficacy of the proposed algorithm, it is compared with the algorithm (PSP) proposed by Yamada et al. (2002), the APA algorithm, which is a well-established adaptive algorithm (Gay and Tavathia, 1995) when the input signal is highly-colored and the NLMS algorithm which a low complexity reference. The true system to be estimated is w064. The input signal is generated by filtering a white, zero-mean, gaussian random sequence through a first-order system G(z) = 1/1 - 0.95z-1. This input is highly-colored. The noise is white, zero-mean and gaussian with . The system mismatch defined as, , is evaluated. The PSP algorithm uses q = 1 and for the parameter that define the corresponding property sets (Yamada et al., 2002), where r = 8 and σ2 is the variance of the noise. The order of the APA algorithm is p = 8 and μ = 1. The regular-ization of the APA algorithm take the value of 20 times the power of the input signal, thus following Gay and Tavathia (1995). For the NLMS algorithm μ = 1 and the regularization factor is the as the APA. The proposed algorithm uses M = 8. The parameters are computed with chi-squared tables to obtain Pk = 0.99 k = 0, 1, ..., M -1. The coefficients are set equal to 1/Mn, k in the proposed algorithm and in the PSP algorithm. The technique developed by Yukawa and Yamada (2004) could be applied to this algorithm to improve its convergence properties. The curves shown are the result of the ensemble of 50 independent trials.

In Fig. 2 the proposed algorithm is compared with the APA algorithm and the PSP algorithm. The proposed algorithm presents almost the same speed of convergence than the APA algorithm with a lower final error. The PSP algorithm, under this kind of input signal, shows a lower speed of convergence, but a lower final error than the APA algorithm. The good performance of the proposed algorithm with respect to the PSP algorithm is due to the more complete information provided by the property sets in (8). The PSP algorithm uses information about the total power of the noise while the proposed algorithm uses information about the power of the noise at each frequency. The NLMS has a good final error but a poor speed of convergence. In Fig. 3 the proposed algorithm is tested under different values of M. The speed of convergence and the final error are improved as this parameter becomes larger. However, the performance of the algorithm with M = 2 is still good. The algorithm was tested in other conditions (other input signals, different filter lenght, etc.) and its performance was very good.

Figure 2: Proposed algorithm versus APA, PSP algo-

Figure 3: Proposed algorithtm with M=2, M=8 y rithm and NLMS under SNR=20 dB. M=16.

Finally, we compared the computational cost of the algorithms. In Fig. 4 we computed the normalized average number of "effective" projectors ( = 0) per iteration. This could be thought as an estimator of the probability of computing an "effective" projector at each iteration. In this simulation, both algorithms had nearly the same mismatch curve (not shown). As we can see, the proposed algorithm requires less computations. The total average number of "effective" projectors for the PSP algorithm was 32719.92, and for the proposed one, it was 3022.08.

Figure 4: Proposed algorithm (M = 16) versus the PSP algorithm (q = 24, r = 1) under SNR=20 dB. (1998).


A novel adaptive algorithm has been proposed in which information about power spectral density of the noise is used. The algorithm has a reduced number of updates and shows excellent convergence properties under highly-colored inputs. This is very important because exists different algorithms that have a lower computational load but have poor convergence properties under colored inputs (LMS algorithm is a good example). It is in these situations where the proposed algorithm should be used instead of those algorithms. This fact make the algorithm suitable for treating problems like echo cancellation. The information about the power spectral density of the noise can be used to improve the convergence behavior of the algorithm when the noise is not white. Finally the reduced number of updates of the proposed algorithm could be exploited to obtain an improvement in the resources for its numerical implementation.

The authors would like to express their deep gratitude to Prof. Alejandro De la Plaza from University of Buenos Aires for his help in preparing the manuscript.

1. Combettes, P. and J. Trussell "The Use of Noise Properties in Set Theoretic Estimation", IEEE Transactions on Signal Processing, 39, 1630-1641, (1991).         [ Links ]
2. Combettes, P., "The Foundations of Set Theoretic Estimation", Proceeding of the IEEE, 81, 182-208, (1993).         [ Links ]
3. Combettes, P., "Convex Set Theoretic Image Recovery by Extrapolated Iterations of Parallel Subgradient Projections", IEEE Transactions on Image Processing, 6, 493-506, (1997).         [ Links ]
4. Dasgupta, S. and Y.F. Huang, "Asymptotically Convergent Modified Recursive Least-Squares with Data-Dependent Updating and Forgetting Factor for Systems with Bounded Noise", IEEE Transactions on Information Theory, IT-33, 383-392, (1987).         [ Links ]
5. Gay, S. and S. Tavathia, "The fast affine projection algorithm", Proc. IEEE ICASSP, 3023-2026, (1995).         [ Links ]
6. Gollamudi, S., S. Nagaraj, S. Kapoor and Y. F. Huang, "Set-Membership Filtering and a Set-Membership Normalized LMS Algorithm with Adaptive Step Size", IEEE Signal Processing Lett., 5, 111-114, (1998).         [ Links ]
7. Huang, Y., "A Recursive Estimation Algorithm Using Selective Updating for Spectral Analysis and Adaptive Signal Processing", IEEE Transactions on Acoustics, Speech and Signal Processing, ASSP-34, 1331-1334, (1986).         [ Links ]
8. Luenberger, D., Optimization by Vector Space Methods, Wiley Professional Paperback Series, John Wiley and Sons (1969).         [ Links ]
9. Nagaraj, S., S. Gollamudi, S. Kapoor and Y. F. Huang "BEACON: An Adaptive Set-Membership Filtering Technique with Sparse Updates", IEEE Transactions on Signal Processing, 47, 2928-2941, (1999).         [ Links ]
10. Trusell, J. and M. R. Civanlar, "The Feasible Solution in Signal Restoration", IEEE Transactions on Acoustics, Speech and Signal Processing, 32, 201-212, (1984).         [ Links ]
11. Haykin, S., Adaptive Filter Theory. Prentice Hall. Fourth Edition, New Jersey, (2002).         [ Links ]
12. Yamada I., K. Slavakis and K. Yamada, "An Efficient Robust Adaptive Filtering Algorithm Based on Parallel Subgradient Projection Techniques", IEEE Transactions on Signal Processing, 50, 1091-1101, (2002).         [ Links ]
13. Yukawa, M. and I. Yamada, "Acceleration of Adaptive Parallel Projection Algorithms by Pairwise Optimal Weight Realization", EUSIPCO 2004, European Signal Processing Conference, Sept. 6-10, Austria, 713-716, (2004).
        [ Links ]

Received: September 21, 2005.
Accepted for publication: February 6, 2006.
Recommended by Guest Editors C. De Angelo, J. Figueroa, G. García and J. Solsona.