Universal Measurement Matrix Design for Sparse and Co-Sparse Signal Recovery

Article History:Received:11 november 2020; Accepted: 27 December 2020; Published online: 05 April 2021 ABSTRACT : Compressed Sensing (CS) avails mutual coherence metric to choose the measurement matrix that is incoherent with dictionary matrix. Random measurement matrices are incoherent with any dictionary, but their highly uncertain elements necessitate large storage and make hardware realization difficult. In this paper deterministic matrices are employed which greatly reduce memory space and computational complexity. To avoid the randomness completely, deterministic sub-sampling is done by choosing rows deterministically rather than randomly, so that matrix can be regenerated during reconstruction without storing it. Also matrices are generated by orthonormalization, which makes them highly incoherent with any dictionary basis. Random matrices like Gaussian, Bernoulli, semi-deterministic matrices like Toeplitz, Circulant and full-deterministic matrices like DFT, DCT, FZC-Circulant are compared. DFT matrix is found to be effective in terms of recovery error and recovery time for all the cases of signal sparsity and is applicable for signals that are sparse in any basis, hence universal.


Introduction
CS is the technique which allows sub-Nyquist sampling of sparse signals. Signal may be inherently sparse in co-ordinate basis or it can be made sparse (known as co-sparse signal) by applying a transform (dictionary) in orthonormal basis or non-orthonormal basis (redundant dictionary). CS enables the reconstruction of a sparse signal from far fewer measurements [1]. The two conditions for which recovery is possible in CS are: 1. Sparsity, which requires the signal to be sparse in some domain. 2. Incoherence, which is a sufficient condition for sparse signals.
In CS survey paper [2], different sufficient and necessary conditions like mutual coherence, Restricted Isometry Property (RIP), Null Space Property (NSP) for exact recovery of the inherent sparse signal are given. Authors in [3], [4] introduced conditions for stable recovery in terms of D-RIP and D-NSP for dictionary sparse signals, but those are computationally very hard and do not provide any criteria to compare and choose a good measurement matrix. In [5] mutual coherence between rows of N×N orthonormal measurement matrix U and columns of dictionary matrix Ψ is used as performance measurement metric. This can be effectively verified for sparse signals in any domain, hence used in this work.
Random matrices have low coherence with any dictionary [6], but as these are unstructured their elements are highly uncertain which require huge memory and are costly to implement in hardware. Semi-deterministic matrices like Toeplitz and Circulant [7] have reduced randomness as they have only first row to be random. In [8] this first row is replaced with polyphase Frank-Zadoff-Chu (FZC) sequence which causes the measurement matrix to be full deterministic. Even though computational complexity reduces, huge memory requirement still persists, as they used random sub-sampling which necessitates storage of the entire measurement matrix for reconstruction.
In this paper deterministic sub-sampling is done which eliminates randomness completely so that matrix need not be stored for reconstruction purpose. Also measurement matrix is orthonormalized so that it becomes incoherent with any dictionary. The 3 cases of signal sparsity i.e. when signal is sparse in co-ordinate basis, orthonormal basis and redundant dictionary [9], is used to study the effect of mutual coherence on signal recovery and also performance of different measurement matrices (both random and deterministic) shown in Fig.1 is evaluated in terms of recovery error and recovery time. Hadamard and other deterministic matrices like chirp sensing codes [10] are not considered in this paper as they have restriction on length of the input signal. Since both Toeplitz and Circulant matrices give similar performance only one i.e Circulant matrix is considered for comparison.

Fig.1. Measurement matrices 2 Methodology
In CS, far few measurements of signal x are taken by multiplying with a measurement matrix Φ which is an m×n matrix and m<<n. Reconstruction is seeking a solution to the under-determined linear equation z=Φx, x is the sparsity coordinates and z is the compressed coordinates. The sparse representation of the signal x can be reconstructed by solving the minimization problem in Eq. (1), known as Basis Pursuit (BP): When the signal is sparse in orthonormal basis or non-orthonormal basis, signal x becomes sparse when multiplied by the dictionary matrix Ψ, i.e. y=Ψx makes the signal sparse. Therefore when we take m measurements of the signal x as z= Φx, and x=Ψ -1 y which gives z=ΦΨ -1 y =Ay is inferred, where A=ΦΨ -1 . Matrix A and z values are now fed to the optimization algorithm in Eq. (1) for obtaining the sparse representation y and later by finding x=Ψ -1 y, signal x can be reconstructed. CS recovery process is shown in Fig.2. (2) Where x is the actual signal and x̂ is the reconstructed signal.

Generation of Measurement Matrix (Φ)
Ortho-normalizing the measurement matrix, makes it to be mutually incoherent with any dictionary Ψ, thus recovery is possible with high probability. Whether the signal is inherently sparse in the time domain or sparse in some other basis, exact signal reconstruction can be achieved by orthonormalizing Φ. Otherwise if the signal is sparse in DFT or DCT domain, semi deterministic measurement matrices like Toeplitz, Circulant and full deterministic matrices like DFT, DCT cannot be applied since these are highly coherent with DFT or DCT dictionary. Generally images are sparse in DCT domain. In such case choice of Toeplitz, Circulant, DFT, DCT measurement matrices would be inappropriate. Therefore to make the measurement matrix universal it has to be orthonormalized before applied to any signal that makes it highly incoherent with any dictionary basis and thus leads to perfect reconstruction. As shown in Fig.3, N×N measurement matrix U is first orthonormalized and then M rows out of it are selected that gives Φ of order M×N.

Deterministic Sub-sampling
To eliminate storage of matrix Φ for usage in the reconstruction process, M rows of U are selected deterministically rather than randomly. Not only selection of random rows could effectively reconstruct the signal, deterministic selection of rows, by chosing first M rows of U, could also effectively reconstruct the signal with DFT matrix. Polyphase deterministic matrices could effectively reconstruct only when the rows are generated by 3 rd order polynomial function, whereas DFT matrix could effectively reconstruct either with selection of first M rows or 1 st /2 nd /3 rd order polynomials. A polynomial function generates sampling values deterministically. The rows determined by those sample values are selected. This process when implemented in hardware is known as deterministic sub-sampling as shown in Fig 4.

Fig.4. Signal Acquisition
Practically input sparse/co-sparse signal is multiplied with orthonormalized measurement matrix using a matrix multiplier followed by an Analog to Information Converter (AIC) which measures only M rows at sample times determined by polynomial function or first M rows. The compressed measurements can be stored or processed to the receiver along with reconstruction matrix as shown in Fig.5.

Fig.5. Signal Reconstruction 2.3 Mutual Coherence
The choice of measurement matrix for better recovery of sparse or co-sparse signal with both incoherent and highly coherent dictionaries is verified based on mutual coherence between rows of Φ and columns of Ψ as given in Eq. (3), where U is orthonormal N×N matrix of which Φ is obtained by selecting M rows of U.
When the signal is sparse in co-ordinate basis, N×N identity matrix is chosen as dictionary matrix. CS algorithms allow perfect reconstruction since µ(U,Ψ) gives a very low value closer to 1 for any measurement matrix chosen. Therefore reconstruction of co-sparse signal is perfect even when the dictionary is redundant or has high correlations.

Results and Discussion
Matlab simulations are performed for the 3 cases of signal sparsity with deterministic sub-sampling of first M rows. BP optimization algorithm of SPGL1 package [11] is exploited for signal recovery from compressed measurements. The input signal x is chosen with N=848 samples, sparsity K=4 and number of measurements M=65.

Case 1: Sparsity in co-ordinate basis
A signal x that is sparse in time-domain is chosen with K randomly spread peaks. The Fig.6 above shows the actual signal with K=4 peaks and bottom one the reconstructed signal with DFT measurement matrix. The Fig.7 and Fig.8 below show recovery error and recovery time with respect to variation in sparsity and number of measurements for different measurement matrices.  Mutual coherence µ(U,Ψ) for different choice of measurement matrices is independent of sparsity K and number of measurements M. It is least for DFT, Bernoulli and FZC paired circulant matrices. Maximum value of mutual coherence is √N=29.12. There is no linear relation between mutual coherence and recovery error. But a low value of mutual coherence closer to 1 signifies possible recovery of signal from the compressed measurements. Recovery error and recovery time are low for DFT matrix for M=65 measurements and sparsity K=4 as shown in Table I.

Case 2: Sparsity in orthonormal basis
A signal x that has K randomly spread sinusoids is chosen. Then signal will be sparse in frequency domain. Therefore dictionary matrix (Ψ) can be DFT matrix so that y=Ψx will become sparse. Time domain signal x is measured with very few samples using DFT measurement matrix (Φ), as z=Φx. As discussed before A=ΦΨ -1 and z are fed to BP algorithm, to recover frequency domain signal y and then x can be reconstructed back by doing x=Ψ -1 y, shown in Fig.9.

Fig.9. Sparsity in orthonormal basis
It can be seen from the above figure that perfect recovery is possible even when both measurement matrix and dictionary matrix are of DFT, as the measurement matrix is orthonormalized. Orthonormalization makes the measurement matrix incoherent with any dictionary. Most of the natural images are sparse in DCT domain. Once the measurement matrix is orthonormalized, these can also be applied to signals that are sparse in DCT domain.   Fig.11 show nearly similar recovery performance with all the measurement matrices. Table II indicates that mutual coherence value is closer to 1 and nearly equal for all the matrices. All the orthonormalized measurement matrices are highly incoherent with DFT dictionary. From the above figures it can be interpreted that optimum values of recovery error and recovery time are obtained with DFT matrix.

Case 3: Sparsity in Redundant Dictionary
A radar LFM echo signal that is sparse in waveform matched dictionary is chosen.

Fig.12. Sparsity in redundant dictionary
CS radar that could accurately detect location of targets with very few measurements is shown in Fig.12. Here signal reconstruction is not necessary as the requisite information of the targets can be known from sparse approximation itself. DFT matrix shows optimum performance above all in terms of recovery error and time that can be seen from Fig.13 and Fig.14. The waveform matched redundant dictionary has columns that are highly coherent, but still recovery is possible since µ(U,Ψ) is a low value closer to 1, as can be seen in Table III for different measurement matrices by orthonormalizing the measurement matrix.

Conclusion
In this paper, performance of different measurement matrices is evaluated in terms of recovery error and recovery time. Mutual coherence performance metric is evaluated for the 3 cases of signal sparsity. Mutual coherence does not follow a linear relationship with recovery error, but still low value of coherence µ(U, Ψ) guarantees perfect signal recovery. Orthonormalization of measurement matrices make them to be highly incoherent with any dictionary, which causes the sparse signal to be exactly recovered from the compressed measurements. With random measurement matrices, when length of the signal is large, size of the matrix would be very high, that need to be stored for reconstruction. Deterministic matrices with deterministic sub-sampling avoid randomness totally so that matrix need not be stored. When compared to deterministic FZC-Circulant matrix with deterministic sub-sampling, proposed deterministic DFT matrix with deterministic sub-sampling is found to be effective in terms of recovery error and recovery time. Also it is universal as it is orthonormalized, it is applicable with any dictionary basis. As randomness is completely eliminated memory space and computational complexity are greatly reduced.