Projects

I noticed that I’ll have to find a representation of images as vectors, and not as matrices as usual. Because otherwise I won’t be able to describe linear operations as matrices on vectors. To be able to do things like an SVD or a look at eigenvalues or eigenvectors, I’ll have to evaluate matrices on vectors. For building lattices or Gabor systems I’ll have to try to use the algorithms which have been developed for 1D-signals. This leads me to Gabor Analysis on locally compact abelian groups, but group theory is a little bit away from ordinary signal analysis. Reading first intros, I asked myself:

70. What are compact groups? What are locally compact groups? What is a compactly generated group? What is an open compact subgroup?
    As an example, seems to be a compact group whenever A is real-valued invertible d×d. d=1 makes and a compact group.
71. What is a discrete (sub-)group? Why is the dual of a compact group discrete?
72. Why is for and a real-valued invertible d×d-matrix A?
    Maybe because for invertible square matrices and , and clearly .
73. How can I implement the tensor product of separable lattices to get non-separable lattices like the quincunx lattice or a hexagonal lattice?

I finally managed to scale the reconstructed images appropriately such that one can see at what locations certain 2D-frequencies occur. I FT’ed an image of a zebra and “windowed” the FT with a shifted Gaussian. Doing the inverse FT of that cutout yields a convolution of the input image with a modulated Gaussian, corresponding to

This is the zebra, a 480×480 pixel sample I clipped from an image I found in the Wikimedia Commons, and its FFT2:

Again, I rotated the image of the FT by 90° to match the orientation of the “jets” with the line patterns in the zebra. Clearly, the vertically oriented frequencies dominate the image. Now I window the FT-image by placing a Gaussian at the origin. This results in a low-pass filter. The IFT gives a reconstructed image which only contains the lowest frequencies:

The left half shows the (unmodulated) Gaussian with which the original image has been convolved. The right half shows the reconstructed image—the animal has lost all its zebra patterns! This effect is identical to a Gaussian blur.

I found out that the mentioned symmetry on the FFT2-picture occurs because the modulation in the input image only has a real part and no imaginary part. If one looks at fft2(real(Mg)) where Mg is a modulated Gaussian, one really gets two symmetrically shifted Gaussians as output.

And the Gaussian in the center still comes up because the line patterns don’t span completely between -1 (black) and 1 (white).

I also found a bump2d.m in the NuHAG toolbox which incorporates gaussnk.m, so I can trash my gauss2.m-attempt.

The modulated Gaussian I showed previously was just illustratory and did not consider mathematical properties such as FT-invariance. For that, it has to be periodized, normalized and centered at the borders. The MATLAB script gaussnk.m by N. Kaiblinger from the NuHAG toolbox implements this by evaluating the Gauss function on a wider interval and then wrapping it into the desired signal length. I tried to do that for the 2D case, but here I’m not sure about the term “signal length”, it should actually be the number of pixels. I wrote a first gauss2.m which creates a 2D-Gaussian on a square with almost all desired properties. However, the normalization seems to be wrong, as it does not fulfill , but at least it has and as desired. I have to become familiar with the discrete definition, I don’t know how the norming factors come up; maybe they consider the dimension d as a power.

What I was trying to do was doing localized FFT2’s on a test image. I shifted an arbitrarily selected Gaussian like a spotlight aver an image and did FFT2’s of that. However, on white walls the result didn’t show the desired FT-invariance of the Gaussian. So I had to think of the aforementioned normalizing tasks. Then I was confused that shifts of a Gaussian actually result in modulations on the FT-side. But shouldn’t we want to stay the FT all the same while shifting the Gaussian over a white wall in the input image (and having an unchanged Gaussian at every spot)? Yes, but only as long as we don’t want to be able to invert the FT! Here, we’d need the modulations to get back to the corresponding shifts on the input image. So, if only the spectra are interesting, one should look at the absolute values of the FT which will make the modulation factors disappear (because they have an absolute value of 1).

Another thing I came over was this; look at the following example image: On the left half is the input, and the right half shows the absolute values of the FFT2. The input almost looks like a modulated Gaussian. Why doesn’t it simply shift on the output image? Because in the input, the black values of the surroundings match the black values of the line pattern! If the input should represent a modulated Gaussian, the black lines should have value -1, and the surroundings a gray level of 0. I’ll correct this soon.

I like to rotate the output by 90° to match the orientation of the shifts with that of the lines in the input. Notice that I shifted the mid tones in the output image and that I always get another Gaussian in the center. This is the one I have to get rid of. Also, I get some kind of symmetry, the cause of which I haven’t found out yet but might also be due to the scaling issue.

For that, I visualized the TF- and FT-behavior of my 2D-Gaussian:

I tell a little about what I did with Octave during the last days. This is the first part and it's about creating a video of a modulated Gaussian. The second part is about doing FFT's of time-frequency shifted Gaussians.

I figured out how to merge images into a video stream. I rely on Linux tools rather than trying to do it from within Octave, because I don't know whether libraries for creating videos are somehow available there. However, I wrote an M-file which creates images of a modulated 2D-Gaussian. The 2D-frequency changes by going on a spiral in the complex plane starting from the origin. This yields a modulation which is rotating while the line-pattern gets narrower:

I merged the bunch of 900 images to a video by using MEncoder, the open-source encoder from the MPlayer suite. I did it this way:

I know, the usage of a proprietary container format is a contradiction to using open-source tools, but I only want to care about such details later.

I won't publish the M-file here, as it's too basic.

HGFei is leaving Austria in summer 2007 to go to Australia until Christmas 2007. I’d have to have my thesis finished by the end of June 2007. He asked me to give a schedule to be able to finish it by the end of the current term. Of course, regarding my scholarship, I should have finished it by July 2007 anyway. But I currently don’t know what the schedule could look like. I don’t know what my thesis could or should look like in the end, and therefore I don’t know what path to take to go there, and therefore I don’t really know my next steps. He said I shouldn’t be too ambitious and should produce a spate of demos now. But I don’t actually know what these should demonstrate. Maybe that kind of panic is somehow natural, but I’m not comfortable with that. Sure, I have time now for doing my thesis, and I really have to make use of it. I still have plenty of literature to read.

What I had done in February was starting to write the introductory chapter with LaTeX. That was a step what had to be done anyway, I can concentrate mostly on the content now. I wrote some pages about the foundations of TF-analysis, including some more side notes that are usually not mentioned in the other Master’s theses at my institute. As one of the next things I want to get familiar with the computing language of MATLAB/Octave. During the last days I did some experiments with FFT2, and I might write down some results here in the blog. Now I know how to do different things in Octave, I produced some pictures and rendered them to videos. This might provide some munition for doing 2D-Gabor Analysis.

Now I should look how to finally do image processing using Gabor Analysis methods. In our last conversation at the end of December 2006, HGFei told me some results on how to use the 1D-algorithms for higher dimensions, because their tools actually only provide algorithms for 1D-signals. I’d be the first to actually use that in the 2D-case. To understand how this is obtained I’d have to be familiar with the basics of GA on groups—a rather abstract approach. And mapping 1D to 2D implies that I can do it in the 1D case also. At least, if I can’t provide a schedule and don’t know how to be done by the end of this term, I know what my next steps are.

He also mentioned that I simply should do some basic experminents such as thresholding, i.e., doing reconstructions by leaving out coefficients with low contribution; this is a leering to the task of image compression.

Tomorrow, the summer term is starting, and I’ll attend HGFei’s project seminar on numerical harmonic analysis, where we’ll have to approach different problems using MATLAB.