What is a spectrogram?

A spectrogram is a visual representation of the frequency content of a signal. A spetrogram shows how the quantity of energy in different frequency regions varies as a function of time. On a spectrogram, the signal is divided into many small time sections and each section is analysed in terms of what frequency components are present in the section. This analysis is called spectral analysis because the spectrum of each section is calculated and the quantity of each frequency component (that is each sinusoid) is measured from the spectrum. The quantity of each component is then converted to a grey level in which (normally) low energy components are converted to a white colour, while high energy components are converted to a black colour. These colours are then plotted on a vertical strip corresponding to the time at which the original signal segment occurred. The height of the coloured element on this vertical strip represents the frequency of the component.

Thus a spectrogram is a 3-dimensional analysis of a signal, the horizontal dimension is time, the vertical dimension is frequency, and the grey-scale shows the amount of energy occurring in the signal at each time and frequency.

What is the difference between a wide-band and a narrow-band spectrogram?

From the description of a spectrogram above, you will see that one part of the analysis process involves dividing the signal up into sections so that each section can be spectrally analysed. What we didn't say is how big these sections should be. Clearly if we chose very large sections, say of half a second, we wouldn't be able to see much of what is going on in a speech signal - each half second chunk of the picture would be a static set of colours. We know that speech signals change rapidly and we want to choose relatively small sections so that we can see the spectral content of the speech changing from moment to moment.

There is a problem however, as we make the sections smaller and smaller, it becomes more and more difficult to determine precisely which frequency components are present in the signal. You can see this in a qualitative way: if you only look at a small fraction of a single cycle of a sinusoid, it is very difficult to estimate the frequency: you can guess the duration of a whole cycle, but you might be considerably in error. In the same way, the smaller the section of signal we put into spectral analysis, the poorer the estimates of the frequency components it contains.

So we need to come to some kind of compromise: we want sections short enough to see the temporal detail in the speech signal, but long enough to see the frequency detail too. It turns out that there is not single best compromise value for speech signals. If we choose sections long enough that we can see the individual harmonics of larynx vibration, then these sections will be too long to see the time response of the vocal tract resonances (formants) as they respond to an excitation pulse. However, if you need to see source harmonics, then you need sections of this length. This is called a "narrow-band" spectrogram, and the sections are about 20ms long. On the other hand, if you want to see the detailed formant vibrations that occur within the larynx cycle, you need to use sections smaller than a singhle glottal cycle (which is about 5ms for women), so sections of about 3ms are commonly used. These short sections give us bettern temporal detail but poorer frequency detail. This is why the analysis is called a "wide-band" spectrogram. Wide-band analysis is most useful for finding formant frequencies.

What should I look for in a speech spectrogram?

If you study a wide-band spectrogram of say a couple of words of speech you should be able to see some of the following events:

• Larynx excitation pulses: these appear as vertical dark lines at intervals of between 5 and 10ms or so. These are also called striations. Each one of these is caused by the sudden pressure change that arises above the larynx when the vocal folds close suddenly, cutting off the flow of air from the lungs. This change is so sudden it creates a kind of pressure "pulse" which contains energy at a wide range of frequencies, commonly up to 4kHz or more.
• Formant vibrations: between the striations you will see dark regions which only occur at particular frequencies. These regions, which often appear several hundred Hz across because of the limitations of the wide-band frequency analysis of the spectrograph, are caused by the ringing of the vocal tract resonances (or formants) as each pulse from the larynx excites them. If you look carefully, you may see that these vibrations are larger (darker) just after the pulse, and get paler as the energy in the vibrations is lost from the vocal tract.
• Changes in formant frequency: You should see that the dark regions caused by formant vibration change in frequency through the utterance. You may see that the resonances have a kind of continuity in time, and slowly rise and fall in frequency through a syllable, and from one syllable to the next. These slow and smooth changes in formant frequencies are because the frequencies of the vocal tract resonances are set by the shape of the vocal tract tube, which in turn is controlled by the position of the articulators. Since the articulators move relatively slowly (a few syllables per second) the formant frequencies appear to move slowly too.
• Turbulent sounds: In regions where there is no larynx vibration and hence no striations you should see some "speckled" rather noisy unstructured regions of dark colour, often towards the high frequency end of the picture. These are "noise" sounds caused by turbulence in the vocal tract, for instance: bursts, aspiration and frication. Bursts are often short vertical bars, a lot like a striation, caused by the sudden pressure change in the vocal tract when a stop articulation is released. Aspiration is turbulence that occurs in the larynx, caused by a narrowing of the airway from the lungs made by the vocal folds coming close together. Frication is turblence that occurs at other points of narrowing in the vocal tract, made with the tongue or the lips. If you look carefully you may see differences in the frequency content of bursts or fricatives originating from different places of articulation. This is because the different articulator configurations shape the sound generated by the turbulence in different ways depending on the size and shape of the vocal tract tube in front of the constriction.