SourceForge Logo

Animation of swimming fish

TunaFish

"You can tune a piano, but you can't tune a fish"

What's TunaFish?

TunaFish does non-realtime pitch correction to monophonic .wav files, hopefully without introducing audible artifacts.

For example, if you have a monophonic recording of a singer that's off-pitch in places, you can use TunaFish to correct the errors and put the singer back in tune. TunaFish should work "cleanly", so the average listener can't tell it's been altered.

TunaFish is written written in Java, so it runs on Windows, Linux and OS X.

What Can't TunaFish Do?

Task List


Currently TunaFish is in the early code stage, so it's not really usable. Here's a task list of things I expect need to be done before TunaFish is usable, and their current status:

Finished

  1. Create a trivial Swing window for TunaFish with pull-down menus.
  2. Load monophonic .wav files
  3. Playback the .wav file
  4. Trivially modify the .wav file (ex: change the volume or pitch)
  5. Playback the modified .wav file
  6. Save the modified .wav file
  7. Display the waveform in a scrollable window
  8. Perform pitch analysis
  9. Implement retune speed logic
  10. Write cubic spline resampling

In Process

  1. Zoom in and out of the wave
  2. Add a timeline for playback.
  3. Display pitch analysis

Pending

  1. Perform pitch correction
  2. Make the user interface interactive (i.e.: the user can select notes to adjust)
  3. Clean up the user interface and make the application usable
  4. Write documentation

Technological Overview

Determining the Pitch

Before TunaFish can correct the pitch, it first has to determine what pitch is being sung at a particular point in time. Common approaches include:
  1. Zero-crossing applies a bandpass filter to remove harmonics higher than the fundamental, and then searches for sign changes (where the wave moves up instead of down, or down instead of up) and measures the distance between the waves. Because wave shapes are complex and change over time, this method tends to be the most error-prone.
  2. Autocorrelationtries to find the signal phase by "sliding" the target wave forward until it comes into phase with another segment of the wave. The distance between the start and the point of alignment is the frequency of that wave.
  3. Fourier Analysis analyzes the wave for harmonic content. This is quite popular, but it requires an understanding of FFT of DFT. One of these days I'll get around to learning FFT, but probably not for this project.
TunaFish uses autocorrelation, mainly because it give excellent results. It's also trivial to implement.

Changing the Pitch

Once TunaFish has determined where the pitch should be, it becomes necessary to resynthesize a new wave at the new frequency. Again, there are a number of available approaches, including:
  1. Fourier analysis can be used to determine the harmonic analysis of the wave, and recreate the wave at a new frequency. It also has a tendancy to introduce "smearing" and "pre-echo" audio artifacts into the sound, which makes it less than an ideal candidate.
  2. PSOLA (Pitch-Synchronous Overlap Add) reuses cycles from the original wave to recreate the wave by spacing the cycles at the new frequency. Pitch is shifted without changing the characteristics of the sound (i.e.: the spacing of the formants is preserved), but it introduces a distinct "chorus" effect, which make it obvious the sound has been processed.
  3. Resample the original samples at the new frequency. Unlike fourier analysis and PSOLA, it alters all the frequencies in the sound, instead of only altering the fundamental frequency, which at the extreme range can lead to the singer sounding like a chipmunk.
The primary argument for using a more complex method (i.e.: fourier analysis or PSOLA) is to properly handle formant correction. However, because pitches shouldn't be moved by more than a semitone or so, this shouldn't be much of an issue.

Since resampling introduces the least number of audible artifacts, it's the preferred method for changing the pitch.

Finding the Target Pitch

A naive approach to pitch correction would be to have TunaFish clamp all out-of-tune pitches to their correct pitch. Unfortunately, this leads to the so-called Cher effect (named after the use of that effect in the song "Believe").

Unlike a musical instrument like a flute or violin, a singer can't just leap directly from one pitch to the next. Instead, their voice "slides" from one pitch to the next.

Further, expressive effects such as vibrato (which intentionally move the pitch above and below the pitch center) would be lost if TunaFish clamped pitches to set frequencies.

The simplest solution is to implement a retune speed parameter, which determines how quickly TunaFish should pull an errant pitch back to the correct pitch.

Questions?

If you've got any questions about TunaFish, feel free to drop me an email.