Audible Art

What we do here is to let you point at some point in a picture and translate the color there into sound. If you want to know how that is done, it is described in the "How We Translate" part, below.

Abstract acrylics by Tanya Khachiyan. (The old version is here.)
Two Eccentric Buddhist Monks.
Artists' Corner – Make "music" from your own image or a photograph.

What is the Point?

Of course, a painting is meant to be looked at, not to be listened to using a scheme that the painter didn't even know about. The point of the sound is to add something.

First, sound points out certain interesting features of the painting – interesting sounds come out when the mouse moves over interesting things in the painting, at least certain kinds of interesting things. Light and color contrasts, for example, but also subtle shading in the background.

Sound also gives the viewer a way to interact with the painting, become more involved with it, and notice more things about it. When I start to play with the sound, I end up looking at a painting much longer and, I think, more deeply than if I just looked at it until I started to glaze and then moved on.

Note About Something Confusing

On some browsers, if you click the top margin, the audio will be muted. (I don't know why.) If you click again somewhere inside the browser window, the audio will be unmuted.

NEW!! Multi-Touch!!

You can now get sound from more than one place in a picture by touching it with more than one finger – if your touch screen supports that. (My laptop only supports two touches and cancels everything when I put down a third finger.)

Nothing is stopping you from using a mouse at the same time, but be aware that putting touching a finger will cancel what the mouse is doing. (This happens because of the way mouse and touch work, not because of how I have programmed things.)

Background

This is my second try at translating colors into sounds and seeing what happens when I do with the colors of an actual painting. My first try, also using Tanya Khachiyan's paintings, is at the "Old version" link, above.

The new method seems more musical to me and can produce results that fit a wider variety painting styles. But the old method does have more of a soaring strangeness to it.

How We Translate Color To Sound

We first decompose a color into three values using a well-known method.

Hue – the "color of the color", given as an angle from 0 to 360: Red-Yellow-Green-Cyan-Blue-Magenta-Red.
Saturation – the intensity of the color. The value 1 means "colorful color", 0 means "no color": white-grey-black.
A measure of brightness or luminance (luma).

There are several such systems. I have used HSL (Hue, Saturation, Luma) which is the most commonly used color model of this kind. Somehow it produces a sweeter sound than HSI (Hue, Saturation, Intensity), another common color model.

We use this representation this way:

Hue determines a tone or note within an octave.
Luma determines sort of fractional octave of the sound. This allows us to seem to raise the pitch of a sound without seeming to change the tone or note being produced.
Saturation determines how loud a sound is. Pure white, grey, or black are silent. Colors close to white will make a faint, high-pitched buzzing. Colors close to black will make a low grumbling sound, or possibly a rude noise.

There is a control button, described below, to switch the roles of luma and saturation.

Summary of Controls

When viewing a painting, a set of controls will appear on the right. I have given them initial settings that sound good to me, so you don't really need to mess with the controls, unless you want to do better, which you probably can.

At present, there is no way to save the settings that you set, so if you might want to use them again, write them down.

The initial settings will be restored if you press the Set Defaults button at the bottom of the control panel. If you Return to Menu and then select the same painting again, you will be back again at the initial settings.

From top to bottom, the control settings are:

The colored square at the top is the hue canvas. As you draw your mouse over it from left to right, the pitch will rise by an octave.
Lowest tone is the lowest note that can, theoretically, be produced with these settings. It should be produced if you put your mouse over the lower left corner of the hue canvas, but the color there is black, so no sound will be made.
If you draw the mouse over the hue canvas vertically from bottom to top, the pitch will seem to rise by the number of octaves set in Octaves, though the tone within an octave will not change.
Increasing the number of octaves gives a larger tonal range, which is good, but makes light colors shrill, which may not be so good.
Base hue indicates the hue at the left edge of the hue canvas. (At the right edge of Invert is checkded.
The hue at the right edge (left edge if Invert is checked) is the Hue Range value plus the base hue value (minus 360, if it is greater than that).
If you choose hue base and range so that the colors in the painting, and few others, appear on the hue canvas, the painting will produce a wider tonal range, which makes it acoustically more interesting.
But note that 1) hues that are not within the range will not make any sound, and that 2) colors that are close to black or white may have hues that you do not perceive. Check for hearly black and nearly white areas that seem too silent.
Checking Invert reverses the left-to-right order of hues in the hue canvas. I find it less confusing to uncheck invert when playing with base hue and hue range.
Luma min/max give a restricted range for lightness. Colors with lightness outside the range will be silent.
I added this mainly for paintings with a light-colored background, to make the background silent instead of letting it make a high-pitched hum.
"Sat = Volume Luma = Pitch" button. This is a toggle button so you can control whether saturation is volume and luma is pitch or vice versa.
Round to semi rounds each tone produced to the geometrically nearest semitone. This makes images with relatively structured colors sound more musical.
Pressing Set Defauilts restores the initial settings for this painting.
Pressing Return to Menu returns to the menu view from which this painting was reached.
HSI shows the current hue-saturation-intensity (luminence) value for the color under the mouse pointer.