Audible Art
What we do here is to let you point at some point in a picture and
translate the color there into sound. If you want to know how
that is done, it is described in the "How We Translate" part, below.
What is the Point?
Of course, a painting is meant to be looked at, not to be
listened to using a scheme that the painter didn't even know
about. The point of the sound is to add something.
First, sound points out certain interesting features of the
painting – interesting sounds come out when the mouse
moves over interesting things in the painting, at least certain
kinds of interesting things. Light and color contrasts, for
example, but also subtle shading in the background.
Sound also gives the viewer a way to interact with the
painting, become more involved with it, and notice more things
about it. When I start to play with the sound, I end up looking
at a painting much longer and, I think, more deeply than if I
just looked at it until I started to glaze and then moved on.
Note About Something Confusing
On some browsers, if you click the top margin, the audio will be
muted. (I don't know why.) If you click again somewhere inside
the browser window, the audio will be unmuted.
NEW!! Multi-Touch!!
You can now get sound from more than one place in a picture by
touching it with more than one finger – if your touch screen
supports that. (My laptop only supports two touches and cancels
everything when I put down a third finger.)
Nothing is stopping you from using a mouse at the same time,
but be aware that putting touching a finger will cancel what the
mouse is doing. (This happens because of the way mouse and touch
work, not because of how I have programmed things.)
Background
This is my second try at translating colors into sounds and
seeing what happens when I do with the colors of an actual
painting. My first try, also using Tanya Khachiyan's paintings,
is at the "Old version" link, above.
The new method seems more musical to me and can produce results
that fit a wider variety painting styles. But the old method
does have more of a soaring strangeness to it.
How We Translate Color To Sound
We first decompose a color into three values using a well-known method.
-
Hue – the "color of the color", given as an angle from 0
to 360: Red-Yellow-Green-Cyan-Blue-Magenta-Red.
-
Saturation – the intensity of the color. The value 1
means "colorful color", 0 means "no color": white-grey-black.
-
A measure of brightness or luminance (luma).
There are several such systems. I have used HSL (Hue, Saturation,
Luma) which is the most commonly used color model of this kind. Somehow it produces
a sweeter sound than HSI (Hue, Saturation, Intensity), another common color model.
We use this representation this way:
- Hue determines a tone or note within an octave.
- Luma determines sort of fractional
octave of the sound. This allows us to seem to raise the
pitch of a sound without seeming to change the tone or note
being produced.
- Saturation determines how loud a sound is. Pure white,
grey, or black are silent. Colors close to white will make
a faint, high-pitched buzzing. Colors close to black will
make a low grumbling sound, or possibly a rude noise.
There is a control button, described below, to switch the roles
of luma and saturation.
Summary of Controls
When viewing a painting, a set of controls will appear on the
right. I have given them initial settings that sound good to me,
so you don't really need to mess with the controls, unless you
want to do better, which you probably can.
At present, there is no way to save the settings that you set,
so if you might want to use them again, write them down.
The initial settings will be restored if you press
the Set Defaults button at the bottom of the control panel.
If you Return to Menu and then select the same painting again,
you will be back again at the initial settings.
From top to bottom, the control settings are:
-
The colored square at the top is the hue canvas. As
you draw your mouse over it from left to right, the pitch will
rise by an octave.
-
Lowest tone is the lowest note that can, theoretically,
be produced with these settings. It should be produced
if you put your mouse over the lower left corner of the hue
canvas, but the color there is black, so no sound will be
made.
-
If you draw the mouse over the hue canvas vertically from
bottom to top, the pitch will seem to rise by the number of
octaves set in Octaves, though the tone within an
octave will not change.
Increasing the number of octaves gives a larger tonal range,
which is good,
but makes light colors shrill, which may not be so good.
-
Base hue indicates the hue at the left edge of the hue
canvas. (At the right edge of Invert is checkded.
-
The hue at the right edge (left edge if Invert is
checked) is the Hue Range value plus the base hue value
(minus 360, if it is greater than that).
If you choose hue base and range so that the colors in the
painting, and few others, appear on the hue canvas, the
painting will produce a wider tonal range, which makes it
acoustically more interesting.
But note that 1) hues that are not within the range will not
make any sound, and that 2) colors that are close to black or
white may have hues that you do not perceive. Check for
hearly black and nearly white areas that seem too silent.
-
Checking Invert reverses the left-to-right order of hues in the
hue canvas. I find it less confusing to uncheck invert when
playing with base hue and hue range.
-
Luma min/max give a restricted range for lightness.
Colors with lightness outside the range will be silent.
I added this mainly for paintings with a light-colored
background, to make the background silent instead of letting
it make a high-pitched hum.
-
"Sat = Volume Luma = Pitch" button. This is
a toggle button so you can control whether saturation is
volume and luma is pitch or vice versa.
-
Round to semi rounds each tone produced to the
geometrically nearest semitone. This makes images with
relatively structured colors sound more musical.
- Pressing Set Defauilts restores the initial
settings for this painting.
-
Pressing Return to Menu returns to the menu view from
which this painting was reached.
-
HSI shows the current hue-saturation-intensity
(luminence) value for the color under the mouse pointer.