← Projects
RGB MIDI Controller
Spring 2026 · ENAS 3440 Musical Acoustics and Instrument Design
A MIDI controller with controls programmed onto X-Y position on an RGB gradient. Timbre is controlled by breath, and X-Y controls are arbitrary. A color sensor reads an RGB pattern printed on the inner pipe to tell where your hand is, and turns it into MIDI.

A 3D model of the instrument I made :-)
Details
Date
Spring 2026
Course
ENAS 344 Musical Acoustics and Instrument Design, Yale
Instructors
Konrad Kaczmarek, Larry Wilen
Parts
Teensy 4.1 with Audio Shield, TCS34725 color sensor, HX710B pressure sensor, 128×64 OLED, PVC pipe, 3D-printed mounts
Tools
Onshape, Processing, Arduino, Logic Pro, Audacity
Overview


Color sensor (purple) on the outer pipe. Teensy 4.1 with audio board, OLED and breath sensor on the board.
This was my final project for ENAS 344 Musical Acoustics and Instrument Design, taught by Konrad Kaczmarek and Larry Wilen. As a woodwind and EWI player, I wanted to explore ways to add additional controls to the EWI, and explored designing a two-bodied controller, where the right-hand/lower body could be twisted and extended for additional controls. The rotation and extension are both tracked by a color sensor reading a RGB gradient inside the tube, and an inverse matrix decodes the color back into X/Y position. A pressure sensor measures amplitude, and an OLED displays position.
On using color
To track the position of the right hand body, I originally tested using a time-of-flight sensor, but this proved to not be as accurate as required due to the bodily movement of the player. After testing a few sensors, I eventually landed on using an RGB gradient with an RGB sensor to track fixed position granularly.
It turns out relations between color and sound have been theorized for centuries! Scriabin scored colored light into Prometheus: The Poem of Fire (performed at Yale!), and Kandinsky wrote about color as if it were sound. They both offer alternative ways to imagine music, through color — otherwise interpreted as relational waveforms meshed through symbolic logic. Of course, in this project, the relations are arbitrary, since our perception of light waveforms are, but I found kinship with my work with Woodshed in imagining new possibilities for notation, this time through color.
For novice musicians, color registers as a readily understood, visual instrument to interact with music, and encourages playfulness.

Testing out color gradient mixtures
Pure RGB
However, the above gradients proved to be inconsistent because most printers mix cyan, magenta, yellow, and black ink, and each printer has proprietary dithering and halftoning that makes granular controllability unpredictable. In Processing, I then adopted a modulo algorithm where every pixel in the final pattern is pure red, green or blue. The pattern is divided into small cells. Across the sheet, each cell’s share of green grows in one direction and its split between red and blue shifts in the other. The sensor sees a whole patch of pixels at once, so it reads how much red, green and blue the patch holds. The Teensy turns that into two numbers: x for pull and y for twist. I rendered cells at 128, 96, 64 and 32 pixels and printed the 32-pixel one.


The same rule with 96-pixel cells (left) and 32-pixel cells (right), the one in the instrument.
Gradients
Writing the patterns in Processing (built on top of Java) instead of drawing them gave me granular control of color. I first experimented with writing different smooth, linear gradients, all a mixture of RGB — first in one direction (red x to green y), then in two (red x to green y to blue z). A printer mixes cyan, magenta and yellow ink, not red, green and blue light, so calibrations had to be done manually.



Mechanism

Pull, twist
plus breath
→
RGB pattern
on the inner pipe
→
Color sensor
reads R, G, B
→
Inverse matrix
un-mixes the reading
→
Position
how far pulled, twisted
→
MIDI
into Logic instruments
Videos
Clips from the build and different playing modes
Linear sawtooth wave, rotation is timbre
pull: pitch · twist: filter
V7 organ
pull: root note · twist: adds chord notes, up to a dominant seventh
V7 arpeggios
Mapped onto drums
4-part sample
Guitar frets
pull: fret · twist: string
Isfahan ( an attempt at translating a piano roll to RGB)
twist: phrase · pull: notes
My friend Shomari playing with the instrument for the first time
Prototyping
What I learned
Before this class, I had little experience with hardware. I learned engineering CAD through OnShape, basic electronics and soldering, and a robust understanding of the physical properties of music as a factor of materials and waveforms.
Outcome
I loved this process! It was a great elective during my last semester, and I’ve been tinkering with music hardware and electronics more. Please reach out if you want to build some instruments together!I view the RGB controller as just a prototype, and would love to expand on it with more advanced color sensor capabilities, custom transparent structures to take the guesswork out of position and for users to see the registered color, and to eventually build a two-handed EWI.
Next
All projects →
← Projects
RGB MIDI Controller
Spring 2026 · ENAS 3440 Musical Acoustics and Instrument Design
A MIDI controller with controls programmed onto X-Y position on an RGB gradient. Timbre is controlled by breath, and X-Y controls are arbitrary. A color sensor reads an RGB pattern printed on the inner pipe to tell where your hand is, and turns it into MIDI.

A 3D model of the instrument I made :-)
Details
Date
Spring 2026
Course
ENAS 344 Musical Acoustics and Instrument Design, Yale
Instructors
Konrad Kaczmarek, Larry Wilen
Parts
Teensy 4.1 with Audio Shield, TCS34725 color sensor, HX710B pressure sensor, 128×64 OLED, PVC pipe, 3D-printed mounts
Tools
Onshape, Processing, Arduino, Logic Pro, Audacity
Overview


Color sensor (purple) on the outer pipe. Teensy 4.1 with audio board, OLED and breath sensor on the board.
This was my final project for ENAS 344 Musical Acoustics and Instrument Design, taught by Konrad Kaczmarek and Larry Wilen. As a woodwind and EWI player, I wanted to explore ways to add additional controls to the EWI, and explored designing a two-bodied controller, where the right-hand/lower body could be twisted and extended for additional controls. The rotation and extension are both tracked by a color sensor reading a RGB gradient inside the tube, and an inverse matrix decodes the color back into X/Y position. A pressure sensor measures amplitude, and an OLED displays position.
On using color
To track the position of the right hand body, I originally tested using a time-of-flight sensor, but this proved to not be as accurate as required due to the bodily movement of the player. After testing a few sensors, I eventually landed on using an RGB gradient with an RGB sensor to track fixed position granularly.
It turns out relations between color and sound have been theorized for centuries! Scriabin scored colored light into Prometheus: The Poem of Fire (performed at Yale!), and Kandinsky wrote about color as if it were sound. They both offer alternative ways to imagine music, through color — otherwise interpreted as relational waveforms meshed through symbolic logic. Of course, in this project, the relations are arbitrary, since our perception of light waveforms are, but I found kinship with my work with Woodshed in imagining new possibilities for notation, this time through color.
For novice musicians, color registers as a readily understood, visual instrument to interact with music, and encourages playfulness.

Testing out color gradient mixtures
Pure RGB
However, the above gradients proved to be inconsistent because most printers mix cyan, magenta, yellow, and black ink, and each printer has proprietary dithering and halftoning that makes granular controllability unpredictable. In Processing, I then adopted a modulo algorithm where every pixel in the final pattern is pure red, green or blue. The pattern is divided into small cells. Across the sheet, each cell’s share of green grows in one direction and its split between red and blue shifts in the other. The sensor sees a whole patch of pixels at once, so it reads how much red, green and blue the patch holds. The Teensy turns that into two numbers: x for pull and y for twist. I rendered cells at 128, 96, 64 and 32 pixels and printed the 32-pixel one.


The same rule with 96-pixel cells (left) and 32-pixel cells (right), the one in the instrument.
Gradients
Writing the patterns in Processing (built on top of Java) instead of drawing them gave me granular control of color. I first experimented with writing different smooth, linear gradients, all a mixture of RGB — first in one direction (red x to green y), then in two (red x to green y to blue z). A printer mixes cyan, magenta and yellow ink, not red, green and blue light, so calibrations had to be done manually.



Mechanism

Pull, twist
plus breath
→
RGB pattern
on the inner pipe
→
Color sensor
reads R, G, B
→
Inverse matrix
un-mixes the reading
→
Position
how far pulled, twisted
→
MIDI
into Logic instruments
Videos
Clips from the build and different playing modes
Linear sawtooth wave, rotation is timbre
pull: pitch · twist: filter
V7 organ
pull: root note · twist: adds chord notes, up to a dominant seventh
V7 arpeggios
Mapped onto drums
4-part sample
Guitar frets
pull: fret · twist: string
Isfahan ( an attempt at translating a piano roll to RGB)
twist: phrase · pull: notes
My friend Shomari playing with the instrument for the first time
Prototyping
What I learned
Before this class, I had little experience with hardware. I learned engineering CAD through OnShape, basic electronics and soldering, and a robust understanding of the physical properties of music as a factor of materials and waveforms.
Outcome
I loved this process! It was a great elective during my last semester, and I’ve been tinkering with music hardware and electronics more. Please reach out if you want to build some instruments together!I view the RGB controller as just a prototype, and would love to expand on it with more advanced color sensor capabilities, custom transparent structures to take the guesswork out of position and for users to see the registered color, and to eventually build a two-handed EWI.
Next
All projects →
← Projects
RGB MIDI Controller
Spring 2026 · ENAS 3440 Musical Acoustics and Instrument Design
A MIDI controller with controls programmed onto X-Y position on an RGB gradient. Timbre is controlled by breath, and X-Y controls are arbitrary. A color sensor reads an RGB pattern printed on the inner pipe to tell where your hand is, and turns it into MIDI.

A 3D model of the instrument I made :-)
Details
Date
Spring 2026
Course
ENAS 344 Musical Acoustics and Instrument Design, Yale
Instructors
Konrad Kaczmarek, Larry Wilen
Parts
Teensy 4.1 with Audio Shield, TCS34725 color sensor, HX710B pressure sensor, 128×64 OLED, PVC pipe, 3D-printed mounts
Tools
Onshape, Processing, Arduino, Logic Pro, Audacity
Overview


Color sensor (purple) on the outer pipe. Teensy 4.1 with audio board, OLED and breath sensor on the board.
This was my final project for ENAS 344 Musical Acoustics and Instrument Design, taught by Konrad Kaczmarek and Larry Wilen. As a woodwind and EWI player, I wanted to explore ways to add additional controls to the EWI, and explored designing a two-bodied controller, where the right-hand/lower body could be twisted and extended for additional controls. The rotation and extension are both tracked by a color sensor reading a RGB gradient inside the tube, and an inverse matrix decodes the color back into X/Y position. A pressure sensor measures amplitude, and an OLED displays position.
On using color
To track the position of the right hand body, I originally tested using a time-of-flight sensor, but this proved to not be as accurate as required due to the bodily movement of the player. After testing a few sensors, I eventually landed on using an RGB gradient with an RGB sensor to track fixed position granularly.
It turns out relations between color and sound have been theorized for centuries! Scriabin scored colored light into Prometheus: The Poem of Fire (performed at Yale!), and Kandinsky wrote about color as if it were sound. They both offer alternative ways to imagine music, through color — otherwise interpreted as relational waveforms meshed through symbolic logic. Of course, in this project, the relations are arbitrary, since our perception of light waveforms are, but I found kinship with my work with Woodshed in imagining new possibilities for notation, this time through color.
For novice musicians, color registers as a readily understood, visual instrument to interact with music, and encourages playfulness.

Testing out color gradient mixtures
Gradients
Writing the patterns in Processing (built on top of Java) instead of drawing them gave me granular control of color. I first experimented with writing different smooth, linear gradients, all a mixture of RGB — first in one direction (red x to green y), then in two (red x to green y to blue z). A printer mixes cyan, magenta and yellow ink, not red, green and blue light, so calibrations had to be done manually.



Pure RGB
However, the above gradients proved to be inconsistent because most printers mix cyan, magenta, yellow, and black ink, and each printer has proprietary dithering and half toning that makes granular controllability unpredictable. In Processing, I then adopted a modulo algorithm where every pixel in the final pattern is pure red, green or blue. The pattern is divided into small cells. Across the sheet, each cell’s share of green grows in one direction and its split between red and blue shifts in the other. The sensor sees a whole patch of pixels at once, so it reads how much red, green and blue the patch holds. The Teensy turns that into two numbers: x for pull and y for twist. I rendered cells at 128, 96, 64 and 32 pixels and printed the 32-pixel one.


The same rule with 96-pixel cells (left) and 32-pixel cells (right), the one in the instrument.
Mechanism

Pull, twist
plus breath
→
RGB pattern
on the inner pipe
→
Color sensor
reads R, G, B
→
Inverse matrix
un-mixes the reading
→
Position
how far pulled, twisted
→
MIDI
into Logic instruments
Videos
Clips from the build and different playing modes
Linear sawtooth wave, rotation is timbre
pull: pitch · twist: filter
V7 organ
pull: root note · twist: adds chord notes, up to a dominant seventh
V7 arpeggios
Mapped onto drums
4-part sample
Guitar frets
pull: fret · twist: string
Isfahan ( an attempt at translating a piano roll to RGB)
twist: phrase · pull: notes
My friend Shomari playing with the instrument for the first time
Prototyping
What I learned
Before this class, I had little experience with hardware. I learned engineering CAD through OnShape, basic electronics and soldering, and a robust understanding of the physical properties of music as a factor of materials and waveforms.
Outcome
I loved this process! It was a great elective during my last semester, and I’ve been tinkering with music hardware and electronics more. Please reach out if you want to build some instruments together!I view the RGB controller as just a prototype, and would love to expand on it with more advanced color sensor capabilities, custom transparent structures to take the guesswork out of position and for users to see the registered color, and to eventually build a two-handed EWI.
Next
All projects →