Guitar+Interface

ArDuiN^Gu#TARR INT8rface   Guitar gesture input, DSP analysis, Video Game Interaction  **Arcade Buttons, Visible Light sensing, Accelorometer** pitch detection, Joystick, Infrared Sensors

A few words about audiovisual performance
Creating an audiovisual performance is something that has been done for a long time from color organs, to groups like Single Wing Turquoise Bird in the sixties, to recently the advent of music videos and the vj. Currently with the increased performance in laptops, recent open source software trends, microcontrollers and a growing history of arts interaction audiovisual performances are being performed and attempted more and more by one person and new custom instruments are showing up for these performances. Below are some pictures of recent video/visual based instruments and audiovisual performances.

from left to right: video bass, video guitar, Christopher Willits Live from left to right: My UCSD ICAM Senior Project, Electroception at the Loft

How to make meaning out of guitar gestures?
the Arduino platform allows for easy interaction with the processing and pd programming environments, making various interpretations of interaction available. I feel that this part of the research will be an ongoing project that has no correct answer only different interpretations. My initial idea was based on adding video game controls, which spark a certain nostalgia for most. This engagement allows performer and audience to correlate the musical experience to other memories and therefore interpret the music differently.


 * on/off interplay
 * time based interactions
 * position sensing
 * continuous joystick inputs
 * tapping, both on/off and time based
 * creating modes, as well as allowing for user programmed modes
 * d-beam like distance sensing, up/down motion with hand


 * adding new controls
 * reinterpreting existing techniques

The Joystick was an idea given to me by Jaime Oliver during a conversation about extending the guitar. Basically adding a joystick is meant to be a modernization of the traditional whammy bar. Also guitarists frequently use a method called tapping to increase speed and create a more legato sound, so after talking with Adam Wilson we came up with an idea to use the photodiodes not just for position sensing but also as a way to reinterpret tapping. As well with the arcade buttons using them in conjunction with pitch and amplitude information they can create visual fx which become more intertwined with the music, this can be thought of as simply a reinterpretation of turning on/off audio fx pedals, just this time the controls are on the guitar and not on the ground.

Sensing Methods
<span style="font-family: 'Lucida Console',Monaco,monospace;"> joystick <span style="font-family: 'Lucida Console',Monaco,monospace;"> momentary buttons <span style="font-family: 'Lucida Console',Monaco,monospace;"> on/off toggle switches <span style="font-family: 'Lucida Console',Monaco,monospace;"> photodiode array <span style="font-family: 'Lucida Console',Monaco,monospace;"> infrared range finder <span style="font-family: 'Lucida Console',Monaco,monospace;"> midi pickup with custom preamp

<span style="font-family: 'Lucida Console',Monaco,monospace;">Connections
<span style="font-family: 'Lucida Console',Monaco,monospace;"> to connect all of these inputs i will use ic chips that allow for multiplexing. Multiplexing allows <span style="font-family: 'Lucida Console',Monaco,monospace;"> one to conserve pin space on the Arduino itself by hooking up multiple sensors to an external chip <span style="font-family: 'Lucida Console',Monaco,monospace;"> and polling that chip for the current state. Within the arduino community there examples on how to <span style="font-family: 'Lucida Console',Monaco,monospace;"> use these chips, i have links below.

<span style="font-family: 'Lucida Console',Monaco,monospace;">Software
<span style="font-family: 'Lucida Console',Monaco,monospace;"> The photodiode array will take a bit more consideration to be done in code. Firstly a method has to <span style="font-family: 'Lucida Console',Monaco,monospace;"> be developed for initializing the current amount of ambient light in the room, then dealing with hand <span style="font-family: 'Lucida Console',Monaco,monospace;"> position. I plan on keeping track of the edges. This means while your hand is in the middle of the array <span style="font-family: 'Lucida Console',Monaco,monospace;"> the front, thumb side, and back, pinky side, will count as edges. In between each edge the sensors will <span style="font-family: 'Lucida Console',Monaco,monospace;"> read low levels of voltage coming in, due to the lack of light.

<span style="font-family: 'Lucida Console',Monaco,monospace;"> At this point the software is meant only to receive data and pass it on over a serial connection to <span style="font-family: 'Lucida Console',Monaco,monospace;"> another program such as processing or pd. Within processing or pd is where the artistic application <span style="font-family: 'Lucida Console',Monaco,monospace;"> occurs controlling visuals and music.

<span style="font-family: 'Lucida Console',Monaco,monospace;"> in general i would like parameters such as these to be controlled


 * visual filters such as invert, blur, dodge, thresholds
 * wipes
 * brightness
 * object location
 * color
 * size of objects
 * blending or masking

<span style="font-family: 'Lucida Console',Monaco,monospace;">Arduino Pin Connections
<span style="font-family: 'Lucida Console',Monaco,monospace;"> digital pins <span style="font-family: 'Lucida Console',Monaco,monospace;"> 0 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 1 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 2 CD4021B x2 (for 16 multiplexed inputs) ,12 momentary buttons, 4 on/off toggle switches <span style="font-family: 'Lucida Console',Monaco,monospace;"> 3 clock pin for above <span style="font-family: 'Lucida Console',Monaco,monospace;"> 4 latch pin for above <span style="font-family: 'Lucida Console',Monaco,monospace;"> 5 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 6 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 7 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 8 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 9 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 10 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 11 CD14067B <span style="font-family: 'Lucida Console',Monaco,monospace;"> 12 CD14067B <span style="font-family: 'Lucida Console',Monaco,monospace;"> 13 CD14067B <span style="font-family: 'Lucida Console',Monaco,monospace;"> 14 CD14067B

<span style="font-family: 'Lucida Console',Monaco,monospace;"> analog pins <span style="font-family: 'Lucida Console',Monaco,monospace;"> 0 joystick 1 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 1 joystick 2 <span style="font-family: 'Lucida Console',Monaco,monospace;"> 2 infrared sensor <span style="font-family: 'Lucida Console',Monaco,monospace;"> 3 infrared sensor <span style="font-family: 'Lucida Console',Monaco,monospace;"> 4 CD14067B mulitplexed photodiode input array <span style="font-family: 'Lucida Console',Monaco,monospace;"> 5

updates...
july 21 finally got the CD14067B and hooked up 12 photodiodes, in minimum it worked. the photodiodes seem to have inconsistent resistance, however they all seemed to have the same range. my guitar body, neck, and hardware should arrive this week so i am excited about getting started on the actual routing which i think will actually be the hardest part. I am looking to use a CNC router to do this which will be clean if i don't destroy the body in the process. july 22 rewriting arduino code from online examples to make sense with the analog multiplexers. very easy changes, i just implemented a table lookup system to match the analog input to the corresponding truth table, such that the ABCD pins send out the correct binary data to make the analog pin in question available. as time permits I'll upload all of this code. july 23 problems with the photodiodes it seems they were working poorly enough to fool me into believing in them however, after really looking at the data sheet it seems they have a "daylight blocking filter" and seem sensitive to ir more then anything. So I guess its time to go buy some photoresistors and try again. As for the CNC business after talking to Robert Twomey he helped me decide on making a jig, uhhh errr template, with the cnc and then using a hand router on the actual body. this should be much safer to the body itself. aug 1 no updates for a week but in that time i have gotten the new photodiodes up and running and just finished making a toolpath to cut the jig on the cnc router. Also have been doing a some video editing and I am scheming to come up with ways to make this guitar work with a piece of vj software called Resolume Avenue. aug 4 ok i cut my template out on the cnc router, which took quite a bit of time as the router was out of commission initially. looking to get the actual guitar body routing done in the next couple of days. also Adam Wilson gave me the idea of using the photodiode array a bit differently, as in playing them individually in a tapping fashion to create new fx, sounds great that will be implemented. aug 5 routed out guitar body, cut plexiglass covers and scavenged new joysticks from video game controllers. aug 8 did some more woodworking on the guitar and after collecting some more parts yesterday I am now starting on figuring out how to mount everything properly. The more I think about this the more I think I will need to custom print myself a pcb board, the process continues.

__**<span style="font-family: 'Lucida Console',Monaco,monospace;">reference links **__

<span style="font-family: 'Lucida Console',Monaco,monospace; font-weight: normal;"> [|DIY touch sensor] <span style="font-family: 'Lucida Console',Monaco,monospace; font-weight: normal;"> [|Arduino Joystick] <span style="font-family: 'Lucida Console',Monaco,monospace; font-weight: normal;"> [|Arduino Shift In]

<span style="font-family: arial,helvetica,sans-serif; font-size: 14px; font-weight: normal; line-height: 21px;">inspirational links <span style="font-family: arial,helvetica,sans-serif; font-size: 14px; font-weight: normal; line-height: 21px;"> [|video guitar] <span style="font-family: arial,helvetica,sans-serif; font-size: 14px; font-weight: normal; line-height: 21px;"> [|video bass] <span style="font-family: arial,helvetica,sans-serif; font-size: 14px; font-weight: normal; line-height: 21px;"> [|guitarduino] <span style="font-family: arial,helvetica,sans-serif; font-size: 14px; font-weight: normal; line-height: 21px;"> [|the sad dj] []

Thanks.