Processing and Arduino

Exclusively for CalArts Art+Technology students!

and others with internet access



WORK IN PROGRESS

The code and projects on this page are suitable for novices, but assumes you have mastered the basics of loading programs into the Arduino (eg. the BLINK program, at least) and can run demo programs in Processing.

Goal

The code and program examples here will provide you with a way to extract event information from a very simple analog sensor. By itself, the sensor/resistor pair (see schematic below) simply generates a voltage that is analogous to the amount of light impinging on it at the moment you sample it. We usually want more than that though -- we want to know when someone has waved their hand over the sensor, for example; possibly how fast or slow, or how often. This is what I am calling an event here.

A lot of computer interaction at this (low) level of abstraction requires a kind of thinking alien to most people (i've found, when trying to explain it). Imagine that you see the world as the light sensor does; it sees the "room" simply as the aggregate of the ambient light, bouncing off every surface, some tiny portion of it hitting the sensor.

Picture yourself sitting on a chair in the middle of a room with your eyes closed. What do you see? Not much -- if there's sufficient light, a reddish grey cast. Pass your hand in front of your eyes; what do you see? Momentary darkness as your hand passes before your eyes. Light, dark, light -- that is an event in this scheme of things. That is how the sensor, described below, sees the world.

The code on this page will let you extract that event information, and more, from no more data than that provided by a simple light sensor.

This is a model for giving Processing access to sensors attached to an Arduino. The goal here is simplicity and reliability. It's very easy and convenient to read sensors within the Arduino, but the Arduino does not have access to your host computer's resources (video, sound, network, etc) to do interesting media-related things. Processing is great for media, but "seeing" sensors can be difficult and complex (or expensive).

The code presented here is an attempt to bridge the divide. Simple code shown below does the low-level work of reading sensors (which could be simple analog proportional sensors like a pot, a CdS photocell or phototransistor and a resistor, or store-bought analog sensor) or digital inputs such as sensors.

Sample "theramin" programs that demonstrate all this are below. All they require are an Arduino with a simple light sensor (CdS photo cell or a photo transistor), an Arduino, and Processing.

Please note that the Arduino is not the appropriate input device for data streams such as audio or video. Generally speaking, without additional hardware, or non-trivial programming, the Arduino is not capable of streaming data. Never say never, but it's outside the scopy of this project.

Arduino

The code in the Arduino is very simple. It reads the various sensors -- with code you provide, if the example below is not adequate -- and simply "prints" the result, as decimal numbers. If you have only sensor schematic one sensor the task is utterly trivial. if you have multiple sensors, then all of the values must be printed on one line, separated by a space. That's it.

If your sensor needs code to massage it's output into a numeric value, you will need to provide that code. However most simple sensors -- switches (digital) or pots or light sensors (analog) built-in statements are enough.

Sample Rate

The Arduino prints (outputs) the sensor value(s) repeatedly, until the power is turned off. A simple delay() statement determines the rate that Processing will receive updates from the Arduino -- this is called the sample rate and it is fairly important to determine. Luckily it's easy to specify and doesn't require too much cogitation to work out.

Sample rate is nothing complicated -- how often do you need to "see" the sensor to make sense of it? If you are detecting daytime vs. nighttime, then reading light level once per minute is plenty fast. If you are detecting a ball rolling down a chute, you have to look often enough to not miss the ball.

In general, there's no serious penalty for sampling too often, up to the point where the sample rate overwhelms the host machine due to processing overhead. For example, if you are loading a video file into memory and outputting to a projector, attempting to do so 1000 times a second is likely to cause epic fail.

That said, use common sense to make a first guess, then test. But mainly, think out how often you would need to "look" to perform the desired outcome...

Example Arduino Code

The code below is small enough to simply paste into a sketch window in the Arduino IDE. This reads two (MAXPORTS) analog sensors tied to analog inputs (0 and 1) and "prints" the result. You can (and should) use the serial monitor function in the IDE to check that it's doing what you think it should.

Note that the sample rate is set by the delay() function; after printing the values, it simply kills time for one millisecond. Therefore this loop cannot run more than 1000 times per second. Please note that the print command takes time to work -- at a bit rate of 9600, each character sent takes approximately 1 mS to print! Therefore printing a four-digit number consumes 4mS. If timing is critical you need to take this into account. It's rarely that important however.

/*
 Very simple but more than adequate multiple-sensor reading
 program for the Arduino/Processing multiple value code.
 
 Tom Jennings
 11 Jan 2011
*/

int MAXPORTS = 2;     

void setup () {
  Serial.begin (9600);
}

void loop() {
  for (int i= 0; i < MAXPORTS;) {
    int v= analogRead (i);          // read Nth port,
    Serial.print (v);               // print the value
    ++i < MAXPORTS ? Serial.print (" ") : Serial.println();
  }
  delay (1);                        // 1mS means 1000 times/sec
}

Once you have the Arduino spitting out values to the serial monitor window you're ready to go work on the Processing side of things. (HINT: if they scroll by too fast to see, set the delay() value to someting like 500 to slow it down while you work on it.)

Processing

The processing side is more complex, but luckily there's more and convenient resources on the host computer. There are two difficulties to overcome to get those sensor values the Arduino is spitting out into your Processing program; telling Processing how to find, and connect to, the Arduino, and actually parsing the values it's sending you. The example code below solves both of these problems.

Arduino to Processing communication

You (better) know that the USB link is involved. The problem here is that to the operating system (Windows, MacOS, linux) the Arduino is just a generic serial data device. The USB subsystem attaches the Arduino usually without a problem (this is where i handwave, and refer you to your experience making the BLINK program go). There is no a priori knowledge in there that says "Arduino connects to Processing". You have to tell Processing where (eg. what USB device name or port) the Arduino is.

The example code below, seSerial, contains code that will search for available serial devices, the first of which is assumed (rightly or wrongly, but generally correct) to be an Arduino awaiting connection.

In your Processing program, create a new source file tab, name it seSerial, and cut and paste the seSerial code below into that tab. Then in the main program tab, insert the following code into your setup() function:


  seArduino();                   // connect to the Arduino

That will find and attach the Arduino for you. It prints out, in the black debug window, progress information. At the moment it's not very helpful if there is an error.

The seSerial code handles all data retrieval from the Arduino. The fetched data is stored in an array called ardVals[], and updated as often as data arrives from the Arduino. Remember sample rate, above? That is how often the data here is updated.

There is one additional datum provided by seSerial: a boolean flag called ardValid. Each time the array is updated, ardValid is set to true. If you want to only process newly-arrived values, simply clear this flag (set to false) when you have used the data. seSerial will re-set it every update. You can safely ignore the flag and simply use the array as the "most recent" data from the Arduino.

The Arduino implicitly tells Processing how many datums it has received; the method ardVals.length is the number of numbers in the array.

Signal processing functions

TBD. Code is there, docs not written. Enjoy!

The code below is meant to extract solid 'event' information from simple (even trivial) analog proportional sensors, for example, the lowly photocell/resistor pair attached to analog in. This sensor simply tells the arduino how much light is impinging on it; these functions extract change and rate of change information in a robust way that is easy to turn into event actions. They extranct change, filter noise and smooth te result.

Function return values

All of the functions accept as input the current sensor sample value, and return a simple integer result. The return values all follow this model: 0 means nothing happened; no event.

The "analog" functions return signed integer values (plotted, the resulting values look exactly like math textbook graphs, a sinewave centered on zero, with a zero-crossing indicating the middle of the event).

The "digital" functions return a boolean; true (event) or false (no event), or an unsigned quantity, depending on the function.

About noise

No analog sensor produces a truly constant output; noise is always present, though sometimes small enough toignore. The functions here have enough "gain" (amplification) that noise becomes an issue, and you must deal with it. The defaults are such that small amounts of noise are ignored, but you can increase the threshhold parameter for many functions to decrease sensitivity (to both noise and signal).

Increasinng smoothing (filtering) of input values can be used also (there are settings for this too) but these cause slower response.


Differentiation

Differentiation takes a time series of values (eg. values read from the sensor...) and returns a value that represents the change and rate-of-change in the values. For the case where the sensor is seeing ambient light and returning some relatively unchanging but arbitrary value, this function removes the unchanging portion (returns as 0, or a small number representing noise). Larger changes exceed the noise threshhold and the fucntion returns a value containing the magnitude fo the change (waved your finger, vs. waved your hand) and direction (darker or lighter) as the sign (negative or positive).


// in the static common area

seDiff di;

void setup() {

  di= new seDiff();           // instantiate one,
  di.dlen (10);               // OPTIONAL set history size
  di.setThresh (1);           // OPTIONAL set (low) noise threshhold
  di.setGain (10);            // OPTIONAL amplify (by 10) the result

Usage is very simple. Combining with the Arduino communication examples here:


  if (ardValid) {
    int output= di.diff(ardVals[0]);                // CdS/resistor sensor on analog 0
    ...
 

Averaging and integration

The average object performs an ordinary running average of the last N values, where N is the size of the history buffer, settable with the average.len method. This becomes good-enough integration when the values averaged are a time series, as they are in this instance, the Arduino periodically handing us sensor values.

A long (seconds to minutes) average of the light sensor will give you a good idea of the persistent background, ambient light in the room, for example.

Setting length to 1 causes the average function to simply return it's input, so you can leave it embedded in code and not worry about performance overhead.


// static somewhere

seAverage av;

// in setup()

	av = new seAverage();
    av.len (100);							// set history length to 100 values

// usage in your code...

	int n= av.average (n);
 

Sequential differentiation/integration

Sequential differentiation then integration is a classic signal processing technique, and the one i actually set out to implement when i wrote all of this code. Differentiation and integration can be 'computed' simply and physically by resistors and capacitors. (In fact that was my model for this.)

This method produces very reliable, accurate results, is very immune to noise, and is very tunable. The down side for in event processing is that it introduces a substantial delay. This function produces a textbook sinusoidal response when plotted. if speed is not of the essence then it's hard to wrong with this.

diffInt() is reasonably tunable. See the code and comments for now, until documentation appears here (if ever).


// in static common area

seDiff di;


// in setup()

    di= new seDiff();              // (seDiff contains diffInt method)
    di.dlen (4);
    di.ilen (10);                  // OPTIONAL tuning...
    di.setThresh (1);              // noise threshhold
    di.setGain (10);               // differentiator gain


// usage, as always is

    n= di.diffInt (sensor);

PID (proportional integral derivative) adapted to sensor processing

PID is a very well known method/algorithm for controlling closed-loop servomechanisms; i was surprised to find that i was unable to find it already adapted by someone else to this this signal processing task, for which it is nearly ideal. Without going into too much detail (for that, see Wikipedia) PID here is used to "correct" the change from the sensor signal, as if it were an error; the resulting "correction" then must follow the changes the sensor undergoes. PID is entirely tuneable.

The main advantage of PID here is that differentiation and integration are done in parallel; therefore it is much faster (more responsive) than sequential differentiation/integration. The output, if graphed, is not as pretty (sinusoidal) as diffInt, however this hardly matters for simply picking events from outside stimuli.

The main disadvantage to PID is thatit may require some tuning to avoid noise and retain sensitivity. This isn't hard, and once done (for a given sensor and program) it needs no further attention and is quite repeatable.

 
 // in the static common area
 
  sePid pid;		// handle for the object we'll create in setup()
  
 
// in setup()

  pid = new sePID();             // instantiate one!
  pid.setKg (20);                // makes event larger; "-" if sensor change goes LOW
  pid.setKd (14);                // larger (> 20) makes it sensitive to changes
  pid.setKt (3);                 // threshhold; set above "noise" (output on no input)

 

Simply pass each sensor value received through the PID function (there's a couple variants to choose from) and the returned value tells you about the event, if any.

Open each in Processing, then create a new tab for seSerial as above, and also one for seDiffInt which contains objects that perform differentiation, integration, sequential differentiation/integration, double sequential diff/int, and a special PID (proportional integral differential) function that is a very tunable way to handle.

downloads

Arduino to Processing serial communication

The first one is the Arduino code, above. The second one is Processing code. Create a new tab (source file) in your Processing program, paste seSerial in.

Processing, sensor signal-processing

The differentiation, integration, PID code mentioned above.

Sample programs

These programs create sound and plot a graph of sensor input as processed by the three types of functions; basically they create a theramin-like sound and are quite sensitive and playful (though they'd need a bit of work to make sound good). They use the Beads audio library; follow the simple instructions here to install the library for Processing.