THE FLOCK RADIO MESSAGING SYSTEM

SUMMARY

Flock is a radio messaging system for Arduino and similar machines. it uses an absurdly low-cost ($1.00) radio module containing a Nordic nRF24L01+ radio chip. Flock passes information about events, sensors, actuators, and configuration data between remote machines, optionally with a "desktop" base computer. Flock uses the SR Message system to exchange information between and within physically disparate computers, via radio, USB, serial monitor and stored EEPROM.

Flock is receive-driven, (reciever always on), a many-to-many protocol with a bias towards a central node called Base (flockBase). Flock was originally designed for communicating with/between a number of stand-alone devices, each with sound cards and radios, that independently makes bird sounds (hence "Flock"); a desktop computer uses flockBase to send receive status from and send configuration information to individual birds (volume, play schedule and order, etc). since the receiver is always on power consumption is a continuous 19mA at 3.3V, approximately the same as a typical LED.

Peep is a send-driven, (reciever usually off); Flock with transmit-driven power management. Peep is designed for ultra-low-power (eg. tiny solar) remote controllers that sporadically transmit status to the base and its peers. Peep is stealthy as a side effect of it's power management. since the receiver is on only during Peep burst, quiescent current is in the microamp range, and 19mA while receiving. with an average 5-second on time once per minute average power consumption is 1.6 mA equivalent average.

Flock and Peep comes with my SR24 hardware driver for the Nordic nRF24L01+ (plus) radio chip and all of it's variants. it is compact and fast with a robust API.

Flock and Peep are designed from-scratch protocols for a very small scale environment, not a top-heavy commodity protocol force-fit like 802.11 wifi, and uses cheap hardware unlike Xbee. the Flock protocols are frugal, open, and cooperative, auto-negotiate channels, do dynamic bad-channel mapping and fully tunable if so desired.

the purpose of Flock is to reasonably maintain a channel (in Shannon's sense) between all birds in a flock, and with the base. each bird in the flock maintains it's own sense of connection to the channel.

HOW FLOCK PROTOCOL WORKS

Flock collectively negotiates the radio channel used and avoids bad (noisy) channels using two major metrics, a receive-timeout/request-acknowledgement scheme, and a channel quality scheme called ack balance. three methods control Flock operation. they determine how strong the sense of "connection" amongst the flock by tracking the time since the last successful packet receipt and how often to request channel acknowledgement. these have little effect on Peep which is intentionally sporadic and (unwilling, unable) to recieve packets.

the sense of "connection" is based solely on recent successful receipt of packets.

radio is inherently a low-reliability medium, in theory and in practice. the essence of radio channel reliability is that the probability of success decreases as time passes; eg. the longer you wait the less likely it is to work. even if every packet sent is acknowledged by the receiver, that acknowledgement could be lost. trying to solve this absolutely results in an infinite regress of acks-to-acks, stumbling upon Gilb's 9th law:

Investment in reliability will increase until it exceeds the probable cost of errors, or somebody insists on getting some useful work done. -- GILB'S LAW OF COMPUTER RELIABILITY #9

ack balance allows base and birds to tolerate a settable amount of packet loss before giving up on the channel and attempting to renegotiation. implmentation is simple and relies on the occasional use of the optional ack request feature: the ack-balance count is incremented by one whenever any packet is sent with the ack-request flag is set, and decremented when any packet is received. therefore the ack-balance counter reflects the ratio of lost packets. when this single number threshold is exceeded, the channel is presumed to be unusable and is renegotiated.

PROTOCOL NEGOTIATION AND OVERHEAD

radio is an inherently unreliable medium. in a perfect world we'd just pick a radio channel and send and recieve packets without worry of loss, corruption or delay. alas that is not how radio works.

the underlying radio hardware allegedly supports 127 channels in 2400MHz 802.11 allocation; in reality about 120 of those are usable. one-dollar radios are lousy radios, very low power and have relatively poor receivers, and all devices in the 2.4GHz band must tolerate inevitable interference. wifi, microwave ovens, etc all create interference and it varies with time and location.

therefore the reality of radio communication is that it is dominated by channel negotiation, error detection, error correction, retries and delays. this is what Flock protocol handles.

from the programmers point of view Flock is simple. each bird (radio) in the flock has an identity (ID) consisting of a single ASCII character in the range of A..Z (and a..z) and additionally, @ for the base identity (flockBase). once configured the protocol's main state machine method is invoked repeatedly from loop().

here's a brief description of the Flock protocol. it is different for base and bird, which are complementary.

flockBase

if base never hears from any bird(s) ever, it will periodically change channels. it will remain on the same channel as long as packets are exchanged, within parameters described below. when channel fails due to error criteria that channel is marked "bad" and a random non-bad channel selected. (bad channels are randomly un-marked bad at a low rate, ensuring that previously bad channels are eventually retried and to prevent running out of channels.)

bird

bird protocol is only slightly more complicated.

FLOCK BASE PROTOCOL DISCUSSION

three methods control Flock operation. they determine how strong the sense of "connection" amongst the flock by tracking the time since the last successful packet receipt and how often to request channel acknowledgement. these have little effect on Peep which is intentionally sporadic and (unwilling, unable) to recieve packets. the sense of "connection" is based solely on recent successful receipt of packets.

RxAT determines the time after which no valid packet receipt (addressed to "us") causes channel renegotiation. RxAT should be long relative to RxAR.

RxAR is the rate at which acknowledgement-request packets are transmitted, when no valid packet has been received. this should be short relative to RxAT. a good starting point is to allow for one or two ack-request packets at this rate to be lost; eg. RxAR requests an ack response every two seconds, with RxAT set to 4, 5, 6 seconds.

for the base, the meaning of RxAR is different, though the action is the same. when the paradigm for the flock is that the base is essentially a passive log of bird data, with only sporadic need to write to birds, then RxAR can be set larger than RxAT to effectively disable it; this leaves it up to the birds to maintain channel connection to meet their needs. (Peep takes this concept even further.) if however the flock paradigm is that the birds must be ready to respond quickly to commands from the base then a shorter RxAR is called for.

note that RxAR and RxAT are in deciseconds (tenths).

since Peep is bird-driven, eg. only when there is data to send to the base, not only is the bird's receiver turned off, after channel negotiation packet transmission and reception keeps the channel open for the time specified by PeepUpTime.

in addition to the no-packet-received timeout determined by RxAR and RxAT, ackThreshold sets a limit on channel (low) quality, eg. packet loss rate through a technique called ack balance. ack balance increases by one for every packet transmitted that contains an acknowledgement request; ack balance decreases by one with every valid packet received. if ack balance exceeds ackThreshold the channel is abandoned and renegotiated.

the sender can add an optional flag to a sent packet called ack request. the receiver honors this by immediately transmitting an empty (but with to, from addresses) packet back to the sender as acknowledgement. ack balance counts up with every ack-request sent, and counts down with any packet received. if this value grows past a threshold the channel is renegotiated.

FLOCK PROTOCOL ERROR SITUATIONS

if nothing ever went wrong we wouldn't need protocols.

given the unreliability of radio propagation errors are common. rather than attempt a typical brute-force approach with complex hardware and software and attendant cost and overhead that would likely prevent you from considering radio in the first place Flock makes peace with errors by involving usage and code practice into the protocol.

the simplest case is a single bird and base. close together (but not too close if a high-power version is used, else receiver overload increases errors -- what do you want for a dollar?) error rates are ignorably low. moving the radios apart and they become sensitive to signal strength (inverse square law) and increased effects of interference. this is obvious, and packet re-sends and eventual channel renegotiation handles much of it, up to the point where the transmitted signal (physics!) is too weak and there is nothing that can be done about it (other than to move them closer again or buy the $10 version with PA and LNA).

things grow complex when there are many bird clients talking to one base and to each other. as the number grows it is inevitable that one or a few birds will have trouble communicating reliably on a channel the other birds find acceptable. if high reliability is required then you must arrange for signals strength and quality improvements eg. move them close or buy better radios -- truly weak signal can't be fixed in a protocol. but radio is rarely fail/success, it usually manifests as increase in error rate, increase in packet loss. when that is the case, as it is most of the time, then software design schemes are the best solution. in my original application, the Meow Wolf sound boxes, each bird client is a standalone sound making machine, and Flock is used to configure them (volume, play rate, play selection, etc), to silence and enable them, and for telemetry. designing for non-zero errors rates by having a desktop computer running a Processing application gathering Flock data via the base adapter and updating an on-screen display with data -- and the age of that data -- allows monitoring and controlling the Flock.

FLOCK vs. PEEP

in brief, Flock is receiver-driven where Peep is transmit-driven. in Flock, the receiver is on 100% of the time, and always ready to accept packets, and has the features and advantages above.

Peep takes a different approach though it is completely compatible with Flock -- with one accomodation needed. Peep keeps the radio reciever turned of to save power. standby current is therefore close to zero, ideal for battery-powered applications.

Peep is designed for the needs ot remote devices to report back to base with sensor readings, telemetry, events and the like, with minimal information coming from base to the birds. Peep birds "pop up", connect, write, disconnect, power off; therefore the time window for flockBase to transmit to each bird is sporadic and brief -- the accommodation is that base -- or more likely the desktop using flockBase to talk to the flock -- has per-bird outgoing messages queued up ready to go when that bird connects to the base to deliver data. the time window width is configurable.

as of this writing (aug 2015) Peep is in use by me but the code base is still undergoing ripple-effect changes (eg. those mentioned to the per-bird connected() status) and i don't have a lot of usage experience to document yet.

...MESSAGING

of almost greater importance than the radio system is the messaging system -- a decievingly simple way to send and receive event, sensor and configuration data. these are not desktop machines -- we don't use them to move bulk data, but to respond to events, dedicated to small embeddable tasks. the critical characteristic of an event is mainly its occurence, or timing, and very little information is needed to signal them. the messaging system used by Flock and Peep also exists as it's own library (SRMessage) and is generally useful.

SRMessage uses the same message format in all mediums: to and from USB link (including manual commands in the Serial Monitor), any hardware or softwareSerial link, stored in EEPROM, Flash, RAM or constant string. messages can be created with the supplied message-creation methods or a simply formatted string in memory via any C/C++ code.

SRMessage supports multiple channels and multiple priorities if needed (eg. EEPROM stored commands have higher priority than radio messages).

the messaging system is deceptively simple. though this is a summary introduction, and normally not a place to jump right into the techie details in this case i think it is work explaining in modest detail what a command message looks like before continuing with the overview, partly because it is so critically important, but mainly because it's just so damn simple and human-compatible.

a message is a text string containing one or more commands. commands are single ASCII letter characters (A..Z, a..z) and a numeric argument consisting of zero or more digits. arguments are in the range of 0..65535. a command letter with no associated digits has an argument of 0.

commands are in postfix notation -- the numeric argument precedes the command. some examples:

123X command X, argument is 123 decimal
0X command X, argument is 0
X command X, argument is 0.

messages, aka command strings, can contain one or more commands, are read left to right, and may contain spaces for clarity. for example:

123X 50V 22A M four commands (X, M, A, M) with numeric arguments. M's argument is implicitly 0.
123X50V22AM he same, but less readable

Arduino programs often contain fairly small datums at the top of the source code generally considered to be program "configuration": LED blink rates, addresses, that sort of thing. with very slight rearrangement of your code these can become command strings. instead of inline C++ code command strings are handed to the message system to parse and execute.

but why add unnecessary complexity when simple variable-assignment works?

because once configuration is considered a command string, they can be moved about via the Flock system -- configuration once requiring editing, recompile and upload, can now be set under program control; command strings stored in EEPROM read at power-up time, or from commands received via USB or radio, or typed in manually in the Serial Monitor window.

the "linkage" from the message decomposer to your code is easy.

HOW COMMAND PARSING WORKS

Flock invokes a support package, called SRMessage. SRMessage decomposes the command string into command and numeric argument, handling errors, and passes the result to a function that you have written and provided to SRMessage one time, in setup(). the function you write is called a dispatcher, and a perfectly correct example follows here:

void dispatcher (int c, unsigned nnn) switch (c) { case 'X': someVariable= nnn; break; case 'V': someFunction (nnn); break; case 'A': analogWrite (SOMEPIN, nnn); break; case 'M': digitalWrite (SOMEOTHERPIN, nnn); break; } }

(the command character c is passed as an int instead of a char because SRMessage also passes information about the message stream as values outside ASCII char range. as shown above that metadata is ignored.)

as SRMessage decomposes a command string from left to right each (char command, unsigned int argument) pair is provided to your dispatcher code. your dispatcher is always passed correct data, and does not need to do any error checking. i prefer switch..case but if..else works too.

all command sources you choose (USB, EEPROM, radio, string array, etc) can have up to four dispatchers, and up to four levels of priority.

message composition is much simpler, and described elsewhere.

the radio portion of Flock consists of two software components, the nRF24L01+ driver and the Flock protocol, and of course requires a radio module. XXXX LINK HARDWARE PAGE XXX

the Nordic nRF24L01+ chip and module has been around for years, is very inexpensive, under a dollar when bought directly from China, and for the very small data packets it works with, high speed. it operates within the 2.4GHz band, dividing it into 125 channels. it was designed to be embedded in industrial and consumer gear, things like wireless keyboards and displays. it has, accordingly, fairly lousy RF performance all in all.

the nRF24L01+ uses the Arduino SPI interface plus two additional pins (typically 9, 10). it consumes about as much power as an LED -- peak draw is 19mA at 3.3V transmitting or receiving. it sends and receives packets up to 32 bytes in length, at a 2 MHz data rate.

the nRF24L01+'s small size and modest requirements make it ideal for ubiquitous low-volume radio networking. it is however a tricky chip to interface, given some of it's peculiar feature set oriented towards the embedded market, but my driver and Flock layer solve them all (eg. the peculiar dedicated "Pipe" addressing scheme).

the nRF24L01+ driver can be used standalone, without Flock. the API for that is fairly easy to use.

Flock protocol was designed for a specific project XXXXX MEOW WOLF BIRDS LINK XXXXX where complex electronic objects are mounted in physically difficult-to-reach locations, high up on the wall or in a ceiling, where simple tasks like configuration, making adjustments or monitoring problems requires a High-Lift or scary ladder. Flock provides both control (setting volume, enable/disable, etc) and telemetry (internal problems, etc).

very specifically this required that the radios not need any configuration at all, nor adjustment should (when!) the RF/EMI environment changed. the 2.4GHz industrial band is noisy and crowded (eg. Wifi), and the large number of radios required necessitated an inexpensive solution. the catch is that cheap radios are cheap for a reason: they are highly susceptible to noise and existing neighborhood radio and wifi traffic. specifically this meant that the radio system has to adapt to channels that are, or later become, unusable, and do so automatically. worse yet, problems are often local to a place and time, so that some birds cannot connect while others are fine. managing this is what Flock does.

nRF24L01+ radio module wiring

the Nordic Semiconductor NRF24L01+ chip was designed for super-low-end data radio such as wireless keyboards, with ranges measured in meters to tens of meters. it contains a dedicated command processor and 2400 MHz radio and packet assembly in silicon, and an antenna. it can send or receive one small packet at a time. by desktop computer standards, it is very low performance, but it is ideal for event based microcontroller communication.

NRF24l01+ modules can be purchased for under one US dollar each, retail. i buy them from AliExpress.com; my last purchase was 10 modules for $8.65, postage-paid, shipped from China (takes three weeks to arrive). there are two minor variations out there; one has a single row of 0.05" spaced holes for wiring, the other has a 2x8 pin header with 0.1" spacing. makes sure you get the PLUS model, that trailing "+" is important! there is a NRF24L01 (non PLUS) version; it's older, no longer common, but is missing critical features and my driver does not support it. DO NOT! buy the cheapest modules with the epoxy blob on the board. almost uniformly those are junk, low quality often incompatible clones. it's hard to imagine an industry bsaed upon cloning an already rock-bottom-priced chip, but there you go.

NRF24L01+-based data radios are so cheap and easy to use there is little reason to not add communications to a microcontroller project.

the NRF24L01+ module requires 3.3V and ground, two general-purpose output pins (you choose), and the three pins dedicated to the standard Arduino SPI interface and library (determined by the board manufacturer). that's seven wires; power and ground plus five I/O. the module's interrupt pin is not used.

pin Uno Mega2560 TM4C123XL (SPI #2) 
3.3V     J1-1 
GND     J2-1 
CE 9 9   as desired
CSN (SS) 10 10 J2-9 (PA_3) as desired
SCK 13 52 J1-7 per hardware specification
MOSI 11 51 J2-6 per hardware specification
MISO 12 50 J2-7 per hardware specification

WIRING

NOTE: you absolutely must install two capacitors, a 0.1uF ceramic disc and a 2.2uF electrolytic, across the radio's VCC and GND pins, as close to the module as possible. preferably on the module itself. you will have hard to diagnose erratic performance if you do not. i tested this carefully; the .1uF disc alone is insufficient; the electrolytic is required. larger is fine but not required. you can see the two caps in the pic to the right of an early test rig. the two single-row pin headers plug directly into an Arduino.

NRF24L01+ hardware limitations and caveats

the Nordic NRF24L01+ chip is mildly complex, and programming it effectively -- meaning real error detection and recovery -- is tricky. the following are the general areas of difficulty with this chip, and inherently with data radios that do not follow the consumer model, eg. all resources available at all times, unconcerned with the cost of that consumption.

maximum power consumption is low (under 20mA peak) but real savings (microamps!) requires some knowledge of how radio works, and a strategy to manage it. at issue here is knowing when to listen (it isn't "always"). the receiver consumes more power than the transmitter; leaving it on consumes as much power as an LED. turned off, the radio is almost zero load. therefore, for true low power consumption, a strategy and a protocol to accomplish that strategy is needed. this is the Peep protocol. of course there are plenty of times when leaving the receiver on all the time is just fine, eg. USB-powered. for that, Flock protocol is more appropriate.

radio communications requires two or more participants; when more than a pair exists each generally requires a unique identity, aka network addresses. given that the NRF24L01+ was designed for disposable-quality remote controls and the ilk, it comes with a packet addressing scheme ("pipe" address) that has side effects that make general purpose radio communication difficult. the usual approach in NRF24L01+ drivers is to make you manually assign 64-bit unique addresses to every radio. my drivers manage this automatically, with some caveats; if Flock or Peep protocol is used the caveats do not apply. the API also supports very clean ways of handling pipe addressing should you need it.

due to the above my driver and protocols do not use Auto-Acknowledge and automatic retransmission. those features are what make/made this chip successful in the consumer field, but they have severe limitations when used in a general networking applications. if you are willing to manually configure unique 64-bit pipe addressess for each radio in your network then it works great; without that the false-acknowledgement rate will make them useless. this is fully discussed in comments in the driver code, and by understanding the precise operation of the address matching in the receive pipes. Flock and Peep protocol side-step the problem by a high-level rethink of communications needs and doing ack/nak explicitly. rtfm.

(if you have some familiarity with this chip, my code relies on Dynamic Payload but disables Auto-Acknowledgement. only one pair of pipe addresses is used (eg. Multiceiver not used), and these are alternated between RX and TX, so that all receivers are address A, and all transmitters are address B. all receivers therefore receive a transmitted packet (if in range, etc). )

FLOCK NETWORKING CHARACTERISTICS

Flock is a flat, non-heirarchical system, with one (significant) non-linearity: a designated "base" radio that also serves as desktop/laptop entry into the Flock network.

each radio (node) in Flock is called a bird. each bird has an address, a single letter A..Z (though there is no reason it can't be (A..Z,a..z), or for that matter, any of the 96 printable characters). the address is the sole network configuration datum in a bird.

the radio allegedly supports up to 127 channels, but in practice the lower-most channels don't work well, and some of the higher-power modules are worse. by default Flock uses channels 20..125 which maps to 2400..2125 MHz with 1 MHz channel spacing.

an entire Flock uses one channel, so collisions happen. with a modest number of birds collisions are quite rare; 32-byte, 2 megabit/sec packets are a mere hundred microseconds long, and packets are generally sent at low rate (every few seconds max). in the chip collisions are indistinguishable from non-received packets (eg. CRC error, no preamble sync, etc). multiple Flocks could be restricted to different channel ranges, or the entire flock hard-coded for a different Nordic Pipe address, or both but that's an unlikely problem to have.

Flock is a semi-(connection-based) system. each bird independently attempts to maintain "constant" contact with the Base, "constant" defined generally as 1..60 seconds (the Meow Wolf Flock uses 20 seconds).

channel negotiation is simple. at power-up, or after failure of an error-metric described below, Base chooses a (new) channel, at random, from the allowed range.

at power-up, or after failure of the error-metric below, each bird searches for the base by selecting a random channel from the allowed range, then transmitting an ack-request packet and awaiting a response for a stochastic interval. if the correct response is received the search is complete and the bird is on-channel, "connected". if not, the process is repeated. channel discovery generally takes 50 milliseconds to complete.

the existence of Base implies that the topology is a heirarchy. it is actually flat. though packets have distinct and customary TO: and FROM: addresses, each bird receives all network traffic, and packets are selectively (by TO: address) delivered to userland code. this has the substantial advantage of allowing each bird to see responses from Base to other birds and avoid the need for sending "keep alive" ack requests, minimizing network collisions. this is then a full mesh network,

the error metric for both Base and Bird is called ack-balance. each keeps a simple integer tally that increases whenever a packet is transmitted that requires a response (ack-request) and decrements it whenever an ack-request is honored. various threshholds for this are settable in both Bird and Base. this means that channel quality is tracked only in the aggregate, not per-bird (which would require much more overhead and further entrenchment of a connection paradigm).

the worse case for the ack-balance scheme is when a large portion of the flock is having a low error rate talking to Base, with one or few outlier Birds having poor error rates.

when the total number of birds is small, however, even one "bad bird" will skew the tally towards error, and new channel selection occurs reasonably rapidly (and is tuneable). for a larger flock, another, external, pressure applies: the whole point of a large flock is to become independent of [the failure of] one or few birds. with a large number of birds, frequent channel re-negotiation itself would degrade total flock accessibility. (per bird connection attempt/success tallies received via even sporadic telemetry would identify the problematic birds).

the base maintains a bad-channel list. when a new channel is selected due to ack-balance error, that channel is marked "bad" in a table, and is avoided in future channel selections. with over 100 channels to choose from running out of available channels is unlikely, but not impossible. to cover the latter case, each time a new channel is searched for a random channel is "un marked" as bad, ensuring two things: (1) that some channel will be available to attempt communication, and (2) a once-bad channel may in fact become usable, as often "bad channel" meant only interference in that channel from some other device.

LIMITATIONS AND SHORTCOMINGS

inherent in the nRF24L01+'s particular design are many features that become shortcomings when used for general-purpose data communication. the most significant is the built-in "Pipe" addressing, which puts a 64-bit address into each transmitted packet, that must match one of the configured receive pipes. the problem with this in a general environment is that to "do it right" one needs to individually code each radio's driver with a unique 64-bit address, utterly precluding general purpose use and the design of Flock. to allow any radio to talk to any other radio (eg. to do address and other negotiation/configuration in software) my driver, and most others, set all radios to use the same pair of receive and transmit pipe addresses. to make a very long story short(er) (rtfm driver comments) the useful-seeming "Auto-Acknowledgement" feature, where the receiving chip itself, in hardware, transmits a packet-receipt acknowledgement to the sender, utterly fails when it encounters it's own pipe address, generating a substantial number of false positives. tl;dr but while my nRD24L01+ driver code does in fact support a very flexible pipe addressing scheme that *almost* solves the problem, it in fact is a vastly superior thing to simply do acknowledgement in the protocol layer. no complaints about the length of this, you didn't have to read it.

[10 aug 2015 note: at the moment Flock uses a 4-deep transmit queue that is managed by the main state machine, though the meaning of "connected" as aggregate behavior is true, new code will bring truly point-to-point status into the delivery status calculation, and with the queue, will raise delivery reliabiity about an order of magnitude, approximating a fully per-packet-acknowledgement reliability rate without the overhead.]delivery of Flock packets (and messages) is not fully guaranteed. the problem is more philosophical than technical; the pieces are all in place (ack-request, packet sequence, etc) for the common solution of retry until ack, with retry count, etc. but i am deeply working on a different paradigm for networking, partially visible in the code and protocol description, that in fact violates the usual isolation of ISO layers. the Flock packet format conforms too to the overt openness of the protocol and addressing scheme, in that "protocol overhead" (ack request, channel-change hint) are delivered partially in the payload and quite visible to userland code. this is intentional, a scaling of accommodation acknowledging the succinct Arduino environment. the complexity and bloat of traditional error detection and especially error correction gets quickly out of hand, in a machine with 2048 bytes of RAM. part of the solution, i'm convinced, is to move some error handling "up the stack" to the user, but not in the traditional way -- but by incorporating very high level abstract knowledge of self and neighbor into the intended function of the machine. news at 11.

THE FUTURE

these are some items being considered for future release.

within the initial channel-discovery exchange birds will deliver to Base both uptime and a channel-quality telemetry metric, consisting of the number of channel-discovery attempts. base can use this to determine channel quality from the birds point of view, for report and to make dynamic bad-channel determinations.

Flock requires that the receiver be left powered on and enabled, which is the chip's maximum power consumption state. for many projects this is just fine and the advantages are obvious. but for remote-sensor type applications, where the majority of messages are delivered from bird to base this is not required, and when those applications are solar-powered, not really tolerable. the Peep protocol will address this, which is actually a minor variant of Flock: bird initiates all communication; base cannot arbitrarily message bird. from the bird's standpoint, the chip will be entirely powered down between use, reducing consumption to nanowatts. when it is time to send message(s), the chip is powered up, and the channel-discovery phase executed, and message(s) delivered as per Flock protocol. after a suitable interval (to accommodate asynchronous reception from base or peers) the radio is powered down. from Base's point of view, either Base, or the/an attached application will maintain a (very short) queue of outstanding messages to deliver to the bird when it "comes online".




how to use Flock

the Flock code is a C++ object contained Arduino library, installed and instantiated in the usual manner. details are below. Flock is initialized in setup() and the main method, generally Flock.message() but possibly Flock.packet() is invoked within loop() continuously. Flock is a state machine, and never blocks. it assumes that code surrounding it does not block.

the main method handles all protocol and channel negotiation and error handling, though there are other methods to get/set parameters and status (rummage around in the API for details). message/packet reception is indicated by the main method's result code. message/packet transmission is done asynchronously with the write() method.

i assume that the biggest impediment to using Flock will be not Flock itself, but structural changes to your program(s) to accomodate non-blocking code and to think in terms of message-passing. i [am working on] a thorough exposition on a particular strategy that i have found enormously successful for structuring solutions on Arduino and the like. that document can be found XXXXXXXXXXXXXXXXXXXXXX.

likely the best way to understand Flock is to use it. this requires two Arduinos and two radios obviously. FlockTestSender and FlockTestReceiver are a pair of matching minimal programs. there may be variants of these in the EXAMPLES folder in the distribution by the time you read this.

MESSAGE SOURCES (EVENTS)

messages are in fact the most fundamental form of communication, computerized or otherwise. a message, any message, human or machine, selects between predetermined choices. a message does not fully contain meaning, a message only means something within some (narrow) context. "TURN ON LED" assumes that an LED exists, that the recipient knows which LED is being referred to, and that "LED" means anything at all.

messages are most useful when they communicate change, and generally speaking, in Flock the implication is that you send messages when you intend to signal change, and through most of this conversation, messages convey commands and events.

but messages have another use, too, one that is very common, and simply not thought of as "messaging": a perfect example are commands on a command-line interpreter. the internet is quite literally a message-passing system. messages can convey anything that the sender and receiver agree that they mean. messages exchanged between -- or within -- a computer are fundamentally important.

Flock messages are a generalized way to communicate both events -- eg. "TURN ON LED" -- and relatively small amounts of data, which i call commands, or configuration. it simply turns out that a command (letter, number) system works well for both.

events containing a bit of nuance -- eg. "SET LED BRIGHTNESS TO 7" is still an event, probably different in effect, if not kind, from "SET LED BRIGHTNESS TO 251". both are change; both are events.

MESSAGE HANDLING

message reception requires the use of a "callback" function, the Dispatcher referred to in the introduction section. while Flock handles all reception, error handling and other overhead, Flock can't know what you want to do with the individual messages; it can't know what each message means. this is the purpose of the dispatcher.

the dispatcher is very simple: a switch..case statement, or an if..then..else chain. whenever the SRMessage object is given a command string to parse -- the message arriving from any source (USB, serial, EEPROM, text string or Flock radio) -- your dispatcher is called with the (letter, number) pair. your code can do anything with that data, such as store it or act on it immediately (turn something on/off, etc).

the key here is to think in terms of events, not data, not software, but events -- physical acts that may turned into symbols and sent somewhere, over wire or wirelessly. as little as a single letter can signal an event.

Flock messaging code reduces the abstract concept of a message to a simple pair of datums: the command letter and numeric payload. and often the mere presence of the letter command is sufficient.

consider a simple but reasonable application: an installation contains a pushbutton switch (or motion sensor, etc) a change in which must be communicated somehow to a desktop application across the room (that may play a video or sound, etc). with the pushbutton switch or sensor surrounded by an Arduino, nRF24L01+ module and Flock, the switch press -- an event -- is tranmitted to a Flock Base attached to the desktop computer, which receives the message and performs whatever action is necessary.

to be clear, the continuous state of the switch or sensor, whether it is open or closed/on or off, is not an event. in this hypothetical installation, it could be hours or even days before someone enters the room and presses the switch. switch on or off is not "interesting" and continuously sending the switch's state is actually a bad idea; and it doesn't solve the problem of communicating that someone pressed the switch. it is crucial to write code that detects the change in status of the switch, and transmit only that change.

PERFORMANCE/RESOURCE CONSUMPTION

on an ATMEGA328 processor at 16MHz, the Flock code takes anywhere from 150 microseconds (99% typical) to an average-worst-case peak of 2 milliseconds (channel negotiation and certain errors), measured on an Uno.

exactly none of the Flock code blocks, ever. the invokation of the top-level Flock state machines, Flock.message() and Flock.packet(), will return quickly -- average-worst-case is 1 mS on an ATMEGA328 to run the complete protocol stack. a "lightweight" program like the Flock test-sender and test-receiver run over 6000 loop() iterations/second. on a lightweight program Flock consumes about 400 bytes of RAM including statics and runtime heap. a relatively heavyweight program running on an 8MHz, 3.3V Pro Mini runs over 2600 loop()/second, with an average loop time of 375 microseconds.

SPECIFIC PROJECTS

i have installed Flock and nRF24L01+'s in a number of projects with great success. a sampling of these is below, and some will end up on github (see download section).

the initial impetus for Flock was for the Meow Wolf bird flock which are now complete.

i am developing the Meow Wolf boxes into more fully standalone, solar-powered devices, using the same Adafruit Mini FX Soundboard, a 3.3V arduino clone, an RGB LED, motion sensor, temperature sensor and radio initially to stick outside as yard lighting. this is also the test platform for the Peep protocol, with which radio power consumption is down to approximately 1mA average. the outside ambient temperature data will be used by a larger, solar-powered temperature control homeostat system for my lab building. sitting out in the yard it reports operating status (mainly LiPo battery state) via Peep to a flockBase in my lab. here the ugly prototype is sitting on towel outside. it runs adequately with a "3 watt" cheapo solar panel getting 4 - 5 hours of sun per day.

a slightly different application for Flock is as a credentials dongle for my roadster (automobile) project. at the moment the car is running a more traditional instrument panel while i complete details on the fully software panel, seen here with the black box "wireless instrument panel" dongle. the USB cable is for power, upload and logging; the "dongle" runs on an internal LiPo battery. the real dongle will be a tiny version minus LCD and plus one momentary switch and will be very small (true keychain dongle). the LCD dongle shown can lock/unlock the car and display all engine and chassis information delivered by Flock telemetry (those handy SR Messages) in real time, and can be use from anywhere near the car (eg. while tuning up under the hood). for this application the Flock protocol was tuned for extremely rapid convergence (50 mS) at the slight expense of increased radio traffic and power consumption; on the car side the 20 mA is insignificant, and the keychain dongle's "on" time will be under a second for each operation.

when i started the Flock project i cobbled up a number of sloppy test kludges to play with. those turned out to be very useful. the modules are small enough to successfully dangle from #22 solid wire soldered directly onto header row pins. this has been adequate for Flock Base modules, which i generally install in a simple plastic box with an milled hole for the USB cable.

this unpackaged Uno with a "high power" plus preamp version, below, is my desk-bound Flock/Peep base test rig, and listens to the test flock. there exists Processing code that searches attached USB devices to locate the base and displays Flock telemetry data, and allows configuring individual birds or the entire flock at once.

for an instrumented gesture glove i needed actual computing horsepower, so chose a TI Launchpad based upon the TM4C123XL (automotive) processor. it's inexpensive ($13) and comes with a fully ported Arduino IDE, compiler, libraries, etc. it's quite great. my nRF24L01+ driver works fine on it (last i checked). early Flock protocol did too, i just haven't checked recently, and no inherent reason why it won't still.

the modules come in a number of physical and electrical configurations. the most common is the simple module above with integral antenna. there are a number of enhanced modules, some simply with external antenna (2dB gain, but vastly better than the lossy serpentine PCB antenna) and some with additional LNA and PA. and very many modules from China are sadly clones of poor quality, and some outright fakes that do not work. DO NOT BUY THE EPOXY-BLOBBED VERSIONS. to the left is a version with LNA and PA mounted underneath an inexpensive proto shield, RF connector sticking out. these fit very nicely in common plastic project boxes from AllElectronics.

radio is really applied physics, and field testing is mandatory. to that end i've made this test instrument, aka the pinger. it is a Flock Bird client, has multiple LEDs to display send/receive, indicates signal strength, and optionally transmits battery voltage as test data. it's functionality will expand shortly. the code for this is on github, and shortly will have it's own page with schematics and such. the box has a socket that accommodates the most common (and inconvenient) pinout. with the high rate of junk parts from China a test box was mandatory, and it also allows field-testing the performance claims of the various modules. shown is a selection of the modules i've used. i've only received one batch of non-functional parts, and due to AliExpress's payment system i received a full refund. and there are plenty of high quality sellers (hint: do not get the absolute cheapest, look at feedback, and never get the epoxy-blob modules. and the real Nordic chips have laser-etched part numbers!)

DOWNLOADS

all of my code is on github, http://github.com/tomjennings. you will need to get the SRResources library as well as Flock. SRResources is fairly lightweight, and only the components needed are loaded at compile time.

Arduino libraries are not real libraries; they are source-code include files. "real" (sic) libraries are pre-compiled and exist as object modules linked at linkage time. the Arduino system is peculiar, and has advantages and disadvantages. i say all this to warn you that i take advantage of this peculiar system in a way that violates the recommendations but is quite "legal" -- my source code completely exists in the .H files only. i see no reason to duplicate the .cpp file nonsense when they're all includes to begin with. if this comes around to bite me in the ass it will be trivial to reconfigure. in the mean time, enjoy the simplicity.