version 2.2.3

The Adaptive Use Musical Instruments (AUMI) software interface enables the user to play sounds and musical phrases through movement and gestures. This is an entry to improvisation rather than "hitting the right notes" or playing set pieces of music. Instead, the software uses music as a way for participants to express a range of musical effects, both by themselves and in response to, or in conversation with, others.

The original focus of the AUMI interface was on children with profound disabilities and their teachers, therapists, caregivers, and families. In taking these participants as its starting point the project attempts to make musical improvisation and collaboration accessible to the widest possible range of individuals and communities. This approach also opens up the possibility of learning more about the relations between ability, the fullest possible range of bodies and minds, creativity and improvisation, from within a cultural context that does not always acknowledge or accept people with disabilities. The AUMI user base includes adults with and without disabilities, seniors, participants of independent living centers, artists with and without disabilities, and participants of mixed-ability, cross-generational community jam sessions, and many others.

AUMI continues to be revised and improved with feedback from anyone who uses it. We want to hear from you. Send feedback to info@aumiapp.com .

AUMI for iOS was developed for the Deep Listening Institute by Henry Lowengard, and adapted from the desktop version of AUMI, which is also available for download from aumiapp.com

AUMI websites:

• The Official AUMI for iOS Web site is at http://aumiapp.com.
• AUMI also has a presence on Facebook at :https://www.facebook.com/AdaptiveUse.

You can get software support by emailing: info@aumiapp.com

This link points to a PDF copy of these instructions. You can export them to iBooks or print them out for reference.

AUMI was developed with the support of the The Gould Family Foundation, Improvisation, Community, and Social Practice (ICASP), and International Institute for Critical Studies in Improvisation (IICSI).
Further AUMI development is supported by a grant from the Craig H. Neilsen Foundation.

This is AUMI for iOS version 2.2.3

Screenshots used in these instructions are taken from various models and simulations of iOS devices and may not look exactly like what is displayed on your device!

AUMI for iOS is also available for MacOs 12.4 and later, and on Apple Silicon devices in a somewhat untested but workable capacity.

This section is a broad overview of setting up and playing AUMI. Many more details are covered in specifically related sections.

SETTING UP THE AUMI PLAY SPACE
AUMI uses either the front- or back- facing camera to track motion to control which sounds are played. For this to work optimally, good lighting, a neutral background, and a steady support for the iPad, iPhone, or iPod Touch being used are needed. The more neutral the background, the better AUMI will be able to find intentional motion. The Apple iPad folding case is a pretty good support, but you can also use commercially available stands and mounts, and in a pinch, some big rubber bands, a chair or a pole, and some creativity will also do.
Good lighting means your subject is not backlit, which will obscure details that need tracking for motion and face tracking and dulls colors for color tracking. Make sure there is no prominent coloration in your lighting, which can happen if the AUMI player is shadowed by a colored tent or window.
By turning on the "See Spots" feature of the Motion tracker, you can visualize what that tracker thinks is "interesting" and make adjustments so that an AUMI player's motion is emphasized. More hints for this are covered in the Motion Tracker section of these instructions.

PREPARING YOUR DEVICE
AUMI runs on all iOS devices - iPhones, iPod touches, iPad, iPad minis, iPad PRO - running iOS 9.3 and above which have cameras. You will need to confirm that you want to use the camera when a permissions dialog box appears the first time you use AUMI.
For a better experience, you may want to:

• Adjust the screen brightness for the ambient lighting in the room
• Turn off gestures that relate to multitasking and the dock (iPad only). This is controlled in the Settings app, under "General", then "Gestures", then turn off the 4 and 5 finger swipes. Sometimes I find it useful to switch back and forth to MIDI instruments running in the background, but usually, multi-touch gestures are used to play and control AUMI and shouldn't be misinterpreted!
• Turn off keyboard clicks. In the Settings app, look for "Sounds" then "Keyboard Clicks"
• If it's an iPhone or iPad with cell service, and you are in a performance situation, go into "Airplane mode" and it won't spend time looking for cell service or WiFi, which AUMI doesn't need for performance. You will need networking if you intend to download new AUMI Instruments, though.
• Some MIDI instruments may need to be started before you start AUMI if you want AUMI to play them.
• AUMI supports "Dark Mode" in iOS13 and beyond. These instructions mix light mode and dark mode screenshots though.

PLAYING AUMI
You can play AUMI right away when it starts up, or by using the Play AUMI tab bar button to change the screen to the play screen. You will see a live video image, a circle with a red dot in the center (the Cursor), and a number of rectangles (the "sound_boxes").
The Cursor moves when AUMI detects movement in the video image. When the Cursor enters a sound box, a sound plays. You can put the Cursor anywhere on the screen with a single touch, and play AUMI with a single touch to move around the Cursor just as you can with video tracking.
The sound boxes can be rescaled by touching the corners of the box that encloses them with two fingers.

Two important buttons are also on the play space:

• The SILENCE button will immediately silence all sound and MIDI events. It toggles, so it will remain silenced until it its tapped again. On a desktop, you can use the space bar to do this!
• The RESET button loads the setup named Default so that AUMI can be put in a predictable state. You can save any setup as "Default" and it wil use that. If you erase the setup named Default, one will be created that uses a motion tracker to play the Piano.
  

CONFIGURING AUMI
There are four major sections that configure AUMI's processing and UI:

• INSTRUMENTS is where you choose an _instrument, as well as the number of available notes, polyphony options, scales, runtime parameters like volume, cutoff time, and velocity, and MIDI parameters.
  This button is on the main video screen for iPads and in the tab bar on smaller devices.
  On iPads, as a reminder, the name of the current instrument swaps with the Instruments button label every 5 seconds.
• CAMERAS & TRACKERS Settings selects the camera to use and its zoom amount, which tracker to interpret the video stream, and parameters to refine that tracker. Trackers are algorithms that detect different kinds of motion that moves the Cursor. There are a number of adjustments that can be made to the tracker software that can help it work best for your device and the environment it will be working in.

• TIMING & CHORDS, can turn a single choice of a sound box into a specific rhythm and chord combination. This can help make a continuous texture of sound to improvise over. These controls can solve some problems with getting an intended sound choice to be played in a specific rhythm.
• LOOKS controls the placement and colors of the interface elements on the play screen.

A Setup stores all the parameters for an AUMI session: the instrument, its looks, a fully configured tracker, and any timing and chords that are to be used.
Setups are maintained by using the Setup screen.
This includes naming, renaming, loading, deleting, and sharing setups either in a peer-to-peer fashion or as a shared document.

AUMI has the ability to store some data associated with its use, and export this data as CSV files to be used with reporting software. This reporting is associated with the Setup name, which could be set to name a user and a session name for particular tests, and other metadata useful in data in making the reports.
By default, all reporting is disabled. If you are considering the use of reports, please see the LEGAL document. If you are not interested in reporting, you can leave the metadata information spaces blank.

The name is also used to identify saved instrument _setups. If you have already saved some setups, you can load one from a list with the "List Setups" button.

The Setup page in no way affects the ability to play AUMI, nor is this information sent to any server.

AUMI can refine the way it turns the Cursor position relative to the sound_boxes into sound. Read the _Instruments section to find out more.

Some AUMI users are only capable of extremely limited voluntary motion. AUMI can be adjusted to accommodate their needs in a number of ways:

• You can rescale the sound_boxes by "pinching" the screen. If you put a lot of sound_boxes in a small space, smaller motion is more likely to initiate a sound!
• You can zoom the camera in to get faster response. It's not really "zooming" as much as taking an excerpt of the middle of the image, but that may help AUMI find the motion it needs to play sound.
• The "Smooth" slider smooths out the cursor movement if it seems too wiggly!
• The "Mult" slider multiplies the amount of motion that the motion tracking software finds, so that it can move the Cursor more with each motion. Even an eye blink or a tongue movement can control the music.
• AUMI can make the timing of its note events regular so it can play with a regular beat.See the TIMING & CHORDS configuration section
• You can get feedback on what AUMI's trackers are "interested" in tracking by turning on the "Show Spots" (or "Show Areas" if using color tracking) button in the VIDEO CAMERA AND TRACKER SETTINGS screen.
  For the Motion tracker, the screen will display little labels at the positions of the tracking_spots it has found. The blue labels are points that have moved enough to be considered for moving the Cursor, while the paler ones are currently being ignored until they move more later.
  For the Color Trackers, areas with the selected colors are shown as a paler version of itself.
• AUMI can play over other music making programs and synthesizers if they can play in the background. Improvise with AUMI over a Music track!
• AUMI's' MIDI playing feature allows a user to play anything that understands MIDI "Note on", "Note off", and "Program selection" commands which are not only synthesizers and samplers, but lighting consoles and other media devices. AUMI can control other MIDI instruments on a laptop connected via Wi-Fi, for instance, in Apple's Garage Band, and also other MIDI apps running on the same device. When hosted in iOS based hosts, those other apps can add effects, recording, and sharing features missing from AUMI. That said, AUMI is not an Audio Unit (AUv3) plugin or Inter App Audio (IAA) provider.
• The Color Tracker is very fast and precise. If you make a circle layout and use a scale of your choice, you can actually play a melody. Using the timing feature can help keep the melody regularly timed.

As of iOS11 and Yosemite (Mac OS X 10.10) or later on a Mac OS system, you can record directly from a connected iPhone or iPad by using the QuickTime Player app. Simply select "New Movie Recording" and in the on-screen control, pick the devices name in the drop down menu (the "v") for both video and audio. Then record and trim as usual.
If you have another kind of computer that can record from an HDMI stream, you might be able to do it by using hardware: an HDMI adapter and cable to a video recording device. Modern iPads and iPhones can transmit video and audio over WiFi to an AirPlay(™ Apple) equipped device, such as a flat screen TV or video recorder.
In MacOS Ventura and later, there is built in AirPlay emulation on your Mac. See "AirPlay Receiver" in the "AirDrop & Handoff" section of the System Setting menu. There is also AirPlay emulating software for older versions and other operating systems: the device transmits video (and audio) over to a computer that can pick it up and record it, using a program called AirServer, among others. Sometimes it's a little finicky, but that's how I've made some earlier videos of AUMI.

I'd suggest just using an external video camera to record both AUMI and the real performer using it, which gives a better idea about what is happening!

This is a quick overview of the toolbar and interface buttons. Click on them for more details!

MAIN TOOLBAR
	Play AUMI The interactive video interface.
	Instruments Pick _Instruments, set the volume, pick a scale and other instrument related features. On iPhones, this is in the toolbar, and on iPads, this is accessed from the side menu. On iPads, as a reminder, the name of the current instrument swaps with the Instruments button label every 5 seconds.
	MIDI Configure AUMI's MIDI playing features. On iPhones, this is in the toolbar, and on iPads, this is part of the instruments dialog box.
	Instructions Leads to these instructions
	Setup Load and Save AUMI settings like scales, instruments, video and timing, send and receive Setups to other devices, set logging metadata, enable logging to create reports, and read and export reports.
AUMI PLAY PAGE TOOLBAR
	Looks Change aspects of how AUMI looks, such as colors, sound_box layouts, and sizes. This is launched from the "Looks" "button at the left of the main video screen.
	Instruments Pick _Instruments, set the volume, pick a scale and other instrument related features. On iPhones, this is in the toolbar, and on iPads, this is accessed from the side menu. On iPads, as a reminder, the name of the current instrument swaps with the Instruments button label every 5 seconds.
	Sound Order Assign sounds to sound_boxes. This is launched from the "Sound Order" button at the left of the main video screen. This button does not appear when using melodic instruments, which order sounds by using scales.
	Video Camera and Trackers Settings Change the size, resolution of the video cameras, choose tracker, and adjust tracking parameters.
	Configure Timing & Chords Change the options and speeds of AUMI's timing quantization feature. It also configures chords and arpeggios.
	SILENCE or CLEAR MIDI Immediately stops the playing sound, whether it's from AUMI or via MIDI. It will rename itself to "RESUME", and a "SILENCED" message will appear on screen. Tap it again to resume playing. If you have a physical keyboard, on a Mac or via Bluetooth, typing a space will act like this button!
	RESET Will load the Setup named Default. This will help you reset the system to a known state. You could also go to Setups and load any other Setup you like, but this is right there on the play screen. If there is no setup named Default, one will be created that is a Piano with a Motion Tracker. Touching this button will show an optional notice that warns you that it's about to reload the Default setup. You can cancel that action with "DON'T", approve it with "YES", or keep the optional notice from coming back with "Don't show this notice again". The notice will time out after 6 seconds and not approve the action. If the notice is set to not show again, it will approve the resetting action.

AUMI works on iPhones and iPod touches, but is best on iPads, iPad Pros, and iPad Minis. Newer models will be faster, especially for tracking!

AUMI occasionally puts up notices which can be dismissed by tapping the "done" button. Most of these informational notices go away by them selves after two seconds, and the "optional" notices go away after 6 seconds. Notices about movement and event reporting, though, require a manual confirmation.

On the Setup Screen, you can save and load _instrument and other configuration data, which together is called a _Setup , by using the "Save Setup" button and the list of already saved setups.
The Save Setup button saves the current instrument and its layout, colors, and sound order to a file named [setup name].aumi. It is in Apple "plist" format, related to XML.

• Which category of instrument it is as seen in the instrument menu, by using icons
• The instrument name
• Which scale it's using if it's melodic
• Which tracker it's using
• Which timing style it's using, if any
• How many steps or echos it's using, if any
• Which chord type it's using, if any

Listening and Sending
AUMI can be configured so that you can send a Setup from the device it's running on to another copy of AUMI running on a different device. This allows one device to change aspects of another device without needing to touch the screen.
This feature works even if the receiving device has a screen in Installation Mode that can't be easily unlocked.
Listening and Sending is using Apple's "Multi-peer Connectivity" feature. That means all the devices involved should be on a local network, or have Bluetooth enabled.
You can designate a device to listen for setups by touching the "Listen for Setups" button. You can actually do this with up to 7 devices!
AUMI Devices normally have very generic names, which is not very useful. You can now name your AUMI device, for the purpose of listening and sending, in a field on the right side of the Setup screen.
On another device running AUMI, you can then send the current Setup on that device to the listening device(s) by touching "Send Setup". You'll be presented with a list of "nearby" devices that are listening as it discovers them. Touch the device names of the devices you wish to send the Setup to. It should connect and change the Setup on the listening device, and put up a notice that tells you so. If it fails for some reason, there will be another notice telling you that.

AUMI can tell the listening device to load an AUMI Instrument that it doesn't have if it knows where to get it from. "Send Setup" has either a "smiley face" or an "X smiley face" next to the name if the Setup is using a user loaded instrument. If that instrument is an "AUMI Instrument" file loaded from a website, there will be a "smiley face", meaning that the receiver can attempt to download it from the original source if it's not on the device already. If there's an X, that instrument cannot be sent. AUMI will not send audio data in a Setup, it only sends the URL of where to find audio data. If you send a setup that has User Sounds in it, it will work if the receiving device has those sounds installed under the same name. If the sender is sending an AUMI_Instrument that was loaded on that sending device, it will tell the receiver to load that instrument (because it remembers where it was loaded from), but otherwise, it will use the Piano.
Once the setup is received, it can be saved on the device that received it by tapping "Save Setup".

Another way to distribute setups is to use the "Share Setup" button. This is a way to send a Setup through messaging, email, AirDrop, or third party tools that share documents. Since they are special temporary documents, they are named with the suffix "_EXPORT.aumi". These in turn can be collected as setups in a cloud-based directory, or on a website, so that people using AUMI remotely together can easily load Setups used for their performances.

Reports
The Setup page is also where those who are using the reports feature may set up the metadata used in recording activity during an AUMI session.
The report button brings up an interface for exporting the AUMI report files. These files are a comma separated value files (CSV), supported by most spreadsheet software. See the reports section for more details. See the legal section prior to enabling the reports feature.

This is the main interface to be used while playing AUMI (iPad version).

Over a live video image, AUMI shows a set of rectangular "sound_boxes" and a circular Cursor, which is what initiates sounds as it moves into the sound_boxes.
You have the choice of either showing the live video on the screen, or replacing it with a neutral background. Some users prefer not to see themselves while playing, while others enjoy the visual feedback of watching their movements interact with the Cursor and Sound Boxes. Others go back and forth, sometimes turning off the video to reduce the distraction of trying to make a movement make a particular sound, in order to focus more on listening, or to turn the attention of a group from watching individual screens to relating to one another.
You can also set the position of the Cursor by touching the screen, and use this method exclusively for controlling AUMI. There is a tracker designed solely for use by touches, that does not use video at all, called the Touch Tracker.

For most instruments, when the Cursor enters a sound_box, the corresponding sound is played, and the box highlights relative to the volume of the sound that is playing.
For Loop instruments, the nearness of the Cursor to a Sound Box controls the mix of a sound loop playing in that box.
For Relative Instruments, the relative motion of the Cursor chooses which sound to play.

AUMI's Motion Tracker works by detecting and tracking the movement of "interesting" parts of a video image. "Interesting" in this case usually means high contrast and angular shaped parts of the image. The Motion Tracker works best when there is clear lighting, and a blank background behind the user.

The relative motion of the tracking_spots moves the Cursor on the screen. The Cursor has no direct relationship with the tracking_spots. This allows any motion it detects in the image to control the Cursor, and thereby initiate the sounds.
There's a toggle button, "Show Spots", ("Show Areas" for Color Trackers, "Show Poses" for the Pose Detector) in the Video Camera and Tracker Settings screen, that makes the tracking_spots visible or other visual debugging feedback when clicked. Click it again to hide the spots. On a desktop, you can yse the Tab key as a toggle for this
For the Face Tracker, little eye and mouth icons show the position of the eyes and mouth that it finds.
For the Color Tracker, you can see the areas that are the selected group of colors highlighted in paler versions of themselves.

The Face Tracker detects faces with machine learning and put the Cursor right on the face's nose. It's not as flexible as the motion tracker, but it is pretty fast - which might make it the right tracker for your situation!

The Color Tracker detects colors and put the Cursor in the middle of where the largest block of that color is onscreen. While it's very fast, you need to select the color to track by double tapping the screen and sliding your finger around to pick up variants of that color.

The Distance Tracker is the same as the Color Tracker, except the Y axis is replaced with value derived from the area of the largest block of color. That roughly corresponds to distance, and so morion towards and away from the screen (camera) can be used!

The Touch Tracker uses just a single touch to play the screen, with no video used at all. The video image is replaced with the background color.

The Pose Detector looks for video to match pre-trained images, and if so, it moves the cursor to a corresponding sound box.

You can change the size and shape of the rectangle the sound_boxes are laid out in by touching the screen with two fingers. The touch points move the closest corners of the top and bottom of the containing rectangle.
The layout of the rectangles can also be changed, except for Loop and Relative instruments, by using the Looks screen controls. The layouts are Horizontal, Vertical, Gridded, and Circular. The Gridded and Circular layouts enable users to play notes out of their usual order.
The Looks" screen controls can change the color, transparency, and thickness of the sound_box outlines, and the color, transparency, and size of the Cursor.

On an iPad, the "Instruments" button appears on this screen, while on an iPhone, that button is on the navigation bar.

There is a big red "SILENCE" button in case you want to stop the sound that's playing and clear the queued Timing based notes. The trackers and sound will be disabled and even loops will stop playing. This is called "Silent Mode". When you tap it again, the trackers will work again and the loops will play. While in "Silent Mode", the button label will change to "RESUME"
If the instrument is a MIDI instrument, this button will change its label to "CLEAR MIDI" and will silence MIDI notes.

To reset AUMI to a predictable configuration, tap the "RESET" btton. It actually loads a Setup named "Default". "Default" is automatically added to the list of Setups, but you can save any setup you like under that name, and the Reset button will load it.

You can hide and show all the toolbars, and make a completely clean (and safe to touch) screen by tapping with three fingers.

Note: in iOS 13 and on, a three finger tap is now interpreted as a part of a cut, copy, or paste operation. To do a three finger tap that will be recognized in AUMI, you may have to put each finger down slowly (one-uh-two-uh-three-uh) for it to be detected properly. This was fixed in iOS 13.1 and later!
When AUMI is used in a public installation there's a chance that you may want to lock its control interface. One way would be to put it behind glass, but if the device is exposed, it needs another method.
There is an Installation Mode which locks the screen from revealing the toolbars. This is enabled and disabled by touching down and holding two fingers - possibly resizing the Sound Boxes' bounding rectangle - while moving a third finger a long distance of more than 100 pixels. When enabled, it prevents a "normal" three-finger tap from revealing the controls, and also disables a four-finger tap from locking or unlocking the size of the bounding rectangle. This is designed to be a little obscure! It also locks the Sound Boxes' rectangle size, even when you go back to showing the normal interface.
There is an optional notice that warns you that it's about to go into installation mode. You can cancel that action with "DON'T", approve it with "YES", or keep the optional notice from coming back with "Don't show this notice again". The notice will time out after 6 seconds and "Not approve" the action. If the notice is set to not show again, it will approve going into installation mode.

On iPads, the Instruments screen is accessed with a button on the left of the Play AUMI screen. On iPhones and iPod Touches, it's in the tool bar at the bottom of the screen.
The Instruments screen is where you select an instrument and its properties, and set the general volume, and a few other options.
A big scrolling instrument picker is divided into five parts. Some instruments don't use all the parts, so they are blanked out.
Instruments are collections of sounds that AUMI can play in a number of ways. AUMI comes with a good variety of sounds, some conventional, some unconventional. AUMI does not currently play audio though Apple's Inter App Audio or Audio Unit interfaces, nor with Audiobus, but it can send MIDI to apps which do support these features.
Instruments are categorized into sections, each with its own icon and background color, accessed by the scrolling control on the far left. The instruments associated with those categories are correspondingly colored.
You can search for instruments within those sections by typing in the search field. It will match that string in the name and description of the instrumnet. If there are more matches, you can use the ">" button to get to the next one.

• Melodic instruments (green): They have a range of chromatically tuned samples that can be used to play melodies. Most melodic instruments have a range from MIDI note 36(C2) to MIDI note 96 (C7).
• Sample or percussive instruments (blue): They simply play sounds that are not tuned or in any particular order. The order can be rearranged by using the Sounds button.
• Looped instruments (yellow): All the sounds play simultaneously in a continuous loop. The closer the Cursor is to a sound's sound_box, the louder that sound will play. If you make the size of the Cursor bigger, it will cover more sounds and you will hear more of them simultaneously in the mix. The choices for loop sounds and the number of them on the screen can be changed by using the Sound Order screen. There is a maximum of 32 sounds.
• Relative instruments (light purple): These sounds are played by using the relative motion of the Cursor, instead of its absolute position. The sounds are the same as all the melodic and all the Sampled sounds, and can be selected by using the Sound Order screen. However, only four sounds can be played.
• MIDI instrument (blue): Makes no sound by itself. It sends MIDI messages to other programs or devices.

• Melodic User instruments (green): like the built-in melodics, they can be organized into scales and tuned.
• Sampled/Percussive User instruments (blue): like the built in samples, they can be reordered in the same way.
• Looped User instruments (yellow): like the built-in loops. This uses the non-melodic sounds. You must have at least 4 sounds for looping to be enabled. They can be reordered to choose which sounds to loop by using the Sound Order screen. There is a maximum of 32 sounds.
• Relative User Instruments (pink): like the built-in Relatives. You must have at least four sounds for this to be enabled.

• An instrument selector. Each instrument name has a small image in front of it to help identify it.
  
  The instruments are:
  • Melodic sounds: these can use scales in different keys.
    For reference: MIDI note numbers for these pitches: c2=36, c3=48, c4=60, c5=72, c6 = 84, c7 = 96
    
    • Piano: piano, C2-C7
    • Guitar long: synthetic strings, C2-C7
    • Guitar short: synthetic strings, shorter than before, C2-C7
    • Tines: a sampled music box, C4(60) to C7(96)
    • Reedy: synthetic reed instrument, C2-C7
    • Bowed: synthetic string instrument, C2-C7
    • Fifths: Looping Fifths, pulsing every second, C2-C7
    • Steel Drum: C2-C7
    • Kalimba: C4-C7
    • Synth Drum: C2-C7
    • Woodblock: C2-C7
    • Orchestra Hit: C4-C6
    • Sitar: C2-C7
    • Timpani: C2-C4
    • Tubular Bells: C2-C7
    • Glockenspiel: C5-C7
    • Tenor Recorder: C3-Ab4
    • Hand Pan: C3-B4, C5-C6 have harmonics (Thanks, Jesse Stewart)
    • Singer Henry: A2-D5
    • Singer Eleni: D3-D6
    • Choir: a mixed choir C3-C6
  • Non-Melodic sounds: these can be arranged in any order, but do not have scales.
    • Percussion: sampled percussion sounds.
    • Noises: experimental rock band noises (Dr. Nerve's Nerve Events).
    • Kitchen Percussion: sounds of pots and pans (The Institute of Contemporary Music).
    • Cartoon Sounds: cartoon sound effects. (Screwy Squirrel, MGM)
    • String Quartet: The Soldier String Quartet and vocalist Shelley Hirsch.
    • Dogs: Dogs barking (http://www.freesfx.co.uk).
    • Cats: Cats meowing (from a YouTube compilation).
    • On The Farm: Farm animal sounds (https://freeanimalsounds.org/farm-animals/) .
    • Waterphone: Waterphones from Jesse Stewart's Xenophora: Music for Solo Waterphone.
  • Loops: these are a circle of continuous sounds that can be mixed together.
    The closer the Cursor is to a box, the louder the loop it represents will play.
    This way, slow movements can still play along with other musicians playing loops as the same rate, although some loops are more ambient and do not mesh synchronously with other loops.
    There can be from 4 to 32 sound_boxes to play with loops. You can adjust the number with the slider in the Sound Order screen. The sounds are laid out like a clock, atarting from the top of the circle and choosing the next sound in clockwise order.
    • Loops: An instrument where the mix of 2 second long loops is controlled by the Cursor position.
    • Marsh Loops: The sounds of a marsh, from Dave Soldier's Marsh Fugue
    • Fifth Loops: Looping Fifths, pulsing every second which are arranged so they harmonize
    • Tran Phu: Recorded in Vietnam by Kala Pierson
    • Pin Pia: A traditional Thai instrument recorded by Dave Soldier
    • Synthy Loops: A collection of 2 second electronic loops.
    • LP Lock Grooves: A collection of loops that will synch with vinyl lock grooves at 33 1/3 RPM. (1.8 second loops)
    • Glyptodont: Sounds from a recording of Skip La Plante's Glyptodont.
    • Troglodytes: Sounds from Deep Listening Band's recording Troglodyte's Delight.
    • Widow Jane Mine: Tenor Recorder played in the Widow Jane Mine.
    • Waterphone Loops: Waterphones from Jesse Stewart's Xenophora: Music for Solo Waterphone.
    • Leaf Drums: Drumming loops from Leaf Miller. 110 BPM..
    Loops can be made up of sounds that are exactly the same length, or they can be different lengths. When they are all the same length, the loops will play together (be "in synch") and not move in time from each other (or "phase"). If they are different lengths, the will not stay together and get mixed up pretty well. Sometimes this is a good feature!
    However, sometimes you do want your loops to be in synch all the time, and all be a particular length. You can make this happen by setting the (|♪|) Monophonic Whole polyphony switch on, and selecting a length from the "BPM" time in the Timing Menu. Each sound will the be stretched to fit that new length, up to 3 seconds (20 BPM). If the new timing is pretty near what the sounds usually are, they will sound fine, but if not, they will sound distorted, which might be something you want.
    With some cleverness, you can get different Loops on different devices to stay in synch with each other, and stay in sync with similarly Timing quantized percussive, MIDI, and melodic instruments.
  • Relative Motion: All the sample playing instruments - melodic or not - can also be used in relative motion mode. Relative Motion versions of these instruments are marked with a "REL" sign at the end of their names. Relative motion uses the tracker to determine whether the motion was left, right, up or down, and plays a sound corresponding to that motion. The same Sound Order screen is used to assign sounds to the directions (in the order Down, Right, Up, Left). "Relatives" are limited to four sounds.
  • MIDI
    • MIDI: This will send MIDI events to either a connected MIDI device (via the "Camera Connection Kit" or other MIDI interface), or to a software MIDI synth running on the same device or within the network.
      Read more in the MIDI section.
  • User Instruments
    Adventurous users can add their own sound files to AUMI to be turned into instruments. An aesy way to do this is to touch the More button. Read the User Sounds section for details.
• The base note, if the instrument is melodic. This is like the tonic of the "key" of the selected scale. Note that many of the scales are a little more complicated than usual major, minor, and chromatic scales.
• The kind of scale to use if the instrument is melodic:
  • Chroma: a chromatic scale
  • Major: a major scale
  • Penta: a pentatonic scale
  • Minor: a minor scale
  • Maj Triad: a major triad I-III-V scale
  • Min Triad: a minor triad I-IIIb-V scale
  • 7th: a seventh I-III-V-VIIb scale
  • up down: a melodic scale that goes in a pattern of up a major third, up a minor third, down a fourth, and up a minor third again.
  • Blues: a bluesy scale
  • Pelog: a beautiful scale based on Pelog Selsir, a Javanese scale.
  • Madenda: a scale based on Madenda, a Javanese scale.
  • 
    Modes: diatonic scales that start on other notes. Examples given in the key of C.
  • Ionian: Ionian mode. C D E F G A B C
  • Dorian: Dorian mode. C D Eb F G A Bb C
  • Phrygian: Phrygian mode. C C# Eb F G Ab Bb C
  • Lydian: Lydian mode. C D E F# G A B C
  • Mixolydian: Mixolydian mode.C D E F G A Bb C
  • Aeolian: Aeolian mode. C D Eb F G Ab Bb C
  • Locrian: Locrian mode. C C# Eb F F# Ab Bb C
  • 
    Fifth repeating scales: like "up down", they don't repeat by octaves but by fifths. These names are to make it easy to refer to them, and are non-standard.
  • Sun: a major second, a minor third, a major second
  • Earth: a minor third, a major second, a major second
  • Star: a major third, a minor second, a major second (like Pelog)
  • Moon: a major third, a major second, a minor second
  • 
    "Modern music" scales.
  • Whole: a whole tone scale
  • Jazz Minor: Jazz minor scale C D Eb F G A Bb C
  • Tone Row: Serialistic 12 note tone row, repeated in octaves
  • Tone Row D: Serialistic 7 note diatonic tone row, repeated in octaves
  • Tone Row P: Serialistic 5 note pentatonic tone row, repeated in octaves
  The assignment of sounds to sound boxes is built dynamically from the scale picked, base note (key) for that scale, and what sound samples are available for the instrument. They start on the "base note" and create the scale by adding a cycle of intervals to that base note.
  If that base note is lower than the lowest note available in an instrument (like C4 for the Tines), the scale will be played in the base note's key, but only using notes within the range of the instrument.
  If the scale is a "mode", and the base note you choose is lower than the first note available in that instrument, it will start the scale on that base note in the first higher octave that is available for that instrument.
  The Chord feature, found in the Timing dialog, takes advantage of the scales choices to build chords and arpeggios.
• The number of sounds to put on the screen in "sound_boxes". Loop Instruments can play from 4 to 32 sounds or the maximum number of sounds available if less than 32. You can also set the number of sounds from within the Sound Order control.

• Show/Hide Names: This controls whether labels show up in sound_boxes.
• Volume (Vol): The volume for playing internal instruments. If you are playing a MIDI Instrument, it will send a MIDI channel volume message.
• Cutoff Time: Some sounds play for a long time, and overlap each other. This can keep a user from hearing that their motion has started a new sound. By setting the cutoff time, the sound will stop playing after that time in seconds has elapsed. Cutoff time is roughly set in tenths of a second, and runs from 0.1 to 8 seconds. Setting it to 0 will let the sound play its natural length. You can set the time with the slider, and get more precise by using the "<" and ">" buttons, which change the time in 0.1 second increments.
• Use/Ignore Velocity: Normally, the Cursor's motion doesn't control the loudness of a sound. If this switch is set on, melodic instruments and sampled instruments will play more loudly if the Cursor moves into it faster!
  

  There are two sliders that set the range of the velocity. The Velocity High("High Vel" on iPhones) is probably more important. If you set it to a low value, it will take more velocity to make it play loudly (and it will be more sensitive!), whereas setting it higher means nearly any movement will be loud. The Velocity Low ("Low Vel" on iPhones) sets a level for the least velocity, so if this is higher than 0, you should always hear something, but it might be very quiet.

  Normally, MIDI instruments will use the direction perpendicular to the layout direction the sound areas to control MIDI velocity. If the velocity switch is set on, the speed of the Cursor will be reflected in the MIDI velocity of the played notes. Even when the Velocity switch is off, the low and high values are used as the low and high values of the sound boxes. You can actually set the "Low" value higher than the "High" and it will work backwards perfectly well!
  The setting for "volume" is also figured into the MIDI "velocity" value.
  When the Velocity switch is on, and using MIDI, the perpendicular axis is now used to send MIDI's Continuous Controller #1 (CC1), usually associated with a control wheel, at a rate of up to 30 times a second, even when not entering a new sound box. If AUMI is in Mono Mode, a note played in MIDI stays on until the Cursor leaves the box, so this way, you can control a MIDI based sustaining sound that responds to CC1.
• Polyphony AUMI has a number of ways to control how long a sound is playing. This affects Melodic, Percussive, and to a lesser extent, MIDI instruments.
  • (♫) Polyphonic Normally, when any Sound Box is entered by the Cursor, its associated sound plays until it's done or until its cutoff time is reached. This can make for a lot of chaotic sound!
  • (♪) Monophonic When monophonic mode is on, only a single sound plays at a time. However, a chord will play that may have been set up with the Timing control screen, but only one chord at a time. This works best if there's no quantization going on. Entering a new sound box will immediately cut off the sound playing and play the new sound.
    In MIDI mode, staying in a Sound Box will keep the MIDI note playing until you leave the box, that is it will ignore the MIDI Off Delay setting. This is a good way to play MIDI sounds hat have a lot of evolution or arpeggiation!
  • (|♪|) Monophonic Whole This is a variant of Monophonic where the whole sample always plays (or as much as the Cutoff time allows). While the sound is playing, it's not interested in starting new sounds. This is a very specialized way of playing, designed for playing whole musical phrases, sentences, or words or phonemes, one at a time, that need to be played in their entirety. It acts like Monophonic in MIDI Mode, since there's no way to know when a MIDI note is over.

The Looks screen controls lets you change the sizes, layouts, and colors of some interface elements.
Pick which element you want to change from the "Cursor", "Boxes", or "Backgr. (background)" selector button.

By touching in the array of colors, you change the color of the selected element.
The box under the Opacity slider will show you the current color setting.
The "Opacity" slider lets you change the color's opacity (transparency).
For Boxes, this chooses the color that the sound_box is colored. Boxes are always transparent, but you can make them more transparent with the Opacity slider.
The top line is purely black and white, the line underneth shows very subtle shades of color!
The "Size" "slider is used to change these sizes:

• For Cursor, you can change the size of the Cursor with the slider. It can be set from 70 to 300 pixels.
• For Boxes, you can change the size of the sound_box borders with the slider, from 1 to 20 pixels.
• For Background, (Backgr.) you can change the color of the background when you are not showing live video.

The Lock switch locks the shape of the layout. This disables the ability to rescale the bound box of the layout with two-fingered touches. This switch can also be interactively turned on and off with a four-finger touch, if the screen is not in Installation Mode. The best way to do this is to put one finger down on the screen and use the other hand to do a three finger tap! This way, you don't accidentally resize the layout.
Another way to toggle the layout lock is to resize as usual with two fingers, then keep holding them down until more than 3 seconds has passed. This is also true if the layout is already locked, just touching and holding for 3 seconds will unlock it.
When the layout is locked, the Looks button will show a Lock icon on it to remind you of that fact.

The Video switch lets you choose between showing live video or just using the background color. It has the same function as the "Hide Video" switch in the video camera and tracker configuration dialog.

The layout buttons select whether the sound_boxes are laid out horizontally, vertically, in a grid, or in a circle.
The direction arrow is used to reverse the order that the sounds are laid out in for the Horizontal, Vertical, Circular, and Semicircular layouts.

Vertical

Horizontal

Grid

Circle

Semicircle



The Box Width slider can be used to make the sound boxes skinnier, thus putting more space between them. This way, it's easier not to play an adjacent sound box by mistake! AUMI will play the sound if you cross over a box quickly as well, even if it's only 1 pixel wide, so being skinnier is not a disadvantage that way.

When you are using the MIDI instrument, and you are not using the Cursor's velocity, and if your layout is vertical, the position between top and bottom of the row of sound_boxes is used as the note's "velocity," so that near the bottom of the grid is "low velocity" and near the top of the row is "high velocity". For a horizontal layout, left is "low velocity" and right is "high velocity"." These values are taken from the low and high velocity settings in the Instrumnet control.

This screen is accessed by touching the "Sound Order" button on the "Play AUMI" screen.
The Sound Order dialog allows you to assign specific sounds to Sound Boxes. This is the underlying order of sound files that is then used to determine the files used in scales and otherwise associated with sound boxes.

• The slider at the top sets the number of sounds that this instrument will use, which will be from 1 to 121. On the iPad, you can see the layout of the sound_boxes change as you move it. If you "long press" on the number label, it will set the number of sounds to the maximum number available for that instrument, or if the highest selected sound's number! This makes it easy to allow a subset of sounds, namely , the first N of them.
• The Reset button restores the order of the sounds to what it was when the control was first displayed.
• You can change the order of the sounds by dragging the sound using the 'dragging' button on the right of each line.
• You can select any number of sound names and send them either to the top of the list or the end of the list using the appropriate arrow buttons. This makes it easier to move sounds you want to hear into place.
• You can select several sounds quickly by substrings in their name, in case the instrument uses a naming convention for their sounds. Type a string to be matched in the text field, and then tap "Select" or "DeSelect" to change the selection status of those sounds. This operation does not clear selections first, so you can accumulate an interesting, fine-tuned collection! Then you can use the "Up" and "Down" arrows to move them in and out of the way.
  
  
• You can insert a silent sound into this list, after the most recently selected sound, by touching the "Silence" button.
• You can duplicate the most recently selected sound (and later move it) by touching the "Copy" button.
• You can scramble the order of the sounds by touching the "Scramble" button.
• Advanced users can put in a list of numbers and it will choose the sounds corresponding to those numbers when you use the "Use Pattern" button. This is a shortcut for moving and duplicating sound files. For example "1 3 5 2 4 6 1". "0" means silence. If you end with "*" and a number, the pattern will repeat with that number as an offset. This number can be negative! It will stop repeating when all 121 notes are created or you go over the top number of sounds or below the bottom (0). With melodic sound sets, you can set up scales and little melodies with this!. Note: since the files are reordered, melodic instrument scales can be very scrambled!

This alternative order of sounds is saved with the instrument if you use the "Save Setup" button on the Setup screen.

MIDI controls are a separate screen on iPhones:


The MIDI screen is included as part of the Instruments screen on iPads.


If the current instrument is a MIDI instrument, you can use these controls to select a MIDI channel and MIDI program settings.

• You must enable and disable MIDI transmissions with the MIDI switch.
• You can select a MIDI channel to use with the Channel slider.
• Because most MIDI instruments are not percussive, and need to be told to stop playing a note, there's a slider to set the delay time until a "Note off" event is emitted. But if AUMI is set to be in monophonic mode, MIDI notes will stay on until the Cursor is in a new Sound Box.
• When a MIDI instrument is selected, the SILENCE button turns into a CLEAR MIDI button, in case there are stuck notes.
• You can select a program with the picker on the right. The names of the programs are the standard General MIDI names, and may not correspond to sounds that are actually played by the MIDI synthesizer you are connecting to.

1. Start AUMI.
2. In Instruments, choose a MIDI instrument. Pick a scale and note range too, if you like.
3. Make sure the MIDI switch is turned on.
4. Click the home button (on devices that have them) or swipe on the Home indicator (those without a Home button) to put AUMI in the background (it's still running)
5. Start an iOS MIDI synthesizer like Garage Band, Bismark BS-16i, KQ Dixie, Animoog, Thumbjam, or another MIDI compatible iOS app.
6. Make sure the synthesizer can play using background audio. This is usually a switch in the MIDI section of the app's settings and information section
7. Click the home button (on devices that have them) or swipe on the Home indicator (those without a Home button) to put the synth in the background, and click on AUMI's icon again.
8. It should be able to play the MIDI synth now.

AUMI is capable of generating two kinds of reports if this feature is enabled. If you are considering the use of the reports feature, please see LEGAL. These reports are in the form of CSV files. .

There are two kinds of events that AUMI optionally logs: Movements and Sound Box Events. These are enabled by switches on the Setup screen.

When you enable logging, you'll see a privacy notice in a red box appear with the text:

                AUMI collects usage data on this device when this
                switch is turned on, data which is only stored
                locally on this device.
                You can switch this option off at any time. You can
                also either export or remove this stored data.
                
                For more information, read the Records and Legal
                sections of the Instructions.
                

Saved Setups does not save the state of these switches. This is to make sure these switches are always set manually. This is true for remotely sent and initially loaded setups as well.

These fields open up when you choose to enable reporting. (iPad version) They are included in the csv files that the reporting makes, so that you can track several players (entered as part of the setup names) and programs on the same device. For instance, you could call a setup "Janet's Kitchen Percussion".

Movement event logging

When logging is turned on, a tiny logging notice appears in the upper left corner of the screen to remind you that it's logging.

When movement logging is enabled with the optional switch "Log Movements", every 15 seconds, a record is cut with some information that may be used to make reports in an external spreadsheet program like Numbers or Excel. This information may be useful for tracking the progress of individual AUMI users in different programs at various institutions.

The movement file is created with a name in the format AUMI-YYMMDD-HHMMSS.csv, so that the date and time are part of the file name, e.g.: AUMI---121206-092201.csv

Also, when the program transitions from foreground to background, it will also log this event. In this record, the timestamp and identifying information are retained, and the position related fields will be blank, but the Scale field will say FOREGROUND or BACKGROUND.

Sound Box event logging

There's a second kind of report that is enabled with the optional switch "Log Sound Box Events".
It tracks the entry and exit from Sound Boxes, and records the time spent in each one, and its file has a slightly different name from the movement report file:
AUMI-EL-[institution]-[program]-[date and time].csv

Reviewing, deleting, and exporting logs

The Reports button will present you with a list of all the report files AUMI has made. They are ordered by date, and the two types of files are in different sections of the list.

• You can select a file for export by tapping on it. You can unselect it by tapping it again. You can select all the files by tapping all. Whenever all the files are selected, the "All" button turns into a "None" button, so all the selections can be cleared. The header for each section keeps track of the number of items and how many are selected.
• Each line has an "eye" icon on it. By tapping that, a kind of précis of the data shows up on a new screen, which you can examine to make sure it's the one you want. Tap the title bar on this screen to dismiss it and return to the list.
• You can delete the all the selected files by touching the delete button. There will be a confirmation dialog just in case you hit it accidentally!
• By dragging the name of a file to the left, a delete button will show up. You can tap this to delete a file individually.

All these files, are also available in the iTunes Document interface for AUMI. The main log file is also written whenever you leave the program.

Each record has the following information:

• "Version",
• "Date Time Stamp",
• "Name",
• "Instrument",
• "Scale",
• "Notes",
• "Notes Played",
• "min X",
• "max X",
• "min Y",
• "max Y",
• "avg Vel X",
• "avg Vel Y",
• "Institution",
• "Program",
• "Administrator"

• "Version",
• "Date Time Stamp",
• "Name",
• "Instrument",
• "Note",
• "TimeInBox"
• "Name"

For example:

1. plugging in the device into a computer that knows about it. Usually, that's the one that was used to set it up or back it up.
2. open iTunes
3. find the device in iTunes's top bar:
   
4. click on that and click on 'Apps' in this list of buttons:
   
5. scroll down to where it says "File Sharing"
   
6. scroll down that list and click on AUMI:
   
7. The box on the side marked "Documents" will show all the sessions it has recorded:
   

All the personal set-ups are also in the Documents Directory.
You can select them all and click "Save to..." to save them in a directory that you can remember. They don't take up a lot of space, and you can also just delete them after you've saved them.

Camera picker: AUMI can use any cameras that are available. This is also true for the desktop version. By default, cameras facing the user will have mirroring turned on, and cameras on the back of the device have mirroring off. You have the option to flip and mirror the images so they meet your needs. There can be cameras attached to the desktop version that are neither front nor back, so these options are useful!
Camera choices are now saved with their names, so that sharing Setups with other devices can try to match the kind of camera. If AUMI can't find that camera, it will use the first camera in the list.

Hide Video: This switch lets you control whether you see the live video on the screen or not. Some players may not want to look at themselves while playing, or you may just prefer a solid color instead of video for some performances. When Hide Video is off, the screen will show what the camera sees and display the live video image behind the Sound Boxes, controls, and Cursor. When Hide Video is on, the screen will show a solid colored background instead. This background color can be set using the Set Looks screen controls.The Cursor, sound_boxes, and tracking_spots are unaffected by this switch. Select the segment "No" or "Yes". There is an identical switch in the Set Looks screen control dialog.

• Multiply Motion: When the Multiplier is at its lowest setting, all "meaningful" motion, which is relative to the last frame it analyzed, is simply added to the Cursor's position. "Multiply Motion" takes that motion offset and multiplies it up to 4 times. This increases the range of motion, so that a very slight motion can have a bigger effect at moving the Cursor.
  Multiply has no effect on the Face, Color, Touch Tracker, or Pose detectors, which directly set the cursor.
• Motion Track Spots controls the number of tracking spots (features)that AUMI uses to detect motion. The more there are, the better the chance that the motion will be accurate. However, lower powered devices may need to keep this number low. The range is from 32-400 spots. The default is 50, to accommodate slow devices.
• Motion Track Closeness: This controls how close together the tracking spots will be located in pixels. It's good to keep them apart so that general motion can be detected. If there are are parts of the image with a lot of high contrast points close together, the tracker will find that a little too interesting and ignore other parts of the image which you may want to track. Increasing the number of spots will mean it has more chance to find the motion it needs! The range is from 0 to 16 pixels.
• Motion Sensitivity: This number sets a limit on how much motion a tracking_spot must have for it to be considered "meaningful motion". A setting of 0.5 is good, but in visually noisier environments, a higher setting may be needed. For more sensitivity, make it low, like .1 or even 0.0.
  The range is 0.0 to 4.0, in increments of 0.1 , which it compares to the average pixel motion.

Motion Tracking Settings Hints

• Many motion tracking problems are relieved by increasing the frame rate, which can be done by choosing a lower resolution camera setting or zooming in on the AUMI player. Analyzing frames that occur closer to each other in time tends to make the recognition of correlated spots more accurate!
• If there is too much detail concentrated in a small area, but the motion is spread out, increasing the distance between tracking points will distribute the analysis more evenly.
• If there are significant, but not very busy tracking points detected that are not related to the AUMI player, increasing the number of points will allow spots of lower contrast to participate in the motion tracker's flow analysis.
• If there is significant "motion noise" in an image, which is a kind of jittery tracking due to ligting conditions, usually, setting a higher Sensitivity limit will filter out that noise at the expense of losing some accuracy in the "non noisy" tracking points.

• To select the colors that are tracked, double tap the screen on a color that you want to set, which puts AUMI in "color setting mode". It turns out that you will want to track many variations of a color, since it changes in different lighting conditions. While AUMI is in the color setting mode, the cursor will change to a multicolored ring. While still touching the screen, the colors touched are added to the list of tracked colors, and you can "collect them" my moving the colored object on screen or your touch. Monochrome colors like black or white cannot be tracked. If "Show Areas" is turned on, the areas of that color will "light up". This is a good way to insure that the color you want tracked is concentrated in a specific area. The color of the cursor is set to one of these colors as a reminder.
• You can reset the color of the cursor using the Looks screen.
• There are 12 buttons that you can use to save these tracking color sets. Tap on a black button to save the current color set there. Tapping on a colored button will change to that color set. Long pressing on a color set button will clear it back to black. These color sets are only saved on the device being used.
• The current color set is saved with the Environment for this Setup, and can be transmitted to other devices using the the Listen and Send interface or other Setup sharing methods.

• None: No Timing Quantization is used. All sounds play immediately.
• Last: If several Sound Boxes are passed through, only the last box before the beat comes by is chosen. This way a user can have more time to choose the right sound correctly.
• Dotted: This behaves like "Last", except instead of an even distribution of beats, it puts them in a "dotted" rhythm: a long beat followed by a short beat (or a short followed by a long, depending on how you hear it!)
• Echo: Like "Last", except every time a Sound Box plays, it plays the same note again the specified number of times, each time a little quieter. New events override these "Echos". This has the effect of giving you a lot of sound even if there isn't a lot of motion.
• Up: Like "Echo", except each play will queue up the next Sound Box's note in the Sound Boxes on screen. If the scale is a chord, this means it will arpeggiate the chords upward!
  If the layout is reversed, this will instead play "Down".
• Mordent: Like "Echo", except each step will queue up the note, the next box's note in the boxes on screen, back to the original note, the box before the note, and it will repeat this the more Echos you provide for. Yes, technically, it's a "Grupetto".
• Walk: A random walk - it either repeats, goes one step below (if it can) or one step above (if it can), for "Echos" steps.
• Bach: The first 8 notes of Bach's Prelude 1, BVW 846. This sounds recognizable no matter what the scale! If you are in the chromatic scale, it will transpose it to whatever key you start in, if not, it will treat the scale as diatonic and you'll get modal (or weirder) variations!
• Saron: Plays a traditional Indonesian gamelan saron pattern. This sounds good with Diatonic, Pelog, Madenda, and the fifth-based scales. The whole pattern is 16 steps long.
• Claps: Plays the clapping pattern for Steve Reich's Clapping Music. It's 12 beats long.
• Paradiddle: Plays the classic Left-Right-Left-Left Right-Left-Right-Right paradiddle drum pattern. This chooses notes based on the setting of the chord type. It's 8 beats long.
• Maqsoum: Popular Arabic rhythm. This chooses notes based on the setting of the chord type. It's 8 beats long.
• Cha Cha: Cha Cha Cha!. It's 8 beats long.
• Take 5: a 5/4 classic!. If you are in the chromatic scale, it will transpose it to whatever key you start in, if not, it will treat the scale as diatonic and you'll get modal (or weirder) variations!
• Yankee: Yankee Doodle! If you are in the chromatic scale, it will transpose it to whatever key you start in, if not, it will treat the scale as diatonic and you'll get modal (or weirder) variations!
• Birthday: Happy Birthday to You! If you are in the chromatic scale, it will transpose it to whatever key you start in, if not, it will treat the scale as diatonic and you'll get modal (or weirder) variations!
• Row, Row,Row: your boat! If you are in the chromatic scale, it will transpose it to whatever key you start in, if not, it will treat the scale as diatonic and you'll get modal (or weirder) variations!
• Memory: plays back the last notes you've played in a loop. The "steps" setting lets you set how many. to clear the memory, use the "Silence" or "Clear MIDI" button.

• BPM "Beats Per Minute" , which can be used to set up the general pace of quantization. AUMI doesn't know about other programs' rhythms, but if it's set to the same BPM, it will at least be roughly in phase with them. (6 to 300)
• SPB "Sounds Per Beat" divides the beat equally and plays this many sounds per beat. It's an easy way to increase the closeness of sounds and still be "in time" with other programs. (1 to 8). Turning this up also lets the arpeggiation and chord timing features be used in a way that is nearly operating in real time.
  BPM and SPB multiply to show the "Sounds Per Minute" (SPM) in the titlebar.
• Echos AUMI's Timing Quantization can also act like a mini-arpeggiator, echo, or assign new note sequences with each touch of a Sound Box. This is the number of scheduled sounds (usually echos) in that mini-sequence (1 to 32, or infinite (∞) which is really 64).
• Chord Type AUMI can also play several notes simultaneously - or nearly so - by choosing sounds from the layout to make a chord. For example, with Chord Type set to "triad", playing a C in a C major scale will result in "C E G", which is a C major triad, playing an "A" will result in "A C E", an A minor triad, and so forth. The choices of chord type are made based off of the original note picked (the "root"). These intervals are adjusted for chromatic scales to be the major third, the perfect fifth and major 7th!
  • NoneOnly the root is played
  • 3rdThe root and the third sounds are played.
  • 4thThe root and the fourth sounds are played.
  • 5thThe root and the fifth sounds are played.
  • 8veThe root and the octave(8th) sounds are played.
  • TriadThe root, third and fifth sounds are played.
  • TetradThe root, third, fifth and seventh sounds are played.
  • ArpThe root, third, and fifth are played in an arpeggio.
  
  Using Timing Styles with Chord Types will let a chord play in a Timing Styled pattern. It can play a series of triads, say, ascending the scale.
  Using these chord types with more interesting scales, you'll get more interesting chords!
  If the notes in a chord go higher than the available notes, it will play as many as it can.
  It may help to shorten the instrument's cutoff time (or MIDI off delay time) so you don't have a massive number of notes playing at once.
  This will also work on non-melodic sounds, adding to the cacophony, depending on the instrument!
  Reminder: The Chord Types don't work in monophonic mode, instead only playing the last note scheduled.

AUMI makes it easy to add pre-packaged sets of sounds as optional instruments. This way, custom sounds for a particular user or group can be easily installed on a number of devices. Since they are optionally loaded, they don't enlarge the original AUMI download or clutter up the lives of people who might not be interested in them.
A collection of pre-made AUMI instruments is available on aumiapp.com's AUMI Instrument page (http://AUMIapp.com/aumiinst.php). Just tap on one and it will ask to launch AUMI, and install it right away and let you play it. This page is constantly growing with a collection of unusual instruments and sound sets.
If you are connected to the internet, these instruments are also easily available from within AUMI by touching the "More" button on the Instruments screen. Read about MORE here There are two ways to delete a User Instrument: One is to use the "Delete Instrument" button in the instruments controls, or you can delete individual sounds in the Sounds Control. If you choose to delete all the sounds, the option to delete the whole instrument will present itself.
An AUMI Instrument can be prepared by the AUMI staff out of your sound files and put in this list for the whole AUMI community, just contact info@aumiapp.com! Or you can make one yourself:

• Using the MacOs Finder, you can make a directory of sound files that follow the naming conventions, and compress it from the Finder.
• Rename the file's suffix from zip to aumiinst.

• Right-click the folder with the sounds in it
• Choose Send to and from that menu, choose Compressed (zipped) folder
• Rename the file's suffix from zip to aumiinst.

• Put the sound files into a folder. Make sure they are named correctly if you want them to be "melodic", and that they are edited and equalized and normalized to your taste. tar/zip the files. I use this UNIX command in a shell script:
  /usr/bin/tar --exclude ".DS_Store" -vczf "$1.aumiinst" "$1"
• note that the tar/zip file should have the suffix ".aumiinst"

Somewhat adventurous AUMI users or administrators can add their own sounds into AUMI in a limited fashion.

• You can "share" sound files from other applications that can record or create audio files and they can be made available inside AUMI, like Voice Memos or Garage Band and many more! This will work if the sharing dialog says "Copy into AUMI". It will not work if it says "Open In AUMI".
• You can use the "Files" app to move sounds in and out of AUMI and rename them. If you "Share" from another app into AUMI's folder in "Files", it will show up!
• You can also use the iTunes Document interface for AUMI, you can drop in MP3 and other sound files and have them automatically turn into instruments from another computer running iTunes ( or, from the Finder, in MacOS Catalina and later).

• Plug in your device and open the Finder.
• When the device becomes available, it will appear as a device in the "Locations"" list, like other disk drives. Click on the button named for your device.
• Look for the menu item "Files". Inside of Files should be the "AUMI" folder, which is in fact where you can put files that AUMI can use. You can treat this folder like any other folder and copy sounds into it, or rename, or delete them, and otherwise organize them. Logs, .aumi setup files are also available in there.
• Back in the device, your sounds should be available in AUMI when you launch it again.

• Plug in your device and open iTunes.
• When the device becomes available in iTunes, it will appear as an icon next to the drop down menu in the toolbar on the left. Click on this button.
• Pick "File Sharing" in the button menu. (Called "Apps" before iOS11)
• Scroll down below the listing of Apps to the "File Sharing" section
• Find AUMI in that section and click on it. It should look like the image shown below.
  
• You can now drag files in (or out) from this window. Or you can use the "Add ..." button.
• Drag in any sound files you want to use. They must have a suffix of ".mp3", ".aif", ".aiff", ".m4a", or ".wav" (lower case).

Naming Conventions

• Files which you want to be interpreted together as a melodic instrument - that is, able to use scales and keys like Piano - should be named as the corresponding MIDI note numbers, and have a contiguous range of chromatic notes. The name should start with 36 (which is C2) and not go higher than 96 (C7). You can provide a smaller range of sounds within those limits, say, starting at 60 (Middle C). It's important that they be contiguously numbered - otherwise they won't be recognized as melodic!
  For example,36.mp3, 37.mp3, 38.mp3, etc.
  There's no reason to constrain yourself to sounds with pitches corresponding to the MIDI note number that it is named for, or melodic sounds at all! You should have a file for each MIDI note in the range of notes you want to support.
  These files are turned into a "melodic instrument" named "My Melodic". They are also put in as a percussive instrument, and you can reorder the sounds!
• If you don't want your sounds to be considered melodic, but you still would like to number them, try starting the numbers from 100 rather than 0 or 1!
  You can also mix the two kinds of instruments. A percussive instrument can reveal non-melodic, but related sounds as part of the instrument as well. These special sounds simply must not end in a number from 36 to 96.
• Any sound file is also put in the list for simple sampled instruments (My Instrument), loops (My Loops) and a relative instrument (My Instrument REL). You have to have at least 4 of these sounds to make a "My Loops" instrument or a "My Instrument REL".

• (Melodic audio files) Marimba_60.mp3, Marimba_61.mp3, Marimba_62.mp3,...
• (Percussive audio files) Peter_hi.mp3, Peter_there.mp3, Peter_AUMI.mp3, Peter_players.mp3

• You can remove sound files from a User instrument by going to the sounds menu while a user instrument is selected. A "delete" button shows up in the interface. Select the files you want to delete, and tap the delete button. It will ask to confirm the deletions. If you are deleting all of the sound files, it will ask to confirm that you want to remove the instrument as well. If the current instrument is deleted, AUMI will use the "Piano" instrument when done.
• You can remove files from AUMI using the desktop File Sharing interface. The .csv AUMI logging files will be visible there and are also able to be transferred for reporting purposes or removed.
• Once the files have been transferred, you can close iTunes.
• When you now restart AUMI, you can see the new instruments appear at the bottom of the list named My Melodic, My Instrument, My Loops, and My Instrument REL or classified with the names given with the naming convention mentioned above.
  If there are no samples in that particular category, the instrument won't be created.
  
  

Many apps like "Audio Share" or other editing, recording, and synthesis programs, have a sharing button that can send a sound file to other programs. AUMI is now a destination for this kind of sharing. The supported sound formats are : mp3, aif, aiff, wav, and m4a. I'd suggest you keep the sounds under 30 seconds long.

AUMI now can read extra metadata in a file included along with the AUMI Instrument sounds named "metadata.json". If this file is missing, generic information is provided, but this gives a little more flexibility to the AUMI Instruments. At some point, all the official AUMI Instruments will have this metadata included!

                {
                    "uiname": "Bamboo Clarinet",
                    "desc": "Bamboo Clarinet in the Widow Jane Mine",
                    "tags": [
                        "clarinet"
                    ],
                    "types": [
                        "melodic",
                        "loop",
                        "perc"
                    ],
                    "created": "20220613T194521",
                    "icon": "\ud83c\udf8b",
                    "author": "jhhl",
                    "user": true,
                    "source": "http://aumiapp.com/AUMIInsts/Bamboo%20Clarinet.aumiinst",
                    "version": "1.0",
                    "znotes": "melodic"
                }
                

screen, a "More" button becomes available. If you are connected to the internet, you will see this control:

This list of instruments comes from the Official list of AUMI Instruments hosted on

This list has a lot of information in each line, and you can scroll through it, reorder it, or directly search for instruments. Melodic instruments have "♬" in front of the name, percussives have "🥁", and others have "😀".

• Original The sort order as provided by the aumiapp.com website
• Name Sorted by name, A-Z
• Section Sorted by the section, "melodic", "percussive", etc. as in the website
• Credits Sorted by the credit name, A-Z
• Newest Sorted by when it was added to the collection, newest to oldest
• Loaded Sorted by whether it's already loaded (with a check mark).
• Biggest Sorted by how big it is. Sometimes you want to save some space!

There may be a case where you want to access an unofficial sound collection through the More Button. This is a bit of an advanced feature, since you need to set up the API call that provides thisinformation, but it is possible. Email info@aumiapp.com if you wish to try this out. Otherwise, you can always make a web page with saved instruments and setups on it that can load themselves into AUMI with a click , as long as there's an AUMI on the same device. This is detailed in the User Sounds and AUMI Instruments section of these instructions.

You can register for news about AUMI by clicking on this MailChimp link or visiting the AUMI website's "Get Involved" page at http://aumiapp.com/participate.php.

AUMI for iOS collects data about how the app is used. To learn more about what kind of information is being stored, see Reports. This information is stored only on the device AUMI is running on. It is not shared or uploaded to any service.
Using AUMI's reporting interface, these usage records can be reviewed and exported for further reporting needs, such as research on the progress of a group of AUMI users. The export can be done in a number of ways to any apps which accept text files. These records can have metadata included in them that can identify the user, program, administrator, and institution if the person filling out that metadata wishes. Each file can also be deleted from within the app.

It is the responsibility of the person transferring the data to ensure that the privacy of the AUMI Users is in accordance with the laws in their jurisdiction, and the wishes and consent of the people whose actions are recorded in the data which is being collected and retrieved.

A privacy policy document is currently under review. Contact us at info@aumiapp.com if you have any questions. A preliminary version is visible at:http://aumiapp.com/policy

AUMI for iOS was developed by Henry Lowengard for the Deep Listening Institute, now called "The Center for Deep Listening at Rensselaer" and headquartered in the Center for Cognition, Communication, and Culture. AUMI for iOS adapted from the desktop version of AUMI, also available from aumiapp.com. Thanks to Pauline Oliveros, Leaf Miller, Jaclyn Heyen, Emily Halstein, Lisa Barnard Kelley, and Al Margolis of DLI, and the AUMI desktop developers:
Zane Van Duzen,
Zevin Polzin,
Doug Van Nort,
Ian Hattwick,
Aaron Krajeski,
Ivan de Almeida Soares Franco,
John Sullivan

For the AUMI Consortium: Prof. Sherrie Tucker (University of Kansas), Prof. Eric Lewis (McGill University, Montreal), Thomas Cuifo (Mt. Holyoke), Dr. Ellen Waterman, Gillian Siddall, and Jesse Stewart.
For Rensselaer, thanks to Prof. Jonas Braasch, Prof. Tomie Hahn, and Dean Mary Simoni, John Fisher.

Some audio was generously provided by these musicians:

• Nick Didkovsky and Dr. Nerve for the Nerve Event noises from "Beta 14: OK"
• Dave Soldier and the Soldier String Quartet for the Marsh Loops, Pin Pia, and Soldier String Quartet Excerpts from "Sequence Girls", "Paris 1919" (featuring Shelley Hirsch on vocals), and "The Impossible String Quartet".
• Skip La Plante for Glyptodont .
• Deep Listening Band for Troglodyte's Delight.
• Kala Pierson for Tran Phu.
• The Institute of Contemporary Music for Kitchen Percussion. (via ccMixter.com)
• Free SFX for the Dogs
• Funny Cats and Kittens Meowing Compilation 2013 [HD](You Tube) for the Cats
• Farm animal sounds (https://freeanimalsounds.org/farm-animals/)
• Jesse Stewart's Xenophora: Music for Solo Waterphone and the Hand Pan sounds.
• Eleni Reyes, singer extraordinaire!
• Leaf Miller, AUMI pioneer and drummer!
• and Henry Lowengard of jhhl.net supplied the Music Box, Synthy Loops, LP Lock Grooves, Tenor Recorder in the Widow Jane Mine, Fifths, male voice, and prepared or synthesized the other samples and loops.

                    Copyright (c) 2010-2015, Sam Soffes, https://soff.es

                    Permission is hereby granted, free of charge, to any person obtaining
                    a copy of this software and associated documentation files (the
                    "Software"), to deal in the Software without restriction, including
                    without limitation the rights to use, copy, modify, merge, publish,
                    distribute, sublicense, and/or sell copies of the Software, and to
                    permit persons to whom the Software is furnished to do so, subject to
                    the following conditions:

                    The above copyright notice and this permission notice shall be
                    included in all copies or substantial portions of the Software.

                    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
                    EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
                    MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
                    NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
                    LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
                    OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
                    WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
                    

AUMI websites:

• The Official AUMI for iOS Web site is at http://aumiapp.com.
• AUMI also has a presence on Facebook at :https://www.facebook.com/AdaptiveUse.

AUMI © 2012-2014 Deep Listening Institute, Ltd., and 2015-2023, Center For Deep Listening at Rensselaer
Developed by Henry Lowengard
compile date:   build version: