Bio_Logging is a collaboration between Artist Dr Nigel Helyer and Dr Mary-Ann Lea of the Institute for Marine and Antarctic Studies (University of Tasmania, Hobart) which seeks to link scientific bio-logging and GIS techniques with the Artist’s interests in interactive acoustic cartography, data signification and the development of Audio-Portraits to extend the conceptual and intuitive grasp of otherwise abstract data.
The collaboration commenced in 2011 funded as an Artist in Residence under the Synapse Art and Science initiative of ANAT (The Australian Network for Art and Technology) and has gradually evolved to embrace the Conservatorium of Music (University of Tasmania) following the hunch that there are only a few disciplines that encourage and enable people to deal with large and complex data sets, in realtime, and who can communicate their interactions ~ and these are generally called musicians.
Our work during 2011/12 produced three successful public concerts, each providing an opportunity to experiment with different types of data visualisation and graphical scores. The project continued in 2013 funded as a New Art Creative Development project of the Inter-Arts office of the Australia Council for the Arts, facilitating conceptual and technical development and establishing new creative partnerships
The Under the Icecap concert, PowerHouse Museum Sydney, for National Science Week 2012.
The 2011/12 incarnations of BioLogging Vox on the Rox were based upon large sets of data collected from biologging tags attached to Elephant Seals swimming under Antarctic Ice shelves and on Southern Ocean Transits. This information, combined with Ice formation and climate data, was generated as dynamic 3D visualisations, accompanied by a series of graphical scores which a contemporary Jazz improvisation quartet responded to.
Dots on the Rox concert at the Hobart Conservatorium of Music UTAS,
opening National Science Week 2012.
Check out this short documentary video from the Bookend Trust.
And a short sample video showing the cartographic visualisations and graphical scores developed from bio-logging data sets.
And here are three live recordings from the first Dots on the Rox concert at the Tasmanian Conservatorium of Music.
Vox on the Rox ~ Movement No.1
Vox on the Rox ~ Movement No.2
Vox on the Rox ~ Movement No.3
And a short sound mix of an ambient recording of a Southern Elephant Seal convolved with biologging data collected from seals on their deep under-ice cap dives.
Here are two short examples of data-generated 3D cartographic visualisations of Southern Elephant Seal dive and transit patterns. These have been generated using EonFusion software.
DR move3 from Nigel Helyer on Vimeo
Dots on the Rox, EonFusion renders of Southern Elephant Seal BioLogging Data_02 from Nigel Helyer on Vimeo
For more detailed context read the short paper Under the Icecap presented at the World Forum for Acoustic Ecology, Global Composition conference, Darmstadt Germany 2012.
Under the IceCap ~ where to now?
During the past year, our Art and Science journey has reformulated both the title and the scope of the collaboration.
Under the Icecap aims to illuminate the fundamental connection between human activities and planetary dynamics, by creating an experimental installation and performance series that will visualise and sonify scientific and statistical datasets. In essence Under the IceCap renders complex environmental bio-logging data sets collected by Southern Elephant Seals on their under-ice dives and open ocean transits with economic and climatic data, combining them to form 4D cartographic animations, sonifications and live performative and sculptural forms.
The byeline for the Institute of Marine and Antarctic Studies is Turning Nature into Knowledge. The Under the IceCap project supplies a second line Turning Knowledge into Culture encapsulating a powerful Art and Science synthesis and simultaneously raising the expectation but also the risk of the endeavour. Our primary aim is to produce creative work which is compelling and affective but is at the same time a work of scientific utility tapping into both sides of the brain! The key focus is the relationship of the environmental knowledge generated from Antarctic bio-logging data with the Anthropogenic changes in the biosphere.
Under the Icecap is a radical development of our previous collaborative work and whilst the philosophical aims and direction remain consistent with these earlier manifestations (for example the Vox on the Rox and Dots on the Rox concert series) the aesthetic scope, methodology and modality of this proposal have been radically re-defined to encompass spatialised sound-sculpture & data-projection installations, featuring animated GIS mapping projections and data-sound objects, as well as a live performance series (the next incarnation to be hosted by the new Moonah Art Centre, Hobart in early 2015).
We are planning a performance series structured around innovative data sonification methods including; sonification engines reading data directly as audio and/or into musical notation; convolution engines that modulate ambient and field recording audio files with data profiles to produce a hybrid data-music; transcription into traditional musical notation from charted data-profiles and the transcription of data to electro-mechanical instruments such as a Disklavier piano. The performances will hybridise electro-acoustic techniques with both animated graphical scores and conventional notational forms, the latter allowing a musical engagement with the choir or chamber group.
The aesthetic components of the installation will include 3D dynamically spatialised audio; with a focus on Geophony, Biophony and Anthrophony and references to early data-controlled music machines (which pre-figured contemporary computing and data-analysis) operated by punch paper graphical scores derived from Bio-Logging and other data. Media derived from the performative events and installation will be distributed via associated webcast and broadcast domains and incorporated into educational outreach materials and workshops.
Ironically, Science is constrained by Objectivity and Impartiality, whilst Art is constrained by Subjectivity and Partiality. However, both disciplines experience similar difficulties in establishing effective communication with either the public at large or the structures of governance and policy. The Under the Icecap core team is committed to exploring the environmental, social and political issues that are currently transforming our biosphere and is experimenting with radical means of expressing hard scientific research in the form of cultural works that can render this knowledge into a broader cultural discourse.
We have been prototyping various approaches to translate environmental data as sound and animated images. Here is a short test that transcribes four streams of raw biologging data into a simple musical notation, in this case, a String Quartet.
Another test was to transcribe biologging data using a punch tape system on a 20-note range music box.
IceCap_MusicBox_01 test from Nigel Helyer on Vimeo.
As one small example of the sketches we are making, here is a simple test to take complex bio-logging data collected by Southern Elephant Seals on their dives in the Antarctic (see print-out in the background of Surface Wind Speed, Depth with Salinity, Depth with Temperature and Ocean Bottom with Bottom Density)and transcribe the values onto a punch paper music box system. This is a crude but effective Digital to Analogue sonification of data values. This simple sketch illustrates the potential to render tens of simultaneous data streams onto a pianola or discklavier for the Under the IceCap performance.
Why so retro?
The provenance of computer control was originally devised as a connected sequence of individual punch cards, encoding weaving patterns to operate Industrial Revolution Jacquard looms.
The virtues of the punch card system were not lost on Charles Babbage who adopted them to drive his difference engine and they were rapidly adapted to automate mechanical music devices, the punch patterns becoming, in effect a form of graphical score capable of sequencing music boxes’ barrel organs and later Pianolas.
A Jacquard loom showing a sequence of individual punch pattern cards.
The Pianola (or Player Piano) was the most sophisticated manifestation of this development and in terms of reproduction quality was far superior to the nascent technologies of audio recording and transcription, such as the Edison Wax Cylinder or disc-based Phonography, by virtue of being able to not simply encode musical pitch but also performance characteristics.
Ironically it was the punch card and subsequent punch tape technology that enabled the birth of the modern computer and its entwined history with music.
The first public performance of computer-generated music was demonstrated at the Australian Computer Conference in 1951 by a team from CSIRAC (Council for Scientific and Industrial Research Automatic Computer). The CSIRAC fed upon spools of paper punch tape very similar to that illustrated in the video of my small music box.
And now for some background, where we started and how we changed!
The Australian Network for Art and Technology (ANAT) has supported the Bio_Logging Art and Science residency which was structured around a series of four one-month intensive work periods allowing the collaboration to involve a range of research activities and associated technologies.
The artist participated in the working life of the lab, joining field trips to make (for example) hydrophone and ambient recordings as well as water quality measurements and visual documents.
We originally envisaged that each of the four sessions would comprise a ten-day field trip where the Artist would work directly with a Scientific survey team, followed by twenty days in the Hobart lab working with the collected data and developing this into interactive works; however as the project developed we became more involved in questions of data representation and interpretation, together with the potential for experimental ways to convey complex data sets to a wider public.
The images show Bio_Logging Beta exhibited in Hobart in February 2011 as part of the CSIRO International BioLogging Conference. This work in progress, sonifies satellite-tracked bio data from the Sub-Antarctic and layers this into an interactive map interface which can be explored, mixing the layered audio in real-time.
Kerguelen and Heard, satellite image.
Listen to some audio fragments from the sonifications.
Kerguelen Islands