Public examination of the doctoral degree of Uljas Pulkkis
Thesis:
Score-Tool – Target audibility testing in orchestration
Artistic component:
All the Truths We Cannot See – a Chernobyl story
Applied component:
Score-Tool application and its source code
- Supervising board: Prof. Tapio Lokki, Aalto University, Prof. Andrew Bentley, Sibelius Academy of the University of the Arts Helsinki, Dr. Tuukka Ilomäki
- The board which assessed the artistic demonstrations: Prof. Mieko Kanno, Sibelius Academy of the University of the Arts Helsinki, Prof. Sakari Oramo, Sibelius Academy of the University of the Arts Helsinki, Prof. emeritus Atso Almila, Sibelius Academy of the University of the Arts Helsinki, Jean-Baptiste Barrière, Prof. Philippe Esling, IRCAM
- Thesis supervisor: Dr. Marcus Castrén, Sibelius Academy of the University of the Arts Helsinki
- Thesis pre-examiners: Professor Stephen Stephen McAdams, PhD, DSc, FRSC, Schulich School of Music, McGill University – Canada, Composer Jean-Baptiste Barrière – France
- Thesis opponent: Professor Stephen Stephen McAdams, PhD, DSc, FRSC, Schulich School of Music
Custos: DMus Ulla Pohjannoro
Abstract
“Testing the impact of newly acquired orchestration knowledge to my own artistic choices”
This is an applied artistic doctoral project, which is divided into three parts: an orchestral composition, a computer application and its source code, and a written report. In this project, I present an orchestration issue that I have encountered in rehearsals of my compositions. The issue is the inaudibility of an instrument or an instrumental group that I thought to be audible when I was writing the score. This inaudibility can occur even though the orchestration follows the best practices recommended in academic handbooks. The artistic part of the project is an opera All the Truths We Cannot See, which I composed while acquiring new knowledge about orchestration as described in this report.
The developmental part of the project is the Score-Tool computer application, Score-Tool App, which I designed and coded myself. The application is intended for composers to enable them to analyze a score during the composing stage in terms of the audibility of a desired instrumental sound. The audibility is calculated based on real instrument audio analysis and measurements of sound intensity levels in a performance situation. In addition to my own research, the App utilizes various psychoacoustic algorithms, borrowed from lossy audio coding- and speech recognition applications and virtual fundamental research to determine the masking threshold of the orchestration. I have also adjusted the existing algorithms for this project’s purposes. Any sound that has spectral peaks above the masking threshold within at least one critical band should thus be at least partly audible. The inaudibility of a sound can also be caused by its auditory blending into the orchestration. The blending of a sound can happen, for example, when its timbre matches the orchestration or when the spectral centroid of the sound is low in frequency space. The application estimates the blending of the sound by calculating its spectral centroid and by comparing the timbre similarity with orchestration by utilizing the MFCC algorithm, which is borrowed from speech recognition applications. As a result, the App calculates the audibility prediction in percentage values. The audibility prediction is my own term, calculated with the algorithm I developed. This value comes from a combination of masking and blending algorithms whose importance is weighted based on my experiences and my own measurements in orchestral rehearsals. The usability of the Score-Tool App has been tested by me and by other composers by analyzing both existing and in-progress compositions using the App.
The report is divided into three parts. In Part I discuss aspects affecting the sound audibility in orchestration, what is taught about the subject in music universities, my personal experience in this area, the effect of the hall and the performer’s position on stage, and how the issue is addressed in music psychology research.
In Part II describe the development of the Score-Tool App, how the algorithms work “under the hood,” and the features I implemented in the program. Part II also includes my own research in visualizing orchestration masking and timbre data as well as a tutorial for the App. The App manual can be found in the tool itself. The Score-Tool App is also addressed at the end of the report, where I discuss the possibilities for future research in this field using the Score-Tool App and other orchestration features that could be implemented into it.
In Part III of this report, I describe the testing phase and how using this App has changed the way I write for the orchestra while composing the opera All the Truths We Cannot See. In addition, I present cases where this App has helped my colleagues in their orchestrations. I also discuss how the analysis results of the Score-Tool App correlate with live performance and how the audibility prediction algorithm of the Score-Tool App has been adjusted based on my experience and measurements I made in this testing phase. In other words, I test the validity of the results that the Score-Tool App currently gives a composer.
The conclusion is that the Score-Tool App provides relevant information about possible audibility issues in a composition. Using the App does not solve the issues immediately, but it does give reliable specifics about which instruments are causing the issues. This information helps composers during the artistic process to pre-evaluate the functionality of their orchestration. The App guides the composer in a field that, owing to its complexity, has very few definite answers.
More information
Uljas Pulkkis
uljas.pulkkis@gmail.com
uljas.pulkkis.com
score-tool.com
- Supervising board: Prof. Tapio Lokki, Aalto University, Prof. Andrew Bentley, Sibelius Academy of the University of the Arts Helsinki, Dr. Tuukka Ilomäki
- The board which assessed the artistic demonstrations: Prof. Mieko Kanno, Sibelius Academy of the University of the Arts Helsinki, Prof. Sakari Oramo, Sibelius Academy of the University of the Arts Helsinki, Prof. emeritus Atso Almila, Sibelius Academy of the University of the Arts Helsinki, Jean-Baptiste Barrière, Prof. Philippe Esling, IRCAM
- Thesis supervisor: Dr. Marcus Castrén, Sibelius Academy of the University of the Arts Helsinki
- Thesis pre-examiners: Professor Stephen Stephen McAdams, PhD, DSc, FRSC, Schulich School of Music, McGill University – Canada, Composer Jean-Baptiste Barrière – France
- Thesis opponent: Professor Stephen Stephen McAdams, PhD, DSc, FRSC, Schulich School of Music
Custos: DMus Ulla Pohjannoro
Abstract
“Testing the impact of newly acquired orchestration knowledge to my own artistic choices”
This is an applied artistic doctoral project, which is divided into three parts: an orchestral composition, a computer application and its source code, and a written report. In this project, I present an orchestration issue that I have encountered in rehearsals of my compositions. The issue is the inaudibility of an instrument or an instrumental group that I thought to be audible when I was writing the score. This inaudibility can occur even though the orchestration follows the best practices recommended in academic handbooks. The artistic part of the project is an opera All the Truths We Cannot See, which I composed while acquiring new knowledge about orchestration as described in this report.
The developmental part of the project is the Score-Tool computer application, Score-Tool App, which I designed and coded myself. The application is intended for composers to enable them to analyze a score during the composing stage in terms of the audibility of a desired instrumental sound. The audibility is calculated based on real instrument audio analysis and measurements of sound intensity levels in a performance situation. In addition to my own research, the App utilizes various psychoacoustic algorithms, borrowed from lossy audio coding- and speech recognition applications and virtual fundamental research to determine the masking threshold of the orchestration. I have also adjusted the existing algorithms for this project’s purposes. Any sound that has spectral peaks above the masking threshold within at least one critical band should thus be at least partly audible. The inaudibility of a sound can also be caused by its auditory blending into the orchestration. The blending of a sound can happen, for example, when its timbre matches the orchestration or when the spectral centroid of the sound is low in frequency space. The application estimates the blending of the sound by calculating its spectral centroid and by comparing the timbre similarity with orchestration by utilizing the MFCC algorithm, which is borrowed from speech recognition applications. As a result, the App calculates the audibility prediction in percentage values. The audibility prediction is my own term, calculated with the algorithm I developed. This value comes from a combination of masking and blending algorithms whose importance is weighted based on my experiences and my own measurements in orchestral rehearsals. The usability of the Score-Tool App has been tested by me and by other composers by analyzing both existing and in-progress compositions using the App.
The report is divided into three parts. In Part I discuss aspects affecting the sound audibility in orchestration, what is taught about the subject in music universities, my personal experience in this area, the effect of the hall and the performer’s position on stage, and how the issue is addressed in music psychology research.
In Part II describe the development of the Score-Tool App, how the algorithms work “under the hood,” and the features I implemented in the program. Part II also includes my own research in visualizing orchestration masking and timbre data as well as a tutorial for the App. The App manual can be found in the tool itself. The Score-Tool App is also addressed at the end of the report, where I discuss the possibilities for future research in this field using the Score-Tool App and other orchestration features that could be implemented into it.
In Part III of this report, I describe the testing phase and how using this App has changed the way I write for the orchestra while composing the opera All the Truths We Cannot See. In addition, I present cases where this App has helped my colleagues in their orchestrations. I also discuss how the analysis results of the Score-Tool App correlate with live performance and how the audibility prediction algorithm of the Score-Tool App has been adjusted based on my experience and measurements I made in this testing phase. In other words, I test the validity of the results that the Score-Tool App currently gives a composer.
The conclusion is that the Score-Tool App provides relevant information about possible audibility issues in a composition. Using the App does not solve the issues immediately, but it does give reliable specifics about which instruments are causing the issues. This information helps composers during the artistic process to pre-evaluate the functionality of their orchestration. The App guides the composer in a field that, owing to its complexity, has very few definite answers.
More information
Uljas Pulkkis
uljas.pulkkis@gmail.com
uljas.pulkkis.com
score-tool.com