Doing things differently
What does a mentorship program for troubled boys in 1940s Boston, USA, have in common with a social dancing program for older adults in 2010s Sydney, Australia? Both were assumed to benefit participants, and both were subjected to randomized controlled trials (RCTs), which discovered—to the researchers’ surprise and even consternation—that the programs did not have the expected effects. RCTs are often described as the “gold standard” of scientific research, so their results demand close attention. The dance study found that social dance did not reduce the incidence of falls or their associated risk factors in older adults. More strikingly still, the mentorship study found that the social outcomes were worse for the treatment group—the boys who had been mentored—than for the control group.
These studies underline a point that is central to my previous research and my new book, Rethinking Social Action Through Music: one cannot assume the value of any given social intervention. As a plethora of studies across many disciplines reveal, some interventions—even ones that look attractive and whose benefits seem to be a matter of common sense—turn out to have null effects, while others, like the Boston mentoring program, have negative ones. However widely accepted they may be, however “obvious” their benefits may appear, they need to be rigorously scrutinized—and if the social objective is the priority, the results need to be taken seriously, even if they are unexpected or inconvenient.
Taking the results seriously may mean considering a change of direction. The dance study, for example, did not stop there; its authors did not simply conclude that dance doesn’t work. Rather, they proposed the development of “modified dance programmes that contain ‘training elements’ to better approximate structured exercise programs.” In other words, they suggested that ordinary social dance might not work to prevent falls, but a specialized dance curriculum, designed around the desired outcomes, might.
I was pointed to this study by Tim Joss, the founder and head of Aesop (Arts Enterprise with a Social Purpose), which has created Dance to Health. In Aesop’s words, the Australian RCT
demonstrated that dance in general, and social dance in particular (such as folk and ballroom dance workshops) do NOT prevent falls or their associated risk factors and, more broadly, challenges the validity of generalised claims about arts activities achieving health improvements. We realised therefore that Dance to Health needed to do things differently to achieve the outcomes we desired.
Dance to Health created a specialized curriculum, which was found by an external evaluation to reduce falls by 58%.
Two features of Aesop’s philosophy are particularly noteworthy: its insistence on the value of research and engagement with researchers; and its starting point of “being sceptical of broad claims of generic health and wellbeing benefits of arts engagement.” In other words, engagement with researchers is not shorthand for engagement with researchers who tell a story that we’d like to hear.
Imagine if social action through music (SATM) took a similar approach.
Our field has its own RCT: the Inter-American Development Bank’s 2017 study of El Sistema. Like the two RCTs above, its findings confounded expectations: it discovered no evidence that El Sistema boosted social or cognitive skills, contrary to two decades of claims. It also estimated the poverty rate among those who enrolled in El Sistema at 17 percent, while the poverty rate of the states in which they lived was 47 percent. In other words, a program that had long been touted as a model of social inclusion actually seemed to exclude the poor rather than include them. The report concluded that El Sistema “highlights the challenges of targeting interventions towards vulnerable groups of children in the context of a voluntary social program.”
At this point, though, the response was very different to Aesop’s. The researchers—employed by El Sistema’s funder, the IDB—used “creative” methods to extract a couple of positive results from the data, and then all the parties involved (the program, the researchers, the funders, and the supporters) spun the results, exaggerating the positive findings and minimizing the negative ones, and ended up presenting the study as confirming El Sistema to be a resounding success (see Baker, Bull, and Taylor 2018 for detailed discussion).
Now imagine a parallel universe in which El Sistema and its backers had taken a different approach. Rather than sweeping inconvenient findings under the carpet, rather than inventing a success story where there wasn’t one, the key players acknowledged the null and negative findings and resolved to do something about them. They acted on the major caveat in the report—that El Sistema “highlights the challenges of targeting interventions towards vulnerable groups of children in the context of a voluntary social program”—and responded, like Aesop, “we realize that we need to do things differently to achieve the outcomes we desire.” They turned to experts on music education and social development, including those who question conventional approaches, recognizing the value of critical thinking for progress. They drew on that expertise to create a new, specialized, research-based curriculum, designed around the desired social outcomes rather than a particular musical tradition.
This is, of course, a parallel universe. El Sistema did not take this approach, and the chances of it doing so in future are slim. But SATM is not El Sistema, and the broader field could look to the research and examples like Dance to Health and conclude: we need to do things differently, and we can.
Examples exist within SATM—perhaps not of the precise process described above, but of efforts to think critically about conventional assumptions and practices, and to develop new, better approaches. However, we don’t hear enough about them, in part because El Sistema has monopolized the media space. We tend to hear about their similarities to El Sistema rather than their differences.
My new book looks at one such program, the Red de Escuelas de Música de Medellín (the Network of Music Schools of Medellín). The Red started out like El Sistema, but it began to research and critique this model in 2005 and has steadily diverged from it since then. There are other programs out there that have explored new directions, like Argentina’s Programa Social Andrés Chazarreta (which focuses on Latin American traditional and popular music), Orchkids in Baltimore (which has pursued collaborative composition), Sistema Toronto (with its Social Development Curriculum), or Sister Cities Girlchoir (a “girl empowerment choral academy”). It’s time we heard more about them—and particularly, more about what they’re doing differently and why.
Reference
Baker, Geoffrey, Anna Bull, and Mark Taylor. 2018. “Who Watches the Watchmen? Evaluating Evaluations of El Sistema.” British Journal of Music Education 35 (3): 255–69. https://doi.org/10.1017/S0265051718000086
For those without institutional access, a pre-proof draft can be found here.
Geoff Baker 