A/B Testing in the classroom
Most world language textbooks and classrooms do NOT make reference to pronunciation.
Pronunciation is so much more than an accent or a dialect. It can:
Affect how a learner perceives meaning
Without without perceiving the difference between pero and perro in Spanish, you may be confusing but and dog
Impede spoken communication
You might have just mixed up silk (seda) with a footpath (senda) in Portuguese.
Result in awkward sociocultural situations
I accidentally asked a Brazilian friend how her breads (pães) were doing instead of her parents (pais).
How can we make this information meaningful and accessible to total beginners?
My colleague at Florida State and I wanted to try new approaches in the classroom, but immediately ran into the following challenges:
Beginning language classes only meet three times a week and have a packed curriculum-how could we add more material?
Languages classes in our program are conducted in the target language-how would we incorporate pronunciation without confusing the students?
Beginning language students have very limited production skills-How could we test the effectiveness of our experiment?
We got creative and figured out how to incorporate new techniques weekly without impeding the normal curriculum. Then we got to work getting materials ready for the upcoming semester.
Experimental Design
We had 3 university-level Spanish 1 classrooms for a whole semester (16 weeks):
Control: This class got the normal Spanish 1 experience, without any reference to pronunciation
Explicit Instruction: This group got 10 minutes of technical phonetics instruction (in English) each week
Implicit Instruction: This group completed weekly activities where the answered differed based on pronunciation (in Spanish), but no technical phonetics terms were used
We tested their ability to perceive words in Spanish before instruction began (pretest), after the 12-week instruction period (post-test), and 4 weeks after instruction ended (delayed post-test).
How each class received instruction throughout the semester
Testing for perception
In the experimental groups, we used minimal pairs in class, or words whose meanings differ when only one sound is changed (think bat and rat in English). This way, we could focus on meaning and not just dialect or accent.
During the (post)tests that all groups completed, participants heard 240 Spanish words and had to decide whether the word had target-like pronunciation (like a fluent speaker would say it) or if they had heard the word before (distractor question).
Minimal pair in Spanish: carro (left, 'car') and caro (right, 'expensive')
Determining effectiveness
Data Analysis
To test for any effects of instruction, we performed a 3 x 3 ANOVA on their test results, with Group (Control, Explicit, Implicit) as the between subjects variable and Test (pre, post, delayed post) as the within subjects variable. Any participants who scored at random (55% or below) were excluded (this affected 5 participants).
Results
Our analysis revealed a significant effect for Test, in which accuracy on the test was highest on the post-test (week 12), regardless of Group.
We also saw a significant interaction between Test and Group, in which the Implicit group outperformed both other groups on post-test 1. For the Implicit group, their improvement was a full letter grade in percentage, from 64% (D) to 74% (C). For beginning students, this level of improvement was a huge finding for us.
Although minor, the implicit group outperformed the other groups at Post-test 1. There were no significant differences at the pre-test or post-test 2, with the control group having a near identical scoring pattern to the Explicit group.
Interpretation & Impact
Implicit, meaningful lessons made the difference!
While all groups improved in their perception from pre- to post-test, a striking finding was that the Explicit and Control groups performed similarly-this not only provides evidence that the technical phonetics instruction in English was not effective for the Explicit group, but that the regular classroom input that the Control group received was sufficient for learners to improve their perception by nearly 10% (62% to 69% accuracy, nearly a whole letter grade).
Our second insightful finding was that the implicit pronunciation instruction did have a positive effect on learners' perception of sounds, but only during the experimental phase. This suggests that learners need ongoing (implicit, meaning-based) instruction to sustain these gains in perception.
After this experiment, we created sample classroom materials for language teachers, which we presented at the Ignite session at Current Approaches to Spanish and Portuguese Second Language Phonology.
Want to learn more? You can read the full write up of my and Anel Brandl's paper here!
