Assessing Listening and Speaking

I. TEACHING CONTEXT

Information	Details
Students’ age (or age range)	14-15
Students’ proficiency level (beginner, advanced, etc.)	B1（Council of Europe, 2020)
Nature of the course (e.g., university, public school, community centre)	Public schools in China

II. LISTNENING TEXT

The listening script is taken from MissLettyMari's video (2021)

Being a volunteer is one of the best things you can do with your life. It's a great way to help other people. It's also very satisfying to know that you're not wasting your time and are helping people who need help. Many of us could and should be out there doing voluntary activities of some kind. So many volunteer organizations need extra hands. It really is easy – just pick up the phone and offer your services.

I think too many of us settle into a lazy lifestyle. We just want to come home and watch TV. Life is much more interesting when you're a volunteer. I found it really opens your eyes to how some people live. (MissLettyMari, 2021)

Questions	Answers
How is this text suitable for your learners’ age and interests?	The topic of volunteering is suitable for 14-15-year-old students as it aligns with their developmental stage of exploring social responsibilities and personal interests. At this age, students are beginning to understand the importance of community involvement and the value of helping others. The text introduces real-life examples of young people involved in volunteering, which can be inspiring and relatable for them
How is this text suitable for your learners’ language proficiency?	The text is appropriate for B1 level learners as it uses common vocabulary and simple sentence structures. It does not contain overly complex grammar or idiomatic expressions
Do you anticipate any challenges in terms of comprehension? (for example, potentially difficult words, grammar, accent, speech rate)	Words like "voluntary", "satisfying", "organizations", "services", "settle", and "government" might be unfamiliar to some students. Pre-teaching these words or providing a glossary can help mitigate this challenge.
Describe the conditions under which the listening task will take place, such as: - how many times it will be played - whether or not it will be slowed down, and why - whether learners have access to subtitles, transcripts, word lists, etc. during or after listening	The text should be played at least twice. The text should be played at normal speed initially to help students get accustomed to natural speech. Providing the full transcript after the exercise allows students to check their answers and understand the parts they missed.

III. LISTENING ASSESSMENT

Being a Volunteer

Instructions: Listen to the following text about being a volunteer. After listening, answer the questions below based on what you heard. You will hear the text twice.

Questions:

What is one of the best things you can do with your life according to the speaker?
- a) Being a volunteer
- b) Watching TV
- c) Working full-time
Why does the speaker find volunteering satisfying?
- a) It helps him make money
- b) It helps other people and doesn't waste time
- c) It is very easy
What do volunteer organizations need according to the speaker?
- a) Money
- b) Extra hands (more volunteers)
- c) New buildings
What simple action does the speaker suggest to start volunteering?
- a) Write a letter
- b) Visit the organization
- c) Pick up the phone and offer your services
What does the speaker say about life as a volunteer?
- a) It is boring
- b) It is much more interesting
- c) It is difficult

Answer Key:

a) Being a volunteer
b) It helps other people and doesn't waste time
b) Extra hands (more volunteers)
c) Pick up the phone and offer your services
b) It is much more interesting

Questions	Answers
What resources do the students have when completing the activity? (e.g., transcript, thesaurus, word list)	Audio Recording: The students will listen to the audio recording of the text twice. Multiple-Choice Questions: The questions provided with multiple choices help guide the students and focus their listening
What construct does this activity assess?	Listening Comprehension: This activity primarily assesses the students' ability to understand spoken English, specifically focusing on their comprehension of the main ideas and specific details from the audio text.
Why did you choose this test format? (i.e. what makes it appropriate for the construct you identified?)	Focus on Key Information: This format requires students to listen for specific information and main ideas, which are essential skills in listening comprehension. By answering questions that cover different aspects of the text, students must pay attention to both the general meaning and specific details.

IV. ASSESSING YOUR ASSESSMENT

Cornerstone	High	Moderate	Explanation
Construct Validity		X	Construct validity is moderate because the multiple-choice format allows for guessing, which might not 100% accurately reflect the students' true listening comprehension.
Reliability	X		The use of multiple-choice questions ensures that scoring is consistent and objective. The answers are fixed and clear, reducing the risk of scoring discrepancies.
Authenticity		X	The activity is somewhat authentic as it involves listening to a real-life scenario, but volunteering is not a common activity for Chinese students, reducing its relevance.
Washback		X	Washback is moderately high as the test provides feedback on students' listening abilities, but the possibility of lucky guesses means it might not fully reflect their understanding.
Practicality	X		The multiple-choice format is easy for teachers to create, administer, and grade. It also simplifies the process for students to understand and complete the activity.

Trade-offs:

In designing this activity, I prioritized reliability and practicality. Ensuring consistent and objective scoring was crucial, and the multiple-choice format is practical, making it easy for both teachers and students to engage with the assessment. This is important for balancing the effort of students and teachers, as open-ended questions are involved in the speaking sections, so the listening section prioritizes a less demanding format.

Construct validity and washback were sacrificed to some extent due to the multiple-choice format, which allows for guessing. This trade-off was made after considering other formats suitable for B1 14-15 year-old students. Options like cloze tests or fill-in-the-blank questions might overemphasize reading and reasoning skills, while short-answer questions could be too challenging for this level. Multiple-choice questions, despite the potential for guessing, provide a balanced approach that offers some "lucky points" to boost student confidence. Similarly, the ability to guess affects washback, but overall, the test still offers valuable insights into students' listening comprehension abilities.

V. SPEAKING ASSESSMENT

Speaking Task：

As an open day volunteer, please introduce today's schedule to the parents at the school gate according to the following table and images. You have 1 minute to prepare, and please say at least 5 sentences.

The Open Day Schedule

Activity	Time	Place
	2:00 p.m.	Classrooms on the second floor
	2:20 p.m.	Hall
	2:45 p.m.	Playground
	3:00 p.m.	Library on the first floor

Possible answer:

Good afternoon, parents! Welcome to our school's open day. Here is today's schedule:

At 2:00 p.m., we will start by visiting the classrooms on the second floor. You will have the chance to see where your children learn every day.

Next, at 2:20 p.m., we will move to the hall to watch a play performed by our talented students. This will be a great opportunity to see their creativity and hard work.

After that, at 2:45 p.m., we will head to the playground to watch a football match. Our students are excited to show you their skills.

Finally, at 3:00 p.m., we will gather in the library on the first floor. You can have a discussion with the teachers while enjoying some cakes and tea.

Thank you for joining us today. We hope you enjoy the open day and get a good glimpse of our school life.

Benchmark:

-	A	B	C	D	E
Fluency and Coherence	Fluent with occasional hesitation. Well-organized ideas, clear detail.	Some hesitation, generally organized, sufficient detail.	Noticeable pauses, connected ideas, basic detail.	Frequent hesitation, poorly connected ideas, minimal information.	Very hesitant, unconnected ideas, unclear information.
Pronunciation	Clear and accurate, minor errors.	Generally clear, some errors but understandable.	Noticeable errors, mostly understandable.	Frequent errors, often unclear.	Errors make understanding difficult.
Vocabulary Range	Wide range, appropriate use.	Good range, occasional errors.	Basic vocabulary, limited range.	Very limited range, frequent errors.	Extremely limited vocabulary.
Grammatical Accuracy	Accurate use of various structures, few errors	Range of structures, some errors.	Simple structures, frequent errors.	Basic structures, errors impede communication	Limited control, very difficult communication.

-	-
What resources do the students have when completing the activity? (e.g., dictionary, thesaurus, word list)	Students do not need additional auxiliary materials. Firstly, this is a task designed for the high school entrance exam. During the high school entrance exam, additional materials or tools are not allowed, so it is best to let students adapt to this in advance. Secondly, the instructions already provide some information and content that students can use.
What construct(s) does this activity assess?	This activity assesses several key constructs essential for effective communication. It evaluates Speaking Fluency and Coherence, ensuring students can speak clearly and fluently while maintaining a logical flow of ideas. Pronunciation is assessed to ensure correct word pronunciation for comprehensibility. The activity also looks at Vocabulary Range, encouraging the use of varied and appropriate vocabulary to describe the schedule and activities. Grammatical Accuracy is crucial, with correct grammar usage necessary to construct meaningful sentences. Additionally, Interactive Communication Skills are measured, focusing on the ability to engage listeners, in this case, parents, by providing clear and relevant information, reflecting real-life interaction abilities.
Why did you choose this test format? (i.e. what makes it appropriate for the construct[s] you identified?)	By providing a structured yet open-ended task, this format effectively assesses a range of speaking skills necessary for B1 level students. With one minute to prepare, students practice organizing their thoughts and delivering structured information. Also, it encourages practical usage of language in a context that students can relate to and find meaningful. The scenario is engaging and relevant to students, making it a meaningful to complete. The images not only stimulate students' interest but also provide opportunities for students of different levels to demonstrate their abilities.

VI. ASSESSING YOUR ASSESSMENT

Cornerstone	High	Moderate	Explanation
Construct Validity	X		The task’s structured yet open-ended format can help primarily assess the target content and target grammar of the students, while also providing room for students of different levels to demonstrate their abilities. Therefore, it can effectively measure the intended constructs of speaking fluency, coherence, pronunciation, vocabulary range, and interaction.
Reliability		X	If the images are unclear, students may interpret the content differently, which can lead to significant differences in the content and quality of their responses, even if their English speaking abilities are similar. Therefore, such a situation can affect the consistency and stability of the test results.
Authenticity	X		The task is authentic as it simulates a real-life scenario where students explain a schedule on an open day, which is a common communicative need.
Washback	X		The task has a high positive washback effect because it encourages students to actively use their speaking skills in a realistic and meaningful context. By preparing for and performing the task, students practice organizing their thoughts, using appropriate vocabulary, and constructing grammatically correct sentences, which are crucial skills for effective communication.
Practicality		X	The practicality of this speaking task is moderate. It is easy to set up and administer, requiring only a printed schedule and images. However, scoring is subjective despite detailed rubrics leading to potential inconsistencies. Effective teacher training is crucial for consistent evaluation.

Trade-offs: What cornerstones did you prioritize? What did you sacrifice?

I prioritized authenticity by creating a realistic open day experience with a genuine schedule and vivid images. This approach helps students engage in a practical and relevant task, enhancing the authenticity of the assessment. However, the variability in image quality could affect reliability, as differences in how students interpret the images might lead to inconsistent responses. Additionally, to maintain construct validity, I designed the task with relatively open-ended content, allowing for a broader range of student responses. This openness, while beneficial for accurately assessing students' language abilities, introduces subjectivity into the evaluation process, making practicality in scoring more challenging. Consequently, ensuring fair and consistent grading requires significant effort and teacher training, which can be a practical limitation. Overall, while the task effectively measures the intended language constructs and provides a realistic context, it sacrifices some reliability and practicality in scoring.

References：

Council of Europe. (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing. https://www.coe.int/en/web/common-european-framework-reference-languages

MissLettyMari. (2021). B1 listening: Volunteer work #1 talking English [Video]. YouTube. https://www.youtube.com/watch?v=2FQNm34xUPE

Shanghai high school entrance exam speaking mock test:jkpt.koukao.cn

Catalog