In Unit 7 we talked about course design and evaluation.
My course proposal
I was intrigued by Nicky Hockly’s article which we read in Unit 2: Modelling and cognitive apprenticeship in teacher education, so I decided to experiment with this approach.
This was my proposal for a 25-hour PRESET course.
- Methodology: cognitive apprenticeship
- Syllabus: cyclic – closed
- Philosophy: implicit
- Orientation: process and product
- Treatment: Breadth. insufficient time to really cover the range of topics in depth
- Organisation: coherent
- Input: 4 lessons taught by trainers and 7 lessons taught by their peers.
Feedback from trainers
- Output: 1 lesson planned and taught by each trainee.
Group discussions
Handbook
Reflective journal
- Intensive
- designed to meet generic teacher training needs rather than the specific needs of the group taking this course.
As Hockly described in her article, teachers would be exposed to 4 model lessons, as learners and later:
1. reflect individually and collaboratively on:
- the lesson stages, tasks and their aims
- classroom management techniques
2. make daily journal entries.
3. On day 4, teachers would plan their first lesson with or without peer/tutor support.
4. Then, teachers would be divided into two groups, and each one would get 45 minutes to teach their lesson. They would be observed by one of the trainers.
5. There would be time for self-evaluation, followed by group feedback and then a plenary session where the trainers would provide anonymous feedback after having watched 4-6 TPs daily.
6. On the last day, the trainer would invite teachers to compile their own handbook or book of learning (or any other title they choose). There would be A4 sheets of paper passed around for teachers to record their takeaways. They could use their notes/ reflective journals. The sheets could be labelled as follows:
- Grouping and seating
- Giving instructions
- Giving feedback on language
- Giving feedback on task
- Board management
- Teaching receptive skills
- Teaching productive skills
- Teaching grammar/vocabulary
- Dealing with disruptive behaviour
The trainer would collect the sheets, put the handbook together and make copies for everyone.
I included 11 anticipated problems (technological, cognitive, affective, timing, expectations) and solutions.
Feedback I received
Positive
The course allows for a huge amount of discussion and analysis after each lesson.
Plenty of opportunities for reflection (individual and collaborative).
A bold approach! 😊
Strong anticipated problems section.
Great idea to ask teachers to create their own handbook as a take-away product at the end.
To think about
1.Including noticing tasks. Don’t just rely on asking teachers what they noticed. Mind maps, grids, ordering or matching tasks.
2. Supporting reflection. Include specific questions. E.g. rank the error correction techniques starting from your favourite to least favourite and justify your answer.
3. Joining up. Create a link between the noticing /reflecting stage and the experimental part. E.g. Read your notes and identify 5 techniques you’d like to try in your TP.
Course evaluation
We were introduced to Kirkpatrick’s model, which evaluates training courses by taking into account:
- participants’ reactions
- participants’ learning
- transferability of course to participants job
- results/change in work context
- Evaluating teachers’ reactions: I chose a written feedback form, which I believe would help me get more honest answers, especially if they are anonymous. Conducting interviews would be time consuming and perhaps stressful for trainees. It’s fast to design such forms (or make a copy of this one) and will get qualitative data. It can be done on the last day of the course. I borrowed this one from ETpedia Teacher Training.
2. Evaluating learning: By running a pyramid discussion, again on the last day of the course, I would be able to understand which parts of the course were the most beneficial for teachers, and which were not impactful. Their journals and handbooks would also help assess to what extent learning has happened. A self-evaluation questionnaire could also be given to trainees. Also taken from ETpedia Teacher Training.
The main issue with Kirkpatrick’s model is that though it’s easy (or at least easier) to evaluate reactions and learning, the transfer and results tiers are usually ignored at the end of a teacher education course and most teachers can’t/don’t apply new learning. As Simon Smith also said in one of the lectures, it’s tough to measure learning in a short intensive course ‘because change takes a long time’. And as Martyn Clarke added, demonstrable learning that is often evaluated on a course isn’t necessarily evidence of change.
3. Evaluating transfer: To evaluate transfer and results, I thought of a grid that looks like this
I think completing this grid would help teachers to first identify what they want to bring back to their classes, practise, reflect on the results, make an action plan and then reflect again on the experience, so hopefully, a tool that could support experimentation and reflection. I would give it to them at the end of the course and ask them to complete it during their first year of teaching. At the end of the first year, I would send everyone an email and ask them to upload their completed forms to a shared padlet.
You can download it here ⬇️
4.Evaluating results with the one-year-later questionnaire (adapted from Wright and Bolitho, Trainer Development)
To evaluate results, I would steal an idea from Wright and Bolitho’s book Trainer development. I would send a letter to the trainees, a year after the course. There would be some questions, such as :
- Where do you work at present? Briefly describe your role and main professional tasks. How has the course prepared you for these?
- What have been the most valuable and useful benefits of the course in your work and life since completing?
- Looking back, was there any topic or area missing from the course, or an aspect you wish you’d spent more time on?
- Are there any other comments you’d like to make on the impact the course had on your academic/professional life?
My tutor suggested the An email to the Future Me activity. On the last day of the course the tutor asks trainees to write themselves a note saying what they how they hope their practice will change, what they want to find out more about, etc. over the next 6 months. The tutor collects them and emails them out 6 months after the course, along with questions exploring what has actually happened. According to my tutor, who’s already used this activity, self-generated input often provides that ‘ah!’ factor which provokes engagement.
Learning from my peers
I was particularly fascinated by the MSC approach to course evaluation, proposed by one of my peers. It’s a story approach that measures impact. See link in the reference section.
Another interesting idea was including peer and Dos observations after the course, which would help teachers identify where they are and whether they have achieved what you set out to do.
Over to you
How do you evaluate PRESET/INSET courses? Feel free to leave a comment and share what’s worked for you.
The TEFL Zone 