In recent years, there has been a growing call to educators to embrace an “evidence-based” approach, branding it as the science of learning. Teachers have been under significant pressure to incorporate ideas generated from experimental methodologies and randomized control trials from the laboratory settings to determine the effectiveness of educational practices. However, it is important to delve into the philosophy and meaning of being “evidence-based” and also consider the potential hazards that may arise from transforming classrooms into scientific laboratories. In this video by Human Restoration Project captioned ; Schools Are Not Labs: Why “What Works” May Hurt, these ideas are very well articulated by quoting foundational works by scholars like Gerd Biesta and Yong Zhao.
Category: Evaluation/Assessment
-
Evidence-informed Vs Evidence of what(Biesta)
Recently I have got the opportunity to watch an interview of Guy Claxton in which he was asserting the need to ask deep questions about evidence. The first question according to him should be “evidence of what?”. This made me curious to read an article that I have scanned through few years ago by Gert Biesta, i.e. “Good Education in an Age of Measurement”
In which he writes;
“More than just the question of the technical validity of our measurements – i.e., whether we are measuring what we intend to measure – the problem here lies with what I suggest to call the normative validity of our measurements. This it the question whether we are indeed measuring what we value, or whether we are just measuring what we can easily measure and thus end up valuing what we (can) measure.”
He further clarifies the difference between instrumental value, a value which says something about the quality of processes and, more specifically, about their ability to bring about certain outcomes in a secure way.
According to Biesta, whether the outcomes themselves are desirable is an entirely different matter, ie “a matter for which we need value-based judgements that are not informed by instrumental values but by what we might best call ultimate values“
He further expands the ideas discussing “The ‘Learnification’ of Education”. The article is freely available here(link)
I think education needs more scholars trained in philosophy to question many of its popular assumptions more often and more vigorously
-
Schools are not labs: Questioning the blind use of Evidence based approach in Education(Video)
In recent years, there has been a growing call to educators to embrace an “evidence-based” approach, branding it as the science of learning. Teachers have been under significant pressure to incorporate ideas generated from experimental methodologies and randomized control trials from the laboratory settings to determine the effectiveness of educational practices. However, it is important to delve into the philosophy and meaning of being “evidence-based” and also consider the potential hazards that may arise from transforming classrooms into scientific laboratories. In this video by Human Restoration Project captioned ; Schools Are Not Labs: Why “What Works” May Hurt, these ideas are very well articulated by quoting foundational works by scholars like Gerd Biesta and Yong Zhao.
-
Entrepreneurship is ecological. Outside the direct scope of academic credentialism.
If it was for the academicians to decide entrepreneurial outcomes, they would have done it already with Psychometric MCQ tests. Luckily, that is not the case and will never be the case. But unfortunately there are many enthusiastic chaps.(not meant for Prof Mollick if you ever see this).
As I have discussed elsewhere, GRE’isation of entrepreneurship is an unrealized dream for the academic community who lived through many GRE’s and GMAT’s thinking that MCQ tests is equal to generalized intelligence. They will never give-up so easily and this study shared by Prof Mollick is a good example; “What matters more for entrepreneurship success? A meta‐analysis comparing general mental ability and emotional intelligence in entrepreneurial settings”
If it was for psychometric tests Richard Branson as a dyslexic wouldn’t have made it to become a billionaire entrepreneur;
Richard Branson writes
“When I was in my fifties, I was sat in a board meeting being taken through some financial figures when the subject of net and gross came up. I must admit I was quite confused – I didn’t know if the results were good or bad news.”…”As a dyslexic, I thought I’d been hiding my muddling of words and numbers well for years. But I’d been rumbled. I couldn’t tell the difference between net and gross.”
Branson BlogAccumulated advantage matters :
On the other hand accumulated advantage matters. Those with accumulated advantage might provide their kids with all kinds of resources to get admission to elite universities. This can give a false sense that because of higher education they become successful. This is wrong because they already are filthy rich and that’s why they got-in in the first place.
I am ending this point with a Nassim Thaleb BS expose or Exposayyy. He writes(blogpost)
“When the results come from dealing directly with reality rather than through the agency of commentators, image matters less, even if it correlates to skills. But image matters quite a bit when there is hierarchy and standardized “job evaluation”. Consider the chief executive officers of corporations: they not just look the part, but they even look the same. And, worse, when you listen to them talk, they will sound the same, down to the same vocabulary and metaphors. But that’s their jobs: as I keep reminding the reader, counter to the common belief, executives are different from entrepreneurs and are supposed to look like actors.”
-
Comparing Open book exams And Closed book exams
Since the covid-shock, educational thinkers have been vigorously debating alternative assessment methods to suit the changed conditions. A 2016 systematic review(Medical Ed) comparing Open book exams vs Closed book exams(shared by David Carless) provided some insights about the use of open-book exams as an alternative to regular assessment methods. The review looked at six outcome categories: (1) examination preparation, (2) test anxiety, (3) exam performance, (4) psychometrics and logistics, (5) testing effects, and (6) public perception.
Conclusion suggest that no sufficient evidence exist for exclusively choosing either Open book exams or closed book exams. According to the researchers, a combined approach could be more significant. To quote Prof. David Carless “Open book vs closed book exams: mixed evidence but students prepare more diligently for CBEs”
The study also mentioned comments made by students(on OBE); ‘that they were less prepared for a final examination that they knew would be an OBE because they expected to be able to find the answers in the book during the exam. Because of less experience with the OBE format students misperceived its difficulty. As the following tweet by Tim Fawns suggests, students later desired additional practice/training for the OBE format.
My hidden curiosity about this topic was/is to find if success in CBE predicts success in OBE. I am curious about this because it can shed some light on the predicting power of standardized tests(PISA)/IQ, etc. on general performance in an open book real world.
-
Goodhart’s Law & Metric Fixation
Following are 2 similar concepts which maybe useful for understanding rigid nature of measurement and evaluative cultures in organizations.
Goodhart’s Law suggest that, when a measure becomes a target, it ceases to be a good measure.” I.e., when we set one specific goal, people will tend to optimize for that objective regardless of the consequences. (Eg. Exponential changes)
Metric Fixation: According to Jerry Z Muller (from Aeon article), “The key components of metric fixation are the belief that it is possible – and desirable – to replace professional judgment (acquired through personal experience and talent) with numerical indicators of comparative performance based upon standardized data (metrics); and that the best way to motivate people within these organizations is by attaching rewards and penalties to their measured performance.”
-
Tweet: Feedback turned in to labeling and self-fulfilling-prophesy
This thought needs attention. Feedback is important but when feedback becomes labels its destructive. Students should not carry the burden of systemic labeling and the resultant self fulfilling prophecies which haunts them through out their life.#learning #education https://t.co/s6YStvB5B0
— Kiran Johny (@johnywrites) November 28, 2018
-
Tweet: Early evaluation,early feedback and early intervention for student success by @AliceSNKim
Early evaluation,early feedback and early intervention for student success; Driven from the conclusion that the grades students receive early in a course can be used to predict how they will score on a cumulative final exam. https://t.co/f4yUmzSSHr
— Kiran Johny (@johnywrites) November 4, 2018
In the paper“Early, But Not Intermediate, Evaluative Feedback Predicts Cumulative Exam Scores in Large Lecture-Style Post-Secondary Education Classrooms”, researchers Alice S. N. Kim and Sharry Shakory investigated whether early and intermediate evaluative feedback on in-class quizzes were predictive of students’ scores on a final cumulative exam in a third-year Psychology course at a large North American university.The results of the regression analysis showed that early, but not intermediate, evaluative feedback was predictive of students’ scores on the final cumulative exam.