Over the last few years I have examined a dozen or so on-line courses. I do so having begun my teaching career in 1963 and having had Faculty positions at such institutions as MIT, GWU, Columbia and NYU/Poly. Over more than fifty years I have had thousands of students and thus have a modicum of experience, albeit not in the new on-line context. Thus I bring perhaps some bias of the old school to my opinion and my perspective.
To examine this I will consider a few applications. One of the methods the on-line educational systems use for grading their
“students” is the process of what they term “peer review”. Based upon my
limited but to some degree in depth analysis, it consists of the following:
1. Students are asked to answer certain questions in a prose
like manner which means paragraphs and not bulleted sentences. The first
problem here is the very definition of prose. To those of use with some
educational background prose is merely the absence of poetry. Poetry however
may lack structure of rhyme and rhythm such as free verse and thus defining
something by what it is not when what it is not cannot be defined leaves one in
a conundrum. However when one has half the students coming from non-English
speaking backgrounds this leaves them cold, as apparently it does the
instructors. Thus for the sake of simplicity we would then define “prose” as
complete sentences collected together in a paragraph.
2. Now as to the “peer” nature of the students we have a
problem. The students may range from PhD/MDs to High School drop-outs. They may
have no initial knowledge of the subject, they may have some education in the
area, or they may be professionals with publications. Thus the students are
often a true broad based spectrum of individuals. Yet for the sake of this peer
review process they are all considered equal. In a sense it becomes the ultimate
Millennial world view that everyone’s opinion has equal merit. The discovery of
any scientific fact is non-existent. Furthermore there is also a massive
culture clash. In certain countries and in the highest ranking schools one is
judged by one’s ability to seek out facts from the observations. In contrast,
in certain countries, one is judged by one’s ability to memorize exactly what
is taught and to return what is memorized flawlessly. As I will later note both
approaches place the student at risk.
3. Instructors often add to the confusion by contributing
their own personal views on how grades should be determined. The instructor often
seems to believe that by performing this peer review process that the students
will learn. The major problem is that the students who grade others bring to
the table many personal and cultural biases and prejudices. For example, the
students are actually instructed to see if the answer is literally “word for
word” as the instructor may have indicated. If not, then the grade will be lowered
in some totally random manner. If the student responding to the question manages
to phrase the result in their own terms then they will be marked down. In the
Discussion sections one see the instructor and their alleged assistants
reinforcing this bizarre approach! Namely the answer must match what the
instructor wrote “word for word”, yet as we shall also see that would have been
plagiarism or fortune telling!
In addition, if the student wants to improve their own overall
ranking, then they grade everyone lower. If the student does not understand the
material, then whatever they give as a grade is meaningless. Thus we have cultural,
ethical, and intellectual challenges that impact the process and are reflected
in the student’s grade. In reality the grade all too often a reflection of the
biases of the grader and the instructor and does not reflect the students
comprehension. This is in no way a true real peer grading system function. In
such a system, there are true peers, persons with professional credentials
chose by an arbiter Editor for example, and a process of due process to rebut
any claims.
4. The issue of plagiarism is also introduced as an element
to be considered by the students in their process of evaluation. The issue of
plagiarism is highly complex. To claim one plagiarized is to accuse one of what
may, in certain countries, be considered a crime. It is a fraud, a theft, an
act which can have significant consequences. Thus it should not be treated
lightly. To do so there must be three elements; (i) proof that what is
delivered as being expressly represented as the student’s, (ii) a clear
demonstration by objective evidence that what is presented has been purloined
from a second source, (iii) a due process offering whereby the student can have
a hearing to remedy the claim. Accusation without those elements may very well be per se
defamation. It is actionable. Furthermore if the instructor inserts themselves
in the process in any manner, say by altering the existing definitions, then
the instructor becomes a party to the defamatory act.
This somewhat characterizes a peer review process. The
problem, simple, with thousands of students all having different ethical
standards and almost all never having graded a test in their lives and having
confused directions from the woman instructor, this led to chaos. If one did
not answer word for word as the instructor demanded after the fact the grades
were reduced, and plagiarism was alleged a multiple of times without basis, in
my opinion, due to the interference of the naive instructor.
The solution is simple, avoid these "peer" reviews
at all costs. First there are no "peers" in this process. Second, remove
the commentary by both the instructors and the alleged "Community
TAs" who especially seem to be grossly unqualified. I have seen one who
alleges to be an out of work teach-at-home parent with no experience in the
real world! They somehow seem to feel that their world view should be
everyone's. Third, Instructors should let the Honor Codes speak for themselves.
Once they get into the fray they tend to add their own ideas which frankly for
almost all Instructors go well beyond their core expertise. That is what they
have lawyers for; Instructors should teach and not practice law.
Now in Science and Engineering one should be able to avoid
"peer" review of written materials. Yet the worse example of this
practice going awry is in the Science area. The excuse of the Instructor is
that it gives the student a feel for real peer review. The problem is that real
peer review is real peer review, and it has the chance to remedy complaints. It
has the ability to have a due process. On-lines do not.
The net result is that on-lines have so many failings that one
wonders if they are converging on something that works or diverging into chaos.
One awaits the lawsuits. They may very well come.
Now there is another element in on-lines which is a potential
concern; the Discussion sections. Here we have the alleged “student” using this
to voice their own opinions and views as to the material. This can be a
potential mine field, especially one of generational warfare. Again in the
context of the Millennial mindset is that each opinion has equal value. There
is no basis of facts, not ability to judge based upon analysis and prior art,
no true scientific method. This is merely a mechanism for the “students” to
voice their world views, and one must be careful if one’s views, even if based
upon facts, are at odds with some of the participants.
A second dimension is the Discussion boards. I had for two
years looked but kept quiet. They seem to be filled with big ego Millennial who
seem to want to give an opinion of tell the world of their perfect grades. I had
an experience where I decided to post a technical comment along with a
technical basis to support the observation. Then out of nowhere some, in my opinion, rather bizarre
character emerges who begins to lecture me on what I said with no basis in fact.
Bizarre is an understatement for the way the interaction went.
The only remedy
is to remove the observation and let the responses speak for themselves. Such
an interaction was unseen at MIT and frankly would most likely not be
tolerated. However with the “Facebook” like Discussions it becomes a Wild West
of opinion. That in my opinion is not productive and is destructive. But these
Discussion boards seem to foster such an attitude. There is no use of facts, no
logic, no specifics, just gross and oftentimes arrogant anonymous attacks. Pity
humanity if these folks take over, they just most likely will not be tolerated
long. The core problem of on-lines is the inability to have any adult supervision.
Now what makes an on-line work, if anything? I have tried to give
some thought to this process.
The on-line concept has been around now for a couple of years.
I have followed several of them in some detail. I have taken a few courses both
to experience the process and to learn what they are presenting. To be honest,
in all cases I have had the material before and in many cases even have written
on the topics, some but not all. Thus, unlike many students who approach it
fresh I had seen much of what was presented. On the other hand I spent no more
than 3 hours per week on a course that required twelve. Thus, I did not “study”
for exams. Instead I used the exams as a learning experience rather than a
valuation metric. That works well especially if one’s intent is to learn the
material. In fact the issue of grades is itself specious. There is no value to
the grade, there is no validation that you learned anything and moreover even
if there was such an acceptable validation the courses themselves are all too often
limited in scope.
Let me now present some observations on what works and what
does not in several key areas:
1. Lecture Style
There are two elements of a good lecture style: First,
having a live class makes a world of difference. You see others and you feel
part of the process. Second, the classic
MIT approach as demonstrated by Lander which is to develop and explain two or
three topics or ideas at each lecture so that the student gains critical
insight. It is NOT to read through a collection of power point slides or to
ramble on with dozens of facts without any attempt to connect them. Power point
presentations do not work very well. They foster too rapid a presentation. Poor video quality does not work at all. Assuming students are Juniors at a Public
High School does not work. Interjecting one’s political views do not work. Using local colloquialisms do not work. Also
there is the general problem of accents, and that is always a problem.
Unfortunately there is no way to remedy that.
2. Exams
Lander’s exams are perfect. They make you understand the
issues including the details. No one else seems to do this. The multiple choice
exams are the lazy way to deal with the subject. Having others student’s grade
is worse than lazy.
3. Grading
Here I have a conundrum. Some students want to score
perfectly and then let everyone else know their alleged performance. The grades
may or may not reflect a gain in knowledge. In Lander’s course the feedback
often lets you know that you were sloppy, you guessed too quickly. Grading
should be interactive, reflective of new knowledge, fair, balanced. Frankly,
grades are just a mechanism to let you know what you did wrong. The problem is
that many foreign students seem to think they have completed an MIT or Harvard
course. The issue of ambiguity of expectations is a major problem.
4. Peer Grading
There is no advantage conceivable. Peer grading is an
abomination. There are several reasons:
a. There are no real peers. It appears that there is an
amalgam of various egos that want to be self-assured of their own value and
will do anything to get it.
b. Peers generally know next to nothing. Thus, especially if
the English and American systems are different for them, they then interject
their own values. That often leads to massive misunderstandings.
5. Discussion Groups
I guess in the younger set it enables the communicating that
they are used to. The intent of the discussions groups is valid, namely to
discuss the topics, but they all too often turn into a “Facebook” type dialog. There
is all too often a rambling of personal issues and the overly sympathetic
colloquialisms that detract from the intent from the course. I have yet to see
the type of dialog one would have at a truly first class graduate school no
less a competitive Secondary School. The overhead of watching what is said is
not worth the cost. I see no real benefit. There are those complaining that
they got a bad grade, those puffing up about a good grade, those making
politically useless remarks about something that has nothing to do with the
course material. On the other hand, Discussions have great potential,
especially if managed by Teaching Assistants who are educated and trained in
the art of managing them. Having TAs who are unqualified just reinforces the
chaos.
6. Material
I have generally managed many without a text. A few demand a
text just to be able to do the required manipulation. Some courses had what may
be considered notes but they were few and far between. Lack of a text or class
notes can be a disadvantage for many students. Faculty Capability is also a key
factor. Lander is the sine qua non. He knows the topic, he presents it well,
he engages the students, and thus the listener. It appears that some educational
institutions get an instructor who simply has the time and wants to do this.
That approach shows immediately.
7. Expectations
The courses are NOT true University courses. They are
abbreviated at best. Lander’s course is close, and in fact as close as one
gets. The others are long Seminars. Thus one should expect to learn something
but not to be as accomplished as an MIT or Harvard student. There is a balance
which must be met. The MIT Electronics course found some student in Mongolia
who allegedly scored 100%. I gather they then tried to get the student into
MIT. Yet that is one in 100,000 or more. The dropout rate is 95% or greater. In fact of
the courses I took, I dropped more than half. The reason was quality of the
Instructor or the lack of quality in the teaching materials on line.
Thus on-line courses face challenges. There are massive ambiguities of
expectations. The cultural variances are quite large. These factors must be
examined and considered. If all the courses were like Lander’s Biology course
it would be a perfect world. However it is not that way. But improvement is
essential and this can be a powerful tool if well done. It can be a waste of
time if not.