[Bestplus] Fw: scoring BEST Plus

[Iris Broudy]

Here is Lynda's response to my questions.  What still throws me a little is a response of one or two sentences that are highly complex.  That doesn't seem to be addressed directly in the rubric, but since a 3 has to be a "string of sentences,"  the only possibility then is a 2.  I guess we can assume that a high-functioning speaker at some point will give a response with several complex sentences (just as 1's in the warm-up questions don't necessarily indicate a lack of ability).  

On the student who went off-topic, his expanded response was not totally unrelated, but it was far enough from the original question that I think I would give a 1 on listening comp. and then a 3 in language complexity.  We can't know for sure how a student understood a question, so we just have to make a judgment call, I think.  (And as Lynda kept reminding us, it's only 1 point.)  

BTW, I would not tell an examinee in regular testing not to worry about grammar.  I only said it in this case because I knew this student would be concerned about being correct in a testing situation.  But it didn't matter much anyway, did it?  She still worried.  (At least in regular communication she's more concerned about getting her ideas across than being error-free. . . . I wish I could be that way about my Spanish!)

It's fascinating to observe what the software needs to do to consider the test complete.  Yesterday I had an SPL 2 that took 13 minutes, then an SPL 3 that took only 7 minutes.  

Enough rambling.  Take care, everyone.

Iris Broudy
Hampden County Sheriff's Dept.


----- Original Message ----- 
From: Lynda Terrill 
To: Iris Broudy 
Cc: andree.duval at sdh.state.ma.us 
Sent: Monday, March 29, 2004 4:23 PM
Subject: RE: scoring BEST Plus


For Iris Broudy:

Hi, Iris,
You've asked some thoughtful questions, and I will do my best to answer them, one answer for each of the main paragraphs

One well-formed sentence with emerging complexity (because, if) to provide some detail that is minimal but beyond basic is a 2--just as you said. The rubric says "sentence(s)" because a single sentence could demonstrate the emerging complexity/beyond basic detail. Benchmarks video #8 is an example of this. 

The second issue is a little bit harder to talk about because it's harder to explain without seeing the performance.  In some cases--probably rare if the person is high intermediate and above--you might find out that person really didn't understand fully (so you would have to give a 1 in listening). However, I think I mentioned it in the training, a person at the high-intermediate or advanced level, might truly answer the question, and then move a bit to another topic.  For example, someone might answer the seeing the doctor question and start talking about his experience with dentists. That's no problem.  I guess what I am saying is that my answer to your question depends on what you mean by "off-topic." Another scenario would be that a person really goes off topic like, "I'm getting married next week!" The test administrator could smile and look happy and say something like "Let's finish the test." The student's comment was really parenthetical and not part of the test. If a person was rambling so much that you couldn't follow where she was going except that it was away from the original question and answer, you could score a 2 in communication for being sporadically difficult to understand.  I also think that as you become more seasoned in giving the test, you will be able to move to the next question more quickly while still being polite and non-agressive.  Also, we do need to score the whole sample. Remember in the scoring practice #11 (3rd question) where the woman is talking about what rules a person needs to learn, she starts out great, but falls apart later, and we have to score the whole sample. I hope this helps--see what happens as you have further interviews

Again, I think that this third question will be ameliorated as you become more familiar with the test and the processes.  It did for me.  Also, using the keyboard (maybe you already do) rather than the mouse seems to make the scoring quicker.  Beyond program limitations, you wouldn't want to have an interview continue so long that it seemed like a trial to the examinee. In the case you mention, I wouldn't tell the examinee anything much except the most basic information--as little as possible--and in passing.  I don't think it usually helps people to say they don't need to worry about..something.  I usually smile and say something like "I'm going to ask you some questions" or "this examination is to see what class (or level or book) is good for you."  However, I've had examinees who just really want to spend the time to be careful and self-correct. You will get more adept at moving the interview along, but sometimes this happens.

You sound like you are taking the interviews seriously; that's great.  I hope this has helped, and feel free 
to email be anytime.  As you see, it sometimes takes awhile for me to get back  because swamped is my default mode. 

Lynda

  -----Original Message-----
  From: Iris Broudy [mailto:arcoiris17 at msn.com]
  Sent: Saturday, March 27, 2004 9:44 AM
  To: Lynda Terrill
  Subject: scoring BEST Plus


  Hi Lynda,

  I spoke with you the other day about what seemed to be too high a score for my second practice test.  I've now done 10 tests, and overall the scores seem appropriate, although one more did seem a bit high.  So I'm looking at my use of the rubric.  One problem that keeps coming up:  If a student gives a brief answer--maybe just two sentences, but one sentence is complex, with a subordinate clause or imbedded clause, what should the score be?  A 3 requires a "string of sentences," correct?  Can a 2 be just one "well-formed" sentence,  so long as it's not just a S-V-O structure (with "if", "because" or the like) and provides detail beyond the minimum?  I keep looking at the rubric, but it's not always easy to make a call.  

  One other thing that has cropped up:  I encountered a student who likes to talk and tends to go off on tangents.  He seemed to understand the initial questions (the higher level questions) and would answer them, then would produce a lot more language, but off-topic.  Eventually I realized that I had to be more aggressive in cutting him off politely.  (The test lasted for 24 minutes.)  But what about the scoring?  If he goes off-topic, should that be a 1 on listening comprehension?  At the training it was clear that we should score language complexity on what we get, so even though he was talking about other (though related) topics, I gave him 3's where appropriate.  He spoke so much at times that a 4 could be considered, but because of lack of cohesion and organization, a 4 would not apply. 

  Yesterday I tested a young woman who was so concerned about forming perfect sentences (she's Dominican but reminds me of Japanese students I've had in the past!) that the testing process was very slow.  (24 minutes for SPL 6)  I told her in advance not to worry about grammar, just have a conversation with me, but it didn't make much difference.  Do you have any suggestions for how to handle such a situation?  I didn't want to cut her off when she was trying so hard to express her ideas with correct structures.  (If I saw that she was lacking the vocabulary she needed, then I would say, "That's okay," and go on.)  Should there be a concern about timing for the test, other than program limitations?  

  I can see how important this practice phase is in the TOT process.  I expect such questions will come up when I'm giving trainings, and I'd like to be comfortable answering as much as possible. 

  I'm working on getting e-mail at work, but for now I have to communicate from home, which means I can't get any responses until evening.  If you want to get back to me on Monday, you could use the e-mail address of our education manager, Andree Duval, who will pass your message on to me.  Her address is:
  andree.duval at sdh.state.ma.us

  Thanks for your patience with my learning process.  

  Regards,

  Iris Broudy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.literacytent.org/pipermail/bestplus/attachments/20040330/ee5ffcd3/attachment.htm