It's About Time C

[saturnian]

I finally got my hands on the nationals test from last year... and was surprised at the type of questions that were asked: only four required any knowledge of physics (and this is a physics event). The rest were more of a Jeopardy-style. (For example, the frame rate for NTSC TV?

Duh, 29.97.

/nerd

and how long did it take you to think to come up with the solution? Which physics concepts did you have employ? Any formula you had to apply?
Memorization or printing of facts is not physics and not science.

[andrewwski]

None at all. I'm not saying it's a good question. I don't think it is either.

[scienceolympiadist]

So, a member of my team who is writing a practice test loves physics. After all, the rules do say about Newtonian physics. But really, how much of the mechanics you learn in like physics classes is actually covered?

[anonymous123]

So, a member of my team who is writing a practice test loves physics. After all, the rules do say about Newtonian physics. But really, how much of the mechanics you learn in like physics classes is actually covered?

At the national level there were two kinematics problems (linear motion: one with acceleration and one without), one simple pendulum, and one time dilation (out of 25). You can get the test from SO store.

[manutd94]

Those questions were pretty intense (I got the test as well in the summer) making for a challenging National final. Nevertheless, I think questions that hard are good because then it just shows how broad the best resources need to be in order to succeed in such an event.

I cannot believe any of you thought the test was hard. The only problem that I would consider of moderate difficulty is the one about time dilation. And the rest ... How hard is it to lookup facts? If you feel 'brilliant' by being able to google 'dog watch' and then spit it back on the test, you may enjoy that test. Frankly, I think the test was embarrassing for a National Science competition.

Saturnian, though I said that the test was challenging, by no means did I mean the concepts. What I was referring to was the diversity of the questions and the ability to have a great resource in order to succeed on the test. Saying that the only problem you considered to be of "moderate difficulty" was the time dilation one may be true, but that was definitely not the only one you may have gotten wrong because the highest score on the test was a 18/25. If the test was that easy, then you should have gotten at least a 23-25, which from my knowledge you didn't.

[chalker]

I'm finding the ongoing discussion about the test portion of this event interesting, but thought I should chime in briefly in order to clear up what I think are some misconceptions. As I've mentioned before, I'm the National Event Supervisor for It's About Time and am always receptive to suggestions and constructive criticism regarding the event (in fact some of the suggestions people made earlier this summer were incorporated into this year's rules).

Unfortunately, other than at National's and the Ohio State tournament (which I also happen to run), I have no control over what other regional and state event supervisors do regarding this event. Thus my hope is this thread, and my comments in particular, will help steer people in the direction my colleagues and I on the National Physical Sciences Events committee have for this event.

Thus:
1. This is a Physical Sciences event, not explicitly a Physics event. Likewise, it is NOT a Technology & Engineering event. Thus while there is a building component to the event, the 'knowledge testing' component is equally important and weighted the same. Essentially we are trying to include both 'practical' and 'theoretical' aspects related to the very broad field of 'time'.

2. As the description at the top of the event rules states, the test involves a wide range of issues, including "the concept of time, timekeeping, astronomy, physics, and mechanics". Thus the questions should cover an equally broad range, and not just be focused on physics-related calculations.

3. The soinc.org website has a great page outlining the purpose/mission of Science Olympiad (http://soinc.org/mission). Of particular relevance here are the Science Olympiad Tournament goals, which include:
"To bring science to life, to show how science works, to emphasize problem solving aspects of science and the understanding of science concepts" and "To develop teamwork and cooperative learning strategies among students". Note that problem solving is only ONE part of the goals. Understanding concepts and cooperative learning strategies are also emphasized. While 'printing of facts' may not be viewed as 'science', the process of preparing your team to respond to such questions most definitely requires special learning strategies and teamwork, which furthers the SO mission. Thus what's 'hard' about these types of tests isn't necessarily what transpires during the actual testing period, but rather what happens in the months prior trying to prior for all the possibilities.

4. Where possible, we want to avoid testing 'rote memorization'. As an engineer, I can personally attest to the higher 'real world' value of being able to quickly research, reference, and synthesize information from a variety of sources in order to respond to an issue, compared to just 'number crunching'. By allowing a resource binder to be brought into the event we are trying to encourage those types of skills. I've consistently found that the best teams don't bring a huge unorganized binder into the event because there just isn't time to search through the thing. Rather they digest their research down into more manageable chunks, and in the process are LEARNING, which oftentimes allows them to 'mostly know' the answers, and thus usually only need to refine their response via looking at the binders.

Finally, I'd be interested in hearing what anyone who took the test or has gotten a copy of it since has learned as a result of it. For example, I bet a LOT of people had no idea what a 'dog watch' was, but as a result of this now probably know about the way time is kept on ships at sea (which might be of value if any of you end up joining the Navy;). I've already gotten some suggestions for specific 'sub-areas' of things related to Time that weren't covered last year and thus will probably show up on the tests this year;) And I'm always open to more suggestions.

I'll conclude with another quote from Ray Bradbury's Fahrenheit 451: "Most of us can't rush around, talk to everyone, know all the cities of the world, we haven't time, money or that many friends. The things you're looking for, Montag, are in the world, but the only way the average chap will ever see ninety-nine per cent of them is in a book. Don't ask for guarantees.
And don't look to be saved in any one thing, person, machine, or library. Do your own bit of saving, and if you drown, at least die knowing you were headed for shore."

PS: Saturnian, sorry to hear you were the one stuck in the elevator at OSU last year. The situation wasn't brought to my attention until well after all the teams had finished competing in the event and if I would have let your team rerun the device potion it would have been unfair to the other 39 teams since you would have had the advantage of approximately knowing what the selected time intervals were. I know your coach discussed the situation with the arbitrator and they reached the same conclusion. While we try to be as accomodating and fair as possible, we can't account for and resolve all possible situations.

[scienceolympiadist]

PS: Saturnian, sorry to hear you were the one stuck in the elevator at OSU last year. The situation wasn't brought to my attention until well after all the teams had finished competing in the event and if I would have let your team rerun the device potion it would have been unfair to the other 39 teams since you would have had the advantage of approximately knowing what the selected time intervals were. I know your coach discussed the situation with the arbitrator and they reached the same conclusion. While we try to be as accomodating and fair as possible, we can't account for and resolve all possible situations.

I faintly know about that story, but from what I see...couldn't the team have gotten a different time? After all, it's the difference between the recorded time and the actual time, that's counted for scoring, delta t, not the actual t value. Just for future potential situations, could that have been an option?

[chalker]

I faintly know about that story, but from what I see...couldn't the team have gotten a different time? After all, it's the difference between the recorded time and the actual time, that's counted for scoring, delta t, not the actual t value. Just for future potential situations, could that have been an option?

Both I and the arbitrator didn't think that was an option that would be fair to the other teams for a variety of reasons. For example, consider the 5th time trial, which can be any time between 10 and 300 seconds. Several issues typically make the devices much less accurate at the longer time periods, such as pendulum dampening, user fatigue, imprecise periods, etc. etc. I use a computer program to randomly pick the time periods (in this case in question it happened to be 193.4 s). If during the 're-run' the team got a very short time period, such as 20 sec, they would be given an unfair advantage over the other teams and would probably have better accuracy as a result.

As an aside, another issue we took into consideration resulted in one of the rule changes this year. At this particular tournament we were able to have all 40 teams compete at the same time, in the same room. Unfortunately we didn't realize that some of the devices were rather loud during operation, resulting in a strange chorus of clicks and such that were all slightly out of sync and not necessarily the same rate. Many students were noticeably distracted by this and even tried to block out the noise by putting their fingers in their ears. I'm sure this affected their concentration and resulting accuracy. If we had let the team in question re-run by themselves, they would be in a 'pristine' auditory environment and thus having an advantage over the other teams. The addition of rule 3.f. about minimizing impact on other teams had it's genesis in this situation.

[saturnian]

1. This is a Physical Sciences event, not explicitly a Physics event. Likewise, it is NOT a Technology & Engineering event. Thus while there is a building component to the event, the 'knowledge testing' component is equally important and weighted the same. Essentially we are trying to include both 'practical' and 'theoretical' aspects related to the very broad field of 'time'.

2. As the description at the top of the event rules states, the test involves a wide range of issues, including "the concept of time, timekeeping, astronomy, physics, and mechanics". Thus the questions should cover an equally broad range, and not just be focused on physics-related calculations.

...

I agree that the rules state that this is not purely physics event and questions can cover wide variety of areas concerning time; however, I would have liked to see questions that require knowledge of the subject rather than facts around the subject. For example, rather than asking "who invented dead-beat escapement?" (just a fact) you could ask "what is the difference between dead-beat escapement and anchor escapement?" (requires knowledge of the subject). Similarly, you could ask "why the time of vernal equinox changes from year to year?" rather than "what time is the vernal equinox in 2009?"

[chalker]

I agree that the rules state that this is not purely physics event and questions can cover wide variety of areas concerning time; however, I would have liked to see questions that require knowledge of the subject rather than facts around the subject. For example, rather than asking "who invented dead-beat escapement?" (just a fact) you could ask "what is the difference between dead-beat escapement and anchor escapement?" (requires knowledge of the subject). Similarly, you could ask "why the time of vernal equinox changes from year to year?" rather than "what time is the vernal equinox in 2009?"

This is an excellent point and I agree these types of questions are much preferred since they are the best way to test comprehension of the subject. However, I think that the logistics of the event / tournament prevent event supervisors from using these essay type questions. Consider that:
1. The event blocks are typically 50 mins (although sometimes they are 60 mins)
2. The written test is half of the score, and worth 50 points
3. Teams are supposed to be given 20-30 mins for the written test (you really can't extend that due to the time required to do part I)

Thus, my concerns are that:
1. There needs to be at least 25 questions on the test. Any less and you are giving too much weight to individual questions compared to the individual time trials. As a result, a team of 2 working together has at most 1 min per question to read and respond. If the team members split up the questions they have at most 2 mins per question. I don't think that's enough time for most teams to respond in essay form to a significant number of questions. The test needs to 'cater' to a variety variety of levels of team expertise (i.e. in an ideal situation there will be some easy parts most any team with a little preparation can get right, and some hard parts that only the best of the best have a chance at correctly responding to)
2. When you have an essay type response, you often times provide the option for 'partial credit' to be given. However, that introduces a significant amount of subjectivity to the grading process, and can sometimes require some thought / analysis / additional research on the part of the grader. As an example, I once asked a question about what a Blue Moon was. One team gave a scientific sounding response of something like 'a bi-nodal phase alignment' that wasn't the 'correct' response. However because it might have been technically correct I had to look into it and ended up doing a Google search and then finally questioning the team directly about it - they admitted they just put down some technical terms. All of this took a lot of time, and.....
2. Unfortunately grading / scoring time is at a premium. For example, at the Ohio State tournament there are typically 40 teams. Let's say there were only 15 essay type questions on the exam. That equals a total of 600 essay responses to grade. Let's estimate it takes on average 20 seconds to read and grade each response (some will take less, some more obiviously), that means it will take a total of 200 mins (almost 3.5 hours) just to grade this part of the test. Even with the help of a couple volunteers to do other tasks like grading the time trials, helping to record scores, passing out and collecting exams, etc. it is a significant amount of time in an otherwise VERY busy day. The final scores need to be submitted within about an hour or so after the last time block in order for the awards ceremony to be held at a reasonable time. And of course the national tournament is even more challenging since there are 60 teams competing.

As I've said many times, I'm open to suggestions on improving and running the event. Thus I'll put the problem back to you. How would you design the test such that these issues and concerns are accounted for? If you don't think the issues / concerns I've outlined are relevant, why? How many questions and what types of responses would you be looking for? What should be the balance between easy, medium and hard questions? Should they be equally weighted or should partial credit be given (if so, how should it be determined)?

P.S. - Your suggested questions would probably cause a lot of issues. For example:
-A deadbeat escapement is a FORM of anchor escapement. What you are really asking is the difference between a 'traditional design' anchor escapement and a deadbeat design anchor escapement.
-The wikipedia entry about this is pretty good and reasonably concise, but it still requires several paragraphs to detail all the differences (not just physical, but operational and quality of timekeeping). Would the response have to detail all these 'differences' for full credit? If not, would it be fair to give the same score to a team that just mentions the physical changes vs. a team that details the differences in operation and such?