Test Set Exchange Update

[EastStroudsburg13]

As I mentioned earlier, teams don't go to invitationals for the tests. They go there for the experience. The tests simply happen to be a good practice platform that the teams can use for further analysis and reference. Also just because it's unrealistic for every tournament to release tests doesn't mean that we can't do our part. It will hopefully start a precedent that will eventually reach us to that point. And those teams that couldn't attend tournaments would love to pay for those teams. However, they simply can't because they're so new that they realistically can't reach that point in which they can go to invies constantly until maybe Year 3 or 4. Until then those tests will give them a nice starting point so that they don't go into regionals and states totally unprepared.

I know I'm biased, but I'm with Nick. Releasing tests has positive consequences for new teams, but also realize SO is a competition. Yes it's about learning, exploring, and building connections within science. That's great and SO has given me lots of opportunities and knowledge that I wouldn't have had without it. That being said I'm not only here to play nice with other teams and help them. As Nick said Solon fully supports new teams wishing to attend our invitational, but you must also think about experienced teams. You are neglecting their perspective and assume that the only effect of releasing all tests would be for new teams to gain resources whereas in reality experienced teams are losing an edge on competition against their primary competitors as well (ignoring the new teams as competition in the short run). How would people feel if Troy stopped attending invitationals and just asked teams for tests from their invitationals? I guarentee you there would be few teams willing to just hand them tests.

I know this sounds cut throat and maybe even a bit mean, but this is the reality of the program.

Im with Nick and Scott on this one. Teams are free to go to whichever tournaments they want. If you dont like the way one tournament is with their tests/principles, go to a different tournament. I think tournament directors should be free to do whatever they want with their exams and resources.
As a regional tournament director myself, I make all of our exams public immediately after the tournament, but i have an additional motive - I WANT other teams to take our exams, especially those in Florida, realize that our tournament can actually follow the rules for each event, and then choose to come attend here. When I supervise ANYWHERE (including Florida regionals and states, MIT, Princeton, etc), I make certain that my test is made public whether or not the tournament venue wants that or not. I wrote the test - it is my intellectual property and I can choose to share it if I want to.

I can't say that I think it is a GOOD thing for invites to not release exams publically, but I also believe in free-market capitalism and forcing tournaments to share their exams so that "every team has an equal advantage" sounds awfully socialist to me...

Trying to compare SO to a free capitalist market is a very dangerous road to go down. This is, at its foremost, an educational activity for teenagers.

[syo_astro]

I don't think a capitalism vs. socialism debate should be had on this thread unless that's REALLY necessary...I don't think it would end well. Anyway, I don't think the sides will see eye to eye exactly without being more informed. For me, it's more fruitful to then ask the benefits of competitiveness with tests.

What follows from here is me honestly being confused. I'm aware there are other benefits to invites, but clearly I don't get the importance of these tests. Hopefully not dumb to ask...I'm going to type a lot, but to nicholasmaurer, etc feel free to give a brief answer if it's obvious (a lot of these questions are similar):
People can trade digital copies on a whim, which probably (easily) gets back to the big competitive teams. Here, I'm assuming random "big teams of concern" work hard at test collecting, have a wide enough network, etc...fair?. Is this the biggest fear among the most competitive teams running / attending invites? Here, I'm ignoring fundraising or benefits to the host since that's not an incentive to all participants...meanwhile tests are one of the bigger / biggest incentives to all attendees. Did I get that right? Why not then ONLY give back the paper copies to curb other teams getting your tests? I am aware this has been done in the past. I'm sure teams could still take pictures or make scans, but it would at least slow things down since the pictures would need to be retyped or reformatted to make them clean copies.

How do you draw the line between what helps "other teams" less and what helps "our team" more for either teams that host OR attend? Moreover, if tournaments give people a website, google drive, or flashdrive of tests (maybe not everyone does this...but many do something like this now, right?), then what are you actually doing to curb "helping the competition"? Maybe there's an extra factor here, like those are things that attract good teams, but don't those teams want a competitive edge too?

People have said they basically want to help their team more than other ones, is that right? It sounds like people agree that invites are ran for reasons other than tests like seeing how you stack up vs. other teams, etc etc (am I wrong?). Like, if the only reason for invites were tests I'm sure students could come up with some alternative complex "black market" test trading network between individual coaches (can't believe I typed that lol). In fact, why not do that anyway with or without invites?

To summarize, how do you balance everything above in your value "equation" or whatever metric you use? I'm going to go on loop here, but if invites have to be ran, but you're afraid of tests being traded, why isn't more done to curb test trading? If the argument goes back to "because we want more tests than other teams" or "attendees don't like that", then see above...does test trading make your tests not help the competition at least equally? Is test trading not that widespread among teams you care about as I thought in my first point? Hopefully you see why this at least appears contradictory (or if you prefer extremely convoluted) to me.

OK, now my briefer two cents:
I find it hard to believe anyone can practice or make use of 60 tests. Like, just as far as I know about the practical usefulness of tests from a learning / education perspective or even for notesheets...I guess anything is possible. Maybe I don't have a good estimate of this "test value" considering I haven't competed recently.

[nicholasmaurer]

I agree that there’s a quickly diminishing return from acquiring more tests. If you have a few good ones, you can prepare well. There’s no reason to hoard 60 test sets a year.

To answer your broader question, I don’t think there’s any objection to test trading on our part. We’re not trying to prevent trading. I expect that many teams who attend our invitational will trade those tests for other sets. My point is that if we posted our tests publicly, the teams who attend would have nothing to trade. This seems unfair since those teams also wrote many of the tests and volunteered to run events.

In essence, we’d be saying to participating teams: “Thanks for paying to attend, writing these tests, and grading them for us. We’re now going to share them with everyone regardless of whether they paid or contributed.”

[syo_astro]

Thank you very much for your answer, sorry that was repetitive. I can see why you won't want to directly post them online if a school wants to host their own invite, has attendees paying, etc.

How are you or the attendees not against test trading even a little bit? (that one is a little rhetorical...read on) I understand the point earlier by you and others that trading gives your test "continued value". But if teams don't like the competition getting hold of their work when someone shares it online, why do they not fear that (or consider it a non-issue) while trading tests? Note: I repeat, I'm absolutely not saying you should post your tests or be in favor of your tests being posted, hopefully this last question is clear then.

But now say 5 invites are leaked online. Aren't the tests overvalued if they can be leaked or obtained so easily? It sounds like it's acknowledged the big teams will get all the tests anyway...? Again, apologies if I'm being ignorant here.

[nicholasmaurer]

Thank you very much for your answer, sorry that was repetitive. I can see why you won't want to directly post them online if a school wants to host their own invite, has attendees paying, etc.

How are you or the attendees not against test trading even a little bit? (that one is a little rhetorical...read on) I understand the point earlier by you and others that trading gives your test "continued value". But if teams don't like the competition getting hold of their work when someone shares it online, why do they not fear that (or consider it a non-issue) while trading tests? Note: I repeat, I'm absolutely not saying you should post your tests or be in favor of your tests being posted, hopefully this last question is clear then.

But now say 5 invites are leaked online. Aren't the tests overvalued if they can be leaked or obtained so easily? It sounds like it's acknowledged the big teams will get all the tests anyway...? Again, apologies if I'm being ignorant here.

I don't think schools expect the tests they contribute to be kept confidential - they know that when you distribute them to the 70 other teams they will be shared, traded, etc. I think the objection is to tests being given away for nothing. If the tests are leaked and posted online as you state in your hypothetical, that is obviously not ideal. But I can tell you based on conversations I have had with coaches, deliberately posting them publicly as the original intent would raise a lot of objections from our participants.

[Raleway]

Just my two cents:

This feels very much like the prisoner's dilemma, especially from the view of the tournament. In the event of publicly releasing tests, should some invitationals strive to keep an advantage over other invitationals, the dominant strategy is to withhold publically releasing tests. This exists for both sides, so the Nash equilibrium (generalizing the many invitationals) is to simply withhold tests and not publically release them. What I'm seeing, and very much enjoying, is that invitationals are realizing that their tests will be traded (even the watermarking was ineffective). Even if teams attend to get exclusive high-level tests, it makes no sense as a trading tool if every single test becomes publically released. At that point, teams will attend invitationals for quality, competition experience, and seeing how teams do against each other. As a competitor (soon to be a Division D member), I would very much want tournament quality to be the foremost priority. The issue of test trading is inherently the fault of invitationals; if every test was made public a week after the invitational, there would be no test trading.

None. Let that sink in. The onus is not on teams. Invitationals exist to allow teams a chance to compete against other teams and promote the spirit of collaborative science in a competitive setting.

I'm going to conclude with my final thoughts that SOINC should step in and require all invitationals register with SOINC and release tests and keys promptly to centralize these resources. The recent addition of "Invitational Ruleset" has been a foray into this but I hope it becomes more useful later. I applaud Princeton for being the first high-level invitational to publicly release tests and set off the next wave of invitationals to publically release tests (MIT, SOUP, etc.). Kudos to the aforementioned invitationals. Please make all tests public. It would save teams countless hours not worrying about tests to get and get rid of this issue once and for all.

[KC1FVS]

I am glad to hear everyone's take on this; thanks for explaining your thinking.

I think that for those of us who act as both a tournament director and a coach, then our obligation to be impartial should in fact inform the rationale of our test release policies. When deciding who should get our tests after the tournament, we are still acting as directors, not as coaches. Coaches who host or run invitationals should not be using the privilege of that opportunity to wield the advantage of proprietary information for one or more teams. If we are withholding them solely in order to incentivize registration and attendance at our tournament, then I suppose that’s justifiable, but I will explain in a moment why I have concluded that even this rationale is flawed. If the decision to withhold invitational tests from public release is explicitly founded in our desire to disadvantage non-attending teams, then I think we have mixed up our coaching hat with our directing hat.

Our tournament has itself been withholding tests in order to incentivize tournament registration, but I have conlcuded that this practice is misguided. Consider the response I just received from the student whose email initiated my post. When I expressed my hesitation to share the data from our invitational, they stated,

"...nothing can stop me from going to the other schools that attended your invitational and offer them the same deal and I am sure someone will accept it...”

So, we are not in possession of a valuable commodity. Our attempts to limit access will likely be undermined by students such as these. Getting the tests without attending is arguably easier than attending. If we all release our tests, there will be no need for this ridiculousness. The only reason I can see to limit test access is to incentivize tournament attendance, which frankly needs no such incentive. Teams and coaches will still want to attend as many invites as they can.

I am dismayed to find that there is an insider market of test exchanges taking place between teams who are lucky enough to attend several invitationals. Don’t we just want to help all students to learn to do science well and have the best students win on their own merits? Do we want them learning science or hawking information in elaborate schemes to leverage advantage? As coaches, we may be reluctant to surrender the advantage that these test exchanges can give our teams, but as leaders in SO (coaches and directors), we should model good sportsmanship, fair practice, and equity so that the best students can win. The best way to do this is to share information freely. If this results in another team winning, then they have earned their victory and we should congratulate them.

Our tournament has now removed the password protection on our tests, and I hope that other invitationals decide to do the same, regardless of whether they are run by coaches or alumni, for the benefit of ALL students in SO. They are on our site here https://sites.google.com/site/dartmouthscioly/.

[EastStroudsburg13]

I'm going to conclude with my final thoughts that SOINC should step in and require all invitationals register with SOINC and release tests and keys promptly to centralize these resources. The recent addition of "Invitational Ruleset" has been a foray into this but I hope it becomes more useful later. I applaud Princeton for being the first high-level invitational to publicly release tests and set off the next wave of invitationals to publically release tests (MIT, SOUP, etc.). Kudos to the aforementioned invitationals. Please make all tests public. It would save teams countless hours not worrying about tests to get and get rid of this issue once and for all.

If I could thumbs up a post, I would. I think they should also do this with state tournament results, but that's just for personal convenience. This test trading issue, in my opinion, is eating away at the spirit and values of Science Olympiad.

[syo_astro]

How are you or the attendees not against test trading even a little bit? (that one is a little rhetorical...read on) I understand the point earlier by you and others that trading gives your test "continued value". But if teams don't like the competition getting hold of their work when someone shares it online, why do they not fear that (or consider it a non-issue) while trading tests? Note: I repeat, I'm absolutely not saying you should post your tests or be in favor of your tests being posted, hopefully this last question is clear then.

Aren't the tests overvalued if they can be leaked or obtained so easily? It sounds like it's acknowledged the big teams will get all the tests anyway...?

I don't think schools expect the tests they contribute to be kept confidential - they know that when you distribute them to the 70 other teams they will be shared, traded, etc. I think the objection is to tests being given away for nothing.

So to the others above, I don't think nicholasmaurer, etc (sorry, not just directing this to you...just convenient you know >.>) would agree at least as scioly stands right now. If they can get a competitive advantage, they will. And fine, coaches will object for similar reasons as rules stand right now. If you read his posts, I think he makes this clearer than I could say, thank you again! In fact, it's clear we're talking at different issues.

Okay, paraphrasing the above conversation and repeating some questions because I'm not sure I totally get the "why" part of this...Why do schools expect tests not to be confidential and are okay with trading?

[Unome]

I'm going to conclude with my final thoughts that SOINC should step in and require all invitationals register with SOINC and release tests and keys promptly to centralize these resources. The recent addition of "Invitational Ruleset" has been a foray into this but I hope it becomes more useful later. I applaud Princeton for being the first high-level invitational to publicly release tests and set off the next wave of invitationals to publically release tests (MIT, SOUP, etc.). Kudos to the aforementioned invitationals. Please make all tests public. It would save teams countless hours not worrying about tests to get and get rid of this issue once and for all.

This is exactly what I don't want. I'm certain NSO will mess things up the more involved they get. It's a natural consequence of being distant from the ground level.