Society for the Discontinuation of Game On

[iwonder]

My proposal for improving the event is to model it after a UIL Computer Science or USACO competition. The basics are as follows:

-Use a classical programming language like Python or Java

-Instead of a game, give teams 50 minutes to solve a collection of problems like these (http://www.cs.utexas.edu/~scottm/uil/20 ... s_2014.pdf). Each problem has multiple test cases. Teams would be ranked by how many problems they can solve that work with the test cases

-There can also be a written test in addition to the hands on programming. The written test would be similar to this: http://www.cs.utexas.edu/~scottm/uil/20 ... ritten.pdf

In short, I do not feel that the current implementation of Game On is the correct way to implement a computer science event. Instead, I would recommend a USACO or UIL type programming competition of objective algorithmic nature. A written test could also be added.

I speak for nearly all of my fellow Science Olympiad competitors who program when I say that we think this would be the ideal way to run a computer science event in Science Olympiad.

And that, is how every computer science contest I've ever been to has happened. I think I see how Game On was picked because it looks like a fun event for the students, but it's highly subjective and really doesn't test a lot of true comp sci concepts. Personally, I think tests are a more direct approach for SciOly, write the tests in Java or Python, and the just like the UIL contests you can test competitors on search algorithms, syntax, object oriented programming concepts (at least in Java, I think python is less so?), etc, etc. It's easier for contests to run an event that doesn't require computers, it doesn't need as much standardization (versions of python, or versions of scratch), it's quick to grade (same as all the other events), and honestly SciOly competitors will probably not at all complain about a testing event, that's what a lot of them are anyways.

In other words, SciOlyers aren't known for learning the basics of a field. No event here operates at a surface level, yet, it seems like every Computer Science event that's been pitched has been at that beginning, surface level. I think a more SciOly-esque thing to do would be to go in depth into CompSci, which has the added benefit of being easier for competitions to run, and more like the other events.

[chalker]

....It's easier for contests to run an event that doesn't require computers, it doesn't need as much standardization (versions of python, or versions of scratch), it's quick to grade (same as all the other events), and honestly SciOly competitors will probably not at all complain about a testing event, that's what a lot of them are anyways.
.....

I have to profess ignorance of how some of those other comp-sci type competitions are run, and while I could look up the details, it sounds like all of you are familiar with them so I can just ask your opinions. I and several other people involved at the National level have been very interested in having a true comp-sci type event for a while (and I've even posted about it on SciOly in the past). In fact, when I competed in SO back in the early 90's we did have an event called Computer Programming that was exactly like described above: Pascal based, mostly written test, etc.

There are some very good ideas in this thread regarding the content of a potential event. However, the problems we always run into involve the logistics. Consider the following:

-There are over 300 tournaments run each year, nearly each weekend over a multi-month period. That's over 300 event supervisors all over the country, with a wide variety of knowledge and time available to devote to the competitions.

-Tournaments range in size from a handful of teams to over 100 competing, all within the same 6-7 one hour long time blocks on a given day. Event supervisors only have ~2 hours typically after the competition ends to grade and score all teams in order to announce results at the awards ceremony.

-Tournaments are run at locations as diverse as small middle schools to large university campuses. The hardware available in computer labs varies widely, as does the security policies that have to be followed. Supervisors don't always have the ability to have full administrator access to systems.

-There are ~7000 teams that compete each year, ranging from perennial National champions to first time teams that can't even field a full 15 students. All events need to be flexible enough to not be too hard at regional level tournaments to discourage teams from competing, to being hard enough to not result in a lot of ties at the National tournament.

Can you provide some thoughts on what you all have seen in other comp-sci competitions that would help address these issues?

[iwonder]

(I'm on a phone right now, but when I get access to a laptop and some time I'll fill this out more)

So there are two types of compsci competitions that I've competed in, UIL and 'Programming Contests'. UIL is a test compromising Java Syntax, as well as comp sci topics like search algorithms and time classes (O(n), O(nlogn), etc), etc. It's then followed by a programming session where you're given a set of 12ish problems and you have to write Java to solve those problems. The other programming competitions are just the last half of uil. (And of course, there's software to judge stuff and log data and score all this automatically, it's avaliable free but I forget the name).


Personally I make a big distinction between computer science and programming, it's like physics and English, computer science is the study of how everything works together, information and database theory, common algorithms, etc. While programming is the implementation of those algorithms, sort of like expressing the same idea in english vs Chinese, (ie Python vs COBOL). You can write tones of useful stuff without knowing what bubble sort is, but inversely its very difficult to explain bubble sort without knowing a language (being able to speak).

By that description, I think a SciOly event should cover the computer science portion, of course, it has to touch on programming (you have to be able to 'speak'), but algorithms and concepts like inheritance, mutability, etc are more like what I'd expect of the scioly level (those are standard AP exam topics).

Perhaps then the best option is a written test for invites and regionals, written in something like python or Java, covering basic syntax, and the aforementioned comp sci ideas. This is followed by a State/Nationals programming contest where teams must solve 1 or 2 other challenges (they can submit them through one of the numerous programs designed for this task to ease scoring). This means the written code is computer graded, quick and easy, and the test can be scantron based, which is also quick and easy. Also, by limiting the computer based tasks to the state and national level, the tournament should be able to handle the needed infrastructure for it. (I believe everyone was able to run 'Compute This' from B division).

To recap,

-I feel the event required the same from an ES as any event at the lower levels. (I'm not sure if you do anything with a programming language, but what you know the concepts in one its very easy to adapt to another, especially Python or Java)

-The grading at all levels can be machine based if needed.

-Computer labs are only needed at the larger contests, which should make coordination easier.

-The difficulty of the event increases as the level of competition increases the same as any other event.

Also, your concern about computer avaliability is a real issue, most of the contests I've been to have competitors bring their own laptops or computers to use, instead of providing them. (That's a bit of a departure for scioly, so likely not an option)

Edit: PC^2 (Programming Contest Control system) is the software I've used in the past, it's fairly straightforward, but remember the people in these contests are pretty computer savvy.

[laneyoung]

Speaking only for myself, and not having really discussed this with other people before now, I was very surprised to see Game On included in the list of C events for next year. I think in concept Scratch is more appropriate to be used by B level students. Even with in that, I think the subjectivity of the scoring compared to all other events is extreme. I would be concerned if it were going to be a B level event from a fairness perspective. I think it's not, as pointed above, at the same level of field mastery as other SO events in the C Division.

I do support the idea of a SO event around Computer Science. In fact I strongly support it. I think the inclination to make it "hands-on" rather than test based is/was correct. I do not have any experience in programming competitions, but from an event running standpoint the idea of having a set programming language and programming challenges is the right way to go. If the output of a program can be defined then someone without much area expertise could administer an exam if it was written by someone with more expertise. This is no different than many events now and addresses the numbers issue chalker is right to point out.

[chalker]

This is helpful. I wasn't aware of some of the automated code scoring systems that exist. Looking at this website: https://www.cs.utexas.edu/outreach/uil-contest it seems that the UIL contests are setup to occur over a 3 hour period and involve both a written test and 12 coding problems. Do you all think this format could be scaled down to the 50-60 standard SO time block, while still involving enough questions / problems to be able to break ties amongst 50-60 teams?

[iwonder]

If I recall correctly the UIL Test is meant to be taken in a 45 or 50 minute block and the 12 coding problems are made for the remainder of the time. Keep in mind that teams aren't expected to entirely finish either section (most people finish the test, but complete maybe 8-10 problems).

The test could easily be scaled to accommodate the SO time block, the programming section would probably just take a handful of problems (1,4,10 for TX UIL, if you found some sample problems).

I would think an 'average' Texas team could complete the first 1/3-1/2 of the UIL test and say problems 1 and 5 of the problem set in a 50 minute block. (The difficulty rises exponentially as tests/psets progress) But that's just a guess honestly.

Including the significantly more difficult programming problems, and awarding partial credit for solutions (say, the standard cases are handled, but boundary cases or special things aren't) would likely eliminate most ties.

[patil215]

I think the format could be scaled down to a 50 minute block, but I'm not sure if this is possible to do with both hands-on programming and a written test in a 50 minute time block. It would have to be one or the other, or you could have a short test coupled with only one or two programming problems:

The written test could be modeled off of:

• UIL Computer Science http://www.cs.utexas.edu/~scottm/uil/20 ... s_2014.pdf
  The UIL Computer Science topics list http://www.uiltexas.org/files/academics ... st1415.pdf
  The AP Computer Science exam
  https://www.ets.org/Media/Tests/MFT/pdf ... ompsci.pdf (this one is fairly difficult, the questions would have to be simplified greatly for high school level, but the idea remains the same)

I think the test could easily be written by a knowledgeable proctor to be as approachable and proctorable as any other SciO event.

The programming problems could be modeled off of:

• Simple exercises from here: http://adriann.github.io/programming_problems.html
  USA Computing Olympiad problems http://usaco.org/index.php?page=viewproblem2&cpid=514 (one example)

This is more challenging, but it's definitely doable. Just like Game On requires copying the Scratch game file, at the end of the 50 minute time slot the computer would submit their program code. The proctor would run it with a given input and give points based on how many input test cases the competitors algorithm correctly solves.

A good example of a well-thought out computer science competition that has both is indeed UIL Computer Science.

[patil215]

-There are ~7000 teams that compete each year, ranging from perennial National champions to first time teams that can't even field a full 15 students. All events need to be flexible enough to not be too hard at regional level tournaments to discourage teams from competing, to being hard enough to not result in a lot of ties at the National tournament.

I can say for certain that Game On will have serious issues with lots of ties at the national level. Scratch is a program that's fairly easy to master, with not much depth beyond the basic concepts. After that, judging would have to become completely based on game aspects like "originality" or "game polish/aesthetics", which would make the event near-completely subjective at the national level, as it's fairly easy to meet all the other criteria ("sound", "automated movement", "collision detection", etc) with a basic game.

[patil215]

I would also say that Game On as an event does not scale in difficulty. What I mean by this is that, for example in academic events, the tests get more difficult as the level of competition increases (nationals tests are usually lengthier and more difficult compared to regional-level tests, which separates out teams). And with engineering, specific aspects of the competition get harder (variable time for Mission, more fine distance increments for Scrambler, etc). The difficulty in making a game in Scratch doesn't change, which is a serious problem. With the computer science competitions we are proposing, the event can be scaled in difficulty by competition. Regionals tests can test basic computer science concepts or have easier hands-on problems, but at the national level, a knowledgeable proctor would be able to make a seriously challenging test.

[Magikarpmaster629]

Does anyone think this will be worse than Pentathlon?