Clixsense 2014 Second Thread

ms_night_ryder

Sometimes I feel like cs, ask you to do a hell of a lot of work on 1c tasks.

vampire-girl

PL Survey 0.50c with censuswide on wages

vampire-girl

won 0.10c on clixgrid

ms_night_ryder

PL Gambling survey, quite a few SSI surveys, gambling and pensions.

nickyotron

RachelBel wrote: »

Those test questions are sometimes ridiculous aren't they?

You might not have seen this info I posted on the last thread about test questions. I spent ages reading the client side of Crowdflowers website to get a better understanding of how it works... This is what I learned about how test questions are done:

Test Questions: There's actually two ways that test questions are made.

The first way is that the 'correct' answer is provided by the client who built the task. (The majority seem to be done this way because the whole client side of the website is geared towards teaching clients how to upload answers, build explanations & instructions etc)

The second way is more complicated & might be used by the client to save time on their part - Test questions are built through 'the wisdom of the crowd, so to speak. So if, for example, four people, who have a high 'trusted' accuracy level opt for answer A, but one person opts for B... the test question becomes "Correct answer=A". They use a formula to ascertain confidence levels based on the aggragate of answers and the accuracy levels of the people involved. So essentially this means that a task can accumulate more test questions dotted throughout the task as it proceeds.
But its not infallible at all... because those 'trusted' contributors might become 'untrusted' over the course of the task by getting other test questions wrong, which adjusts their % in real time, and therefore adjusts the 'confidence level' of their answer in those test questions. So accuracy get adjusted automatically at the end of the task if you were marked wrong by what turned out to be inaccurate responders.
It's important to note, 'trusted' contributor on a task is not based on global accuracy, but only your performance on that task... so someone might do well on the quiz mode questions, are regarded 'trusted' and then only do a handful of the task questions, and never stay in the task long enough to be properly judged for their accuracy which might not be all that great overall - thats why its worth raising tickets to draw attention to tasks that might be using bad judgement as a benchmark. And some jobs on a task can be tricky and trip up the majority of accurate people too... like those rooftop ones where the pool has been drained & not obvious, or the googlemap view shows solar panels that aren't on the main pic.

In addition to that, I recall that the platform they use means that a spreadsheet can be uploaded by a client with all the correct answers to correspond to the numbered tasks - it only takes one field to be skipped/empty for all the rest of the answers to be badly aligned/mismatch (So explanation doesn't match answer)

There was also a 'Test Question Creator' skills test - last year and again recently, so the ones who passed last year would possibly be writing test questions by now. Not the recent people though, as someone presumed, because they do get trained first, and its such a nice perk I doubt anyone would want to risk getting dropped for shoddy work. I wouldn't be surprised if the 'Official Crowdflower' questions are the ones written by a combination of Contributor Writers and Staff.

Hope that helps. And hope these test questions get a bit better... It can quite annoying to get bumped out of a task for good work.

Thanks Rachel. Really good to read that, an gain more of an understanding how it works, or doesn't sometimes lol :beer:

ms_night_ryder

Ink tasks 1c, rate the pic

vampire-girl

438348
Memory test
level_2_contributors $0.10
66

ms_night_ryder

PL- 83C survey really easy and interesting.

Bryan1983

Very quick SSI survey currently on the homepage. Only need to answer one question, takes 10 seconds.

ms_night_ryder

Tasks are a bit thin on the ground for level 1's! not seen it this scarce before!