Prolific Academic Survey Alerts

olliebean

So regarding the LLM_XAI_Normal study discussed above, did anyone else get this message from the researcher?

Dear Participant,Thank you for your time in attempting this survey. Unfortunately, we encountered a technical issue with the proceed button on the first question, which may have significantly increased the time required to complete the survey.As a result, your response was not recorded, and the survey remains incomplete. Please rest assured that this will not impact your performance on Prolific. We sincerely apologize for any inconvenience this may have caused.May I kindly ask you to return the submission and cancel the reward by selecting the red circular arrow on your Submissions page?We truly appreciate your understanding and apologize again for the inconvenience.

Seems like a bloody cheek, asking people to return and cancel the reward because of a technical issue at their end which "significantly increased the time required." As it turns out, they had already manually approved my submission (regardless of my having already returned the study), and apparently that means I'm now unable to return it anyway. I've referred them to Prolific, where I imagine they will get short shrift. (Also, for studies that can be returned, the circular arrow they refer to isn't red, which just proves they don't know what they're talking about.)

silvercar

olliebean said:

So regarding the LLM_XAI_Normal study discussed above, did anyone else get this message from the researcher?

Dear Participant,

Thank you for your time in attempting this survey. Unfortunately, we encountered a technical issue with the proceed button on the first question, which may have significantly increased the time required to complete the survey.

As a result, your response was not recorded, and the survey remains incomplete. Please rest assured that this will not impact your performance on Prolific. We sincerely apologize for any inconvenience this may have caused.

May I kindly ask you to return the submission and cancel the reward by selecting the red circular arrow on your Submissions page?

We truly appreciate your understanding and apologize again for the inconvenience.

Seems like a bloody cheek, asking people to return and cancel the reward because of a technical issue at their end which "significantly increased the time required." As it turns out, they had already manually approved my submission (regardless of my having already returned the study), and apparently that means I'm now unable to return it anyway. I've referred them to Prolific, where I imagine they will get short shrift. (Also, for studies that can be returned, the circular arrow they refer to isn't red, which just proves they don't know what they're talking about.)

I had the same problem and returned the survey. Mine hasn't been approved and I haven't had the message. I'd guess that they realised too late that they had approved your payment and our now desperately trying to get that money back.

Slinky

House_owner said:

Slinky said:

House_owner said:

Slinky said:

Did anybody do

Executive Function, Metacognition and Mental Health

By Aidan Sammel

Just seemed to stop after the colour game, and no completion code, well I didn't get one anyway

Managed to get through it, but didn't get a completion code.

Was the last section you did the one with the words/colours?

No, I had a mental health questionnaire to do after that, then when I submitted, it took be to a log in page asking me for a password which I didn’t have, so I went back to Prolific and submitted it as no code.

Oh dear, looks like I might get a rejection then.

onemouseplace

Anyone confused about the Dynamic Rewards for the AI - I've just done one and I'm down around $10 and I don't really understand why.  The FAQs say 'If you finish all assigned tasks, you’ll receive the full payment as expected.' - how do we know how many assigned tasks we have per session, especially as the length/ complexity varies so much.  It didn't help that the expected completion time for this one was 50 minutes?

warby68

onemouseplace said:

Anyone confused about the Dynamic Rewards for the AI - I've just done one and I'm down around $10 and I don't really understand why.  The FAQs say 'If you finish all assigned tasks, you’ll receive the full payment as expected.' - how do we know how many assigned tasks we have per session, especially as the length/ complexity varies so much.  It didn't help that the expected completion time for this one was 50 minutes?

I don't think this is Dynamic Rewards in action quite yet. I think they've just set up a 50 minute task. They are probably still tinkering. When I've seen Dynamic Rewards before the reward had the wording 'up to' in front.

If I understand it right its not very dynamic as it allows the researcher to pay you less if you finish early but will not pay you more if a task takes you over the allowed time. A one way street really.

onemouseplace

warby68 said:

onemouseplace said:

Anyone confused about the Dynamic Rewards for the AI - I've just done one and I'm down around $10 and I don't really understand why.  The FAQs say 'If you finish all assigned tasks, you’ll receive the full payment as expected.' - how do we know how many assigned tasks we have per session, especially as the length/ complexity varies so much.  It didn't help that the expected completion time for this one was 50 minutes?

I don't think this is Dynamic Rewards in action quite yet. I think they've just set up a 50 minute task. They are probably still tinkering. When I've seen Dynamic Rewards before the reward had the wording 'up to' in front.

If I understand it right its not very dynamic as it allows the researcher to pay you less if you finish early but will not pay you more if a task takes you over the allowed time. A one way street really.

Yeah - it said nothing about Dynamic Rewards when I started it (and noticed it was a 50 minute study) and the task blurb still has the same stuff about setting a timer etc., but I got a message after I submitted confirming I'd been paid under the dynamic rewards method and I was down on the $ I should have received even bearing in mind it was a 50 minuter.

I suspect they are still tinkering as the Dynamic Rewards thing didn't work very well last time they tried it either.

olliebean

So, did anyone see the study this morning wanting to install software on your phone to automatically take photos of you and your surroundings whenever you're using social media, and not think "You must be bloody joking"?

MollyR

I don't use my phone for Prolific or any other studies.  I don't touch social media.  Such scammers are onto a dead loser with me.

Graceglow

Graceglow said:

MollyR said:

I had three yesterday.  One for 60p, completed no problem, awaiting review.  One for 50p, got taken to the home page of where the study should have been hosted;  messaged researcher;  no reply;  returned study.  Researcher later replied, apologising and saying it was her first study on Prolific, and asked me to try again;  carefully explained to her that after returning a study one cannot just open it again, and that if I had not returned it I would have been timed out by then.  Recommended Prolific's Help pages.  And the third study (Robot Perception by Imed Ben Nasr for 60p) gave a page with brief instructions, then a page asking which of three answers was appropriate to what I had been told (none was;  and if it was an attention check it was in total breach of Prolific's rules) and when I selected one answer option the survey terminated, without any complaints but without any completion code either.  I immediately messaged the researcher to tell him.  Shortly afterwards I received a message saying that my participation had been rejected "Finished the study too quickly".  I responded protesting bitterly that he should not land me with the permanent black mark of a rejection just because his study was not working properly.  I got no reply, and lodged an official complaint with Prolific, for which I got an acknowledgement.  Hopefully they will remove the rejection;  in the past they have done, although it takes a long time.

So yes, I had surveys;  but for all the good they did me I might as well not have done!

Interesting, I also had the same problem with Imed Ben Nast’s study. I don’t know how you can finish the study too quickly when he had controlled the time you spent reading the scenario before answering the questions. Definitely, a technical hitch as I never got to the 2nd scenario the study just closed. I also responded to the researcher to complain but haven’t had a reply. 

I will complain officially to Prolific tomorrow and appeal the rejection. Hopefully, more people have also flagged up this study.

* Update  My appeal about this rejection has been upheld. Email received today,Prolific have changed the submission to returned 🙂

Andrew30000

lou-rose said:

Andrew30000 said:

Have you thought about working directly for an AI annotation company. Then you can do as much as you want. There's Outlier and Data Annotation you could apply to.

Have you applied for either of these? A lot of us may likely struggle to demonstrate our prior experience due to the confidentiality agreements

yes, both. DA I failed the assessment but Outlier I passed. You don't need any prior experience. You add your CV and connect a Linkedin account and then pass an exam. I think having a degree helps.