The NIH has just announced a great new tool for clinical trials and other IRB-approved research!  ResearchMatch.org allows people interested in participating in research to sign up to learn about specific studies they might want to participate in.  It also lets researchers to use the system to recruit participants for their research.  In contrast to our online research focus, this tool fills a gap in research done in the laboratory, where it can also be very hard for researchers to recruit study participants.

Despite being administered by the NIH, ResearchMatch.org is for any IRB-approved researcher (not just clinical trials).  It contrasts in that way with research-participant match-up site ClinicalTrials.gov.  Another difference between the sites is that ClinicalTrials.gov makes the participant do the work of finding research they’re interested in participating in, while ResearchMatch.org makes it the job of the researchers to contact participants that match their needs (the system protects potential participants’ personal information however).

Currently, the site only allows researchers to use the system if their university or institution is a participating member.  But they’re encouraging researchers to sign up for information and express interest even if their institution is not currently participating; I don’t know if the site will eventually stop being mediated by institutions, or if they’re just hoping to get lots more institutions to sign up soon.

A side effect of the institution model is that opportunities are sparse in some parts of the U.S., and if you’re interested in participating in research in those regions, you may be out of luck for a while.  But I don’t know, maybe the network will grow fast — this tool seems like a terrific idea, and I hope lots of research institutions, researchers, and people interested in participating all sign up.

I also hope that if this system is successful, other countries will emulate it. For that matter, maybe the U.S. is lagging behind here and other countries already have such systems; does anyone know of such research matchmaking sites elsewhere?

If you haven’t yet checked out Amazon Mechanical Turk for doing research with human subjects, you should!  We like Turk for running short, simple studies.  For instance, if you want to find out how several hundred people would describe a single picture, or survey a big crowd to find out which name they prefer for a new product you’re releasing — and you want the data really fast, Turk is a really useful tool.

When I did my thesis work in cognitive science, one question I was interested in was how people interpreted words like “tall” and “long” and “big” in different contexts.  I was able to show thousands of people one drawing each and ask them to pick out the tall or long items in each picture.  It was a fast and inexpensive way to get data, compared to bringing people into the laboratory.

So does it work well?  Earlier this year, Language Log had a nice writeup of a linguistic sentiment experiment on Turk that they ran primarily to test Turk’s reliability for simple linguistics experiments.  They found the results to be quite good.  Another great analysis, courtesy of Dolores Labs, concludes that Turk is “fast, cheap, and good for machine learning data.”

Other questions that researchers have is who participates in tasks posted to Turk, and why.  Panos Ipeirotis has some great data on why people participate in tasks on Turk.  FloozySpeak provides more detailed information on what entices people to participate — and how much time they spend on Turk.  And Ipeirotis (a great resource on Turk in general) does a nice job analyzing the demographics and of Turkers.

Mechanical Turk was not designed with scientists or researchers in mind, nor do they have human subjects’ rights on their mind in watching out for the participants.  There are a lot of times when the system can be a bit frustrating when doing research — especially longer or more complex studies.  But it can be a great tool for many things!

Dolores Labs is doing a bunch of cool work finding ways to distribute tasks to large numbers of people online.  As well as crowdsourcing tasks, they also check the results to make sure that you get higher quality results than using unmoderated tools like Amazon Mechanical Turk.

The Dolores Labs blog has an interesting post up about their recent partnership with Gambit (a payment tool used in online games and communities).  Together, they’re offering Facebook gamers the opportunity to earn in-game credit for doing real work through their platform.  This is a timely service, especially given TechCrunch’s recent indictment of offer platforms as scams. Now you can earn your in-game wage by completing simple tasks instead of signing up for credit cards and Netflix accounts!  We like this innovative way of reaching and rewarding large numbers of people via Facebook.

Dr. Markus Jakobsson of PARC has a great article sharing 5 tips on how to perform successful research using Amazon Mechanical Turk.  Jakobsson talks about some of the pros of Mechanical Turk for research (e.g., quick access to a large pool of participants and participant anonymity) and then delves into how to navigate the possible risks and difficulties (detecting cheaters, running multi-part studies, and avoiding biasing subjects, to name a few).  It’s a really nice look at how to best make use of the tools currently available; check it out!

Today I follow up on the rest of the survey data I started analyzing in the last post.  We’ve gotten a number of new survey submissions since my last post; thanks to everyone for your data!  Since this post is a continuation of the last one, I’m going to keep analyzing the same set of responses from before — but the new responses don’t significantly change the results I’m presenting here, in any case.

Last time we talked about researchers’ specific needs in terms of experimental/research design.  This time, I’m going to cover the other aspects of running studies — participant recruitment and payment in particular.

» Continue reading “what researchers want, part 2″

We recently asked a bunch of researchers who work with human participants to tell us about their research needs.  (It’s not too late to give input!  Take our short survey here.)  We love data, and we know a lot of you also love data, so we wanted to share some of our results!

At the time of this analysis, we’ve had 72 complete or nearly complete survey responses, as well as many partial responses.  The breakdown by (not disjoint) fields of research is:

• 38 psychologists/related cognitive scientists
• 21 linguists
• 19 computer scientists
• 10 neuroscientists
• 9 people in other fields (marketing, political science, economics, education, evolutionary biology, human-environment geography)

You can see here the sample bias that our own background led to in our initial research!  Especially if your field is underrepresented in the above list, we’d love to hear about what you do.

» Continue reading “what researchers want, part 1″

Hey,  everyone! For our inaugural blog post, I’d like to tell you a bit about who we are, and what to expect from our company and our blog.

I’m Lauren Schmidt.  I’ve got a background in cognitive science research as well as computer science, and I’ve spent many hours hacking together online experiments using various online tools that were built without scientists in mind — as well as coding experiments from scratch.  I’ve also spent lots of hours trying to recruit participants to come into the lab for experiments, and working to make our schedules align.  Now I’d like to make everyone else’s research a lot easier, as well as my own.   That’s why I’ve co-founded HeadLamp Research.   Working with me on this project is Hilary Mason.  She’s a professor of computer science and an experienced developer who has a history of building technical tools to help people access and analyze data.  We both want to make people’s lives easier and more fun.

We’re setting up an online platform that’s powerful enough for all researchers to design studies on, while also being easy to use — whether or not you have any programming experience.  We’re also going to essentially play matchmaker for researchers and participants; if you’re a researcher who needs participants for a study, we’ll help you find the people you need, with the demographics and skills you require.  If you want to participate in some fun studies and make some money, we’ll make it simple for you to find rewarding studies to participate in.  We’ll  make payment easy as well.

In this blog, we’ll keep you updated about the development of our research tools.  We’ll also be discussing the needs of researchers online, the demographics and interests of people participating in studies online, and other data we collect or run across in our own research.  We’ll be showing some graphs and figures, because we love data!

Stick around!  We’re looking forward to sharing our ideas and hearing yours.