So what do you do once you have your data and want to analyze and present it in a useful and easy to understand way?  Today we do a roundup of some of the cool blogs out there doing work in these areas.

In the realm of social science research, Andrew Gelman is always very thoughtful and thorough in the ways that he displays his own complex data, and on his blog he often shares interesting data visualizations and data visualization tools from other sources as well.  He’s done some blogging for FiveThirtyEight as well, which has had some other nice visualizations too.

Gelman contributed to a book called Beautiful Data, along with Peter Norvig of Google (discussing natural language data), Brendan O’Connor of the AI and Social Science blog, and a number of other folks.  The bits I’ve read look pretty great — I can’t wait to read the whole thing.

In the realm of more playful and off-the-wall, the OKCupid blog has really interesting data analyses.  They have a large amount of data they’ve gathered from users of their dating site, and they regularly discuss on the blog how various opinions and priorities vary geographically, which demographics dates which, what kinds of communications get a response, and so on.

There are a number of cool websites that don’t address a particular topic, but instead apply clever information visualization techniques to a wide array of issues.  I particularly like Information is Beautiful, and have recently discovered Flowing Data (full of useful tutorials and tools!) and Infosthetics.  At the last blog, I would recommend checking out their visualization of Choose Your Own Adventure books, if you ever enjoyed those as a child.  (I did!)

Finally, there are Strange Maps, covering visualizations of everything from the useful to the whimsical to the imaginary, and GraphJam, which is just plain silly, but sometimes pretty great.

Here’s something for the geekier researchers in our crowd, and for those already running batches of tasks on Mechanical Turk.

Have you ever wanted to run a real quick experiment or task on Mechanical Turk — run a quick poll, or ask the same question about each of a few different pictures, for instance?  Do you enjoy using the command line to get things done efficiently?

Mechanical Turk’s own command line API wrappers can be clunky, but Voxilate is here to help!  They’ve created a python script that creates command line functionality for simply and quickly setting up common tasks on MTurk.  Nifty!  Google code is hosting a few other possibly useful Mechanical Turk coding projects as well.

There’s a new survey site called Survs.  The main feature differentiating it from existing tools appears to be that it’s easy to collaborate on surveys and share information across multiple accounts.  That seems like a good idea!  From playing around with it, it seems like they have good usability, and it looks like they also have some nice data analysis tools. The free account won’t let us do anything with  logic or with collaboration, so we can’t test the more complicated features, but it looks like these folks might be worth checking out if you have surveys to run.

There’s a neat blog up called Experimental Turk whose purpose is:

reporting evidence concerning the reliability of Amazon Mechanical Turk as an online subject pool for experiments in economics, psychology, and social sciences in general.

The authors have been running classic experiments from social science on Turk (e.g., some of Kahneman & Tversky’s classic work on judgment heuristics and biases) and posting to the blog with the result, comparing the Mechanical Turk results to known effects found in the laboratory.  This is a great way to start getting a better sense of data reliability on Turk, and they encourage other researchers to collaborate.  Take a look!

The HeadLamp Research co-founders recently attended the Ignite NYC event — a crazy, fun series of 5 minute talks, where each talk consists of 20 slides that auto-advance every 15 seconds.   Both of us spoke at the event, and you can currently see Hilary Mason’s talk, “How to replace yourself with a very small shell script,” on the Ignite site!  It’s fun, and all about how to manage your email and communication data more efficiently.

Eric Waller did a quick experiment to confirm a behavioral economics hypothesis with a small amount of Mechnical Turk data. He found a paper that showed evidence that removing the minimum payment line from a credit card statement causes people to pay more (unless they typically pay the bill in full), and constructed an experiment to confirm the hypothesis.

The entire process took him only three days. I’ll let you jump over to the article to see the results!  It’s a nice example of how quickly and cheaply a short, simple experiment can be run on Turk — Waller spent a total of 3 evenings setting up the experiment, recruiting and running 200 participants, and analyzing the results.  As we’ve discussed before, Turk is a good tool for this kind of experiment.

As this kind of research becomes easier, it also makes it more likely that people will do more research like Waller’s — confirming things that they’re pretty sure are the case, but which should really be double checked, as well as fleshing out existing results a bit more precisely.  That seems like a pretty great meta-result to us.

Our mission involves recruiting large populations of internet users for research tasks, so we’re always interested in innovative crowdsourcing methods.  Crowdsourcing services allow many people to contribute to a project and be compensated in various ways.  We stumbled on 10 ways small businesses can harness big crowds by Ross Kimbarovsky, co-founder of CrowdSPRING, a sometimes controversial marketplace for design services.

The most interesting services highlighted are software testing (uTest), customer support (the always fantastic Get Satisfaction), domain-specific scientific, materials and technology research (InnoCentive), and prediction marketplaces (Inkling). The most successful crowdsourcing projects seem to be those that offer a win for both the business and the community members.  As a company developing tools to help both researchers and participants, this makes intuitive sense to us, and we’re happy to see this strategy succeeding elsewhere.

If you haven’t yet checked out Amazon Mechanical Turk for doing research with human subjects, you should!  We like Turk for running short, simple studies.  For instance, if you want to find out how several hundred people would describe a single picture, or survey a big crowd to find out which name they prefer for a new product you’re releasing — and you want the data really fast, Turk is a really useful tool.

When I did my thesis work in cognitive science, one question I was interested in was how people interpreted words like “tall” and “long” and “big” in different contexts.  I was able to show thousands of people one drawing each and ask them to pick out the tall or long items in each picture.  It was a fast and inexpensive way to get data, compared to bringing people into the laboratory.

So does it work well?  Earlier this year, Language Log had a nice writeup of a linguistic sentiment experiment on Turk that they ran primarily to test Turk’s reliability for simple linguistics experiments.  They found the results to be quite good.  Another great analysis, courtesy of Dolores Labs, concludes that Turk is “fast, cheap, and good for machine learning data.”

Other questions that researchers have is who participates in tasks posted to Turk, and why.  Panos Ipeirotis has some great data on why people participate in tasks on Turk.  FloozySpeak provides more detailed information on what entices people to participate — and how much time they spend on Turk.  And Ipeirotis (a great resource on Turk in general) does a nice job analyzing the demographics and of Turkers.

Mechanical Turk was not designed with scientists or researchers in mind, nor do they have human subjects’ rights on their mind in watching out for the participants.  There are a lot of times when the system can be a bit frustrating when doing research — especially longer or more complex studies.  But it can be a great tool for many things!

Dolores Labs is doing a bunch of cool work finding ways to distribute tasks to large numbers of people online.  As well as crowdsourcing tasks, they also check the results to make sure that you get higher quality results than using unmoderated tools like Amazon Mechanical Turk.

The Dolores Labs blog has an interesting post up about their recent partnership with Gambit (a payment tool used in online games and communities).  Together, they’re offering Facebook gamers the opportunity to earn in-game credit for doing real work through their platform.  This is a timely service, especially given TechCrunch’s recent indictment of offer platforms as scams. Now you can earn your in-game wage by completing simple tasks instead of signing up for credit cards and Netflix accounts!  We like this innovative way of reaching and rewarding large numbers of people via Facebook.

Dr. Markus Jakobsson of PARC has a great article sharing 5 tips on how to perform successful research using Amazon Mechanical Turk.  Jakobsson talks about some of the pros of Mechanical Turk for research (e.g., quick access to a large pool of participants and participant anonymity) and then delves into how to navigate the possible risks and difficulties (detecting cheaters, running multi-part studies, and avoiding biasing subjects, to name a few).  It’s a really nice look at how to best make use of the tools currently available; check it out!