Biases of the scientist citizen

On August 18th, Nature issued an editorial comment acknowledging the “rise of the citizen scientist”, its relevance for science and the reliability of CS-based methods of data collection.

It is a very interesting short piece, which for instance documents how CS in ornithology stems from a long-standing tradition that dates back to the 1900 when “the Audubon Society persuaded Americans to exchange their Christmas tradition of shooting birds for a more productive effort to count them instead”. The piece also claims that technologies, and especially portable and/or wearable technologies, will make CS-based data collection more and more effective.

Interestingly, the author claims that issues with data quality, which have been found to fall short of the current standards in some CS projects, are simple to address, as they are issues that “the professional scientific community is already wrestling with”.

The conclusion is however puzzling. Nature claims that instead issues with conflicts of interests are troublesome in CS. They report cases where conservationists (holding “strong views on conservation that did not reflect those of the broader population”)  are disproportionately represented as data collectors in projects in conservation biology. They also speculated that CS volunteers may further political objectives, for instance “gathering evidence of harmful effects” of fracking because they oppose it.

There are two separate issues here. (1) Politically motivated activists may (even unintentionally) misrepresent or insert bias in their data collection and/or analysis so as to back their politically motivated theses. (2) Scientific questions that are politically motivated may get disproportionately researched.

Both criticisms however do not stand scrutiny, as they apply to “traditional” scientists as well.

  1. Issues of biases and misrepresentations are endemic in science and indeed review process and other sophisticated features of the functioning of science may be thought as de-biasing devices, devices that would ideally filter out biases in CS-based research as well.
  2. As Weber noted for the social sciences, complexity makes it impossible that all scientific questions get researched (cf. Weber 1904: Objectivity’ in Social Science and Social Policy). This holds true for any science, and even “data-driven research” must at some level make some choice regarding what matters, and what does not. That is, value judgements about what is significant and what is not are inevitably going to influence the choice of research question. Philip Kitcher expresses the same concept arguing that there is no “scientific agenda set by nature”, but only scientific agendas set by human interests (cf. Kitcher 2001: Science, Truth and Democracy). This is true for CS as it is true for non-CS research.

Are these “problems” any more compelling in the case of CS than they are in the case of professional scientists? The first issue seems to be common to CS and “professional” science, as they both contribute to a common endeavour with the same rules for publications, replications, etc.
Setting the research agenda in ways that reflect the views of the broader population, as suggested in the Nature editorial, is an altogether different challenge. Is this a good ideal in the first place? And what specific admixture of professionalism and “laypeople” contribution will promote that ideal?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s