Debiasing Collection in Field Biology

1 Why Natural History Collections Matter

Collected biological specimens—animals, plants, fungi, and materials such as bones or shells—were fundamental to the development of the modern field of biology (Funk 2018). Both private and institutional collections have provided key data for the development of biological theory. And collections have been central in shaping the scientific image of the biological world. Darwin and Wallace were inspired to make what’s now called biodiversity a central explanandum of evolutionary theory in part by their encounters with diversity in specimen collections (Burch-Brown and Archer 2017). Contemporary natural history collections continue to provide these formative experiences of the breadth and depth of life to today’s budding biologists. Likewise, modern specimen collections continue to be vital research tools, contributing to understanding evolutionary change, the geographic distributions of organisms, and ecological interactions (Miller et al. 2020). The knowledge contained in these collections is thus crucial to not just taxonomy, but also theoretical fields like ecology and evolutionary biology, and applied fields like conservation biology and restoration ecology—so long as the specimens are appropriately contextualized (Halm 2023).

That caveat is an important one. Biases in the practice of specimen collection risk becoming biases in the scientific work informed by collections. Since this scientific work spans not only core biological theorizing, but also urgent applied science that aims at goals like preventing extinctions and adapting to climate change, biases in biodiversity collection are worth our careful philosophical attention.

For all their informational value, the use of collected specimens in applied contexts like conservation and restoration faces “epistemological hurdles,” since collections tend to be very partial representations of the biological systems they sample, and, moreover, since specimens are often stored without the necessary contextualizing metadata (Halm 2023). For instance, consider a preserved organism in a collection labeled with the year it was taken, who collected the specimen, and the county it was collected from. If the organism has a seasonally-plastic phenotype, then just knowing the year it was collected might not be enough, since we’d want the time of year as well. Was the organism trapped or killed by the collector, or was the specimen found dead? And counties can be ecologically diverse. What specific locality provided the specimen? It’s nearly guaranteed that a natural history collection will be missing much of this type of data, limiting our ability to recognize when the collection might present a misleading picture.

Additionally, researchers have documented biases in which specimens are chosen for collection. For example, there’s a bias towards male specimens in collections of birds and mammals, and this bias is especially pronounced for type specimens used to describe new species—only 27% of bird type specimens are female, for instance (Cooper et al. 2019). Our understanding of vertebrate richness and diversity, based as it is on species descriptions, might thus be skewed by a sex bias. Similarly, field collectors are more likely to focus on specimens with aesthetic value: rarity, beauty, colorfulness, distinctiveness, charisma, etc. This means that natural history collections, when used to estimate population abundances, tend to overestimate the populations of rare species (Gotelli et al. 2023).

Navigating around these epistemic hurdles when drawing on specimen collections requires understanding how they bias the collection, and what this means for the types of inferences we’re aiming for. In some ways, however, under-specific metadata, sex bias, aesthetic bias, and the like are relatively easy to navigate, because they can be identified from careful examination of the natural history collection itself. Less obvious, and less theorized, are ways in which specimen collection is biased as it occurs in the field. That’s where we’d like to turn our attention, focusing on a foundational step in field collection: selection of field site.

2 Locality Bias

Specimen collections are incomplete representations of biodiversity not only because of biases in which specimens we collect, but also because we tend to collect specimens from a limited geographic range.

In field biology, it is common to select locations for a study that are isolated (e.g., an island) or have limited human disturbance (Kohler 2002b). Additionally, field biologists often select locations that make it easier to consider and contextualize certain variables associated with fieldwork, such as weather or time of year. One upshot of this is that field biologists are selective about where they do fieldwork both for practical and scientific reasons. For example, quantifying mean plant mass growth per year in an arroyo may be an overriding goal, but site selection will also be influenced by questions of permits, proximity to roads, and the steepness of the wash.

For similar reasons, biological field studies appear to be heavily biased toward public land, with one study suggesting that only 27% of fieldwork surveyed included any private land (Hilty and Merenlender 2003). Avoidance of private land can stem from many reasons: researchers might not know the value of including private land in their research, they may not know how to interact with landowners and ask permission, or they may worry that their work is too “political,” which may make them not wish to discuss it or explain it to others. Furthermore, perhaps most importantly, it can be a laborious process to collect permission for private land, sometimes taking longer than the study itself (Hargiss and DeKeyser 2014). While useful for providing information to Institutional Animal Care and Use Committees (Paul and Sikes 2013), documenting permissions can be a impediment sufficient to motivate shifting the research site to public lands, to easier locations to permit, or even reimagining the study design.

Military land can be biologically rich, encompassing habitats for numerous threatened and endangered species (Tazik and Martin 2002). Furthermore, military and other restricted-access public lands can comprise a significant land area. Federal and military land in the United States, for example, encompasses around 29% of all land, making its use in biological surveys paramount, despite it often being overlooked (Stein and Benton 2008). However, permitting and permission can make it difficult to secure time on and access to military land, limiting the ability and motivation for biologists to pursue fieldwork on it.

Academic field biology similarly tends to neglect Indigenous or Tribal land. In part, this may be because biologists have had a history of disrespecting, ignoring, or exploiting the Indigenous stewards of the land, rather than putting in the work to pursue collaborative and inclusive research (Trisos et al. 2021; Ramírez-Castaeda et al. 2022; Park et al. 2023). Successful collaboration with Indigenous Peoples and other local communities requires both time invested in relationship-building and training in how to build those relationships appropriately, and that can discourage researchers from pursuing projects on Indigenous and Tribal lands.

An underlying theme is that field biologists have pragmatic, social, and political dimensions that they consider alongside their work. These are all important components in site selection that are often left as subtext, affecting how field research is communicated to both the public and to other scientists.¹ And for present purposes, it means that specimens tend to be disproportionately collected from a subset of the planet’s geography, significantly biasing how well collections represent biological reality.

This bias is likely to be significant for both scientific understanding and for conservation. We’re not aware of a systematic study of locality bias in specimen collection, but there has been some research on biased site selection on biodiversity research in general. One recent metadata analysis (Carvalho et al. 2023), for instance, found that research site selection in the Amazon is driven by travel time and proximity to research facilities, and neglects both Indigenous lands and hilly/mountainous terrain. The analysis documents not only these systematic locality biases, but also significant overlap between the types of sites neglected in research and locations highly vulnerable to severe effects from climate and land use change. Locality bias, then, is likely pervasive and potentially problematic. We turn next towards how we might address it.

3 Tackling Locality Bias

Because the localities in which field collection occurs don’t necessarily compose a representative sample of habitats, the specimens in biological collections tend be skewed representations of the biota out in the world. We’re calling this unrepresentativeness ‘bias’ since it’s an instance of sampling bias in the strict statistical sense, and because it’s the term other researchers use to refer to phenomena like locality bias (Hughes et al. 2021). Biodiversity data skewed by locality bias, for instance, create a database with “spatial bias” (Beck et al. 2014). ‘Bias’ in this statistical sense is normatively neutral. To say that a collection is biased is just to state the fact that it’s a non-representative sample for non-random reasons, without necessarily implying that this is a bad thing. Uses of the term ‘bias’ in other domains can connote lack of objectivity, unfairness, and other normative failings, but we² don’t want to read inherent normativity into ‘locality bias’. For one thing, that would be to misunderstand how the term ‘bias’ is used by scientists working with data informed by field collection. Even more crucially, tackling locality bias in biological specimen collection requires identifying when that bias causes problems. Locality bias isn’t problematic because it’s bias; it’s problematic when, and only when, it causes epistemic, practical, or ethical issues.

Biased collection of data or specimens, that is to say, isn’t necessarily problematic. This is especially true when the bias mirrors the motivating scientific interests. For example, Liboiron (2021) describes their work in pollution science as “biased (in the statistical sense of the term) by design,” since random sampling is generally less effective at finding pollution than sampling according to researchers’ judgment. “If someone thinks their oil tank is leaking, they don’t grid off their entire yard and randomly select some grid points to sample,” Liboiron argues, but instead “they sample around the tank, and often only around the tank.” Biasing the sampling is a more effective way to pursue some research questions. This is just as true in biological specimen collection as it is in pollution science. If a field researcher interested in seeing how the genetics of population differs from that of a neighboring population, they’re probably justified in biasing their collection of eDNA samples towards the microhabitats that species of interest most frequents, rather than systematically sampling the whole landscape.

So, locality bias often occurs for good practical and epistemic reasons, and it can be totally justified. For example, suppose the National Park Service is collecting samples from Joshua trees (Yucca brevifolia) in Joshua Tree National Park to track how the plants are physiologically responding to climate change, with the overarching goal of trying to maintain a healthy Joshua tree population in the park. Given that aim, we aren’t going to object that their collection practice is problematic because it occurs exclusively on public land. Whether biased collection practice poses a scientific problem depends on the scientific purposes of the collection. That raises an important theoretical point: assessing how we should respond to locality bias requires an understanding of the relationship between the practice of collection and the purposes of collections. In a given instance, scientists and technicians might have a good sense of that relationship. But in general, collection as a practice is under-theorized. We lack, for example, general accounts of scientific collection like those we have for other activities such as experiment, observation, and simulation. So the obvious take-home point from the ubiquity of locality bias—that we may sometimes need to correct for it—doesn’t exhaust the lessons we should learn. It also motivates developing a philosophical account of collection as a distinct scientific practice. In the next section, we’ll develop these ideas.

If some biases aren’t a significant problem for science, how should we identify the problematic biases? An analogy to the philosophy of scientific modeling might help. Philosophers of modeling have long recognized that scientific models can be helpful even when they are essentially inaccurate representations of their target system (Potochnik 2017; Rice 2019). Among other things, those inaccuracies (or “idealizations”) might make inference about a complex system tractable without making a relevant scientific difference. They could facilitate understanding by us cognitively limited beings. Or they may help isolate and identify the parts of a system of greatest interest. Recently, a leading view of how to assess whether the distortions and omissions (i.e., biases) in a model are scientifically acceptable is what Parker (2020) calls the adequacy-for-purpose account. No surprises here; the account is just what it says on the label. Identify the scientific role a model is meant to play, and if the model fulfills that function despite its inaccurate representations, the model is adequate for science.

Models may seem to be of a different scientific kind than collections of biological specimens, but there are enough commonalities to suggest that adequacy-for-purpose is a good approach to assess biases in collection, like locality bias, as well. Consider Currie’s (2018) account of modeling in engineering and design contexts, where the scientific model is a material artifact. As Currie argues, the “success” of a material artifact in science and engineering is a function of its adequacy-for-purpose.³ Specimen collections might generally be put to different purposes than material models, but their success can still be evaluated with respect to their own purposes.

That leads to a straightforward approach to assessing whether locality bias is problematic. What scientific functions do the specimens being collected serve? Does this bias undermine those functions? If so, the bias should be addressed. If not, carry on. This approach has some obvious advantages. For one thing, it’s simple and straightforward, and thus generally easy to apply, even in the field. An additional advantage of an adequacy-for-purpose approach is that it won’t demand costly, time-consuming corrective actions where unnecessary, which decontextualized guidelines for how to handle biases could.

On the other hand, adequacy-for-purpose has the drawback of being a less-effective guide to practice when the purpose of a collection is unclear. This may be an especially pointed issue for specimen collections, given that many collections are intended to serve the open-ended needs of future researchers.⁴ To some extent, we can still assess adequacy against broadly specified purposes, such as the conservation aims many contemporary biological collections often possess (Halm 2023). But this doesn’t entirely mitigate the issue. When the purpose of a collection is indeterminate, adequacy-for-purpose alone won’t fully adjudicate when and where locality bias is problematic.

So, even if scientific fieldworkers are going to use adequacy-for-purpose, or something similar, to decide whether and how to address biases, it’s reasonable to want to equip them with additional tools as well. Here, we think, philosophers of biology and our allies could help. Reflective practice in scientific field collection would be aided by a theory of collection as epistemic activity, and on that topic the scholarly literature runs thin. Creating a more viscous literature on scientific collection will require a community effort, which is more than we can provide in this paper. But we’ll take some steps here to encourage and perhaps shape that effort.

4 A Philosophy of Scientific Collection

Fieldwork in biology has long been seen as less prestigious—and less epistemically reliable—than laboratory research (Kohler 2002a). Perhaps for this reason, field methods haven’t received the same attention from philosophers of science that methods like laboratory experiment, modeling, and computer simulation have received.⁵ This is doubly true for field methods that aren’t experimental, such as specimen collection. As we’ve argued, however, collection is a crucial part of biology, and an account of collection as a scientific practice will facilitate addressing biases by helping identify the scientific purposes of collection.

Here, then, are some initial forays into such an account. This won’t be a comprehensive theory of the epistemology of field collection, but we’ll explore some features that are characteristic of natural science field collection.

Let’s start with Currie’s (2017) claim that fieldwork involves “secret epistemologies.” Some of that, according to Currie, is the way in which fieldwork involves “making decisions which finely balance the ‘properly’ epistemological with the pragmatics of extraction: time, money, person-power and so forth.” But an additional dimension he highlights is that much, though not all, fieldwork is prospective, which means that it isn’t hypothesis-driven, or even aimed at answering a specific scientific question. Instead, it involves the fieldworker employing their expertise to identify specimens or samples of potential scientific import. Currie notes that this expertise involves both a perceptual component (“a trained eye”), and theoretical knowledge that grounds “empirical expectations” about what to find, violations of which (“anomalies”) will be the potentially significant specimens. Currie insists that this constitutes a distinct epistemic practice from even exploratory experiments, where ‘exploratory’ means ‘not-hypothesis-driven.’

We agree with Currie on this, but think there’s more to say on multiple fronts. First, Currie’s motivating example of prospective epistemology is paleontologists wandering a field site in search of informative dinosaur fossils. That example, while very cool, might suggest that fieldwork is prospective only when it’s not highly structured. But we’d suggest that the same epistemic features will often apply to more structured field collection as well. Consider a hypothetical group of ecologists conducting transects to identify relative plant abundances. Assume that their method is highly structured, in that they are testing a specific hypothesis and they have systematic rules for where to set transect lines and how to sample plants as they survey along those lines. Although these ecologists might be doing something that looks a lot more experimental than those paleontologists wandering a fossil bed, we’d argue that many of the features of Currie’s secret epistemologies apply, though perhaps less obviously. Even with a rigorous system in place, the pragmatics of the field will still require expert judgment calls on the field scientist’s part. Suppose the ecologists intend to run parallel transects at 50m intervals, start to lay down transect tape, and discover that one transect will cross inaccessible terrain—a cliff, perhaps, or a very thick patch of prickly Salsola tragus.⁶ Do they leave the transects where they originally intended, with an unsampled stretch of terrain? Do they move every transect 10m? If so, which direction? To answer such questions, the fieldworker will need to make judgment calls on the basis of their expert knowledge of both theory and method, and they might employ a bit of intuition on the basis of eyeballing the options. Imagine now that the transects are successfully set, and our semi-imaginary ecologists are now walking down each line counting plants by species. Occasionally, they’ll need to stop and key out⁷ a plant they can’t identify. And sometimes they’ll second-guess the key, and clip a sample of a plant for further identification later on. How do they know when to do so? By those same applications of expertise that allow for identifications of anomalies and epistemically-significant specimens. Even in highly-structured fieldwork, the secret epistemologies are at play. Biological field collection, then, is always exploratory to some extent. We’ve used Currie (2017) to draw out a bit of what that means, but there’s more to be said.

Furthermore, we don’t want to overemphasize the prospective nature of field collection, because it paints too passive a picture of the collector. Collection, even field collection, involves curation. As we discuss above, biologists tend to visit the same field sites over and over again. In their curatorial role, the field collector often shapes those sites to facilitate collection: setting traps, laying down transect lines, rolling over boulders, tromping out a trail, deploying GPS collars, etc. Finding the specimens and samples worth collecting is thus not a matter of pure serendipity and chance encounters, but an application of the fieldworker’s efforts in shaping the space to afford discovery.

A curator, unlike a prospector, has an eye for the systematic as well as for especially valuable novel discoveries. Consider the museum curator, who in the name of comprehensiveness displays mundane quartz and calcite specimens alongside flashy gem minerals. In this, the biological fieldworker is often more curatorial than purely prospective. Their expert eye is attuned to the scientific anomalies, yes, but they are also interested in having what they collect accurately represent the biological systems of interest. The interplay, and sometimes tradeoff, between novelty and systematicity requires the fieldworker to make curatorial judgments on the basis of a constellation of factors, not just the epistemic value of specimens on a one-by-one basis.

Another important feature of curatorial expertise is that it’s not identical to credentialed scientific expertise, even though there’s overlap. Amateurs play a significant role in biological specimen collection, whether they’re uploading data to a citizen science initiative, volunteering to help classify collected materials, or just bringing a serendipitous find to a museum or wildlife agency.⁸ These amateur collectors can have expertise that PhD-wielding biologists might lack, such as place-based experiential knowledge, or a hobbyist’s aesthetic sensibilities that aid in identifying standout specimens or in making intuitive curatorial judgments.

To summarize, a theory of collection as a scientific practice should account for several things. First, much of the epistemology of collection isn’t found in explicit protocols, but instead in ‘secret epistemologies.’ Second, while field collection varies in how structured it is, it always retains an exploratory nature, involving responsiveness to field conditions and in-the-moment application of expertise. Third, unlike experimental manipulations, field collection interventions are curatorial, involving shaping a space or system to facilitate meaningful collection rather than to control for irrelevant variables. Fourth, effective field collection requires balancing a soft tradeoff between systematic representation and novel or anomalous specimens. And fifth, collection is often accessible to amateurs with alternative sources of expertise such as hobbies and local knowledge. These features hardly constitute a full theory of scientific collection, but they’re enough of a starting point that we can test their utility against the case of locality bias.

5 Conclusion: Applying the Philosophy of Collection to Locality Bias

We’ve argued that biological specimen collection predominantly occurs in a systematically skewed subset of habitats. This locality bias is bias in a thin statistical sense, and so is potentially, but not essentially, worrisome. It takes further analysis to demonstrate that a collection skewed by locality bias is a scientific problem, rather than the result of good epistemic and practical considerations. Often, ‘further analysis’ means determining whether the collection, even skewed by locality bias, is adequate for purpose. Given the plural and indeterminate purposes of biological collections, however, we don’t always have all the necessary details to fully determine adequacy for purpose. A philosophy of scientific collection could fill in some of those necessary details and thus aid in analysis of whether and how locality bias is a problem, but such a theory of collection doesn’t exist.

We can’t provide a full theory in this space, but we’ve proposed some draft features of a philosophy of collection. To conclude, we’ll use those features as a lens to think about locality bias.

Let’s start with those secret epistemologies. Currie’s discussion focuses on the secret epistemologies of specimen selection, but they play a similar role in site selection, and this has a couple of ramifications for locality bias. First, when the field collector has a good sense of the purposes of the collecting they’re doing, they may be in a good position to select sites that are adequate for those purposes. That may provide one reason not to worry so much about locality bias in cases where the purposes of collecting are narrow and explicit, and where practical considerations permit choice among a diverse set of field sites. But when the purposes for collection are open-ended or unspecified, the fieldworker is less likely to be able to employ their expertise to select sites that serve those purposes. Locality bias is thus more likely to be a problem for general collections than for task-specific ones.

Relatedly, the ability of the fieldworker to employ their sense of purpose in site selection combines with a second feature of collection, the significant role of amateurs, to suggest another lesson. Amateurs such as hobbyists and community scientists should, where possible, be informed about the purposes of the collection they contribute to, rather than being treated as inexpert labor. This suggests that biological collection projects should adopt models of citizen science and community science that are less ‘authority-driven’ in the sense of Ottinger (2017). Authority-driven models of citizen science try to minimize the influence amateurs’ own knowledge and experience have on the project. We’re suggesting that, particularly in situations where amateurs have freedom to collect where they choose, those amateurs can mitigate potential pitfalls of locality bias better when they have a detailed understanding of the scientific aims of the collection. That should be a consideration in what models of publicly-engaged science biology adopts.

Shifting focus, we’ve argued that scientific collection requires balancing novelty with systematicity. What does this say about locality bias? A fully systematic approach to collection would strive to eliminate locality bias entirely, at least with respect to the systems the collection aims to represent. But systematicity can trade off with other purposes of collection, such as the epistemic⁹ value of novel or distinctive specimens. That tradeoff can become a reason to accept some lack of systematicity, to accept some representational mismatch between a collection and the systems it represents. There is no general rule of thumb about how to strike this balance. Part of the role of the collector as curator is to make informed judgments about how to do so. This is typically clear at the level of specimen selection, but if our analysis is right, it applies just as much to site selection. Determining how much and what sort of locality bias is tolerable is itself an act of curation. That may seem obvious by this point in the paper, but if you’ll think back to how we described locality bias in Section II, you’ll recall that site selection is frequently not handled curatorially. Instead, it’s often driven case-by-case by practical considerations. A key implication of our discussion then is that intentional, top-down management of locality bias is important, not because we need to eliminate all the bias, but because adjudicating the bias is a matter of curatorial judgment. At some scales of collecting, a research team might be able to manage this themselves merely by being more intentional about site selection. But at other scales, this will require institutional support. For instance, where biologists recognize a need to mitigate the locality bias against private land, Tribal land, and so on, museums and universities can provide community engagement resources such as training, earmarked funding, and services of professional community engagement coordinators. Likewise, when a scientific field notices a systematic gap in collection, it might issue challenges and bounties for collecting in that domain, a practice that’s been successful in other disciplines relying on field collection (e.g., Hazen et al. 2016).

One more lesson, this time drawing on the exploratory, responsive, and in-the-moment nature of field collection. These features of collection can make it look like an epistemically impoverished scientific practice, and it has in fact been treated as such in biology (Kohler 2002a). If we think that the ‘scientific method’ is, as we’re taught in school, essentially about testing hypotheses, then exploratory practices look like inferior science. The responsiveness of collecting to the conditions of the messy natural world is the opposite of the valorization of control in gold standard experimental science. And the role of hard-to-articulate expert judgment in making collection decisions contradicts traditional ideas of science as involving a “view from nowhere,” meaning not from the perspective of an embodied human being (see Harding 1995; Haraway 2013). So, for many reasons, it looks like field collection isn’t top shelf science. It’s just scientific foreplay, perhaps with dismissive emphasis on play, that sets the stage for the real science that happens later. That real science, according to this picture, occurs once the collected material is used to generate data for good old view-from-nowhere hypothesis testing.

Our discussion of locality bias reveals just how wrong this attitude is—especially if collected material is going to serve as data for further research. The ways in which collection differs from experiment as a scientific practice don’t make it epistemically inferior. On the contrary, they can create more epistemically valuable collections. Many of the features driving the kind of locality bias that can distort collections qua sources of data are the result of the pressure for collecting to disguise itself as a kind of experimentation. For example, revisiting the same site over and over again gives the appearance that specimens are collected under conditions of something like experimental control, because using an identical site suggests that variables are likely to be similar each collecting trip. That makes sense, when collection is supposed to be some sort of pseudo-experiment. But if the collected specimens are to become data for further research at a scale beyond that one site, it creates a very unrepresentative sample. The collection would be more epistemically valuable if we stopped worrying about controlling variables so much, and thus collected from diverse localities. Similarly, we’ve just argued that identifying where locality bias is scientifically distorting is curatorial, that is, it involves application of expertise. Collections can thus be better sources of information when we let experts make judgments about how to balance systematicity and novelty, and all the other purposes of collection. This is true even when those judgments are in-the-moment judgments (“let’s run the transect there”) or consist of a view-from-somewhere, as they certainly must in the field.

Let’s boil down these last few paragraphs. There’s a connection between collections as sources of data, and collections as result of collection. Allowing collection to look less like experiment can mitigate problematic kinds of locality bias, which means more representative and reliable data. Consequently, there’s significant epistemic benefit to lowering the pressure for field research to look like confirmatory, experimental research, which challenges the second-class status of non-experimental methods, like collection.

If you’re keeping score, that’s four or so lessons we’ve been able to draw by applying some ideas for a philosophy of collection to the case of locality bias. It’s okay if you’re not fully sold on some of them. We think we’re drawing the right lessons, but our overarching goal is to illustrate that a philosophy of collection as a scientific practice can help address a significant topic in biology. We don’t want to just gesture at locality bias as a thing worth thinking about, but take some first steps towards engaging with it meaningfully.