Model transferability and implications for woodland management: a case study of Pinyon Jay nesting habitat
 
Transférabilité de modèles et implications dans l'aménagement de secteurs boisés : étude de cas de l'habitat de nidification du Geai des pinèdes

• Abstract
• Introduction
• Methods
  • Field methods
  • Statistical modeling
  • Predicted probabilities
  • Transferring models
  • Transferring critical covariate values
• Results
• Discussion
  • Management implications
• Responses to this article
• Acknowledgments
• Literature Cited
• Erratum

INTRODUCTION

Model transferability, in which the accuracy of an ecological model developed in one location is evaluated when applied to another location, is increasingly important to ecologists, particularly when models provide the basis for land management (Moon et al. 2017). When model transferability is good, managers can make better-informed conservation decisions about new, unsampled areas (van Reusel et al. 2007). Among studies of birds, transferability is typically assessed for models of resource selection, habitat suitability, or species distribution. Most studies assessing model transferability employ covariates from remotely sensed or GIS datasets such as land cover (Gray et al. 2009, Street et al. 2015). Less commonly, the transferability of these models has been assessed at smaller scales, such as the area immediately around nests of breeding birds (Latif et al. 2016). Transferred models generally perform well within the same environment or climate (van Reusel et al. 2007, Street et al. 2015). However, they may transfer poorly despite apparent ecological similarities between locations (Leftwich et al. 1997) or when projected across geographic or climatic spaces (Randin et al. 2006, Torres et al. 2015, Latif et al. 2016). Thus, model transferability is an important consideration (Wogan 2016).

The transferability of models from areas of model development to areas of application is often measured via rank metrics such as the area under the receiver operating curve (AUC; Fielding and Bell 1997). AUC offers the advantage of being threshold-independent and therefore robust to imbalanced classes. This is desirable in assessing the transferability of models that may employ varying proportions of zeros because of rarity or nonstandardized data collection, or to increase statistical power (Franklin et al. 2009). However, AUC and other threshold-independent metrics are limited when assessing the classification of occurrences (sensitivity) or nonoccurrences (specificity; Lobo et al. 2008). This information may be critical to management, which may place higher conservation value on occurrence locations such as a forest patch than nonoccurrence locations (Latif et al. 2016).

Threshold-dependent metrics such as sensitivity and specificity may adequately identify model transferability under complex management needs (Latif et al. 2016). A requirement for this approach is the availability of the same measures at the application location as those employed at the development location. Unfortunately, remotely sensed or other habitat data may not be readily available for a new area, or model covariates may occur at resolutions that cannot be measured via satellite or other remote platforms, such as the diameter of individual trees (Latif et al. 2016). In addition, even when finer scale data are available for modeling, the consequences of on-the-ground management based on model-predicted thresholds are not readily apparent.

More pragmatic approaches to management have based recommendations on the numerical distribution of one or more finer scale covariates that (1) discriminate locations of animal occurrence from those of nonoccurrence and (2) are typically employed in management, e.g., tree diameter (Bunnell et al. 2002, Marcot et al. 2010). Although most studies using these approaches have not addressed transferability of this information among areas, they have been interested in applying known information on habitat use to guide on-the-ground management in new areas. The impact of transferring critical values such as median nest tree diameter from one area to prescribe management at another is likely to vary, e.g., in percent nest trees removed, depending on the characteristics of the areas involved. Few if any studies have extended model transferability to evaluate the management consequences of transferring covariate critical values across areas.

The Pinyon Jay (Gymnorhinus cyanocephalus) is a medium-sized jay of the corvid family (Corvidae) inhabiting roughly 350,000,000 ha across the western USA, primarily in piñon-juniper (Pinus edulis, P. monophylla, Juniperus spp.) woodlands. Pinyon Jays have a mutualistic relationship with piñon pines whereby piñon trees produce highly nutritional seeds in large crops at irregular intervals (mast crops; Forcella 1981, Parmenter et al. 2018) and Pinyon Jays serve as the trees’ main long-distance seed disperser (Ligon 1978, Lanner 1996). Abundant piñon seeds sustain Pinyon Jays throughout the winter, support successful nesting, and significantly influence population viability (Marzluff and Balda 1992). In turn, Pinyon Jays can transport millions of piñon seeds to caching areas up to several miles from the source woodland and cache them in favorable microhabitats for seed germination (Ligon 1978). Pinyon Jays are social, flocking in winter and nesting colonially, sometimes cooperatively, on traditional nesting grounds (Marzluff and Balda 1992). They nest primarily in piñon and juniper trees (Johnson et al. 2014, 2017). Flock home ranges can cover up to 4000 ha in the breeding season and 6000 ha year round (Johnson et al. 2016). Winter flocks can number in the hundreds (Balda 2002) and may move outside the home range when food is scarce (Balda 2002).

North American Breeding Bird Survey data indicate that Pinyon Jay populations declined by an estimated 3.69% per year during 1967–2015 (Sauer et al. 2017). The Pinyon Jay was listed as Vulnerable on the Red List of Threatened Species by the International Union for Conservation of Nature (BirdLife International 2017). Potential impacts to Pinyon Jays include climate change (Johnson et al. 2017) and woodland thinning (Johnson et al. 2018), a management practice employed worldwide to reduce potential wildfire risk (Huffman et al. 2009), increase timber productivity (Taki et al. 2010), or improve ecological condition, e.g., thinning to restore ecosystem structure and processes (Brown et al. 2019). Because of long-term population declines (Sauer et al. 2017, Boone et al. 2018) and recent and predicted impacts of climate change on their piñon-juniper habitats (Williams et al. 2010, McDowell et al. 2016), regulatory agencies and land managers need guidance on habitat management to benefit Pinyon Jays.

We address the transferability of habitat models in the context of woodland management at fine scales for the Pinyon Jay. Although Pinyon Jay behavior is well-studied (e.g., Johnson 1988a, b, Marzluff and Balda 1988, 1992, Bednekoff and Balda 1996), few studies have investigated variation in patterns of occurrence or habitat use, and none has evaluated transferability of models describing these patterns or the management consequences of transferring critical covariate values from one area to another. In addition to the conservation-related need for information on the generality of habitat models, Pinyon Jays are an appropriate model for a transferability study because, although most nesting colonies are in woodlands composed of the same tree species, woodland structure varies across different colony sites. Hence, it was unclear to what extent nesting habitat use is site-specific vs. fixed across sites.

We use the occurrence of Pinyon Jay nests as the ecological response at several local breeding areas (Johnson et al. 2014, 2015), where varying habitat conditions may confound one-size-fits-all habitat management prescriptions. Our objectives were the following: (1) to fit nest-scale models of Pinyon Jay nest occurrence at four areas in New Mexico, (2) to quantify model transferability among the four areas, and (3) using a management scenario of tree thinning for fuels reduction, assess the implications of managing one site based on the use of nest trees at another site.

METHODS

Field methods

We studied the occurrence of Pinyon Jay nests relative to habitat characteristics and transferability of occurrence models at four nesting areas (nine colony locations) in New Mexico, USA: Farmington Field Office (FFO) of the U.S. Bureau of Land Management (FARM), Crow Mesa in the FFO (CROW), Kirtland Air Force Base, NM (KIRT), and White Sands Missile Range, NM (WSMR; Fig. 1, Table 1). We use “site” hereafter to refer to the four nesting areas, “colony location” to mean the location of a nesting colony within sites, and “plot” to refer to a nest or non-nest plot within a colony.

We located Pinyon Jay nesting colonies by watching flock movements during roadside and walking searches in April and May (protocol described in detail in Petersen et al. 2014). We performed all colony searches in the morning hours. We conducted driving surveys where Pinyon Jays had been reported or in areas of suitable habitat. Pinyon Jays are very vocal and can be heard over road and engine noise if driving slowly. Because only birds close to the road can be detected visually and by calls, we stopped and exited the vehicle approximately every 0.8 km to listen for vocalizations for approximately 10 min.

When we detected Pinyon Jays showing breeding behaviors (Petersen et al. 2014), we remained in the area for at least one hour or until we confirmed nesting, monitoring the birds from a distance to avoid disturbing them. We found nests by watching for nesting behaviors such as nest construction, incubation, and nest attendance. We recorded active nest locations with a GPS waypoint while standing at least 20 m from the nest and recorded GPS points at inactive and old nests. After all birds had finished nesting, we searched thoroughly within and around the area delineated by nests and recorded GPS coordinates of all additional nests.

We collected vegetation data following a modified BBIRD protocol (Martin et al. 1997). Circular plots were centered on a nest tree, and we generated a sample of non-nest plots in each colony location by centering plots on a tree 100 m in a randomly selected direction from each nest tree without evidence of previous nesting. We measured vegetation within radii of 5 m (0.01 ha) and 11.3 m (0.04 ha) of each nest tree and center tree on non-nest plots. We chose covariates that significantly discriminated nests from non-nests at one or more sites in previous analyses (Johnson et al. 2014, 2015), after eliminating those highly correlated (|r| > 0.7) with other covariates, which left nine covariates at three scales for inclusion in models. At the nest tree or center tree on non-nest plots, we measured root crown diameter (RCD, the diameter of the trunk at the soil level, used in lieu of diameter at breast height for tree species having multiple trunks), height, and canopy cover (mean of canopy presence or absence at the plot center and 1–5 m in each cardinal direction). At the 5-m scale, we measured canopy cover and the stem density, mean RCD, and mean height of all trees over 1 m tall. We assigned each tree on the 11.3-m plot to a size class. To estimate RCD from these classes we used the mean RCD values within these classes (small < 7 cm; medium = 7-21 cm; large > 21 cm) across the smaller plot radius to estimate RCD at the larger plot size. These mean values were 5 cm (small), 13 cm (medium), and 35 cm (large). We calculated stem density and RCD at this scale.

Statistical modeling

We modeled the occurrence of nests within colonies using generalized linear mixed effects models with binomial error (lme4 package, Bates et al. 2017), in which year and colony were included as crossed random effects. These random effects were intercepts assumed to be independent and identically distributed. To assess model transferability, we repeated modeling steps using observations of nests and non-nests at (1) single development sites and (2) combinations of three development sites.

We transferred model sets from single development sites to the other single application sites. We applied model sets from all combinations of three development sites, with a model selection process for each site combination (see below), to single application sites. Transferring model sets combining multiple sites reflects scenarios in which knowledge from several known areas is applied to an unknown area.

Another approach to assessing transferability is k-fold cross-validation (Wenger and Olden 2012). In the k-fold process, models originally fit to observations across all development sites are iteratively refit using the same covariate combination, by withholding each site and predicting the probability of occurrence at locations in this withheld site. We did not adopt this approach here, because using a model-selection process fit to a data set of all sites combined is not reflective of a realistic management scenario in which the focal site is unstudied.

Modeling for each site or site combination involved several steps. Because of the tendency for breeding birds to select nesting locations with characteristics having intermediate values relative to their availability (Götmark et al. 1995), we first assessed whether quadratic terms for each covariate were potentially predictive. We made this assessment in mixed effects models as described above, using data from single and combined sites by comparing models with (1) only the linear form of the covariate vs. (2) both a linear and quadratic form of the covariate. If (1) the Akaike information criterion for small samples (AICc) was lower in the quadratic model and (2) the quadratic term was informative (P < 0.15; Arnold 2010), we allowed its inclusion during model selection.

We undertook model selection by first fitting a global model of the nine covariates described above (plus predictive quadratic forms) for each site or combination of three sites. We then performed an all-subsets model selection process (Burnham and Anderson 2004) by identifying subset covariate combinations using the MuMin package (Bartoń 2018). We limited model subsets by removing those models exhibiting high variance inflation factors (> 2; Zuur et al. 2010) or containing uninformative parameters (Arnold 2010). The former reduced the potential effects of multicollinearity while the latter avoided overparameterization by preventing the inclusion of parameters with weak effects. This resulted in candidate sets of 10–194 models. Finally, within these candidate sets, we considered competitive those models within four AICc units of the lowest-AICc model (ΔAICc < 4) in each model set (Burnham and Anderson 2002). Models in which three sites were combined into a single data set employed weighted observations such that each site received the same weight in models and weights summed to the total number of observations. This total varied across combinations of three sites, and was 304 when all sites were combined.

Predicted probabilities

To avoid problems associated with model-averaged parameter estimates (Cade 2015), we used competitive model sets to generate predicted probabilities of nest occurrence. Each model in a set was used to generate a vector of marginal predictions (no random effects of site and year) via linear combinations of parameter estimates (betas) and a matrix of observed covariate values. This produced a matrix with a number of rows equal to the number of observations and number of columns equal to the number of competitive models. To generate a final prediction vector, matrix multiplication was employed between this prediction matrix and a vector of model weights associated with the model set (w_i, calculated from ΔAICc values; Burnham and Anderson 2002). We employed these predictions in calculating internal AUC statistics and to determine prediction threshold values at which sensitivity (correctly classified nests) was 75% (an intermediate level consistent with Latif et al. 2016). AUC was calculated with the AUC package (Ballings and Ven den Poel 2013).

Transferring models

The process of transferring models involved the same steps as described for internal predictions. However, instead of using linear combinations of parameter estimates (betas) and a matrix of observed covariate values used in model fitting (i.e., data from the development site), we used the matrix of observed covariate values from application sites not used in model development.

After generating transferred predictions (via weighted averages if necessary) we employed these predictions to evaluate the ability of models to discriminate nests from non-nests when transferred to other sites. We (1) used AUC as a general, threshold-independent measure of discriminatory accuracy, and (2) predicted probability thresholds at which development-site model sensitivity reached 75%. We expected transferability using the first approach (AUC) to range from ≤ 0.5, indicating transferability no better than random, to 1, indicating perfect transferability. Transferability using the second approach (via thresholds) corresponded to an acceptable loss of 25% of nest sites at model development sites. Under this management scenario, trees with predicted probabilities below thresholds could be removed during woodland management. We expected variation in transferability of these thresholds ranging from a combination of high sensitivity and low specificity (all nests retained, few non-nests removed) to a combination of low sensitivity and high specificity (few nests retained, all non-nests removed).

Transferring critical covariate values

In addition to assessing transferability using model predictions (AUC and probability thresholds), we assessed the transferability of site-specific information about habitat use by nesting Pinyon Jays. This process can be useful in meeting management objectives of conserving the majority of used habitat while allowing alteration of unused habitat. We used a covariate present in all competitive model sets, plot center tree RCD, which is commonly used by foresters to identify trees for removal (Lee and Irwin 2005). We used critical values corresponding to the lower quartile (25%) of RCD values within each site or across combined sites. This is akin to probability thresholds used in determining 75% sensitivity across development locations (Latif et al. 2016). In our scenario of prescribed forest thinning for fuels reduction, critical RCD values from one site are used to guide thinning at another site. We transferred these prescriptions to the other three application sites or withheld site (when critical RCD was calculated for combined sites) and determined percentages of nest plots retained and non-nest plots removed under this thinning scenario. All analyses were conducted in R (R Development Core Team 2019).

RESULTS

Vegetation varied among the four sites and between nest and non-nest plots within sites (Table 2). Although center tree RCD and height varied across sites, nest trees within sites were generally larger diameter, taller, and had higher canopy cover than non-nest center trees. Mean canopy cover and number of trees were higher on the 5-m nest plots than on non-nest plots.

The complexity and number of supported models varied between development sites (Table 3), with 3–15 models supported and 1–7 covariates included per model set. All covariates examined were present in supported models for one or more sites (Table 3) and eight covariates had 95% confidence intervals not bounding zero in at least one single-site model set. Of these eight covariates, three (nest tree or non-nest center tree RCD, nest tree or non-nest center tree height, canopy cover within 5 m) had 95% CIs not bounding zero and appeared in model sets of more than one site. Most covariates had overall positive effects on the probability of nest occurrence. The size of these effects varied between sites; e.g., nest or non-nest center tree root crown diameter had an effect at CROW nearly 10 times that at KIRT. Most effects were linear and trending positive, indicating selection for larger trees, taller trees, higher canopy cover, and denser woodlands. Exceptions in models from FARM included quadratic effects of center tree RCD, indicating selection for intermediate-sized nest trees, and negative or quadratic relationships of several covariates at 5-m or 11.3-m plot scales, indicating selection for nesting sites near smaller, somewhat shorter trees.

The internal accuracy of model predictions as measured by AUC was fair (defined as 0.7–0.8) to good (defined as 0.8–0.9; Table 4), with a mean AUC across single-site models of 0.81 (range = 0.77–0.89; Table 4). Model transferability was overall fair between individual sites (mean = 0.71; range = 0.51–0.86) and fair from combined sites to individual sites (mean = 0.71; range = 0.52–0.88). Development sites varied in how well they transferred to other sites (range of means = 0.69–0.73), with CROW transferring most poorly, and KIRT and WSMR transferring most accurately. Likewise, application sites varied in how well they were predicted (range of means = 0.57–0.82), with KIRT and CROW being lowest and highest, respectively.

Model-predicted probability thresholds associated with 75% sensitivity resulted in mean internal specificity of 53% (range = 25-86%; Table 4). Transferability of these thresholds varied between sites with a mean sensitivity of 42% (range = 0–100%) and mean specificity of 67% (range = 22–100%). Sensitivity was higher (mean = 60%, range = 14–100%) and specificity lower (mean = 62%, range = 35–93%) when sites were combined and thresholds were transferred. Neither transferred sensitivity nor specificity was correlated with AUC (|r_s| < 0.2).

Our hypothetical scenario of forest thinning, in which tree removal would result in a retention of at least 75% of nests (nest tree RCD > 25th percentile of nest tree RCD in each site; Table 4), would allow the removal of a mean 53% (range = 25–86%) of non-nest trees within development sites. Transferring critical RCD values from individual development to application sites would result in both retention of nest trees above and below the nominal 75% (mean = 73%, range = 21–100%) and allow the removal of an average 57% (range = 25–100%) of non-nest plot center trees. Variation in transferability of RCD values in this scenario would be reduced and the average number of correctly classified nests increased if critical RCD values were calculated from multiple (combined) development sites and transferred to application sites (retained nests: mean = 75%, range = 60–93%; removed non-nest trees: mean = 55%, range = 35–93%). Transferring critical RCD values from FARM would result in the lowest average nest retention (mean = 40%, range = 21–53%) and allow the highest non-nest tree removal (mean = 75%, range = 56–100%), while values from KIRT would retain both a high proportion of nests (mean = 90%, range = 79–100%) and allow for adequate removal of non-nest trees (mean = 49%, range = 25–79%). Neither transferred nest retention nor non-nest tree removal was correlated with AUC (|r_s | < 0.5). Although transferred nest retention was not correlated with either sensitivity or specificity (|r_s | < 0.6), transferred non-nest tree removal was correlated with both (|r_s | > 0.7, P < 0.01). No measure of transferability was correlated with the distance between development and application sites (|r_s| < 0.2).

DISCUSSION

Consistent with previous findings (Johnson et al. 2014, 2015), Pinyon Jays appeared to select larger diameter and taller trees in areas with higher canopy cover than available areas within colonies. Despite these similarities, characteristics discriminating nests from non-nests differed across sites, as evidenced by their effects in models and the transferability of both prediction thresholds and critical values of RCD.

As measured by AUC, the internal accuracy of single-site models was on average good. Transferability from single development sites to application sites varied from unsuccessful (AUC < 0.6) to good but was overall fair. The same cannot be stated about transferring model-predicted thresholds between sites; given a target sensitivity of 75%, the mean transferred sensitivity of 42% represents an unacceptable degree of misclassification, particularly if these thresholds are to be employed in targeting areas for conservation. Low agreement between transferred AUC and sensitivity reiterates concerns that while classifiers such as AUC offer the advantage of threshold-independence, they may have limited utility in real-world scenarios (Lobo et al. 2008). Although these patterns are clear, predictions used in calculating AUC and thresholds were derived from a single analysis framework (generalized mixed-effects models); additional approaches, e.g., random forest, may have resulted in better transferability (Wenger and Olden 2012).

Utilizing critical RCD values corresponding to 75% sensitivity resulted in overall good between-site transferability under a tree-thinning scenario. There were some notable exceptions, the most extreme of which would have resulted in a loss of nearly 80% of nests in one site. The variability of critical-value transferability was greatly reduced and average transferability increased when critical values were calculated from combinations of three sites.

Reduced accuracy of models predicting occurrence has been associated with a variety of species traits (McPherson and Jetz 2007, Wogan 2016). That Pinyon Jays share few of these traits, e.g., large body size, nonmigratory rather than migratory, woodland- rather than wetland-associated, may contribute to the generally good accuracy of the single-site models. However, the scale of our study was much smaller than those reviewed in the above studies. In addition, the close association of Pinyon Jay with piñon-juniper woodlands in the southwestern USA, and their reliance on piñon trees for both nesting and food, is a fundamental feature of their biology operating across their New Mexico range. This accounts for the inclusion of nest/center tree measures in all single-site nest models.

The differences among the single-site models may be influenced by necessary trade-offs, e.g., predator avoidance vs. thermal considerations, which likely vary spatially and temporally. The main causes of Pinyon Jay nest failure are avian predators and cold and snowy spring weather (Marzluff 1988; K. Johnson, unpublished data). Pinyon Jays made adjustments in nest placement depending on the success of previous nests, moving nests to warmer locations after failure caused by cold weather and to more concealed locations after failure due to predation, but choosing similar locations after nesting successfully (Marzluff 1988). This suggests that choice of a nest location is influenced by multiple selective factors that interact with habitat variability across the Pinyon Jay’s range.

The question motivating this study came from habitat managers who repeatedly requested specific values of Pinyon Jay habitat covariates, e.g., “typical” canopy cover or tree diameter, to guide management of Pinyon Jay nesting habitat. These requests assumed that habitat use is fixed, i.e., important covariates and their values are similar across sites. Alternatively, habitat choice could be relative, i.e., habitat use depends on what is available at a site.

The differences in effects within covariates across single-site models indicate that Pinyon Jays in this study did not employ fixed habitat use. Likewise, the varying inclusion of covariates in model sets indicates birds were not operating on the same “model” of selection for nesting locations. Our results indicate that choice of nesting habitat is relative to the available woodland structure at a site. It is also sometimes complex, with multiple influential habitat features that differ among sites. We do not mean to imply local adaptation, in which natural selection has differentiated populations by habitat preference. Pinyon Jays may disperse far beyond their home ranges (Balda 2002). The distances between sites in our study, while large, are certainly within known dispersal distances of this species. Our results suggesting adjustable habitat selection are consistent with the intelligence and behavioral flexibility of this bird (Balda 2002, Bond et al. 2007, 2010).

Given the fairly wide geographical separation and variation in vegetation structure of our study sites, transferability between sites was surprisingly high when the sites shared similar covariates and covariate parameter estimates in their competitive model sets. For example, good transferability of models from FARM and WSMR to CROW was likely due to the positive effect of nest or non-nest center tree height in models for each site. In contrast, models for FARM and CROW failed to transfer to KIRT (AUC < 0.6), likely because of strong positive effects of nest or non-nest center tree height in the former sites and its absence from competitive models for the latter site.

Transferable models should “describe biologically meaningful environmental relationships that generally determine species distributions while avoiding location specific relationships that are not generalizable” (Latif et al. 2016:778). Studies of model transferability in woodland birds at the nest scale are rare, but one study of two woodpecker species nesting in burned habitats (Latif et al. 2016) features some points of comparison. In that study, models discriminated nests from non-nests at development locations fairly well, but performance and transferability at application locations were variable, and largely poor. As here, that study concluded that single-area models are unlikely to be generally applicable across the entire range of their study species and recommended integration of data from multiple development locations. However, that study did not include a management scenario as presented here.

Management implications

In our scenario, employing critical values of RCD from one site to dictate the extent of thinning at another site resulted in highly variable rates of nest tree retention and non-nest tree removal. For example, although the transfer of critical RCD values from FARM to other sites would allow managers to thin most non-nest trees (mean = 75%, range = 56–100%), this would also result in unacceptably low retention of nest trees (mean = 40%, range = 21–53%). In contrast, transferring critical RCD values from KIRT to other sites would result in high retention of nest trees (mean = 90%, range = 79–100%) and high allowable thinning of non-nest trees (mean = 51%, range = 21–75%).

RCD appears informative in Pinyon Jay habitat management scenarios because it (1) is used by foresters to identify trees for removal (Lee and Irwin 2005) and (2) was a predictive measure in competitive models across all sites. Nonetheless, it is important to consider the shortcomings of focusing on a single covariate when management planning. First, competitive model sets of nest occurrence for each site included up to seven covariates; one or two measures did not adequately describe patterns of nest occurrence in most sites and are therefore likely to exhibit lower accuracy (AUC, sensitivity, and specificity) in discriminating nests from non-nests in unknown areas. Second, removing trees based on RCD would affect stem density, which was present in model sets of two individual sites and model sets of all combinations of sites. Finally, if site-specific models do not share covariates, model transferability will likely be low, and a management scenario like our RCD example will likely be unproductive.

Based on these results, we recommend the best approach to managing habitat for Pinyon Jays and other woodland-dependent species is to understand the local combination of factors discriminating locations of occurrence from those of nonoccurrence. With this information, management for retention of occurrence locations, e.g., nests, to desired levels can be achieved by identifying critical values of discriminating factors, e.g., RCD, tree density, or canopy cover.

If occurrence is unknown in an area or if habitat measurement and modeling of occurrence vs. nonoccurrence are not possible, we recommend conservative management based on an approach similar to that employed in our scenario of thinning based on RCD. Assuming habitat measurements are available from more than one occupied area, we recommend the use of critical values of relevant woodland covariates combined among these areas, then transferring these to other areas with unknown occurrence. This is akin to recommendations made by Latif et al. (2016) in transferring multiarea thresholds for improved management and reduces the potentially biasing influence of areas with habitat characteristics at more extreme ends of their distribution. Additional testing with more areas and multiple influential covariates would improve confidence in this approach. We suspect this process will assist managers in identifying more optimal critical values of one or more woodland covariates necessary to achieve target retention levels at unknown occurrence locations or nonoccurrence locations.

LITERATURE CITED

Arnold, T. W. 2010. Uninformative parameters and model selection using Akaike’s Information Criterion. Journal of Wildlife Management 74:1175-1178. https://doi.org/10.1111/j.1937-2817.2010.tb01236.x

Balda, R. P. 2002. Pinyon Jay (Gymnorhinus cyanocephalus), version 1.0. In A. F. Poole and F. B. Gill, editors. The birds of North America. Cornell Lab of Ornithology, Ithaca, New York, USA. https://doi.org/10.2173/bna.605

Ballings, M., and D. Van den Poel. 2013. AUC: threshold independent performance measures for probabilistic classifiers. R package version 0.3.0.

Bartoń, K. 2018. MuMIn: multi-model inference. R package version 1.40.4.

Bates, D., M. Maechler, B. Bolker, S. Walker, R. H. B. Christensen, H. Singmann, B. Dai, G. Grothendieck, and P. Green. 2017. lme4: Linear mixed-effects models using Eigen and S4. R package version 1.1-14.

Bednekoff, P. A., and R. P. Balda. 1996. Social caching and observational spatial memory in Pinyon Jays. Behaviour 133(11-12):807-826. https://doi.org/10.1163/156853996X00251

BirdLife International. 2017. Gymnorhinus cyanocephalus (amended version of 2016 assessment). The IUCN Red List of Threatened Species 2017: e.T22705608A110431877. [online] URL: https://www.iucnredlist.org/species/22705608/110431877

Bond, A. B., A. C. Kamil, and R. P. Balda. 2007. Serial reversal learning and the evolution of behavioral flexibility in three species of North American corvids (Gymnorhinus cyanocephalus, Nucifraga columbiana, Aphelocoma californica). Journal of Comparative Psychology 121: 372-379. https://doi.org/10.1037/0735-7036.121.4.372

Bond, A. B., C. A. Wei, and A. C. Kamil. 2010. Cognitive representation in transitive inference: a comparison of four corvid species. Behavioural Processes 85:283-292. https://doi.org/10.1016/j.beproc.2010.08.003

Boone, J. D., E. Ammon, and K. Johnson. 2018. Long-term declines in the Pinyon Jay and management implications for piñon-juniper woodlands. Pages 190-197 in W. D. Shuford, R. E. Gill Jr., and C. M. Handel, editors. Trends and traditions: avifaunal change in western North America. Studies of Western Birds 3. Western Field Ornithologists, Camarillo, California, USA. https://doi.org/10.21199/SWB3.10

Brown, G. W., A. Murphy, B. Fanson, and A. Tolsma. 2019. The influence of different restoration thinning treatments on tree growth in a depleted forest system. Forest Ecology and Management 437:10-16. https://doi.org/10.1016/j.foreco.2019.01.022

Bunnell, F. L., E. Wind, M. Boyland, and I. Houde. 2002. Diameters and heights of trees with cavities: their implications to management. Pages 717-737 in W. F. Laudenslayer, P. J. Shea, B. E. Valentine, C. P. Weatherspoon, and T. E. Lisle, editors. Proceedings of the symposium on the ecology and management of dead wood in western forests. General Technical Report PSW-GTR-181. U.S. Forest Service, Pacific Southwest Research Station, Albany, California, USA. https://doi.org/10.2737/psw-gtr-181

Burnham, K. P., and D. R. Anderson. 2002. Model selection and multimodel inference: a practical information-theoretic approach. Springer, New York, New York, USA.

Cade, B. S. 2015. Model averaging and muddled multimodel inferences. Ecology 96:2370-2382. https://doi.org/10.1890/14-1639.1

Fielding, A. H., and J. F. Bell. 1997. A review of methods for the assessment of prediction errors in conservation presence/absence models. Environmental Conservation 24(1):38–49. https://doi.org/10.1017/s0376892997000088

Forcella, F. 1981. Ovulate cone production in pinyon: negative exponential relationship with late summer temperature. Ecology 62:488-491. https://doi.org/10.2307/1936722

Franklin, J., K. E. Wejnert, S. A. Hathaway, C. J. Rochester, and R. N. Fisher. 2009. Effect of species rarity on the accuracy of species distribution models for reptiles and amphibians in southern California. Diversity and Distributions 15:167-177. https://doi.org/10.1111/j.1472-4642.2008.00536.x

Götmark, F., D. Blomqvist, O. C. Johansson, and J. Bergkvist. 1995. Nest site selection: a trade-off between concealment and view of the surroundings? Journal of Avian Biology 26:305-312. https://doi.org/10.2307/3677045

Gray, T. N. E., R. Borey, S. K. Hout, H. Chamnan, N. J. Collar, and P. M. Dolman. 2009. Generality of models that predict the distribution of species: conservation activity and reduction of model transferability for a threatened bustard. Conservation Biology 23(2):433-439. https://doi.org/10.1111/j.1523-1739.2008.01112.x

Huffman, D. W., P. Z. Fulé, J. E. Crouse, and K. M. Pearson. 2009. A comparison of fire hazard mitigation alternatives in pinyon-juniper woodlands of Arizona. Forest Ecology and Management 257:628-635. https://doi.org/10.1016/j.foreco.2008.09.041

Johnson, K. 1988a. Sexual selection in Pinyon Jays I: female choice and male-male competition. Animal Behaviour 36(4):1038-1047. https://doi.org/10.1016/s0003-3472(88)80063-0

Johnson, K. 1988b. Sexual selection in Pinyon Jays II: male choice and female-female competition. Animal Behaviour 36(4):1048-1053. https://doi.org/10.1016/s0003-3472(88)80064-2

Johnson, K., T. B. Neville, J. W. Smith, and M. W. Horner. 2016. Home range- and colony-scale habitat models for Pinyon Jays in piñon-juniper woodlands of New Mexico, USA. Avian Conservation and Ecology 11(2):6. https://doi.org/10.5751/ACE-00890-110206

Johnson, K., N. Petersen, J. Smith, and G. Sadoti. 2018. Piñon-juniper fuels reduction treatment impacts Pinyon Jay nesting habitat. Global Ecology and Conservation 16:e00487. https://doi.org/10.1016/j.gecco.2018.e00487

Johnson, K., G. Sadoti, and J. Smith. 2017. Weather-induced declines in piñon tree condition and response of a declining bird species. Journal of Arid Environments 146:1-9. https://doi.org/10.1016/j.jaridenv.2017.07.006

Johnson, K., L. Wickersham, J. Smith, N. Petersen, and J. Wickersham. 2015. Nest-scale habitat use by Pinyon Jay and Gray Vireo in the BLM Farmington Resource Area 2013–2014. Natural Heritage New Mexico Report 15-GTR-386. Biology Department, University of New Mexico, Albuquerque, New Mexico, USA. [online] URL: https://nhnm.unm.edu/sites/default/files/nonsensitive/publications//FINAL%202014%20BLM%20P-J%20%20report.pdf

Johnson, K., L. Wickersham, J. Smith, G. Sadoti, T. Neville, J. Wickersham, and C. Finley. 2014. Habitat use at multiple scales by pinyon-juniper birds on Department of Defense lands III: landscape, territory/colony, and nest scale. Natural Heritage New Mexico Report 14-GTR-38. Biology Department, University of New Mexico, Albuquerque, New Mexico, USA. [online] URL: https://nhnm.unm.edu/sites/default/files/nonsensitive/publications//Year%203%20Legacy%20FINAL2.pdf

Lanner, R. M. 1996. Made for each other: a symbiosis of birds and pines. Oxford University Press, New York, New York, USA.

Latif, Q. S., V. A. Saab, J. P. Hollenbeck, and J. G. Dudley. 2016. Transferability of habitat suitability models for nesting woodpeckers associated with wildfire. Condor 118(4):766-790. https://doi.org/10.1650/CONDOR-16-86.1

Lee, D. C., and L. L. Irwin. 2005. Assessing risks to spotted owls from forest thinning in fire-adapted forests of the western United States. Forest Ecology and Management 211(1-2):191-209. https://doi.org/10.1016/j.foreco.2005.02.001

Leftwich, K. N., P. L. Angermeier, and C. A. Dolloff. 1997. Factors influencing behavior and transferability of habitat models for a benthic stream fish. Transactions of the American Fisheries Society 126(5):725-734. https://doi.org/10.1577/1548-8659(1997)126<0725:FIBATO>2.3.CO;2

Ligon, J. D. 1978. Reproductive interdependence of Piñon Jays and piñon pines. Ecological Monographs 48(2):111-126. https://doi.org/10.2307/2937295

Lobo, J. M., A. Jiménez-Valverde, and R. Real. 2008. AUC: a misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography 17:145-151. https://doi.org/10.1111/j.1466-8238.2007.00358.x

Marcot, B. G., J. L. Ohmann, K. Mellen-McLean, and K. L. Waddell. 2010. Synthesis of regional wildlife and vegetation field studies to guide management of standing and down dead trees. Forest Science 56:391-404.

Martin, T. E., C. R. Paine, W. Hochachka, P. Allen, and W. Jenkins. 1997. The breeding biology research and monitoring database (BBIRD) program. Montana Cooperative Wildlife Research Unit, University of Montana, Missoula, Montana, USA.

Marzluff. J. M. 1988. Do Pinyon Jays alter nest placement based on prior experience? Animal Behaviour 36:1-10. https://doi.org/10.1016/s0003-3472(88)80244-6

Marzluff, J. M., and R. P. Balda. 1988. Pairing patterns and fitness in a free-ranging population of Pinyon Jays: what do they reveal about mate choice? Condor 90(1)201-213. https://doi.org/10.2307/1368449

Marzluff, J. M., and R. P. Balda. 1992. The Pinyon Jay: behavioral ecology of a colonial and cooperative corvid. T. & A. D. Poyser, London, UK. https://doi.org/10.5040/9781472597359

McDowell, N. G., A. P. Williams, C. Xu, W. T. Pockman, L. T. Dickman, S. Sevanto, R. Pangle, J. Limousin, J. Plaut, D. S. Mackay, et al. 2016. Multi-scale predictions of massive conifer mortality due to chronic temperature rise. Nature Climate Change 6(3):295-300. https://doi.org/10.1038/nclimate2873

McPherson, J. M., and W. Jetz. 2007. Effects of species’ ecology on the accuracy of distribution models. Ecography 30:135-151. https://doi.org/10.1111/j.0906-7590.2007.04823.x

Moon, J. B., T. H. Dewitt, M. N. Errend, R. J. F. Bruins, M. E. Kentula, S. J. Chamberlain, M. S. Fennessy, and K. J. Naithani. 2017. Model application niche analysis: assessing the transferability and generalizability of ecological models. Ecosphere 8(10):e01974. https://doi.org/10.1002/ecs2.1974

Parmenter, R. R., R. I. Zlotin, D. I. Moore, and O. B. Myers. 2018. Environmental and endogenous drivers of tree mast production and synchrony in piñon-juniper-oak woodlands of New Mexico. Ecosphere 9(8):e02360. https://doi.org/10.1002/ecs2.2360

Petersen, N., K. Johnson, and J. Smith. 2014. Pinyon Jay monitoring program for New Mexico. Natural Heritage New Mexico Technical Report No. GTR-14-382. Biology Department, University of New Mexico, Albuquerque, New Mexico, USA. [online] URL: https://nhnm.unm.edu/sites/default/files/nonsensitive/publications/PIJA%20MONITORING%20FINAL%20REPORT.pdf

R Development Core Team. 2019. R: A language and environment for statistical computing. Version 3.5.3. R Foundation for Statistical Computing, Vienna, Austria.

Randin, C. F., T. Dirnbock, S. Dullinger, N. E. Zimmermann, M. Zappa, and A. Guisan. 2006. Are niche-based species distribution models transferable in space? Journal of Biogeography 33:1689-1703. https://doi.org/10.1111/j.1365-2699.2006.01466.x

Sauer, J. R., D. K. Niven, J. E. Hines, D. J. Ziolkowski, Jr., K. L. Pardieck, J. E. Fallon, and W. A. Link. 2017. The North American breeding bird survey, results and analysis 1966-2015. U.S. Geological Survey Patuxant Wildlife Research Center, Laurel, Maryland, USA.

Street, G. M., L. M. Vander Vennen, T. Avgar, A. Mosser, M. L. Anderson, A. R. Rodgers, and J. M. Fryxell. 2015. Habitat selection following recent disturbance: model transferability with implications for management and conservation of moose (Alces alces). Canadian Journal of Zoology 93(11):813-821. https://doi.org/10.1139/cjz-2015-0005

Taki, H., T. Inoue, H. Tanaka, H. Makihara, M. Sueyoshi, M. Isono, and K. Okabe. 2010. Responses of community structure, diversity, and abundance of understory plants and insect assemblages to thinning in plantations. Forest Ecology and Management 259(3):607-613. https://doi.org/10.1016/j.foreco.2009.11.019

Torres, L. G., P. J. H. Sutton, D. R. Thompson, K. Delord, H. Weimerskirch, P. M. Sagar, E. Sommer, B. J. Dilley, P. G. Ryan, and R. A. Phillips. 2015. Poor transferability of species distribution models for a pelagic predator, the Grey Petrel, indicates contrasting habitat preferences across ocean basins. PLoS ONE 10(3):e0120014. https://doi.org/10.1371/journal.pone.0120014

van Reusel, W., D. Maes, and H. van Dyck. 2007. Transferability of species distribution models: a functional habitat approach for two regionally threatened butterflies. Conservation Biology 21(1):201-212. https://doi.org/10.1111/j.1523-1739.2006.00577.x

Wenger, S. J., and J. D. Olden. 2012. Assessing transferability of ecological models: an underappreciated aspect of statistical validation. Methods in Ecology and Evolution 3:260-267. https://doi.org/10.1111/j.2041-210X.2011.00170.x

Williams, A. P., C. D. Allen, C. I. Millar, T. W. Swetnam, J. Michaelsen, C. J. Still, and S. W. Leavitt. 2010. Forest responses to increasing aridity and warmth in the southwestern United States. Proceedings of the National Academy of Sciences of the United States of America 107(50):21289-21294. https://doi.org/10.1073/pnas.0914211107

Wogan, G. O. U. 2016. Life history traits and niche instability impact accuracy and temporal transferability for historically calibrated distribution models of North American birds. PLoS ONE 11(3):e0151024. https://doi.org/10.1371/journal.pone.0151024

Zuur, A. F., E. N. Ieno, and C. S. Elphick. 2010. A protocol for data exploration to avoid common statistical problems. Methods in Ecology and Evolution 1:3-14. https://doi.org/10.1111/j.2041-210X.2009.00001.x