Introduction
While the “Getting started” vignette covers the standard workflow
(obs_read() → obs_summary() /
obs_plot() / obs_map()), advanced analytical
workflows often require deeper integration with external tools.
This vignette demonstrates how to use obsR for: 1.
Optimising memory usage with large datasets. 2. Integrating observation
data with custom GIS layers. 3. Deriving ecological indicators and
spatial grids. 4. Preparing data for occupancy modelling.
The output of obs_read() is a standard tibble (class
obs_df). From that point onwards, you can leverage the full
power of the R ecosystem.
1. Optimising Memory for Large Datasets
When working with large SQLite exports, it is crucial to manage
memory efficiently. You can use standard dplyr::select() to
retain only the columns necessary for your specific analysis.
slim_obs <- obs %>%
select(id, date, scientific_name, species_code, validation_code, lat, lng)
names(slim_obs)
#> [1] "id" "date" "scientific_name" "species_code"
#> [5] "validation_code" "lat" "lng"Performance Note: If the obs_df is
still a lazy SQLite query, select() pushes
the column reduction down to the database level. It only shrinks the
data footprint when the table is finally brought into R memory via
obs_collect(). For datasets with millions of rows, always
start from the native .sqlite export rather than CSV or
Excel.
2. Integrating with Custom GIS Workflows
obsR provides basic mapping functions, but for advanced
spatial analysis, you will likely want to use your own GIS stack.
Observation coordinates are provided in WGS84 (lng,
lat; EPSG:4326).
You can easily convert the obs_df into an
sf object and join it with your own spatial layers (e.g.,
protected areas, municipalities, or custom study plots). Note that
obsR does not perform geocoding; you must provide the
spatial boundaries.
# Convert to sf object
obs_sf <- sf::st_as_sf(obs, coords = c("lng", "lat"), crs = 4326, remove = FALSE)
# Define a custom polygon (e.g., a study area)
park <- sf::st_as_sfc(
"POLYGON ((-6.1 37.3, -5.8 37.3, -5.8 37.5, -6.1 37.5, -6.1 37.3))",
crs = 4326
)
# Filter observations intersecting the polygon
inside_park <- obs_sf[lengths(sf::st_intersects(obs_sf, park)) > 0, ]
nrow(inside_park)
#> [1] 438Tip: If your external GIS layer uses a different Coordinate
Reference System (CRS), ensure you reproject it to EPSG:4326 (or vice
versa) before using st_join() or
st_intersects(). For more details, see
vignette("spatial-filters").
3. Deriving Ecological Indicators and Grids
Raw observation counts are not direct measures of abundance, but they are highly valuable as proxies for effort and baseline metrics (e.g., number of records, species richness, observer participation).
# Basic annual summary metrics
obs %>%
mutate(year = as.integer(format(date, "%Y"))) %>%
summarise(
n_observations = n(),
n_species = n_distinct(scientific_name, na.rm = TRUE),
n_observers = n_distinct(observer_code, na.rm = TRUE),
active_years = n_distinct(year)
)
#> # A tibble: 1 × 4
#> n_observations n_species n_observers active_years
#> <int> <int> <int> <int>
#> 1 1455 3 61 2For spatial analysis or data sharing, aggregating points into a grid
is a best practice. The obs_grid() function generates a
regular grid of observation counts, which serves as the backend for
obs_map_density().
# Create a 0.05-degree resolution grid
grid_data <- obs_grid(obs, resolution = 0.05)
nrow(grid_data)
#> [1] 149
sum(grid_data$n) # Total observations in the grid
#> [1] 1455Privacy Benefit: Sharing the aggregated
grid_data table is an excellent way to comply with FAIR
data principles while protecting the exact coordinates of sensitive
species or observer locations.
4. Preparing Data for Occupancy Modelling
A critical methodological requirement for occupancy modelling is the ability to distinguish between true non-detections and lack of sampling.
Extracting data for a single protected species is insufficient for this purpose. To reconstruct valid (pseudo)visits, you need to know if an observer was present in a specific location and time, and actively recorded other species within the same taxonomic group. As defined by Fajgenblat et al. (2025), a valid visit requires knowing what was reported (and thus, what was not reported) by that observer in that grid cell on that day.
Recommendation: Always request a group-wide export (e.g., all birds, all odonates) for your study area from Observation.org, rather than a species-level subset.
The bundled Seville sample (containing only two sparrow species) can
illustrate the dplyr data shaping required, but it cannot
represent a real occupancy analysis.
# Illustrative data shaping for pseudo-visits (requires group-wide data in practice)
grid_obs <- obs %>%
filter(!is.na(lat), !is.na(lng)) %>%
mutate(
lng_grid = round(lng / 0.05) * 0.05,
lat_grid = round(lat / 0.05) * 0.05
)
# Define unique visits (observer + date + location)
visits <- grid_obs %>%
distinct(observer_code, date, lng_grid, lat_grid)
# Define detections (adding the species dimension)
detections <- grid_obs %>%
distinct(observer_code, date, lng_grid, lat_grid, scientific_name)
nrow(visits)
#> [1] 851
nrow(detections)
#> [1] 920From a comprehensive, group-wide export, this structured visit table
serves as the direct input for occupancy modelling software (e.g.,
unmarked, spOccupancy) or joint species
distribution models (JSDMs).
Note: obsR is designed to prepare and structure
this data; it does not fit the statistical models themselves.
Reference
Fajgenblat, M., Wijns, R., De Knijf, G., Stoks, R., Lemmens, P., Herremans, M., Vanormelingen, P., Neyens, T., & De Meester, L. (2025). Leveraging massive opportunistically collected datasets to study species communities in space and time. Ecology Letters, 28(3), e70094. https://doi.org/10.1111/ele.70094