Skip to contents

The core function for downloading statistical data. Variable selections are passed as named arguments via ..., or via a prepared query object from prepare_query().

Usage

px_get_data(...)

get_data(
  api,
  table_id,
  ...,
  query = NULL,
  .codelist = NULL,
  .output = "long",
  .comments = FALSE,
  simplify = TRUE,
  auto_chunk = TRUE,
  max_results = NULL,
  cache = FALSE,
  cache_location = NULL,
  verbose = FALSE,
  .status = FALSE
)

Arguments

...

Variable selections as named arguments. Each name is a variable code, each value is one of:

Only variables whose metadata reports elimination = TRUE may be omitted on a v2 API. Omitting a mandatory one (including ContentsCode and the time variable) is an error naming the variables to add — the server's own HTTP 400 does not say which variable is missing. Use get_variables() to see the flag, or let prepare_query() fill the mandatory variables for you. Ignored when query is provided.

api

A <px_api> object.

table_id

A single table ID (character). Vectors are not supported; use purrr::map() + dplyr::bind_rows() for multiple tables. Ignored when query is provided.

query

A <px_query> object from prepare_query(). When provided, table_id, ..., and .codelist are taken from the query object.

.codelist

Named list of codelist overrides (e.g. list(Region = "agg_RegionLan")).

.output

"long" (default, tidy) or "wide" (pivot content variables).

.comments

Include footnotes/comments as an attribute.

simplify

Add human-readable text label columns alongside codes.

auto_chunk

Automatically split large queries that exceed the cell limit into multiple requests. When TRUE (default), the variable with the most values is split into batches, each request staying under the limit. A progress bar is shown. Set to FALSE to error instead.

max_results

Override the API's cell limit. When set, this value is used instead of the limit reported by the API's config endpoint. Useful for keeping result size manageable or for testing chunking behavior.

cache

Logical. If TRUE and cache_location points at a SQLite file (or an nxt_handle from nordstatExtras), the data is cached at cell granularity in that database. Supports concurrent multi-process read/write and cross-query cell reuse. Requires nordstatExtras.

cache_location

Either a path to a .sqlite file or an nxt_handle from nordstatExtras::nxt_open(). Ignored unless cache = TRUE.

verbose

Print request details.

.status

Keep the source's status symbol for cells without a value. PX-Web marks a cell it withholds or cannot give with a symbol such as ".." (not available or suppressed) or "." (not applicable); by default such a cell is simply NA in value, indistinguishable from a cell the response does not carry. With .status = TRUE the result gains a character column status, placed after value: the symbol exactly as the source sent it, and NA where the cell has a value (or no symbol). Default FALSE leaves the result unchanged. Only for .output = "long"; the SQLite cell cache is bypassed, since it stores numeric cells only.

Value

A tibble of data. See Details for column structure.

Details

When simplify = TRUE and .output = "long" (defaults), columns are:

  • table_id: back-reference to the source table

  • One pair per dimension: {code} (raw code) + {code}_text (label)

  • value: the numeric measurement

When simplify = FALSE, only raw codes and value are returned.

When .output = "wide", content variables are pivoted into separate columns.

A v1 API returns the values of a contents variable as separate content columns, and they are folded into a column named ContentsCode even when the table metadata calls the variable something else (as in the *_Cont.px tables of Energimyndigheten). When that happens the result carries the name the source uses in attr(x, "px_contents_variable").

When auto_chunk = TRUE and the query would exceed the API's cell limit, get_data() automatically splits the request. It picks the variable with the most values and batches its values so each request fits under the limit. Requests are paced to respect the API's rate limit.

A split fetch is all or nothing. If any of its requests fails (a connection error, an HTTP error), get_data() stops with an error of class pixieweb_chunks_failed that says how many requests failed and which values of the split variable they asked for. The other requests' rows are not returned.

When the API answers that requests come too fast (HTTP 429), a request waits and tries again, five times, for as long as the server asks or for 2, 4, 8, 16 and 32 seconds. If the API still refuses, get_data() (and every other call that reads the API) stops with an error of class pixieweb_rate_limited. It never returns an empty answer for it.

A name that cannot be masked

px_get_data() is get_data() under a name no sister package exports. rTrafa also exports a get_data(), so in a script that attaches both, the package attached last masks the other and its calls fail on the wrong argument check. Use px_get_data() here and rTrafa::trafa_get_data() there, or prefix every call with its package.

Examples

# \donttest{
scb <- px_api("scb", lang = "en")
if (px_available(scb)) {

  # Fetch with explicit selections. ContentsCode and Tid are mandatory
  # in this table (elimination = FALSE); Region is not.
  get_data(scb, "TAB638",
    Region = c("0180", "1480"),
    ContentsCode = "*",
    Tid = px_top(5)
  )

  # Fetch from a prepared query
  q <- prepare_query(scb, "TAB638", Region = c("0180"))
  get_data(scb, query = q)
}# }
#> ── Query: TAB638 ──────────────────────────────────────────────────────────────── 
#>   Estimated cells: 20 / 150000 (0% of limit)
#> 
#>   Region = c("0180")
#>     user override
#>   Civilstand = <eliminated>
#>     eliminated (4 values available)
#>   Alder = <eliminated>
#>     eliminated (102 values available)
#>   Kon = <eliminated>
#>     eliminated (2 values available)
#>   ContentsCode = "*"
#>     all 2 content variable(s)
#>   Tid = px_top(10)
#>     latest 10 of 57 periods
#> # A tibble: 20 × 8
#>    table_id Region Region_text ContentsCode ContentsCode_text Tid   Tid_text
#>    <chr>    <chr>  <chr>       <chr>        <chr>             <chr> <chr>   
#>  1 TAB638   0180   Stockholm   BE0101N1     Population        2015  2015    
#>  2 TAB638   0180   Stockholm   BE0101N1     Population        2016  2016    
#>  3 TAB638   0180   Stockholm   BE0101N1     Population        2017  2017    
#>  4 TAB638   0180   Stockholm   BE0101N1     Population        2018  2018    
#>  5 TAB638   0180   Stockholm   BE0101N1     Population        2019  2019    
#>  6 TAB638   0180   Stockholm   BE0101N1     Population        2020  2020    
#>  7 TAB638   0180   Stockholm   BE0101N1     Population        2021  2021    
#>  8 TAB638   0180   Stockholm   BE0101N1     Population        2022  2022    
#>  9 TAB638   0180   Stockholm   BE0101N1     Population        2023  2023    
#> 10 TAB638   0180   Stockholm   BE0101N1     Population        2024  2024    
#> 11 TAB638   0180   Stockholm   BE0101N2     Population growth 2015  2015    
#> 12 TAB638   0180   Stockholm   BE0101N2     Population growth 2016  2016    
#> 13 TAB638   0180   Stockholm   BE0101N2     Population growth 2017  2017    
#> 14 TAB638   0180   Stockholm   BE0101N2     Population growth 2018  2018    
#> 15 TAB638   0180   Stockholm   BE0101N2     Population growth 2019  2019    
#> 16 TAB638   0180   Stockholm   BE0101N2     Population growth 2020  2020    
#> 17 TAB638   0180   Stockholm   BE0101N2     Population growth 2021  2021    
#> 18 TAB638   0180   Stockholm   BE0101N2     Population growth 2022  2022    
#> 19 TAB638   0180   Stockholm   BE0101N2     Population growth 2023  2023    
#> 20 TAB638   0180   Stockholm   BE0101N2     Population growth 2024  2024    
#> # ℹ 1 more variable: value <dbl>