Alex Open Research Wiki

BTS: Building Timeseries Dataset

7 min read

  • Important
  • Skimmed

BTS: Building Timeseries Dataset

Source

Status And Credibility

BTS was accepted as a poster in the NeurIPS 2024 Datasets and Benchmarks Track. The official NeurIPS proceedings, poster page, and OpenReview submission identify the same title and authors. The full raw dataset was released on Figshare after the competition; the current canonical snapshot is version 3, posted 2025-04-03.

This is a credible dataset release rather than a code-only proposal: it has a peer-reviewed dataset paper, versioned data DOIs, a data card, raw real-world telemetry, a semantic schema, benchmark code, a concluded public challenge, a permanent competition archive, and published winning solutions.

Core Claim

BTS is interesting because it pairs large irregular real-building telemetry with machine-readable channel and topology context. It contains 14,547 time series and about 2.864 billion timestamp-value observations from three anonymized non-residential Australian buildings, plus Brick 1.2.1 graphs that describe equipment, locations, sensors, setpoints, commands, status points, alarms, parameters, and relationships.

That combination makes BTS more than another forecasting CSV. It is a public substrate for studying channel identity, graph context, cross-building schema drift, long-tailed labels, heterogeneous sampling, and interoperability.

Dataset Shape

PropertyAudited snapshot
Buildings3 anonymized non-residential buildings in Australia
Time coverage2021-01-01 through 2024-01-18, with site-specific end dates
Time series14,547 total; 8,349 / 851 / 5,347 by site in the paper
Observations2,863,795,583 timestamp-value pairs in the repository data card
Current full releaseFigshare v3, 17.68 GB landing-page download size
Raw structurePer-stream pickle files plus per-site metadata CSVs
Semantic structurePer-site Brick 1.2.1 RDF/Turtle graphs linked by StreamID
SamplingIrregular and asynchronous across streams; benchmark pipelines resample

The data were intentionally not cleaned so realistic missingness, errors, flat segments, zeros, irregular rates, and mixed distributions remain part of the research problem.

The Infrastructure Is Part Of The Value

flowchart LR
  BMS[Building management systems] -->|MQTTS telemetry| DCH[CSIRO Data Clearing House]
  DCH --> RAW[Irregular numeric streams]
  DCH --> BRICK[Brick 1.2.1 semantic graph]
  RAW --> FIG[Versioned Figshare releases]
  BRICK --> FIG
  FIG --> BENCH[Classification and forecasting benchmarks]
  BENCH --> BBB[Brick by Brick challenge]
  BBB --> WIN[Leaderboards, winner code, WWW 2025 papers]
  DCH -. same project ecosystem, separate data .-> FLEX[FlexTrack demand-response challenge]

The useful surrounding stack is:

  1. Collection and semantic modeling. Building-management-system streams enter CSIRO’s Data Clearing House over MQTTS; expert engineers build Brick semantic models that connect points to equipment and locations.
  2. Versioned public artifacts. Figshare hosts the full raw release and a separate permanent archive of the competition data.
  3. Exploration surface. The GitHub repository contains a data card, a 50-stream snippet, Brick files, and inspection notebooks.
  4. Benchmark surface. The repository carries preprocessing code and modified Time-Series-Library copies for ontology classification and cross-building forecasting.
  5. Challenge surface. Brick by Brick adds hidden test data, a formal evaluation script, a leaderboard, published winning methods, and permanent artifacts.
  6. Follow-on ecosystem. FlexTrack moves toward demand-response event detection and capacity estimation, but uses separate digital-twin-generated 15-minute data and MUST NOT be described as another BTS split.

Benchmark Notes

Hierarchical ontology classification

The paper cuts streams into 2-, 4-, or 8-week chunks and predicts a hierarchical Brick label vector. The true class and its superclasses are positive; subclasses are masked; unrelated classes are negative. The development/test partition changes both time and building, making this a transfer and interoperability test rather than an ordinary random split.

The paper’s tuned Transformer reaches F1 0.3615 and mAP 0.3489, while simple baselines expose extreme imbalance: the always-negative baseline has accuracy 0.8484 but F1 0. The concluded competition reports a higher winning F1, but it uses a changed partition, 94-class label surface, and separate evaluation protocol, so those scores are not directly comparable.

Cross-building zero-shot forecasting

The appendix benchmark uses one month of data, resamples to 10-minute intervals, and uses a 2-hour context and 2-hour horizon. A model trained on one building is evaluated on the others. The tested DLinear path is multivariate, while PatchTST, Informer, and iTransformer are applied channel-wise because their tested interfaces cannot absorb a different number of building channels.

This is useful baseline evidence but not yet a decisive native-multivariate benchmark: the protocol removes raw irregularity, uses only a one-month slice, and gives different model families different channel interfaces.

Why It Is Interesting For Foundation Time-Series Models

BTS can test whether a model treats channels as typed parts of a system rather than anonymous array positions. A particularly useful experiment would compare:

numeric history only
vs. numeric history + Brick class text
vs. numeric history + Brick graph context

under cross-building ontology classification, forecasting, missing-channel transfer, and rare-class evaluation. The strongest result would preserve irregular timing and use the graph to bind channels across buildings with different inventories.

The dataset is also a bridge between passive telemetry and control. Brick exposes Command, Setpoint, and Status point families, but class names alone do not prove that a stream is a clean action, control input, or intervention. An action-conditioned extension would need aligned command issuance, target, requested value, execution receipt, failure/partial-execution status, NOOP, downstream observation window, and an outcome or reward contract.

Repository Engineering Assessment

The official repository is a rich archive but a weak maintained software package:

  • dated notebooks and scripts are the main entry points;
  • modified Time-Series-Library copies are committed directly;
  • the visible requirements pin old dependencies, including PyTorch 1.7.1;
  • generated output, compiled bytecode, slides, posters, competition archives, and winner artifacts are committed;
  • the audited tree has no release tags, GitHub Releases, CI workflow, test suite, package metadata, or dataset checksum manifest.

A reproducible study SHOULD pin the Git commit, Figshare version, split, preprocessing path, ontology version, environment, and metric implementation. Treating main as an installable or stable benchmark package would overstate the infrastructure maturity.

Gotchas

  • Class-count drift: the paper says 240 unique classes; the repository data card says 215; the Brick by Brick competition uses 94 modified Brick point subclasses. These are not interchangeable counts.
  • Size drift: the paper/data-card snapshot says 18.77 GB, while the current Figshare v3 page says 17.68 GB. Artifact versioning or accounting likely explains the difference, but the public metadata inspected here does not prove one reconciliation.
  • Historical access text: the README still contains June 2024 language saying zero-shot data were unavailable; the full raw release superseded this in April 2025.
  • License layers: Figshare data are CC BY 4.0; repository code is MIT; competition starter assets and winning submissions have artifact-specific terms.
  • Privacy: UUID anonymization and hidden building identity reduce risk but do not eliminate re-identification through external data.
  • Deployment scope: three non-residential Australian buildings are not enough to support direct deployment claims for other climates, regulations, or building types.
  • Pickle safety: official raw streams use Python pickle; load only trusted versioned artifacts.

Foundation TSFM Relevance

Agenda slotVerdictEvidenceMissing pieces
Context interfacepartially closesBrick classes and RDF relationships provide machine-readable channel, equipment, location, and topology context linked to telemetry.Needs matched context ablations and a cross-building typed channel interface used by the model rather than only stored beside it.
Native multivariate encoding and high-channel scalingpartially closesThe raw sites expose hundreds to thousands of heterogeneous asynchronous streams and building-specific channel inventories.Published Transformer forecasting baselines become univariate under channel-count drift; no full graph-conditioned high-channel benchmark is established.
Time representation and irregular event streamspartially closesRaw streams have explicit irregular timestamps and different sampling rates.Published baselines mostly resample to fixed grids; missingness and sampling process are not first-class targets.
Data diversity, curriculum, and long tailpartially closesReal errors, long-tailed values, rare Brick classes, cross-building domain shift, and severe class imbalance are preserved.Only three buildings; needs rare-class/state protocols that separate corruption from meaningful tail behavior.
Benchmarks: what level of modeling is tested?partially closesProvides cross-building ontology classification, zero-shot forecasting, and a concluded hidden-test competition.Needs a unified versioned harness and protocol separation across paper, full release, and competition artifacts.
Control and counterfactualsinsufficient evidenceCommand, setpoint, and status streams are control-input-like semantic hints.No validated typed actions, execution receipts, rewards, NOOP, counterfactual pairs, or closed-loop simulator.

Graph View

Backlinks