Skip to main content

Data Nesting

Manipulating arbitrary data

Introduction

The nest object provides a mechanism to define a list of operations that can be used to arrange an array of data objects into a nested hierarchy that maps to the chart data hierarchy of series, x values, and y values.

The goal of this utility is to simplify the process of arranging any arbitrary data sets into chart series and points. It can be enough to look at the code examples below to quickly comprehend how the utility can be used.

The operations are divided into 3 groups: nest, aggregate, and generate. The nesting key() function groups entries by key values and includes date grouping capabilities. At least one key() call must be used to establish a hierarchy or to specify a property that maps to the resulting data point x value. Aggregation specifies values that map to point properties from the nested entries and provides additional output mapping customization options. Only one of each of the aggregation functions rollup() and pointRollup() can be used within a nest call chain. The final tail end calls are used to pass the data for processing to generate the final output. The different tail end functions series(), points(), and entries() determine how the data is formatted.

Basic Overview

Nesting

The key() function call identifies a data field of interest. It can be used once to map to point x value which results in a single series. Or it can be used twice to establish a series grouping property and point x values.

JSC.nest()
  .key("seriesName")
  .key("xField")

key() can also take a function as an argument that gets a key value for a given data entry.

A property path can be used with the key() and rollup() function as well. For example

JSC.nest()
  .key("name.last")
  .key("xField")

Aggregation

The most basic aggregation operation defines a data property that maps to point y values using the rollup() call. Combining a key() call with a rollup() call results in mapping the data entries into a list of [x,y] points. It is similar to using array.map() when key values don't repeat, however, the key() call merges entries with matching key values into a grouping and rollup combines the grouped y values with a calculation such as 'sum' by default. A different calculation can be specified as a rollup argument.

JSC.nest()
  .key("seriesName")
  .key("xField")
  .rollup("yField", "mean")

For more customization control, the pointRollup() function provides a way to manually map a grouping of entries into point objects given a key value and the entries that are part of that grouping.

Generation

Finally, the data can be passed to the nest call chain to be processed into chart series (series()), points (points()), or the raw data using entries()

JSC.nest()
  .key("seriesName")
  .key("xField")
  .rollup("yField", "mean")
  .series(dataArray);

A basic example that maps data properties to points.

See the Pen Nest Property Mapping (@jsblog) on CodePen.

Date Grouping

The key function can also accept on object with properties key, pattern, and range. Calendar patterns can represent intervals like day, month, etc. when set with configuration {month:'*'} translating roughly to [every month]. A string setting can be used as a shortcut to these interval configurations. For example setting 'month' is equivalent to {month:'*'}.

A date based key can be used in a nesting call chain with code such as

JSC.nest().key("dateField")

This will only group data based on unique values of dateField properties, but it will not apply date grouping. The following code will group data entries by day.

JSC.nest().key({ key: "dateField", pattern: "day" })

This creates groups for each unique date that exists in the data. However, date grouping can also fill in the gaps with empty elements for a given date range using code such as:

JSC.nest().key({ key: "dateField", pattern: "day", range: ["1/1/2019 0:00", "1/31/2019 23:59:59"] })

If this is used to generate series points, the range could be applied to X axis range instead:

{ xAxis_scale_range: ["1/1/2019 0:00", "1/31/2019 23:59:59"] }

However, if this is used to generate values for microcharts like 'columns' then it will ensure the correct number of values are generated to represent each day in the given date range.

The date range used in this property should be inclusive where values that fall within the range are determined by min <= value <= max. If the range ['1/1/2019', '2/1/2019'] is used, all days in January would fall within the range including any dates at midnight February 1st. When using dates like this, it is recommended to negate 1 millisecond from the end date.

Examples

Multiple Series

Using two key() calls to generate multiple series

See the Pen Nest Multi Series (@jsblog) on CodePen.

Custom pointRollup() example

Using pointRollup() function to generate custom point configurations. In this example, an additional Z value is included in points.

See the Pen Nesting Point Rollup (@jsblog) on CodePen.

Custom pointRollup() Range

A more involved point rollup example

See the Pen Nesting pointRollup() Range (@jsblog) on CodePen.

Count occurences

See the Pen Nest Ratings Summary (@jsblog) on CodePen.

Reuse Nest Objects

Reuse a nest object to generate series mapping different columns to point y values

See the Pen Nesting Multi Tails (@jsblog) on CodePen.

Date Grouping

See the Pen XyrGyG (@jsblog) on CodePen.

Point Microcharts Metadata

Using nest inside a pointRollup function to apply date grouping on metadata that is shown in microcharts.

See the Pen Nesting Microcharts (@jsblog) on CodePen.

Another trick this example demonstrates is using pointRollup() to create a list of numbers. This approach can be used do process data and generate any type of output.