Tag: ECharts

Where the time goes

A chart render in Superset has five stages. Each can be the bottleneck, and the fix for each is different.

Measure before you change anything. Three tools give you the whole picture:

• Superset’s own timing. Open the chart in Explore, use “View query” and “View as table” to see row counts, and check the network tab for the /api/v1/chart/data response time and size.
• Chrome Performance panel. Record a dashboard load and look at the flame chart. Long yellow blocks under your plugin’s function names are transform time; long purple and green blocks are layout and paint.
• React Profiler. Shows how many times your component rendered and why. A chart that renders four times per load is doing four times the work.

Write down the numbers. Ninety per cent of the time one stage dominates, and it is usually transform or render.

Fix 1: Do not fetch what you will not draw

A chart 900 pixels wide cannot display 100,000 distinct points. If the plugin fetches them anyway, every downstream stage pays for it. The right place to fix this is buildQuery, because it is the only stage where the database does the work.

Aggregate. If the chart shows a value per category, group by the category. If it shows a time series, apply a time grain. buildQueryContext does this from the standard controls; make sure the plugin’s controls expose them rather than pulling raw rows.

Limit series, not just rows. For multi-series charts, series_limit and series_limit_metric keep the top N series and drop the tail that nobody can read anyway.

Set a sensible `row_limit`. The default row limit is often far higher than any chart needs. A row limit control with a default matched to the chart’s capacity protects users from themselves.

Use server-side post-processing. Superset’s query object accepts a post_processing list of pandas operations that run on the backend after the query: pivot, aggregate, rolling, resample, cum, contribution, sort, rename, flatten. Rolling averages, pivots for a heatmap, and percentage-of-total calculations belong here, not in transformProps:

return buildQueryContext(formData, baseQueryObject => [
  {
    ...baseQueryObject,
    post_processing: [
      {
        operation: 'pivot',
        options: {
          index: ['__timestamp'],
          columns: formData.groupby,
          aggregates: { [metricLabel]: { operator: 'mean' } },
        },
      },
      {
        operation: 'rolling',
        options: { rolling_type: 'mean', window: 7, min_periods: 1, columns: { [metricLabel]: metricLabel } },
      },
    ],
  },
]);

The payload that reaches the browser is already in the shape the chart needs. This one change often removes the transform stage entirely.

Fix 2: Keep transformProps cheap and pure

transformProps runs on every render trigger, which means every time the user touches a cosmetic control. Anything expensive in it is paid repeatedly.

• No per-row string formatting. Format values at draw time for the visible elements, or in the tooltip, not for every row up front.
• No sorting of large arrays if the query can return them sorted (orderby in buildQuery).
• No nested loops. Building a lookup map once and indexing into it is O(n); the nested find is O(n²) and 100,000 rows makes that ten billion operations.
• Return plain data, not React elements. Keep transformProps framework-free so it is cheap to test and impossible to accidentally allocate components in.

A useful guardrail is a unit test that runs transformProps on a 100,000-row fixture and asserts it completes under a threshold. It fails the moment someone adds a quadratic step.

Fix 3: Choose the rendering technology for the data size

This is the change people reach for first, and it is the third most effective, because the first two usually make it unnecessary. When it is necessary, the guidance is simple.

SVG is fine up to a few thousand elements. Each element is a DOM node with layout, style and event handling. Past roughly 5,000 to 10,000 nodes the page becomes sluggish regardless of how well the code is written.

Canvas draws pixels, not nodes, and handles hundreds of thousands of points. ECharts, which Superset’s own charts use, renders to canvas by default. If your plugin wraps ECharts, three options matter for large data:

series: [{
  type: 'line',
  sampling: 'lttb',      // downsample to the visible pixel width while preserving shape
  large: true,           // enable large-data mode for bar and scatter
  largeThreshold: 2000,
  progressive: 4000,     // render in chunks so the UI stays responsive
  progressiveThreshold: 10000,
  data,
}]

sampling: 'lttb' (largest triangle three buckets) is the single most valuable one for time series: it reduces a 100,000-point line to the number of pixels available without losing peaks and troughs.

WebGL through deck.gl is for the cases canvas cannot handle: millions of points, geospatial layers, 3D. Superset ships deck.gl plugins, and the patrol analytics guide shows the pattern in a real deployment. The cost is complexity and a harder testing story, so reach for it only when the data justifies it.

Fix 4: Virtualise tables and lists

Table-style plugins are a special case. Users expect to see every row, so aggregation is not an option, but rendering 100,000 table rows as DOM nodes is exactly the SVG problem in a different costume.

The fix is virtualisation: render only the rows in the viewport plus a small buffer, and swap them as the user scrolls. react-window and @tanstack/react-virtual both do this well. Superset’s own table chart also supports server-side pagination, where each page is a separate query; that is the better choice when the dataset is large enough that even transferring it is slow.

Fix 5: Stop re-rendering

The React Profiler often shows a chart rendering several times per dashboard load: once with empty data, once with data, once when the dashboard finishes laying out, once when a filter indicator updates. Each render repeats any work not memoised.

• Wrap expensive derived values in useMemo keyed on the data reference and the relevant props.
• Memoise the component with React.memo if its props are stable.
• For ECharts, call setOption with notMerge: false and update only the series that changed, rather than recreating the instance.
• Make sure transformProps returns the same object references for unchanged data. A new array on every call defeats memoisation downstream.

Fix 6: Make interaction cheap

Tooltips that recompute on every mouse move, hover highlighting that touches every element, and cross-filter emissions that fire on drag are all ways to make a fast chart feel slow.

• Throttle hover handlers to animation frames.
• For canvas charts, use the library’s built-in hit testing rather than your own.
• Emit cross-filters on click or on the end of a brush, not continuously.

Fix 7: Let long queries run asynchronously

When the database stage genuinely is slow, the dashboard should not freeze waiting for it. Superset’s global async queries feature runs chart queries through Celery workers and lets the browser poll or receive the result over a websocket. The dashboard renders immediately with loading states, and charts fill in as their data arrives. This is a deployment change (GLOBAL_ASYNC_QUERIES feature flag, Redis, Celery workers) rather than a plugin change, but plugin authors should test that their chart handles the loading and error states it produces.

Async queries do not make a slow query fast. They make a slow query stop blocking everything else. Pair them with caching and query tuning, which is a platform question covered in our production operations guide and in the Architecture Review.

A worked order of operations

Faced with a plugin that takes nine seconds on 100,000 rows:

• Measure. Say the query is 0.8 s, transfer 0.6 s, transform 3.1 s, render 4.2 s.
• Move aggregation and the rolling calculation into buildQuery with post-processing. Rows drop to 8,000. Transfer is now 0.1 s, transform 0.3 s, render 1.9 s.
• Switch the SVG line to ECharts canvas with LTTB sampling. Render drops to 0.2 s.
• Add useMemo around the option object. Renders per load drop from four to one.

Total: under two seconds, and most of it is the database. No WebGL, no virtualisation, no heroics. That is the usual shape of these fixes: the first two steps do most of the work.

Frequently Asked Questions

How many rows can a Superset chart plugin handle before it gets slow?

It depends on what you render with, not on Superset. SVG is comfortable up to a few thousand elements, because each one is a DOM node with layout, style and event handling, and becomes sluggish somewhere past five to ten thousand nodes however well the code is written. Canvas draws pixels rather than nodes and handles hundreds of thousands of points; WebGL through deck.gl goes further again, at the cost of complexity. Match the technology to the expected row count before writing the plugin.

Should aggregation happen in the browser or in the query?

In the query, nearly always. Aggregating by category, applying a time grain, setting a realistic row_limit, and limiting the number of series with series_limit all keep rows from being fetched at all. Rolling averages, pivots and percentage-of-total belong in the query object’s post_processing list, which runs on the backend, rather than in transformProps. The fastest transform is the one that has less data to transform.

Why does my chart render several times on every dashboard load?

Usually because a prop identity changes on each render, so memoisation never holds. The React Profiler shows the render count and the reason. A chart that renders four times per load does four times the transform and paint work, which is why re-render control often buys more than micro-optimising the transform itself.

What should transformProps avoid doing?

Per-row string formatting, sorting large arrays that the query could have returned sorted through orderby, nested loops, and allocating React elements. A nested find inside a loop is O(n squared), which at 100,000 rows is ten billion operations; building a lookup map once and indexing into it is O(n). Keep the function plain-data-in, plain-data-out and it stays cheap and testable.

Build It With Andolasoft

Performance is much cheaper to design in than to retrofit. When we scope a plugin, expected row counts and interaction patterns are part of the conversation, and they decide the rendering approach before any code is written. If you have a chart in mind, or one that has started to struggle, our Plugin Scoping Call is free and 45 minutes, and the written estimate you get afterwards will say which of the approaches above the design needs. Where the bottleneck turns out to be the platform rather than the plugin, that is the territory of a Superset Architecture Review.

If a dashboard in front of real users is freezing and you need to know whether the fix is the plugin, the query or the deployment, talk to us. We will profile a real load and tell you which of the three it is before anyone commits engineering time.