Superset Upgrade Playbook: Moving a Production Instance from 3.x to 4.x Without Downtime

Phase 1: Read

Before any environment is touched.

`UPDATING.md`. The Superset repository keeps a running file of breaking changes by version. Next, read every entry between your current version and your target. Then copy each one into a checklist with a column for “affects us” and “action”. For 3.x to 4.0 the entries most deployments hit:

• Filter box removed. The legacy filter box chart type is gone. Dashboards that used it need native filters. Superset provides a migration, but it is a data change, so it is rehearsed like everything else.
• Legacy chart migrations completed. Several older chart types (the legacy area, bar, line and others) were migrated to their ECharts replacements and the originals removed. Saved charts of those types must be migrated with the superset viz-migrations command.
• Talisman on by default. Flask-Talisman and its content security policy are enabled. Anything that loads external resources (map tiles, fonts, scripts) or embeds Superset in an iframe needs TALISMAN_CONFIG entries. If you use embedded dashboards, frame-ancestors is now mandatory.
• Python version. 4.x requires Python 3.9 or later (3.10 or 3.11 recommended at the time of 4.0). A custom image on an older base needs a new base.
• Feature flag defaults. Several flags flipped, notably around generic chart axes, cross-filters and native filters becoming default-on. Diff superset/config.py between the two tags for the DEFAULT_FEATURE_FLAGS dictionary and note every change.
• Removed configuration keys. Deprecated settings are removed at major versions. Superset logs a warning for unknown keys at startup; your rehearsal will surface them.

What your own environment adds

Next, your own inventory. From the metadata database: count of charts by viz_type, dashboards using filter boxes, custom plugins and their peer dependency pins, custom security manager or other Python extensions, CSS templates (they target internal class names), and alerts and reports (the report worker’s browser dependency may change).

Finally, dependencies. Database drivers pinned in your image, the Celery and Redis versions, the reverse proxy configuration for websocket support if you use it.

At the end of Phase 1 you have a checklist of actions and an estimate. Therefore, if the checklist has surprises, this is the cheap moment to find them.

Phase 2: Rehearse

Never run the first upgrade against production. Instead, rehearse against a copy of the production metadata database, because the metadata is where the risk is.

• Snapshot production metadata. pg_dump the metadata database (or your provider’s snapshot). Restore it into a staging database.
• Stand up staging on the current version against that copy, with the same superset_config.py and the same Redis topology (a fresh Redis is fine). Confirm it works: log in, open the top ten dashboards.
• Build the target image. Same Dockerfile, new base tag, plugins rebuilt against the new @superset-ui versions (see the plugin compatibility guide), updated superset_config.py with the Phase 1 actions applied.
• Run the schema migration against the staging copy:

superset db upgrade
superset init

Timing the rehearsal

Next, time it. On a metadata database with years of logs the logs table can make migrations slow; if it is over a few minutes, plan to prune old logs (reports.prune_log in Celery, or a direct delete of logs rows older than your retention period) before the production run.

• Run the visualisation migrations for the chart types listed in Phase 1:

superset viz-migrations upgrade --viz_type area
superset viz-migrations upgrade --viz_type bar
# repeat per type named in UPDATING.md for your versions

Each rewrites saved charts’ form data to the new chart type and keeps a record so it can be downgraded with the matching downgrade command if needed before cutover.

• Migrate filter boxes to native filters using the provided migration, then open every affected dashboard and check the filters do what they did.
• Start the target image against the migrated staging database. Read the startup log for unknown configuration keys and deprecation warnings. Fix them.
• Verify using the checklist in Phase 4. Fix what fails, rebuild, and repeat from step 1 with a fresh copy until the rehearsal is clean end to end.

The rehearsal produces the exact command sequence and timing for production, and a list of dashboards and charts that needed attention. So, keep both.

Phase 3: Cut over

The migration constraint means true zero-downtime blue-green, where old and new run side by side against the same database, is not safe across a major version: the old version may fail or corrupt data on the new schema. What is achievable is a short read-only window measured in minutes, with a tested rollback.

The night before

• Announce the window. Ten to twenty minutes is realistic for a rehearsed upgrade.
• Confirm backups are running and a restore has been tested recently.
• Pre-pull the target image on every node so the deploy is a restart, not a download.
• Prune old logs rows if the rehearsal showed the migration was slow.

The sequence

• Freeze writes. Put the web tier into maintenance (a static page at the load balancer, or scale web replicas to zero). Stop Celery beat so no scheduled reports or warm-ups run mid-migration. Let in-flight Celery tasks finish, then stop workers.
• Snapshot the metadata database. This is the rollback point. Do not skip it because you have nightly backups; you want the state from thirty seconds ago.
• Run the migration with the rehearsed commands from a one-off container on the target image: superset db upgrade, superset init, the viz-migrations commands, the filter box migration. Compare the timing to rehearsal.
• Flush the caches you configured: metadata cache certainly, data cache preferably. Cached metadata from the old version can render incorrectly on the new one.
• Deploy the target image to workers first, then beat, then web. Health check /health on each web replica.
• Smoke test while still in maintenance: log in, open two dashboards, run one SQL Lab query, load one embedded dashboard if you have them.
• Open traffic.

Overall, elapsed time for a rehearsed upgrade on a moderate metadata database: five to fifteen minutes, dominated by the migration.

Rollback

However, if the smoke test fails and the cause is not obvious within a few minutes:

• Stop the target web, beat and workers.
• Restore the metadata database from the step 2 snapshot.
• Redeploy the previous image to workers, beat and web.
• Flush caches, smoke test, open traffic.

Also, rehearse the rollback at least once in staging. A rollback you have never run is a plan, not a procedure.

Phase 4: Verify

Do this in staging during rehearsal and again in production after cutover. Also, automate as much as you can, because you will do it again next year.

• Authentication. SSO login, role assignment, an admin action, a viewer’s restricted view. Custom security managers are a common casualty.
• Top dashboards. Every dashboard in the top twenty by views over the last month, opened and visually checked. A script that requests each dashboard’s chart data endpoints and asserts 200 with rows catches most breakage without eyes.
• Migrated charts. Every chart touched by viz-migrations, opened in Explore. Colours, stacking and axis formatting are what typically drift.
• Filters. Every dashboard that had a filter box, with each filter exercised.
• Custom plugins. Every saved chart using each plugin, plus one new chart created from scratch.
• SQL Lab. A query on each database connection, one async.
• Alerts and reports. Trigger one report manually and check the screenshot renders with the new Talisman policy.
• Embedded dashboards. Load one from the embedding application; the Talisman frame-ancestors change is the number one post-upgrade embedding failure.
• Performance. Compare the dashboard load times to the numbers you noted before. New defaults occasionally disable a cache you relied on.
• Logs. Tail the web and worker logs for the first hour for tracebacks and deprecation warnings.

Between major versions

Minor and patch upgrades (4.0 to 4.1, 4.1.1 to 4.1.2) use the same playbook with less reading. The migration step is still one-way, so the snapshot before migration is still mandatory, but the rehearsal can be lighter and the window shorter. So, do them regularly; the longer the gap between upgrades, the more entries in UPDATING.md and the more painful the jump. Overall, quarterly is a good cadence, with security patches applied as they land.

Common failures, from experience

Before the next one

Two habits make the next upgrade cheaper. First, keep your superset_config.py free of anything not deliberately set, so the diff against defaults stays small. Second, keep a UPGRADE.md in your deployment repository with the exact commands, timings and gotchas from this run; the person doing the next upgrade will thank you, and it is often you.

Frequently Asked Questions

What breaks when upgrading Apache Superset from 3.x to 4.x?

In short, the entries most deployments hit are the removal of the legacy filter box, the completed legacy chart migrations (older area, bar and line types must be migrated with superset viz-migrations), Flask-Talisman and its content security policy being on by default, the Python 3.9-or-later requirement, several feature-flag defaults flipping, and removed configuration keys. Read every UPDATING.md entry between your current version and your target, and turn each into a checklist row with “affects us” and “action” columns.

Can a Superset major-version upgrade really be done without downtime?

For read traffic, yes, with a rehearsal against a restored copy of production, a blue-green or rolling cut-over, and a tested rollback. However, the part that is not free is the database migration: schema changes on a large metadata database take time, which is why the rehearsal exists, to measure that time before the change window rather than during it.

What is the most common post-upgrade failure?

Embedded dashboards going blank, because Talisman is on by default in 4.x and frame-ancestors is now mandatory for any origin that frames Superset. Similarly, custom security managers are the other frequent casualty, so SSO login and role assignment are the first things on the verification list rather than the last.

What should be verified after the cut-over?

Authentication end to end, every dashboard in the top twenty by views over the last month, every chart touched by viz-migrations, every dashboard that previously used a filter box, every saved chart from each custom plugin plus one new one, a query per database connection including an async one, one manually triggered report, one embedded dashboard loaded from the embedding application, and a load-time comparison against the numbers you recorded beforehand. Then tail the web and worker logs for the first hour.

Build It With Andolasoft

Upgrade readiness is one of the six areas in our Superset Architecture Review: the version gap, the UPDATING.md entries that apply to your deployment, the plugins and extensions that need rebuilding, and the state of your metadata database. In addition, it is a fixed-scope five-day review with a written report you can run the upgrade from. Under Managed Support we run the upgrades on a schedule, including the rehearsal, so the playbook above is something you read once rather than execute. The production configuration the upgrade has to preserve is covered in Superset in production, and the plugin side in what breaks between 3.x and 4.x for plugins.

If you have an upgrade on the calendar and a deployment people depend on, talk to us. We will tell you what in your specific instance breaks, how long the migration takes on your metadata database, and what the rollback looks like.