DataHub Migration to StarRocks and what changes for customers
Eightfold is upgrading the engine behind DataHub from Amazon Redshift/Databricks to StarRocks. This guide explains why we are making the change, how DataHub report emails now look, and what to expect in how some column types are handled after the migration.
At a glance
| Area | What changes | Action needed |
| Query engine | DataHub now runs on StarRocks, a faster engine built for many users at once. No downtime. | None. We migrate your curated and scheduled queries for you. |
| Your data | Same tables, columns and rows as before, checked before and after the switch. | None. |
| Migration notice | A banner in DataHub confirms the switch and links to the StarRocks syntax guide. | None. |
| Report emails | Subjects and bodies now name the query or widget and include the date. Saved and scheduled query emails no longer show SQL text. | Update any inbox rules that match the old subject “Data Hub Report”. |
| Column types | Some column types are handled differently in StarRocks. | Review Section 4 (details to follow). |
| CSV report format | In CSV files delivered by email or SFTP, every value and header is wrapped in double quotes, NULL appears as \N, and booleans appear as true/false. | If another system reads these files, check that it handles quoted values and \N (see Section 4). |
| SQL syntax | New or edited custom queries use StarRocks (MySQL-compatible) SQL. | Use the syntax reference when writing new queries. |
1. Why we are migrating
As data volumes and the number of people using DataHub have grown, our previous warehouse started to limit performance. It could only run a fixed number of queries at once, so reports queued up at peak times. It also needed regular maintenance that caused performance to vary. StarRocks is a modern analytics engine designed for large data volumes and many users running queries at the same time.
What you get
- Faster queries. Typical reporting workloads run with noticeably lower latency, and most of the slowest queries go away. Large scans, joins, aggregations and window functions benefit the most.
- Better performance at peak times. Scheduled reports, exports and interactive queries no longer compete for a small number of query slots, so less waiting during busy reporting hours.
- Fresher data. New data can be queried sooner.
- More consistent performance. Maintenance has less effect on performance, and the platform scales without disruptive resizes.
What stays the same
- The DataHub SQL editor, customer views, saved, scheduled and curated queries.
- Scheduled email and SFTP deliveries. The delivery schedule and SFTP file naming do not change. The CSV file format has some differences (see Section 4).
- Tenant isolation and row-level security. You still query only your own data, enforced the same way as today.
- Download links are still delivered by email and expire after the same period.
2. What to expect during the migration
No downtime or disruption
The switch to StarRocks happens behind the scenes, with no downtime for DataHub or anything connected to it. Before we switch your tenant, we run StarRocks alongside your current warehouse, with new data written to both. When we switch, DataHub simply starts reading from StarRocks. Nothing goes offline, and no refresh cycles or scheduled deliveries are missed. We switch tenants one at a time, so we can move a tenant back quickly if we find an issue.
- Scheduled queries keep running on their usual schedule.
- Email and SFTP deliveries continue as before.
- Saved and Eightfold curated queries stay in your DataHub, ready to run.
- Integrations keep receiving DataHub exports on the same schedule. Check any that parse the files, as the CSV format has some differences (see Section 4).
Your data stays the same
Every table you query in DataHub is available in StarRocks with the same name, columns and data. Before we switch a tenant, we check each table and compare results between the two warehouses:
- Table structure. Column names, column counts and data types are compared against the original tables. Any intentional difference is documented (see Section 4).
- Row counts. Total rows in each table must match exactly. Any difference is investigated and fixed before the tenant is switched.
- Values. Sample rows are compared value by value, including edge cases such as empty and null values.
- Your queries. Your saved and scheduled queries are converted to StarRocks syntax, then run on both warehouses. We compare the number of rows and the column names each query returns.
- Ongoing checks. While both warehouses run side by side, automated checks run every day and fix any differences automatically.
You don’t need to re-check your tables or rebuild your reports. They work the same way on StarRocks.
How you’ll know your DataHub has been migrated
Once your tenant is switched, a “Datahub Migrated” banner appears at the top of the DataHub editor. The banner confirms that DataHub now runs on StarRocks and includes a View syntax guide button that opens the StarRocks syntax guide.
The banner shown in the DataHub editor after migration
The syntax guide lists the differences to keep in mind when you write new queries or edit existing ones. You can also read it any time here: Redshift to StarRocks migration guide.
3. Changes to report emails
Previously every report email used the same generic template: the subject was always “Data Hub Report”, the body showed the full SQL query, and the download was always named data_hub_report000.csv. Customers told us this made reports hard to tell apart, and that showing SQL in email was not ideal. With StarRocks, each type of report gets its own email:
- The subject names the query or widget, so you can identify a report without opening it.
- The send date is added to the subject (DD/MM/YY, in UTC). Most email clients group messages with identical subjects into one long thread. Adding the date gives each day’s run its own thread.
- SQL text is removed from saved and scheduled query emails. The query name is shown instead.
- Downloaded files are named after the report and include a timestamp.
- These updates affect only the email and the file name, not the CSV values. For how StarRocks writes CSV values, including quoting, see Section 4.
Case A: Scheduled DataHub query
Sent every time a scheduled query runs. The email identifies the report by its query name and does not show the SQL.
| Subject: Your requested data for {Query Name} is ready — {DD/MM/YY} |
|
Hi {User Name}, You requested data for the following query: {Query Name} The download link below will expire in 1 day(s) [ Download Report ] [ Data Hub ] Cheers, |
Example subject: Your requested data for Candidate Pipeline Daily is ready — 30/09/26
Case B: Ad hoc query (“Send full report” in DataHub)
Sent when you run a query directly in the DataHub editor and choose to email the full results. A query run this way has no saved name, so the email includes the SQL to show which query the results belong to.
| Subject: Your DataHub query results are ready |
|
Hi {User Name}, You requested data for the following query, run directly from DataHub:
The download link below will expire in 1 day(s) [ Download Report ] [ Data Hub ] Cheers, |
Case C: Native Dashboard widget export (“Export raw data”)
Report delivery from Native Dashboards uses the same email service, so it was updated at the same time. The email names the widget and its dashboard. It includes no SQL and no DataHub link, since the export did not come from DataHub.
| Subject: Your requested data for {Widget Name} is ready — {DD/MM/YY} |
|
Hi {User Name}, You requested data for the following widget: {Widget Name} from the {Dashboard Name} dashboard. [ Download Report ] Cheers, |
Large results
If a result is too large for a single file, it is split into parts. The email shows one link per part (“Download Report part 1”, “Download Report part 2”, and so on), and each file ends in a part number (_000, _001, …).
Downloaded file names
| Report type | File name pattern | Example |
| Scheduled DataHub query | {Query_Name}_{YYYYMMDD-HHMMSS}_000.csv | Candidate_Pipeline_Daily_20260930-041721_000.csv |
| Ad hoc DataHub query | DatahubQueryResults_{YYYYMMDD-HHMMSS}_000.csv | DatahubQueryResults_20260930-041721_000.csv |
| Native Dashboard widget export | {Widget_Name}_{YYYYMMDD-HHMMSS}_000.csv | Employee_Distribution_by_Gender_20260930-041721_000.csv |
Timestamps are in UTC. Special characters in names are replaced so the file name is valid on every operating system, for example “Hires / Attrition (Q3)” becomes “Hires__Attrition_Q3”.
Before and after
| Before | After | |
| Subject | Always “Data Hub Report” | Names the query or widget, plus the date (except ad hoc runs) |
| SQL in body | Always shown | Hidden for scheduled queries; shown for ad hoc runs; not applicable to widget exports |
| DataHub link | Always included | Included for DataHub reports; removed for Native Dashboard exports |
| Email threading | All runs grouped in one thread | One thread per day |
Action: check your inbox rules
If your team has email filters or forwarding rules that match the subject "Data Hub Report", they will stop matching after the change. Update them to match the new subject (for example, "Your requested data for" or the name of the query).
4. Changes to column type handling
StarRocks and Redshift return a few data types in slightly different formats. The underlying data is the same, but some values look different in the DataHub results table (UI) and in scheduled report files (CSV, delivered by email or SFTP). We confirmed every difference below by running the same test query on both warehouses, using sample values only.
The list covers the differences you are most likely to notice, and where you may need to adjust a downstream process that reads DataHub output.
Differences at a glance
| Data type | Where | Before (Redshift) | After (StarRocks) | What to do |
| Empty values (NULL), all types | CSV |
Blank field (JSON columns: null)
|
\N
|
If a downstream system treats blank fields as empty, configure it to treat \N as empty too. |
| Quoting, all values and headers | CSV |
A field is quoted only if it contains a comma, a quote or a line break. An empty string is a blank field.
|
Every value and every header name is wrapped in double quotes, for example "hello", "3", "2024-01-02" and "mixedCaseAlias". NULL stays unquoted as \N. An empty string is "".
|
A standard CSV reader removes the quotes, so text, numbers and dates read the same. Check any process that splits lines on commas or matches the raw text. |
| Boolean | CSV |
t / f
|
true / false
|
Update any process that looks for t or f. |
| Boolean | UI |
true / false
|
1 / 0
|
None. 1 means true and 0 means false. |
| Decimal | CSV |
12.50 (keeps declared decimal places)
|
12.5 (drops trailing zeros)
|
Use ROUND() or CAST in the query if you need a fixed number of decimal places. |
| Floating-point numbers (double, real) | CSV |
0.333333333333333311.0000000000000001e-051.1000000210000000000000000
|
0.33333333333333331e-051.11e+16
|
Same values, shorter form. Very large values use scientific notation. Use ROUND() or CAST if you need a fixed format. |
| Floating-point results of calculations | UI |
0.30.333333333333333
|
0.300000000000000040.3333333333333333
|
StarRocks shows the exact stored value. Use ROUND() to display fewer digits. |
| Timestamp with fractional seconds | CSV |
2024-01-02 03:04:05.12
|
2024-01-02 03:04:05.120000
|
Always six digits after the decimal point. Timestamps in whole seconds are unchanged. |
| JSON | UI and CSV |
{"a":1,"b":[1,2]}
|
{"a": 1, "b": [1, 2]}
|
A space is added after each colon and comma. Use a JSON parser rather than an exact text match. |
| Column names | UI and CSV header |
mixedcasealias (always lower case)
|
mixedCaseAlias (case kept as written; in the CSV header it is also wrapped in double quotes: "mixedCaseAlias")
|
If a downstream process matches column names, write aliases in lower case or update the process. |
For a detailed table-level comparison across all tables, download the DataHub StarRocks column type changes (ZIP). It has one CSV file per table.
Note: The link downloads a ZIP file to your computer. Open it to get a folder with all the files, one CSV file for each table.
What stays the same
Text values, whole numbers (smallint, integer, bigint), dates, timestamps in whole seconds, and simple decimal values such as 3.5, 15 and 4.25 have the same values on both warehouses. In the CSV file, StarRocks wraps them in double quotes, which a standard CSV reader removes. In the UI, empty values still show as null.
Tip: keep number formats stable
If a report feeds another system that expects a fixed format, set it in the query, for example ROUND(score, 2) for numbers or CASE WHEN flag THEN 'Yes' ELSE 'No' END for booleans. The output then looks the same whichever warehouse runs it.
Frequently asked questions
Do I need to do anything for the migration?
No. There is no downtime, and we migrate and validate your tables and queries before we switch you over. The only suggested action is updating inbox rules that match the old email subject.
Will the data in my CSV exports change?
The email changes do not affect CSV values. On StarRocks, scheduled report CSV files wrap every value and header in double quotes, and some column types are formatted differently; see Section 4.
Do I need to rewrite my SQL?
Existing scheduled queries are migrated for you. New or edited custom queries should use StarRocks (MySQL-compatible) syntax. The migration guide lists the common differences.
How will I know my DataHub has moved to StarRocks?
A “Datahub Migrated” banner appears in the DataHub editor, with a link to the StarRocks syntax guide. Your Eightfold team will also confirm by email.
Who can I contact with questions?
Contact your Eightfold account team or raise a support ticket.