What I did
Sirius is a GPU-native SQL engine. It is designed as a drop-in acceleration layer for systems like DuckDB, keeping analytical execution on the GPU where thousands of threads and high memory bandwidth can speed up joins and aggregations. That speedup only appears when the query exposes enough parallel work to outweigh the fixed costs of GPU execution: kernel launches, data conversion, and moving data between host and device.
I evaluated Sirius on the Join Order Benchmark (JOB) and worked on the code paths that JOB broke. JOB is a suite of 113 multi-table join queries over a real IMDB dataset: 2.5 GB, 74,190,187 rows, and 21 tables. It is a more demanding test than a handful of hand-picked queries because it combines large probe sides, selective filters, variable-length strings, and aggregates in many different shapes.
Two things came out of the work. The first was a benchmark comparison of Sirius against DuckDB across the suite. The second was two fixes that I contributed upstream and that were merged: safe handling for empty row-ID conversions (PR #115), and support for materializing string aggregate results (PR #124). The goal was not only to make Sirius fast on JOB. It was to make the entire benchmark run correctly.
The test bed
I ran the comparison on a Chameleon testbed node with an Intel Xeon Gold 6126 CPU and an NVIDIA Quadro RTX 6000 GPU. Sirius was given an 8 GB GPU cache for resident data and a 12 GB processing region, which comfortably fit JOB’s 2.5 GB dataset.
Each query ran five times. I recorded the final cache-warm execution, using Sirius’s engine logs and DuckDB’s native timer. This isolates query execution from one-time initialization and host-to-device transfer costs. It also makes the scope of the result explicit: these are hot-run numbers, not a claim that GPU offloading wins immediately on cold data.
- Intel Xeon Gold 6126 CPU and NVIDIA Quadro RTX 6000 GPU.
- 8 GB GPU cache and 12 GB processing region.
- Five executions per query; the final cache-warm run was recorded.
- Identical query topology and tables on Sirius and DuckDB.
Six times faster, once data was resident
The full JOB workload could not complete at first, because Sirius did not materialize string aggregate scalars. To measure the join pipeline on its own, I removed the aggregations over string columns while keeping every join, filter, table, and row. I ran the same modified queries on DuckDB so the comparison stayed fair.
On cache-warm runs, Sirius averaged roughly a 6× speedup over DuckDB. Cold runs fell to about 0.2×, because initialization and data transfer overwhelmed the execution gain. The point of reporting both is that GPU acceleration paid off after the dataset was resident, not before.
The average hides why individual queries behave so differently. Query 6f is close to an ideal GPU workload: an eight-row build side makes hash-table construction almost free, while a roughly 4.5-million-row probe side exposes enough parallel work to saturate the GPU. Query 27c produces only about 128 bytes after highly selective filtering, so the fixed kernel-launch and conversion overhead dominates the useful computation.
Query 10c regressed. Its LIKE predicate scans a variable-length note column that Sirius materialized in full and then deep-copied through later pipeline stages, even though the final result never projected that column. The redundant string movement saturated GPU memory bandwidth. DuckDB, by contrast, could scan the strings from CPU cache and prune the column earlier.
Bug 1: zero-row joins triggered invalid CUDA launches (PR #115)
Some JOB joins legitimately return zero rows. Sirius crashed on those results while translating row-ID buffers between cuDF’s 32-bit representation and its own 64-bit representation. The conversion helpers still launched CUDA kernels when the element count was zero, which produces a <<<0, BLOCK_THREADS>>> launch that CUDA rejects as an invalid configuration.
I added explicit N == 0 guards to both conversion directions, convertInt32ToUInt64 and convertUInt64ToInt32. An empty input now returns nullptr immediately instead of launching a kernel with no work. Non-empty buffers take the original cross-width conversion path unchanged. A zero-row join result is a normal database state, and the smallest edge case should not be able to abort the run. The fix was merged upstream as PR #115.
- Removed invalid zero-block CUDA launches.
- Preserved the existing behavior for every non-empty result.
- Turned zero-row join output into a supported database state.
- Merged upstream as PR #115.
Bug 2: string aggregates were computed but never materialized (PR #124)
cuDF could already compute MIN and MAX over string columns, but Sirius could not translate the resulting cudf::string_scalar into its own GPUColumn representation. The computation succeeded inside cuDF and then stopped at the engine boundary, because the variable-length string bytes, offsets, and metadata were never materialized for Sirius.
I extended GPUColumn::setFromCudfScalar with a string-scalar path. It conditionally allocates device storage, copies the character bytes and offsets, and sets the metadata needed to represent the result as a VARCHAR GPU column. This connects cuDF’s existing aggregate implementation to the rest of Sirius, rather than reimplementing the aggregate itself.
The same PR closed two adjacent edge cases. I replaced an unaligned memcpy in the CUDA inline-string materialization kernel with a byte-wise copy, and I changed the aggregate checks so that AVG, MIN, and MAX accept valid empty inputs instead of being rejected before cuDF sees them. The fix was merged upstream as PR #124.
- Materialized
cudf::string_scalarvalues as Sirius GPU columns. - Copied the variable-length string bytes device-to-device without unnecessary host staging.
- Avoided unaligned GPU memory access for inline strings.
- Allowed supported aggregates to receive empty inputs.
- Merged upstream as PR #124.
Running the full, unmodified suite
After string-scalar materialization landed, I ran the unmodified JOB suite, including its original string MIN and MAX aggregates, and compared the output against DuckDB. Sirius stayed faster on most queries, though the speedups were generally smaller than in the join-only variant.
That reduction has a concrete cause. String aggregation adds kernel launches, synchronization, scalar materialization, string-metadata handling, and the transfer of the final scalar. The modified benchmark exposed maximum join parallelism; the unmodified workload measures the additional cost of producing the complete SQL result.
I validated the change at three levels: the full JOB benchmark now ran to completion, the query results matched DuckDB, and Sirius’s make test suite passed. That combination mattered more than a single passing query, because performance, cross-engine correctness, and regression coverage all had to agree.
The broader lesson is not that GPUs are fast. It is that GPU databases amplify boundary conditions. An empty vector becomes an invalid launch. A scalar becomes a multi-buffer representation. A single unaligned copy can invalidate a whole execution path. A benchmark suite is useful precisely because it keeps surfacing those boundaries until the engine can handle the workload end to end.
- Benchmark the real workload, not only the queries already known to pass.
- Separate cache-warm execution gains from cold-start transfer costs.
- Validate every GPU result against a mature CPU engine.
- Treat empty results, strings, offsets, and alignment as first-class cases.