Table 2.

SF-Relate scales efficiently to large data sets

Data setSF-RelateAll-pairwise
RuntimeCommunicationRuntime (estimated total)Comm. (estimated total)
Step 1Step 2 (MHE)TotalStep 1Step 2 (MHE)Total
Phase 1Phase 2Phase 1Phase 2
UKB-200K1.8 min14.0 h0.5 h14.5 h46.6 TB0.5 GB46.6 TB1.3 years32.5 PB
UKB-100K49.5 sec7.05 h0.23 h7.29 h23.85 TB241.7 MB23.85 TB112 days9.8 PB
AoU-20K18.6 sec5.65 h0.11 h5.79 h6.2 TB77.6 MB6.2 TB18.8 days2.31 PB

[i] We report the runtime and communication costs for individual steps of SF-Relate described in Methods. The runtime and communication costs for setting up the cryptographic keys are 40.4 sec and 1.7 GB, respectively, constant across all experiments. We also show the estimated total costs of running all-pairwise comparisons and determining the closest relationship for each individual using MHE.