The Spring Dataset & Benchmark

Spring: L. Mehl, J. Schmalfuss, A. Jahedi, Y. Nalivayko, A. Bruhn — University of Stuttgart

RobustSpring: V. Oei, J. Schmalfuss, L. Mehl, M. Bartsch, S. Agnihotri, M. Keuper, A. Bulling, A. Bruhn — University of Stuttgart, University of Mannheim, MPI for Informatics

Not logged in | Login | Create Account

RAFT-3D (K)

Nov. 8, 2022, 5:15 p.m. — Public

💡 submitted by spring team | Z. Teed, and J. Deng. "RAFT-3D: Scene Flow using Rigid-Motion Embeddings." In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

Code: https://github.com/princeton-vl/RAFT-3D/

Scene Flow

1px total	1px low-detail	1px high-detail	1px matched	1px unmatched	1px rigid	1px non-rigid	1px not sky	1px sky	1px s0-10	1px s10-40	1px s40+
37.262	36.796	60.227	34.340	75.021	32.866	70.522	33.231	98.531	43.799	24.546	63.914

SF total	SF low-detail	SF high-detail	SF matched	SF unmatched	1px rigid	1px non-rigid	SF not sky	SF sky	SF s0-10	SF s10-40	SF s40+
17.345	16.998	34.446	14.320	56.422	14.367	39.880	13.200	80.341	20.238	9.431	35.169

Disparity 1

1px total	1px low-detail	1px high-detail	1px matched	1px unmatched	1px not sky	1px sky	1px s0-10	1px s10-40	1px s40+
32.309	32.096	45.189	30.071	76.734	27.963	98.355	39.354	22.522	48.158

Abs total	Abs low-detail	Abs high-detail	Abs matched	Abs unmatched	Abs not sky	Abs sky	Abs s0-10	Abs s10-40	Abs s40+
7.042	7.059	5.994	5.670	34.271	3.244	64.766	13.584	3.151	6.247

D1 total	D1 low-detail	D1 high-detail	D1 matched	D1 unmatched	D1 not sky	D1 sky	D1 s0-10	D1 s10-40	D1 s40+
14.174	14.140	16.206	11.863	60.041	10.051	76.837	19.487	9.482	18.100

Disparity 2

1px total	1px low-detail	1px high-detail	1px matched	1px unmatched	1px not sky	1px sky	1px s0-10	1px s10-40	1px s40+
32.948	32.735	45.810	30.571	71.760	28.644	98.367	39.367	22.654	51.487

Abs total	Abs low-detail	Abs high-detail	Abs matched	Abs unmatched	Abs not sky	Abs sky	Abs s0-10	Abs s10-40	Abs s40+
7.111	7.127	6.119	5.752	29.296	3.294	65.115	13.651	3.157	6.498

D2 total	D2 low-detail	D2 high-detail	D2 matched	D2 unmatched	D2 not sky	D2 sky	D2 s0-10	D2 s10-40	D2 s40+
14.544	14.505	16.919	12.168	53.340	10.445	76.843	19.482	9.532	20.129

Optical Flow

1px total	1px low-detail	1px high-detail	1px matched	1px unmatched	1px rigid	1px non-rigid	1px not sky	1px sky	1px s0-10	1px s10-40	1px s40+
13.962	13.539	80.464	12.963	55.254	8.932	52.013	11.822	46.479	8.895	14.726	54.283

EPE total	EPE low-detail	EPE high-detail	EPE matched	EPE unmatched	EPE rigid	EPE non-rigid	EPE not sky	EPE sky	EPE s0-10	EPE s10-40	EPE s40+
2.528	2.401	22.473	2.229	14.883	1.508	10.239	2.016	10.309	0.671	1.358	21.227

Fl total	Fl low-detail	Fl high-detail	Fl matched	Fl unmatched	Fl rigid	Fl non-rigid	Fl not sky	Fl sky	Fl s0-10	Fl s10-40	Fl s40+
6.889	6.557	59.261	6.129	38.333	4.026	28.553	5.329	30.596	2.890	7.225	39.438

WAUC total	WAUC low-detail	WAUC high-detail	WAUC matched	WAUC unmatched	WAUC rigid	WAUC non-rigid	WAUC not sky	WAUC sky	WAUC s0-10	WAUC s10-40	WAUC s40+
81.267	81.624	24.923	82.127	45.658	85.308	50.693	83.082	53.671	86.026	80.124	44.539

Robustness Evaluation (Overall)

Disparity 1			Disparity 2			Optical Flow
1px total	Abs total	D1 total	1px total	Abs total	D2 total	EPE total	Fl total	1px total
45.260	12.110	17.930	2.890	0.140	0.460	5.030	17.360	31.200

Robustness per Corruption

Method: Error metric:

Robustness Evaluation (Per-Corruption)

Corruption	Disparity 1			Disparity 2			Optical Flow
	1px	Abs	D1	1px	Abs	D2	EPE	Fl	1px
Brightness	10.740	2.110	3.390	1.480	0.070	0.210	1.380	3.870	8.230
Contrast	23.140	3.940	6.740	1.650	0.070	0.220	1.420	5.070	10.710
Saturate	13.530	2.700	3.860	1.330	0.060	0.180	0.930	3.310	6.720
Defocus Blur	12.340	2.460	3.160	0.880	0.040	0.100	0.660	2.440	5.270
Gaussian Blur	13.760	2.690	3.450	1.040	0.050	0.140	0.780	2.730	5.850
Glass Blur	19.420	2.610	3.150	0.820	0.040	0.090	0.650	2.390	5.290
Motion Blur	13.120	2.310	3.610	1.600	0.080	0.280	1.620	6.850	14.660
Zoom Blur	59.890	7.290	11.210	2.840	0.140	0.500	2.680	11.990	34.060
Gaussian Noise	85.780	33.350	45.020	3.640	0.200	0.710	5.250	25.430	43.330
Impulse Noise	85.000	38.940	50.450	4.430	0.220	0.750	6.730	33.160	59.860
Speckle Noise	83.700	29.650	41.900	3.170	0.180	0.640	4.860	26.110	51.120
Shot Noise	81.490	28.200	39.890	3.260	0.180	0.670	4.650	22.910	42.070
Pixelate	59.610	3.700	4.070	1.020	0.050	0.100	0.820	2.830	7.660
JPEG Compression	59.520	6.760	10.100	2.590	0.130	0.410	2.730	10.550	33.930
Elastic Transform	76.470	4.850	5.050	1.610	0.080	0.200	1.700	5.990	21.820
Fog	20.550	9.680	9.750	1.230	0.060	0.150	2.290	11.670	18.150
Frost	47.400	11.200	24.310	3.750	0.160	0.520	7.490	24.260	45.070
Rain	59.220	26.500	42.340	10.750	0.470	1.960	27.890	59.770	74.230
Snow	45.880	17.240	33.300	6.790	0.310	0.840	19.080	60.010	80.490
Spatter	34.580	6.040	13.860	3.820	0.170	0.530	7.060	25.800	55.550

Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error
Reference Frame	Disparity 1 visualization	Disparity 1 error	Disparity 2 visualization	Disparity 2 error	Flow visualization	Flow error	Scene flow error

blue to white to red error visualization color code — Error visualization color code