Spring Dataset and Benchmark Logo

Spring: L. Mehl, J. Schmalfuss, A. Jahedi, Y. Nalivayko, A. Bruhn — University of Stuttgart

RobustSpring: V. Oei, J. Schmalfuss, L. Mehl, M. Bartsch, S. Agnihotri, M. Keuper, A. Bulling, A. Bruhn — University of Stuttgart, University of Mannheim, MPI for Informatics

Not logged in | Login | Create Account

M-FUSE (K)

Nov. 1, 2022, 2 p.m.  —  Public

đŸ’¡ submitted by spring team | L. Mehl, A. Jahedi, J. Schmalfuss, and A. Bruhn. "M-FUSE: Multi-frame Fusion for Scene Flow Estimation." In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023.

Code: https://github.com/cv-stuttgart/M-FUSE

Scene Flow

1px
total		 1px
low-detail		 1px
high-detail		 1px
matched		 1px
unmatched		 1px
rigid		 1px
non-rigid		 1px
not sky		 1px
sky		 1px
s0-10		 1px
s10-40		 1px
s40+
62.490 62.291 72.306 60.574 87.247 60.385 78.416 60.032 99.850 82.966 49.200 75.963
 
SF
total		 SF
low-detail		 SF
high-detail		 SF
matched		 SF
unmatched		 1px
rigid		 1px
non-rigid		 SF
not sky		 SF
sky		 SF
s0-10		 SF
s10-40		 SF
s40+
25.232 24.935 39.807 22.333 62.680 22.616 45.023 20.362 99.244 31.990 15.060 44.937

Disparity 1

1px
total		 1px
low-detail		 1px
high-detail		 1px
matched		 1px
unmatched		 1px
not sky		 1px
sky		 1px
s0-10		 1px
s10-40		 1px
s40+
52.232 52.185 55.051 50.647 83.685 49.104 99.771 80.774 34.056 52.347
 
Abs
total		 Abs
low-detail		 Abs
high-detail		 Abs
matched		 Abs
unmatched		 Abs
not sky		 Abs
sky		 Abs
s0-10		 Abs
s10-40		 Abs
s40+
7.890 7.907 6.908 6.715 31.222 4.143 64.840 16.260 2.836 7.103
 
D1
total		 D1
low-detail		 D1
high-detail		 D1
matched		 D1
unmatched		 D1
not sky		 D1
sky		 D1
s0-10		 D1
s10-40		 D1
s40+
19.836 19.798 22.085 17.671 62.816 14.618 99.136 34.182 11.009 18.971

Disparity 2

1px
total		 1px
low-detail		 1px
high-detail		 1px
matched		 1px
unmatched		 1px
not sky		 1px
sky		 1px
s0-10		 1px
s10-40		 1px
s40+
57.029 57.013 58.037 55.430 83.141 54.216 99.791 83.191 39.742 58.972
 
Abs
total		 Abs
low-detail		 Abs
high-detail		 Abs
matched		 Abs
unmatched		 Abs
not sky		 Abs
sky		 Abs
s0-10		 Abs
s10-40		 Abs
s40+
8.076 8.093 7.057 6.893 27.392 4.336 64.916 16.396 2.983 7.486
 
D2
total		 D2
low-detail		 D2
high-detail		 D2
matched		 D2
unmatched		 D2
not sky		 D2
sky		 D2
s0-10		 D2
s10-40		 D2
s40+
21.536 21.510 23.126 19.252 58.823 16.436 99.050 35.615 12.224 22.609

Optical Flow

1px
total		 1px
low-detail		 1px
high-detail		 1px
matched		 1px
unmatched		 1px
rigid		 1px
non-rigid		 1px
not sky		 1px
sky		 1px
s0-10		 1px
s10-40		 1px
s40+
20.979 20.600 80.743 19.942 63.882 15.953 59.005 19.500 43.455 10.131 30.966 84.713
 
EPE
total		 EPE
low-detail		 EPE
high-detail		 EPE
matched		 EPE
unmatched		 EPE
rigid		 EPE
non-rigid		 EPE
not sky		 EPE
sky		 EPE
s0-10		 EPE
s10-40		 EPE
s40+
2.526 2.408 21.176 2.236 14.545 1.598 9.547 2.098 9.031 0.773 1.477 20.035
 
Fl
total		 Fl
low-detail		 Fl
high-detail		 Fl
matched		 Fl
unmatched		 Fl
rigid		 Fl
non-rigid		 Fl
not sky		 Fl
sky		 Fl
s0-10		 Fl
s10-40		 Fl
s40+
8.480 8.177 56.222 7.737 39.239 5.849 28.383 7.109 29.309 3.178 9.726 49.467
 
WAUC
total		 WAUC
low-detail		 WAUC
high-detail		 WAUC
matched		 WAUC
unmatched		 WAUC
rigid		 WAUC
non-rigid		 WAUC
not sky		 WAUC
sky		 WAUC
s0-10		 WAUC
s10-40		 WAUC
s40+
76.182 76.501 26.016 77.049 40.302 80.041 46.989 77.427 57.269 84.450 69.614 24.786

Robustness Evaluation (Overall)

Disparity 1 Disparity 2 Optical Flow
1px
total		 Abs
total		 D1
total		 1px
total		 Abs
total		 D2
total		 EPE
total		 Fl
total		 1px
total
40.950 21.900 20.770 4.200 0.290 0.460 3.390 11.170 22.000

Robustness per Corruption

Robustness Evaluation (Per-Corruption)

Corruption Disparity 1 Disparity 2 Optical Flow
1px Abs D1 1px Abs D2 EPE Fl 1px
Brightness 6.380 1.270 1.780 1.530 0.140 0.180 0.830 2.800 5.540
Contrast 19.000 3.330 6.450 1.710 0.170 0.170 0.990 3.600 7.860
Saturate 6.430 1.240 1.710 1.220 0.120 0.140 0.670 2.430 4.940
Defocus Blur 8.550 2.020 2.490 1.370 0.150 0.150 0.840 2.710 5.260
Gaussian Blur 9.640 2.160 2.650 1.560 0.160 0.180 0.940 2.920 5.810
Glass Blur 11.560 2.170 2.550 1.320 0.160 0.140 0.800 2.650 5.170
Motion Blur 10.590 1.820 2.740 2.500 0.180 0.350 1.510 6.810 15.100
Zoom Blur 63.520 6.380 9.740 3.740 0.280 0.410 2.280 9.520 27.880
Gaussian Noise 80.740 80.890 62.280 6.560 0.410 0.800 6.490 14.810 29.220
Impulse Noise 85.390 85.240 65.420 8.110 0.430 0.880 5.980 19.160 37.320
Speckle Noise 84.060 84.540 65.370 5.680 0.350 0.570 3.730 12.220 29.390
Shot Noise 79.410 76.530 59.940 5.600 0.360 0.690 4.870 12.340 26.320
Pixelate 35.190 3.850 4.110 1.510 0.190 0.130 0.860 2.510 5.950
JPEG Compression 55.180 9.200 10.840 3.620 0.320 0.360 1.980 6.820 27.210
Elastic Transform 71.960 8.020 10.920 2.280 0.220 0.220 1.150 3.920 14.930
Fog 17.950 14.250 10.880 2.430 0.190 0.190 2.350 10.130 15.390
Frost 38.430 7.280 18.510 6.550 0.380 0.780 7.910 23.410 41.600
Rain 56.550 22.140 34.580 12.790 0.700 1.290 10.210 28.990 41.780
Snow 47.030 20.510 32.230 7.670 0.460 0.800 6.360 33.550 47.060
Spatter 31.430 5.130 10.190 6.210 0.390 0.800 7.000 22.100 46.350
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
reference frame
Reference Frame
color visualization of disparity 1
Disparity 1 visualization
grayscale visualization of disparity 1 error
Disparity 1 error
color visualization of disparity 2
Disparity 2 visualization
grayscale visualization of disparity 2 error
Disparity 2 error
color visualization of optical flow
Flow visualization
grayscale visualization of optical flow error
Flow error
grayscale visualization of optical flow error
Scene flow error
Error visualization color code
blue to white to red error visualization color code