ECCV 2026

DeLux : Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data

Bartosz Stachowiak & Dariusz Brzeziński

Institute of Computing Science, Poznań University of Technology, Poland

Conventional RGB cameras suffer from lighting artifacts such as flare, glare, flicker, and overexposure, leading to irrecoverable information loss that necessitates computational restoration. However, existing approaches treat these problems in isolation, failing to recover structural details completely obscured by complex spatially discrete image degradations.

We propose a novel cross-modal restoration paradigm and present DeLux, a modular proof-of-concept pipeline that leverages neuromorphic event streams as a structural prior to guide the targeted detection and inpainting of lighting artifacts in RGB video. Validation on synthetic benchmarks and real-world automotive footage demonstrates that DeLux effectively suppresses local artifacts and restores affected regions, outperforming RGB-only baselines and event-guided HDR models.

cross-modal restoration lighting artifact removal event-based vision multimodal fusion video inpainting
Paper PDF arXiv Code

Why event cameras?

When an RGB sensor saturates, the information is permanently lost. Neuromorphic event cameras asynchronously capture brightness changes with microsecond precision and an effective dynamic range exceeding 140 dB, preserving scene structure even in the presence of extreme illumination.

Comparison of an overexposed RGB frame, raw event data, and the event-based reconstruction that recovers the lost scene details at a tunnel exit.
Left: overexposed RGB image loses all structure. Center: raw event data retains brightness changes. Right: event-based reconstruction restores the scene details.
Glare artifact example

Glare

Lens flare artifact example

Lens flare

Overexposure artifact example

Overexposure

Flicker artifact example across four frames

Flicker

The DeLux pipeline

DeLux explicitly decouples artifact detection from multimodal fusion and image inpainting. It takes a corrupted RGB frame together with a temporally aligned event window, reconstructs a grayscale event video, detects the degraded region, and inpaints only the affected pixels.

Event Window 1. Event-to-video Event Reconstruction 3. Multimodal Fusion Artifact-Free RGB RGB Frame 2. Artifact Detection Artifact Map Fused Representation 4. Artifact Removal

Architecture & training

Detector, fusion module, and removal network are trained jointly under a composite objective that balances artifact localization and image reconstruction quality. The event-to-video reconstructor remains frozen.

DeLux U-Net architecture and end-to-end training pipeline.
Left: shared U-Net backbone and building blocks. Right: end-to-end training with synthetic artifact generation, multimodal inputs, and composite losses.

Results

Drag the slider to compare the corrupted RGB input (left) with the restoration produced by DeLux (right).

DeLux output Corrupted input
❮
❯

Real-world glare — sun2

DeLux output Corrupted input
❮
❯

Real-world lens flare — sun10

DeLux output Corrupted input
❮
❯

Real-world overexposure — sun11

DeLux output Corrupted input
❮
❯

Real-world mixed artifacts — highway1

DeLux output Corrupted input
❮
❯

Synthetic lens flare — highway1 synthetic

Video comparisons

Side-by-side comparisons on real-world automotive recordings. Each video shows the corrupted RGB input on the left and the DeLux restoration on the right.

highway1

sun10

sun11

sun14

Quantitative evaluation

DeLux is evaluated on both synthetic benchmarks and real-world automotive recordings. Best values are bold, second-best are underlined.

MS-SSIM
Multi-Scale Structural Similarity Index Measure — compares a restored image with its clean reference frame.
PSNR
Peak Signal-to-Noise Ratio — reconstruction fidelity relative to a clean reference frame.
MAPE
Mean Absolute Percentage Error — relative reconstruction error against a clean reference frame.
Accuracy
Pixel-wise classification accuracy of the predicted artifact mask.
F1-score
Harmonic mean of precision and recall for the artifact detection mask.
FIV
Frame Intensity Variance — variance of the mean luminance across video frames; lower indicates less global brightness fluctuation.
MAR
Mean Artifact Reduction — reduction in the mean estimated artifact-map intensity relative to the input.
SAS
Strong Artifact Suppression — decrease in pixels classified as strong artifacts before versus after restoration.
Δ SAS
Relative change in Strong Artifact Suppression compared with the corrupted input.

Synthetic reconstruction & removal

Metric DAD†F7KWu et al.SHDRHDRev-DiffDeLux
MS-SSIM ↑0.9700.9540.9490.868—0.991
PSNR ↑29.5326.5825.8514.42—36.46
MAPE ↓0.0600.0710.0780.537—0.028
Δ SAS ↑-352.50%58.87%18.54%-255.65%-1402.87%63.22%

Synthetic artifact detection

Metric DAD†F7KWu et al.DeLuxDeLux-D
Accuracy ↑0.9490.9150.8090.9250.971
F1-score ↑0.6340.2120.2810.6130.785

Real-world removal effectiveness

Metric DADDAD*F7KWu et al.SHDRDeflickerDeLux
FIV ↓3910.523893.353534.974195.34—3278.413427.30
MAR ↑17.49%0.89%34.08%41.39%-13.56%—53.99%
SAS ↑-2.949-3.5250.5890.185-2.557—0.632

Citation

@misc{stachowiak2026deluxcrossmodallocalartifact,
  title={DeLux: Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data},
  author={Bartosz Stachowiak and Dariusz Brzezinski},
  year={2026},
  eprint={2606.27576},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2606.27576},
}