Admin
Members
Home

Abstract

Zero-reference deep learning-based methods for low-light image enhancement sufficiently mitigate the difficulty of paired data collection while keeping the great generalization on various lighting conditions, but color bias and unintended intrinsic noise amplification are still issues that remain unsolved. In this paper, we propose a zero-reference end-to-end two-stage network (Zero-LEINR) for low-light image enhancement with intrinsic noise reduction. In the first stage, we introduce a Color Preservation and Light EnhancementBlock (CPLEB) that consists of a dual branch structure with different constraints to correct the brightness and preserve the correct color tone. In the second stage, Enhanced-NoiseReduction Block (ENRB) is applied to remove the intrinsic noises being enhanced during the first stage. Due to the zero-reference two-stage structure, our method can enhance the low-light image with the correct color tone on unseen datasets and reduce the intrinsic noise at the same time.

Proposed Method

Overview of Zero-LEINR. CPLEB preserves the color while correcting the illuminance by combining the results from dual branches. ENRB removes the intrinsic noise enhanced unintendedly through a denoising network trained by two independent noisy pair subsampled from same noisy image. Dotted line indicate the paths used only in training stage.

Contributions

  1. We propose a two-stage end-to-end solution for low-light image enhancement. At the first stage, a zero-reference low-light enhancer CPLEB is applied to boost the brightness and preserve the correct tone. Next, the unintendedly amplified noise can also be removed in a zero-reference manner by appending a denoiser ENRB. Our structure can alleviate the heavy burden of collecting training data.
  2. We propose a simple but effective dual branch structure inspired by DCE-Net with different constraints to enhance the illumination while preserving the color during the low-light enhancement. Furthermore, this structure inherits the great ability of generalization and perform averagely well in unseen datasets.

Experimental Results

  1. Image enhancement on the LOL dataset[1]. Compared with exiting state-of-the-art methods, our method achieves well performance(PSNR on y-axis) with few parameter(x-axis).
  2. Qualitative Results
  3. Demo video

Ablation Study

  1. Table1 shows different setting with correspond PSNR, SSIM on LOL dataset[1]. We train all the models from scratch using the images of MIT-Adobe FiveK[9], LOL[1], and Part1 of SICE[10] like we mentioned in paper. In "Only Single Branch" setup, we use both the color constancy(global) loss and the color angle(local) loss for training in one branch. The results show that the local loss would dominate. Also, "Only Branch1" and "Only Branch2" settings are two extreme cases when tuning the ratio between color constancy loss and the color angle loss.
    2. CPLEB ENRB PSNR/SSIM
    Branch1 Branch2
    ✔ ✘ ✘ 16.84/0.54
    ✘ ✔ ✘ 9.79/0.39
    ✔
    Only single branch
    		 ✘ 9.85/0.40
    ✔ ✔ ✘ 17.02/0.50
    ✔ ✘ ✔ 16.97/0.76
    ✘ ✔ ✔ 11.01/0.13
    ✔
    Only single branch
    		 ✔ 10.23/0.47
    ✔ ✔ ✔ 18.02/0.78
  2. Visual Comparisons

Reference

  1. C. Wei, W. Wang, W. Yang, and J. Liu, "Deep retinex decomposition for low-light enhancement," in British Machine Vision Conference, 2018.
  2. X. Fu, D. Zeng, Y. Huang, X.-P. Zhang, and X. Ding, "A weighted variational model for simultaneous reflectance and illumination estimation," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2782–2790.
  3. M. Li, J. Liu, W. Yang, X. Sun, and Z. Guo, "Structure-revealing low-light image enhancement via robust retinex model," IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 2828–2841, 2018.
  4. Li, X. Zhang, and D. Chen, "CSRNet: Dilated convolutional neural networks for understanding the highly congested scenes," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  5. M. Afifi, K. G. Derpanis, B. Ommer, and M. S. Brown, "Learning multi-scale photo exposure correction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021, pp. 9157–9167.
  6. Y. Jiang, X. Gong, D. Liu, Y. Cheng, C. Fang, X. Shen, J. Yang, Pan Zhou, and Z. Wang, "EnlightenGAN: Deep light enhancement without paired supervision," IEEE Transactions on Image Processing, vol. 30, pp. 2340– 2349, 2021.
  7. Z. Ni, W. Yang, S. Wang, L. Ma, and S. Kwong, “Towards unsupervised deep image enhancement with generative adversarial network,” IEEE Transactions on Image Processing vol. 29, pp. 9140-9151
  8. C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong, and R. Cong, "Zero-reference deep curve estimation for low-light image enhancement," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  9. V. Bychkovsky, S. Paris, E. Chan, and F. Durand,“Learning photographic global tonal adjustment with a database of input / output image pairs,” in The Twenty-Fourth IEEE Conference on Computer Vision and Pattern Recognition, 2011
  10. J. Cai, S. Gu, and L. Zhang, “Learning a deep single image contrast enhancer from multi-exposure images,” IEEE Transactions on Image Processing, vol. 27, no. 4, pp. 2049-2062, 2018