使用J-不变性校准消噪器的完整教程

在这个例子中，我们展示了如何找到任何去噪算法的最佳校准版本。

该校准方法是基于 noise2self 一种快速算法 1.

1

作者J.Batson&L.Royer。Noise2Self：自我监督的盲降噪，国际机器学习会议，第524-533页(2019)。

参见

文中给出了该方法的一个简单示例 利用J-不变性校正去噪器 。

校准小波去噪器

import numpy as np
from matplotlib import pyplot as plt
from matplotlib import gridspec

from skimage.data import chelsea, hubble_deep_field
from skimage.metrics import mean_squared_error as mse
from skimage.metrics import peak_signal_noise_ratio as psnr
from skimage.restoration import (calibrate_denoiser,
                                 denoise_wavelet,
                                 denoise_tv_chambolle, denoise_nl_means,
                                 estimate_sigma)
from skimage.util import img_as_float, random_noise
from skimage.color import rgb2gray
from functools import partial

_denoise_wavelet = partial(denoise_wavelet, rescale_sigma=True)

image = img_as_float(chelsea())
sigma = 0.2
noisy = random_noise(image, var=sigma ** 2)

# Parameters to test when calibrating the denoising algorithm
parameter_ranges = {'sigma': np.arange(0.1, 0.3, 0.02),
                    'wavelet': ['db1', 'db2'],
                    'convert2ycbcr': [True, False],
                    'multichannel': [True]}

# Denoised image using default parameters of `denoise_wavelet`
default_output = denoise_wavelet(noisy, multichannel=True, rescale_sigma=True)

# Calibrate denoiser
calibrated_denoiser = calibrate_denoiser(noisy,
                                         _denoise_wavelet,
                                         denoise_parameters=parameter_ranges
                                         )

# Denoised image using calibrated denoiser
calibrated_output = calibrated_denoiser(noisy)

fig, axes = plt.subplots(1, 3, sharex=True, sharey=True, figsize=(15, 5))

for ax, img, title in zip(axes,
                          [noisy, default_output, calibrated_output],
                          ['Noisy Image', 'Denoised (Default)',
                           'Denoised (Calibrated)']):
    ax.imshow(img)
    ax.set_title(title)
    ax.set_yticks([])
    ax.set_xticks([])

输出：

/scikit-image/doc/examples/filters/plot_j_invariant_tutorial.py:50: FutureWarning:

`multichannel` is a deprecated argument name for `denoise_wavelet`. It will be removed in version 1.0. Please use `channel_axis` instead.

/vpy/lib/python3.9/site-packages/scikit_image-0.20.0.dev0-py3.9-linux-x86_64.egg/skimage/restoration/j_invariant.py:144: FutureWarning:

`multichannel` is a deprecated argument name for `denoise_wavelet`. It will be removed in version 1.0. Please use `channel_axis` instead.

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

自我监督损失与J-不变性

这种校准方法的关键是J不变性的概念。如果去噪函数对每个像素的预测不依赖于该像素在原始图像中的值，则该去噪函数是J不变的。相反，对每个像素的预测可以使用包含在图像的其余部分中的所有相关信息，这通常是相当重要的。可以使用简单的掩码过程将任何函数转换为J不变函数，如中所述 [1] 。

J不变去噪器的像素级误差与噪声无关，只要每个像素中的噪声是独立的。因此，去噪图像和噪声图像之间的平均差、 self-supervised loss ，与去噪图像和原始清洁图像之间的差相同， ground-truth loss (最大为一个常数)。

这意味着，通过选择最小化自监督损失的去噪器，可以仅使用噪声数据来找到给定图像的最佳J不变去噪器。下面，我们将对一类小波去噪器进行演示，这些去噪器 sigma 参数。自我监督损失(实线蓝线)和地面真实损失(虚线蓝线)具有相同的形状和相同的最小化。

from skimage.restoration.j_invariant import _invariant_denoise

sigma_range = np.arange(sigma/2, 1.5*sigma, 0.025)

parameters_tested = [{'sigma': sigma, 'convert2ycbcr': True, 'wavelet': 'db2',
                      'multichannel': True}
                     for sigma in sigma_range]

denoised_invariant = [_invariant_denoise(noisy, _denoise_wavelet,
                                         denoiser_kwargs=params)
                      for params in parameters_tested]

self_supervised_loss = [mse(img, noisy) for img in denoised_invariant]
ground_truth_loss = [mse(img, image) for img in denoised_invariant]

opt_idx = np.argmin(self_supervised_loss)
plot_idx = [0, opt_idx, len(sigma_range) - 1]

get_inset = lambda x: x[25:225, 100:300]

plt.figure(figsize=(10, 12))

gs = gridspec.GridSpec(3, 3)
ax1 = plt.subplot(gs[0, :])
ax2 = plt.subplot(gs[1, :])
ax_image = [plt.subplot(gs[2, i]) for i in range(3)]

ax1.plot(sigma_range, self_supervised_loss, color='C0',
         label='Self-Supervised Loss')
ax1.scatter(sigma_range[opt_idx], self_supervised_loss[opt_idx] + 0.0003,
            marker='v', color='red', label='optimal sigma')

ax1.set_ylabel('MSE')
ax1.set_xticks([])
ax1.legend()
ax1.set_title('Self-Supervised Loss')

ax2.plot(sigma_range, ground_truth_loss, color='C0', linestyle='--',
         label='Ground Truth Loss')
ax2.scatter(sigma_range[opt_idx], ground_truth_loss[opt_idx] + 0.0003,
            marker='v', color='red', label='optimal sigma')
ax2.set_ylabel('MSE')
ax2.legend()
ax2.set_xlabel('sigma')
ax2.set_title('Ground-Truth Loss')

for i in range(3):
    ax = ax_image[i]
    ax.set_xticks([])
    ax.set_yticks([])
    ax.imshow(get_inset(denoised_invariant[plot_idx[i]]))
    ax.set_xlabel('sigma = ' + str(np.round(sigma_range[plot_idx[i]], 2)))

for spine in ax_image[1].spines.values():
    spine.set_edgecolor('red')
    spine.set_linewidth(5)

输出：

/vpy/lib/python3.9/site-packages/scikit_image-0.20.0.dev0-py3.9-linux-x86_64.egg/skimage/restoration/j_invariant.py:144: FutureWarning:

`multichannel` is a deprecated argument name for `denoise_wavelet`. It will be removed in version 1.0. Please use `channel_axis` instead.

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

到J-不变性的转换

该函数 _invariant_denoise 充当给定消噪器的J不变版本。它的工作原理是遮罩一部分像素，对它们进行内插，运行原始去噪器，并提取遮罩像素中返回的值。对图像进行迭代会产生完全J不变的输出。

对于任何给定的参数集，去噪器的J不变版本与原始去噪器不同，但它不一定更好或更差。在下面的图中，我们看到，对于猫的测试图像，J不变版本的小波去噪器在较小的方差减小值时明显好于原始版本 sigma 对于更大的值，情况会在不知不觉中变差。

parameters_tested = [{'sigma': sigma, 'convert2ycbcr': True,
                      'wavelet': 'db2', 'multichannel': True}
                     for sigma in sigma_range]

denoised_original = [_denoise_wavelet(noisy, **params)
                     for params in parameters_tested]

ground_truth_loss_invariant = [mse(img, image) for img in denoised_invariant]
ground_truth_loss_original = [mse(img, image) for img in denoised_original]

fig, ax = plt.subplots(figsize=(10, 4))

ax.plot(sigma_range, ground_truth_loss_invariant, color='C0', linestyle='--',
        label='J-invariant')
ax.plot(sigma_range, ground_truth_loss_original, color='C1', linestyle='--',
        label='Original')
ax.scatter(sigma_range[opt_idx], ground_truth_loss[opt_idx] + 0.001,
           marker='v', color='red')
ax.legend()
ax.set_title(
    'J-Invariant Denoiser Has Comparable Or '
    'Better Performance At Same Parameters'
)
ax.set_ylabel('MSE')
ax.set_xlabel('sigma')

输出：

/scikit-image/doc/examples/filters/plot_j_invariant_tutorial.py:176: FutureWarning:

`multichannel` is a deprecated argument name for `denoise_wavelet`. It will be removed in version 1.0. Please use `channel_axis` instead.


Text(0.5, 14.722222222222216, 'sigma')

比较不同类别的消噪器

除了为单个类别选择参数外，自监督损失还可用于比较不同类别的去噪器。这允许用户以无偏见的方式为给定图像的最佳类别的去噪器选择最佳参数。

下面，我们展示了一张添加了显著散斑噪声的哈勃深场图像。在这种情况下，在非局部平均、小波和TV范数三类去噪器中的每一种中，J不变校准去噪器都比默认去噪器好。此外，自监督去噪法表明，对于这种噪声图像，电视标准去噪器是最好的。

image = rgb2gray(img_as_float(hubble_deep_field()[100:250, 50:300]))

sigma = 0.4
noisy = random_noise(image, mode='speckle', var=sigma ** 2)

parameter_ranges_tv = {'weight': np.arange(0.01, 0.3, 0.02)}
_, (parameters_tested_tv, losses_tv) = calibrate_denoiser(
                                    noisy,
                                    denoise_tv_chambolle,
                                    denoise_parameters=parameter_ranges_tv,
                                    extra_output=True)
print(f'Minimum self-supervised loss TV: {np.min(losses_tv):.4f}')

best_parameters_tv = parameters_tested_tv[np.argmin(losses_tv)]
denoised_calibrated_tv = _invariant_denoise(noisy, denoise_tv_chambolle,
                                            denoiser_kwargs=best_parameters_tv)
denoised_default_tv = denoise_tv_chambolle(noisy, **best_parameters_tv)

psnr_calibrated_tv = psnr(image, denoised_calibrated_tv)
psnr_default_tv = psnr(image, denoised_default_tv)

parameter_ranges_wavelet = {'sigma': np.arange(0.01, 0.3, 0.03)}
_, (parameters_tested_wavelet, losses_wavelet) = calibrate_denoiser(
                                                noisy,
                                                _denoise_wavelet,
                                                parameter_ranges_wavelet,
                                                extra_output=True)
print(f'Minimum self-supervised loss wavelet: {np.min(losses_wavelet):.4f}')

best_parameters_wavelet = parameters_tested_wavelet[np.argmin(losses_wavelet)]
denoised_calibrated_wavelet = _invariant_denoise(
        noisy, _denoise_wavelet,
        denoiser_kwargs=best_parameters_wavelet)
denoised_default_wavelet = _denoise_wavelet(noisy, **best_parameters_wavelet)

psnr_calibrated_wavelet = psnr(image, denoised_calibrated_wavelet)
psnr_default_wavelet = psnr(image, denoised_default_wavelet)

sigma_est = estimate_sigma(noisy)

parameter_ranges_nl = {'sigma': np.arange(0.6, 1.4, 0.2) * sigma_est,
                       'h': np.arange(0.6, 1.2, 0.2) * sigma_est}

parameter_ranges_nl = {'sigma': np.arange(0.01, 0.3, 0.03)}
_, (parameters_tested_nl, losses_nl) = calibrate_denoiser(noisy,
                                                        denoise_nl_means,
                                                        parameter_ranges_nl,
                                                        extra_output=True)
print(f'Minimum self-supervised loss NL means: {np.min(losses_nl):.4f}')

best_parameters_nl = parameters_tested_nl[np.argmin(losses_nl)]
denoised_calibrated_nl = _invariant_denoise(noisy, denoise_nl_means,
                                            denoiser_kwargs=best_parameters_nl)
denoised_default_nl = denoise_nl_means(noisy, **best_parameters_nl)

psnr_calibrated_nl = psnr(image, denoised_calibrated_nl)
psnr_default_nl = psnr(image, denoised_default_nl)

print(f'                       PSNR')
print(f'NL means (Default)   : {psnr_default_nl:.1f}')
print(f'NL means (Calibrated): {psnr_calibrated_nl:.1f}')
print(f'Wavelet  (Default)   : {psnr_default_wavelet:.1f}')
print(f'Wavelet  (Calibrated): {psnr_calibrated_wavelet:.1f}')
print(f'TV norm  (Default)   : {psnr_default_tv:.1f}')
print(f'TV norm  (Calibrated): {psnr_calibrated_tv:.1f}')

plt.subplots(figsize=(10, 12))
plt.imshow(noisy, cmap='Greys_r')
plt.xticks([])
plt.yticks([])
plt.title('Noisy Image')

get_inset = lambda x: x[0:100, -140:]

fig, axes = plt.subplots(ncols=3, nrows=2, figsize=(15, 8))

for ax in axes.ravel():
    ax.set_xticks([])
    ax.set_yticks([])

axes[0, 0].imshow(get_inset(denoised_default_nl), cmap='Greys_r')
axes[0, 0].set_title('NL Means Default')
axes[1, 0].imshow(get_inset(denoised_calibrated_nl), cmap='Greys_r')
axes[1, 0].set_title('NL Means Calibrated')
axes[0, 1].imshow(get_inset(denoised_default_wavelet), cmap='Greys_r')
axes[0, 1].set_title('Wavelet Default')
axes[1, 1].imshow(get_inset(denoised_calibrated_wavelet), cmap='Greys_r')
axes[1, 1].set_title('Wavelet Calibrated')
axes[0, 2].imshow(get_inset(denoised_default_tv), cmap='Greys_r')
axes[0, 2].set_title('TV Norm Default')
axes[1, 2].imshow(get_inset(denoised_calibrated_tv), cmap='Greys_r')
axes[1, 2].set_title('TV Norm Calibrated')

for spine in axes[1, 2].spines.values():
    spine.set_edgecolor('red')
    spine.set_linewidth(5)

plt.show()

• 
• 

输出：

Minimum self-supervised loss TV: 0.0034
Minimum self-supervised loss wavelet: 0.0035
Minimum self-supervised loss NL means: 0.0037
                       PSNR
NL means (Default)   : 25.8
NL means (Calibrated): 27.3
Wavelet  (Default)   : 26.1
Wavelet  (Calibrated): 29.1
TV norm  (Default)   : 28.1
TV norm  (Calibrated): 29.5