图像的视差匹配（Stereo Matching）

这里要求用我们自己计算得到的视差图和给的视差图作比較来比較我们得到的视差图的好坏程度，我视差图返回的值是计算得到的视差乘以3之后的图，所以在计算时我不是两个值相差大于1，而是大于3。由于两个图像都乘3了。所以要大于3。我传入的參数是两个图像的矩阵。由于我是写了一个脚本咯跑全部測例的。在脚本里边已经把图像读出来了

<span style="font-size:18px;">function [percentnumberbadpixels] = PercentBadPixels(mydisparitymap,groundtruthdisparitymap)
mydisparitymap = double(mydisparitymap);
groundtruthdisparitymap = double(groundtruthdisparitymap);
[Rmydisparitymap, Cmydisparitymap] = size(mydisparitymap);
numbadpixels = 0;
for i = 1:Rmydisparitymap
    for j = 1:Cmydisparitymap
        if (abs(mydisparitymap(i,j) - groundtruthdisparitymap(i,j)) > 3)
              numbadpixels = numbadpixels+1;
        end
    end
end
percentnumbadpixels = numbadpixels/(Rmydisparitymap*Cmydisparitymap*1.0);
percentnumbadpixels = floor(percentnumbadpixels*100);
str = strcat(num2str(percentnumbadpixels),'%');
percentnumberbadpixels = str;
end
</span>

SSD:

左视差图

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

右视差图

实现细节：

输入的图像是24位的图像，可是这些计算都是基于强度（intensities）来计算的，在matlab中把彩色图转为灰度图的函数rgb2gray实质上就是转为其强度灰度图像。

原理：

如图：

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

对于同一图像。左眼看到的图像相对于右眼看到的图像的位置是不一样的，如上图，对于同样位置在右眼看到的图像的位置在左眼看到的图像的位置的前边，所以左视差图和右视差图的搜索方法不同，对于左视差图：

因为其位置比右眼看到的位置要远，则对于在右眼图像同样的物理位置应该往该位置的前边搜索才可能找到最相近的部分，即要减去d。

同理。对于右视差图则刚好相反。要加d。

实现细节：

1）首先把图像通过intensities转为灰度图像：

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

2）设定搜索深度d的值和patch的值。然后对图像进行补边界，使得能够从原图像的第一个像素開始进行匹配，我用的补边界的方法和曾经做滤波时候原理一样。我都是补周围的值

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

3）对左图或右图中的每个像素（i。j）作为patch的中心在另外的左图或右图上来寻找与之最相近的patch部分的搜索深度d，详细为：

对于模板图像中每个不移动的patch。把该patch里边的每个像素和另外一张目标图相应得patch（加上搜索深度d的）上的像素点做差的平方和然后再累加起来，每个d相应一个patch，每个patch相应一个值（sum），然后把求出在搜索深度d的范围内最小的sum，从而得到该最小的sum的搜索深度d的值，当遇到边界时用推断来处理，详细实现例如以下：

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

假设是求右视差图的话，上边原理以及介绍了，要加上d来进行搜索

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

NCC

左视差图：

右视差图:

实现细节：

图像转为灰度图及边界处理和SSD同样，公式实现：

对左图或右图中的每个像素（i，j）作为patch的中心，在另外的左图或右图上来寻找与之最相近的patch部分的搜索深度d。

1）对于模板图像中每个不移动的patch，把该patch里边的每个像素和另外一张目标图相应的patch（加上搜索深度d的）上的相应像素点分别相乘相加，把模板图中的patch和目标图的patch中的像素值分别平方相加，然后再两个值相乘开方。

2) 每个d相应一个patch。每个patch相应一个值（ncctemp），然后把求出在搜索深度d的范围内最大的ncctemp，从而得到该最大的ncctemp的搜索深度d的值，当遇到边界时用推断来处理

NCC与SSD的差别：

NCC抗光照性强，SSD抗光照性弱

如：Add a small constant amount of intensity (e.g. 10) to all right eye images, and re-run the above two methods. Analyze how the intensity change affects
the results (i.e. the quality) of the two methods. Explain in which ways that NCC is a better matching cost than SSD.

这里我在view5加了常数10

NCC结果例如以下：Aloe disp1坏点率为24%

View5强度没有加10是坏点率为：24%

SSD 结果例如以下：Aloe disp1坏点率为38%

View5强度没有加10是坏点率为：26%

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQv/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="">

分析：

由上边公式可SSD是依据两幅图上patch里两个像素点差值的平方求和来找到最小的值。依据这个最小值我们觉得距离为d的patch是匹配的部分，可是在拍照片是。会受到一些因素的影响，比方光照强度等，这样会照成左眼和右眼看到的图像的强度不一样，假如右眼看到的图像强度添加10，这样对于patch里边像素值相减得到的结果就会有非常大的变化，从而造成求得的最小值的d也会不一样，因此的图像和右眼图像没有强度变化的结果有非常大的不同。

而对于NCC来说，分子上是两个patch里边的像素分别相乘相加，而分母是每一个patch里边的像素值平方然后求和在相乘开方，即使右眼的亮度加了。可是分子和分母的值都会同一时候增大，并且增大的值差点儿相同同样。增大的值相除接近为1，所以得到的结果和右眼强度没有加10的结果基本上一样，没有多大的变化。

所以假设左眼图和右眼图受光照强度干扰比較大的情况下，NCC比SSD好。

nccdispl.m

function [output_img] = nccdispl(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
disparitymap = zeros(nrright,ncright);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
d = 70;
winsize = 11;
disparitymap = zeros(nrright,ncright);
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrright+win
    for j = 1+win:ncright+win
        min = -9999999;
        for k = 0:d
            numerator = 0.0;
            powerrightwin = 0.0;
            powerleftwin = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b-k > 0
                        numerator = numerator+(leftimg(i+a,j+b)*rightimg(i+a,j+b-k));
                        powerleftwin = powerleftwin+(leftimg(i+a,j+b)*leftimg(i+a,j+b));
                        powerrightwin = powerrightwin+(rightimg(i+a,j+b-k)*rightimg(i+a,j+b-k));
                    end
                end
            end
            ncctemp = numerator/(sqrt(powerrightwin*powerleftwin));
            if (min < ncctemp && j+b-k > 0)
                min = ncctemp;
                recordk = k;
            end
        end
        disparitymap(i-win,j-win) = recordk;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\nccdisp1.png'));
end

</pre><pre name="code" class="plain"><pre name="code" class="plain" style="font-size:18px;">ssddispl.m

<pre name="code" class="plain">function [output_img] = ssddispl(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end

if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
disparitymap = zeros(nrleft,ncleft);
d = 70;
winsize = 11;
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrright+win
    for j = 1+win:ncright+win
        max = 99999999;
        for k = 0:d
            sum = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b-k > 0
                         temp = leftimg(i+a,j+b)-rightimg(i+a,j+b-k);
                         temp = temp*temp;
                         sum = sum+temp;
                    end
                end
            end
            if (max > sum && j+b-k > 0)
                max = sum;
                mind = k;
            end
        end
        disparitymap(i-win,j-win) = mind;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\ssddisp1.png'));
end

<pre name="code" class="plain">nccdispr.m

function [output_img] = nccdispr(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
d = 70;
winsize = 11;
disparitymap = zeros(nrleft,ncleft);
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrleft+win
    for j = 1+win:ncleft+win
        min = -999999;
        for k = 0:d
             numerator = 0.0;
             powerrightwin = 0.0;
             powerleftwin = 0.0;
            for a = -win:win
                for b = -win:win
                    if j+b+k <= ncleft+2*win
                        numerator = numerator+(rightimg(i+a,j+b)*leftimg(i+a,j+b+k));
                        powerrightwin = powerrightwin+(rightimg(i+a,j+b)*rightimg(i+a,j+b));
                        powerleftwin = powerleftwin+(leftimg(i+a,j+b+k)*leftimg(i+a,j+b+k));
                    end

                end
            end
            ncctemp = numerator/(sqrt(powerrightwin*powerleftwin));
            if (min < ncctemp && j+b+k <= ncleft+2*win)
                min = ncctemp;
                recordk = k;
            end
        end
        disparitymap(i-win,j-win) = recordk;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\nccdisp5.png'));
end

</pre><pre name="code" class="plain"><pre name="code" class="plain" style="font-size:18px;">ssddispr.m

<pre name="code" class="plain">function [output_img] = ssddispr(leftimg,rightimg)
[nrleft,ncleft,nleft] = size(leftimg);
[nrright,ncright,nright] =size(rightimg);
if (nleft > 2)
   leftimg = rgb2gray(leftimg);
end
if (nright > 2)
    rightimg = rgb2gray(rightimg);
    rightimg = rightimg+10;
end
disparitymap = zeros(nrleft,ncleft);
d = 70;
winsize = 11;
leftimg = double(leftimg);
rightimg = double(rightimg);
leftimg = supplyborder(leftimg,winsize);
rightimg = supplyborder(rightimg,winsize);
win = (winsize-1)/2;
for i = 1+win:nrleft+win
    for j = 1+win:ncleft+win
        max = 999999999;
        for k = 0:d
            sum = 0.0;
            for a=-win:win
                for b = -win:win
                    if j+b+k <= ncleft+2*win
                         temp = rightimg(i+a,j+b)-leftimg(i+a,j+b+k);
                         temp = temp*temp;
                         sum = sum+temp;
                    end
                end
            end
            if (max > sum && j+b+k <= ncleft+win)
                max = sum;
                mind = k;
            end
        end
        disparitymap(i-win,j-win) = mind;
    end
end
disparitymap = uint8(3*disparitymap);
output_img = disparitymap;
% imshow(disparitymap);
% imwrite(disparitymap,strcat('C:\Users\samsung-\Desktop\output\ssddisp5.png'));
end