問題描述

我目前正在使用 opencv (CV2) 和 Python Pillow 圖像庫來嘗試拍攝任意手機的圖像并用新圖像替換屏幕.我已經到了可以拍攝圖像并識別手機屏幕并獲取角落的所有坐標的地步，但是我很難用新圖像替換圖像中的那個區域.

我目前的代碼:

導入 cv2從 PIL 導入圖像image = cv2.imread('mockup.png')edged_image = cv2.Canny(圖像, 30, 200)(輪廓，_)= cv2.findContours(edged_image.copy()，cv2.RETR_TREE，cv2.CHAIN_APPROX_SIMPLE)輪廓=排序(輪廓，鍵= cv2.contourArea，反向=真)[:10]screenCnt = 無對于輪廓中的輪廓:peri = cv2.arcLength(輪廓，真)約= cv2.approxPolyDP(輪廓，0.02 * peri，真)# 如果我們的近似輪廓有四個點，那么# 我們可以假設我們已經找到了我們的屏幕如果 len(大約)== 4:screenCnt = 大約休息cv2.drawContours(image, [screenCnt], -1, (0, 255, 0), 3)cv2.imshow("屏幕位置", image)cv2.waitKey(0)

這會給我一個看起來像這樣的圖像:

我也可以使用這行代碼獲取屏幕角的坐標:

screenCoords = [x[0].tolist() for x in screenCnt]//[[398, 139], [245, 258], [474, 487], [628, 358]]

但是，我終生無法弄清楚如何拍攝新圖像并將其縮放到我找到的坐標空間的形狀并將圖像覆蓋在上面.

我的猜測是，我可以使用我改編自

如果我使用不同的垂直高度非常高的圖像，我最終會得到一些太長"的圖像:

我是否需要應用額外的轉換來縮放圖像?不知道這里發生了什么，我認為透視變換會使圖像自動縮放到提供的坐標.

我下載了你的圖片數據并在本地機器上重現了問題以找出解決方案.還下載了 lenna.png 以適應手機屏幕.

導入 cv2將 numpy 導入為 np# iPhone 的模板圖片img1 = cv2.imread("/Users/anmoluppal/Downloads/46F1U.jpg")# 用于擬合白色空腔的樣本圖像img2 = cv2.imread("/Users/anmoluppal/Downloads/Lenna.png")行，列，ch = img1.shape# 硬編碼白色空腔的 3 個角點，用綠色矩形標記.pts1 = np.float32([[201, 561], [455, 279], [742, 985]])# 在要擬合的參考圖像上硬編碼相同的點.pts2 = np.float32([[0, 0], [512, 0], [0, 512]])# 將樣本圖像仿射變換為模板.M = cv2.getAffineTransform(pts2,pts1)# 應用轉換，注意傳遞的 (cols,rows)，這些定義了轉換后輸出的最終維度.dst = cv2.warpAffine(img2,M,(cols,rows))# 僅用于調試輸出.最終 = cv2.addWeighted(dst, 0.5, img1, 0.5, 1)cv2.imwrite("./garbage.png", 最終)

I'm currently using the opencv (CV2) and Python Pillow image library to try and take an image of arbitrary phones and replace the screen with a new image. I've gotten to the point where I can take an image and identify the screen of the phone and get all the coordinates for the corner, but I'm having a really hard time replacing that area in the image with a new image.

The code I have so far:

import cv2
from PIL import Image

image = cv2.imread('mockup.png')
edged_image = cv2.Canny(image, 30, 200)

(contours, _) = cv2.findContours(edged_image.copy(), cv2.RETR_TREE,     cv2.CHAIN_APPROX_SIMPLE)
contours = sorted(contours, key = cv2.contourArea, reverse = True)[:10]
screenCnt = None

for contour in contours:
    peri = cv2.arcLength(contour, True)
    approx = cv2.approxPolyDP(contour, 0.02 * peri, True)

    # if our approximated contour has four points, then
    # we can assume that we have found our screen
    if len(approx) == 4:
        screenCnt = approx
        break

cv2.drawContours(image, [screenCnt], -1, (0, 255, 0), 3)
cv2.imshow("Screen Location", image)
cv2.waitKey(0)

This will give me an image that looks like this:

I can also get the coordinates of the screen corners using this line of code:

screenCoords = [x[0].tolist() for x in screenCnt] 
// [[398, 139], [245, 258], [474, 487], [628, 358]]

However I can't figure out for the life of me how to take a new image and scale it into the shape of the coordinate space I've found and overlay the image ontop.

My guess is that I can do this using an image transform in Pillow using this function that I adapted from this stackoverflow question:

def find_transform_coefficients(pa, pb):
"""Return the coefficients required for a transform from start_points to end_points.

    args:
        start_points -> Tuple of 4 values for start coordinates
        end_points --> Tuple of 4 values for end coordinates
"""
matrix = []
for p1, p2 in zip(pa, pb):
    matrix.append([p1[0], p1[1], 1, 0, 0, 0, -p2[0]*p1[0], -p2[0]*p1[1]])
    matrix.append([0, 0, 0, p1[0], p1[1], 1, -p2[1]*p1[0], -p2[1]*p1[1]])

A = numpy.matrix(matrix, dtype=numpy.float)
B = numpy.array(pb).reshape(8)

res = numpy.dot(numpy.linalg.inv(A.T * A) * A.T, B)
return numpy.array(res).reshape(8) 

However I'm in over my head a bit, and I can't get the details right. Could someone give me some help?

EDIT

Ok now that I'm using the getPerspectiveTransform and warpPerspective functions, I've got the following additional code:

screenCoords = numpy.asarray(
    [numpy.asarray(x[0], dtype=numpy.float32) for x in screenCnt],
    dtype=numpy.float32
)

overlay_image = cv2.imread('123.png')
overlay_height, overlay_width = image.shape[:2]

input_coordinates = numpy.asarray(
    [
        numpy.asarray([0, 0], dtype=numpy.float32),
        numpy.asarray([overlay_width, 0], dtype=numpy.float32),
        numpy.asarray([overlay_width, overlay_height],     dtype=numpy.float32),
        numpy.asarray([0, overlay_height], dtype=numpy.float32)
    ],
    dtype=numpy.float32,
)

transformation_matrix = cv2.getPerspectiveTransform(
    numpy.asarray(input_coordinates),
    numpy.asarray(screenCoords),
)

warped_image = cv2.warpPerspective(
    overlay_image,
    transformation_matrix,
    (background_width, background_height),
)
cv2.imshow("Overlay image", warped_image)
cv2.waitKey(0)

The image is getting rotated and skewed properly (I think), but its not the same size as the screen. Its "shorter":

and if I use a different image that is very tall vertically I end up with something that is too "long":

Do I need to apply an additional transformation to scale the image? Not sure whats going on here, I thought the perspective transform would make the image automatically scale out to the provided coordinates.

I downloaded your image data and reproduced the problem in my local machine to find out the solution. Also downloaded lenna.png to fit inside the phone screen.

import cv2
import numpy as np

# Template image of iPhone
img1 = cv2.imread("/Users/anmoluppal/Downloads/46F1U.jpg")
# Sample image to be used for fitting into white cavity
img2 = cv2.imread("/Users/anmoluppal/Downloads/Lenna.png")

rows,cols,ch = img1.shape

# Hard coded the 3 corner points of white cavity labelled with green rect.
pts1 = np.float32([[201, 561], [455, 279], [742, 985]])
# Hard coded the same points on the reference image to be fitted.
pts2 = np.float32([[0, 0], [512, 0], [0, 512]])

# Getting affine transformation form sample image to template.
M = cv2.getAffineTransform(pts2,pts1)

# Applying the transformation, mind the (cols,rows) passed, these define the final dimensions of output after Transformation.
dst = cv2.warpAffine(img2,M,(cols,rows))

# Just for Debugging the output.
final = cv2.addWeighted(dst, 0.5, img1, 0.5, 1)
cv2.imwrite("./garbage.png", final)

這篇關于如何使用 Python 用新圖像替換圖像中的輪廓(矩形)?的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！