Add code and explaination to blog post

This commit is contained in:
Lynx 2024-08-11 21:08:05 +02:00
parent 0830558cce
commit 3f8404d53e
2 changed files with 77 additions and 5 deletions

View file

@ -31,3 +31,7 @@
- The test images were generated using - The test images were generated using
- https://moonlit.technology/cqql/frost_patterns - https://moonlit.technology/cqql/frost_patterns
# Build Blog
- pandoc --standalone blog.md -o shape_matching.html --metadata title="Shape Matching"

View file

@ -1,6 +1,6 @@
# Acknowledgements # Acknowledgements
I blame [cqql](https://tech.lgbt/@cqql) for this existing. I blame [cqql](https://cqql.site) for this existing.
# Disclaimer # Disclaimer
@ -9,10 +9,10 @@ I largely know nothing about this topic and everything you see here is knowledge
# The Idea # The Idea
![hi! :neocat_floof:\nI'm looking for help with programmatic shape retrieval. I need to rank a couple million shapes by how similar each is to a shape given as input. \ndo you know anything about this or anyone that could help me? I'd really appreciate boosts! :blobcatheartR:](Idea.png) ![hi! :neocat_floof:\nI'm looking for help with programmatic shape retrieval. I need to rank a couple million shapes by how similar each is to a shape given as input. \ndo you know anything about this or anyone that could help me? I'd really appreciate boosts! :blobcatheartR:](Idea.png)
This is the post that threw me down this rabbit hole and started it all. This is the post that threw me down this rabbit hole and started it all.
After doing a bit of searching around, for ways to maybe solve this problem I stumbled on opencv, which also happens to have a python port. Perfect for quickly hacking something together. After doing a bit of searching around, for ways to maybe solve this problem I stumbled on OpenCV, which also happens to have a python port. Perfect for quickly hacking something together.
# The Process # The Process
@ -22,6 +22,19 @@ First I imported the image as grayscale and let OpenCV find the contours for me.
Which we can trace to form the complete outline as seen here: Which we can trace to form the complete outline as seen here:
![Contours](contours.png) ![Contours](contours.png)
Actually Implementing this was as simple as
```py
def GetContoursFromFiles():
cnt = []
for i in range (0, MAX, 1):
img = cv2.imread(str(i)+'.png',0)
ret, thresh1 = cv2.threshold(img, GrayscaleThreshhold, 255,cv2.THRESH_BINARY_INV)
contours,hierarchy = cv2.findContours(thresh1,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
print("Number of Shapes detected in Image "+str(i)+":",len(contours))
cnt.append(contours[0])
return cnt
```
Next, I had OpenCV compare the different outlines with each other, which I used to build a NxN Matrix of weights. Next, I had OpenCV compare the different outlines with each other, which I used to build a NxN Matrix of weights.
(I love when libraries already do everything for me, especially all the math I wouldn't even know where to start with learning it). (I love when libraries already do everything for me, especially all the math I wouldn't even know where to start with learning it).
Then I just had to pretty print the matrix and that could be considered Task Complete: Then I just had to pretty print the matrix and that could be considered Task Complete:
@ -36,6 +49,54 @@ So that's exactly what I did. I stitched together all the source images to form
But the result was well worth the effort: But the result was well worth the effort:
![Graphical Matrix](Matrix.png) ![Graphical Matrix](Matrix.png)
So, what actually took all the effort?
This:
```py
def CreateMatchingColorMatrix(mat):
im_res = cv2.imread(str(0)+'.png',cv2.IMREAD_COLOR)
height, width, channels = im_res.shape
# (Coordinate 0/0) Color (white)
im_temp = np.full((height, width, 3), 255, np.uint8)
norm = matplotlib.colors.Normalize(vmin=0.0, vmax=NormUpperBound)
# Build Topmost row (just iterate through all images and concat them sequentially)
for i in range(0, MAX, 1):
img = cv2.imread(str(i)+'.png',cv2.IMREAD_COLOR)
imgg = cv2.imread(str(i)+'.png',0)
ret, thresh1 = cv2.threshold(imgg, GrayscaleThreshhold, 255,cv2.THRESH_BINARY_INV)
# find the contours in the binary image
contours,hierarchy = cv2.findContours(thresh1,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
img = cv2.drawContours(img,contours[0] , -1, (255, 0,0), 3)
im_temp = cv2.hconcat([im_temp, img])
# This top row is now our first row
im_res = im_temp
# Build The matrix row by row
for i in range(0, MAX, 1):
im_temp = cv2.imread(str(i)+'.png',cv2.IMREAD_COLOR)
img = np.full((height, width, 3), 255, np.uint8)
img[:] = (0, 0, 255)
# Individual row here, current sequential image gets chosen above, so here, we can do square coloring
for j in range(0, MAX, 1):
cmap = matplotlib.cm.get_cmap('brg_r')
cmap.set_over((0.0,0.0,0.0))
# Gets current weight, normalises it, looks it up in color map, converts it to full scale, colors
img[:] = NtoF(cmap(norm(mat[i*MAX+j]))[:-1])
# build up row
im_temp = cv2.hconcat([im_temp, img])
#build up matrix
im_res = cv2.vconcat([im_res, im_temp])
DebugDrawImage(im_res)
```
That's because I'm assembling the Matrix piece by piece and since I was learning as I went, I incrementally built it in such a way, that one part completely worked before moving on to the rest.
Specifically, I first tried to build the row and column headers, by filling all the other spaces with a dummy image 0.png. Then I had to figure out how to color them white. Afterwards, I tried to color code them, at which point I pulled in matplotlib for the colormaps. But because it uses different value ranged, I needed to do a bit of back and forth conversion, which led to this fun line which does all the heavy lifting for the coloring right here:
```py
# Gets current weight, normalises it, looks it up in color map, converts it to full scale, colors
img[:] = NtoF(cmap(norm(mat[i*MAX+j]))[:-1])
```
# Rotation # Rotation
@ -56,7 +117,7 @@ To Demonstrate, here we have 45° Rotations of the same object, performed with d
![Rotation similarities displayed Graphically](rot2.png) ![Rotation similarities displayed Graphically](rot2.png)
![Rotation similarities displayed using weights](rot2w.png) ![Rotation similarities displayed using weights](rot2w.png)
The second algorithm seemed to perform more consistantly when using 45° rotations. Sadly, it completely failed with 22.5° rotations (which may be something I could fix, but I've already spent enough time on a "let's just do 10-15 minutes of research" project.) The second algorithm seemed to perform more consistently when using 45° rotations. Sadly, it completely failed with 22.5° rotations (which may be something I could fix, but I've already spent enough time on a "let's just do 10-15 minutes of research" project.)
# Why write this post in the first place? # Why write this post in the first place?
@ -76,4 +137,11 @@ I used the Frost Pattern Generator written by cqql for my sample images, as they
Now, keep in mind, this was quickly hacked together while I was learning how to even do this thing. With this disclaimer, the code can be found here: Now, keep in mind, this was quickly hacked together while I was learning how to even do this thing. With this disclaimer, the code can be found here:
[https://moonlit.technology/NixLynx/shape_matching](https://moonlit.technology/NixLynx/shape_matching) [https://moonlit.technology/NixLynx/shape_matching](https://moonlit.technology/NixLynx/shape_matching)
(Make sure to follow the license) (Make sure to follow the license :3)
``` {=html}
<style>
body { min-width: 40% !important; }
</style>
```