NixLynx/shape_matching
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add code and explaination to blog post

Browse source
This commit is contained in:
Lynx 2024-08-11 21:08:05 +02:00
parent 0830558cce
commit 3f8404d53e
2 changed files with 77 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
Readme.md
View file

@ -31,3 +31,7 @@
- The test images were generated using
- https://moonlit.technology/cqql/frost_patterns
# Build Blog
- pandoc --standalone blog.md -o shape_matching.html --metadata title="Shape Matching"

78
blog/blog.md
View file

@ -1,6 +1,6 @@
# Acknowledgements
I blame [cqql](https://tech.lgbt/@cqql) for this existing.
I blame [cqql](https://cqql.site) for this existing.
# Disclaimer
@ -9,10 +9,10 @@ I largely know nothing about this topic and everything you see here is knowledge
# The Idea
![hi! :neocat_floof:\nI'm looking for help with programmatic shape retrieval. I need to rank a couple million shapes by how similar each is to a shape given as input. \ndo you know anything about this or anyone that could help me? I'd really appreciate boosts! :blobcatheartR:](Idea.png)
![hi! :neocat_floof:\nI'm looking for help with programmatic shape retrieval. I need to rank a couple million shapes by how similar each is to a shape given as input. \ndo you know anything about this or anyone that could help me? I'd really appreciate boosts! :blobcatheartR:](Idea.png)
This is the post that threw me down this rabbit hole and started it all.
After doing a bit of searching around, for ways to maybe solve this problem I stumbled on opencv, which also happens to have a python port. Perfect for quickly hacking something together.
After doing a bit of searching around, for ways to maybe solve this problem I stumbled on OpenCV, which also happens to have a python port. Perfect for quickly hacking something together.
# The Process
@ -22,6 +22,19 @@ First I imported the image as grayscale and let OpenCV find the contours for me.
Which we can trace to form the complete outline as seen here:
![Contours](contours.png)
Actually Implementing this was as simple as
```py
def GetContoursFromFiles():
cnt = []
for i in range (0, MAX, 1):
img = cv2.imread(str(i)+'.png',0)
ret, thresh1 = cv2.threshold(img, GrayscaleThreshhold, 255,cv2.THRESH_BINARY_INV)
contours,hierarchy = cv2.findContours(thresh1,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
print("Number of Shapes detected in Image "+str(i)+":",len(contours))
cnt.append(contours[0])
return cnt
```
Next, I had OpenCV compare the different outlines with each other, which I used to build a NxN Matrix of weights.
(I love when libraries already do everything for me, especially all the math I wouldn't even know where to start with learning it).
Then I just had to pretty print the matrix and that could be considered Task Complete:
@ -36,6 +49,54 @@ So that's exactly what I did. I stitched together all the source images to form
But the result was well worth the effort:
![Graphical Matrix](Matrix.png)
So, what actually took all the effort?
This:
```py
def CreateMatchingColorMatrix(mat):
im_res = cv2.imread(str(0)+'.png',cv2.IMREAD_COLOR)
height, width, channels = im_res.shape
# (Coordinate 0/0) Color (white)
im_temp = np.full((height, width, 3), 255, np.uint8)
norm = matplotlib.colors.Normalize(vmin=0.0, vmax=NormUpperBound)
# Build Topmost row (just iterate through all images and concat them sequentially)
for i in range(0, MAX, 1):
img = cv2.imread(str(i)+'.png',cv2.IMREAD_COLOR)
imgg = cv2.imread(str(i)+'.png',0)
ret, thresh1 = cv2.threshold(imgg, GrayscaleThreshhold, 255,cv2.THRESH_BINARY_INV)
# find the contours in the binary image
contours,hierarchy = cv2.findContours(thresh1,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
img = cv2.drawContours(img,contours[0] , -1, (255, 0,0), 3)
im_temp = cv2.hconcat([im_temp, img])
# This top row is now our first row
im_res = im_temp
# Build The matrix row by row
for i in range(0, MAX, 1):
im_temp = cv2.imread(str(i)+'.png',cv2.IMREAD_COLOR)
img = np.full((height, width, 3), 255, np.uint8)
img[:] = (0, 0, 255)
# Individual row here, current sequential image gets chosen above, so here, we can do square coloring
for j in range(0, MAX, 1):
cmap = matplotlib.cm.get_cmap('brg_r')
cmap.set_over((0.0,0.0,0.0))
# Gets current weight, normalises it, looks it up in color map, converts it to full scale, colors
img[:] = NtoF(cmap(norm(mat[i*MAX+j]))[:-1])
# build up row
im_temp = cv2.hconcat([im_temp, img])
#build up matrix
im_res = cv2.vconcat([im_res, im_temp])
DebugDrawImage(im_res)
```
That's because I'm assembling the Matrix piece by piece and since I was learning as I went, I incrementally built it in such a way, that one part completely worked before moving on to the rest.
Specifically, I first tried to build the row and column headers, by filling all the other spaces with a dummy image 0.png. Then I had to figure out how to color them white. Afterwards, I tried to color code them, at which point I pulled in matplotlib for the colormaps. But because it uses different value ranged, I needed to do a bit of back and forth conversion, which led to this fun line which does all the heavy lifting for the coloring right here:
```py
# Gets current weight, normalises it, looks it up in color map, converts it to full scale, colors
img[:] = NtoF(cmap(norm(mat[i*MAX+j]))[:-1])
```
# Rotation
@ -56,7 +117,7 @@ To Demonstrate, here we have 45° Rotations of the same object, performed with d
![Rotation similarities displayed Graphically](rot2.png)
![Rotation similarities displayed using weights](rot2w.png)
The second algorithm seemed to perform more consistantly when using 45° rotations. Sadly, it completely failed with 22.5° rotations (which may be something I could fix, but I've already spent enough time on a "let's just do 10-15 minutes of research" project.)
The second algorithm seemed to perform more consistently when using 45° rotations. Sadly, it completely failed with 22.5° rotations (which may be something I could fix, but I've already spent enough time on a "let's just do 10-15 minutes of research" project.)
# Why write this post in the first place?
@ -76,4 +137,11 @@ I used the Frost Pattern Generator written by cqql for my sample images, as they
Now, keep in mind, this was quickly hacked together while I was learning how to even do this thing. With this disclaimer, the code can be found here:
[https://moonlit.technology/NixLynx/shape_matching](https://moonlit.technology/NixLynx/shape_matching)
(Make sure to follow the license)
(Make sure to follow the license :3)
``` {=html}
<style>
body { min-width: 40% !important; }
</style>
```