The results for "Mark Rothko", "Paintings by Mark Rothko", "Paintings similar to mark rothko" etc does not bring up anything that I was expecting. NGA has a large collection of Rothko paintings but none of them come up.
We are right now not including the artist name. Which will be done in the next iteration of the model (next week). Right now the search is only based on what the model can "see". And it seems like that the model does not understand the art of Mark Rothko.
The next version can see the image and read the metadata.
A bit more context: We are include everything in the latent space (embeddings) without trying to maintain multiple indexes and hack around things. There is still a huge mountain to climb. But this one seems really promising.
And this seems like a hard limitation of this approach as art (v craft) is concerned with interpretation and reception whereas this is more like unsplash-for-galleries in that the searches have to be very literal I guess? (eg search for something abstract, like 'dreams', something that you will find depicted in the collection, produces quite the mixed bag of results).
A search for : "character studies of old farmers" yielded good results.
The results are drawings / engravings, which may reflect the balance of the collection, and perhaps this subject is more used in practice than in marketable oil paintings.
Since this is a semantic search, using a vector embedding, it will handle meanings better than a text search, which would handle names better.
The results for "Mark Rothko", "Paintings by Mark Rothko", "Paintings similar to mark rothko" etc does not bring up anything that I was expecting. NGA has a large collection of Rothko paintings but none of them come up.
This NGA link returns over a thousand pieces by Rothko: https://www.nga.gov/artists/1839-mark-rothko/artworks
We are right now not including the artist name. Which will be done in the next iteration of the model (next week). Right now the search is only based on what the model can "see". And it seems like that the model does not understand the art of Mark Rothko.
The next version can see the image and read the metadata.
A bit more context: We are include everything in the latent space (embeddings) without trying to maintain multiple indexes and hack around things. There is still a huge mountain to climb. But this one seems really promising.
And this seems like a hard limitation of this approach as art (v craft) is concerned with interpretation and reception whereas this is more like unsplash-for-galleries in that the searches have to be very literal I guess? (eg search for something abstract, like 'dreams', something that you will find depicted in the collection, produces quite the mixed bag of results).
A search for : "character studies of old farmers" yielded good results. The results are drawings / engravings, which may reflect the balance of the collection, and perhaps this subject is more used in practice than in marketable oil paintings.
Since this is a semantic search, using a vector embedding, it will handle meanings better than a text search, which would handle names better.