AIrt Critic

Complimentary critiques from our robot overlords

By Luke Watson-Davies

February 2, 2024

We had an AI-themed hack day as a lot of us at Cogapp have recently been dabbling in using Artificial Intelligence and Large Language Models like ChatGPT as tools within our work.

I was originally inspired to create an AI Art critique generator to compliment my 5-year-old daughter’s enthusiastic attempts at artwork… often revolving around the theme of unicorns, family & friends and of course poo. We had previously played around with using ChatGPT to help her stories come to life, she would give me the characters and the general narrative of a story and within a few iterations, we had delightful tales of prancing fairies going through portals and stealing gems from dragons.

The process

I planned out the steps of what I wanted to achieve within the day (well, a day and a half) as follows:

Set up a Next.js application
Add basic interface elements
Upload button for the user’s image
Display the AI-generated image-to-text description
Display the AI-generated Art critique text
Send this image to Replicate.com* and use the clip-interrogator model to generate a text description of the image
Send the generated description to OpenAI (the underlying technology that ChatGPT is built upon) and ask for an art critique

*Replicate.com allows running remote machine learning models with an API call or two.

As I cheated slightly and spent a little extra time on this during the week, I added some stretch goals:

Additional critic personas
Save the user’s image to AWS S3
Give users the ability to save their artwork and critique to a public gallery
Allow other visitors to view the gallery
Make it all pretty(ish)

The outcome

I asked for a few submissions from fellow Cogappers… oh, and I implemented a “Mean critic” who dislikes every piece of art they see.

The user interface of an application showing a pencil drawing on the left and a description of the artwork on the right

Here is Ben Rubinstein’s drawing of me and the generated text description that was generated with the clip-interrogator model:

“a child’s drawing of a man with glasses, professional portrait drawing, portrait of sam hyde, portrait of nick offerman, portrait of bob ross, rugged man portrait, portrait of rugged male ranger, portrait of an old man, accurate portrait of a bob dylan, man with beard, slavoj zizek, clothed non — nude portrait, portrait of rick sanchez”

And here’s the response from the “Mean critic”:

“This child’s drawing lacks skill and technique. The proportions are off, the lines are uneven, and there is no sense of depth or shading. It lacks any artistic merit or creativity.”

Here’s an example of the “Encouraging parent” mode on one of my daughter’s drawings:

The user interface of an application showing a child's drawing on the left and a description of the artwork description on the right

Generated description:

“a child’s drawing on a piece of paper, by Anato Finnstark, cg society contest winner, goddess of the sea, finnian in grungerock alien sky, over-the-shoulder shot, i_5589.jpeg, numerous dimly glowing eyes, she is happy, very blurry, fisherman, amber, in the cosmos, toddler”

Critique:

“Wow, this is such an amazing artwork, just like the paintings by the famous artist Anato Finnstark! I love how you drew the goddess of the sea in a grungerock alien sky. The numerous dimly glowing eyes really make her look magical and special. And the fisherman in the cosmos adds a sense of adventure to the whole scene. The blurriness of the drawing gives it a dreamy and mysterious feel. You did such an incredible job, my little artist!”

In the prompt to OpenAI to generate the critiques, I asked it for structured JSON data that included similar artists and artworks. That way I was easily able to display these next to the critique and provide onward journeys for those wishing to explore the recommendations. Here are the related artists and artworks mentioned above, she’s a child prodigy!:

“Vincent Van Gogh<br />Pablo Picasso<br />Starry Night by Vincent Van Gogh<br />Guernica by Pablo Picasso”

What I learnt

The mean critiques are the funniest, especially when putting photos of your colleagues through it. Here’s one I got for a photo of Tristan Roddis:

“The subject matter, a man standing in front of a chain link fence, is uninteresting and mundane.”

I need to find a more accurate image-to-text model, as the keyword stuffing that it did often threw the OpenAI-generated critique off
Speed feels like the biggest blocker to this being usable in a real-time manner at the moment. The time taken to generate the text description is often 1 minute and interrupts the fun. Whilst it’s possible to use better hardware for this, it would get quite expensive, fast