INTRO

MoMA (The Museum of Modern Art) is a cultrual epicenter in New York City. With their extensive collection, MoMA is an incredble destination for art lovers. MoMA publishes their collection data on Github, making it easy to work with. After my many years living in New York City, I've always wanted to create an analysis with the MoMA collection data. Today, we dive into an exploration in colors, using the paintings from the collection to find new artists.

EARLY EXPLORATIONS

I actually started this project many years ago, in an effort to programatically create Image Quilts, a concept promoted by Edward Tufte. While preliminary results were okay, but there were a couple issues with this approach:

1. Observable notebooks were not performant enough to handle such a large dataset.
2. CORS errors made it hard to scale.
3. The method for identifying colors used binning to make a new primary color. This meant that generated colors colors from artworks may not actually exist in the paiting.

I knew that to do this better, I'd have to try a different solution, one day...

FAST FORWARD

I had the great opportunity to work on the Observable team, and while there, we launched Observable Framework – a static site generator for building data apps. It would be a good choice for this project, as it would address the previous concerns.

I migrated my earlier work from notebooks to see if I could work with larger datasets, like analyze _every_ pixel in an artwork, rather that binning results. I began to sort these colors by different methods – redness, blueness, greenness (rgb), hue, saturation, lightness (hsl), etc. but none of it worked well. No matter what sorting logic I used, I would get strange artifacts, little dashes of hues that appeared in the wrong place.

There was an article by the New York Times called The True Colors of America's Political Spectrum Are Gray and Green, where they do a similar analysis of organizing colors in two-dimensions, and I was surprised to see the same sorting issues. If you look closely at this sorted picture of Lido Isle, you'll find speckles of orange mixing randomly within gray sections.

If the New York Times couldn't figure it out, maybe sorting colors in two dimensions was a hopeless effort...

MUNSELL COLOR SYSTEM

After trying what felt like a hundred different sorting methods, I remembered this great visualization I found years ago of the Munsell Color System. From Wikipedia:
Color is by nature three dimensional – whether you're looking at RGB values, HCL, LAB. Organizing color within two dimensions will inherently cause problems, while three dimensions can create a more intuitive layout.

While this analysis is conceptually straightforward, implementation proved difficult. The pixel data in these artworks is quite large, and visualizing all those pixels at once can be taxing for the browser. I tried Zdog andThree.js, until I figured out a solution using re.gl, a light-weight open-source library for building with webgl. This proved effective for visualizing hundreds of thousands of points perfomantly.

Click the play button to see the results.

I've been calling this visualization a Munsell Plot. It's a novel way of analyzing color distributions. Pixels are sorted along the Munsell Color System, then sized by their frequency. The outer cube shows the possible extent of values, and the inner cube shows the extent from that artwork.

Let's compare to Andy Warhol's Lita Curtain Star [Lita Hornick], and see very different distribution of color, with an explosion of neon pinks, yellows, greens, teals, and more uniform distribution across colors.

It appeared the Munsell Color System worked for sorting colors in an effective way. It made sense when you looked at it – colors appeared intuitively, and it allowed you to see interesting distributions that were difficult to ascertain from the original work.

MoMA IN MUNSELL

We can apply a Munsell Plot to not only a single artwork, but to a collection of artworks. Here, I've visualized the distribution of color across all paitings in the MoMA collection that are on-display. You'll see that there are significant trends into which colors are use more predominately – reds, oranges, yellows have higher frequencies than your blues, purples and teals. In general, you see a lack of neon colors, like we saw in the Warhol examples, since most of those hues only occur in more recent paintings.

While these visualizations were cool (at least I thought so!), where they effective? Did they provide value into understanding artworks? What could they be used for?

We talked to friends in the art community, and their feedback was incredibly helpful - while this is interesting, it's perhaps just a step in where you can dervive value. Could we use this analysis to find similar artworks? People have often looked for art based on color, but could you search for art based on color distributions? Could you uncover underrepresented artists by analyzing color?

SIMILAR PAINTINGS

Let's show how this works. We can compare the distributions of color for each painting to each painting to develop a similarity score. Here are two paintings that scored high in their similarity score, and while you may not think about these two paintings as similar given their topics, the artists, the time period, you can see from their distributions that they both have lots of very similar colors.

Here's another example of two paintings that scored high on the similarity scale. The paintings here are Piet Mondrian's Composition in Brown and Gray and Diego Rivera's Young Main in a Gray Sweater (Jacques Lipchitz). I love how for these two paintings, you can see the clear area where they are similar and where Rivera's work departs from the main distrubtions with it's accents of blue and red.

Now that you've learned about the inspiration and analysis, go explore paintings from the MoMA collection and find similar artworks based on their color distributions!