The Zen of Compression

Our world is in peril

Our planet is in trouble, and we as software engineers aren't helping. research suggests that, the positive environmental impact of the advances and power efficiency, and technologies like virtualization. Is being erased by the software industry's ever increasing demand for resources.

This may seem to be a systemic and unsolvable problem that we as individual developers are powerless to solve. The real solution is simple, we as software engineers must learn to live with less. Digital scarcity, is not new. In fact it is one of the oldest problems in computing. If our ancient 12bit ancestors could write whole operating systems in four kilo words of memory, surely we can do better. That's the topic of today's article, where I ask.

How can we live with less on the, web while still creating aesthetically pleasing user experince?

AND

Can jpeg save the planet?

Solution?

The image at the top of this post, is a full fat 4k portable network graphic image with alpha transparency. And it weighs in add an astonishing 4.5MB. Or to put it in real terms more ram than my family's first computer had when it was new. Yikes, we need some size reduction, and fast.

One of the fastest ways to reduce image size is to convert PNGs to to jpegs. In modern javascript this is fairly simple using the sharp library.

const sharp = require("sharp")
inImage = sharp("IMG_0713.png")
inImagt.jpeg().toFile("713.jpeg")

Et vollia, an order of magnitude size reduction on the default settings and I still look fly. And still technically in 4K. The software developers of the nineties really did know what they were doing.

We're down to 480KB My example code wraps sharp in a web interface for convenience of experimentation. But you could just as easily do this as a command line application or whatever. I think the more important question is why does this work, and more importantly can we abuse the algorithm to get smaller, and more aesthetically pleasing results?

Why it works

It comes down to compression methods, PNG's compression is designed to be lossless, and works on a per pixel basis. At an absurd level of abstraction, it is basically a highly optimized form of gzip. On the other hand jpeg's compression algorithm is lossy. Basically it converts regions that have similar pixels, into tiles and rebuilds the image from those tiles upon render. You can think of it as analogous to how a retro game console such is the super Nintendo, or sega mega drive handles the background layer.

Also worth mentioning the jpegs color model is only RGB as opposed to RGB with alpha. So there is some size reduction based on the color pallate.

How low can we go?

You might be tempted not unreasonably, to think that the default settings are good enough an order of magnitude it's plenty for everybody. But I think we can do better. Here's how there are two settings of interest in the encoder, that control how size works out basically. The quality setting, and chroma Subsampling.

Quality is basically a proxy setting for a bunch of fiddly bits, that control how similar an image region needs to be in order to be converted to a tile. The lower it goes, lower the threshold of similarity needs to be met. If we set it at about 50, the image is free of compression artifacts while having a pleasant retro asthetic.

Chroma subsampling controls how much of the color data is discarded during compression. There are basically three settings which are safe to use, in the sense they don't produce error prone images that refuse to render which are.

4:4:4: All color data retained (24 bit) 4:2:2: Color reduced to 16bits 4:2:0: color reduced to 8bits (256 colors total)

There are also safe values 4:1:1 and 4:1:0. These have something to do with luminance, and I couldn't really distinguish them from 4:2:0 impact on size was negligible. For completeness I'll mention the interlacing setting, but experimenting with this is left is an exercise for the reader.

inImage.jpeg({
     quality: 50,
     chromaSubsampling: '4:2:0'
   })

311KB And giving off distinct Win95 vibes, still in 4K. Aren't mashups of modern and retro technology wonderful.

limbo lower

The most obvious way to get the image sizes down even further, without resorting to shenanigans. Is to curtail color. Color in images is like yeast in bread. It increases enjoyment immensely, but going without it sometimes is good for the soul.

inImage.toColorspace('grey16')
inImage.jpeg({
      quality: 50,
      chromaSubsampling: '4:2:0'
    })

Shaves nearly 70KB off the image 249KB we're going places. And still 4k resolution. All this is great. but.

Honey I broke the compression model!

For any further shenanigans to work effectively we need to switch to my native language which is python. I created the same web interface and gave it the same functionality this time using the python imaging library. PIL. Similar results were achieved. I did reduce the grayscale image in size, by converting its color palette from sixteen shades of gray to four. shaved about 10KB compared to the javascript, and was more aesthetically pleasing in my opinion. But this is my most dubious change from the standpoint of the scientific method.

End of the line?

And then I remembered a technology used to use for image rendering on Apple IIs and other eight and sixteen bit computers. Called dithering.

Dithering reduces the color palette to a subset of the images full colors, and imposes a dot matrix on the image so that, you have the illusion of color depth, through the magic of psychology.

In fact the jpeg encoder still supports this, but the algorithm it uses to impose the dot matrix, Floyd–Steinberg dithering. Is designed to be as subtle and CPU efficient as possible, and looks almost uncanny valley in my opinion.

There are other algorithms, and after some experimentation and a look at other software artists that do this. I settled on Bayer Dithering.

adding a simple package from PyPi called, appropriately enough dithering. Makes this relatively simple.

 
m = im.convert('RGB')
img_dith_a = ordered_dither(np.array(im), "Bayer4x4") # the headfake
img_dith = Image.fromarray(img_dith_a)

A Blue owl, in a santa suit, comic book style

This looks amazing, it has the vibe of a comic book or pulp magazine from a hundred years ago. One couldn't ask for a better aesthetic.

One problem, our image size has increased it's 1.1MB. Why is this. Well friends we've broken the compression model, by dithering we have unwittingly broken, that homogeneity among image regions, which jpeg relies on to produce an efficient result.

However if we switch to a more pixel based and lossless model, such as PNG... Our results are staggering.The PNG version of the image clocks in at 270KB in color, which nearly matches our first round grayscale result. And the greyscale comes in at 116KB. Better than anything we've been able to achieve with jpeg.

Final Thoughts

We've come full circle, now what. The main takeaway from this article, is that it's not the compression algorithm that matters, but how you use it. I think we've also demonstrated something else, as well.

That it is possible to create, a pleasing user experience using fewer resources. this is what we must do, if we as software engineers want to start making the world better instead of worse.