Displaying Images on Framebuffers with Pipelines

Overview Link to heading

The audience for this guide is for beginner, but not an absolute beginner: This guide assumes that you are familiar with the following:

Software Basic terminal usage, including using a text editor and installing software through apt. Hardware Basic understanding of interfacing with Raspberry Pi gpio.

This should work on any raspberry pi. I only tested with a 4B and Zero 2 W. For this guide I used the latter.

This was my software setup:

         _,met$$$$$gg.           user@pi
      ,g$$$$$$$$$$$$$$$P.        OS: Debian 12 bookworm
    ,g$$P""       """Y$$.".      Kernel: aarch64 Linux 6.6.51+rpt-rpi-v8
   ,$$P'              `$$$.      Uptime: 12h 33m
  ',$$P       ,ggs.     `$$b:    Packages: 1594
  `d$$'     ,$P"'   .    $$$     Shell: bash 5.2.15
   $$P      d$'     ,    $$P     Disk: 5.4G / 15G (39%)
   $$:      $$.   -    ,d$$'     CPU: ARM Cortex-A53 @ 4x 1GHz
   $$\;      Y$b._   _,d$P'      RAM: 137MiB / 416MiB
   Y$$.    `.`"Y$$$$P"'
   `$$b      "-.__
    `Y$$
     `Y$$.
       `$$b.
         `Y$$b.
            `"Y$b._
                `""""

🔴 This is a continuation of a previous tutorial covering displaying images to an SSD1351 display using linux framebuffers. Make sure you go in order, each one builds on top of the last.

If you don’t understand pipelines already, there’s no better explanation than the manual, I suggest you read it. Otherwise, below is my take on explaining it but feel free to skip to the next part.

For this article, you only need to know that a pipe (| symbol) lets the output of one command feed into another command. How is that helpful and what does it do? I think by example makes the most sense:

Let’s try to find how many unique words are in the book Moby Dick.

Make a directory to not have clutter:

mkdir ~/pipeline-example
cd ~/pipeline-example

Download a .txt file of the book locally

# Save to mobydick.txt
wget -O mobydick.txt https://www.gutenberg.org/cache/epub/2701/pg2701.txt

First let’s think about the problem. I know I will probably need to run a program called uniq, which will remove duplicate lines in a file. The problem is that we want duplicate words to be removed, so we need to put each word in it’s on line before calling uniq.

# This won't do anything, besides print every unique line, which is the entire book.
uniq mobydick.txt

Instead we want to turn output that looks like this:

When this last task was accomplished it was noon, and the seamen went
below to their dinner. Silence reigned over the before tumultuous but
now deserted deck. An intense copper calm, like a universal yellow
lotus, was more and more unfolding its noiseless measureless leaves
upon the sea.

Into this:

When
this
last
task
was
accomplished
it
was
noon
and
the
seamen
...

To start, lets just print the entire book to the terminal with the cat command:

cat mobydick.txt
You should see a nice big wall of text

Now, let’s try to replace ever empty space character with a newline character. To do this let’s use the sed command. Let’s tell sed to replace space with newline. We can use the “substitue” command with the following format "s/wordsearched/wordreplace/g". Our word searched should be an empty space, and our word to replace should be the newline character \n. Giving the following command:

sed "s/ /\n/g" mobydick.txt

Great! It’s printing everything on a newline. However, there is a problem. The punctuation is still included, and that will mess with our results. Luckily, there’s another command tr or, “translate”. tr has a built in function to remove punctuation from text tr --delete '[:punct]'. Now this is where pipelines come in handy. Let’s use cat tr in combination with a pipe | to redirect the output of cat into the input of tr.

cat mobydick.txt | tr --delete '[:punct:]'
You should see the book without any punctuation.

We can then pipe the output of cat mobydick.txt | tr --delete '[:punct:]' into the previous sed command to put every word on a line.

cat mobydick.txt | tr --delete '[:punct:]' | sed "s/ /\n/g"

We are not done yet! Let’s keep up the momentum and make every letter lowercase to prevent duplicate entries. To do this, we can pipe the output into another tr command:

cat mobydick.txt | tr --delete '[:punct:]' | sed "s/ /\n/g" | tr [:upper:] [:lower:]

Now let’s put this output into a command to sort every line, as required by uniq. Again we pipe the output into a function, we can sort lines with the sort command:

cat mobydick.txt | tr --delete '[:punct:]' | sed "s/ /\n/g" | tr [:upper:] [:lower:] | sort

Again, we further need to process this output. We can finally use that uniq command to remove all duplicate lines (words).

cat mobydick.txt | tr --delete '[:punct:]' | sed "s/ /\n/g" | tr [:upper:] [:lower:] | sort | uniq

Now let’s wrap it all up by using wc to count all the words left after running the pipeline:

cat mobydick.txt | tr --delete '[:punct:]' | sed "s/ /\n/g" | tr [:upper:] [:lower:] | sort | uniq | wc --word

You should see 25731

Sweet! That’s it. Pipelines are extremely powerful tools and I’m sure you can imaging how many uses it could have processing text. Let’s utilize pipelines in our linux framebuffer project.

Using Pipelines for Image Processing for our Linux Framebuffer Link to heading

Remember the previous python script? With little effort we can modify this script to utilize pipelines. Let’s just put it into a new file:

touch convert-stream-rgb565.py # create file
chmod +x convert-stream-rgb565.py # mark as executable
nano convert-stream-rgb565.py # Edit the file

Paste the following:

#!/usr/bin/env python3 

# convert-stream-rgb565.py

import sys
import struct

def convert_rgb_to_rgb565(input_stream, output_stream):
    while True:
        # Read 3 bytes (1 RGB pixel)
        rgb_data = input_stream.read(3)
        if len(rgb_data) < 3:
            break

        # Extract RGB components
        r, g, b = rgb_data[0], rgb_data[1], rgb_data[2]

        # Convert to RGB565 format
        rgb565 = ((r >> 3) << 11) | ((g >> 2) << 5) | (b >> 3)

        # Write RGB565 as 16-bit little-endian
        output_stream.write(struct.pack('<H', rgb565))

if __name__ == "__main__":
    try:
        convert_rgb_to_rgb565(sys.stdin.buffer, sys.stdout.buffer)
    except BrokenPipeError:
        pass

If you notice, we no longer have hardcoded values for the names of the files. Instead we take an input and an output stream as arguments to the function, and pass stdin for reading stdout for writing.

For those unfamiliar, stdout can be thought of the output leaving a program in a pipeline and stdin as the input stream into the next program.

So, we are now able to read a stream of 8-bit image data into the program and write out a stream of 5/6/5 image data with our python script.

Putting It All Together Link to heading

Let’s use our previous ImageMagick command with our tux.png image from the first tutorial.

convert tux.png -resize 128x128! -depth 8 rgb:tux.rgb

We can use a little “trick” to output the raw image data into stdout (print it to the terminal).

# You CAN run this, but it will spit out a jumbled mess of characters since the data is not supposed to be readable
convert tux.png -resize 128x128! -depth 8 rgb:-

Now, we can use a pipe to pass it into our recently modified python script `convert

Next, create a file named convert-for-screen.sh and use your favorite text editor to add the following:

# This will also print out unreadable data
convert tux.png -resize 128x128! -depth 8 rgb:- | ./convert-stream-rgb565.py

Now, you can finally write to the framebuffer by redirecting the output with the > character into /dev/fb0:

# This will also print out unreadable data
convert tux.png -resize 128x128! -depth 8 rgb:- | ./convert-stream-rgb565.py > /dev/fb0

That’s it! You should see the image on your display if everything went well. Let’s finish up by writing a script that can take the image name as an argument, so we can remove the hardcoded tux.png.

Open a terminal and perform the following

touch display-image.sh # create file
chmod +x display-image.sh # mark as executable
nano display-image.sh # Edit the file

Paste the script contents:

#!/bin/bash

convert "$1" -resize 128x128! -depth 8 rgb:- | ./convert-stream-rgb565.py > /dev/fb0
Notice that the only difference is the first line #!/bin/bash which tells the computer to run this as a bash file, and the "$1" instead of tux.png. "$1 is just a way of getting the second parameter in a command.

For example with the command cat mytext.txt if you were to print "$1" it would give “mytext.txt”.

Now you should be able to run this script on any image, of most any format thanks to ImageMagick:

display-image.sh coolpicture.png

I put some pictures on a timer, sorry for the bad gif:

examplegif

Congrats! You can now display any image easily thanks to the power of pipelines. In a future post I’ll get into how we can add a few more steps and get a (albeit choppy) video going.

Overview Link to heading

Pipelines Link to heading

Using Pipelines for Image Processing for our Linux Framebuffer Link to heading

Putting It All Together Link to heading