Can Programming Be Easier?

I like creating diy projects. Most of them involve writing some code.

Note: Hereafter, by using the expression « friends » to refer to people that have some basic computer and logical skills. They are not at ease with a programming language, yet they can easily understand the concepts underneath.

When I discuss those with non programmer friends, this is basically how it goes:

friend: This project looks cool. Too bad I cannot do the same.
me: Well, it’s not complicated, you totally can!
friend: Well… I don’t know… Really?
me: For sure!
friend: Where should I start?
me: Let’s me think… Do you know about git? python? raspberry pi? do you have an IDE?
friend: …

I think that I experience here the curse of knowledge. I’ve become blind about the obstacles people have to overcome.

A few days ago, a similar conversation took place. It was about making photographs appear along with a GNSS track, on a map.

Once again, I felt like it was easy, but my friend did not. I confidently suggested I would provide a minimal example to get my friend started. I tried to do something and eventually did not manage to make it as simple as I wanted it to be.

So, let’s decompose this project to find out how I do it and why it is not that easy.

the story

Let’s consider those pictures of mushrooms, that were taken during a walk whose gpx track is here.

Here comes the first difficulty I can spot.

Difficulty: one that wants to create such a project must be aware that gnss positions need to be stored somehow and that there exist formats of data − gpx being one of them − to do so.

Let’s try to put those photographs on a map, along with the track.

The wanted end result should look like this.

setting things up

I will use a python virtual environment for the sake of this article. This is not something important do understand to realize this project. I am just showing this so that the references to venv bellow don’t appear like coming from nowhere.

python -m venv venv

installing the dependencies

I need libraries to:

read gpx data: gpxpy
generate a map: folium
deal with the subtelties of datetime localization: pytz
show a progress bar when the code takes some time computing stuff: tqdm

Difficulty: here is to find libraries that will help me. In the end, I don’t much more than gathering libraries made by clever people and assemble them together in a way that suits me.

./venv/bin/pip install gpxpy folium pytz tqdm

I also need a few command line tools to create the thumbnails and medium size images and to extract the exif data from images.

Difficulty: using command line tools. As a linux/debian user, I use those without even thinking I do. But for windows non programmer users, it might be more complicated to have such tooling.

sudo apt install imagemagick extract

gathering the input files

Let’s assume the images are in the images folder.

ipfs get https://konubinix.eu/ipfs/bafybeigwxsg6hc7tsbcrxtj7lqdu4evlkptz7cc7uf3sc22vrw4vmupixe -o images

And that the gpx data is in track.gpx

ipfs get https://konubinix.eu/ipfs/bafkreigktftwpvrc5wlydn3vb2phzqootei2jkgc6rhhcnfaiqxj7mq4t4 -o track.gpx

creating the images that will actually be shown

A real size picture is generally too big to be downloaded and rendered in a browser in a smooth way.

I will create thumbnails that will be shown on the map and medium sized version that will appear when clicking on the thumbnail.

Now, let’s first create thumbnails of the images.

images=($(ls images))
mkdir -p dist/thumbnail dist/medium
for image in "${images[@]}"
do
    convert images/${image} -scale 256 dist/thumbnail/${image}
    convert images/${image} -scale 1024 dist/medium/${image}
done

Difficulty: one needs to be at ease with scripting language to easily create those. We can also assume that the person doing this project uses a graphical tools that perform batch conversions, so this might be not a big difficulty.

Now, we have everything we need to start programming.

Let’s take a look at how are organized the files.

find -name venv -prune -o -type f -print

./images/IMG_20231001_160015.jpg
./images/IMG_20231001_162837.jpg
./images/IMG_20231001_172833.jpg
./images/IMG_20231001_155909.jpg
./images/IMG_20231001_160838.jpg
./images/IMG_20231001_160832.jpg
./images/IMG_20231001_162651.jpg
./images/IMG_20231001_155226.jpg
./images/IMG_20231001_172703.jpg
./images/IMG_20231001_163023.jpg
./images/IMG_20231001_155532.jpg
./images/IMG_20231001_172856.jpg
./images/IMG_20231001_170226.jpg
./dist/thumbnail/IMG_20231001_160015.jpg
./dist/thumbnail/IMG_20231001_162837.jpg
./dist/thumbnail/IMG_20231001_172833.jpg
./dist/thumbnail/IMG_20231001_155909.jpg
./dist/thumbnail/IMG_20231001_160838.jpg
./dist/thumbnail/IMG_20231001_160832.jpg
./dist/thumbnail/IMG_20231001_162651.jpg
./dist/thumbnail/IMG_20231001_155226.jpg
./dist/thumbnail/IMG_20231001_172703.jpg
./dist/thumbnail/IMG_20231001_163023.jpg
./dist/thumbnail/IMG_20231001_155532.jpg
./dist/thumbnail/IMG_20231001_172856.jpg
./dist/thumbnail/IMG_20231001_170226.jpg
./dist/medium/IMG_20231001_160015.jpg
./dist/medium/IMG_20231001_162837.jpg
./dist/medium/IMG_20231001_172833.jpg
./dist/medium/IMG_20231001_155909.jpg
./dist/medium/IMG_20231001_160838.jpg
./dist/medium/IMG_20231001_160832.jpg
./dist/medium/IMG_20231001_162651.jpg
./dist/medium/IMG_20231001_155226.jpg
./dist/medium/IMG_20231001_172703.jpg
./dist/medium/IMG_20231001_163023.jpg
./dist/medium/IMG_20231001_155532.jpg
./dist/medium/IMG_20231001_172856.jpg
./dist/medium/IMG_20231001_170226.jpg
./track.gpx

the code

extracting data from images

I need some code that, given an image location, gets

the date it was shot,
the location of its thumbnail version,
the location of its medium sized version.

from subprocess import check_output
from datetime import datetime
from pathlib import Path
import pytz
import time

class Image:
    def __init__(self, location):
        self.location = Path(location)
        self.thumbnail_location = f"thumbnail/{self.location.name}"
        self.medium_location = f"medium/{self.location.name}"

    @property
    def _extracted_lines(self):
        return check_output(
            ["extract", self.location],
        ).decode().splitlines()[1:] # getting past the header

    @property
    def _extracted_data(self):
        return {
            line.split(" - ")[0].lower(): line.split(" - ")[1] for line in self._extracted_lines
        }

    @property
    def date(self):
        naive_date = datetime.strptime(self._extracted_data["creation date"], "%Y:%m:%d %H:%M:%S")
        return pytz.timezone(time.tzname[0]).localize(naive_date, is_dst=False)

Difficulties: There are several difficulties that we can see here:

I chose to implement the algorithm using classes because it feels natural to me, yet it might make the code harder to read for people not used to it,
calling and extracting the output of the extract command need several trials and errors. It might also feel less than obvious for people not used to it,
the date needs to be parsed and then localized. Even among programmers, date localization is a complicated topic,

extracting data from the gpx file

Now, I need an object that can extract the data of my gpx track. I need:

the bounding box (in latitude and longitude) of the data so that I will be able to zoom the map appropriately in the later code,
extract the points and segments that make the tracks of the gpx file.
provide a way to get coordinates (latitude, longitude), provided a date. This will be used later when positioning the images on the map.

The algorithm I thought about to get the coordinates is very simple: find the latest point that is before the given date and return its location. This is very simple and one could easily think of a more precise one, but this is not the point (no pun intended) of this article.

from pathlib import Path
import gpxpy

class GPX:
    def __init__(self, location):
        self.location = Path(location)
        self.data = gpxpy.parse(self.location.read_text())
        self.min_lat = min(point.latitude for point in self.points)
        self.max_lat = max(point.latitude for point in self.points)
        self.min_lon = min(point.longitude for point in self.points)
        self.max_lon = max(point.longitude for point in self.points)

    def get_coord(self, date):
        points_before_date = sorted([point for point in self.points if point.time < date], key=lambda point: point.time)
        matching_point = max(points_before_date, key=lambda point: point.time)
        return (matching_point.latitude, matching_point.longitude)

    @property
    def tracks(self):
        yield from self.data.tracks

    @property
    def segments(self):
        for track in self.tracks:
            yield from track.segments

    @property
    def points(self):
        for segment in self.segments:
            yield from segment.points

Difficulties: Once again, this is a short code, but not without issues to understand

a gpx file is composed of tracks, that are composed of segments that are composed of points. As a mentioned above, I discovered this by playing using ipython.
sorting and getting the max of points using a custom key might be hard to grasp,
python stuffs, like yield from or property. Yet this is more an opinionated choice rather than an intrinsic difficulties of this project,

creating the map

Now, we want to read all the images, the gpx file and create a nice map out of it.

import folium
from pathlib import Path
from folium.plugins import MarkerCluster
from tqdm import tqdm

def main():
    gpx = GPX("track.gpx")

    center = ((gpx.max_lat + gpx.min_lat) / 2, (gpx.max_lon + gpx.min_lon) / 2)

    map = folium.Map(
        location=center,
        control_scale=True,
        max_zoom=19,
    )

    map.fit_bounds([(gpx.min_lat, gpx.min_lon), (gpx.max_lat, gpx.max_lon)])

    for segment in tqdm(gpx.segments, "Drawing segments"):
        locations = [(p.latitude, p.longitude) for p in segment.points]
        if not locations:
            print("Got a segment without locations, going on")
            continue
        folium.PolyLine(locations=locations).add_to(map)

    mCluster = MarkerCluster(name="Cluster",
                             options={
                                 "maxClusterRadius": 30,
                                 "spiderfyDistanceMultiplier": 4,
                             }).add_to(map)

    images = [Image(image) for image in Path("images").glob("*")]

    for image in tqdm(images, "Drawing images"):
        folium.CircleMarker(
            fill=True,
            location=gpx.get_coord(image.date),
            fill_color="red",
            popup=folium.Popup(
                f'<img src="{image.thumbnail_location}" onclick="img_box(\'{image.medium_location}\')")/>',
                lazy=True)).add_to(mCluster)

    map_root = map.get_root().render()

    header = map.get_root().header.render()
    body_html = map.get_root().html.render()
    script = map.get_root().script.render()

    result = f"""
    <!DOCTYPE html>
       <html>
           <head>
               <title>Example of map</title>
               {header}
               <script src="https://konubinix.eu/ipfs/bafkreiaxghyfbzfnpq7rkakxswmdihnutkfhjjfw7br5fwoyumaovncczu?img_box.js"></script>
           </head>
           <body>
               {body_html}
               <script>
                 {script}
               </script>
           </body>
       </html>
            """

    Path("dist/index.html").write_text(result)

Difficulties:

understanding how folium works, with a map and stuff that adds to it,
understanding how to write html,

Once again, I did not come to this code at first. I tried a lot, using ipython. The fact that the map has a method called show_in_browser helps a lot, as I could try things and see the result in the browser very easily.

For instance, I did not think of using tqdm to show a progress bar at first, but the fact it took several seconds to compute made me want the program to show that it was not stuck.

Also, the use of MarkerCluster, a custom html code (you could instead use map.save("dist/index.html")) to use img_box.js came along the way.

all in one

This is what the final code should look like.

from subprocess import check_output
from datetime import datetime
from pathlib import Path
import pytz
import time

class Image:
    def __init__(self, location):
        self.location = Path(location)
        self.thumbnail_location = f"thumbnail/{self.location.name}"
        self.medium_location = f"medium/{self.location.name}"

    @property
    def _extracted_lines(self):
        return check_output(
            ["extract", self.location],
        ).decode().splitlines()[1:] # getting past the header

    @property
    def _extracted_data(self):
        return {
            line.split(" - ")[0].lower(): line.split(" - ")[1] for line in self._extracted_lines
        }

    @property
    def date(self):
        naive_date = datetime.strptime(self._extracted_data["creation date"], "%Y:%m:%d %H:%M:%S")
        return pytz.timezone(time.tzname[0]).localize(naive_date, is_dst=False)

from pathlib import Path
import gpxpy

class GPX:
    def __init__(self, location):
        self.location = Path(location)
        self.data = gpxpy.parse(self.location.read_text())
        self.min_lat = min(point.latitude for point in self.points)
        self.max_lat = max(point.latitude for point in self.points)
        self.min_lon = min(point.longitude for point in self.points)
        self.max_lon = max(point.longitude for point in self.points)

    def get_coord(self, date):
        points_before_date = sorted([point for point in self.points if point.time < date], key=lambda point: point.time)
        matching_point = max(points_before_date, key=lambda point: point.time)
        return (matching_point.latitude, matching_point.longitude)

    @property
    def tracks(self):
        yield from self.data.tracks

    @property
    def segments(self):
        for track in self.tracks:
            yield from track.segments

    @property
    def points(self):
        for segment in self.segments:
            yield from segment.points

import folium
from pathlib import Path
from folium.plugins import MarkerCluster
from tqdm import tqdm

def main():
    gpx = GPX("track.gpx")

    center = ((gpx.max_lat + gpx.min_lat) / 2, (gpx.max_lon + gpx.min_lon) / 2)

    map = folium.Map(
        location=center,
        control_scale=True,
        max_zoom=19,
    )

    map.fit_bounds([(gpx.min_lat, gpx.min_lon), (gpx.max_lat, gpx.max_lon)])

    for segment in tqdm(gpx.segments, "Drawing segments"):
        locations = [(p.latitude, p.longitude) for p in segment.points]
        if not locations:
            print("Got a segment without locations, going on")
            continue
        folium.PolyLine(locations=locations).add_to(map)

    mCluster = MarkerCluster(name="Cluster",
                             options={
                                 "maxClusterRadius": 30,
                                 "spiderfyDistanceMultiplier": 4,
                             }).add_to(map)

    images = [Image(image) for image in Path("images").glob("*")]

    for image in tqdm(images, "Drawing images"):
        folium.CircleMarker(
            fill=True,
            location=gpx.get_coord(image.date),
            fill_color="red",
            popup=folium.Popup(
                f'<img src="{image.thumbnail_location}" onclick="img_box(\'{image.medium_location}\')")/>',
                lazy=True)).add_to(mCluster)

    map_root = map.get_root().render()

    header = map.get_root().header.render()
    body_html = map.get_root().html.render()
    script = map.get_root().script.render()

    result = f"""
    <!DOCTYPE html>
       <html>
           <head>
               <title>Example of map</title>
               {header}
               <script src="https://konubinix.eu/ipfs/bafkreiaxghyfbzfnpq7rkakxswmdihnutkfhjjfw7br5fwoyumaovncczu?img_box.js"></script>
           </head>
           <body>
               {body_html}
               <script>
                 {script}
               </script>
           </body>
       </html>
            """

    Path("dist/index.html").write_text(result)

if __name__ == "__main__":
    main()

Its time to put that code into map.py and running this.

./venv/bin/python map.py

Drawing segments: 0it [00:00, ?it/s]
Drawing segments: 2it [00:00, 3483.64it/s]

Drawing images:   0% 0/13 [00:00<?, ?it/s]
Drawing images:   8% 1/13 [00:00<00:04,  2.85it/s]
Drawing images:  15% 2/13 [00:00<00:03,  3.01it/s]
Drawing images:  23% 3/13 [00:00<00:03,  3.08it/s]
Drawing images:  31% 4/13 [00:01<00:02,  3.15it/s]
Drawing images:  38% 5/13 [00:01<00:02,  3.09it/s]
Drawing images:  46% 6/13 [00:01<00:02,  3.12it/s]
Drawing images:  54% 7/13 [00:02<00:01,  3.12it/s]
Drawing images:  62% 8/13 [00:02<00:01,  2.92it/s]
Drawing images:  69% 9/13 [00:02<00:01,  3.07it/s]
Drawing images:  77% 10/13 [00:03<00:00,  3.05it/s]
Drawing images:  85% 11/13 [00:03<00:00,  3.02it/s]
Drawing images:  92% 12/13 [00:03<00:00,  3.13it/s]
Drawing images: 100% 13/13 [00:04<00:00,  3.15it/s]
Drawing images: 100% 13/13 [00:04<00:00,  3.08it/s]

You end up with the index.html file in dist that shows the pictures in a map.

And here comes the result

what I learned, writing this all down

I feel at ease doing a project like this, but this is because I forgot all those tiny details that need to be overcome:

using a “sandbox” environment to trial and errors,
finding suitable libraries, such as gpxpy, folium or img_box.js.
finding out how each library and data format is meant to be used, like the tracks -> segments -> points structure of gpx,
understanding the hairy details of datetime localization,
being at ease with python advanced concepts like generator and properties allowed me to end up with a short code when it would have been more verbose without those.
using some the command line tools to prepare my project,

is that the end then?

A little voice in my head tells me that although those are needed skills, they are mostly about programming. Most of the difficulties had little to do with getting a bunch of photographs and display them on a map. They were more about generic programming stuffs.

In other terms, if we could find a way to make those easier, then my friends may find those projects less daunting (see the TEA model).

what about using low-code/no-code/visual programming?

Let’s consider the difficulties mentioned above one by one and see how a visual programming environment would have helped a non-programmer deal with them.

a visual programming tool already comes with a playground, inviting more naturally to try stuffs,
a visual programming tool already comes with a package manager of usable libraries. If one does not exist, skilled programmer people like me could write them,
using the playground of the visual programming tool, it feels manageable to me,
visual programming tools generally use high level data structure. For instance, it could make sure that there is no such thing as a naive datetime in the first place,
this is indeed a difficulty. Any environment has its own paradigms that need to be understood to use them,
hopefully those would be made available in the visual programming environment, but I’m not confident,

When looking for visual programming tools, I found either program that were dedicated to education or programs specialized to very specific domains and not very adjustable to become more generic programming environments.

The only one that seemed promising to me was orange data mining. As the name suggests, it focuses on data mining, but it allows creating custom widgets.

conclusion

In my humble opinion, we could create tools that would make easier the world of programming. If I had to choose, I would bet on visual programming. But it appears to need quite some energy, from programmer people (to create the needed core stuff) and also from non programmer people (to test and talk about what it feels like to use the tool).

Therefore, for the time being, I would tend to conclude that no, programming cannot be made easier. No with the current tools available at least.

Notes linking here

trying to make programming easier (braindump)

Konubinix' site