gcloud on Python 3.10: module ‘collections’ has no attribute ‘Mapping’

The CPython documentation says about collections.Mapping and other abstract base classes:

Deprecated since version 3.3, will be removed in version 3.10: Moved Collections Abstract Base Classes to the collections.abc module.

Fedora 35 uses CPython 3.10 as its default Python interpreter. Here, invoking Google’s gcloud CLI fails with

ERROR: gcloud failed to load: module 'collections' has no attribute 'Mapping'

Observed with Google Cloud SDK release 363.0.0 (2021-11-02).

I addressed this by installing Python 2.7 and then instructing gcloud to use that:

$ sudo dnf install python2.7
$ export CLOUDSDK_PYTHON="/usr/bin/python2"
$ gcloud

I reported that here.

Twitter’s H.264 video requirements

I tried adding a video to a tweet. A video I encoded myself with ffmpeg.

It’s not like I know nothing about H.264 and ffmpeg, and I chose what I thought were sane and common parameters (mainly ffmpeg defaults), also sane aspect ratio and resolution etc.

Yet, no matter what I tried, I got the following error message when trying to send the tweet:

Your video file could not be processed. Please see tips for upload videos.

It linked to https://help.twitter.com/en/using-twitter/twitter-videos.

That is a rather unspecific help page. At the bottom it talks about constraints with respect to resolution, frame rate, aspect ratio — I got all that right.

I found other people’s ramblings about this error message:

But those resources were not really helping.

I then found https://developer.twitter.com/en/docs/twitter-api/v1/media/upload-media/uploading-media/media-best-practices which has a section “Video specifications and recommendations”. I double-checked things like pixel aspect ratio etc, and then I found two rather rather specific boundary conditions:

  • “Only YUV 4:2:0 pixel format is supported”
  • “H264 High Profile” (emphasis mine)

I was indeed using the “high” profile (see this for learning more about these profiles). However, the video file(s) I tried all used the yuv444p pixel format, as shown by this line of output of ffprobe:

Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 1280x720 [SAR 1:1 DAR 16:9], 271 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)

So, I followed this lovely resource and added the following command line flags to ffmpeg to explicitly set the profile High, level 4.0, and to set the pixel format to yuv420p:

-profile:v high
-level:v 4.0
-pix_fmt yuv420p

With these choices, ffprobe emitted

Video: h264 (High) (avc1 / 0x31637661), yuv420p

and Twitter accepted my H.264 video. Hope this helps!

By the way, the following resources help understanding the role of the pixel format and also the differences between yuv444 and yuv420:

Fedora: reinstall kernel and update GRUB config

This is to show how dnf reinstall can be used to rewrite the GRUB config to boot a specific kernel on Fedora.

My Fedora 34 ended up in a bit of a confused state. From DNF’s point of view (only) kernel 5.14.13 was installed:

$ dnf list kernel
Last metadata expiration check: 0:00:57 ago on Fri 22 Oct 2021 02:41:15 PM CEST.
Installed Packages
kernel.x86_64 5.14.13-200.fc34 @updates

But grubby showed that 5.14.9 was configured for boot:

grubby --default-kernel
/boot/vmlinuz-5.14.9-200.fc34.x86_64

This reinstall command set things straight again:

dnf reinstall kernel-core-5.14.13-200.fc34.x86_64

Outcome:

$ grubby --default-kernel
/boot/vmlinuz-5.14.13-200.fc34.x86_64

Lightroom: how to remove dark spots as of water droplets on the lens

I brought home a sunrise landscape photograph from Tuscany (Italy) that I like: the scenery is quite lovely, and the result is pretty decent, technically:

If it were not for those circular shadows scattered across the field of view — artifacts caused by tiny water droplets on the lens. I missed that during the shoot (been there before; always try to pay attention, but these mistakes happen).

Do you see the spots? Especially within the upper half of the picture, they stand out pretty clearly over the otherwise homogeneous background. What follows is the same photograph developed in grayscale (with enhanced contrasts), to visualize the dark spots a little more clearly:

I was able to remove these artifacts with Lightroom’s Spot Removal tool. With surprisingly great success.

Here is a screenshot of the “visualize spots” view (which you can activate after selecting the Spot Removal tool in Heal mode):

This view clearly exposes the circular artifacts, especially in the top half of the photograph.

Here is how this view changed after obliterating these artifacts one by one (by applying one spot removal operation per artifact):

Quite tidy, right? Indeed, in the actual photograph, the dark spots magically disappeared:

That’s pretty neat!

 

Robust child process management in NodeJS?

Robust child process management in NodeJS? Not so easy :-).

For example, calling child_process.spawn() with a bad path does not throw an error right away. Although that error (ENOENT in this case, on Linux) is indeed known to the runtime in a synchronous fashion (NodeJS uses uv_spawn() for invoking the new process and that reports errors synchronously to its caller).

Even when child_process.spawn() fails to create a process it will return a process handle. Which you can try to kill(). And that would not throw an error right away.

To discuss this topic I submitted https://github.com/nodejs/node/issues/30668 a while back (as of which we improved the documentation for kill()) and https://github.com/nodejs/node/issues/33566.

Some more aspects and links, in no particular order:

A successful (but I think a little too involved) recipe for handling startup errors might be this:

  1. spawn()
  2. attach error handler, and upon error store the error state in a magic variable.
  3. implement and start a “success criterion watcher” (may even be based on polling) and in that logic consider the magic variable (for example, if it is known to be an error object, throw it).