Category Archives: technical things

gcloud on Python 3.10: module ‘collections’ has no attribute ‘Mapping’

The CPython documentation says about collections.Mapping and other abstract base classes:

Deprecated since version 3.3, will be removed in version 3.10: Moved Collections Abstract Base Classes to the collections.abc module.

Fedora 35 uses CPython 3.10 as its default Python interpreter. Here, invoking Google’s gcloud CLI fails with

ERROR: gcloud failed to load: module 'collections' has no attribute 'Mapping'

Observed with Google Cloud SDK release 363.0.0 (2021-11-02).

I addressed this by installing Python 2.7 and then instructing gcloud to use that:

$ sudo dnf install python2.7
$ export CLOUDSDK_PYTHON="/usr/bin/python2"
$ gcloud

I reported that here.

Twitter’s H.264 video requirements

I tried adding a video to a tweet. A video I encoded myself with ffmpeg.

It’s not like I know nothing about H.264 and ffmpeg, and I chose what I thought were sane and common parameters (mainly ffmpeg defaults), also sane aspect ratio and resolution etc.

Yet, no matter what I tried, I got the following error message when trying to send the tweet:

Your video file could not be processed. Please see tips for upload videos.

It linked to https://help.twitter.com/en/using-twitter/twitter-videos.

That is a rather unspecific help page. At the bottom it talks about constraints with respect to resolution, frame rate, aspect ratio — I got all that right.

I found other people’s ramblings about this error message:

But those resources were not really helping.

I then found https://developer.twitter.com/en/docs/twitter-api/v1/media/upload-media/uploading-media/media-best-practices which has a section “Video specifications and recommendations”. I double-checked things like pixel aspect ratio etc, and then I found two rather rather specific boundary conditions:

  • “Only YUV 4:2:0 pixel format is supported”
  • “H264 High Profile” (emphasis mine)

I was indeed using the “high” profile (see this for learning more about these profiles). However, the video file(s) I tried all used the yuv444p pixel format, as shown by this line of output of ffprobe:

Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 1280x720 [SAR 1:1 DAR 16:9], 271 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)

So, I followed this lovely resource and added the following command line flags to ffmpeg to explicitly set the profile High, level 4.0, and to set the pixel format to yuv420p:

-profile:v high
-level:v 4.0
-pix_fmt yuv420p

With these choices, ffprobe emitted

Video: h264 (High) (avc1 / 0x31637661), yuv420p

and Twitter accepted my H.264 video. Hope this helps!

By the way, the following resources help understanding the role of the pixel format and also the differences between yuv444 and yuv420:

Fedora: reinstall kernel and update GRUB config

This is to show how dnf reinstall can be used to rewrite the GRUB config to boot a specific kernel on Fedora.

My Fedora 34 ended up in a bit of a confused state. From DNF’s point of view (only) kernel 5.14.13 was installed:

$ dnf list kernel
Last metadata expiration check: 0:00:57 ago on Fri 22 Oct 2021 02:41:15 PM CEST.
Installed Packages
kernel.x86_64 5.14.13-200.fc34 @updates

But grubby showed that 5.14.9 was configured for boot:

grubby --default-kernel
/boot/vmlinuz-5.14.9-200.fc34.x86_64

This reinstall command set things straight again:

dnf reinstall kernel-core-5.14.13-200.fc34.x86_64

Outcome:

$ grubby --default-kernel
/boot/vmlinuz-5.14.13-200.fc34.x86_64

Robust child process management in NodeJS?

Robust child process management in NodeJS? Not so easy :-).

For example, calling child_process.spawn() with a bad path does not throw an error right away. Although that error (ENOENT in this case, on Linux) is indeed known to the runtime in a synchronous fashion (NodeJS uses uv_spawn() for invoking the new process and that reports errors synchronously to its caller).

Even when child_process.spawn() fails to create a process it will return a process handle. Which you can try to kill(). And that would not throw an error right away.

To discuss this topic I submitted https://github.com/nodejs/node/issues/30668 a while back (as of which we improved the documentation for kill()) and https://github.com/nodejs/node/issues/33566.

Some more aspects and links, in no particular order:

A successful (but I think a little too involved) recipe for handling startup errors might be this:

  1. spawn()
  2. attach error handler, and upon error store the error state in a magic variable.
  3. implement and start a “success criterion watcher” (may even be based on polling) and in that logic consider the magic variable (for example, if it is known to be an error object, throw it).

 

 

 

VS Code: I cannot quite figure out how to search and replace in a selection

Gosh, search and replace in a selection of text in VS Code drives me nuts.

I can’t even describe how exactly it does so, but it works against my intuition.

It’s funny how in 2019 we haven’t quite settled on UX for that!

For me, it’s super hard to grasp how this (search and replace in a selection of text) is supposed to work in VS Code, as in: how would the people who designed this part of the product like me to use it? Which workflow did they have in mind? I don’t seem to understand the magic behind it. I often seem to end up replacing things in the entire document as opposed to in the selection that I just carefully created a second before. All I can conclude so far is that trial and error don’t seem to get me far :-).

Trying to understand the behavior based on docs and reading GitHub issues then reveals that this thing is complex, that it can be configured with a number of special configuration options, and that many users experience WTF moments on a daily basis. Users keep posting videos and GIFs of their weird experiences (thanks!). Two of these WTF moments are well captured in the GIFs in this comment: https://github.com/microsoft/vscode/issues/17563#issuecomment-542656591 (from October 2019).

I can say that search and replace in a selection of text works much better, so much more intuitively, in Sublime Text 3 (by default).

In VS Code it turned out that setting editor.find.autoFindInSelection to true helps me quite a bit towards getting more predictable outcomes.

Feel free to leave a comment below if you have an anecdotal opinion about all this.

If you’d like to read along, I find these issues pretty entertaining:

I particularly agree with this statement from November 2019:

I appreciate all the work and attention that has gone into this, but I don’t think I will ever be truly comfortable with Find/Replace in VS Code. The issue for me is that the “find in selection” button looks and feels like a toggle switch, not a trigger (or “fire button”).

If you flip a toggle switch from one position to the other, and then back to the original position, it should be as though you had never flipped the switch at all. With this mental model, it is always a shock and a disappointment when flipping the switch irreversibly destroys my carefully crafted manual selection.