Robust child process management in NodeJS?

Robust child process management in NodeJS? Not so easy :-).

For example, calling child_process.spawn() with a bad path does not throw an error right away. Although that error (ENOENT in this case, on Linux) is indeed known to the runtime in a synchronous fashion (NodeJS uses uv_spawn() for invoking the new process and that reports errors synchronously to its caller).

Even when child_process.spawn() fails to create a process it will return a process handle. Which you can try to kill(). And that would not throw an error right away.

To discuss this topic I submitted https://github.com/nodejs/node/issues/30668 a while back (as of which we improved the documentation for kill()) and https://github.com/nodejs/node/issues/33566.

Some more aspects and links, in no particular order:

A successful (but I think a little too involved) recipe for handling startup errors might be this:

  1. spawn()
  2. attach error handler, and upon error store the error state in a magic variable.
  3. implement and start a “success criterion watcher” (may even be based on polling) and in that logic consider the magic variable (for example, if it is known to be an error object, throw it).

 

 

 

“Dr. Erickson COVID-19 Briefing” — a quick assessment

The “Dr. Erickson COVID-19 Briefing” keeps popping up.

A quick assessment, mainly derived from the first few minutes of this video which contain the strongest claims.

I am certain that “Dr. Erickson” overestimates his scientific capabilities, his ability to derive knowledge, his actual ability to contribute a novel aspect the wider discussion of how to deal with COVID-19 in society.

I am certain that at the same time he underestimates the previous findings of other research groups all over the world. He ignores them.

He claims to take a scientific approach, but at the same time he does not trust and consult the scientific community around him. It is very likely that he didn’t talk to other experts to get a critical eye on his conclusions before going viral. Isn’t that suspicious? To you?

If you like the content in this video and if you feel enthusiastic about it, here is what I think happened:

  • he made you like him
  • he didn’t contribute productively to a complex problem

Aren’t these key ingredients of populism? I might think so.

In any case we can try to asses who took advantage here. I think it’s him. It’s not you, it’s not society.

His communication is built on “simple truths”. The way “Dr. Erickson” describes the situation: it’s all so obvious, from the numbers they have looked at.

I hope that you can believe me that his house of cards is built on methodological flaws in his statistical analysis (I have studied physics and a PhD, so please allow me to judge). They had an intuition and found numbers to support that intuition. Their conclusions, though, are as wrong as they are dangerous and destructive.

With respect to complex problems you should not trust your intuition. You’re better off trusting your rationally developed, peer-reviewed, triple-checked model that has survived the test of time and thousands if not millions of critical reviews. At least when your goal is to productively contribute to a wider set of problems.

This is where my gut hurts: “Dr. Erickson” tries to make his points sound strong and convincing by saying that they have a lot of experience in immunology and microbiology. This is deceiving, this is ironic. Maybe you can’t (and that’s fine) but I can tell that this is a flash bang, a smoke grenade. They are clearly not experts in these fields (because then you have to master the basic tools of statistics), and they are clearly overestimating the sanity of their method, and the relevance of their findings.

Another way to say this is that the “statistical method” that they applied to derive their conclusions is utter nonsense. It doesn’t even deserve to be called a “method”. They didn’t discuss their findings in the context of published studies trying to address the same questions. That’s the least you have to do before going public so widely.

This is a great example for how easy it is to put bullshit into the world without “fact checkers” (I don’t like this terM) standing a chance to keep up with the sheer influx of bullshit.

I would love to discuss the details in-depth. But we all have limited time and resources and while it’s an important job to call out bullshit, it’s also incredibly frustrating.

A few quotes from a reddit thread about this video that I agree with:

I’m reading comments on the video (bad idea) of people who are angry that this guy is “the only one reporting hard numbers and facts” and that no one is listening to him and its frankly giving me an aneurism. He’s not doing that. He’s putting forward dubious extrapolation

If they can’t understand the importance of using representative data, then they really shouldn’t be posing as experts.

They also adopt the same misleading approach when comparing Norway and Sweden and tried to claim that both countries have similar numbers of cases and deaths. In reality, Sweden has 18,177 cases and 2,192 deaths (217 deaths per 1 million population) and Norway has 7,493 positive cases and 201 deaths (37 deaths per 1 million people). So contrary to their claims, Norway appears to be doing much better than Sweden, which supports the use of social distancing methods.

The doctor in the video grossly interpreted data incorrectly and “extrapolate” them into misleading conclusion (whether intentionally or not, that is another question).

Again these two are not experts. Yes they are physicians but I bet they do not have the type of training background as infectious disease specialist or epidemiologist. I do not see any type of credibility from them. And what they say could easily be used by certain political groups to instigate falsehood information. And this is not good because it affects public safety.

 

The scientific method: in normal times you don’t give a fuck about it

“The statistics are flawed”, “the numbers don’t convince me” are some of the more common things I hear and read these days in view of the ongoing pandemic of coronavirus disease 2019 (COVID-19).

One cannot disagree with that. I cannot disagree with you when you say that. In fact, I can very much relate to that. I feel you. And I’d love to work together with you to discuss the numbers, just as much as I like to do that for and with myself.

Every single person who says that “the numbers are not correct” is correct. Do you want to be one of them? I can understand that.

All the skeptical people are right. Really. You are absolutely right when you don’t quite trust the current statistics and numbers.

But some of you are also fucking fundamentally wrong and destructive.

When you think and propagate that it isn’t all as bad as “they” are trying to tell us.

When you use that argument to morally justify not following guidelines.

When you try to convince your peers of your “insight”.

That disappoints me so much. What I hear and read these days is a reminder for how deeply we as a humanity have to worry about our own ignorance and stupidity. It’s our own collective stupidity that we suffer from the most.

Of course you would agree with that. Because clearly “they” are all idiots, right?

You may even think that you contribute something useful when you say “the numbers are not correct”. I am pretty sure: it’s not a productive contribution. You need to care a little more, and try a little harder (please read on).

Gladly, relevant people have already considered the uncertainty of data for us. They know that they base their decisions on potentially thin ice. They work with the fact that all models are wrong, some are useful (cheers to Mia). Finding more of the latter is hard work.

 

There is an important difference between them and you.

Most of “them” are trying to help. For example, by leaning on the side of caution. Some of “them” are making decisions that matter. Because they are asked to do so. And at least in Germany I am quite grateful for the average decision-making that happened to date in the context of COVID-19. I am especially grateful that it’s not people like you who get to making decisions that matter.

And here we are: the big difference between “them” and you is that you don’t need to make decisions that matter. You just use this opportunity to express how little you trust the people around you to do the right thing, how little you trust their desire to actually protect you from severe illness. I would love to see a shift in perspective: how would you think if you had to make difficult decisions that matter? Think about it, please.

 

You are Captain Obvious.

For undeniable, credible, bulletproof statistics we need dense sampling. We need exhaustive spatial and temporal coverage.

That requires a large amount of time, and many countries being affected by definition. It will take months and years to quantitatively and with great confidence answer all these questions that we seek answers for right now.

However, we needed to make decisions in the meantime. And we still need to make decisions right now, and we are going to have to make decisions every minute from now. All in view of uncertainty.

There is a fundamental conflict here that implies that we’re in a shitty situation.

 

You don’t get kudos for finding out that we’re in fact in a shitty situation.

Working with the uncertainty at hand is exactly what every single scientist working in the field of epidemiology tried their best to be prepared for. Dealing with uncertainty is the biggest challenge of every single politician who needs to make decisions right now.

What do you think are most of these discussions about in meeting rooms and phone conferences? I would assume: uncertainty.

Now after we’ve made some decisions (in view of uncertainty) you come in, Captain Obvious.

Glad we have you. Genius. You’ve come up with some incredible findings.

For example, you have found that some test methods might be affected by false positives and false negatives. You have found that the time evolution of the case count numbers in region X is not conclusive because we don’t quite know how the time evolution of the corresponding testing effort looked like. You have heard that some COVID-19-attributed deaths might actually have been misclassified. No! Why didn’t we think of THAT?

 

Thanks, Genius. You’re right.

Genius! Fortunately you had a look at the website of the European mortality monitoring institution and found that it doesn’t quite show anything unusual for Europe in March 2020. Thanks for looking. I mean, really! We almost missed looking at their website. Now we know that it probably isn’t even unusual that all these people die. Probably just a local outlier!

In view of that: you are of course right! It’s very likely that whatever is happening in Italy, Spain, France is completely anecdotal. Of course! They’re probably just in bad luck over there, don’t have their shit together, or are just plain stupid. I mean, right, of course! There must be some plausible explanation for what’s happening there other than “this is dangerous”. And even if we don’t find a plausible explanation: we cannot yet conclude that “this is dangerous” because all the data we have so far are so damn inconclusive! We don’t really know anything! How can we act not knowing? You are right, the best course of action is to eat some crackers and every now and then refresh the website of the European mortality monitoring institution. Really cool plots they have.

You’re right. We should amend our conclusions: it’s unlikely that anything bad is going to happen to us because we don’t have bulletproof statistics yet about Italy, Spain, France. China’s numbers? Made up. And about the U.S…. I mean, their president simply fucks it all up, right (yes, he does)?

 

Denial of observation

In view of “anecdotal evidence” is where some of you even flip into conspiracy land. You happily ignore or trivialize the qualitative in the absence of the quantitative. (thanks, Gustav).

I really wish you would try to understand what I just tried to say here.

Example: you do not actually show genuine, deep interest for the situation in Italian hospitals (and refuse to analyze how we got there) until someone presents bulletproof evidence on a silver plate convincing YOU that .. oh, fuck, there were indeed unusually many people “dying”.

 

Did I just say “dying”? Hah.

In your view it seems to be only about dying or surviving. You seem to pick your battle with a potential number of deaths, from specific disease.

I would love to remind us that the spectrum of suffering includes more than just people dying. Do you know what it’s like to be intubated? I don’t. And I don’t want to find out. Seriously. Did you already hear that when you “survive” (which is likely, yes) that you might do so with some long-term lung tissue damage?

I don’t want to create fear. But at a higher level I would like you to consider what it means to fight a disease that we barely understand yet. There is a spectrum between “dying” and “surviving” that very much defines how we experience this disease. Please, differentiate.

From your arguments I infer that you all seem to understand what COVID-19 really is, how straight-forward it is to treat. You seem to know your enemy very well. Cool.

I can tell you that our measures of precaution are fortunately not one-dimensionally based on the potential number of people dying. Yes, it’s what most of us talk about. Because of our stupidity, and inability to talk more than one-dimensional through mainstream media.

But in the meeting rooms and in the phone conferences “they” talk about details. Details that matter.

You may want to consider for yourself that you know NOTHING about all that, compared to all the things that one can know about this disease, and how it spreads through populations.

 

Stop telling your peers that the lockdown measures are too much, that things are “not as bad as they are trying to tell us”.

Shut up. Until you know. You should pause here for a moment. And fucking think about the word “know”.

Also think about the following: given your limited efforts it is unlikely that you understand more about this than those people who try really really hard. Try harder, and then we can talk (see below).

My most important point really is: don’t influence the people around you with your personal risk assessment.

Yes, that assessment that you pulled out of your ass. Based on how severe this sickness was for you and your friends. OH WAIT, don’t have personal experiences yet? Maybe wait for things to happen. This. has. just. begun.

Free of irony: the timescale of the observation period that you base your intuition on is likely to be vastly different from the timescale that this pandemic will change things for you and us. Maybe account for the possibility that your intuition can mislead you.

Please, do not assume that the majority of the people providing guidance towards decision-making do not know what they are doing.  Stop believing that you are a productive member of society by finding simple flaws in the “statistics” (call it whatever you’d like) underlying the political decisions of the past weeks. I mean it’s cool that you care! But you should realize that (tens of) thousands of people working in scientific fields are actually doing their job here pretty well and you suggesting otherwise is not helpful. It creates damage.

Have you ever thought about how hard it is to have an in-depth expert-level discussion AND at the same time communicate the subtle findings to the public so that we would be able to “understand”? It’s close to impossible. What you see through the media is (sadly, yes!) a very very very very tiny fragment of what actually matters.

Statements will be refined and revised. Measures will be extended and relaxed. That’s all part of the process. Personally, as of the risk implied by the unknown, I quite like when this process starts erring on the side of caution, and then relaxes over time. Yes, you will find many statements that appear to be “wrong”. Especially in hindsight. That does not indicate that we are taking a bad path through this crisis.

 

Invitation to contribute

I want to say that if you really believe that you can contribute anything substantial to the discussion about models, statistics, numbers: you can. You’re even invited to. All you need to do is take some time and choose the right words. The scientific community is more open and accessible than ever.

Please, join the discussion. Be productive. This discussion leads to iteration. That iteration results in a refined understanding of the current situation. We derive knowledge.

This is called the scientific method.

In normal times you don’t give a fuck about it.

More knowledge leads to better decision-making. This is how this world works, on average.

I am calming myself down and genuinely ask you to please please have some faith, please assume good intentions. At the very least when you think about the science behind all that.

VS Code: I cannot quite figure out how to search and replace in a selection

Gosh, search and replace in a selection of text in VS Code drives me nuts.

I can’t even describe how exactly it does so, but it works against my intuition.

It’s funny how in 2019 we haven’t quite settled on UX for that!

For me, it’s super hard to grasp how this (search and replace in a selection of text) is supposed to work in VS Code, as in: how would the people who designed this part of the product like me to use it? Which workflow did they have in mind? I don’t seem to understand the magic behind it. I often seem to end up replacing things in the entire document as opposed to in the selection that I just carefully created a second before. All I can conclude so far is that trial and error don’t seem to get me far :-).

Trying to understand the behavior based on docs and reading GitHub issues then reveals that this thing is complex, that it can be configured with a number of special configuration options, and that many users experience WTF moments on a daily basis. Users keep posting videos and GIFs of their weird experiences (thanks!). Two of these WTF moments are well captured in the GIFs in this comment: https://github.com/microsoft/vscode/issues/17563#issuecomment-542656591 (from October 2019).

I can say that search and replace in a selection of text works much better, so much more intuitively, in Sublime Text 3 (by default).

In VS Code it turned out that setting editor.find.autoFindInSelection to true helps me quite a bit towards getting more predictable outcomes.

Feel free to leave a comment below if you have an anecdotal opinion about all this.

If you’d like to read along, I find these issues pretty entertaining:

I particularly agree with this statement from November 2019:

I appreciate all the work and attention that has gone into this, but I don’t think I will ever be truly comfortable with Find/Replace in VS Code. The issue for me is that the “find in selection” button looks and feels like a toggle switch, not a trigger (or “fire button”).

If you flip a toggle switch from one position to the other, and then back to the original position, it should be as though you had never flipped the switch at all. With this mental model, it is always a shock and a disappointment when flipping the switch irreversibly destroys my carefully crafted manual selection.

 

 

 

Bokeh: disable touch interaction (disable drag, zoom, pan)

Bokeh is quite cool. I was looking for a way to disable touch interaction with a plot, though. Was a little tedious to find a solution on the web. Found it, buried in the docs. Therefore this quick post.

Say you have created a figure object with

fig = bokeh.plotting.figure(...)

Then you can disable the individual touch controls with e.g.

fig.toolbar.active_drag = None
fig.toolbar.active_scroll = None
fig.toolbar.active_tap = None

Very easy to do, you just need someone to tell you :-).

(Tested with Bokeh 2.0.0)