Wannabe scientists as lens benchmarkers

Topic photography. Unfortunately, many lens reviews mainly discuss quantitative aspects that are of negligible relevance in practice.

Regarding the optical performance of a lens, especially image sharpness, there are plenty of metrics that can be measured. Not surprisingly, these metrics can be measured uber precisely — given expensive lab equipment. Obviously, lenses differ in these metrics. The issue is that many of these photography review sites out there forget putting these differences into context. They forget to explain that often these differences are insignificant when doing photography under real-world conditions. At the same time they treat those differences among lenses that actually make a difference in practice as a side note (such as focus speed, bokeh, material and weight). Why is that? I guess people are easily blinded by their cool and uber precise lab equipment as well as by their ‘research results’ — so that they forget what actually matters.

Talking bottlenecks

The concept of bottlenecks is a concept every engineer understands and should cherish. Actually everybody understands, I guess. However, many people do not apply it on a day-to-day basis (including those wannabe scientists writing lens reviews). When applying the concept of bottlenecks, one identifies the weakest component/entity in a system regarding a certain output criterion. That thing, the weakest component, the bottleneck, determines the output quality.

  • Insight 1: improving everything else except for this bottleneck does not change the output quality. As a direct consequence, any effort in improving other components is wasted.
  • Insight 2: it is possible to improve the bottleneck component that much so that now a different component becomes the new bottleneck. This would be a serious improvement.

Now, regarding image sharpness, there are three major components in the optical system that could potentially be bottlenecks, depending on the scenario:

  • Subject movement vs. exposure time
  • Subject location and dimension vs. focal plane location and orientation
  • Optical performance of the lens (or: “how sharp is the image under ideal conditions, i.e. subject is ideally aligned with the focal plane and the subject does not move”)

When reading these points, you probably do already realize that in real-world photography, the first two are real bottlenecks. Whereas the third point is about physical limitations imposed by the lens, the first two points are physical limitations imposed by the scene itself. There is no way to trick these physical limitations.

Unbreakable limits

Let’s explicitly state what these physical limitations are:

  • Subjects move. Think people, animals, or plants moving in the wind. Things just move. There is at least some micro movement. When things practically don’t move, we’re under laboratory conditions.
  • Depending on the exposure time and the camera movement, the subject movement yields a certain degree of blurriness in the image. This can be very large or very small. Also, in the image it might appear that the subject is frozen whereas the background is blurry. In any case, any movement in the scene is responsible for decreasing sharpness somewhere in the image.
  • Taking an image of a scene means projecting 3D on 2D. There is only one focal plane where this projection is sharp (ignoring movement). It helps to vividly imagine a real plane standing around somewhere in the scene. Imagine how this plane cuts the scene and the actual subject. Clearly, almost all of the scene is *not* in focus, i.e. does not cut the focal plane. These parts of the scene are not sharp, even if not moving. Regarding the background this might be what you want. Regarding the actual subject, however, we also need to appreciate that only a tiny fraction of its 2D projection ever is in focus. In other words, if the subject has an obvious 3D shape, such as a face, an animal, a building or a plant, most of the subject is *not* in focus (arguably, it might be pretty close to focus). If the subject is flat and the alignment of the subject with the focal plane is perfect, we’re under laboratory conditions.

As you can see, the physical limitations can be minimized under laboratory conditions. No surprise, this is what a lab is good for. The right question to ask at this point: Honestly, considering your shooting habits, how large is the fraction of photos you’re taking under laboratory conditions? 1 % would already be a large fraction, I guess.

Insight: we usually don’t shoot under idealized conditions and, therefore, there are severe physical limitations due to scene movement and focal issues. Given a certain scene, these limitations are unbreakable theoretical limits.

The photographer is the bottleneck. Only then, …

I stressed the term in the optical system above, ignoring the photographer as part of the system. But he/she does belong to the real system, right? So, a little abstraction here: What are the bottlenecks “Subject movement vs. exposure time” and “Subject location and dimension vs. lens focus” considering that the photographer needs to master focus and exposure? Let’s bring things in order: regarding image sharpness and thinking in the concept of bottlenecks,

  • the skill and abilities of the photographer him(her)self in mastering focus and exposure are the most relevant bottleneck.
  • Considering a quite skilled photographer, the physical limitations in focus and exposure are the very next bottlenecks to be considered as relevant.
  • Only under idealized (laboratory) conditions the physical limitations w.r.t. subject movement and alignment with the focal plane can be controlled in a way that the actual sharpness of the lens plays a role.

Lens sharpness? Seriously?

What we know is that practically *all* ((D)SLR) lenses on the market, even the cheapest ones, provide an image sharpness which is so good (good enough), that the physical limitations discussed above are by orders of magnitude more limiting. According to the concept of bottlenecks, the differences of these lenses in terms of their image sharpness under laboratory conditions do not matter in real-world photography. So, why do many lens reviews spend 90 % of their texts talking about the optical limitations of the lenses? An open question.

A quote from Ken Rockwell, a popular lens reviewer who writes about lenses what actually matters:

It’s the least skilled hobbyists who waste the most time blaming fuzzy pictures on their lenses, while real shooters know that few photos ever use all the sharpness of which their lenses are capable due to subject motion and the fact that real subjects are rarely perfectly flat.