Which Lens Wins in Sharpness vs Expense?

It took a little while, but we finally have the “desert island” 24-70mm f/2.8 lens for every major full-frame mirrorless system on the market, with some lovely crossover as well thanks to the L-Mount Alliance and Sigma’s continued support of E-mount. Though RF and Nikon Z are a bit on their own at present, E-Mount and L-Mount, which encompass a huge number of cameras, now have a lot of options.

With Panasonic Lumix and Sigma getting their 24-70m f/2.8 lenses to market here in 2020, I wanted to take their offerings along with the Sony 24-70mm f/2.8 G-Master, the Canon RF24-70mm f/2.8 L, and the Nikon Z 24-70mm f/2.8 S out to photograph the same subject from the same distance and see which one I preferred.

The Lenses and Their Prices

This test compares the sharpness performance of five lenses, each of them available for one or more full-frame mirrorless mounts, and all of them at different (though sometimes similar) price points. Below they are listed from least to most expensive:

  1. Sigma 24-70mm f/2.8 DG DN Art for E-Mount and L-Mount : $1,099
  2. Panasonic Lumix S PRO 24-70mm f/2.8 for L-Mount: $2,197
  3. Sony FE 24-70mm f/2.8 GM (G Master) for E-Mount: $2,198
  4. Nikon NIKKOR Z 24-70mm f/2.8 S for Z-Mount: $2,296
  5. Canon RF 24-70mm f/2.8 L IS USM Zoom for RF-Mount: $2,299

The Testing

The images below were all captured within 15 minutes of one another on a tripod at 24mm, 35mm, and 70mm at both f/2.8 and f/8. The shutter was on a 2-second delay to remove any chances of camera shake from my finger hitting the shutter button. All the cameras were set to center point, single AF mode and focused on the exact same space in the frame. All lenses are in “like new” condition and provided to us directly from each manufacturer.

Before I get into showing the results of this non-scientific test, I wanted to make a few things clear:

1.) As I just said, this isn’t a scientific test. Sure, the idea was to compare five lenses against one another in an environment that I considered to be fair and unbiased, but I’m not a lens testing specialist and I’m not interested in being one, to be honest. For those looking at this test, you might disregard all the results because I used five different lenses on three different cameras, a few of which have varying degrees of megapixel difference. Those differences do matter in serious, hardcore benchmarking, but this is not that. This is a guy with a box of equipment taking photos of a building.

2.) The Sigma and Sony lenses were tested on an a7 III. The Lumix was tested on the Panasonic Lumix S1. The Canon was tested on the EOS R. The Nikkor was tested on the Z6.  We chose to use the near-24-megapixel cameras currently available because Canon does not have a high megapixel mirrorless camera compatible with their latest 24-70mm lens at present. So while Sony, Nikon, and Panasonic all could have played nice near the high-end of the megapixel spectrum, it would put Canon at a serious disadvantage here should we have chosen to just use the EOS R against them.

But that said, Canon did not have any spare EOS RP cameras that they could loan us for this test, which means the only camera I was able to use was the EOS R. As you might note, the EOS R has about six more megapixels than the Sony A7 III, Panasonic Lumix S1, and Nikon Z6. As such, the frames taken from the Canon are going to be larger and therefore the perception of sharpness may be affected. This obviously sucks and wasn’t ideal, but I think we all can look at these results and still talk about them in at least a “well, that’s interesting” mindset.

3.) When we get into looking at the sharpness at the edge of the frame, you’re going to see a huge variance in the way the subject is presented. Since this isn’t a truly scientific test and these shots are all just taken exactly as the cameras they are attached to show them, there is clearly something happening internally on some of these cameras. For example, it’s pretty well known that Lumix cameras hard-bake in their lens corrections into the RAW file. There are known ways to get around this, but I opted to just show you exactly what comes out of the camera.

In effect, when looking into the “rules” of how this test would be conducted, we decided that it would be best to just see how these lenses would work on the cameras they were designed to work with, rather than testing to see the pure image rendering power of the lens. If you want tests like that, DxO Mark, LensRentals, and other websites do hardcore MTF testing that will satisfy your needs. Here, we just wanted to take some pictures.

4.) Getting the best idea for what you, the consumer, can expect from a lens requires testing more than one lens. It requires testing more than a dozen. That’s why LensRentals tests a huge swatch of lenses and averages the results. We highly recommend you use them as well as looking into the results here. As much as you should judge a lens by its average, you should also know how individual lenses that are sent out at random perform against one another.

5.) Finally, this test is all about sharpness. We could get into color fidelity and distortion control, but we think that’s another story for another day. Here, we just want to look at sharpness versus price, and as such all images have been converted to black and white so that color doesn’t interfere with our perception of sharpness.

So you have an idea of what was being photographed without having to download the original files, this is the scene:

The below are side-by-side excerpts from images taken with all five lenses at 100%. You may click the images below to open them in a new tab to be viewed at maximum resolution.

Center Point Sharpness

With that, let’s take a look at how all five of these lenses performed at 24mm and wide open at f/2.8:

It’s close, but I think that the Lumix 24-70mm has the best overall sharpness here, with Sigma not too far behind it.

Next, let’s look at the same 24mm shot, but this time at f/8:

This is a much closer, harder to gauge comparison. As expected, the jump to f/8 gave all five lenses a boost in sharpness. Looking at the head of the statue and the bricks above the window, I’m leaning towards giving this one to the Lumix and then the Sigma. There are a few bricks above the window that have vertical lines in them, and those lines are more defined out of the Lumix than out of any of the others, though the Sigma is quite good as well.

The next two shots were taken at 35mm. The first was again at f/2.8:

Looking specifically at the vertical hash marks on those same bricks, the Lumix again keeps drawing my eye. All five of these are good, but I think the Lumix and the Sigma yet again look the cleanest. Canon would be my third pick.

Here is 35mm again, this time at f/8:

Again, this was much harder to discern major differences. They all perform really well, but if you twisted my arm I would lean towards the Lumix yet again. The lines are just a bit more defined than the competition, though Canon and Sigma are close here.

These final two shots were captured at 70mm. The first is again at f/2.8:

I really like the Canon here, it’s standing out to me as being the sharpest of the bunch. The Lumix, Sigma Sony, and Nikon are all pretty much playing in the same sandbox right behind it though.

Let’s look at 70mm again, this time at f/8:

I am not willing to call a winner here, they all look pretty fantastic. As expected, all five lenses are doing their best work at the longest point of the lens with a middle-of-the-road closed down aperture. No losers here at all, that’s for sure.

Center Sharpness Preference: The results here really surprised me, as I wasn’t expecting the L-Mount Alliance to pull into the frontrunner position, but I can’t argue with my eyes. Though all five lenses are doing a pretty fine job across the lens range and both wide open and closed down, the Lumix is my top pick if you’re looking at just pure performance. That said, the Sigma was almost as good everywhere in this test. Canon really impressed me at 70mm, and that’s certainly worth noting as well.

But we all know that center sharpness doesn’t mean anything if your edges are bad. So, with that in mind, let’s take a look at an edge, shall we?

Edge Sharpness

These images are all the same images as above, just our perspective now shifts to the upper right-hand corner. This is the toughest place to be consistent with zoom lenses, and adding into that equation the different methods by which the camera sensors read and adjust to the lenses means that what is the absolute corner of these images is pretty different from lens to lens, and camera to camera. This is a lot less clean to look at, but that’s edges of a zoom lens for you.

So to be clear, each of these snippets is the absolute upper right-hand corner of all of the shots.

Starting off, let’s again look at 24mm at f/2.8:

Honestly, these all are not great. Every single one of them has some level of softness ranging from really soft to tolerable. As a note, take a look at the Lumix example. That truly is the far, upper-right-hand corner of the frame that everyone else captured, but you’ll notice the statue is completely missing. My intuition tells me this is due to the in-camera lens correction just cropping that part off to account for distortion. So, that noted, it’s probably no surprise that the Lumix looks the cleanest to me out of the bunch. Perhaps more impressive is how much of the statue is visible on the Sigma, and how well the sharpness competes with the Lumix despite that.

Next, let’s look at 24mm at f/8:

Oh boy, is that much better. Compared to the last group, this is like turning on the overhead lights in a dimly lit attic. Just about all of them see huge, marked improvement, but I like Sigma here the best of them all. I personally think the details on the wall and the tree are more defined than out of any of the others.

Ok, let’s zoom in a bit to 35mm and see how the edges look at f/2.8:

Well, while better than at 24mm, I still am not super impressed with anyone here. I personally think Sony, Sigma and the Lumix are all pretty close, though none of them look “good” in my book.

Alright let’s clean that up and go for f/8 now:

Ah that’s better. All five of these cleaned up really nicely, with my preference here being the Sigma first, and the Canon and Lumix being closely behind that in that order.

The final two images here were both taken at 70mm, looking at the absolute far upper-right edge of the frame. Here are the results at f/2.8:

I like the Lumix here the most, but the Sigma and Nikkor are looking pretty nice right behind it. These are easily the least bad of all the f/2.8 corners we’ve looked at, which makes sense given we are at 70mm.

Let’s close it down to f/8 and take a look:

For me, the clear winner here is the Lumix. Though the Sigma, Canon, and Nikkor all see improvements thanks to the closed aperture, the Lumix stands out the most as being truly sharp. How much of that is the in-camera correction? Probably a lot. But that’s what you get with a Lumix straight out of the camera. The biggest surprise for me her was how little the Sony improved between wide open and f/8. It almost looks like the same shot.

Debriefing These Results

I want to start by saying I think all five of these lenses are very good, and any professional could make magic with them on an assignment. In the end, it’s the shooter and not the equipment that makes great photos.

That said, I do have a preference here. Though the Lumix was most commonly my personal favorite out of each of the tests, I don’t know that I would say it’s the best purchase. If you recall, initially I said this was a test of sharpness vs price, and the Lumix is nearly twice as expensive as the Sigma.

Is the Lumix twice as good as the Sigma? That’s ultimately up to you to decide, but I personally don’t think it is. It’s definitely better, and I think if money were no issue I would select it over the field. But since money is always an issue, it’s hard to argue against the Sigma.

Current Favorite: Sigma 24-70mm f/2.8 DG DN Art Lens

Not only is the Sigma 24-70mm f/2.8 DG DN Art remarkably consistent throughout the zoom range and through aperture values, but it’s also the most versatile to consumers. It’s cheaper, namely and perhaps most importantly, by a gigantic margin. I think this is going to be the most attractive feature of this lens, and the fact it is also a very good performer is just icing on the cake. It’s also available for both E-Mount and L-Mount, which opens it up to use on far more cameras than any of the other lenses tested here.

When you are looking at just sharpness vs price, if you have an E-Mount or L-Mount camera, the Sigma feels like the obvious choice. Mix that with the rest of the things this lens does right, and you have a pretty tight package that’s hard to argue against.

Source link