Comment by hypersoar

Just in case not everyone are in the loop, LMG is Linus Media Group [1] which is the publishing agency behind the popular YouTube channel "Linus Tech Tips" [2]. It is a different Linus, not Torvalds. :)

[1]: https://linusmediagroup.com/

[2]: https://www.youtube.com/c/LinusTechTips

I'm hopeful for LMG's lab too. It's still a bit of a gamble, but from the sound of it the company is set up such that they can review products in an objective, data-backed way and tank any blowback from manufacturers that occurs as a result.

It's much more focused on enthusiast computer hardware, but Gamers Nexus[0] is doing good things in this space too. Their style is much more dry and data-dense than LMG's though, which isn't everybody's cup of tea.

[0]: https://www.youtube.com/channel/UChIs72whgZI9w6d6FhwGGHA

Do you trust Linus, though? He often promotes himself as without bias, but he very clearly hates Apple (except the watch). He also loves things he already understands (anything Microsoft). He's got heavy duty fanboyitis. And he's clearly someone you can buy demonstrated by his flip flopping AMD/Intel/NVidia praise.

I don't think he outwardly lies (at least not in a way that matters), or anything, but he's got pretty good soft selling skills which he definitely uses for evil/to make money.

All LMG channels are great. But to me anyways, they're great because they're basically comedies.

Phones and anything Apple are reviewed to oblivion. There are some incredible consumer product review YouTube channels out there too.. The Best one imo is project farm (https://m.youtube.com/watch?v=vO3UX4oEnZI). If you're into headphones, Crinacle's site/YT are on a completely different level than any other review site

I really hope LMG does videos in these styles with their lab

I’m curious about this too. He did a breakdown of how much money they get from ads and it’s not much considering how many employees they have. They have a lot of sponsorships and selling swag but I’m just not sure what their maximum size is as just another YouTube channel, even one with lots of revenue streams.

The big problem with most YouTube review channels is that they're reliant on the vendors to provide them with free test units and other forms of access in order to grow their channels, and Linus is no exception.

I actually came across a "why I'm shutting down my channel" video a while back in which the host showed emails from the company whose product she was reviewing (it was a drawing tablet or something like that) pressuring her to show the product in a more favorable light.

It's so hard to tell who's actually objective in those review videos and who's censoring themselves at the request of a vendor.

I think the review industry is so bad that even a mediocre quality endeavor could gain a ton of traction. The problem with the current tech review industry IMO is that it seems like the benchmarking and review part of it are treated like separate business units that need to be self sustaining / profitable.

If you go by what LMG says on their podcast it sounds like the intent is for the lab to give them credibility and to act as an eyeball funnel, even if it needs to be subsidized by the entertainment side of the business. They've already shown that it's possible to make entertaining reviews if you keep the technical details light, so what they really need is hard data to back them up when they trash a product or get accused of being a corporate puppet.

I personally find their videos to be entertaining, so if I'm looking to buy something and I know they evaluate tech products, I'll go to their labs site, look for entertainment videos that are produced from that data, and watch those videos. Then when I find something I think looks like a good fit for me I'll jump back to the labs side to look at the details.

IMO the thing that might make LMG's effort different is that they're going into the space as a new participant. I think they realize the technical aspect of the lab is basically going to be content that needs to exist, but that no one reads (enough to be profitable) and their monetization is set up to accommodate that scenario. Compare that to traditional reviewers (and SEO spammers) that rely on page views for their revenue.

The whole review industry is going to keep shifting towards video and the low cost, low value SEO spam sites are a big part of that. Any existing review businesses that aren't shifting towards a hybrid model like the LMG / Labs plan are going to get crushed IMO. Even if it's not LMG doing it, it's going to happen eventually.

It will be nice if they do come up with an experiment based approach to reviews. Personally I really enjoy Gamer Nexus since they already do this.

Their coverage of the Nvidia cooler design change was really interesting to watch, and they went into depth on their testing methodology with both its strengths and weaknesses.

Their channel really convinced me to take a more critical look at other “reviews” and how they conduct them with either lazily held thermal camera or smoke machines.

What's the profit model for LMG? Ads? I'm not sure how long that will sustain even a smallish channel with no expenses, let alone electronic testing equipment and products to test.

Gamersnexus has been similarly expanding their capabilities with very expensive fan testing equipment. They seem to be one of the better sources of consumer tech journalism these days.

ATK is, as far as I am concerned, the gold standard for kitchen reviews.

I like a lot of project farm's videos but his electrical tape video was far off the mark of what actually matters. They were good tests for tape but bad tests for Electrical tape.

I already mentioned it above, but repeating it here.

I would also recommend The Torque Test Channel as very similar in approach to Project Farm, maybe a bit more technical and thorough. Their focus was on impact drivers, but they have branched out into other power tools, LED lights and hand tools.

https://www.youtube.com/channel/UCZem9C5rWjSb0B8tV3k2EZg

Project Farm rules for the types of products he tests, which is pretty narrow.

It seems like the only way to really get a review is to find someone on youtube who shows the product being used in a way you plan to use it, or someone who does a re-review after a period of time like a year etc

> I like America's Test Kitchen for kitchen-y stuff.

For me, America’s Test Kitchen compromises on quality too much for the sake of convenience. And perhaps that is their target audience, but they dismissed Demeyere’s cookware out of hand for being too heavy and unwieldy, whereas a site like centurylife uses IR cameras and probe thermometers to actually measure heat distribution and retention across different cookware sets.

I don’t understand the “too heavy” complaint anyway; people cook with cast iron (Lodge / Le Creuset) all the time, and it is significantly heavier than Demeyere.

I’m going to dissent here on this thread because I’m not seeing any references. I personally feel the quality of Wirecutter has gone down since NY Times just a bit. However, after almost a decade of reading Wirecutter they have overwhelming provided a decent “why you should trust us” section for staple consumer items. There is a good example from just today. [1] You can always say they should do more, but honestly they do more research that many others in the space.

[1]: https://www.nytimes.com/wirecutter/reviews/best-air-conditio...

"I actually subscribe to Which a UK consumer reports guide."

I used to too, for quite a few years.

However, their IT related reviews boiled down to "Windows PC: Good, Apple: Pretty, Linux and Open Source: Not on my watch". A Consumer Forum "for good" completely ignores Open Source - why? Personally I think it is down to a lack of imagination rather than anything politically motivated.

I did find many of their reviews useful - you get some great details on their working and they spend a decent amount of time on reviewing non IT stuff. The content articles were also often very decent, well written and often thought provoking. Their consumer campaigning has got as far as making changes to Laws too in the past so I do think Which is a general force for good.

I just got pissed off that as soon as a laptop or desktop or software article came along, the usual turgid crap would come out. Perhaps this has improved since around 2015 when I ditched them after being a subscriber for over 10 years.

I used to laugh at my dad for subscribing to Which, but now I genuinely see the value in it.

Today’s modern review sites are skewed in favour of whoever pays the most commission. It warps recommendations which is a problem Which never had.

On the topic of displays, and specifically monitors, also the excellent TFT Central ( http://tftcentral.co.uk/ ).

I wish rtings had a Boolean on tvs so we could search explicitly for non smart models. That is basically the only thing else I’d want from that site, it’s really good.

Yes, for the things they cover Rtings is excellent.

I haven't trusted them for years, their testing is too subjective and the "objective" tests aren't considering the right things.

For example I bought hiking boots based on their recommendation. They were the most comfortable hiking boots I've ever worn but their terrible traction literally nearly got me killed despite their claims of having excellent traction. I angrily returned those boots.

I also bought a backpack based on their recommendation. They have this volume test filling a backpack with pingpong balls. It sounded like a great objective test in theory and my new pack had a higher volume than my old pack but I couldn't fit everything into it as the shape changed too much with a sleeping back and bear can in it reducing usable volume.

Finally I gave up on them when I was looking to buy a new headlamp. They ranked a headlamp lower because it's battery life was less than all the other headlamps being tested. But that headlamp max brightness was 3x the lumens of the other, batter life should have been tested at a comparable brightness level.

I like to read their subjective discussions as one point of view but its really hard to get much use from their rankings. One obvious example is at one point in time all their highest ranked ultralight sleeping bags were quilts (ie. open back, no zippers) and then all of a sudden the quilts dropped to the bottom and were replaced with more traditional zip up bags. I assume the reviewer changed and simply doesn't like quilts, which is totally reasonable, they don't work for everyone but it wasn't clear how the rankings are useful when they just shuffled so drastically.

These guys were the first thing I thought of but to be honest for me, their suggestions have been a bit off. Ofc it's all subjective but I remember distinctly buying two full face helmets they had on their list because their ratings were so different from the concensus from reviews. I could'nt tell who to trust. The gearlab suggestion was very obviously inferior beyond first impressions

I like this site as well. I trust them because I feel they are upfront with the level of subjectivity they are introducing. Also, it seems they at least buy and try out the gear.

Being open about when you are in a sponsorship "relationship" is not something that earns you an internet cookie, it's required by the FCC https://www.fcc.gov/consumers/guides/sponsorship-identificat...

Heh. The biggest problem I have with Amazon isn't even the fake reviews, it's the people who leave reviews and don't even know what a review is, which is almost all of them.

"My gadget just arrived today and I haven't even used it yet but it looks well-made and I'm sure it will last forever. Five stars!"

I (about half-honestly) blame Amazon, and more recently Google Maps.

All reviewers are not created equal.

In the early days before mass-SEO, you at least got the benefit of most reviewers being authentic, even if inept.

Now, we have the worst of all possible words: mass fake reviews + a public trained to expect only amateur-level reviews.

I trust their measurements, I just don't like how they score things, and people tend to just use their scores instead of looking at the pros+cons and measurements. (they weight all the different subscores, and add them up, so eg. if there was an excellent monitor except it had a 100:1 contrast ratio, it'd still get great scores despite having such a huge flaw that most people would consider it to be essentially unusable).

It's really bad for HDR monitors, where an edge lit "fake HDR" monitor can get a 7, while failing the basics that are necessary to give a proper HDR experience. Something like TFTCentral or HardwareUnboxed's HDR checklists, and just straight up failing monitors that don't meet all the requirements would be much better than their current (imo misleading) system that can give good SDR monitors high HDR scores, when they're terrible at HDR.

Agreed, minus the headphones. Their headphone reviews are a joke. It's also worth noting some products have a lot of variation due to poor QC (PC monitors) and they may get an unusually good/bad unit from time to time, skewing the review.

Definitely trust rtings. They buy every product in store to do their testing.

Agree - they updated the review for Logitech G PRO X WIRELESS build quality while the SoundGuys still show build quality 9/10.

I used to subscribe and they were generally good, but they made no account of cost.

So you might have (made up example) an Electrolux vacuum getting a score of 73, but a Dyson gets a score of 74 and wins their "recommended buy" then you see the Dyson is, like, twice the price.

I can see they might do the review price-blind, but it does make one suspicious that they get some sort of financial benefit from having top picks be vastly more expensive products.

Useful reviews though.

I used to subscribe to Consumer Reports back in the day, and basically regretted it. They rarely described their testing methodologies and more often than now, when they did, I wasn't impressed. Their testing usually just boiled down to whether or not the specs met the manufacturers claims, not anything useful like how well it was built and how long it is likely to last.

Australia: https://www.choice.com.au/ New Zealand: http://consumer.org.nz/

I wouldn't use Reddit for anything but general product usage information. You can get some honest reviews from Reddit users, but I find a lot of it is people justifying their purchase instead of honest feedback.

site:reddit.com works and I use it often. But I do wonder how long it will be until reddit starts to get gamed as well (if it isn't already).

100% this. YouTube's fantastic for real reviews.

I watched a few of these a while ago and I can somewhat see why they’re popular as they have this fast-paced data-dump look-at-all-this-testing format but I didn’t really think they were very good. I thought many of the tests were likely poor metrics for actual quality and that results would therefore be misleading. A stupid example would be trying to measure how much torque a Phillips head screwdriver can apply before camming out because the point of the screw design is that screw drivers should cam out at a certain torque (so better screw drivers shouldn’t necessarily let you go tighter).

I love it. I love the accent, cadence and volume.

Yes, assessing devices is far from trivial. In case of air purifier reviews, there's little focus on two crucial elements:

* Real energy consumption and performance when the filter is a bit dirty

* Fan noise and vibrations when the device has already been in use for a year or two

For example, in my experience, the top pick from Wirecutter (Coway) excels on the first item but fans tens to become misaligned after a while and vibrate a lot at low speeds. It happened to 2/2 units I bought.

He's great at getting there first with the unboxing or review with insane visuals and editing, but the content itself is very lacking. He's very heavily biased towards Apple devices, and doesn't dig deep at a technical level, preferring more subjective judgements which are difficult to compare across devices. I don't find his advice any more objective than a Reddit comment.

I think he is entertaining, but I don't find his reviews critical enough. He typically reads off a spec sheet and shares subjective opinions of just a few days of use. I need deeper, more critical reviews

MKBHD is good for entertainment purposes only.

The reviews as actual reviews are terrible. And the retro tech series - well if you know anything about the items being discussed you'll find quite a few inaccuracies. It's pure entertainment.

I have a service tech for appliances. I just ask him what I should buy. He usually has suggestions from all the cost-ranges. Sometimes, I buy used through him (built-in fridge); sometimes I buy new. Since I had the opportunity to buy a bunch of equipment this year (a huge power surge from my HVAC fried appliances, and bad luck):

Built in fridge: GE monogram;

Dish washer: anything that is quiet (below 42 dB);

Dryer: anything with turn-timers; and,

HVAC: American Standard.

I found similar at the UK take, Which. Everything is boiled down to star ratings and then Mail Merge creates the review text.

Apparently, each air purifier which can handle a large room is big, heavy and loud. And the air purifiers that score highly on being quiet have the downside that they can only handle small rooms. Oh, and they did measure the CADR, and will tell you that "this air purifier scored five stars on our CADR test".

I've noticed a tendency for them to review spec sheets; the whole point of a reviewer should be to do the in-depth checking and verification that I cannot do. I want someone to speak to how long the model has been sold, parts availability, repairability, etc.

Some of this can't be entirely determined until years after the product is released but you can check the company.

As for me, I went with SpeedQueen for the washer/dryer and wish I could find an equivalent company for refrigerators, but I basically consider those disposable.

My library has a subscription to consumer reports, don't have to pay for it. I only bring this up because you said you paid for it. Worth checking if you have some local resource that has a subscription already.

Wirecutter is exceptionally "affiliaty".

Nonsense. As a non-profit founded in 1926 with 50 testing labs and partnerships with outside labs, the proof is in the test after test after test after test that is unbiased and, more importantly, the criteria and testing methods are always available and reproducible! So when you don't agree with their rankings you can at least agree that there methods are clear, not based on SEO, not based on spam, not based on money, and pro-consumer.

I gave up on Consumer Reports many, many years ago when they got caught taking kickbacks from tire manufacturers.

Agree. https://www.tweakers.net rocks !

Also, to use Amazon I search for the 1/2/3-star reviews and see if I disagree with the given reason for the low review. If I disagree, I will contemplate buying the product.

I also subscribe to choice. They do lots more than product reviews. They have general guides on how to get the best use out of products, mystery shoppers reviewing customer service, tools to compare health insurance, etc. The reason I stay subscribed is they campaign about consumer rights issues which actually cause industries as a whole to change, e.g. bank fees. They also have the shonky awards which usually gets a bit of media attention each year, where they shame companies for poor behaviour, quality, outright scams, etc. I think of it like a lobby to help us consumers out, which we need more of. Too often we are listening to the advice of the industry bodies that represent the companies rather than the consumers.

ServeTheHome does some review-like stuff, but its not entirely detailed though they do actually run the hardware and measure things like noise, power, etc.

Yeah but it’s little more than an SEO farm these days.

rtings.com only covers a few categories but they cover those categories really well and they do a excellent job of testing those products.

I'm also really skeptical about their impartiality; I wouldn't be surprised if they have a lot of underhanded deals with whatever they review and advocate for.