10G networking on an Apple M1 Mac Mini is wow.

There we go... 10 gig networking on an M1Mac mini that's just, wow! Ah, well, maybe, the thing is just because aport is present on your computer doesn't mean that it willoperate at full speed, and it certainly doesn't mean it will operate at fullspeed all the time. USB for example, is ratedby the speed that is shared across multiple ports, high speed I/O and SSDslots often share resources, meaning that both of them will slow down if you use them simultaneously. And in the early daysof gigabit networking basically no motherboards could run those ports Apple's speed because they were connectedto the lethargic PCI bus, you know the old non Express PCI bus. Of course, we only know these things because vendors like Intelpublished block diagrams of their platforms to help usunderstand their capabilities. Apple on the other handdoes no such thing, meaning that if thereare internal bottlenecks, the only way to find out about them is to have your performance drop while you're in themiddle of doing something. That makes today's adventureinteresting for two reasons, one, it'll tell us if Apple delayed thisversion of the M1 mini perhaps over performance concerns and two, it might giveus some fresh new insight into the black box that is Apple's M1 SoC and it also might give us some insight into our sponsor, Honey. Honey is the free to use shopping tool that helps search for someof the best promo codes on lots of your favorite sites. Get it today a joinhoney.com/LTT. (upbeat music) The M1 Mac mini blew us awaynot just with its performance, but also with its price. It beat out every previous Intel Mac and single-threadedbenchmarks and a Ryzen 5 5600X one of the fastest single threaddesktop CPU's in the market all without breaking the bank. It didn't have the cores to back that up in heavily multithreaded tasks, but so far that hasn't generated many, if any user complaints thanks to M1's tightintegration with macOS and of course theavailability of the Mac Pro if you really need more horses. What has generated complaints is the I/O, with just two USBs, two Thunderbolt ports, and a single one gigabit ethernet jack M1 mobile pedigree seemedto be holding it back for people like me wholoved the CPU performance but need to work with heavy video footage or other shared resources over a network. If one gig wasn't enough you had to use a bulky expensive dongle and use up one of your Thunderbolt ports, or you had to buy into adead end Intel platform, which was so disappointingfor me personally, after seeing Apple push 10 gig forward over the last few years, first with the iMac Pro,then with the Mac mini, but it turns out I justdidn't have enough faith, in my hand right now is ashiny new 10 gig version of the M1 Mac mini and we're gonna be runningIperf on it as a server to give us full controlover what other ports or devices might be hoggingPCI Express bandwidth causing potential bottlenecks. So to start, we were running an absolutebest case scenario here with nothing plugged into it other than networking andpower, not even our display. Once I come over here andhave a look, that is... Dang! That is pretty stable. 9.25, 9.45 gigabits per second. Not too shabby 'cause remember guys 10 gigabit is the line speed, and it's perfectly normalfor the actual data speed to be a bit lower. What's not normal is the price for our 40 ounce waterbottles on lttstore.com, they're the same as 21 ounce, what a deal. Now let's up the ante andconnect a Pro Display XDR but Linus you might ask, why would plugging in a monitor affect how much bandwidth is leftover for something like high-speed networking? I'm so glad you asked,the M1 Mac mini is notable for being the first mini with DSC or Display Stream Compression. And what that does, is it allows it to haveenough leftover bandwidth to run the integratedUSB 3 hub on this monitor at full speed. Previous Intel-based Macminis ended up running either not at the full 6K resolution or in an uncompressed tiled mode that some users have reportedactually took this hub and knocked it all theway back to USB 2 speeds. That's because uncompressed, this monitor will suck up about 34 gigabitsper second of bandwidth. To be clear, I'm notactually expecting this to throttle our network speed. It's just an excellent illustration of how a system has afinite amount of resources that needs to be carved up. Also, now that we've got our display up, you can see that the terminal on our mini matches our SSH session so no smoke and mirrorshere ladies and gentlemen. Well, let's go ahead andrun it again, shall we, and as expected we are exactly the same. I mean, no computermanufacturer in their right mind would rate the speed of their ports assuming that no display has plugged in especially not a display they make. But yeah, we're gonnakick things up a notch and to do so Anthony repurposed, some of these shell scriptthat he wrote previously that constantly writes data to an SSD and then spits out speedinfo regular intervals for graphing purposes, and he targeted it at this high performance Thunderbolt SSD. Maybe with this thing running, we'll finally start tosee some bandwidth sharing in our Iperf test. All right, Apple, allright, I'm impressed. Our SSD right now is pullingover 10 gigabits per second in reads putting us darn close to the, up to 40 gigabits per second number that's advertised for these ports but all that tells us so far is that the two Thunderboltports don't share bandwidth, so let's continue. Remember the USB hub I mentioned on the back of the Pro Display XDR well we're gonna loadit up with USB-C SSDs. We're gonna run that SSD script on all three of these at the same time, then we're gonna run our network test and see if we can makethis poor Mac mini cry. And good news, sort of,our SSDs did slow down, that means we managed to find a bottleneck but it's in the USB hubof the Pro Display XDR which presents us witha couple of problems. Number one, that doesn'tactually tell us anything about internal bottlenecksof the M1 Mac mini, and number two is we arefresh out of USB-C ports to plug SSDs into. Unless a Thunderbolt dock, ah, yes! We need a Thunderbolt dockand more SSDs. (laughs) We're gonna plug all these in and then we're gonnarun the script on them and then see how you like that. There we go. Five SSDs plugged into thehub and then for good measure an additional oneanother Thunderbolt 3 one plugged into the Thunderbolt Daisy-chain. Guys, if that 10 gig ethernet is stealing bandwidth from somewhere we are going to find it. Our pre-planning did not havethis actually go down at all. We figured it out, one of the discs wasnot showing up properly and running the commandto a missing target was causing something, I don't know, maybe somekind of CPU overhead. Anyway, now that allthe disks are showing up we are getting the results we expected, which is anywhere from 9.3to 9.42 gigabit per second and this is even with allof the drives running. - The CPU usage while we'rejust doing this is around 18% and that's with it allthe drives plugged in. If I add that drive that's not plugged in, what's our CPU usage now 32, 36 significantly higher. - Interesting. - I didn't test this on thebench, I didn't expect this. - Hey, it's done. - 40, 77. (laughing) - Okay, Wow! Way to write a script that hurts computers when you do it wrong. But while our side adventure interesting, you know, Hey it turnsout runaway CPU usage is going to affect networktransfer performance. It wasn't really thepoint of the video today. So let's refocus and talk aboutwhat we've learned so far. There's a few things actually. So one, we now know that the 10 gig M1 is legit non-shared 10 gig, no switch chips, no controller hubs just a whole Gen 4 PCI Express lane. That's why nothing that wedid was able to bottleneck it. So clearly Apple's delay hadnothing to do with performance and you can buy thispuppy with confidence. That's cool, but we're not done, diving deeper while ourtheoretical max speed when testing the drives individually works out to just over58 gigabits per second. It capped out somewherearound 50 in the real world and that means that wedid throttle somewhere. The SSDs connected to the Pro Display XDR were obviously going to throttleas they were being crowded by that chunky 13.7 gigabit per second display port a signal. But the Thunderbolt 3 hubsthrottling is another story, and this one comesprimarily down to the fact that each Thunderbolt 3 portappears to be capping out at around 20 to 22 gigabits per second. But hang on a second Linus, Apple said each of these ports could do up to 40 gigabit per second, and you said they didn'tshare bandwidth, what gives? So remember how our 10 gigNIC is expected to be less than 10 gigabit persecond in the real world? Well, the same is actuallytrue for Thunderbolt 3, USB for whatever the hell this thing is. So around 7.6 gigabit per second of that is set aside for our display port, which are Thunderbolt 3 dock has, and then the rest of it getseaten up by the signal encoding bring us down to a theoretical maximum of only around 26 gigabits per second. And that's close enough to our numbers that the difference probably comes down to the fact that we'regoing through busy hubs rather than attaching directly, hence the wavy patterns on the graphs compared to the individual drive results. That means that if you've got applications that are sensitive toI/O variants, like audio or anything involving real-time data you would do best to directly attach or use only one high-speed device on a hub or dock at a time. Of course, for mostpeople this is academic and it won't make a difference real world. Well sort of becauseM1 is still I/O limited and it has affected the portoptions on Apples M1 products, like for example, the newiMac which we'll be reviewing, so make sure you're subscribed. The question just becomes,how limited is it? What it looks like is we'vegot two PCI Express Gen 4 lanes going to Apple's Thunderbolt controller which steps down to twoThunderbolt three channels, then we've got one Gen 4lane for the 10 gig NIC, there's 10 gigabits to goaround for the Type-A ports, so that's another lane that is maybe sharedwith the Wi-Fi 6 adapter, and then there's an HDMI portconnected directly to the GPU via an internal displayport to HDMI converter. That's your I/O, four lanes accounted for. We ran the I/OLGA terminal command to try and find out for sure but while it does show device hierarchies it doesn't show how they'reconnected to the main system just that they are connected. Of course, most PC systems have a lot more than four PCI Express lanes. Surely M1 does as well then right? Well, not necessarily. You might think it would haveanother four lanes at least for its SSD, but it doesn't. Remember how the T2 chipthat we loved so much was the storage controller for the final generation of Intel Macs and then it just disappeared with the M1. Well, the reverse engineering effort for getting M1 running on Linux has actually discoveredthat Apple wasn't kidding. The M1 SoC has its securitychip built right into it and it's controlling the SSD directly through some kind ofmemory map arrangement, not through PCI Express and you got to rememberNVMe is a protocol, so while we associate it with PCI Express it doesn't technically haveto run over PCI Express. All of which is to saythen that the M1 Mac mini is probably near the limitsof what Apple can manage with the M1 SoC but it's got far more I/O than we initially gave it credit for because remember wehaven't even accounted yet for whatever bus they'reusing for the internal GPU. So the reason they heldoff on the 10 gig version was probably more to dowith logistics or strategy, but whatever it wasApple has done a lot here with little and that's pretty admirable kinda like how Apple used to be back in the good old daysand kinda like my segues to sponsors like MicroCenter, get the best prices and best selection on computer hardware and everything else technologyat any one of Micro Center's 25 locations across the United States. Micro Center's custom PC builder will help you spec out thebest PC for your budget by making sure all yourparts are compatible finding stock that's available at your nearest Micro Center location making it so you canjust add it to your cart and arrange same day in store pickup. For a fee you can check the boxmarked Same Day Pro Assembly to have a Micro Center expert technician assemble the PC for youand if you want help deciding what parts to putin your new custom gaming PC just join the new onlineMicro Center community. It's a great place to talktech with other enthusiasts. Check out the link in the description for a free pair of wirelessBluetooth headphones valid in store only no purchase necessary. So thanks for watching guys, check out our review of the M1 Mac mini when it first launched to get just a little bitmore of an appreciation for how much of a game changer this modest piece of Silicon could be.