The 250TB Project

– Hey guys, this is Austin And this is the 250 terabyte project

Now this has been one of the biggest things that we've ever undertaken as we literally started working on this last year and we've literally been shooting this video ever since This entire project was really built around these two units from 45Drives The original Storinator which we did a couple years ago with Linus, as well as the brand new Stornado which is an all SSD server which allows us for everyone in the office to be connected via 10 Gig Ethernet, to share files, to edit things with tons of space and tons of speed Building an entire server for the first time has definitely not been a smooth process So this video actually begins in April when we first got our Stornado

So inside here, we have the Stornado Now unlike the Storinator which we used for our previous project, this is all SSD flash storage And 45Drives was awesome enough to hook us up with this for free because we're trying to do something a little bit bigger than what we've done in the past It's so heavy (laughing) So inside this Stornado is essentially what will be our new editing server

So right now, we use a series of Samsung T3 and T5 drives to do all of our edits Now they actually work pretty well and it's nice to have the portability but the problem is that say, Jimmy's working on a project tonight and I wanna take over, he has to eject the drive and hand it to me We basically have no redundancy if a drive fails or if someone loses something Whereas with this, everyone will be connected via 10 Gig Ethernet and theoretically, we can all work on the same project at the same time I wanna get this thing open

This is rad Wait, did they load us up with 64 GB of RAM? 8, 16, 32, wow Are you for real right now? (laughing) Dude, look how cool this is They gave us a custom front plate Did you design this? – Yes

– Oh, you did Oh, dude That's gonna be awesome So we can basically just swap that out? Oh, dude This is so cool

The next task was actually to get the Stornado up and running with a ton of super fast SSDs which happened in June So inside here, we have a whole bunch of 800 GB SSDs So these are the Nytro Series straight from Seagate who were very gracious to provide these for the Stornado project But essentially, with Actually how many drives do we have again? Four, five, six, seven So the cool part about these SSDs is that they're not just standard drives So let's actually open one of them up You'll see that these are SAS based

Now this is not something that you can just, say, throw into an average desktop PC That makes sense These are actual, proper enterprise class drives which is important 'cause this server is going to be up for years at a time We need to make sure that all these drives are as reliable as possible and importantly also, as fast as possible Especially when you talk about 10, 20, 30 drives in one single system, reliability becomes a huge, huge factor

And the last thing I wanna do is start editing a video and realize that, "Oh "Right "I lost my 31st drive "and the entire thing just caught on fire" Which thankfully is not the case here Also, it'd be weird if the project caught on fire but that would still suck

Now in July, I actually thought that we were pretty much done with this project The Stornado was up and running and it seemed like we were going to be ready to go That is surprisingly quiet for an open server All right, so we are up and running So right now we have 22

4 terabytes available Now this is not actually completely set up so the main thing I wanna do right now is just test to see what kind of optimal speeds We're going to wanna build some redundancy and we're going to wanna make sure that if multiple of these drives fail, Actually, no, what number are we gonna aim for? Probably like two drive failures before we lose anything? Look at that So we have our two 240 or 250 GB SSDs and then we have all of our 800 GB SSDs That my friends, Oh look, look

Wait, sorry Hang on There's a whole other page Keep going That's so many discs

Oh, I love it All right I think we're ready to actually start speed testing and see what we're able to get over our 10 Gig network So again, not totally final We still need to get our switch involved and kind of run lines to everyone

But this should give us a pretty good idea of what kind of speed we're going to be able to get over the network with all of our SSDs So to start with, we have a 28 GB file Now if I just paste it over here We're getting 4 gigabits a second Nice

All right, that's pretty fast So actually, really our bottleneck right now is just purely the SSD that we're using This is literally just isn't fast enough to saturate the 10 Gigabit network Oh, oh, I'm excited All right

So let me actually try to transfer something which is on the internal SSD on the MacBook which should be somewhere in the neighborhood of like, 2 GB per second Oh, that's fast That is 8 gigabits per second Or like, 30 seconds to transfer 25 GB file over the network Ooh, that's fast

So that is actually, 8 gigabits per second is about 1 gigabyte per second Yo, that's fast Oh Oh, this is good Fast forward to August and while it seemed like our server build was missing a few critical components

So we actually started to, you know, put that together So you guys have already seen the Stornado We also have our original Storinator which will make its way in here We also have some fun goodies This is one of my favorites

So this doesn't look like anything particularly cool However, inside, this houses a Mac Mini So there's a lot of reasons why we've chosen to go with a Mac Mini First of all, we actually do need something, especially for our LTO drive that does have Thunderbolt 3 But importantly, the Mac Mini is fairly affordable and it also has a 10 Gigabit Ethernet port, which means that this computer can live in here, live on this shelf

We'll also have a monitor, we'll have a keyboard so you can really easily walk up to the rack, do whatever it is that you need to do configuration wise, although hopefully a lot of that stuff will also be able to be done upstairs on the network but something such as the LTO will need to be run from the Mac Mini I love the fact that for a power cable up front, it literally hit the button, it just presses the power button on the back of the Mac Mini That's so cool So on top of the Storinator, we actually are going to have another Storinator at some point We have our

I forgot how heavy this is We have our Synology NAS with 70 terabytes of additional storage Dude, this, so heavy

– I thought you were the shredded one – So all of these different units, the Storinator, the Stornado, the Mac, the LTO drive, all this stuff needs to communicate somehow and primarily, we're using this 10 Gigabit Ethernet switch to run all these things upstairs so everyone upstairs will have 10 Gig Ethernet Honestly, we're just stealing Linus' setup – Kind of Yeah

– I mean We're stealing Linus' setup But to be fair, Linus was the one who literally hooked us up with the Storinator in the first place Like, at the very beginning, it was Linus' fault So we're just- – Completely his fault

– [Austin] We're just literally just always a couple years behind Linus So stay tuned We'll have 80% more ads two years from now So especially when you're dealing with the server, a UPS is incredibly important – The big thing that's cool about this though is that in the event of a power outage, there are a couple different banks of outlets in there

So when the power goes out, we can tell it, "Hey, prioritize these devices "Keep these on even longer "Shut down this one, this one first" And they can actually send a signal to the Storinator and the Stornado to say, "Hey I've lost power

"Start your safe shutdown process" So that way, nothing hard crashes – All right, let's get this stupid thing out Oh my God It's so heavy

All right Coming down And I am down So with that, we have our UPS installed So do you wanna try do the Stornado next? – Sure

– Be very careful not to drop our incredibly fast and expensive Stornado – That'd be rad So we're gonna just line it up Yup, line it up – Okay, okay

– We're good We're good That's fine That's fine We're good

Yup – Hey – Yup – It's working Oh my god

We got it – Boom – I mean, some slight tightening But we have the Stornado in the server rack – It's earthquake proof

– And now the server is finally done So it is October and now well, okay, I say done It's always going to be a work in progress But it is actually fully functional We're all using it right now

Now what happens when our storage is full? Or more importantly, what happens when we need more storage that needs to be safely backed up? Well, that's where our LTO drive comes in You see this empty hole in the rack? Yeah So LTO is magnetic tape and actually I wanna do a whole video on this at some point but essentially, this is a really cheap and long term backup solution However, currently our LTO drive is being repaired So it normally goes right here

This has been an unexpectedly huge project I did not think when we first start talking about doing this late last year, that it would take anywhere near this long to get everything up and running But the best part is that it is finally up and running now So as you can see, this has a lot of room to grow We have lots of additional space

And realistically, there's a lot more that I wanna do with this But the most important thing here is that we have a really easily accessible bit of storage for everything we need here in the office So as we edit videos, we can use the Stornado As we wanna back them up, we have the Storinator as well as we have our LTO drives And the cool thing is that there's so many more things we can do in the future that part 2 of this video will come out in 2021

Be the first to comment

Leave a Reply

Your email address will not be published.


*


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

This site uses Akismet to reduce spam. Learn how your comment data is processed.