{"id":444,"date":"2021-10-17T12:00:02","date_gmt":"2021-10-17T12:00:02","guid":{"rendered":"https:\/\/andrejacobs.org\/?p=444"},"modified":"2022-04-11T20:22:59","modified_gmt":"2022-04-11T20:22:59","slug":"setting-up-2x-14tb-seagate-ironwolf-pro-drives-raid-1-on-my-ubuntu-linux-nas","status":"publish","type":"post","link":"https:\/\/andrejacobs.org\/linux\/setting-up-2x-14tb-seagate-ironwolf-pro-drives-raid-1-on-my-ubuntu-linux-nas\/","title":{"rendered":"Setting up 2x 14TB Seagate IronWolf Pro drives (RAID 1) on my Ubuntu Linux NAS"},"content":{"rendered":"\n
Photo by Gerald Sch\u00f6mbs<\/a> on Unsplash<\/a><\/p>\n\n\n\n I run an Ubuntu server locally in my house that primarily serves as a NAS. In the past I have used a couple of [redacted] consumer NAS devices, but this ended up giving me a really bad experience and thus I would rather now run my own setup.<\/p>\n My NAS has been filling up of late and it is time to upgrade the storage to have 2x 14TB Seagate IronWolf Pro drives that I will be running in RAID 1 as a mirror.<\/p>\n The plan is:<\/p>\n <\/p>\n To give you context of my server. It is a normal PC running Ubuntu 18.04 LTS (for the moment). The OS runs off a 240GB SSD drive and is full disk encrypted. Which means I need to enter the passphrase at every boot. Inside of the PC are various other RAID 1 and single drives all full disk encrypted as well. Most of the important data is stored over 4TB (RAID 1) and 3TB (RAID 1) which I want to transfer over to the new 14TB drives.<\/p>\n Once the tasks are done, I will remove the 2x 4TB drives and store them somewhere for a few months until I know for sure the 2x 14TB drives have no issues. At that point I will reuse the drives for auxiliary data.<\/p>\n Later in the process I will be setting up scripts to run long running tasks and they will need to email me when the tasks are done.<\/p>\n I have previously setup the server to send out email to me for events like when the CPU gets too hot etc.<\/p>\n For now I just need to verify that I can still receive email as expected. Who watches the watchmen?<\/strong><\/p>\n I will be first setting up the new drives and backing up the old drives before clean installing Ubuntu 20.04 LTS. Why not do this first? Well I need to confirm that the drives actually work etc. and I know what a black hole installing and setting up Linux can be and don’t want to waste time on that and then have issues returning faulty drives if needed.<\/p>\n <\/p>\n <\/p>\n <\/p>\n Now this is the part where if you are not being careful then you will be crying<\/strong> over data lost<\/strong> very soon. TRIPLE CHECK<\/strong> every single command and make sure it is the correct drives being used. In fact I would suggest you be super paranoid and unplug every drive in the system that you don’t want to screw up.<\/p>\n I will be using partitions for the raid devices instead of the full disk. The reason for this is that I will reserve 200MB of unused space in the event that in the future we need to replace a disk that is slightly smaller than the working disk (because of different brands etc.)<\/p>\n <\/p>\n NOTE:<\/strong> I decided to not use LVM to further partition the RAID, but instead use it as a single full disk encrypted device.<\/p>\nOverview<\/h2>\n
\n
Backing up<\/h2>\n
\n
screen<\/code> so that I can return to the shell session at any time.<\/li>\n<\/ul>\n
$ screen\n... press enter\n<\/code><\/pre>\n
\n
$ sudo su\n$ mkdir -p \/media\/tiger\/sda_backups\/\n$ dd if=\/dev\/sda of=\/media\/tiger\/sda_backups\/sda_20211001.img\n<\/code><\/pre>\n
\n
Ctrl+a d\n<\/code><\/pre>\n
You’ve got mail!<\/h2>\n
$ echo "Howzit boet! Kyk net gou of die ding not lekker werk." | \/usr\/bin\/mail -s "[SIMBA] Testing 123" someone@email.address\n<\/code><\/pre>\n
Setting up the new drives<\/h2>\n
Installing the new drives<\/h3>\n
\n
$ sudo shutdown -h now\n<\/code><\/pre>\n
\n
\n
\n
Run SMART test to check the disks are ok<\/h3>\n
\n
$ lsblk\n...\nsdb 8:16 0 12.8T 0 disk\nsdc 8:32 0 12.8T 0 disk\n<\/code><\/pre>\n
\n
# First check the drive supports SMART\n$ sudo smartctl -i \/dev\/sdb\n...\nSMART support is: Available - device has SMART capability.\nSMART support is: Enabled\n\n# Do the same for \/dev\/sdc\n<\/code><\/pre>\n
$ sudo smartctl -H \/dev\/sdb\nsmartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.15.0-159-generic] (local build)\nCopyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org\n\n=== START OF READ SMART DATA SECTION ===\nSMART overall-health self-assessment test result: PASSED\n\n# Do the same for \/dev\/sdc\n<\/code><\/pre>\n
\n
$ sudo smartctl -c \/dev\/sdb\n...\nShort self-test routine\nrecommended polling time: \t ( 1) minutes.\nExtended self-test routine\nrecommended polling time: \t (1252) minutes.\nConveyance self-test routine\nrecommended polling time: \t ( 2) minutes.\n<\/code><\/pre>\n
\n
$ screen\n$ sudo smartctl -t short \/dev\/sdb\n...\n=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===\nSending command: "Execute SMART Short self-test routine immediately in off-line mode".\nDrive command "Execute SMART Short self-test routine immediately in off-line mode" successful.\nTesting has begun.\nPlease wait 1 minutes for test to complete.\nTest will complete after Sat Oct 2 14:22:50 2021\n\n# I waited till after the suggested time and viewed the results\n$ sudo smartctl -l selftest \/dev\/sdb\n...\nNum Test_Description Status Remaining LifeTime(hours) LBA_of_first_error\n# 1 Short offline Completed without error 00% 3 -\n\n# Note you can also use the following to get lots of info: sudo smartctl -a \/dev\/sdb\n<\/code><\/pre>\n
# Time for the long test\n$ sudo smartctl -t long \/dev\/sdb\n...\n=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===\nSending command: "Execute SMART Extended self-test routine immediately in off-line mode".\nDrive command "Execute SMART Extended self-test routine immediately in off-line mode" successful.\nTesting has begun.\nPlease wait 1252 minutes for test to complete.\nTest will complete after Sun Oct 3 11:17:42 2021\n\n# Repeat the above for \/dev\/sdc\n# Interestingly enough, the test is estimated to be quicker on this drive\nPlease wait 1237 minutes for test to complete.\nTest will complete after Sun Oct 3 11:04:12 2021\n<\/code><\/pre>\n
\n
$ screen -ls\nThere are screens on:\n\t14810.checksum-hh\t(03\/10\/21 11:17:26)\t(Detached)\n\t14722.checksum-gw\t(03\/10\/21 11:14:28)\t(Detached)\n\t3206.pts-0.simba\t(02\/10\/21 11:34:09)\t(Detached)\n$ screen -r 3206.pts-0.simba\n \n$ sudo smartctl -l selftest \/dev\/sdb\n...\n=== START OF READ SMART DATA SECTION ===\nSMART Self-test log structure revision number 1\nNum Test_Description Status Remaining LifeTime(hours) LBA_of_first_error\n# 1 Extended offline Completed without error 00% 21 -\n\n# Repeat for \/dev\/sdc\n<\/code><\/pre>\n
Setup RAID 1 and full disk encryption<\/h3>\n
\n
$ lsblk\n...\nsdb 8:16 0 12.8T 0 disk\nsdc 8:32 0 12.8T 0 disk\nsdd 8:48 0 3.7T 0 disk\n\u2514\u2500sdd1 8:49 0 3.7T 0 part\n \u2514\u2500md0 9:0 0 3.7T 0 raid1\n \u2514\u2500greatwhite 253:5 0 3.7T 0 crypt \/media\/greatwhite\nsde 8:64 0 3.7T 0 disk\n\u2514\u2500sde1 8:65 0 3.7T 0 part\n \u2514\u2500md0 9:0 0 3.7T 0 raid1\n \u2514\u2500greatwhite 253:5 0 3.7T 0 crypt \/media\/greatwhite\n...\n<\/code><\/pre>\n
\n
$ sudo wipefs --all --backup \/dev\/sdb\n$ sudo wipefs --all --backup \/dev\/sdc\n<\/code><\/pre>\n
\n
$ history\n...\n1384 sudo wipefs --all --backup \/dev\/sdb\n1390 sudo wipefs --all --backup \/dev\/sdc\n...\n\n$ history -d 1384\n$ history -d 1389 # Note the line numbers change when you delete a line (duh!)\n<\/code><\/pre>\n
\n
$ sudo cfdisk \/dev\/sdb\n\n** Select label type: gpt (press enter)\n\nDisk: \/dev\/sdb\n Size: 12.8 TiB, 14000519643136 bytes, 27344764928 sectors\n Label: gpt, identifier: 2CE64109-8EDF-6D45-8E68-45EFA6E7DFDD\n\n Device Start End Sectors Size Type\n>> Free space 2048 27344764927 27344762880 12.8T\n~~~~\n\n** Select [ New ] and press enter\n** Enter the new desired size. In my case 14000319643136\n\nDevice Start End Sectors Size Type\n>> \/dev\/sdb1 2048 27344375807 27344373760 12.8T Linux filesystem\n Free space 27344375808 27344764927 389120 190M\n\n** Select [ Type ] and press enter\n** Scroll down in the list and select Linux RAID (press enter)\n\n<\/code><\/pre>\n
\n
\n
$ lsblk\n...\nsdb 8:16 0 12.8T 0 disk\n\u2514\u2500sdb1 8:17 0 12.8T 0 part\nsdc 8:32 0 12.8T 0 disk\n\u2514\u2500sdc1 8:33 0 12.8T 0 part\n...\n<\/code><\/pre>\n
Create the RAID 1 (mirror)<\/h3>\n
\n
$ cat \/proc\/mdstat\n\nPersonalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]\nmd2 : active raid1 sdf1[0] sdg1[1]\n 1953285120 blocks super 1.2 [2\/2] [UU]\n bitmap: 0\/15 pages [0KB], 65536KB chunk\n\nmd0 : active raid1 sdd1[1] sde1[0]\n 3906788352 blocks super 1.2 [2\/2] [UU]\n bitmap: 0\/30 pages [0KB], 65536KB chunk\n\nmd1 : active raid1 sdh1[1] sdi1[0]\n 2930037760 blocks super 1.2 [2\/2] [UU]\n bitmap: 4\/22 pages [16KB], 65536KB chunk\n<\/code><\/pre>\n
\n
$ sudo mdadm --create --verbose \/dev\/md5 --level=mirror --raid-devices=2 \/dev\/sdb1 \/dev\/sdc1\n...\nContinue creating array? yes\nmdadm: Defaulting to version 1.2 metadata\nmdadm: array \/dev\/md5 started.\n\n$ cat \/proc\/mdstat\n\nPersonalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]\nmd5 : active raid1 sdc1[1] sdb1[0]\n 13672054784 blocks super 1.2 [2\/2] [UU]\n [>....................] resync = 0.0% (7685568\/13672054784) finish=1136.3min speed=200404K\/sec\n bitmap: 102\/102 pages [408KB], 65536KB chunk\n<\/code><\/pre>\n
\n
$ cat \/proc\/mdstat\n\nPersonalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]\nmd5 : active raid1 sdc1[1] sdb1[0]\n 13672054784 blocks super 1.2 [2\/2] [UU]\n bitmap: 0\/102 pages [0KB], 65536KB chunk\n<\/code><\/pre>\n
\n
# First backup the existing mdadm.conf\n$ sudo cp \/etc\/mdadm\/mdadm.conf \/etc\/mdadm\/mdadm.conf.bak\n\n# Append to mdadm.conf (this will make editing easier)\n$ sudo -s -H\t\t # become root\n$ mdadm --detail --scan >> \/etc\/mdadm\/mdadm.conf\n$ exit\t\t\t\t\t # exit root shell\n\n# Check the diff\n$ vimdiff \/etc\/mdadm\/mdadm.conf \/etc\/mdadm\/mdadm.conf.bak\n\n# Now to edit the file\n$ sudo -s -H\n$ vi \/etc\/mdadm\/mdadm.conf\n\n# This is what my new file looks like md0 became md7 and the new 14TB drives become md0\n# DO NOT change the name (e.g. name=simba:X) because it won't work then\n\n # Megalodon 14TB\n ARRAY \/dev\/md0 metadata=1.2 name=simba:5 UUID=4b2397d7:5cb88116:eee8b5a1:1c72b091\n # Hammerhead 3TB\n ARRAY \/dev\/md1 metadata=1.2 name=simba:1 UUID=11a0d9f1:49b7ccf4:313c4161:871a3c6b\n # Timemachine 2TB\n ARRAY \/dev\/md2 metadata=1.2 name=simba:3 UUID=e659b734:4dc6956e:7b6e6df7:3d3da262\n # Greatwhite 4TB (will be stored offsite)\n ARRAY \/dev\/md7 metadata=1.2 name=simba:0 UUID=de63af76:8148120c:cd7cf3e6:75a6d28e\n\n# You must update-initramfs!\n$ update-initramfs -u -k all\n<\/code><\/pre>\n
\n
$ sudo shutdown -r now\n...\n<back online>\n$ cat \/proc\/mdstat\n...\nmd7 : active raid1 sdd1[1] sde1[0]\n 3906788352 blocks super 1.2 [2\/2] [UU]\n bitmap: 0\/30 pages [0KB], 65536KB chunk\n\nmd0 : active raid1 sdb1[0] sdc1[1]\n 13672054784 blocks super 1.2 [2\/2] [UU]\n bitmap: 0\/102 pages [0KB], 65536KB chunk\n...\n<\/code><\/pre>\n
Encrypting the disks<\/h3>\n
\n
\/dev\/md0<\/code><\/li>\n<\/ul>\n
$ lsblk\n...\nsdb 8:16 0 12.8T 0 disk\n\u2514\u2500sdb1 8:17 0 12.8T 0 part\n \u2514\u2500md0 9:0 0 12.8T 0 raid1\nsdc 8:32 0 12.8T 0 disk\n\u2514\u2500sdc1 8:33 0 12.8T 0 part\n \u2514\u2500md0 9:0 0 12.8T 0 raid1\n<\/code><\/pre>\n
\n
task-enc.sh<\/code> DO NOT RUN THIS<\/strong> as is without modifying it for your own purposes. This will override \/dev\/md0!<\/strong><\/li>\n<\/ul>\n
#\/!bin\/bash\n\necho "Writing random data to \/dev\/md0"\nopenssl enc -aes-256-ctr -pass pass:"$(dd if=\/dev\/urandom bs=128 count=1 2>\/dev\/null | base64)" -nosalt < \/dev\/zero | pv -pterb | dd bs=4096 of=\/dev\/md0\n\nEMAILTO="someone@email.address"\nEMAILBODY="Finished writing random data"\necho $EMAILBODY | \/usr\/bin\/mail -s "[SIMBA] Writing random data" $EMAILTO\n\necho ""\necho "Done!"\n<\/code><\/pre>\n
# Start screen because this is going to be a long task\n$ screen -S enc\n$ sudo -s -H\n$ .\/task-enc.sh\n\nWriting random data to \/dev\/md0\n*** WARNING : deprecated key derivation used.\nUsing -iter or -pbkdf2 would be better.\n4.94GiB 0:00:16 [ 245MiB\/s] [ <=> ]\n...\n# A day later, it finished\ndd: error writing '\/dev\/md0': No space left on device <=> ]\n12.7TiB 21:02:49 [ 176MiB\/s] [ <=> ]\nerror writing output file\n3418013697+0 records in\n3418013696+0 records out\n14000184098816 bytes (14 TB, 13 TiB) copied, 75787.4 s, 185 MB\/s\n<\/code><\/pre>\n