This is a reformat of the setup I first did for these storage units in 2010. The current setup is 8 drive RAID-6 stripes and a bit different, but this is here for past documentation.
High large-file-performance NAS system, great for media storage, designed for:
- Full GbE throughput for read/write (125MB/sec read/write target) on single large file
- RAID-5 redundancy
- Low noise, low power, low cooling needs
- 10TB usable storage
- Stock Linux system, no special drivers/kernel/experimental code/etc, and still easily extended as a Linux server if desired
Part Cost "Case - Antec Three Hundred Gaming Case
Roomy space for lots of drives, quiet fans, and good ventilation. 6 x 3.5” bays, 3 x 5.25” bays." 59.82 "Proc - Intel Celeron E3400, 2.60Ghz, 1MB CACHE, 800MHZ FSB, LGA775
Best Intel bang for the buck if you need more power than an Atom-based system but want to keep things on a budget. Alternative for a very low power system is Atom board/cpu combo, sacrificing some peak throughput which will be cpu bound by samba on an Atom cpu, but still offering 40MB/sec+ throughput over network. More expensive/faster processors are of course an option if the NAS will dual purpose with other cpu-intensive tasks." 50.24 "MB - Intel Core2 Quad/LGA 775/Intel G41/FSB 1333/2DDR2-800/Intel GMA X4500/GbE/5.1-CH/VGA Micro ATX Motherboard
Stable no-frills LGA775 board with 4 onboard SATA ports. Another option would be to go with a more extravagant board with more onboard SATA with the possibility of dropping the extra SATA controller card below." 60.99 "Boot Drive - Western Digital 80 GB Caviar Blue SATA 7200 RPM 8 MB Cache Bulk/OEM Desktop Hard Drive WD800AAJS
Faster than using a USB boot drive with gads of room for logs, etc. Low capacity 2.5” boot drives are also a great option and can be a few dollars cheaper. The downside of a SATA boot drive over USB is the use of a SATA port that could be used for another share drive." 36.99 "Power - Antec EA-380D Power Supply EA-380D
We’re willing to spend a bit more than the minimum here, as it more than pays for itself in power bills within a few months over a less efficient supply. This Antec is an 80plus power supply that still runs at 79% efficiency at only 10% power draw (38W), with plenty of overhead for lots of drives or a more powerful processor. The high efficiency at very low power draw makes this attractive for a machine that will probably be sitting idle 90+% of the time." 44.99 "RAM - 2 x Kingston ValueRAM 1GB 800MHz PC2-6400 DDR2 Desktop Memory (KVR800D2/1GR)
Not much point in going over 2GB for basic operation. Anything over 512MB is basically for some filesystem caching and any optional daemons/apps that might be installed on the NAS." 45.00 "Syba PCI Express SATA II 4 x Ports RAID Controller Card SY-PEX40008
PCI-E is an absolute must here for bandwidth reasons. This card uses the SIL3124 chipset which works great as raw SATA devices under any recent Linux distribution." 58.50 StarTech.com METAL 5.25 TO 3.5IN DRIVE-ADPT BRACKET 4.23 7 x Tripp Lite P940-19I Serial ATA (SATA) Signal Cable (19 Inches) 28.77 Syba SY-CAB40007 Molex 4 Pin to 2x 15 Pin SATA Power Cable (5.5 Inches) 2.54 6 x Western Digital 2 TB Caviar Green SATA Intellipower 64 MB Cache Bulk/OEM Desktop Hard Drive WD20EARS 509.94
Subtotal for everything except the raid drives: $392.07 With 12TB raw drives for 10TB usable RAID-5: $902.01
- Ubuntu 64-bit server LTR 10.04
- During install, pick sshd and Samba metapackages. Don’t do anything with the 2TB drives. Just install on the 80GB drive and leave those drives untouched and unpartitioned. No software raid or LVM during install.
- Install iostat, very useful for monitoring.
# apt-get install sysstat
We’ll need to install xfs utilities since they aren’t installed by default on Ubuntu.
# apt-get install xfsprogs
# dmesg | grep "512-byte" [ 0.670573] sd 2:0:0:0: [sda] 156301488 512-byte logical blocks: (80.0 GB/74.5 GiB) [ 0.670835] sd 2:0:1:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB) [ 0.671329] sd 3:0:0:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB) [ 0.671543] sd 3:0:1:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB) [ 3.497021] sd 4:0:0:0: [sde] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB) [ 6.142257] sd 5:0:0:0: [sdf] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB) [ 8.836352] sd 6:0:0:0: [sdg] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
# mdadm --create /dev/md0 --chunk=64 --level=5 --raid-devices=6 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg
(going to use xfs for speed/reliability)
# mkfs.xfs -d sunit=128,swidth=640 -f /dev/md0
Add the following line to /etc/fstab:
/dev/md0 /opt/fs1 xfs sunit=128,swidth=640 0 2
# mdadm --examine --scan ARRAY /dev/md0 level=raid5 num-devices=6 UUID=587b1607:e939d29e:b948c19b:e2871408
In /etc/mdadm/mdadm.conf, put the output of the mdadm examine line above in the section for known arrays.
# df -B 1000000 Filesystem 1MB-blocks Used Available Use% Mounted on /dev/sda1 72837 1315 67822 2% / none 1029 1 1029 1% /dev none 1034 0 1034 0% /dev/shm none 1034 1 1034 1% /var/run none 1034 0 1034 0% /var/lock none 1034 0 1034 0% /lib/init/rw none 72837 1315 67822 2% /var/lib/ureadahead/debugfs /dev/md0 9999858 6 9999852 1% /opt/fs1
# dd if=/dev/zero of=testfile bs=1000000 count=10000 10000000000 bytes (10 GB) copied, 76.9677 s, 130 MB/s
# dd if=testfile of=/dev/null bs=1000000 10000000000 bytes (10 GB) copied, 26.1015 s, 383 MB/s
# bonnie++ -d /opt/fs1/share1/ -s 10g -f -b -u samba-anon Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP zare1 10G 111449 19 88581 23 387771 42 198.3 6
- SIMPLE NO ACCESS CONTROL SAMBA CONFIG SECTION (user samba-anon should own share mount point)
[share1] comment = Share1 path = /opt/fs1/share1 force user = samba-anon force group = users read only = No guest ok = Yes
- hdparm -S (for drive idle standby)