Original Zare NAS setup 2010

From dtype.org

This is a reformat of the setup I first did for these storage units in 2010. The current setup is 8 drive RAID-6 stripes and a bit different, but this is here for past documentation.

Description

High large-file-performance NAS system, great for media storage, designed for:

  • Full GbE throughput for read/write (125MB/sec read/write target) on single large file
  • RAID-5 redundancy
  • Low noise, low power, low cooling needs
  • 10TB usable storage
  • Stock Linux system, no special drivers/kernel/experimental code/etc, and still easily extended as a Linux server if desired

Parts List

Part Cost "Case - Antec Three Hundred Gaming Case

Roomy space for lots of drives, quiet fans, and good ventilation. 6 x 3.5” bays, 3 x 5.25” bays." 59.82 "Proc - Intel Celeron E3400, 2.60Ghz, 1MB CACHE, 800MHZ FSB, LGA775

Best Intel bang for the buck if you need more power than an Atom-based system but want to keep things on a budget. Alternative for a very low power system is Atom board/cpu combo, sacrificing some peak throughput which will be cpu bound by samba on an Atom cpu, but still offering 40MB/sec+ throughput over network. More expensive/faster processors are of course an option if the NAS will dual purpose with other cpu-intensive tasks." 50.24 "MB - Intel Core2 Quad/LGA 775/Intel G41/FSB 1333/2DDR2-800/Intel GMA X4500/GbE/5.1-CH/VGA Micro ATX Motherboard

Stable no-frills LGA775 board with 4 onboard SATA ports. Another option would be to go with a more extravagant board with more onboard SATA with the possibility of dropping the extra SATA controller card below." 60.99 "Boot Drive - Western Digital 80 GB Caviar Blue SATA 7200 RPM 8 MB Cache Bulk/OEM Desktop Hard Drive WD800AAJS

Faster than using a USB boot drive with gads of room for logs, etc. Low capacity 2.5” boot drives are also a great option and can be a few dollars cheaper. The downside of a SATA boot drive over USB is the use of a SATA port that could be used for another share drive." 36.99 "Power - Antec EA-380D Power Supply EA-380D

We’re willing to spend a bit more than the minimum here, as it more than pays for itself in power bills within a few months over a less efficient supply. This Antec is an 80plus power supply that still runs at 79% efficiency at only 10% power draw (38W), with plenty of overhead for lots of drives or a more powerful processor. The high efficiency at very low power draw makes this attractive for a machine that will probably be sitting idle 90+% of the time." 44.99 "RAM - 2 x Kingston ValueRAM 1GB 800MHz PC2-6400 DDR2 Desktop Memory (KVR800D2/1GR)

Not much point in going over 2GB for basic operation. Anything over 512MB is basically for some filesystem caching and any optional daemons/apps that might be installed on the NAS." 45.00 "Syba PCI Express SATA II 4 x Ports RAID Controller Card SY-PEX40008

PCI-E is an absolute must here for bandwidth reasons. This card uses the SIL3124 chipset which works great as raw SATA devices under any recent Linux distribution." 58.50 StarTech.com METAL 5.25 TO 3.5IN DRIVE-ADPT BRACKET 4.23 7 x Tripp Lite P940-19I Serial ATA (SATA) Signal Cable (19 Inches) 28.77 Syba SY-CAB40007 Molex 4 Pin to 2x 15 Pin SATA Power Cable (5.5 Inches) 2.54 6 x Western Digital 2 TB Caviar Green SATA Intellipower 64 MB Cache Bulk/OEM Desktop Hard Drive WD20EARS 509.94

Subtotal for everything except the raid drives: $392.07 With 12TB raw drives for 10TB usable RAID-5: $902.01

OS/RAID/Filesystem Setup

  • Ubuntu 64-bit server LTR 10.04
  • During install, pick sshd and Samba metapackages. Don’t do anything with the 2TB drives. Just install on the 80GB drive and leave those drives untouched and unpartitioned. No software raid or LVM during install.
  • Install iostat, very useful for monitoring.
# apt-get install sysstat

We’ll need to install xfs utilities since they aren’t installed by default on Ubuntu. 

# apt-get install xfsprogs

# dmesg | grep "512-byte"
[    0.670573] sd 2:0:0:0: [sda] 156301488 512-byte logical blocks: (80.0 GB/74.5 GiB)
[    0.670835] sd 2:0:1:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    0.671329] sd 3:0:0:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    0.671543] sd 3:0:1:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    3.497021] sd 4:0:0:0: [sde] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    6.142257] sd 5:0:0:0: [sdf] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    8.836352] sd 6:0:0:0: [sdg] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)

# mdadm --create /dev/md0 --chunk=64 --level=5 --raid-devices=6 /dev/sdb 
/dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg

(going to use xfs for speed/reliability) 

# mkfs.xfs -d sunit=128,swidth=640 -f /dev/md0

Add the following line to /etc/fstab: 

/dev/md0	/opt/fs1	xfs	sunit=128,swidth=640	0	2

# mdadm --examine --scan
ARRAY /dev/md0 level=raid5 num-devices=6 UUID=587b1607:e939d29e:b948c19b:e2871408

In /etc/mdadm/mdadm.conf, put the output of the mdadm examine line above in the section for known arrays.

Performance

# df -B 1000000
Filesystem          1MB-blocks      Used Available Use% Mounted on
/dev/sda1                72837      1315     67822   2% /
none                      1029         1      1029   1% /dev
none                      1034         0      1034   0% /dev/shm
none                      1034         1      1034   1% /var/run
none                      1034         0      1034   0% /var/lock
none                      1034         0      1034   0% /lib/init/rw
none                     72837      1315     67822   2% /var/lib/ureadahead/debugfs
/dev/md0               9999858         6   9999852   1% /opt/fs1

# dd if=/dev/zero of=testfile bs=1000000 count=10000 
10000000000 bytes (10 GB) copied, 76.9677 s, 130 MB/s

# dd if=testfile of=/dev/null bs=1000000 
10000000000 bytes (10 GB) copied, 26.1015 s, 383 MB/s

# bonnie++ -d /opt/fs1/share1/ -s 10g -f -b -u samba-anon
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
zare1           10G           111449 19 88581  23           387771 42 198.3   6

Samba Configuration

              • SIMPLE NO ACCESS CONTROL SAMBA CONFIG SECTION (user samba-anon should own share mount point)

For /etc/samba/smb.conf: 

[share1]
   comment = Share1
   path = /opt/fs1/share1
   force user = samba-anon
   force group = users
   read only = No
   guest ok = Yes

Additional Considerations 

- hdparm -S (for drive idle standby)
Retrieved from "https://dtype.org/index.php?title=Original_Zare_NAS_setup_2010&oldid=278"