OneRNG on Arch Linux

If your a Linux Crypto guy you probably care about your entropy.  If you don’t, you problably should.

I recently purchased a OneRNG device from the kickstarter project.  Unfortunately it didn’t work out of the box with Arch Linux.  This blog post documents what I hit and how I overcame some of the issues to get my OneRNG working.

First thing I did was install the prerequisites:

yaourt -S at python-gnupg rng-tools
sudo systemctl enable atd.service
sudo systemctl start atd.service

as per the instructions at http://onerng.info/onerng/.  Note that we do NOT enable/start rngd.service as the daemon management is done from the udev stuff.

I then downloaded the tar file from https://github.com/OneRNG/onerng.github.io/blob/master/sw/onerng_3.4.orig.tar.gz?raw=true, verified the md5sum and the sha256 sum, and installed it with slightly tweaked instructions:

tar -xvzf onerng_3.4.orig.tar.gz
cd onerng_3.4
sudo make install
sudo udevadm control --reload-rules

I now plugged in my OneRNG with my fingers crossed.  At this point it died.  I did some brief reverse engineering and ended up figuring out how the /sbin/onerng.sh worked.  I then tried to run it manually and got:

$ sudo bash -ax /sbin/onerng.sh daemon ttyACM0
[snip]
Exception in thread Thread-7:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.4/site-packages/gnupg.py", line 753, in _read_response
result.handle_status(keyword, value)
File "/usr/lib/python3.4/site-packages/gnupg.py", line 284, in handle_status
raise ValueError("Unknown status message: %r" % key)
ValueError: Unknown status message: 'NEWSIG'
[snip]

This issue is logged upstream against python-gnupg and has been fixed but the current release doesn’t have it active. See the patch file at https://bitbucket.org/vinay.sajip/python-gnupg/commits/1337e6ce364f.

I manually applied the patch to /usr/lib/python3.4/site-packages/gnupg.py because I didn’t feel like rebuilding the package with the patch included.

Now when I unplugged the OneRNG and plugged it back in I noticed rngd successfully started in the background:

rngd -f -n 1 -d 1 -p /var/lock/LCK..ttyACM0 -r /dev/stdin

It is reading from stdin since there is an openssl pipe for aes whitening in front of it.  Unfortunately, removal didn’t result in the rngd process being killed. My udev foo is a little lacking so I hacked the onerng.sh script as follows:

--- /sbin/onerng.sh.orig        2015-06-20 21:27:33.000000000 -0700
+++ /sbin/onerng.sh     2015-07-04 16:31:45.478019740 -0700
@@ -158,7 +158,10 @@
 #      when something is removed kill the daemon
 #
 if [ "$1" = "kill" ]; then
-       if [ -e /var/lock/LCK..$2 ]
+       if [ -e /var/lock/LCK..ttyACM0 ]
+       then
+               kill -9 `cat /var/lock/LCK..ttyACM0`
+       elif [ -e /var/lock/LCK..$2 ]
        then
                kill -9 `cat /var/lock/LCK..$2`
        else

This enabled my pull of the OneRNG to kill the rngd process. Unfortunately this is a hack since if anything else used the OpenMoko ttyACM stuff it would kill it but I don’t have such things.  At this point I hit the “good enough for me” wall.

I verifed running

cat /dev/random > /dev/null

dimmed the LED on the OneRNG and I was good to go.  Time to regenerate my ssh hostkeys 😉

Banshee + Rockbox + Playlist Sync

Generally I find opensource pretty easy to use but recently my wife and I acquired Sandisk Clip+ devices along with 32 Gigabyte class 4 SD cards to put some of our music on.

I don’t personally use many playlists as I like to play my music based off Album or Artist and playlists don’t seem like they are worth the hastle to me.  My wife on the other hand lives by her playlists.  This means she really wanted me to find a workflow that allowed her to share playlists between her Banshee player on Linux and her new Clip+.

This was not intuitive…..

Originally I had hoped to use the stock OF (Original Firmware) from Sandisk for her and simply have Banshee write playlist files out for her.  The immediate issues I ran into were (1) playlists had to be in m3u format and HAD to be in the MUSIC directory with everything else.  This wasn’t a show stopper once I figured it out.  What came next was much harsher.  (2) The files had to be in DOS format and had to have ‘\’ separators.  Banshee wrote the files out with ‘/’ separators in Unix format.  This means I wrote the following script:

cd '/media/path/to/sdcard/Music'

sed -i 's/\//\\/g' *.m3u
todos -uo *.m3u

Unfortunately this meant the following workflow was required:

  1. Open Banshee
  2. Copy music to the device
  3. Create playlists on the device
  4. Add music on the device to playlists on the device
  5. Unmount the device (to cause Banshee to write the playlists to the device, which only seemed to happen on unmount).
  6. Re-mount the device,
  7. Run my above script to fix M3U files up.
  8. Unmount the device
  9. Play

Running through this workflow once made me realize it was putting undue hardship on my poor wife.  What I really wanted was a solution where she could:

  • Create playlists on Banshee
  • Sync the playlists and Music automatically to the Device
  • Play the playlists

This is the solution I found that worked and fulfilled the above requirements:

  1. Install Rockbox on the Clip+
  2. Setup a proper .is_audio_player for Rockbox on the SDcard
  3. Create a smart playlist that is the OR of all the playlists she wants to sync
  4. Tell the device’s SDcard to sync Music to that smart playlist
  5. Sync the device
  6. Play the playlists via the file browser in Rockbox

Lets look at each of these steps more closely:

Install Rockbox on the Clip+:

This is pretty well documented at http://www.rockbox.org/manual.shtml. The reason I did this was Rockbox allows for ‘/’ and ‘\’ in its M3U processing so Banshee generated M3U files would play properly.  Rockbox of course brings lots of other goodnesses with it, but its more robust M3U processing is my necessity.

Setup a proper .is_audio_player for Rockbox on the SDcard:

In order for Banshee to recognize the SDcard, I you have to tell Banshee that the SDcard is a music device.  I decided not to put any music onto the  NAND in the actual device itself.  Instead, all music would live on the SDcard.

The file I ended up using looked like:

name="Wifes Sandisk Clip"
audio_folders=MUSIC/
output_formats=audio/mpeg,audio/x-ms-wma,application/ogg
playlist_formats=audio/x-mpegurl,audio/mpeg-url
playlist_path=Playlists/

endoffile=PlaceThisHereToEnsureThereIsABlankSpaceBeforeTheLastLineOfTheFile

Note that this file was formed as the results of unioning many Google search outputs.  Basically the name is the name, and the outputs tell everything where to go and in what format they can go.

Create a smart playlist that is the OR of all the playlists she wants to sync:

This is the part that was tricky.  I found lots of posts saying “Make a smart playlist and everything will work”.  I kept trying to make a smart playlist on the device with a rule of “is in other playlist on computer”, but it never worked.

What I had to do instead was make a smart playlist ON THE COMPUTER that was a union of all the various playlists I wanted on the device.

For example, lets say I wanted a playlist of all my Britney Spears music and another playlist with all my Handel music.  To do this, I would search my music for Britney Spears, send it to a playlist named “Britney”, then search for Handel and send that to a new playlist named “Handel”.  This would result in the following:

playlistsThese steps would obviously be repeated as many times as needed until all the desired playlists have been created.

Now comes some of the tricky parts.  What we are going to do is define a new “Smart Playlist” that selects the files from all the other playlists we care about.  I mistakenly thought that syncing that playlist would only sync the files and create a single playlist on the device, but what Banshee actually does is sync all the files and then copies all the playlists and edits them down to only contain the files you sync’d.  This means all our playlists like Britney, Handel, and others will show up on the device.

So what do we do now, ON YOUR COMPUTER, right click on music and say “New Smart Playlist”.  Call it something like “MusicToSyncToDevice”.  Change the criteria to match “any” of the following and repeatedly press the plus button and set Playlist is <playlist_we_care_about>.  So in this case we will have two rows for Briney and Handel that look like:

smartsync

 

 

 

 

 

 

Now that we have this defined, we need to tell Banshee to sync this stuff to our device.  To do this, we click on the device at the left (note if you just created the .is_audio_device file you may need to unmount and remount the device), and we point to our smart playlist as our music sync source.

Tell the device’s SDcard to sync Music to that smart playlist:

In the menu setup the Music sync source to our smart playlist:

setupsync

Sync the device:

Now right click on the device and select Sync or select Sync from the upper right.  Banshee will sync all the music down to the player.  After that is done, all the playlists will be written out to the Playlists directory for playing on the device.

Play the playlists via the file browser in Rockbox:

All that remains is on Rockbox browse to Files->microSD1->Playlists and select a playlist to begin playing all the songs on that playlist imported from Banshee.

Note I tried to change the Playlist Catalog to point to /<microSD>/Playlists, but wasn’t successful in that endeavor.  Mainly because as soon as I started looking I realize that I wanted to keep Banshee playlists separate from dynamically created playlists I made on the device so this worked out well.

Hope this helps someone,

Barry

NAS Part 2

In December of 2010 I married the love of my life.  She brought forth into my home the woman’s touch.  She also brought forth all her data!  My poor struggling original NAS was already 80% full.  There was no way I was going to RIP all her movies, MASH episodes, and everything else onto my poor old NAS.  About this time I set out in search of new solutions.

In a previous life I was a systems administrator.  I had a very bad day on the job once.  When I arrived at work, one of the drives in our 18 drive RAID5 Sun Solaris raid array had failed.  Being the good administrator I was, I carefully replaced the drive and started a rebuild.  About half-way through the raid rebuild another drive in the RAID failed.  RAID5 protects against 1 drive failure.  The second drive failure was too much for the raid array and it went down never to come back up.

In hind sight, creating a RAID5 array with 18 drives was not a smart move and we shouldn’t have done that.  We had incorrectly convinced ourselves that the enterprise drives we were using were robust and the likely-hood of two drives failing at the same time was low.  What we hadn’t fully considered is how much stress the rebuild would put on the other drives.

Haunted by that experience, I was worried that if one of the drives in my raid5 failed another might fail in the rebuild and I would lose my data.  I had backups of it, so it wouldn’t be a disaster, but it sounds like a pretty major inconvenience.  Add this to the fact I am a nerd and I didn’t want to take the risk.

Armed with this information I wrote a list of requirements for my new NAS:

  • At least 8 TB of storage
  • Ability to have 2 drives fail and no data loss
  • Hot swap capable
  • Still quiet and energy efficient

About this point in time I started reading about the various options available as far as operating systems/software is concerned.  I read about Unraid, ZFS, FreeNAS, OpenMediaVault, and others.  In the past, I had worked a lot with Solaris and the *BSDs.  I have to admit that I was missing playing with that technology.  This pushed me towards OpenSolaris or FreeNAS so that I would use ZFS with a RAIDZ2 array.  Using RAIDZ2 would allow me to lose two drives and still have no data loss.  ZFS supports up to 16 exbibytes of data (giga->tera->peta->exa).  ZFS also added scrubbing capabilities to protect against bit rot.  All in all, it had a TON of features I wanted to play with that my Linux md raid didn’t have.

I placed a order for the following components:

  • 1 x LIAN LI PC_Q25B Black Aluminum Mini-ITX Tower Computer Case
  • 1 x APC BE650G1 650 VA Back-UPS 650
  • 1 x SILVERSTONE 500W ATX Power Supply
  • 2 x 8 GB Crucial Ballistix DDR3 1600 Ram Sticks
  • 1 x ASUS C60M1-I AMD Fusion CPU C-60
  • 6 x Seagate Barracuda 7200 ST3000DM001 3TB 7200 RPM Hard Drives
  • 1 Intel EXPI9301CT 1000Mbps PCI-Express Card
  • 1 x Silverstone PP05 Short Cable Set for Modular PSU

One of the coolest things about the C60M1-I was that it had 6 onboard SATA connectors so I wouldn’t need any sort of IO expansion card.

I assembled the system after all the components arrived.  I can’t say enough good things about the Lian-Li case.  It is a very quality piece of equipment and worth the price.  I have never had such a nice case before and I will seriously be considering them for my next PC build.

I built this server in May of 2013.

IMG_7417 IMG_7416 IMG_7415 IMG_7413

Overall I was very happy with this build except for the errors I made which I will outline below.

Pros:

  • ~ 12 TB of usable storage
  • Ability to lose two drives without loss of data
  • Lian-Li case/backplane is Hot Swap capable
  • Fast
  • Quiet
  • Doesn’t use much electricity
  • Better Airflow = Cooler Drives (Hover around 37 degree C)
  • ZFS Awesomeness

Cons:

  • Lian-Li case can’t hold more than 8 drives
  • No ECC
  • No Encryption
  • Limited Expandability

Honestly the biggest issues with this NAS device stemmed from my lack of understanding of ZFS at the time I built it.  ZFS can do some really bad things if memory gets corrupted. See the ECC vs non-ECC RAM and ZFS discussion on forums.freenas.org for more information.

Another related issue is my processor (AMD C-60) doesn’t support the AES-NI instruction set.  This means I couldn’t encrypt my drives and still have acceptable performance.

The lack of ECC and AES-NI could be fixed by moving to an AMD Kabini GX mini-itx board, but as of January 2014, they still are not readily available.

At the core, all of these issues come about because I chose commodity hardware over server hardware.  My next build will overcome these limitations, but it will come at a cost of more expensive components, more electricity usage, and more noise; but that is a post for another day.

Barry

 

 

 

 

 

NAS Part 1

As with many modern computer users, I have a large collection of digital data ranging from pictures, manuals, backups, game-data, etc.  During college, I had a few external hard drives and I would routinely backup my data to those drives but it was always a pain and it meant I couldn’t access all the data concurrently without plugging all my drives into the various USB ports on my computer.

After graduating and having a “real” job, I decided to build my first NAS.  For those not in-the-know, a NAS is a “Network Attached Storage”.  It is basically a computer with lots of hard drive space that sits on your network and allows other computers on the network access to its storage pools.

My first adventure into the world of NAS was composed of the following items:

  • 1 x APEX TX-381 Black Steel MicroATX case w/300 Watt PS
  • 4 x Hitachi GST 1TB 7200 RPM hard drives
  • 2 GB of Crucial 240-Pin DDR2
  • Intel mini-itx motherboard with 4 SATA ports w/Atom 330 processor

TX381aThe OS for this particular system was Ubuntu LTS.  I setup the drives in Linux
using softward RAID (mdadm) using RAID5.  This gave me a total of 3 TB of storage that I could export to my clients using Samba and NFS. It also meant that if any drive failed my data wouldn’t be lost.

I built this system in May of 2009 for ~$800.  It served me well until I replaced it in April of 2013.

Designing home storage solutions is about compromise, and this server was no different.  Here are some of the pros and cons of this particular build:

Pros:

  • Fairly Cheap
  • Provided 3TB of disk storage
  • Provided a dedicated storage NAS
  • Compact
  • Quiet
  • Low power consumption

Cons:

  • No hot-swap capability
  • Hard drives ran hot (would hit 40 degree C at ambient temp)
  • Linux’s Raid wasn’t as awesome as some of the competition
  • “Only” 3TB of storage started filling up!

These Con’s all lead me to build my next NAS which will be covered in another post.

Barry