WTPA2 Sample Packer Extractor

Mike Dooley, sage of all things object oriented, took my weird hard to use sample packers and extractors and made them into a sensical python script.
This should work on all your machines.  If you have Python.  Who doesn't have python?

This (remember) is used for converting AIFFs to WTPA2 samples and back again.  So to store your live samples or to put samples you've made on your computer onto WTPA2.

Here's how you use it:

python implementation of the WTPA2 sample packer/extractor

Commands take the form of python wtpa2.py COMMAND ARGUMENTS

Currently supported commands:

  • pack - create a WTPA2 readable binary file from a list of AIFF files and directories
  • extract - extract all samples from a WTPA2 formatted binary file or raw device to a target directory. An optional argument can be supplied to limit the number of sample slots examined

Usage

Here are some example uses:

python wtpa2.py pack samples.bin file1.aiff dir1/ dir2/ file2.aiff
python wtpa2.py extract samples.bin samples/
python wtpa2.py extract --slots 8 /dev/sdc samples/

And here's where you find it:

Try packing some samples!  And please, comments or suggestions are welcome.
TB

«1

Comments

  • could this be done to also convert wave?

  • It could be but it probably won't be.  AIFFs and WAVs are actually pretty similar internally.  I'd use a program like sox or audacity to convert your WAV to an AIFF (there won't be any data loss) and then just use the packer as is.
    Thanks!
    TB

  • could you post a little more "python for dummies" info on how this works?

    a little more literal step by step?

    I am a complete python idiot newbie


    I downloaded python 3.4.2 and got it to run but after that I am  totally illiterate.
  • Python is an easy-to-use, cross-platform more-or-less-scripting language.
    It makes if very easy to make tools (like this one) which work on more than one kind of OS.
    Running a python script requires you to have the python interpreter (usually just called Python) installed on your machine.
    Most Linuxes comes with Python.  Not sure about you other dudes, but it's easy to install.

    Python scripts are (often) invoked via the command line.
    You type the word "python" first to call the python interpreter, then the name of the script.
    So if you had a python script to sort your fart noises, you might use it by typing:

    $ python sortFarts.py <enter>

    In examples like that, the dollar sign is the command prompt.

    In the above examples, the "python wtpa2.py" part says "call the python interpreter and have it execute the script named wtpa2.py".
    So here's the "pack" example again:
    python wtpa2.py pack samples.bin file1.aiff dir1/ dir2/ file2.aiff
    In this example, the "pack" command is a word that wtpa2.py is looking for, and specifies that the script should "pack" aiffs into a WTPA2 specific blob file.
    The arguments are "samples.bin" which will be the output file (the blob you put onto WTPA2) and the remaining "files" and "dirs" are the inputs which tell the script what to pack into the output file. 
     
    Make sense so far?
    If you're confused, try it out.  You won't break anything.  If you type something wrong, the script will tell you what it didn't like.

    You can get python here:  https://www.python.org/downloads/
    I generally use python 2, but I suspect this works just fine with python 3.

    @msd might know for sure.

    Best,
    TB



  • Hi Todd!
    the WTPA2 seems to work fine, good. 
    But my big problem since 1 week, is how to put correct samples into sd card.
    I have installed Python2.7.9, and tried a lot of things in all possibilities i was thinking about... , digital translate your comments to be sure i understand correctly your instructions, but nothing work like it should.
    I think (i hope :-)) i'm not the only one.
    The best result i have is this Python message with Python Shell after open wtpa2.py file:

    usage: wtpa2.py [-h] {pack,extract} ...
    wtpa2.py: error: too few arguments
     I have a little experience with coding arduino, compile for pic etc..
    But here, it is a difficult action when you are newbie.
    I think, for all your customers, you should explain step by step, with screen photos,  how we have to convert samples exactly, step by step.(where we put samples, how to pack etc.)
    Your build it manual is very good explained, user manual too, but put correct samples into sd card is the dark side of WTPA2.. :-)

    I hope you will do something, i think i've tried all possibilities to understand but no result.
     Thanks!
    Loic



  • I seem to be having (sort of) the same problem. My wtpa2 seems to be working fine up until I try the sd functions. The card I got was standard density, $2, ebay. nothing seems to happen other than a led turning off for a second and then back on- think its a normal sample card, but I cant get to the sd card menu. I too have tried to figure out using python & the wtpa2 packer/extractor, and I am having trouble understanding something. Maybe a more detailed description of how to do this would help. I have had a ton of fun with the wtpa2 and am looking forward to using more of the functions. Thank you. 
  • edited January 2015
    Some wtpa.py tips that I managed to figure out. I'm on OSX but a lot of this is the same for Linux.

    To get the samples on the card, you either need to put the card device as the output file (I need sudo to write to the device)
    ~/bin/wtpa2.py pack /dev/disk2 ./samples/

    or make a bin file and use dd to get it on your card (also needs sudo)
    ~/bin/wtpa2.py pack samples.bin ./samples/
    sudo dd if=samples.bin of=/dev/disk2

    To find out what that magic device number is so you don't clobber your hard drives run
    OSX:
    diskutil list

    Linux:
    fdisk -l

    That will list all your drives. Look for the one that is the size of your card (2GB) with an unrecognized file system.
    /dev/disk2
    #: TYPE NAME SIZE IDENTIFIER
    0: *2.0 GB disk2
    There might be an easier way....? dd scares the crap out of me, as well as sudo-ing random Python scripts.


    To get the samples in the right format, sox is your dream program.

    sox audiofile.aiff -b 8 -r 22050 -c 1 watpwillbehappy.aiff

    Broken down that is sox (source file) -b (bit rate needs to be 8) -r (sample rate needs to be 22050) -c (needs to be one channel / mono) (output file)


    Take a directory of wav files and turn them into watp aiffs:

    for i in *.wav; do sox $i -b 8 -r 22050 -c 1 $i.aiff ; done


    Hopefully that helps some of you guys. Unfortunately I don't know what the Windows equivalents would be for the first part.
  • Man, where's that nice GUI tool Batsly wrote? I want that. 
  • I've successfully changed a .wav file to a proper .aiff and put the image on a microSD card, but when I play it back on the WTPA2, the audio is truncated.  Here are the command lines:

    sox 24940__vexst__the-winstons-amen-brother-full-solo-4-bars.wav -b 8 -r 22050 -c 1 test.aiff
    python ../wtpa2.py pack test.img test.aiff
    sudo dd if=test.img of=/dev/disk1

    When I play back test.aiff on the computer, I get the whole 7 seconds, but playback on WTPA2, either direct from the card or loaded into RAM, is only the first second of the sample. Am I doing something wrong here?  test.aiff is only 151K, so should be small enough.
  • Interesting. Can you cat the first 1024 bytes of the image and upload it somewhere? And also your test wav?
  • Also. You can try "cat test.img > /dev/whatever" instead of dd
  • The full image, the .aiff, the .wav, and the first 1024 bytes of the image are at http://ben.potswift.org/wtpa2/ --

    Using cat gets the same result as dd.


  • And I should have mentioned that I get the same result with two different microSD cards.
  • I fanched up the sample packer. First I made it auto convert problem files with sox if it is in your path. I also added a packmap command so you give it a txt file with one sample path per line and it will load it to the appropriate slot. Skips blank lines. I need this to map everything to particular keys on my controller.

    Hope this is useful for you guys!

    https://github.com/scragz/wtpa2-python
  • Awesome!  I think it's really cool you're contributing code.
    And.  I love sox.  I use it manually to convert WTPA2 files and about a million other things for embedded audio.

    I think we originally didn't autoconvert files because we didn't want dependencies other than python.  The non-penguin types are already having a hard time accessing raw devices, nawmean?

    Either way, kickass.  Did you get with Dooley about merging those changes in?  As long as you don't get a crash if sox isn't installed, and as long as you can specify weird sample rates (other than 22050) if you want, this seems like a big improvement all around.

    You want a WTPA2 t-shirt?


  • @bens
    Lemme look at those files and get back to you.  Thanks!
  • @ToddBailey I will test it some more, make the sample rate configurable, and get with Dooley. Would love a shirt!
  • I wanna know how to get a shirt - unfortunately I can't program.  Well, maybe a simple program, something like, "Hello World.  What is your name?"  Enter Name.  "Hello entered name, my name is Elisa.  How are you feeling today?"  See what i did there, nerd and age test all in one.
  • @bens --
    Found it!  Here's the story.

    First thing:
    if you have sox installed (speaking of how awesome it is), this is handy:

    play -t raw -r 22050 -b 8 -c 1 -e unsigned-integer ./blob 

    This lets you play the whole sample image and since the data is in the same encoding as an aiff it will tell you if the data got stuck in correctly.
    I made an image using the python script too (and your aiff) and got the same problem, although both blobs played fine with sox.

    Next I looked at the first 1024 bytes in a hex editor.  Each sample on the card starts with a 4-byte length.  The one from our blobs resolved incorrectly.
    Then I tried using the packer I originally wrote in C.

    Check the photo.  The C output is on the right.  Cursors are placed on the first byte of the sample length.
    image

    So you can see the output of the python script only put out 2 bytes, which told WTPA2 to play for a shorter time, even though all the sample bytes were there.  You can see the 32-bit number on the right resolves to the right length.

    So, _somebody_ has to fix the python packer.  I'll get around to it if Dooley or one of you does not.

    In the meantime, you can use the C packer tool on github.

    Good catch!
    TB

  • Wow, Todd, that is excellent sleuthing. Thanks! Let me take a look at the python script and I'll see if I can fix it. I'll use the C packer in the meantime.
  • msd
    edited February 2015
    @bens
    Good find! I guess I didn't play around with sufficiently large samples during my testing.

    @ToddBailey
    Thanks for the deep dive, helped me figure out what was up.

    What was happening, is that when I used two slices to add the WTPA SD card ID to the header, it appended a newline and grew the header's bytearray by two bytes. So when I wrote the header out to disk (the last thing done in the packer) it overwrote the first two bytes of the first sample's length.

    I updated the packer on github to limit the header write to 512 bytes, but I'm all ears for a better solution.





  • @msd -
    Thanks! Nice work. I'll point the link on the website at the packer from github.
  • Thanks, Mike! I grabbed the new python script, the output looks right.  I'm 3,000 miles from my device right now, but will test next week.
  • The new script at https://github.com/mdooley/wtpa2-python works great, thanks!

  • OK so I am getting around to using the packer/extractor. I am on windows 10. I got python 2.7 and I downloaded latest wtpa2.py from mdooley's github 

    I got the script to run a test on 1 file and successfully got samples.bin to my windows desktop. 
    Error codes told me that not 8bit mono, so I got a copy of sox and I am learning how that works 

    but HOW DO YOU GET SAMPLES.BIN TO LOAD ON TO THE SD CARD IF ITS FORMATTED AS WTPA SO WINDOWS CANT SEE IT OTHER THAN AS BAD FORMATTED DRIVE E: 

    A little help on this would be appreciated , do I need to get dd for windows to force it on the card? yuk

  • and fwiw I have a mac runnng 10.6.8 that also has the same issue, how do I get the wtpa2 packer to write samples.bin on the sd card if the Mac says bad formet???
  • @bcommes Use dd. Probably easier on Mac since Windows cmd is fucky. First do `diskutil list` to find what the drive is then `sudo dd if=samples.bin of=/dev/disk2` (replace disk2 with your card).
  • @scragz is right, I think.
    I use "cat" on Linux but dd would work too.  I've never done it on Mac or Windows but from what little I know Mac is probably easier.

    FWIW if you are writing a virgin SD card with a "correct" FAT file format on it already, sometimes the system will automount and you'll have to unmount the card first.
    Once you write samples to it (and trash the FAT table) this won't happen.

    Thanks, @scragz.
Sign In or Register to comment.