Whether you are receiving natural radio or amateur radio, it is most useful to record the received signal continuously so that you don't miss anything interesting. Amateur radio transmissions are not always announced in advance, and natural radio events are almost never announced in advance!
The idea is, you record continuously to a disk reserved especially for the job. The recording is stored in chunks, each in its own file, of say an hour long. The chunks follow each other without a gap. As the disk fills, the oldest chunks are deleted to make way for new ones. You effectively have a long recording loop which can go back several days or even weeks or months, depending on how many channels you are recording, the sample rate, and the size of the disk.
For example, suppose you want to record one channel at the full 96k/sec sample rate of the audioinjector. Using 32 bit floating point format (f4 in vlfrx-tools terminology), each hour of recording takes up 1320 Mbytes, a day takes 31.7 Gbytes. That seems like a lot but you can fit three months onto a 3000 Gbyte disk drive, and they don't cost very much. You can manage with much less, just keeping a few days of signal is long enough for you to hear about interesting activity.
In practice, you would prefer to only record the part of the spectrum that you are interested in. For example for natural radio you might record at only 32k samples/second to store a baseband of 16kHz width. For amateur radio, you could record just the 8270 Hz band, at say 10 samples/second. You can record as many bands as you want, each to its own directory on the data disk.
You'll need to be able to estimate how much space your recording will need. The easiest way to do that is to measure it. Use the vtgen program to generate a test signal in f4 format with your desired sample rate and number of channels. For example,
vtgen -r 96000 -c 1 -T now,+1m -- -,f4 | wc -c
generates one minute of single channel 96k/sec stream in f4 format and pipes this into the unix 'wc' program which reports the number of bytes. Run the byte count through awk with a command to print formatted the size of a day's recording in GBytes:
vtgen -r 96000 -c 1 -T now,+1m -- -,f4 | wc -c | awk '{ printf( "%.3f Gbytes/day\n", $1 * 60 * 24 / (1024 * 1024 * 1024))}'
This section describes how to set up the Pi for continuous recording of the VLF band up to 16kHz, at a sample rate of 32k samples per second. I'll assume that you have hum-filtered GPS timestamped signal passing through the buffer @filtered. It may be a single channel or five channels, depending on what type of audioinjector card you're using. For this exercise I'll just assume you want to record only the first channel, in other words @filtered:1 and you can substitute your own choice of which channels to record.
Start by creating an empty directory on your mounted data disk.
mkdir /d/vlf
Now the following command will convert the signal in @filtered:1 to f4 format and write it to /d/vlf in one hour chunks:
vtresample -r 32000 @filtered:1 -- -,f4 | vtwrite -G 3600 /d/vlf
You should now find a data file growing under /d/vlf. From another shell run
ls -l /d/vlf
and if you repeat the command, you'll see the file size is steadily increasing. The -G 3600 option tells vtwrite to start a new file every hour (3600 seconds) and files will start on the hour. If you used -G600, files would start every 10 minutes, at XX:00, XX:10, XX:20, and so on. It's up to you to choose a suitable chunk size. Hour chunks are convenient if you're keeping just a few days. If the disk is big enough for a month or more, you might prefer daily files with -G 86400.
vtwrite will also start a new file if it gets an interruption to the stream, perhaps caused by loss of GPS PPS pulses. A new file will also be started each time vtwrite is restarted.
Now edit /root/radio to add the recording command, with the usual -B options to put the programs into background. The radio script now looks like
#!/bin/bash
killall -q vtcard
killall -q vttime
killall -q vtfilter
killall -q vtresample
killall -q vtvorbis
killall -q vtwrite
if [ "$1" = start -o "$1" = restart ]
then
sleep 2 # Delay needed when using Octo, the driver is slow to initialise
vtcard -B -d hw:0,0 -r96000 -b32 -c2 -g-1 -L /run/vtcard.log -v @raw
vtwait -t @raw
vttime -B -m pulse+ -c2 -h 30 -L /tmp/vttime.log -v @raw @timed
vtwait -t @timed
vtfilter -B -a th=5 @timed:1 @filtered
vtwait -t @filtered
vtresample -B -r32000 @filtered |
vtvorbis -B -ep -q0.4 -ktu shout,localhost,8000,/vlf,XXXXX
if [ -d /d/vlf ]
then
vtresample -B -r32000 @filtered:1 -- -,f4 | vtwrite -B -G 3600 /d/vlf
fi
fi
The additions are highlighted in blue. The recording command is contained in an 'if' statement which checks that the directory /d/vlf exists before starting the recording. That way, it wont try to start if for some reason the data disk failed to mount. Save the changes and restart the radio system with
systemctl restart radio.service
and after a minute or two, signal will appear in @filtered and vtwrite will begin recording a new file.
So far so good. The Pi is now recording your selected channels continuously. It we leave it at that, the data disk will eventually fill up and the recording will stop. We must now arrange something to periodically remove the oldest files from /d/vlf. Use your text editor to create a new file /root/cleardown containing the following:
#!/bin/bash
find /d/vlf -type f -mtime +2 -exec rm {} \;
We are instructing the 'find' program to find all ordinary files (-type f) under the /d/vlf directory, which have a modification date more than 2 days old (-mtime +2) and execute on each of them the rm command to remove them.
Make this script executable in the usual way:
chmod +x /root/cleardown
You can run it with
/root/cleardown
but it will do nothing for a couple of days until you have files that are that old. If you're impatient to test it, you can change the find command to use -mtime +1 to get a cleardown after one day, or specify a time in minutes with, say, -mmin +300 to get a cleardown after 5 hours.
Now to make the cleardown script run automatically, say, every hour. This is very easy. We just need to put the cleardown script into the directory /etc/cron.hourly and the system will run it once per hour. Rather than copy /root/cleardown into /etc/cron.hourly, we'll use a symbolic link:
ln -s /root/cleardown /etc/cron.hourly
That way, your cleardown script continues to reside in your home directory /root along with your other scripts, but it also appears in the /etc/cron.hourly directory so that the 'cron' program knows to run it. Symbolic links are very handy for that sort of thing.
From time to time you can run
ls -l /d/vlf
and you'll see that when files reach your chosen cleardown age, they will disappear within an hour. You'll be using the /d filesystem for other things too, so from time to time you might decide to revise the cleardown age of /d/vlf in order to make room for other data on the disk. You just need to edit the find command in the cleardown script.
Now you have continuous recording. It will run for years so long as you keep power applied to the Pi. It will probably stop working when the flash memory stick is worn out when you've exhausted its write cycle capability. That might be a few thousand cycles, that's a few thousand times the cleardown period, so it could be a long time!
If you're new to Linux, you'll have to get used to the idea that when you start something running, it will run until either it finishes or you tell it to stop. You never have to reboot the system to 'unblock' it, and programs don't just 'crash'. On professional systems it's not usual to find a program that has been running for many many years - so long that everyone has forgotten what it does!
Check out the vt programs in the process table now, with
vtps
and you'll get something like
PID RT %CPU %MEM COMMAND 29820 99 5.7 2.0 vtcard -B -d hw:0,0 -r96000 -b32 -c2 -g-1 -L /run/vtcard.log 30082 - 15.7 4.8 vttime -B -m pulse+ -c2 -h 30 -L /tmp/vttime.log 30152 - 17.6 2.6 vtfilter -B -a th=5 @timed:1 @filtered 30161 - 13.2 0.7 vtresample -B -r32000 @filtered:1 30163 - 5.4 0.4 vtvorbis -B -ep -q0.4 -ktu shout,localhost,8000,/vlf,XXXXX 30167 - 13.1 0.7 vtresample -B -r32000 @filtered:1 -- -,f4 30169 - 0.1 0.1 vtwrite -B -G 3600 /d/vlf
In total that's about 70% of a CPU used, less than a quarter of the quad core Pi's capacity, so there's still plenty of room to load more work onto it.
In later sections of these notes, you'll discover how easy it is to retrieve signals from this recording. But for now you can run a quick demo once you've collected a few minutes of signal. Run something like
vtread -T 2018-02-14_21:30,+5m /d/vlf | vtstat -i
This command extracts 5 minutes of signal starting at the given time. You'll have to pick a time that's within your recording. You'll get an output from vtstat that looks like
format: float8 channels: 1 samples: 9600000 clips: 0 breaks: 0 sample rate: 32000, correction 1.00000000 duration: 300.0000000 seconds start: 2018-02-14_21:30:00.000000 end: 2018-02-14_21:35:00.000000, interval 300.0000000 seconds mean,rms,peak: 1 1.559e-05,1.946e-05,1.024e-04
A vtread command will be the starting point of most post-processing pipelines. The Pi doesn't have enough CPU capacity to do any series post processing. It can do it, but slowly! Most of the time you'll retrieve signal data from the Pi and process it on your workstation.
For example, from your workstation shell:
ssh -n root@rp5 vtread -T 2018-02-14_21:30,+5m /d/vlf | vtstat -i
The vtread program runs on the Pi, but vtstat or whatever else follows in the pipeline, runs on your workstation.