Bash Scripting

One of the things I like about Linux is shell scripting which for me is scripting with Bash (Bourne again shell).

Linux systems have much in common with Unix systems including a design philsophy of having discrete tools that do one job and do it well. Bash provides me with a powerful set of tools and also allows me to use a host of other command line applications when there is something I either want to do my own way or I can not find an application that suits the task.

Below you will find some shell scripts written by me. I learned how to script by reading and playing with scripts and perhaps these might be useful to someone else either for dissection or modification and use.

Some of the scripts listed below were used in a Bash Scripting tutorial I gave at the Melbourne Linux Users Group although the versions linked below may be more recent. You can find the complete tutorial here. If you want to learn a little more about the Unix Philosopy you can find an introduction here, and if you are familar with BASH and scripting, you might be interested in reading Steve Bourne's original shell tutorial, An Introduction to the Unix Shell written at Bell Labs in1978.


This page is being revised. The next next four scripts have not been worked into this page yet.


Countbytes:

Iptraf is handy to get a realtime ncurses display of network traffic listed by interface but it also has a logging function. This script takes a log created by iptraf run in a given time period and then parses out the inward TCP traffic, totals it and appends it to a logfile. I had always wondered how I could log downloads on individual workstations but never really found anything simple. When someone on the MLUG list asked if anyone knew of something he could use to log downloads on his networked workstations I decided to have a go at scripting something up using iptraf.

This script uses sleep and will take a time period for logging as minutes or hours as part of the command. If there is no incomming TCP traffic it will say so in the log entry. Otherwise the download total will be given as true kilobytes, i.e. total bytes by 1,000. Bash only supports math with integers so a decimal of kilobytes is achieved by using the modulus of total bytes by 1,000.

You can find countbytes here and below is a snippit from the logfile iptraf_summary created by countbytes.


Begin: Monday-February-22:14;   End: Monday-February-22:15;     Total download on eth0 is: 1155.98 Kb.

Begin: Tuesday-February-05:19;  End: Tuesday-February-05:19;    There was no incoming TCP traffic during this period.

Begin: Tuesday-February-05:20;  End: Tuesday-February-05:40;    Total download on eth0 is: 8064.720 Kb.

Top


Mtnfs:

We have a home network with static IP's that includes a server for printer/scanner shares, file shares and timeshifting DVB-T broadcasts. If this server is not turned on it takes forever for the workstations to boot if the NFS shares are listed in their /etc/fstab but can't be found.

The solution is to wait for the system to boot and then ping the server to see if it is up. If it is up the NFS shares can be mounted using the mount command.

This is a basic script that fulfills a need. While there is nothing fancy about it, mtnfs shows how handy a little scripting can be . Since we use Slackware here, we can run mtnfs at the very end of a boot by listing the command in /etc/rc.d/rc.local. Mounting NFS shares like this might cause a shutdown to hang. Read the comments in the script to see how that can be avoided.

You can find mtnfs here.

Top


Autoshtdn:

The server for our home network is only required if there are work stations up on the lan that need to access it. In order to save some electricity and avoid heating up the room in which it resides unnecessarily, we use the script autoshtdn to shut it down when it is not needed.

Run as a cron job, autoshtdn checks to see if:
  1. there are any workstations up,
  2. /var/spool/atjobs is empty, since our save-dvb script schedules timeshifts as at jobs,
  3. dvbstream is running which indicates that there is a show being saved to disk, and
  4. create-dvd, our encoding script is running.

Autoshtdn uses " 2 " to indicate up or running and " 1 " to indicate down or not running. As you can see from the log snippit below the first and third entries have one workstation up so the server is not shut down. However, in the second entry, the number one was returned for all 5 tests so the server was shutdown.


Mon Feb 23 21:47:11 EST 2009 ran autoshutdn script Mon Feb 23 21:47:11 EST 2009
rick=2, leila=1, atjobs=1, create-dvd=1, dvbstream=1, no shutdown now.


Mon Feb 23 22:47:14 EST 2009 ran autoshutdn script Mon Feb 23 22:47:14 EST 2009
rick=1, leila=1, atjobs=1, create-dvd=1, dvbstream=1, shutting down now.


Tue Feb 24 05:47:05 EST 2009 ran autoshutdn script Tue Feb 24 05:47:05 EST 2009
rick=2, leila=1, atjobs=1, create-dvd=1, dvbstream=1, no shutdown now.

You can find autoshtdn here.

Top


Chkserver:

A user in another room has no idea whether or not the server is up and more often then not if they are the first one out of bed or home in the evening they just turn on their machine and later will wonder why they cannot access shared directories or print.

Chkserver is run from ~/.kde/Autostart and checks to see if the server is up with ping.

If there are no return packets chkserver assumes that the server is down and uses kdialog to display an error message on the user's desktop.

Click the thumbnail above to see a screenshot of the chkserver popup window and click here to see the script

Top


Save-dvb:

We use this script to schedule and record, i.e. timeshift, free to air DVB-T broadcasts. It requires dvbstream and a channels.conf for our locality (Melbourne, Australia). Everything else is bash and commands standard on just about any Linux distro.

I originally wrote this script so that we could timeshift using a DVB-T card on our server that usually sits in the corner without too much to do. We could ssh into the server and run save-dvb from our workstations. Shows are recorded to disk and we can access them via NFS fileshares in real time as they are recording or view them some other time. Later on when prices came down I purchased a second DVB-T card for my workstation where this script is also used for timeshifting. Scheduling is faster CLI than opening up and point/clicking my way through a GUI interface.

Save-dvb will parse a standard channels.conf file and display only those stations which have not been commented out. Uncommented stations are offered for selection after which the user is prompted for running time, a name for the recording, etc. The four thumbnails below will give you screenshots in sequence left to right of save-dvb in use.

This script parses the necessary variables from a channels.conf then combined with other variables based on user input, it writes a temporary script called .sched-show which is run either "now" or as an at job. The temporary script is overwritten the next time save-dvb is run.

The .sched.show script for the above screenshots is here. You can find save-dvb here and my channels.conf (Melbourne, Mt. Dandenong) here. If you are interested in how to use command line tools for DVB you may find this page I put together a while back of interest although it is getting a bit long in the tooth.

Top


Get-weather:

Get-weather is a script we all use around the house rather than hope and wait for a weather forecast on the radio. This script downloads our current weather forecast from the Australian B.O.M and the removes unnecessary text, does some reformatting and adds a few lines to make the resulting output to a terminal easier to read. The information comes from a text file that feeds the publicly accessable http://www.bom.gov.au/products/IDV17102.shtml. I want something a little more concise and either available in a terminal or one click away with an icon.

I had previously made up a script called get-forecast that I could run in a terminal or be used to provide content for a kdialog popup several years ago. However, several months ago B.O.M screwed everything up with some website redesign. Luckily for me the text based web page I was using, www.bom.gov.au/cgi-bin/wrap_fwo.pl?IDV17102.txt, still existed even if it was no longer the default page for the Central District forecast. The layout and content had changed and I had to figure out a new way to parse the content as well do a little more to make the output presentable.

Get-weather turned out to be a bit simpler than my previous get-forcast being a one liner made up of piped sed commands that either prints to stdout or is redirected to a temporary text file which is then displayed in a kdialog popup. Get-forecast had been used in a bash script to generate a forecast page on my web site. The get-weather one liner has now been worked into my Mornington-weather.cgi to feed the latest forecast into the web page.

The thumbnails below are links to screenshots of local weather displayed in a terminal, in a popup and on a web page left to right. There are links to the scripts directly below each thumbnail. If you want to see what the forecast is right now go here: http://www.turtlespond.net/Mornington-weather.cgi

If you have had a look at this or several of the other scripts above you will have noticed that I often use stream editors and pipes to work text.files and the output of other commands into something useful. There is usually more than one way to edit things on the fly and I don't claim to be the most elegant but if you are not familiar with stream editing you might find the explanantion of how get-forecast works helpful if limited introduction.

Top


Create-dvd:

Create-dvd was written solely for the purpose of shrinking, cropping, encoding as an mpeg-ps (.vob) the free to air digital TV programs that we have recorded and then burning them to a dvd. Any editing out of commercials or cropping the video before or after the desired program is done with Gopdit, my prefered editor prior to processing with create-dvd.

This script is used both on my workstation or on our server. The server is my preference as I can pop a blank dvd in the drive, ssh in, start the script, turn my machine off and then walk away and go to bed, go to work, whatever. When the script is done and the dvd is burned, the server will just shut itself doen unless there is a recording scheduled.

Create-dvd offer four different options:
  1. Burn a dvd without encoding, if the video file is not properly encoded the nothing will be burned.
  2. Encode to dvd quality with with a 16/9 aspect ratio then burn to a dvd.
  3. Crop, then encode to dvd quality with a 4/3 asopect ration then burn to a dvd, this is used on older movies.
  4. Encode as above with a 16/9 aspect ratio, shrink to a requant factor provided by user input then burn to a dvd.
After several years I have found these are the only options I require. There has never been an old movie with a 4/3 aspect ratio that required shrinking, and I've never had the need or desire to crop black borders out of video with a 16/9 aspect ratio.

This script requires empty space. I have written it so that when finished it will have left a directory containing the original video, the encoded video and the iso. temporary working directory will require additional room while shrinking/encoding. I usually leave the line that deletes this directory at the close of encoding commented but thats not neccesary.

Theres not much in the way of screenshots for this script other than the selection menu and the user being asked for a requant value:

Top


Fun with stream editors:

You might have noticed that I often use stream editors and other CLI aps that sort, sift and edit text in my scripts. While intimidating at first and I lay no claim to mastering regular expressions, I find using these applications fun since using them involves problem solving, i.e. figuring out how I can get what I want using which application(s).

Case in point is my script save-dvb above where I use a variety of small aps in various combinations to select digital TV stations and then create the variables required to lock onto a dvb-t signal.

Below are several simple scripts I have written to do something trivial but useful (to me) which may be of interest to someone wanting to dissect them to learn little more about stream editing.

Top