passing bash variables to other programs

[santner]

In this code I am reading a text file:

 

'Mall Santa'

'Christmas Eve'

Christmas_Morning

blank

blank

blank

DONE

 

The goal is to pass anything except 'blank' and 'DONE' to another program. So when I read in 'Mall Santa', I want to literally pass exactly that exact phrase including the single quotes to the external program. Here is the code:

 

titlefile="list.txt"
if [ "$titlefile" != "$emptystring" ]; then
count=0
j=0
old_IFS=$IFS
IFS=$'\n'
lines=($(cat $titlefile)) 
IFS=$old_IFS

while [ "${lines[count]}" != "DONE" ]
  do
  if [ "${lines[count]}" != "blank" ]; then
      menu_options=$menu_options" -f "$outdir"/"${vob[count]}".vob -t "${lines[count]}
      count=$(($count+1))
  else
      count=$(($count+1))
      remove_title[j]=$count
      j=$(($j+1))
  fi
done
fi

echo $menu_options
dvd-menu $menu_options

 

Now the problem is that the: echo $menu_options prints it correctly. i.e.

-t 'Mall Santa' -t 'Christmas Eve' -t Christmas_Morning

 

However, dvd-menu reads in 'Mall

 

Any idea what I am doing wrong?

[Steve Scrimpshire]

I'm no bash guru like aru, but maybe try:

 

dvd-menu "$menu_options"

 

or

 

dvd-menu ${menu_options//\'/\"}

(the above snippet stolen from aru, but may not be appropriate in this situation, though I think it is.)

Edited January 11, 2005 by Steve Scrimpshire

[aru]

This one is not trivial, you need to realize that you are passing a string to the dvd_menu program so in order to work correctly it first needs to be processed by the shell and then passed to dvd_menu.

 

To show you what is happening and how to solve I'll post three different examples of the same. The frist just shows what is already happening, the second illustrates how to do it, and the latter shows how to *even* pass parameters with quotes within.

 

First here is a 'dvd_menu' emulator:

#! /bin/bash
# it shows you values as are passed to the 'dvd_menu' program 
for i in $@; do echo $i; done

 

so as you are passing parameters in your script you are getting:

aru@mandrakeusers ~/bin$ menu_options="-t 'Mall Santa' -t 'Christmas Eve' -t Christmas_Morning"
aru@mandrakeusers ~/bin$ ./dvd_menu ${menu_options}
-t
'Mall
Santa'
-t
'Christmas
Eve'
-t
Christmas_Morning
aru@mandrakeusers ~/bin$

 

Which is certainly what you don't want, quotes are there, but aren't being evaluated by the shell before running dvd_menu, so we need to evaluate them before calling dvd_menu:

aru@mandrakeusers ~/bin$ menu_options="-t 'Mall Santa' -t 'Christmas Eve' -t Christmas_Morning"
aru@mandrakeusers ~/bin$ eval ./dvd_menu ${menu_options}
-t
Mall Santa
-t
Christmas Eve
-t
Christmas_Morning
aru@mandrakeusers ~/bin$

 

We are done, use eval to call dvd_menu; now arguments are passed as they should, probably this will be enough for *real* dvd_menu. So test it and check that it does what it must do.

 

Just in case you get problems with just using "eval", you may then use escaped quotes and later evaluate the expression to get the quotes passed to dvd_menu, as in:

aru@mandrakeusers ~/bin$ menu_options="-t '\"Mall Santa\"' -t '\"Christmas Eve\"' -t Christmas_Morning"
aru@mandrakeusers ~/bin$ eval ./dvd_menu "${menu_options}"
-t
"Mall Santa"
-t
"Christmas Eve"
-t
Christmas_Morning
aru@mandrakeusers ~/bin$

 

HTH

 

PS: side note to Steve:

I'm no bash guru like aru, but maybe try:

dvd-menu "$menu_options"

that way you are passing just one parameter to dvd-menu, as in:

aru@mandrakeusers ~/bin$ ./dvd_menu "${menu_options}"
-t 'Mall Santa' -t 'Christmas Eve' -t Christmas_Morning
aru@mandrakeusers ~/bin$

It will depend on how dvd-menu is written to see if it evaluates rightly the command line arg string, but certainly the shell wont do. ;)

 

 

PSS: I can see several ways to improve your script to make it nicer, just ask in case you want a different approach :D

 

EDITED: added note to Stive Scrimpshire

Edited January 11, 2005 by aru

[santner]

aru, first off thank you very much. eval worked like a champ. Secondly, I never turn down an opportunity to improve my software.

 

Let me give you a brief background on what I am trying to accomplish. The script I am writing will automatically capture video from a dv camcorder, transcode it to mpeg2, make a menu for the dvd, author it and optionally burn it to disc as well. This started as a project written in C, however it just makes more sense to use a shell script. If you want more background, the project is listed under the workbench forum and is on sourceforge.net as dvdhomevid.

 

Let me make a couple of adjustments to the code and I will repost it here tomorrow for your review.

 

Thanks again!!!

[Steve Scrimpshire]

I think I need a whole lesson in 'eval'. lol

[santner]

Here is the code. Like I said before, I am open to any feedback. I would like this software to be as robust as possible because I am looking to release it on sourceforge within a week or so. Thanks. :D :P

dvd_homevideo.txt

Edited January 12, 2005 by santner

[aru]

Thanks I will review your code. It will be a pleasure :)

 

I've done first and quick sight (I'm going to work right now), but I want to give you two advices, first one is with functions and code maintenance: code in bash as you would have done in C (or in another language), use functions as islands don't use global variables unless strictly needed, hence pass arguments to the functions as you would have done in any other language, and use local variables. That will simplify mainteinance and refactoring in the future.

 

The second one is more terrenal, there is a bash builting to handle command line arguments called getopts (see my example), but if you need enhanced CL args features, you can also use the program /usr/bin/getopt. So this is how I would have coded that piece of code:

# This equates to main() in C...C++
####################################################################
# read in command-line arguments
while getopts ":Bho:m:s:b:a:f:q:" option; do
   case ${option} in
       o) outdir=${OPTARG};; # dvd directory
       m) min=${OPTARG};; # dv capture minutes portion
       s) sec=${OPTARG};; # dv capture seconds portion 
       b) bgimage=${OPTARG};; # background jpeg
       a) audiofile=${OPTARG};; # ...
       f) titlefile=${OPTARG};; # ...
       q) quality=${OPTARG};; # ...
       B) burn=1;; 
       h|*) help && exit 0;;
   esac    
done    

 

also I've seen in different places of your code, some checks such as:

conv_check=$(grep -i error log/transcode.log | wc | awk '{print $1}')
   if [ $conv_check -ne 0 ]; then
   <do something>
fi

That is like killing flies with tanks, grep itself returns error 1 when it doesn't find a string,so this code will be better:

if grep -iq error log/transcode.log; then
  <do something> # error found
fi

 

 

Later on (when I come back from work) I'll do a deeper check of your script. Please keep attaching new versions as you made them.

[santner]

First of all, thanks for the advice and recommendations. :D

 

hence pass arguments to the functions as you would have done in any other language, and use local variables.

 

Are you talking about the vob and remove_title arrays, or are you also talking about the command line arguments that are passed to the program. Could you illustrate with an example.

 

Most of the code works as it should. My final task is to implement the '-f' option passed to the program which is where a user supplies a text file titles used in the DVD menu. This has been difficult so far. My main issues arise in the author function where I have to delete lines from vmgm.xml. This is where sed comes in.

 

Here is an example vmgm.xml file (the file I want to edit)

 

<dvdauthor dest="DVD/" jumppad="0">
       <vmgm>
          <menus>
              <pgc entry="title" >
                   <vob file="./menu.vob" pause="inf"/>
            	 <button> jump title 1; </button>
            	 <button> jump title 2; </button>
            	 <button> jump title 3; </button>
            	 <button> jump title 4; </button>
            	 <button> jump title 5; </button>
            	 <button> jump title 6; </button>
            	 <button> jump title 7; </button>
            	 <button> jump title 8; </button>
            	 <button> jump title 9; </button>
            	 <button> jump title 10; </button>
            	 <button> jump title 11; </button>
            	 <button> jump title 12; </button>
            	 <button> jump title 13; </button>
            	 <button> jump title 14; </button>
                   <post> jump vmgm menu 1; </post>
               </pgc>
          </menus>
       </vmgm>
       <titleset>
<titles>
 <pgc>
 <vob file="avifile-001.vob"  />
 <post> jump title 2; </post>
 </pgc>
 <pgc>
 <vob file="avifile-003.vob"  />
 <post> jump title 3; </post>
 </pgc>
 <pgc>
 <vob file="avifile-005.vob"  />
 <post> jump title 4; </post>
 </pgc>
 <pgc>
 <vob file="avifile-004.vob"  />
 <post> jump title 5; </post>
 </pgc>
 <pgc>
 <vob file="avifile-005.vob"  />
 <post> jump title 6; </post>
 </pgc>
 <pgc>
 <vob file="avifile-006.vob"  />
 <post> jump title 7; </post>
 </pgc>
 <pgc>
 <vob file="avifile-007.vob"  />
 <post> jump title 8; </post>
 </pgc>
 <pgc>
 <vob file="avifile-008.vob"  />
 <post> jump title 9; </post>
 </pgc>
 <pgc>
 <vob file="avifile-009.vob"  />
 <post> jump title 10; </post>
 </pgc>
 <pgc>
 <vob file="avifile-010.vob"  />
 <post> jump title 11; </post>
 </pgc>
 <pgc>
 <vob file="avifile-011.vob"  />
 <post> jump title 12; </post>
 </pgc>
 <pgc>
 <vob file="avifile-012.vob"  />
 <post> jump title 13; </post>
 </pgc>
 <pgc>
 <vob file="avifile-013.vob"  />
 <post> jump title 14; </post>
 </pgc>
 <pgc>
 <vob file="avifile-014.vob"  />
 <post> call vmgm menu 1; </post>
 </pgc>
</titles>
</titleset>
</dvdauthor>

 

And here is an example text file passed to the program:

 

'Mall Santa'
blank
blank
blank
'Christmas Eve'
blank
blank
blank
blank
blank
Christmas_Morning
blank
blank
blank
DONE

 

So this is how it works: lines 2,3,4,6,7,8,9,10,12,13 and 14 are all listed as 'blank'. This signals to my program to remove "<button> jump title 2", "<button> jump title 3", "<button> jump title 4", "<button> jump title 6".... from vmgm.xml.

 

However, my sed command does not seem to work correctly.

 

   $(sed -e '/<button> jump title '${remove_title[$count]}'/d' vmgm.xml > vmgm.xml.new)

 

Any ideas why it would not work? :unsure:

 

I have attached the latest and greatest with updates you have suggested. :D

dvd_homevideo.txt

Edited January 12, 2005 by santner

[Steve Scrimpshire]

'${remove_title[$count]}' <- When you single quote, the special meaning of $ is lost. Also, sed is only seeing:

 

sed -e '/<button> jump title '

 

and probably fails with an error.

 

$(sed -e "/<button> jump title ${remove_title[$count]}/d" vmgm.xml > vmgm.xml.new)

 

Should work. I'm sure aru will correct me if I'm wrong :P

[aru]

First of all, thanks for the advice and recommendations.  :D

 

hence pass arguments to the functions as you would have done in any other language, and use local variables.

 

Are you talking about the vob and remove_title arrays, or are you also talking about the command line arguments that are passed to the program. Could you illustrate with an example

As an example let's test your grab function to see if the variables are local or global. Here is a bizare test that strips the variables used in the grab function (using your full script as source) and check if the same variables are referenced elsewhere in the script:

 

aru@mandrakeusers ~/__dvd_project/misc$ VARS=$(export r='\$\<[[:alnum:]]\+\>'; sed -n "/^grab ()/,/^}$/ {h;s/.*\($r\).*/\1/gp; G; s/.*\(${r}\).*${r}.*/\1/gp}" dvd_homevideo.v1.sh | sort -u )

aru@mandrakeusers ~/__dvd_project/misc$ for var in $VARS; do  
> if sed  "/^grab ()/,/^}$/ {d}" dvd_homevideo.v1.sh | grep -q "\<${var//\$/}\>" 
> then echo "$var -- is reference and/or assigned outside grab function. -- CORRECT ME."
> else echo "$var --> is only used inside grab function, probably is local -- OK"; fi; done

$diff --> is only used inside grab function, probably is local -- OK
$end --> is only used inside grab function, probably is local -- OK
$min -- is reference and/or assigned outside grab function. -- CORRECT ME.
$outdir -- is reference and/or assigned outside grab function. -- CORRECT ME.
$pid --> is only used inside grab function, probably is local -- OK
$sec -- is reference and/or assigned outside grab function. -- CORRECT ME.
$start --> is only used inside grab function, probably is local -- OK
$total --> is only used inside grab function, probably is local -- OK

 

As per the above bizarre test, you are using three global variables inside grab() $min, $sec and $outdir. While correct (since grab() does the job) this could lead to mainteinance problems and is makes code reading a pain. Think in what you'll say three months ahead when you want to improbe the script.

 

What I suggested is to use the functions as hermetic as possible, and self documented, such as in:

 

grab () {
   # Description: Capture audio/video from dv camcorder
   #
   # Usage: -> grab output_directory record_time
   # Inputs:  $1 == output directory.
   #          $2 == time in seconds to perform the recording...            
   # Outputs: None
   # Returns: 0 if right, 
   #          1 if ...

   local pid diff start end total # whatever local variables 
   local OUTPDIR="$1" REC_TIME="$2"
   
   <your code>
   
   return 0;
}

 

and call that function as:

grab "$outdir" $(($min*60+$sec))

 

Is just a matter of taste, but hope you follow this advice. It will make your life easier.

 

...about the rest, tomorow will be another day. my wife needs me ;)

Edited January 12, 2005 by aru

[santner]

I've revised my functions to pass arguments - except I don't know how to pass an array or how to return an array. This seems pretty tricky...

 

This version doesn't have many changes, just the functions now receive arguments.

 

Should I return 1 from my functions if my "grep -iq error log/file.log" call fails and then exit from the program, or should I keep it the way it is now and exit from within the function call?

 

Also, I use emptystring="" extensively in most functions, should I pass this as a variable also or keep it as a global?

dvd_homevideo_revision2.txt

[Steve Scrimpshire]

Here's something about how to pass and return arrays:

http://linuxreviews.org/beginner/abs-guide/en/x14648.html

 

(It's like halfway down the page.)

[aru]

Here's something about how to pass and return arrays:

http://linuxreviews.org/beginner/abs-guide/en/x14648.html

 

(It's like halfway down the page.)

<{POST_SNAPBACK}>

yep, that's the way.

 

I've revised my functions to pass arguments - except I don't know how to pass an array or how to return an array. This seems pretty tricky...

you don't need to do that. The array is only used within the function. In your case I would have done the follwoing (which I don't mean that is a better way, it is just the way I would have done it). The fuction "menu()" needs it's own arguments, TITLE_FILE as an optional argument, as is now. But just return the "$menu_options" as string (a plain echo "$menu_options" does the trick. Then in the main script I would have called the function in this way (it's an example, you'll need to fit to the script):

 eval dvd-menu "$(menu <menu function args>)" &> $OUTPDIR/log/menu.log

 

Should I return 1 from my functions if my "grep -iq error log/file.log" call fails and then exit from the program, or should I keep it the way it is now and exit from within the function call?

to do things right you shouldn't kill the program from inside a function, just use the return builtin to return a value different from 0, then in the main script handle the return value. An example:

foobar () {
   if <test>; then
      <code>
    else 
      echo >&2 "ERROR WARNING STRING" && return 1
    fi
    retrun 0
}

# the short way (script exits fi foobar doesn't return 0):
foobar <args> || exit 1

# the same but the long way (more readable):
if ! foobar <args>; then
    exit 1
fi

# or just another way:
foobar <args>
if [ $? -ne 0 ]; then
   exit 1
fi

 

Also, I use emptystring="" extensively in most functions, should I pass this as a variable also or keep it as a global?

Keep as global what is global (such kind of variable is a good example ($outdir is another, keep reading). But in this particular case you may not need that variable, as "" in if tests is clearly an empty string.

 

- About $outdir: since you seem to use it broadly, as in:

# Lets call some functions!
grab "$outdir" $(($min*60+$sec+5))
conv "$outdir" "$quality"
menu "$outdir" "$bgimage" "$audiofile" "$titlefile"
author "$outdir"

 

you may use it globaly inside your functions, but for keeping readablilty you may comment inside that is a global variable or better as:

grab ()
{
   # Description: Capture audio/video from dv camcorder
   #
   # Usage: -> grab record_time
   # Inputs: $1 == time in seconds to perform the recording
   # Outputs: None
   # Returns: 0 if right
   #          1 if...
   local pid diff start end
   local OUTPDIR="$outdir" # from global name space
   local REC_TIME="$1"
   ....
}

# then call the fuction with just one argument:
grab $(($min*60+$sec+5))

 

HTH

Edited January 13, 2005 by aru

[aru]

Back to yesterday's question, could you explain better what you want to do. ie what is the initial scenery (which files do you have initially, their sources), and which is the result you are looking for (and why --for what is needed--). Seems that I don't fully understand your question (because I don't do dvd recording). Is the question about the author() function. Btw Steve Scrimpshire is right in his post (if that's what you wanted to do, that way you remove the undesired lines)

 

thanks.

 

Another advice. Error outputs should go to stderr (I know you log them in different files, but there are some that are still being output to stdout).

 

 

Once we get fixed that kind of API between functions and main script we can go and check for robustness the script and each of the functions (if you wish).

Edited January 13, 2005 by aru

[santner]

First of all, your suggestions have been extremely helpful.

 

Starting with the question from yesterday, that has actually solved itself. Steve's suggestion worked:

 

'${remove_title[$count]}' <- When you single quote, the special meaning of $ is lost. Also, sed is only seeing:

 

sed -e '/<button> jump title '

 

and probably fails with an error.

 

$(sed -e "/<button> jump title ${remove_title[$count]}/d" vmgm.xml > vmgm.xml.new)

The purpose was this: the vmgm.xml file is passed to dvdauthor and it basically tells dvdauthor when you press a button on the menu what vob (video) file to play. Well, depending on how the user sets up their "title" file, instead of button one mapping to vob file one, button two to vob file two etc. you can have button two go to vob file five, and button three to vob file eight etc.

 

I have fixed the "emptystring" issue and have also changed $outdir back to a global variable that is used locally in my functions. I have also 'fixed' it so that I return a 1 from my function upon an error, and then exit from the program in the 'main' code.

 

Now the only thing I didn't understand was this:

 

But just return the "$menu_options" as string (a plain echo "$menu_options" does the trick. Then in the main script I would have called the function in this way (it's an example, you'll need to fit to the script):

 

eval dvd-menu "$(menu <menu function args>)" &> $OUTPDIR/log/menu.log

What I don't understand is why is that method beneficial as opposed to calling dvd-menu in the menu function. I tested it and it does work, I was just wondering why is one better than the other.

 

Another advice. Error outputs should go to stderr (I know you log them in different files, but there are some that are still being output to stdout).

Which errors are you getting that are still being printed to stdout?

 

Once we get fixed that kind of API between functions and main script we can go and check for robustness the script and each of the functions (if you wish).

Definitely!

 

you don't need to do that. The array is only used within the function.

I use vob[] in both conv() and menu() and I use remove_title[] in both menu() and author()....is this ok?

 

I have attached the latest version of the code.

dvd_homevideo_revision3.txt

Edited January 13, 2005 by santner