Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scripting question: how to use quotation marks inside a script
#1
Hello!

I'm working on my custom backup script with rsync, and I have stumbled upon something that is not obvious for me (I'm quite new to bash, so forgive me if I'm wrong here).

Before doing the sync, I define the options and other things in variables:

Code:
OPTS="-aAXi"
EXCLUDE='--exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/home/*"}'
SRC="/"
BCKP="/media/data/backup"
LINK="--link-dest=$BCKP/$LAST/"
When it is done, I try to invoke the command by
Code:
rsync $OPTS $EXCLUDE $LINK $SRC $BCKP/$NAME/ >> "$LOG"
Which interestingly doesn't respect my --exclude array. NB, I also print the command into a file with
Code:
echo "Running: rsync $OPTS $EXCLUDE $LINK $SRC $BCKP/$NAME/" >> "$LOG"
and in that case, the string that's written to the log file is a valid rsync command, which respects the exclude array (tried it and ran it).

My strong suspicion is that there's some trickery regarding the " quotation marks I am not familiar with, as if I change the variables to
Code:
SRC="\"/\""
to contained escaped quotation marks, the whole rsync command falls apart, and starts reporting missing directories, while the rsync command written to the log file with the quotation marks present, work properly.

Anyone more familiar with bash, could you please give me a resource where to look? Thanks.
I am discovering Arch Linux as a total Linux newbie, and try to share my experience. I've managed to produce a working bootable USB and got my main PC up and running!
Reply
#2
(10-05-2019, 04:30 AM)TarsolyGer Wrote:
Code:
rsync $OPTS $EXCLUDE $LINK $SRC $BCKP/$NAME/ >> "$LOG"

The first thing I'd try it putting quotatin marks around the variables, like this:

Code:
rsync "$OPTS" "$EXCLUDE" "$LINK" "$SRC" "$BCKP/$NAME/" >> "$LOG"

Now you should be able to escape the quotation marks inside the variables.
My website - My git repos

"Things are only impossible until they’re not." - Captain Jean-Luc Picard
Reply
#3
(10-05-2019, 10:09 PM)leon.p Wrote: The first thing I'd try it putting quotatin marks around the variables, like this:

Code:
rsync "$OPTS" "$EXCLUDE" "$LINK" "$SRC" "$BCKP/$NAME/" >> "$LOG"

Now you should be able to escape the quotation marks inside the variables.

What I have found that in case of the exclude={"/dev/*","/proc/*",...} case it's not rsync that understands this expression, but it's a bash curly braces expansion o what, that passes exclude=/dev/* exclude=/proc/* etc to rsync, and if I put it inside a variable, it doesn't get expanded, so that's why rsync doesn't know what to do with it.

Code:
VAR=exclude={"/dev/*","/proc/*"}
echo $VAR             #exclude={"/dev/*","/proc/*"}

VAR=$(echo exclude={"/dev/*","/proc/*"})
echo $VAR             #exclude=/dev/* exclude=/proc/*

Is there any more elegant way to do this?

In the other places, it seems like I shouldn't use escaped quotation marks at all in the variables as they will be sent to the command as literal quotation marks, so rsync will think those are part of the filename. Quoting around the variable names also has some weird behavior, but I haven't got around testing it to detail yet.


Thanks for the help!
I am discovering Arch Linux as a total Linux newbie, and try to share my experience. I've managed to produce a working bootable USB and got my main PC up and running!
Reply
#4
So I have ran into this:


Code:
EXCLUDE=(--exclude={"Some Files","Shared files"})  #makes an array with two elements, both containing spaces
echo ${EXCLUDE[@]}                                #prints all the elements separated by spaces
                                                 #--exclude=Some Files --exclude=Shared files
rsync $OPTS ${EXCLUDE[@]} $LINK $SRC $BCKP/$NAME/ #fails as the spaces inside the folder names stop being escaped and the second words are treated as separate arguments by rsync after that.

Provided I don't have any spaces inside the exclude array, this works, but I have no idea how to force bash to interpret the elements of the array as their own parameters. If I do ${EXCLUDE[0]} ${EXCLUDE[1]}, that also fails to use the whole variable as parameter to rsync. If I introduce quotation marks escaped inside the quotation marks around my folder names  
Code:
(--exclude={"\"Some Files\"","\"Shared files\""})
this on the other hand, makes the escaped quotation marks literal, meaning, when passed as an argument to rsync, rsync will fail to find the files with the quotation marks in the file name.

Does anyone have any idea what's going on?
I am discovering Arch Linux as a total Linux newbie, and try to share my experience. I've managed to produce a working bootable USB and got my main PC up and running!
Reply
#5
Okay, for future reference, if you have trouble with quoting stuff passing as parameters when you do curly braces expansion:


Code:
OPTS="-aAXi"
EXCLUDE=({"/Directory One/*","/Shared files/*"})  # Curly braces expansion into an array of folders
SRC="/home/username/Documents/"
BCKP="/media/bkp/Documents"
LOG="$BCKP/logs/$NAME.log" # some variables I define in other places of the script
LINK="--link-dest=$BCKP/$LAST/"

rsync $OPTS "${EXCLUDE[@]/#/--exclude=}" "$LINK" "$SRC" "$BCKP/$NAME/"


This will expand the EXCLUDE array based on the pattern I found here:



Code:
--exclude=/Directory One/* --exclude=/Shared files/*


And due to the whole stuff being inside the quotation marks, for some reason, each expanded element of the array will be passed as an individual parameter, not affected by white spaces.
I am discovering Arch Linux as a total Linux newbie, and try to share my experience. I've managed to produce a working bootable USB and got my main PC up and running!
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)