Bash script to replace markers in file

Hi,
I have a file of the form (testfile.txt):
here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here
$file1.txt$
Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more $file2.txt$
Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here Some text in here and more here
$file3.txt$
I also have text files with names of the form "filen.txt" which have Markdown tables in them.
age   weight   sex 
15    45       m   
20    56       f   
25    65       f   
30    72       m   
I want to run a script to search for all the markers, and "inject" the relevant table at the marker by opening the file of that name in the same directory and doing a replace with it.
I have the "shell" of the script (sorry about the pun). I cannot however get it to do the injecting part.
I've tried Awk, sed, perl....you name it.
The script so far is:
#!/bin/bash
line=""
places=""
while read -r line; do
    places=$(awk '/\$*\$/ { print $0 }')
done < testfile.txt
cp testfile.txt try.txt
echo "$places" | {
    while read line; do
        strip=$(echo $line | sed 's/^\$//')
        strip=$(echo $strip | sed 's/\$$//')
        curtext=$(cat $strip)
         cat try.txt | {
            while read line; do
            # Something in here to take $strip add a $ at each end and replace
            #+with $curtext
            done
    done
I would really appreciate some help to fill in the area where the comment is. I am successfully getting all the other bits to work.
Many thanks
Mike

sed gives me a headache.  It has its place in the world, but I would never use it unless I had no other option. You do. Here's the same thing in AppleScript:
-- set main paths
set master_file to "/path/to/master file.txt"
set main_folder to "/path/to/folder/"
-- read master text, and break it down around '$' marks
set master_text to read master_file
set text_chunks to tid({input:master_text, delim:"$"})
-- loop through chunks, replacing file names with file contents
repeat with this_chunk in text_chunks
          if this_chunk ends with ".txt" then
                    set contents of this_chunk to (read (main_folder & this_chunk))
          end if
end repeat
-- write it all back out to the master file
set fp to open for access master_file with write permission
write tid({input:text_chunks, delim:""}) to fp
close access fp
on tid({input:input, delim:delim})
  -- generic handler for text delimiters
          set {oldTID, my text item delimiters} to {my text item delimiters, delim}
          if class of input is list then
                    set output to input as text
          else
                    set output to text items of input
          end if
          set my text item delimiters to oldTID
          return output
end tid
A bit wordier, but so much easier on the brain.

Similar Messages

  • Free script for replacing multiple footage files

    Hello!
    I made a simple script for easily changing multiple footage paths. I'm a 3D artist and found it a burden to replace all the vray render passes every time I rendered a new version. I'm sure it has other uses as well, like checking and setting which of your footage files are local and which ones are on a fileserver.
    You can also copy and move footage from one file to another with this tool.
    I'm not a professional scripter and would like some comments and crits. What would be the best place to publish something like this? For 3ds max there's scriptspot.com, is there something similiar for adobe scripts?
    Use at your own risk. Download here.

    looks useful for managing multipasses. Have a look at aescripts.com for publishing your script.

  • Simple bash script in SL to remove files

    1. SL on MBP
    2. launch Terminal
    3. pico remove-files
    4. in my script are two simple rm /path to file type statements
    5. save file in user/bin directory
    6. type name of command
    7. get "command not found" error
    no wonder 99.9999% of the world doesnt program - why doesnt this simple setup work right ?

    Chris Chamberlain1 wrote:
    just guessing, if it type ./remove-files this script works.
    That's the traditional way to invoke a program or command file if its location is not in your path. If that file is in /usr/bin and /usr/bin is in your path, I'm not sure why your situation doesn't work. On my Mac the directory bin in /usr is owned by root:wheel. If yours is the same, how did you manage to create your script there without permission manipulations? What exactly did you mean when you first mentioned "save file in user/bin directory"? Did you really mean /usr/bin or something else? If you meant some other location, you might want to add that location to your path with this command:
    PATH=yourotherdirectory:$PATH

  • Flack - a menu-driven BASH script to tag your FLAC files

    No need to use GUI to edit FLAC tags
    Project page
    Screenshots
    AUR
    Last edited by dyscoria (2009-03-04 15:24:04)

    Ok, I just tested the new version :
    - It's harder to read than before
    - If you select ie: remove tags and select the other option (o), you can't cancel your choice. You could always leave the field empty (nothing will be removed), but a simple option to leave would be better IMHO.
    - To quit, you sometimes have to enter b, b, b, q. An exiting option always available could be a nice improvement (I'm aware of ctrl+c, that's not the point).
    Something else, I don't know if it is a bug or not, but I currently have a track, where the field Comment contains "Track 1". This field isn't listed by flack.
    Sorry if can't give you feedback more often, but I don't tag my music collection everyday

  • Bash script - sqllite insert problem

    So, I'm making a bash script that inserts a config file into a database, but I can't get it to work.
    This is my best attempt so far:
    name="bashrc"
    location="/home/user/.bashrc"
    content=`cat "$location"`
    sqlite3 testdb "insert into dbase values ('`echo $name`','`echo "$location"`','`echo "$content"`')"
    the table is
    create table dbase(name text, location text, data text)
    I had a hard time trying to figure out how I'd be able to get $variables to work (because of the quotes) and this echo style seems to work, but there seems to be no way to insert the config file into the database.
    plaintext files seem to work no problem, but when I'm trying to insert some config file, it spits out errors. (It seems like it's trying to execute the file somehow?)
    Last edited by initbox (2009-08-29 15:28:49)

    initbox wrote:I was thinking the single quotes were stopping bash from replacing $variables with the content, hence the echoes.
    your single quotes are inside a double quote, so to bash they're just another character in the string.
    I'm still getting an SQL syntax error, so bash is no longer getting in the way, but this thing is a true nightmare. What next? This turned out to be way harder than I expected, I'm just doing this as a "fun" learning experiment.
    Your error is probably because I was escaping the quotes for bash, and I should have been escaping them for sqlite. In sqlite, escaping a quote is done by doubling it, so ' is escaped to '' (that's two apostrephes side by side, not a double quote. Compare '' and ").
    This untested sed command might work:
    content = $(cat $location | sed "s/\'/\'\'/g")
    The \'s are escaping the quote from bash, what its really doing is replacing ' with ''.
    Dusty

  • Bash script run via cron not executing MYSQL command

    I have a bash script which is run from a cron,
    It is executing as I have it write to a log file which it does correctly.
    I am wanting the bash script to restore a mysqldump file.
    When I run the bash script manually the dump file gets loaded fine. But when I run it through the cron the mysql command appears to be ignored.
    The mysqldump file is 54MB and I have checked to make sure that MYSQL is included in the global users path in /etc/profile
    Does anyone know why this maybe??
    Here is the bash file
    #!/bin/bash
    date >> /home/user/crons/crons.log
    echo "Started loadbackup" >> /home/user/crons/crons.log
    cd /home/user
    dbuser=root
    dbpass=password
    dbname=databasename
    filename=backup
    mysql -hlocalhost -u"$dbuser" -p"$dbpass" "$dbname" < " >> /home/user/crons/crons.log
    My crontab looks like
    02 17 * * * /home/user/crons/loadbackup.sh
    Many thanks
    Richard

    Hi Richard,
    Have you tried redirecting the script output in the cron to see if an error is being reported?
    I.e.
    02 17 * * * /home/user/crons/loadbackup.sh > /tmp/loadbackup.log 2>&1

  • How to run gdb from a bash script?

    I'm trying to use gdb for the 1st time.  I'd like to start it up by launching a bash script which will provide a file name for gbd commands and a file name for the executeable.  I've tried every which way but nothing works.  Can someone help?

    With respect to .gdbinit, the following is from the gdb.pdf documentation
    <http://www.gnu.org/software/gdb/documentation/>
    When you start gdb, it automatically executes commands from its init files, normally called ‘.gdbinit’. During startup, gdb does the following:
    1. Reads the init file (if any) in your home directory.
    2. Processes command line options and operands.
    3. Reads the init file (if any) in the current working directory.
    4. Reads command files specified by the ‘-x’ option.
    So you can have a different .gdbinit in the current working directory, which can be different for each program you are writing, assuming you wish to use separate directories for your projects.
    gdb has a 'source filename' command, which you can use to execute commands stored in a file anytime you want.  So you could put your breakpoints in a file that you then exeucte within gdb using
    source file.with.your.breakpoints
    Of course the file could have ANY gdb commands in it, not just breakpoints.
    Did you mean you compile your program with -g as in
    gcc -g program.c -o program
    Or did you mean -ggdb I've always used -g, but reading the gcc man page, I do not see any problems using -ggdb.
    Message was edited by: BobHarris

  • Using Bash script to edit config file

    This is a really simple question, but given that I'm just learning Bash scripting and having this solved now would be really illustrative for me, I would really thank some help here.
    I'm using uzbl, and running Tor+Polipo. So, as you will see below in the tail of the config file, there is a line to redirect the requests of uzbl through Polipo.
    # === Post-load misc commands ================================================
    sync_spawn_exec @scripts_dir/load_cookies.sh
    sync_spawn_exec @scripts_dir/load_cookies.sh @data_home/uzbl/session-cookies.txt
    # Set the "home" page.
    #set uri = https://duckduckgo.com
    # Local polipo proxy
    set proxy_url = http://127.0.0.1:8123
    # vim: set fdm=syntax:
    What I want to accomplish is to comment in/out that line with a key shortcut on Awesome. I've thought of doing 2 scripts to do so and using 2 differente key shortcuts, but I want to "toggle" the proxy redirection with only 1 shortcut. To do so, I suppose that the script should go something like:
    if
    tool 'set proxy_url = http://127.0.0.1:8123' config_file
    then
    tool '#set proxy_url = http://127.0.0.1:8123' config_file
    else
    if
    tool '#set proxy_url = http://127.0.0.1:8123' config_file
    then
    tool 'set proxy_url = http://127.0.0.1:8123' config_file
    fi
    fi
    I know little about sed, but I think is the tool for this job. The most intriging part to me is to ask sed to print the regular expression when it finds it in the config file, and use that as an input in the conditional statement.
    Well, this is a mess I have done here. Hope there is a simple answer to this.
    Thanks in advance.-

    You can do this with a single sed command:
    sed -i 's/^#set proxy_url/set proxy_url/;
    t end;
    s/^set proxy_url/#set proxy_url/;
    : end' config_file
    This edits the file in-place (-i) and first tries to replace the commented with the uncommented line. If that suceeds, sed jumps to the "end" label. If not, it tries to replace the uncommented with the commented line. Thus you don't have to include any logic about the current state: if the first substitution succeeds, the line was obviously commented, if not, it was uncommented, and the second substitution should succeed.
    Note that my knowledge of sed is very limited. There might be a simpler way to do this.
    EDIT: For the sake of example, here's how to do the same in bash using regular expressions. Note how this script needs to use a temporary file to simulate in-place editing, how it needs to process the file line by line manually, etc. All things that sed does out of the box...
    #!/bin/bash
    tmp=test.conf.tmp
    echo -n "" > "$tmp"
    while read line; do
    if [[ "$line" =~ ^#set\ proxy ]]; then
    echo "${line/\#/}" >> "$tmp"
    elif [[ "$line" =~ ^set\ proxy ]]; then
    echo "#$line" >> "$tmp"
    else
    echo "$line" >> "$tmp"
    fi
    done < test.conf
    mv test.conf.tmp test.conf
    To answer your original question, the line
    if [[ "$line" =~ ^#set\ proxy ]]; then
    reads: if the line begins with a "#", followed by "set proxy", then...
    Last edited by hbekel (2011-03-20 10:40:16)

  • Shell Scripting to replace file type

    Hey all,
    I'm having some trouble with a shell script. I'm trying to run a script that will look through all the folders in a given directory for a certain type of file (.data) and then will run a getfileinfo -t on the .data file and, if the type is "" (empty), replace it with a given string.
    So far I can get the command to look through directories and I can get it to echo the getfileinfo -t, but I can't put the result of the command into a variable so I can check to see if it meets the "" condition. Any help would be greatly appreciated.
    Also, I am doing this in CSH, but I am no opposed to doing it another way, just so long as it works.

    Hi JohnSka7,
       I really like Niel's solution but it can be done in a script without AppleScript if you have Apple's Developer Tools installed. I don't do csh so I'll give it to you in bash and leave it as an exercise to translate:
    #!/bin/bash
    oldIFS=$IFS
    IFS=$'\n'
    for FILE in $( find "$1" -type f -name "*.data" ); do
       IFS=$oldIFS
       if [ \"\" = $( /Developer/Tools/GetFileInfo -t "$FILE" ) ]; then
          /Developer/Tools/SetFile -t "XXXX" "$FILE"
       fi
    done
    This temporarily sets the Input Field Separator, IFS, to a newline so that the snippet can handle filenames containing spaces. It assumes that the first argument to the script is the directory to be searched. Of course you should change "XXXX" to the type code you want and you can arrange for that to be specified in another argument. If you copy-and-paste this into a script, be sure to replace the non-breaking spaces I've used for indention with real spaces.
    Gary
    ~~~~
       Maslow's Maxim:
          If the only tool you have is a hammer, you treat
          everything like a nail.

  • Strange noise problem with music layer after replacing the music file using script in CS6

    Hey guys,
    I got a strange problem. Sometime when I import a music file, or replace a music footage with another music file using the "replace()" or "importFile()" method, the music layer added one second noise at the beginning. like this,
    Does anyone know what's going on here?
    I'm using After Effects CS6 windows Server 2008 64bit.
    Thanks in advance!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    Hi Myleniu,
    Thanks for your replay. Actually, it doesn't matter what makes up my comp, this problem just randomly happen when I render any comp. For example, I have a comp named "noise", I run a script to replace the music then render it, I use a "for" loop to do this 5 times and the music noise problem just happen one time.
    I'm starting to think that win server might be the problem.

  • How to download file using ftp in bash script

    Hi! I'm runnig a bash script in solaris i want within the script to dowload file using ftp
    How can i do it?
    Tanks a lot

    hello,evgchech
    please try this way:
    1. In the bash script, try following command:
    ftp -n < ftpcmdfile2 in the ftpcmdfile (which is a file),coding the interactive commands of FTP such as:
    user anonymous  [email protected]
              cd /var/sun/download
              bi
              mget *.*
              bye
         try it and good luck!
    Wang Yu
    Developer Technical Support
    Sun Microsystems
    http://sun.com/developers/support

  • Dowload file using ftp in bash script

    Hi! I'm runnig a bash script in solaris i want within the script to dowload file using ftp
    How can i do it?
    Tanks a lot

    hello,evgchech
    please try this way:
    1. In the bash script, try following command:
    ftp -n < ftpcmdfile2 in the ftpcmdfile (which is a file),coding the interactive commands of FTP such as:
    user anonymous  [email protected]
              cd /var/sun/download
              bi
              mget *.*
              bye
         try it and good luck!
    Wang Yu
    Developer Technical Support
    Sun Microsystems
    http://sun.com/developers/support

  • A bash script to backup system only with modified files

    Hi,
    I've made a simple bash script to exam which files are modified and needed to be backed up.
    It will use:
    1. the hash in Pacman's database (/var/lib/pacman/local/<pkg_ver>/files
    2. if no hash in the database, calculate it by our self from cache (/var/cache/pacman/pkg/<pkg_ver>.pkg.tar.gz
    3. if no cache found, compare build date and last modified date
    4. otherwise, this file better be backed up.
    Currently it will only print which files are needed to be backed up, but no backup actually happens. (dry run)
    And it is only in early development stage, please be careful.
    USAGE:
    <the script name> <where to backup files, a directory> <the files, or directories, separated by space, that are going to be examined>...
    Here is the code.
    #!/bin/bash
    # usage:
    # $1: file to backup
    function do_backup() {
    echo
    function smart_bak() {
    pkg=$(pacman -Qo "$1" 2>/dev/null)
    if [ 1 -eq $? ] ; then
    # No package owns $1 (locally created)
    if [ "$1" != "${1%.pacsave}" ] ; then
    echo "skip $1 (left by removed package)"
    else
    echo "backup $1 (local file)"
    fi
    else
    pkg=${pkg#$1 is owned by }
    # evaluate
    # by hash
    cur=$(md5sum $1)
    cur=${cur%% *}
    pkg_ver=${pkg/ /-}
    org=$(grep "^${1:1}" "/var/lib/pacman/local/$pkg_ver/files")
    org_hash=${org##* }
    if [ "$org" != "$org_hash" ] ; then
    # the org hash is in Pacman's database
    if [ "$org_hash" = "$cur" ] ; then
    echo "skip $1 (original config)"
    else
    echo "backup $1 (modified config)"
    fi
    else
    # no hash
    # find out hash myself?
    ARCH=$(uname -m)
    if [ -r "/var/cache/pacman/pkg/$pkg_ver-$ARCH.pkg.tar.gz" ] ; then
    org=$(tar -Oxzf "/var/cache/pacman/pkg/$pkg_ver-$ARCH.pkg.tar.gz" "${1:1}" | md5sum -)
    org_hash=${org%% *}
    if [ "$cur" = "$org_hash" ] ; then
    echo "skip $1 (original)"
    else
    echo "backup $1 (modified)"
    fi
    else
    # no cache, may be a AUR package
    # fall back to built date?
    date=$(pacman -Qi ${pkg% *} | grep "^Build Date")
    date=${date#*: }
    date=$(date -d "$date" +%s)
    mod=$(ls -l $1)
    mod=${mod% $1}
    mod=${mod#* * * * * }
    mod=$(date -d "$mod" +%s)
    tmp1=$(expr $mod "+" 60)
    tmp2=$(expr $mod "-" 60)
    if [ "$date" -le "$tmp1" -a "$date" -ge "$tmp2" ] ; then
    echo "skip $1 (the same date)"
    else
    echo "backup $1 (unknown)"
    fi
    fi
    fi
    fi
    function smart_bak_dir() {
    for i in $(ls -A "$1") ; do
    tmp="${1%/}/$i"
    if [ -f "$tmp" ] ; then
    smart_bak "$tmp"
    elif [ -d "$tmp" ] ; then
    smart_bak_dir "$tmp"
    else
    echo "skip $tmp (not a regular file nor a directory)"
    fi
    done
    # usage:
    # $1: the directory to store this backup
    # $2: directory to evalualte for backup
    # function main()
    # init
    target="$1"
    shift
    # check
    if [ ! -d "$target" -o ! -x "$target" -o ! -w "$target" ] ; then
    exit 4
    fi
    for i in $* ; do
    if [ -f "$i" ] ; then
    smart_bak "$i"
    elif [ -d "$i" ] ; then
    smart_bak_dir "$i"
    else
    echo "skip $i (unknown argument)"
    fi
    done
    Good luck,
    bsdson
    Last edited by bsdson.tw (2008-12-30 07:53:05)

    Thanks for this script. Nice work.
    Can we traverse the pacman.log backwards up and rollback each operation (including "installed" in this). Something like:
    history=$(tail -n +$(grep -m 1 -n "$1" "$LOG" | cut -d ":" -f 1) "$LOG" | tac | grep -E "(upgraded|installed)" | cut -d " " -f 3-)
    Last edited by g33k (2008-11-04 15:08:07)

  • Bash script to dumpstream many files simultaneously with mplayer

    hi guys
    i have a problem which i´m sure can be solved with the power of bash scripting
    unfortunately i´m no bash scripting guru and all my experiments failed so far
    the problem:
    i have a file in which are links(streaminglinks)
    mplayer offers the funtion to dump such a stream with simply issuing
    mplayer -dumpstream mms://path/to/video -dumpfile video1
    for example.
    now i want mplayer to download this streams specified in the links-file automatically.
    basically all it required is a bash script which goes through the link file and generates a command like mplay -dumpstream <link> -dumpfile video<n>
    (where n is the nth link) and execute it.maybe there a even simpler solutions
    well since i´m not that experienced with bashscripting i can´t solve that problem at my self....
    i´m grateful for any help

    hey guys
    thx for the two scripts.
    my approach was nearly the same as your´s kraluz with the difference that it doesn´t work
    but they both have a little blemish
    they download the files sequentially not simultaneously
    how could that be realised
    thx in advance

  • Passing Files/Folders to BASH script...

    I'm trying to pass a list of files and folders selected in finder to a bash script so that I can use VLC to perform some transcoding and concatenation on them.
    I'm using "Get Selected Finder Items" to pass the "Files/Folders" output "As Arguments" to my script.
    Problem is with spaces in the filenames. Apparently Automator delimits the list of files with spaces. So I have spaces in the filenames and spaces between the files. So my bash script just sees a list like this:
    /Users/johnt/Documents/3rd Party Audio For Conversion/WFWC - 3.6.2009 3.7.2009/01c Director Report March 6 2009.wav /Users/johnt/Documents/3rd Party Audio For Conversion/WFWC - 3.6.2009 3.7.2009/01b Comm Disc Mar 6 2009.wav
    ...with no way to differentiate each file. Is there a way to tell automator to escape the spaces in the file paths so that my loop doesn't try to run through $@ as:
    /Users/johnt/Documents/3rd
    Party
    Audio
    for
    ... etc etc
    Thanks

    Thanks Neil, that was helpful. I'll play with that AppleScript and see what happens.
    I figured this was better suited in the Automator forum since I'm trying to format the output of what Finder sends before it ever hits the Bash script. My Bash script itself has no problem, so I didn't see any point going over to the UNIX forums.
    You gotta love the ability of OSX to be able to handle all sorts of technologies and integrate them well. Apparently, there's nothing Apple can do to make it's users do the same thing.

Maybe you are looking for

  • My ipod wont let me download anything

    My ipod wont let me but anything from the app store, its saying i need to sort out billing issues but when i go to type in my security code it says its unavailable

  • How to test RESTful service?

    Hello Everyone! Working on APEX REST services, my query is in regard with the below link: http://docs.oracle.com/cd/E37097_01/doc/doc.42/e35128/restful_svc.htm#autoId6 In the above link, heading '5.5.2 Retrieving Data Based on a Parameter (employees/

  • Can you restore from a backup that was made on a previous version of iOS?

    Hello. I had an iPhone 3GS unning whatever the latest proper version of iOS currently is at the time of writing (4.3.5, is it?) but on Saturday I was on a rather drunken night out and I lost my phone. Silly me. But I was out of contract and the new i

  • Problems with Windows 7 home premium and ovi

    Excuse me for my english. I've a big problem with Windows 7 and NOKIA N97 32G I install the latest version of Nokia Ovi Suite 2.0.2.42. When I connect the phone with USB and select PC suite mode in the phone, OVI don't see the phone and say method no

  • Uml code development

    can you develop forms from uml tools like rational for example? -- and can that code be managed ? am attending this online seminar http://ad.doubleclick.net/clk;5756991;8195099;i?http://www.rational.com/events/forms/webinar-reg.jsp?EVENTID=2774 but n