Batch remove of all ^M characters

[Sapphiron]

Hi everyone

 

I have an application that needs to run, both in windows and in linux.

 

Do any of you know about a tool or command that I can use to remove or detect all the ^M characters from all the configuration files in the folder.

 

thanks in advance.

[ramfree17]

in vi i do this

 

:%s/^M//g

 

note that ^M is produced by doing the following combination : ctrl+v ctrl+m

 

you can also do this in batch using sed.

 

ciao!

[Sapphiron]

sed?

 

please explain

[Gowator]

sed?

 

please explain

<{POST_SNAPBACK}>

Its a line editor...

However if you need to look that up try

dos2unix (which strips off the ASCII 13) It should be urpmi'able if not try a RPMFIND.NET search... its a really simple unix utility since before time existed!

[ramfree17]

oh yeah, that one too.

 

im just waiting for aru to come here and start using ed for inline editing.

 

ciao!

[aru]

aru stepping in!!!

 

this can be a one line command to emulate dos2unix (it also keeps a backup copy of the original DOS file):

 

~$ for file in $(egrep -l '^M$' *); do cp $file $file.bak && sed 's/^M$//' $file.bak > $file; done

 

ciao!

 

Edited: explantion, the command processes all files found with carriage returns (^M characters at the end of line) in the current directory keeping a .bak copy of the original file as it was.

Edited November 30, 2004 by aru

[aru]

oops, I've just noticed the subtitle of the thread. Then in case you have hundreds of files you may experience memory problems using a for loop because of variable limits, so use a pipe plus a while loop instead:

 

~ $ egrep -l '^M$' * | while read file; do cp $file $file.bak && sed 's/^M$//' $file.bak > $file; done

 

remember that ^M is <ctrl-v><ctrl-M>

 

hth

[ramfree17]

gAru, you disappoint me :( . i thought you are going to teach us how to use ed for this scenario so you wont need to have use a second file (only ed's buffer).

 

im taking down your guru ratingg by a itsy-bitsy-teenie-weenie notch for this one.

 

ciao!

[aru]

bah, sed was created because ed is awful for doing such uninteractive editions, but your desires are orders for me, so here you are :P

 

~$ egrep -l '^M$' * | while read file; do ed -s $file <<END
> %s/^M$//
> w
> q
> END
> done
~$

 

As you can see the sed version (IMHO) is more elegant, but ed does the job well too.

 

Sapphiron, be careful with this way because no bak file is created

 

:deal: aru climbing up to ramfree's top guru rank again

[paul]

might be useful

#!/bin/sh
#
# Script to remove ^M from files for DOS <-> UNIX conversions
#

if [ $# != 1 ]
then
 echo "Usage: remove_control_m.sh <extension of files>"
 echo ""
 echo "Example: remove_control_m.sh php3"
 exit
fi

for i in `find . -name "*.$1"`
        do
        echo $i
        tr -d '\015' < $i > ${i}.new
        rm $i
        mv ${i}.new $i
       done;

[ramfree17]

bah, sed was created because ed is awful for doing such uninteractive editions, but your desires are orders for me, so here you are  :P

<{POST_SNAPBACK}>

 

 

i knew you only need a little prodding. sometimes backups just add to the hassle. and its up to the user which one he wants to employ. :)

 

:deal:  aru climbing up to ramfree's top guru rank again

<{POST_SNAPBACK}>

 

ok, im putting you up only by an itsy-bitsy step because you are slipping because the old gAru would not need to be prodded to list down all options (even if some of those seem like gibberish and only you can understand :P). now get back on the scripting sweatshop!

 

ciao!

[aru]

Old times have been always better. I'm sure old ramfree would have done much more blood on this, BUT here is the most complicated way of doing this I can think at this time of the morning:

 

#! /bin/bash

exec 8<&0; exec < "$1"  # saving and redirecting stdin to $1

exec 9>&1; exec > "$2"  # saving and redirecting stdout to $2

tr -d '\015'            # processing using paul's tip; no need to put in or out files

exec 0<&8 8<&-          # restoring stdin

exec 1>&9 9>&-          # restoring stdout

 

 

Ofcourse this code is just for twisting purpose so it is not meant to be productive though it works (just add the loop as in my #2 post to process the whole directory):

 

~$ egrep -l '^M$' * | while read file; do cp "$file" "$file.bak" && bizarrescript "$file.bak" "$file"; done

 

[ramfree17]

 

and i dont even know what that means if not for the comments. heck, even with the comments i cant even fathom what they do. hehehehehe...

 

ciao!

[Sapphiron]

might be useful

#!/bin/sh
#
# Script to remove ^M from files for DOS <-> UNIX conversions
#

if [ $# != 1 ]
then
 echo "Usage: remove_control_m.sh <extension of files>"
 echo ""
 echo "Example: remove_control_m.sh php3"
 exit
fi

for i in `find . -name "*.$1"`
        do
        echo $i
        tr -d '\015' < $i > ${i}.new
        rm $i
        mv ${i}.new $i
       done;

<{POST_SNAPBACK}>

 

Forgive me, but i know very little about shell scripting, but will this script recursively step into subfolders?

[aru]

yes

 

man find

 

;)