Small script to run a instance of mysql in my home directory using the binary provided by the OS. Maybe some day I will make it more usable but for the time being it suits my needs.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
| #!/bin/bash
MYSQL_HOME=$HOME
MYSQL_USER=$USER
MYSQL_PORT=13306
ACTION=$1
if [ $ACTION == 'init' ]; then
echo "initalising new mysql installation at $MYSQL_HOME/var/lib/mysql"
rm -rf $MYSQL_HOME/var/lib/mysql
mkdir -p $MYSQL_HOME/var/lib/mysql $MYSQL_HOME/mysql/data $HOME/var/lib/mysql/tmp
mysql_install_db --user=$MYSQL_USER --datadir=$MYSQL_HOME/var/lib/mysql/data/
cat < $HOME/.my.cnf
[mysqld]
user=$MYSQL_USER
datadir=$MYSQL_HOME/var/lib/mysql/data
log-error=mysqld.log
pid-file=mysqld.pid
socket=$MYSQL_HOME/var/lib/mysql/tmp/mysql.sock
port=$MYSQL_PORT
[client]
user=$MYSQL_USER
socket=$MYSQL_HOME/var/lib/mysql/tmp/mysql.sock
port=$MYSQL_PORT
[safe_mysqld]
log-error=mysqld.log
pid-file=mysqld.pid
port=$MYSQL_PORT
EOT
echo " Done"
elif [ $ACTION == 'start' ];then
echo -n "Starting mysqld as $MYSQL_USER on $MYSQL_PORT"
nohup mysqld_safe &>/dev/null &
echo " Done"
elif [ $ACTION == 'stop' ];then
PID=$(cat $MYSQL_HOME/var/lib/mysql/data/mysqld.pid)
echo -n "Stopping mysql instance [$PID]"
kill `cat $MYSQL_HOME/var/lib/mysql/data/mysqld.pid`
echo " Done"
else
echo "Unknown command '$1'"
fi |
UPDATED SCRIPT:
I decided to update the script and make it less hacky
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
| #!/bin/bash
MYSQL_HOME=$HOME
MYSQL_USER=$USER
MYSQL_PORT=13306
function init {
echo "initalising new mysql installation at $MYSQL_HOME/var/lib/mysql"
rm -rf $MYSQL_HOME/var/lib/mysql
mkdir -p $MYSQL_HOME/var/lib/mysql $MYSQL_HOME/mysql/data $HOME/var/lib/mysql/tmp
mysql_install_db --user=$MYSQL_USER --datadir=$MYSQL_HOME/var/lib/mysql/data/
cat <<EOT > $HOME/.my.cnf
[mysqld]
user=$MYSQL_USER
datadir=$MYSQL_HOME/var/lib/mysql/data
log-error=mysqld.log
pid-file=mysqld.pid
socket=$MYSQL_HOME/var/lib/mysql/tmp/mysql.sock
port=$MYSQL_PORT
innodb_file_per_table
[client]
user=$MYSQL_USER
socket=$MYSQL_HOME/var/lib/mysql/tmp/mysql.sock
port=$MYSQL_PORT
[safe_mysqld]
log-error=mysqld.log
pid-file=mysqld.pid
port=$MYSQL_PORT
EOT
echo " Done"
}
function start {
echo -n "Starting mysqld as $MYSQL_USER on $MYSQL_PORT"
nohup mysqld_safe &>/dev/null &
echo " Done"
}
function stop {
if [ -e $MYSQL_HOME/var/lib/mysql/data/mysqld.pid ]; then
PID=$(cat $MYSQL_HOME/var/lib/mysql/data/mysqld.pid)
echo -n "Stopping mysql instance [$PID]"
kill $PID
echo " Done"
else
echo "mysqld.pid file doesnt exist"
fi
}
function usage {
echo "Unknown command '$1'"
}
function status {
if [ -e $MYSQL_HOME/var/lib/mysql/data/mysqld.pid ]; then
PID=$(cat $MYSQL_HOME/var/lib/mysql/data/mysqld.pid)
ps -p $PID &>/dev/null
RTN=$?
else
RTN=-1
fi
if [ $RTN -eq 0 ]; then
echo "Mysql Running, pid:$PID"
elif [ $RTN -eq 1 ]; then
echo "mysql stopped but pidfile exists"
elif [ $RTN -eq -1 ]; then
echo "Stopped"
fi
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart)
stop
start
;;
init)
init
;;
*)
echo $"Usage: $0 {start|stop|status|restart|init}"
exit 1
esac |
Have you ever needed to run a bunch of long running processes from a bash script and get their return codes ? I come across this issue quite frequently in my line of work. The most common one is where i need to run rsync to collect files from many machines then if successful run some other task. Depending on the amount of servers and data this can take several hours to run sequentially and I don’t really like waiting around to check the output so that I can run the next task.
How to speed it up? The obvious way would to be to background the rsysc commands but then I dont know if they were all successful. What if one fails? How would I know which one? Some how I needed to catch the return codes of all the sub-shells and be able to match them to a command. This is where the bash command wait come into play.
~]$ help wait
wait: wait [id]
Wait for job completion and return exit status.
Waits for the process identified by ID, which may be a process ID or a
job specification, and reports its termination status. If ID is not
given, waits for all currently active child processes, and the return
status is zero. If ID is a a job specification, waits for all processes
in the job’s pipeline.
Exit Status:
Returns the status of ID; fails if ID is invalid or an invalid option is
given.
The idea is to collect the PID of each sub-shell using the $! variable and adding it to a list. Then use the wait command to wait for each sub-shell to finish and exit with the return code of the sub-shell command. By adding these return codes to another list then we can iterate over them and match them up with the original list. We split the original list into an array towards the end so we can reference the individual items by an index.
In the following example I have used wget and also deliberately changed testfile03.txt to testfile06.txt to show an example of a non 0 return code.
testfile01
testfile02
testfile03
testfile04
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
| #!/bin/bash
my_list="
http://www.thegoldfish.org/wp-content/uploads/2010/08/testfile01.txt
http://www.thegoldfish.org/wp-content/uploads/2010/08/testfile02.txt
http://www.thegoldfish.org/wp-content/uploads/2010/08/testfile06.txt
http://www.thegoldfish.org/wp-content/uploads/2010/08/testfile04.txt
"
results=''
pids=''
for X in $my_list ; do
wget -o /dev/null $X &
pid=$!
pids="$pids $pid"
done
for pid in $pids
do
wait $pid
result=$?
results="$results $result"
done
echo $results
i=0
my_array=( $my_list )
for ret_val in $results; do
echo ${my_array[$i]} returned $ret_val
((i++))
done |
From Thrift Site – “Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml.”
When I initially tried to install thrift I had some difficulty because of an error which follows. This error makes you think you need something to do with boost but I am pretty sure the issue was that I was missing gcc-c++
checking for boostlib >= 1.33.1... configure: error: We could not detect the boost libraries (version 1.33 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
I did this on a ‘Desktop’ install on my laptop so this may not be every package required. You will need to install at least the following packages.
yum -y install autocon automake libtool flex \
boost-devel gcc-c++ perl-ExtUtils-MakeMaker byacc
Download the thrift from subversion to get the latest and greatest.
svn co http://svn.apache.org/repos/asf/incubator/thrift/trunk thrift
cd thrift
Then, following the README, you need to do the following.
./bootstrap.sh
./configure
make
sudo make install
And that appears to be it. If you find any issues please leave a comment and I will try on an @base install of Fedora to iron out any other dependencies.
Stunnel is a quick way on taking a non ssl connection and being able to wrap it in ssl for security
stunnel version 4 – Fedora 12/RHEL 5.3 /Centos 5.3
vim /etc/stunnel/stunnel.conf
add in
client=yes
[gmail]
accept = 127.0.0.1:50000
connect = mail.google.com:443
then run
stunnel version 3 – Ubuntu 8.10 (I haven’t used newer versions)
Ubuntu 8.10 has 2 versions of stunnnel: stunnel3 and stunnel4. They have created a symbolic link from /usr/bin/stunnel -> /usr/bin/stunnel3
If you would like to use version 4 you can use the command stunnel4 otherwise if you wish to use the default version, you will need to create a self signed certificate
openssl req -new -x509 -days 3650 -nodes -out /etc/stunnel/stunnel.pem -keyout /etc/stunnel/stunnel.pem
Then to start stunnel use the following command
stunnel -c -d localhost:50001 -r mail.google.com:443
I quite often need to remove a single line from a file by its line number. The most common use case for me is the known_hosts file when I have reinstalled a system, I have in the past used vim and navigated to the line then removed it. This is all well and good but it gets to be a pain having to do it repeatedly, especially when you manage around 1000 servers and the get rebuilt frequently. Finally today I had had enough so wrote a little script to do this task easily. Hopefully someone else finds this useful
Its usage is : delline LINE FILE
#!/bin/bash
LINE=$1
FILE=$2
if [ ! -f $FILE ] ; then
echo "can't read $FILE: No such file or directory"
exit 1
fi
if [ `expr $LINE + 1 2> /dev/null` ] ; then
sed -i "${LINE}d" $FILE
else
echo $LINE is not numeric
exit 1
fi