I’m looking for an ad-hoc (easy) way to share files with my remote server…
I want to call the uploader in a shell environment I created one script for that (maybe with less effort than using ruby or other language) for that, it only needs Curl installed on system and standard Unix commands like grep, tr and sed.
Reference: http://www.pauloamgomes.net/blog/dropbox-shell-uploader
Download: http://dl.dropbox.com/u/885388/DropBoxSimpleUploader.sh
This morning I’ve had issues with my linickx.com cluster, the file system on both nodes went to read-only and I ended up in a world of pain.
[root@georgia ~]# sudo /etc/init.d/httpd start Starting httpd: [root@georgia ~]# tail -f /var/log/messages Jan 9 09:48:35 georgia kernel: [ 474.259265] (httpd,1712,0):ocfs2_reserve_clusters_with_limit:1190 ERROR: status = -22 Jan 9 09:48:35 georgia kernel: [ 474.259271] (httpd,1712,0):ocfs2_lock_allocators:2546 ERROR: status = -22 Jan 9 09:48:35 georgia kernel: [ 474.259276] (httpd,1712,0):ocfs2_write_begin_nolock:1732 ERROR: status = -22 Jan 9 09:48:35 georgia kernel: [ 474.259282] (httpd,1712,0):ocfs2_write_begin:1856 ERROR: status = -22 Jan 9 09:49:31 georgia kernel: [ 530.660071] o2net: no longer connected to node amy (num 1) at 10.176.128.7:7777 Jan 9 09:49:31 georgia kernel: [ 530.661856] ocfs2: Unmounting device (147,0) on (node 2) Jan 9 09:59:46 georgia kernel: [ 1145.772174] o2dlm: Nodes in domain E9447DBE28154DAEA1B988CEC573EB64: 2 Jan 9 10:01:05 georgia kernel: [ 1223.911192] o2net: connected to node amy (num 1) at 10.176.128.7:7777 Jan 9 10:01:09 georgia kernel: [ 1227.933348] o2dlm: Nodes in domain E9447DBE28154DAEA1B988CEC573EB64: 1 2 Jan 9 10:01:09 georgia kernel: [ 1227.938693] ocfs2: Mounting device (147,0) on (node 2, slot 1) with ordered data mode. Jan 9 10:02:35 georgia kernel: [ 1314.467741] OCFS2: ERROR (device drbd0): ocfs2_validate_gd_self: Group descriptor #419328 has bit count 32256 but claims that 45941 are free Jan 9 10:02:35 georgia kernel: [ 1314.467754] File system is now read-only due to the potential of on-disk corruption. Please run fsck.ocfs2 once the file system is unmounted. Jan 9 10:02:35 georgia kernel: [ 1314.467764] (httpd,2389,0):ocfs2_search_chain:1729 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467771] (httpd,2389,0):ocfs2_claim_suballoc_bits:1902 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467778] (httpd,2389,0):__ocfs2_claim_clusters:2185 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467783] (httpd,2389,0):ocfs2_local_alloc_new_window:1204 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467790] (httpd,2389,0):ocfs2_local_alloc_slide_window:1306 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467798] (httpd,2389,0):ocfs2_reserve_local_alloc_bits:695 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467803] (httpd,2389,0):ocfs2_reserve_clusters_with_limit:1190 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467809] (httpd,2389,0):ocfs2_lock_allocators:2546 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467814] (httpd,2389,0):ocfs2_write_begin_nolock:1732 ERROR: status = -22 Jan 9 10:02:35 georgia kernel: [ 1314.467821] (httpd,2389,0):ocfs2_write_begin:1856 ERROR: status = -22 Jan 9 10:02:36 georgia kernel: [ 1315.046965] OCFS2: ERROR (device drbd0): ocfs2_validate_gd_self: Group descriptor #419328 has bit count 32256 but claims that 45941 are free ^C [root@georgia ~]#
What made this odd is that running fsck.ocfs2 as suggested made no difference, as the output said that the disk was clean.
[root@georgia ~]# fsck.ocfs2 /dev/drbd0 fsck.ocfs2 1.4.4 Checking OCFS2 filesystem in /dev/drbd0: Label: linickxcluster UUID: E9447DBE28154DAEA1B988CEC573EB64 Number of blocks: 1048535 Block size: 4096 Number of clusters: 1048535 Cluster size: 4096 Number of slots: 4 /dev/drbd0 is clean. It will be checked after 20 additional mounts. [root@georgia ~]#
I learn that in fact the above output was a lie! For any future googlers seeing the same issue, run:
fsck.ocfs2 -fy /dev/drbd0
The f & y force a check and fix any found issues, the force on my filesystem found the errors and we appear to be back online 
PHP make you think of web app’s right? … well, did you know you can run it from the CLI to?
Recently I’ve been doing a lot of spreadsheet and sub-netting type stuff, whilst doing this mundane work I’ve realised that I can get scripts to work for me. I’ve started to post a few PHP network functions to github which I’ve been using.
Here’s an example: I have a nokia firewall, the routing table in voyager is shown in the following format -
Network / CIDR Mask , Gateway 10.0.0.0 / 8 , 10.0.0.1
I need that same routing in a Cisco ASA format -
Network , Mask , Gateway 10.0.0.0, 255.0.0.0, 10.0.0.1
Now this is not a problem for a few routes but the firewall I’m looking at now has 177 static routes, which I don’t want to convert manually.
Roll in PHP!
Save the original routing table as a .csv file. ColA = ip/mask , ColB = gateway.
Save this a route_conv.php
<?php
/**
Change the below to your CSV File.
**/
$firewall_csv = "./routes_cdr.csv";
/**
Function to convert CIDRs such as "23" to decimall dotted like "255.255.254.0"
I've got more of these: https://gist.github.com/1309388
**/
function cidr2mask($netmask) {
$netmask_result="";
for($i=1; $i <= $netmask; $i++) {
$netmask_result .= "1";
}
for($i=$netmask+1; $i <= 32; $i++) {
$netmask_result .= "0";
}
$netmask_ip_binary_array = str_split( $netmask_result, 8 );
$netmask_ip_decimal_array = array();
foreach( $netmask_ip_binary_array as $k => $v ){
$netmask_ip_decimal_array[$k] = bindec( $v ); // "100" => 4
}
$subnet = join( ".", $netmask_ip_decimal_array );
return $subnet;
}
ini_set('auto_detect_line_endings', true); // detect CR
if (file_exists($firewall_csv)) {
$file = fopen($firewall_csv, 'r');
$counter = 0; // array counter
while (($data = fgetcsv($file)) !== FALSE) {
list($ip, $netmask) = split( "/", $data[0] ); // SPLIT Col A into IP & Mask
$netmask = cidr2mask($netmask); // Covert Mask
$gateway = $data[1]; // Col B
/**
This echo is the CSV style output, but you could change this to echo "route add $ip $mask $ gateway \n" for unix style output.
**/
echo $ip . "," . $netmask . "," . $gateway . "\n";
}
fclose($file);
} else {
echo "404: $firewall_csv \n"; // FILE NOT FOUND.
}
?>
from your CLI run “php route_conv.php” and enjoy the output!
As a cloud server customer you get access to rackspace’s free DNS service.
When I fist saw this product I had an instance light-bulb moment, I could stop paying for a dynamic DNS service and build my own private one. As a broadband (DHCP) user I have a very basic requirement of needing to regularly update an A record so that I can find my pc
To bring my idea into fruition I began researching; I need a cli tool which I could run from cron on my linux box (to send the DNS update requests to rackspace). In my research I found rscurl, a cli tool to control cloud servers, as rackspace have a standard API for all their products I have been able to use rscurl to develop rsdns.
rsdns is a series of cli tools to adding/deleting/changing rackspace DNS records, as part of the tool development I have created a script called rsdns-dc.sh to run on my machine, below is a short how to:
How to get free dynamic dns from rackspace.
Posted here so I can find it again, and yep worked on my SDcard!
This Recovery example guides you through PhotoRec step by step to recover deleted files or lost data from a reformatted partition or corrupted file system.
Something I discovered ages ago (so long ago that my trial license expired) but forgot to post is that you can get an LTM VE to work in Virtual Box.
To get started download the ESX image from the F5 VE Trial Page, when you get the download import the OVA into virtualbox.
The only thing I needed to tweak after the import was the interface settings, you need two intels and a PCNet, the PCNet is the management interface. Set the PCNet to host only networking, give your laptop/pc an ip address on the host only network a 192.168.1 address and you’re good to go!
You may experience HIGH CPU issues after boot, but since these boxes are based on linux, you can use the divider=10 centos trick.
Enjoy your virtual load balancing!
I really hope that the next version of fedora gets called “Beefy Miracle”
You can vote for your favourite name at https://admin.fedoraproject.org/voting/about/relnamef16 but I can’t see how anything can beat the beef!
One of the things I’d really like is process priorities for virtual box. In the forum I posted a couple of shell commands that I regularly type… which gets a bit tedious, following a recent article on lifehacker reviewing mac text expanding I’ve been prompted to automate a few things… below is a little shell script to lower the priority (renice) of all running virtual machines.
The advantage of doing this is that your host machine stays snappy, responsive and won’t get too over-loaded by jobs on your VMs!
#!/bin/bash
ps -xo pid,command | grep -v grep | grep startvm | while read line ;
do
procID=`echo $line | awk '{print $1}'`
sudo renice +10 -p $procID
done
The above code works on a mac; although I haven’t tested it, I recon to get it running on Linux you need to update the PS command, by swapping the x for an e… like this….
#!/bin/bash
ps -eo pid,command | grep -v grep | grep startvm | while read line ;
do
procID=`echo $line | awk '{print $1}'`
sudo renice +10 -p $procID
done
Have fun, suggestions and improvements welcome.
So it turns out my 5 minute vpn doesn’t work in EC2 because the ESP/AH protocols (50 and 51) are blocked on the AWS network.
This is no big deal tho, as NAT-T allows one to tunnel IPSEC over UDP… however getting it to work on CentOS required a bit of a hack.
If you have already tried setting up an IPSEC vpn, shut it down with ifdown ipsec1 and remove your /etc/racoon/192.168.56.101.conf (or whatever IP yours is).
To start the hack on BOTH boxes, you need to edit /etc/sysconfig/network-scripts/ifup-ipsec. Around line 215 you need to insert nat_traversal force;… like this….
BEFORE:
case "$IKE_METHOD" in
PSK)
cat >> /etc/racoon/$DST.conf << EOF
my_identifier address;
proposal {
encryption_algorithm $IKE_ENC;
hash_algorithm $IKE_AUTH;
authentication_method pre_shared_key;
dh_group $IKE_DHGROUP;
}
}
AFTER:
case "$IKE_METHOD" in
PSK)
cat >> /etc/racoon/$DST.conf << EOF
my_identifier address;
nat_traversal force;
proposal {
encryption_algorithm $IKE_ENC;
hash_algorithm $IKE_AUTH;
authentication_method pre_shared_key;
dh_group $IKE_DHGROUP;
}
}
Again, on both boxes update your /etc/sysconfig/network-scripts/ifcfg-ipsec1 files so that AH is disabled… because AH doesn’t like NAT… like this….
[root@CentOS2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ipsec1 DST=192.168.56.101 TYPE=IPSEC ONBOOT=yes IKE_METHOD=PSK AH_PROTO=none [root@CentOS2 ~]#
On your iptables policy make sure that UDP 500 and UDP 4500 are permitted and volia.
# tcpdump -n -i eth1 port not 22 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 20:26:49.257590 IP 192.168.56.101.ipsec-nat-t > 192.168.56.102.ipsec-nat-t: UDP-encap: ESP(spi=0x08de7c32,seq=0xa), length 116 20:26:49.261076 IP 192.168.56.102.ipsec-nat-t > 192.168.56.101.ipsec-nat-t: UDP-encap: ESP(spi=0x03787bd0,seq=0xa), length 116 20:26:50.260942 IP 192.168.56.101.ipsec-nat-t > 192.168.56.102.ipsec-nat-t: UDP-encap: ESP(spi=0x08de7c32,seq=0xb), length 116 20:26:50.262939 IP 192.168.56.102.ipsec-nat-t > 192.168.56.101.ipsec-nat-t: UDP-encap: ESP(spi=0x03787bd0,seq=0xb), length 116 20:26:51.261298 IP 192.168.56.101.ipsec-nat-t > 192.168.56.102.ipsec-nat-t: UDP-encap: ESP(spi=0x08de7c32,seq=0xc), length 116 20:26:51.264974 IP 192.168.56.102.ipsec-nat-t > 192.168.56.101.ipsec-nat-t: UDP-encap: ESP(spi=0x03787bd0,seq=0xc), length 116 20:26:52.262289 IP 192.168.56.101.ipsec-nat-t > 192.168.56.102.ipsec-nat-t: UDP-encap: ESP(spi=0x08de7c32,seq=0xd), length 116 20:26:52.265488 IP 192.168.56.102.ipsec-nat-t > 192.168.56.101.ipsec-nat-t: UDP-encap: ESP(spi=0x03787bd0,seq=0xd), length 116 20:26:53.264008 IP 192.168.56.101.ipsec-nat-t > 192.168.56.102.ipsec-nat-t: UDP-encap: ESP(spi=0x08de7c32,seq=0xe), length 116 20:26:53.267003 IP 192.168.56.102.ipsec-nat-t > 192.168.56.101.ipsec-nat-t: UDP-encap: ESP(spi=0x03787bd0,seq=0xe), length 116 20:26:54.265655 IP 192.168.56.101.ipsec-nat-t > 192.168.56.102.ipsec-nat-t: UDP-encap: ESP(spi=0x08de7c32,seq=0xf), length 116 20:26:54.267264 IP 192.168.56.102.ipsec-nat-t > 192.168.56.101.ipsec-nat-t: UDP-encap: ESP(spi=0x03787bd0,seq=0xf), length 116 20:26:55.267459 IP 192.168.56.101.ipsec-nat-t > 192.168.56.102.ipsec-nat-t: UDP-encap: ESP(spi=0x08de7c32,seq=0x10), length 116 20:26:55.269678 IP 192.168.56.102.ipsec-nat-t > 192.168.56.101.ipsec-nat-t: UDP-encap: ESP(spi=0x03787bd0,seq=0x10), length 116 14 packets captured 14 packets received by filter 0 packets dropped by kernel #
IPSEC VPN Tunnelling over UDP…. done!
OSSEC is my favourite linux HIDS however now that I’m running a SELINUX secured web server I noticed that my active responses were not working after a reboot.
After enabling SELINUX, I started getting alerts about the following problem in my messages file….
Nov 11 12:16:22 amy kernel: type=1400 audit(1289477782.569:8): avc: denied { read write } for pid=2551 comm="iptables" path="socket:[5261]" dev=sockfs ino=5261 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=unix_dgram_socket
This appears to be ossec trying to update iptables, but failing as they’re in different contexts… now I’m no selinx expert but this CentOS Wiki Page helped… run the following command which will create osseciptables.pp in the current directtory…
root@amy# grep iptable /var/log/messages | audit2allow -M osseciptables
This creates a new binary module that can be installed with….
/usr/sbin/semodule -i osseciptables.pp
You can view current selinux modules with …
/usr/sbin/semodule -l
If you want to see what is being created by audit to allow, try the following…
root@amy# grep iptable /var/log/messages | audit2allow -m osseciptables
module osseciptables 1.0;
require {
type iptables_t;
type initrc_t;
class unix_dgram_socket { read write };
}
#============= iptables_t ==============
allow iptables_t initrc_t:unix_dgram_socket { read write };
root@amy#
I hope this helps some future googler!
Browsing the Fedora 14 release notes and I notice that the project will be creating amazon machine images (AMI’s) going forward… now that’s tempting!
See: Fedora 14 Currently Supported EC2 Images
The only issue currently stopping me is that Fedora boxes need updates all the time, perhaps it wouldn’t be so bad with a minimal image!
Whilst learning about Amazon Web Services I noticed that there wasn’t a clean bare-bones version of my favourite server linux – CentOS – to use.
There are various public images available but they all have stuff in there I don’t want!
I have built a 1Gb image of CentOS with the minimum base feature-set… i.e. only the packages you get from typing…
yum groupinstall base
Since I’m not American I can’t sell this using Amazons DevPay program so I’m offering it here… since no-one replied to this post I figure I’m allowed!
I have a CentOS filesystem file (which you can mount via the loopback filesystem) which can be booted within EC2.
To use the file as a private AMI three further steps are required…
Each of these are commands from the AWS tools; all of which I’m happy to do for someone but they would need to handover some secret AWS credentials (it’s your whether you’re comfortable with that or not!).
If you’re interested contact me, I was thinking about £10 ($10->$15USD depending on the exchange rate) was a fair price… obviously you’d paying me for the my time, not the linux or CentOS distribution as they’re free and opensource
I wonder if anyone else finds this a bit irritating…
This new “spatial” user interface presents just one window for each folder, and remembers their location and size.
Now…
Nautilus features a number of user interface changes including a new split view mode and is now set to browser mode by default, replacing spatial mode.
Is this progression or development in circles?
I’ve recently upgraded from F7 to F10 and was surprised to see that ffmpegthumbnailer hasn’t made it into the yum repo’s yet!
As I use ffmpegthumbnailer for PS3 Video thumbnails I needed to find a solution to this.
If I had the time I’d knock up a .spec file & rpm, but for the time being I thought I’d post my bodge…. NUMBER ONE, if you’ve never typed “configure;make;make install” before you need to install the development tools.
sudo yum groupinstall “development tools”
Next pick up a couple of extra dependencies…
sudo yum install libpng libpng-devel ffmpeg ffmpeg-devel
You should now find that, if you download the source, unpack it – gunzip ffmpegthumbnailer-1.5.0.tar.gz;tar -xvf ffmpegthumbnailer-1.5.0.tar and cd ffmpegthumbnailer-1.5.0 hopefully you can complete ./configure without any errors.
If you try make at this point it’ll probably fail with errors, this is how I fixed that….
cd /usr/include/ffmpeg/
ln -s libavcodec/avcodec.h ./
ln -s libavformat/avformat.h ./
ln -s libswscale/swscale.h ./
ln -s libavformat/avio.h ./
Now you can… get back to the build
cd ~/ffmpegthumbnailer-1.5.0
make
sudo make install
If all has gone well you should have a working install