Cisco.com & Wget

Ever since cisco updated their website (you know like, a year ago), I’ve struggled to find a way to wget software onto a box.

This week, I found a bodge using firefox. Sign into cisco.com and go through the normal process, accepting agreements and begin the download.

Once the download starts, pause it… right click and copy the download link:

Copy link from downloads

Copy link from downloads

Then from your terminal/linux box, you can paste the url into wget:

wget -O ise-1.1.3.124.i386.iso "https://secure-us.esd.cisco.com/files/swc/sec/4_SDSP_59/bah/bah/ha?uid=linickxdotcom&key=go"

- Quick gotcha alert, the link you paste must be in speachmarks/double quotes or the full url will not paste correctly.

ipv6 ready

I’ve been testing the AAAA records for linickx.com, I’m supprised how easy it was!

The cloud servers over at rackspace (where this is hosted) come ipv6 ready – i.e. with a native ipv6 address attached to the internet. I’m using CentOS 6 so your experience may vary :)

ifconfig showed the IP address in my network stack, a good start! The first thing is that there are two iptables config files /etc/sysconfig/iptables & /etc/sysconfig/ip6tables, it should be obvious which is which! I’m not sure I agree with Redhat’s decision to have two config files but they are configured the same it is only the IPs that need to be different, therefore I can have the same statement in each to open up access to apache:

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT

By default --dport 22 (SSH) is open on a RHEL/CentOS box, therefore I recommend you change that as appropriate – yes you need to think about ACLs on IPv6 as well as IPv4!

Since there are two config files for iptables there are also two services, you will need to restart the service to load up a new config:

[nick@toad ~]$sudo service ip6tables restart

The apache config file is /etc/httpd/conf/httpd.conf, by default there is a Listen 80 statement, assuming you haven’t changed that apache should be already listening on ipv6 port 80!

[nick@toad ~]$ sudo netstat -nap --inet6 | grep 80
tcp        0      0 :::80                       :::*                        LISTEN      6878/httpd          
[nick@toad ~]$

What happens now when you hit up your server will happen depend now on your config, check you VirtualHost statement, if you have an IPv4 address you want to remove it, so you have something like:

<VirtualHost *:80>
  ServerName localhost
  ServerAdmin root@localhost
  DocumentRoot /var/www/html
</VirtualHost>

If you use a NameVirtualHost *:80 then your ServerName statements will start working when you published AAAA records.

To publish an AAAA record use whatever method you use for ipv4 A records, they’re the same thing :) … I used my rsdns util … kick of a test and watch your access log, all things being equal you should see ipv6 source addresses!

…OK, I have assumed you have an ipv6 source to test with; the quickest method is http://ipv6-test.com/validate.php, the more involved method is to setup a 6in4 tunnel with tunnelbroker.net.

If it all works you can get yourself a cool badge like this one!

ipv6 ready

OpenShift – PHP, APC & hotdeploy

If you are using apc caching on openshift you may have noticed that your cache gets binned each time you git push, the reason for this is that the push restarts your gear’s apache process. To keep the same process, thus keep the cache you can enable hot deployment.

$ touch .openshift/markers/hot_deploy
$ git add .openshift/markers/hot_deploy
$ git commit -m “enabling hot_deploy”

Now you can push as many times as you like without invalidating the cache… of course if you now need to clear the cache or restart the process you have to do it manually with

$rhc app restart -a myapp

HTH!

OpenShift – Backup and Restore

The free plan over at openshift only allows 3 “gears” which you know is a bit of a problem if you have more than 3 apps to test… don’t sweat it tho, simply backup your gear, delete it and re-use the free slot for something else … if you need the gear back as long as you re-create the app with the same name you can restore the snapshot..

To backup your application code, data, logs and configuration, you run:

$rhc app snapshot save -a {appName}

To restore your application, you run:

$rhc app snapshot restore -a {appName} -f {/path/to/snapshot/appName.tar.gz}

REF: KB-E1047

Enjoy!

apc.php for rhel / centos

Note to self: The apc.php (script for monitroing apc performance) is stored in – /usr/share/doc/php-pecl-apc-3.1.3p1

Speed, I am Speed

Recently I’ve been focusing on getting linickx.com not only available, but snappy too!

  1. HA Proxy distributes the load across two cloud servers
  2. Varnish Caches have a hit rate of between 30% & 49%
  3. WP Super Cache serves up static files to Varnish
  4. HypderDB distributes MySQL Requests across my two servers

My next step is to get some minification going, either with page-speed or a WP-Plugin.. they seem to be a bit tricky to troubleshoot tho!

If you are looking to measure your websites performance, I’ve been using pingdom for server response times and webpagetest.org for gathering client load speeds.

Hacking Cisco ISE UDI

ISE Virtual Machine that thinks it a CAM
The back story… you’ve deployed your ISE appliance and the world is great! Your management need you to make a change “right now” but that virtual machine in the lab you have been using for testing is 91 days old and the eval no longer works. You raise a case to get budget and a PO over to Cisco for a lab ISE appliance or license but this change is critical, if only there was a way to use your appliance license on your VM?

Perhaps you should log into your ISE appliance and make a note of the Product Identifier (PID), Version Identifier (VID) and the Serial Number (SN).

What you might want to do now is shutdown your ISE VM and mount the disk… I always have a CentOS server kicking around for this kind of thing, so if I was to do this, I would mount the ISE virtual disk as an extra disk that CentOS has access to.

From within CentOS you can use fdisk -l to view the hard drive partitions… When you’re hacking a VM you mount as many of the ISE partitions as you can (some will fail) to see what’s there. On my test machine /dev/sdb7 was the partition of interest as it had an /opt directory (cisco always install stuff in opt).

Inside mount-point/opt/system/bin/ you might find a file called cars_udi_util, that’s the puppy that the license is bound to.

What you might want to do is rename that file and replace it with something that always gives the “right” answer. Attached is cars_udi_util.txt, a shell script I have been testing, edit the top of the file and insert the PID/VID/SN you found earlier.

Now save the cars_udi_util.txt to mount-point/opt/system/bin/cars_udi_util, that’s right remember to remove the .txt!

Unmount the disk, shutdown Centos and boot up ISE.

Now I’ve been hacking my machine and after this change the services wouldn’t start (show application status ise) to fix that I ran application reset-config ise from the ISE CLI Shell, rebooted and Voila! …The machine booted up with a blank default config.

After changing the default admin password (from cisco) it would now be possible for you to use your proper appliance license on your VM… of course this is only a temporary thing and I fully expect & recommend you undo these changes as soon as your new license arrives from Cisco.

Happy Hacking!

OCFS2 issues

This morning I’ve had issues with my linickx.com cluster, the file system on both nodes went to read-only and I ended up in a world of pain.

[root@georgia ~]# sudo /etc/init.d/httpd start
Starting httpd: 
[root@georgia ~]# tail -f /var/log/messages
Jan  9 09:48:35 georgia kernel: [  474.259265] (httpd,1712,0):ocfs2_reserve_clusters_with_limit:1190 ERROR: status = -22
Jan  9 09:48:35 georgia kernel: [  474.259271] (httpd,1712,0):ocfs2_lock_allocators:2546 ERROR: status = -22
Jan  9 09:48:35 georgia kernel: [  474.259276] (httpd,1712,0):ocfs2_write_begin_nolock:1732 ERROR: status = -22
Jan  9 09:48:35 georgia kernel: [  474.259282] (httpd,1712,0):ocfs2_write_begin:1856 ERROR: status = -22
Jan  9 09:49:31 georgia kernel: [  530.660071] o2net: no longer connected to node amy (num 1) at 10.176.128.7:7777
Jan  9 09:49:31 georgia kernel: [  530.661856] ocfs2: Unmounting device (147,0) on (node 2)
Jan  9 09:59:46 georgia kernel: [ 1145.772174] o2dlm: Nodes in domain E9447DBE28154DAEA1B988CEC573EB64: 2 
Jan  9 10:01:05 georgia kernel: [ 1223.911192] o2net: connected to node amy (num 1) at 10.176.128.7:7777
Jan  9 10:01:09 georgia kernel: [ 1227.933348] o2dlm: Nodes in domain E9447DBE28154DAEA1B988CEC573EB64: 1 2 
Jan  9 10:01:09 georgia kernel: [ 1227.938693] ocfs2: Mounting device (147,0) on (node 2, slot 1) with ordered data mode.
Jan  9 10:02:35 georgia kernel: [ 1314.467741] OCFS2: ERROR (device drbd0): ocfs2_validate_gd_self: Group descriptor #419328 has bit count 32256 but claims that 45941 are free
Jan  9 10:02:35 georgia kernel: [ 1314.467754] File system is now read-only due to the potential of on-disk corruption. Please run fsck.ocfs2 once the file system is unmounted.
Jan  9 10:02:35 georgia kernel: [ 1314.467764] (httpd,2389,0):ocfs2_search_chain:1729 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467771] (httpd,2389,0):ocfs2_claim_suballoc_bits:1902 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467778] (httpd,2389,0):__ocfs2_claim_clusters:2185 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467783] (httpd,2389,0):ocfs2_local_alloc_new_window:1204 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467790] (httpd,2389,0):ocfs2_local_alloc_slide_window:1306 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467798] (httpd,2389,0):ocfs2_reserve_local_alloc_bits:695 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467803] (httpd,2389,0):ocfs2_reserve_clusters_with_limit:1190 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467809] (httpd,2389,0):ocfs2_lock_allocators:2546 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467814] (httpd,2389,0):ocfs2_write_begin_nolock:1732 ERROR: status = -22
Jan  9 10:02:35 georgia kernel: [ 1314.467821] (httpd,2389,0):ocfs2_write_begin:1856 ERROR: status = -22
Jan  9 10:02:36 georgia kernel: [ 1315.046965] OCFS2: ERROR (device drbd0): ocfs2_validate_gd_self: Group descriptor #419328 has bit count 32256 but claims that 45941 are free
^C
[root@georgia ~]#

What made this odd is that running fsck.ocfs2 as suggested made no difference, as the output said that the disk was clean.

[root@georgia ~]# fsck.ocfs2 /dev/drbd0
fsck.ocfs2 1.4.4
Checking OCFS2 filesystem in /dev/drbd0:
  Label:              linickxcluster
  UUID:               E9447DBE28154DAEA1B988CEC573EB64
  Number of blocks:   1048535
  Block size:         4096
  Number of clusters: 1048535
  Cluster size:       4096
  Number of slots:    4

/dev/drbd0 is clean.  It will be checked after 20 additional mounts.
[root@georgia ~]#

I learn that in fact the above output was a lie! For any future googlers seeing the same issue, run:

fsck.ocfs2 -fy /dev/drbd0

The f & y force a check and fix any found issues, the force on my filesystem found the errors and we appear to be back online :)

PHP to solve problems

PHP make you think of web app’s right? … well, did you know you can run it from the CLI to?

Recently I’ve been doing a lot of spreadsheet and sub-netting type stuff, whilst doing this mundane work I’ve realised that I can get scripts to work for me. I’ve started to post a few PHP network functions to github which I’ve been using.

Here’s an example: I have a nokia firewall, the routing table in voyager is shown in the following format -

Network / CIDR Mask , Gateway
10.0.0.0 / 8 , 10.0.0.1

I need that same routing in a Cisco ASA format -

Network , Mask , Gateway
10.0.0.0, 255.0.0.0, 10.0.0.1

Now this is not a problem for a few routes but the firewall I’m looking at now has 177 static routes, which I don’t want to convert manually.

Roll in PHP!
Save the original routing table as a .csv file. ColA = ip/mask , ColB = gateway.

Save this a route_conv.php

<?php

	/**

		Change the below to your CSV File.

	**/

	$firewall_csv = "./routes_cdr.csv"; 

	/**

		Function to convert CIDRs such as "23" to decimall dotted like "255.255.254.0"
		I've got more of these: https://gist.github.com/1309388

	**/

	function cidr2mask($netmask) {

		$netmask_result="";
		for($i=1; $i <= $netmask; $i++) {
		  $netmask_result .= "1";
		}

		for($i=$netmask+1; $i <= 32; $i++) {
		    $netmask_result .= "0";
		}

		$netmask_ip_binary_array = str_split( $netmask_result, 8 );

		$netmask_ip_decimal_array = array();
		foreach( $netmask_ip_binary_array as $k => $v ){
		    $netmask_ip_decimal_array[$k] = bindec( $v ); // "100" => 4
		}

		$subnet = join( ".", $netmask_ip_decimal_array );

		return $subnet;

	}

	ini_set('auto_detect_line_endings', true); // detect CR

	if (file_exists($firewall_csv)) {

		$file = fopen($firewall_csv, 'r');

		$counter = 0; // array counter

		while (($data = fgetcsv($file)) !== FALSE) {
			
			list($ip, $netmask) = split( "/", $data[0] ); // SPLIT Col A into IP & Mask

			$netmask = cidr2mask($netmask); // Covert Mask

			$gateway = $data[1]; // Col B

			/**
				This echo is the CSV style output, but you could change this to echo "route add $ip $mask $ gateway \n" for unix style output.
			**/

			echo $ip . "," . $netmask . "," . $gateway . "\n";

		}

		fclose($file);

	} else {
		
		echo "404: $firewall_csv \n"; // FILE NOT FOUND.

	}
?>

from your CLI run “php route_conv.php” and enjoy the output!

Building a free Dynamic DNS client with rackspace Cloud


As a cloud server customer you get access to rackspace’s free DNS service.

When I fist saw this product I had an instance light-bulb moment, I could stop paying for a dynamic DNS service and build my own private one. As a broadband (DHCP) user I have a very basic requirement of needing to regularly update an A record so that I can find my pc :)

To bring my idea into fruition I began researching; I need a cli tool which I could run from cron on my linux box (to send the DNS update requests to rackspace). In my research I found rscurl, a cli tool to control cloud servers, as rackspace have a standard API for all their products I have been able to use rscurl to develop rsdns.

rsdns is a series of cli tools to adding/deleting/changing rackspace DNS records, as part of the tool development I have created a script called rsdns-dc.sh to run on my machine, below is a short how to:

How to get free dynamic dns from rackspace.

Continue reading

F5 BigIP LTM VE works in Virtual Box

Something I discovered ages ago (so long ago that my trial license expired) but forgot to post is that you can get an LTM VE to work in Virtual Box.

To get started download the ESX image from the F5 VE Trial Page, when you get the download import the OVA into virtualbox.

The only thing I needed to tweak after the import was the interface settings, you need two intels and a PCNet, the PCNet is the management interface. Set the PCNet to host only networking, give your laptop/pc an ip address on the host only network a 192.168.1 address and you’re good to go!

You may experience HIGH CPU issues after boot, but since these boxes are based on linux, you can use the divider=10 centos trick.

Enjoy your virtual load balancing!