There are bucket load of posts on how to use google application engine (GAE) as a CDN but many of them direct you to hosting static content on a google server. For me that approach isn’t practical, every time I did a WordPress update or plugin upgrade I would have to push an update to GAE… annoying!

Origin-Pull is the future then, basically the server acting as a CDN pulls a copy of the original, caches it and serves that to clients. Updates on the main site are easy, just wait for the CDN to age out it’s cache or if you are impatient manually purge.

Over the weekend I stumpbled upon SymPullCDN a GAE app, it’s a bit out of date so I’ve pushed a newer version to github. I’ve made two changes, firstly updated to python2.7 (as per google’s recommendation) and secondly I’ve added a cron job to keep your GAE app snappy

Setting up your own copy is simple, start by signing up for GAE and create a new “application”, mine’s called mygaecdn…

Next get a copy of the Google App Engine SDK for Python also known as GoogleAppEngineLauncher… Install it

Once it’s running, create a new application… give it the same name as the app you created on google.

Take a note of the directory in which the application is being created, mine is Users/nick/Documents/GoogleAppEngine/mygaecdn

Next download this zip file which has the updated SymPullCDN files.

Delete everything from your Users/nick/Documents/GoogleAppEngine/mygaecdn and place in there the contents of SymPullCDN-master.zip

Open app.yaml in a text-editor and edit line 1 to replace *replace*me* with your application name, e.g.:

application: mygaecdn

Next open main.py in a text-editor and edit line 21 and replace http://replace*me/ with your website, e.g.:

origin = "http://www.linickx.com/"

Make sure you save both files and you are done!

Now, test locally in the GoogleAppEngineLauncher app before deploying to google. Click the green “play” and a GAE application will run on your local machine; from the screenshot above you can see mine is listening on “port 10080″, so I can open a web browser to http://localhost:10080 – all things being equal you will see a copy of your website

If that works you’re ready to deploy…. hit the blue “deploy” button to push you app up to google. When that’s finished you should be able to visit mygaecdn.appspot.com… obviously yours isn’t called mygaecdn!

Once the deploy is finished you have a GAE ready and willing to serve cached copies of your site.

What you do next will depend on your website. Me, I use WordPress and wp-super-cache, so I can simply enable the CDN feature in that, e.g.:

You might have to install something, or change some URLs, whatever you do, just remember to only change links to static content such as CSS, JS or IMG – anything dynamic is likely to end in a world of pain.

FOOTNOTE: The term CDN is used loosely in this blog post, GAE is more of a content off-load, IMHO a CDN should server you geographically-local content but in all my tests on webpagetest showed all my content coming from Google-USA, not that is really a problem as their servers are still rocket-quick

mod_security and WordPress with Commands in Permalinks (urls)

For a long while now, one of my oldest posts (nagios ping tool) returned a 403 error and I couldn’t work out why… a recent post about curl also fell foul of the same issue so I’ve been forced to work out why

The main challenge that I faced was that I could not find any errors in my logs, apache’s error_logs were empty, varnish is not catching the error page and my mod_security debuglog didn’t show anything. Now there is clearly a 2do here, I need to look into my logging issues because the issue was mod_security!

modsecurity_crs_40_generic_attacks as a list of system rules which will deny access to commands, on my system ping & traceroute are indeed commands! Looking thru _crs_40 I can see that rule ID 950907 blocks curl, therefore I can create a simple location match to permit access to that page.

<LocationMatch "^/3659/my-lifestream-php-curl-ca-certificate-issues">
    SecRuleRemoveById 950907
</LocationMatch>

Traceroute and Ping are IDs 958837 & 958893 respectively. Going forward I could simply remove those IDs globally, but to be honest I don’t want to, I feel comfort with the restriction they begin… I will just have to be more careful with the titles I use on pages.

Test Facebook post from linickx.com (WordPress)

A while back facebook disabled RSS importing of my linickx.com posts which has resulted in my facebook timeline being very empty.

It’s taken them a while, but facebook appear to have got with the program and released an official wordpress plugin… nice work guys!

This is a test post to see if I have it setup correctly… assuming it works my friends will starting seeing a bit more linickx’y things

My Lifestream – PHP, Curl, CA & Certificate Issues

I use my own lifestream plugin to feed my blog page with activity from other sites.

A (long) while ago HTTPS feeds stopped working, basically due to certificate trust issues.

Now the lifestream plugin uses WordPress built in functions to perform HTTP requests…. and this was not something I wanted to change. It’s taken a bit of exploring to work out why the default configuration did not work… and then find the best way to patch it.

Eventually I found out that you can set the PHP_CURL CA Path in php.ini using the curl.cainfo directive. On CentOS the “well known” CAs are stored in /etc/pki/tls/certs/ therefore the fix was quite simple, add the following to /etc/php.ini

curl.cainfo = /etc/pki/tls/certs/

Job done! Now HTTPS only site (like github) lifestream nicely!

is_blog

Since moving to a static front page I’ve noticed google is indexing /blog rather than individual posts…. this little addition to my header.php should fix it!

<?php
/*
                DO NOT INDEX /blog
        */

        if ( (!is_front_page()) && (!is_page()) && (!is_single()) && (!is_archive()) ) {
                echo '<meta name="robots" content="noindex">' . "\n";
        }
?>

You may notice that WordPress doesn’t appear to be an is_blog function which would do the same thing.

LINICKX LifeStream Version 0.3

It’s been a very long time since I’ve needed to look at this plugin but since linickx.com has moved to twentyeleven I wanted to take advantage of post formats; this has motivated me to make some minor updates.

New Fetures

Post Format Support
Feeds can be assigned post formats, Standard is the default format if you theme doesn’t support this, twentlyelevn does and I think the link format works quite well.
Fail without Bail
A bug I’d just lived with was if the feed was corrupt then the plugin would simply crash and burn, sometimes a factory reset was required to recover. Now the plugin will detect the error, dump the message and carry on… much better!
Admin Page Updates
I’ve been slowly adding context help to my plugins, you know the little help in the top right hand corner, so help is now even closer. The news feed at the bottom will now take advantage of the fail-without-bail feature for when linickx.com is down. The final change here is a dontate feature, there is a tick box to remove this

All the old features still exist and the same old information applies…

Enjoy!

phpbb_recent_topics version 0.7

A little later than planned but the annual release of phpbb_recent_topics is finally here!

I actually started on this release a couple of months ago but couldn’t find the time to write this post and tag the release :-$ The good news is that there have been lots of changes since 0.6, basically I’ve implemented as many bug fixes and feature as I can in the WordPress-Plugin bug tracker.

If you have a feature request or patch, please submit here.

Support questions and requests for help should be directed at the forum.

Without further ado… What’s New?

WordPress 3.1.4 Testing
Yep, with such a slow development cycle I can confirm that PRT works with WordPress version 3, but you knew that right ?
CSS Styleable output
A common request, there is now a UL class called prt_ul and a LI class called prt_li which can be used to make you sidebar look pretty!
Plugin Options Modernisation
I follow planet.wordpress so any tips, tricks and modern ways of doing things have been implemented i.e.
- Collapse multiple options into an array
- Implement register settings to close http://plugins.trac.wordpress.org/ticket/1217
- Implement upgrade.php to migrate all variable to new array form
Resolved: Missing argument 1 for phpbb_topics()
This was a stoooopid bug that has been in the code for ages, I finally found the time to squish it!
Code clean-up (Lots of changes)
I’ve put a lot of effort in 0.7 to remove all the un-necessary code, the kind of stuff that get’s pasted in as a good idea but not needed, hopefully this has some positive impact in your site as well as my kama
Timezone/Off-Set fix (Reads +/- hours from WordPress Settings)
This was perhaps the most requested feature, I figured out a neat way of doing it by reading the WordPress options.
Localisation of date/time fix (http://plugins.trac.wordpress.org/ticket/1173)
I’m planning to complete localisation in 0.8, but this is a start right?
Callback functionality
Do you want to do something cool with PRT? Check out phpbb_topics_callback in the readme!
Contextual Help in the admin dashboard
In the past, this blog post has contained the instructions for getting going, I’ve now put everything either in the readme or actually in the wordpress help, keep a weather eye out for the “help” in the top right hand corner of the WordPress dashboard.

Where do I get it?
Download phpbb_recent_topics version 0.7 from WordPress.org

I hope you enjoy this release!

Footnote: Version 0.7.1 has been pushed out which fixes the date display issue reported below!

linickx on github

For your social coding pleasure, linickx code is now in github!

Yesterday I completed the subversion mirror of my WordPress projects – phpbb_recent_topics , root cookie and linickx lifestream – now this isn’t a migration, it is a mirror! For the time being I’m happy using the subversion tools provided by Automattic and the WordPress team but I understand that git is gaining momentum and many are switching, basically I’m hoping this makes it easier for the WP community to get in touch or make suggestions to the code.

I’m also working on uploading some of my old work, I rely heavily on google to broadcast my wares and perhaps there are some old dinosaurs that need resurrecting by a new community of devs? Well if you’re feeling nosey “A is for Abandonware”

It’s likely that new code and projects will appear on git hub, I’ve been toying with running subversion on linickx.com but now that Xcode4 has git built in this cloud based social service might be a better option…. I guess only time will tell!

Enjoy!

WordPress Custom Query for Custom Post Types and Taxonomy

I wanted to run a custom query against WP3.0 custom post types but all the documentation and google I found all pointed to posts in categories which doesn’t work if your post type isn’t post, this was my solution….

$my_query = $wp_query-&gt;query; // Copy the existing query into a new one
$my_query['posts_per_page'] = "30"; // change the number we want displayed.
$my_query['orderby'] = "title"; // Sort by title.
$my_query['order'] = "ASC"; // 'A' first!
query_posts($my_query);  // Run our query.... normal service resumes.

Hopefully this post will give someone the light bulb moment they’re after.

Xcode for WordPress Plug-in Developers

This post is an extension to this excellent article except with a subtle difference, I’m not interested in hacking the WP core, I’m writing a plug-in

I’m going to assume that you’ve requested access to WordPress.org’s SVN repository, and you’ve found the admin panel and want to get your Xcode goodness hooked up to the WP SVN.

To get started create a new Empty Xcode project.


Xcode project chooser	*Empty Project*

From the menu bar select SCM then Configure Repositories, the window select the + to add a new repo, the username & password pair are your wordpress.org credentials, you can find the URL to your SVN repo on the wordpress.org site.

The WP.org admin interface

Next from the menu bar select SCM then Repositories, highlight the repo you’ve just created and choose checkout; you’re going to be asked for a folder to “save” the contents of the repo to, I chose inside my Xcode project folder.


Repo’ Config	*Code Checkout*

You now need to add those “checked out” files to you project, in the tree on the left, right/control click the blue project icon at the top and select add existing files, accept the defaults and the repo should be imported.

Imported Code

These files now in the project need to be associated with the repo, this is done by highlighting the blue project icon and clicking the “Info” button, then click “Configure Roots & SCM”; click the + button and choose your repo folder (again)… the dropdown on the right should automatically select the correct SCM.


*Project Properties*	*Project Roots*

If you now change a file / add / delete / etc it will appear under the SCM tree, you can right/control click and commit to the repo…. job done

Yay it works!

Very quickly you’re going to find out that you want to test your changes prior to commiting them, to do this you may want to copy your files upto a webserver…. I get Xcode to do that for me, using macfuse to mount my webroot via ssh but for this example I’ll create a directory on the desktop … you also need to create any subdirectories you may need.. I have to create “admin”.

To copy the files we’re use an Xcode target, from the menu bar choose Project then New Target then Copy Files Target, I called my new target CopyFile. Using the info button change the absolute path of the copy – e.g. /Users/Nick/Desktop/LL … now here I ran into a problem (suggestions welcome)… next I created extra copy build phases for each subdirectory. Control/right click your Target ( i.e CopyFiles) then choose Add, New Build Phase, New Copy Files Build Phase… I then set the absolute path of this “sub target” to /Users/nick/Desktop/LL/admin/ to represent my admin subdirectory.

Finally DRAG down the files into their correct target, all being well and a fair wind clicking “BUILD” will cause your files to be copied into your directory. You can now make changes, click build to test, and then commit them to subversion when you’re happy

Finished!

The red sweater post that inspires this now talks about creating a RUN shortcut to automatically open a web browser for testing, however mine didn’t work… I got this error “The program being debugged is not being run“… suggestions to fix welcome!

phpbb_recent_topics version 0.6

UPDATE: This Plugin has been updated, the below links to the new file, but please comment on the new post, thanks.

Screenshot phpbb_recent_topics admin page

Admin GUI

It’s that time again, enough things have happened in the support forum to warrant another version.

phpbb_recent_topics 0.6 owes a debt of gratitude to number3nl he has worked very hard to push in two main new features and his work was so good it only needed a couple of minor tweaks from myself.

New Features!

The loooong awaited phpBB recent posts feature has been enabled, simply tick the box that says “Sort results by post date” and you’re away!
Tooltips are here! Yup if you’re into that kind of thing enabling tool tips will show a snippet of the post content to your visitors prior to them clicking on any links.
Admin UI Error Messages are here; the GUI will now inform you if you’re connected to phpBB.

Bug Fixes.

The variables in display.php now match those of admin-display.php; hopefully this will make community contribution simpler.
Uninstall.php has been added so removal of the plugin is more comprehensive.

A note to upgrader’s.

If you’re using secure database connectivity you’ll need to GRANT access to the posts table….
GRANT SELECT ON phpbb_database.phpbb_posts TO wp_user@localhost;

Download phpBB_Recent_Topics_0.6 from wordpress.org

The usual stuff now follows….

The Installation

Before starting you need to get your database configuration sorted. So, are you going to connect to the PHPBB Database securely or Insecurely?

To connect Securely you need to GRANT access TO WordPress FROM phpbb.

How to GRANT wordpress read only access to phpBB ?
If you don’t know it already you need to find your wordpress mysql user id, it’ll be in wp-config.php

define('DB_USER', 'wp_user');     // Your MySQL username

and you should have already found your phpbb database & table for the above.
You need to type the following syntax into your mysql database

GRANT SELECT ON phpbb_database.phpbb_topics TO wp_user@localhost;

AND

GRANT SELECT ON phpbb_database.phpbb_forums TO wp_user@localhost;

AND

GRANT SELECT ON phpbb_database.phpbb_posts TO wp_user@localhost;

this can be achieved by logging into phpmyadmin as your phpbb user, selecting SQL and pasting the correct GRANT into the text box.

WordPress Installation

Unzip phpbb_recent_topics.tgz in your `/wp-content/plugins/` directory. (You’ll have a new directory, with this plugin in /wp-content/plugins/phpbb_recent_topics)
Activate the plugin through the ‘Plugins’ menu in WordPress
Configure the plugin, you need to tell wordpress about phpbb, this is done in the wordpress menu ‘Settings’ -> ‘phpBB Recent Topics’
The following Settings are required:
- * The name of your phpBB database (e.g phpbb)
- * The name of the table where topics are held (the default is phpbb_topics )
- * The full url of your forum for links (e.g. http://www.mydomain.com/forum)
- * The number of topics to show. (If left blank you get 5)
- * The Date Formatting, i.e. “d/M/y – g:i a” similar to the WordPress “General Settings”
The use of Insecure connectivity is optional – See Below
Tick the boxes of any Forums you don’t want this plugin to show posts from
Hit ‘Update Options”
To output the list of topics in a page or post…
- * create a new page/post, type {phpbb_recent_topics} , hit ‘Publish’ or ‘Create new page’
To output the list of topics in your theme sidebar using the widget…
- * click “design” in the dashboard
- * click “widgets”
- * next to phpBB Recent Topics click “add”
- * click “save changes”
To output the list of topics in your theme sidebar using a template tag…
- * edit sidebar.php and inside <div id=”sidebar”> type…
```
<?php
if (function_exists('phpbb_topics')) {
phpbb_topics();
}
?>
```

What is Insecure MySQL Connectivity & How do I use it?
If you need to you can connect in a less secure manner, the connection details held in your PHPBB config.php can be loaded into WordPress. This is insecure becuase the phpbb3 credentials are store in clear text, this is bad cuase if someone somehow comprised WordPress they’d have full access to PHPBB.

To use, tick the “Enable Insecure Database Connection” box, and submit, when the page re-freshes you’ll have some more boxes to populate, from your phpbb config.php fill in

$dbuser = phpbb MySQL Database UserName $dbpasswd = phpbb MySQL Database Password $dbhost = phpbb MySQL Server

Click update, and you should be connected!

Support
I’ve always been honest about support, there isn’t any. Basically I write this for my own needs and then publish it for you to use for free. You’re more than welcome to post comments here and if you need support the community phpbb-recent-topics forum is available, I do read every post there but we all have busy lives and I make no promises on how quickly I’ll reply to you.

Finally Subversion
The source for this plugin is in the wordpress subversion, you can checkout the trunk directly from here, if you weren’t aware wp.org also auto-magically creates a nightly build of any developments here.

I hope you enjoy my plug-in!

Dear Edgar, in response to your lifestream questions.

I’ve recently received this message thru my contact form, now it is either a very odd spam/phishing message or Edgar made a mistake on his email address as the domain doesn’t exist. Either way I’ve been asked QU 2 before so here’s my response while I update the FAQ

Dear Creator of Plugin, thank you very much for your great job. I am trying to use it in web site.
i am trying to use your plugin LINICKX LifeStream v0.2 and i have 2 questions.
1. I inserted 23 URL of Feeds but plugin loading only 3 of them, why?
2. Could I load also content of posts, if yes than how?
great thanks!
best regards
Edgar

Actually Edgar I’ve only tested with about 5 feeds, so I will load up my test site and see what happens, it could be possible that your feed 4 is corrupt and blocking the rest.

If you wish to create posts from feeds see FeedWordPress by Charles Johnson, I was planning to include this functionality but issues mean that now I’m not bothering, see line L236 of run.php, feel free to submit a patch if you can fix it.

Festive Regards,

LINICKX LifeStream V0.2

Screen-shot of config screen

UPDATE: This Plugin has been updated, the below links to the new file, but please comment on the new post, thanks.

Version 0.2 is the first feature update to my LINICKX Lifestream plug-in. This version incorporates all the minor bug fixes from version 0.1 -> 0.1.3 and includes what I call my “Troubleshooting Feature Pack”.

Version 0.1.x was plagued by a hidden multi-post issue, so not only is this fixed in V0.2, but I’ve included the tools used by myself to fix the issue. If you’ve been effected by this, use the factory reset option to clear out the lifestream plug-in, the next time it runs new posts will be created of all your feeds… so yes you will get double posts… but that will then be the last time it happens!

Plug-in Features:

Admin Feature Pack (NEW)
Verbose Cron Mode, now you can see what the plug-in is doing!
View the contents of the Lifestream Feed DB
View the URLS, Tags & Categories stored in the plugin
Factory Reset, remove all the settings without having to uninstall -> re-install
Stream Any Feed.
Most internet services like digg, twitter, lastfm and the like support RSS feeds of your activity, so you can stream any service even if I haven’t heard about it yet!
Posts are created from the feeds/stream.
Each Feed item creates a new post, you can tag & categorise your posts to give each stream a different look & feel in your WordPress templates.
No dependencies.
LINICKX LifeStream uses WordPress functions to do it’s work, so yes it’ll work on PHP4 & PHP5 and you don’t need to update simplepie or an-other dependancy.
Flexible updating.
By default LINICKX LifeStream should just work, WordPress will fetch the feeds every 5 minutes and store them in the WP Database, but you can change that. If you want to cron your updates every 1 minute you can, if you want the DB store in a file not in MySQL that’s not a problem either.

Bug Fixes

Version 0.2 Multi-Post issue – http://wordpress.org/support/topic/330243
Pushed Version 0.1.3 more bug fixes; 0.1.2 broke auto updates, so 0.1.3 fixes it again!
linickx.com was having execution timeout issues, so theres a patch to fix that for other users
I’ve also introduced a config.php so that upgrades do not overwrite $WPDIR set by users.
Pushed version 0.1.2 to fix error in dashboard reported by Markus .
Pushed version 0.1.1 as the directory path changed when wordpress.org created their .zip file… all should be working again.

Right now on LINICKX.com this plugin is streaming Digg, Google Reader and WordPress Commits… keep an eye for new things streamed such as delicious & last fm

I’m still looking for help.
If you think you can javascript/ajax the admin interface, or my plug-in WordPress MU compatible, please let me know.

Disclaimer
Plug-in development is a hobby, I have a full time job that takes priority. Feel free to leave comments on this post but ask support questions to the WordPress community. I will update this plugin as and when I need features to improve linickx.com and will consider any patches submitted including any good ideas you have.

If you use this plug-in, please consider donating as I give this code away for free and giving something back is good for your kama

Nick is on Google+

Tag Archives: WordPress

Using Google as a FREE origin pull CDN