Tiernans Comms Closet

CDN Hosted Blog: Part 2

admin — Fri, 06 Aug 2010 07:27:00 +0000

So, this morning Amazon has announced that your can set the Root Object of a CloudFront distribution. this now means that you can have an index.htm file (like the front page of this blog) hosted, and tell Amazon to host that when someone goes to the main page. and since you can use a CNAME to make the URL look like (like mine which is cdn.lotas-smartman.net) you can put your contents of your blog on CloudFront and build a fully hosted web site. no more redirects or Proxy servers, as i sugested in my previous post! Theory now would go as follows:

host your website somewhere (like, say, realblog.lotas-smartman.net). this could be a WordPress or any other type of blog for that matter.
have your CloudFront site pointing to blog.lotas-smartman.net
on post (or every few hours) check the WordPress DB (or what ever else your using) and post the new (or updated) HTML to the site.
It probably makes sense not to use WordPress to manage your comments… Disqus might be a better option since its externally hosted…

I now have something to work on this weekend…

A CDN Hosted Blog

admin — Sat, 03 Jul 2010 10:47:59 +0000

So, with the event of Amazon’s Cloud Front and RackSpace Cloud’s Cloud Files (with the backing of Limelight Networks) the world and its mother can now use cheap and cherfull CDNs for hosting images, CSS and JS files for their blogs. but what about the rest of the blog?

In theory, a blog is 99% static. With the help of the likes of Disqus, you dont even need to manage your own comments. so, other than updates to the core template (which is probably not very often) and new blog posts (which for me is currently not very often), the rest of the site is very static. So, why cant that be hosted in a CDN? I am using a plugin on my Wordpress install called W3 Total Cache, and one of the options is to generate the HTML for a page, save it to disk, use a URL Rewrite to point to that HTML file for a certin amount of time, and, in theory, take some load of the server… the first time the page is generated, it should run though the PHP CGI engine (this site runs on IIS7 now) and then be built into a HTML page and passed to IIS to serve. the second time, its just served from IIS from disk, as a standard HTML page… and this got me thinking…

How hard would it be to take that outputted HTML page and upload to Amazon S3 or RackSpace CloudFiles? using the same HTTP Redirect, just serve it from the CDN and bobs your uncle? Now, me not being a PHP dev, the answer chould be “Very”… but the question then comes up of, why cant it not be done anywhere else? this is, to some extent, how the image hosting site I built works. i upload images to a SQL DB, which gives me a nice URL to use. I open the URL, and if the file has not be uploaded to the CDN, it uploads and redirects me to the CDN location. the next request just gets redirected to the CDN.

So, the next question: is it posible to build a Proxy system that will proxy requests to and from the IIS box. request comes in, if its already on the CDN, redirect to the CDN. If not, show the main site, and in the background push the page to the CDN. you would need to do some magic around Search Engines (since they might get confused with CDN links and your main page) and also around logged in users (especially if you are using WordPress). anyway, its a theory… throwing it out there. main reason i was thinking about it is the fact that my server is hosted in house on a Cable Modem… so ping times can be a little slow… as can response times… ideally, this should make visiting this blog a little faster… but also any blog hosted by it… or at least, thats the theory…

[Update] added some code on GitHub with a demo of a HTML Module. some work needs to be done, but will add it in over time…

Out Sourcing V DIY

admin — Sat, 13 Mar 2010 20:33:18 +0000

So, a few days ago, i was talking with some colleagues about printers and DIY printing of your photos, and the consensus was pretty much Don’t DIY, outsource. and it started making me think about my current in-house systems, hosting, etc. so, this is the basis of what this post it about.

Current setup in house

i wont list everything i have in house at the moment, as i did this a few weeks back in the “Whats your setup

Microsoft Windows Phone 7 Series

admin — Mon, 15 Feb 2010 18:17:02 +0000

So, today at the Mobile World Congress, Microsoft unvailed the Windows Phone Series 7. This is the next generation of Windows Phone devices. So, some quick links and some screen shots from around the web:

There is a hands on Demo over at Channel 9 of the Windows Phone 7 Series. the OS is VERY SWEET! check it out.
the main website is at http://www.windowsphone7series.com/. videos with demos and info also available there
Another video on YouTube, with demos of some of the features.

Quick Blurb from the Press Release:

Windows Phone 7 Series creates an unrivaled set of integrated experiences on a phone through Windows Phone hubs. Hubs bring together related content from the Web, applications and services into a single view to simplify common tasks. Windows Phone 7 Series includes six hubs built on specific themes reflecting activities that matter most to people:

People. This hub delivers an engaging social experience by bringing together relevant content based on the person, including his or her live feeds from social networks and photos. It also provides a central place from which to post updates to Facebook and Windows Live in one step.

Pictures. This hub makes it easy to share pictures and video to a social network in one step. Windows Phone 7 Series also brings together a user’s photos by integrating with the Web and PC, making the phone the ideal place to view a person’s entire picture and video collection.

Games. This hub delivers the first and only official Xbox LIVE experience on a phone, including Xbox LIVE games, Spotlight feed and the ability to see a gamer’s avatar, Achievements and gamer profile. With more than 23 million active members around the world, Xbox LIVE unlocks a world of friends, games and entertainment on Xbox 360, and now also on Windows Phone 7 Series.

Music + Video. This hub creates an incredible media experience that brings the best of Zune, including content from a user’s PC, online music services and even a built-in FM radio into one simple place that is all about music and video. Users can turn their media experience into a social one with Zune Social on a PC and share their media recommendations with like-minded music lovers. The playback experience is rich and easy to navigate, and immerses the listener in the content.

Marketplace. This hub allows the user to easily discover and load the phone with certified applications and games.

Office. This hub brings the familiar experience of the world’s leading productivity software to the Windows Phone. With access to Office, OneNote and SharePoint Workspace all in one place, users can easily read, edit and share documents. With the additional power of Outlook Mobile, users stay productive and up to date while on the go.

Availability

Partners from around the world have committed to include Windows Phone 7 Series in their portfolio plans. They include mobile operators AT&T, Deutsche Telekom AG, Orange, SFR, Sprint, Telecom Italia, Telefónica, Telstra, T-Mobile USA, Verizon Wireless and Vodafone, and manufacturers Dell, Garmin-Asus, HTC Corp., HP, LG, Samsung, Sony Ericsson, Toshiba and Qualcomm Inc. The first phones will be available by holiday 2010. Customers who would like to receive additional information about Windows Phone 7 Series and be notified when it is available can register at http://www.windowsphone7series.com.

I am now really wanting one of these!!!

New Year, New Upgrades…

admin — Wed, 13 Jan 2010 18:10:37 +0000

So, we are 2 weeks into the new year (give or take a few hours as of writing this) and i am already starting to think about upgrades… some are more likely than others… but here are my thoughts for the new year and upgrades…

Phones: So, i have had my G1 for a while now (Got it on in the first 2 weeks of July, meaning i have had it nearly 6 months!) and i think it is time to upgrade soon. There are a few phones on the market that are interesting me. here is my current short list in no particular order:

Apple iPhone 3GS (or newer). O2 Ireland loose the Exclusive contract on this soon, and Vodafone are to also get their hands on it. Since i am a Vodafone Customer, and have a contract with them, i only have to resign some paper and pick one up… Rumours are that the next Gen (which will be the 4 Gen iPhone, but not necessarily 4G network support) will be out in April, but my money is on next gen coming around June or July, around the WWDC…
Nokia N900. Very cool looking device. Vodafone Ireland are to get their hands on this soon (maybe this month). looks very cool, fast and well, that keyboard is handy.
HTC HD2. Windows Mobile 6.5, 1Gz Processor, 800×480 display. What more can i say? Very nice device. O2 Ireland might get their hands on this, and if not, Expansys have it…
Google Nexus one. Not sure if this should be on the list, since Android has kind of annoyed me on the G1, but it does look nice. I got to play with out of these over the Christmas, and it was very nice and responsive. 1Gz processor makes it nearly 2x the speed of the G1. Using a phone for a quick demo is all well and good, but the real test is using it as your main phone. Lower Priority than the rest…
Likeliness of happening: Highly. Last year i changed phones 4 or 5 times… This is practically inevitable…

Hardware: After the build of the V.Next Workstation, and the MacPro becoming a HyperV server, there are very few things needed… but improvement can still happen…

The Proliant: This still only has 5Gb of ram and a Dual Core Processor. Not sure if i should go an upgrade this to a Quad core and 8Gb ram, or just build a new box with that kind of spec. Either way, the machine will get upgraded to Windows Server 2008 R2. Likelihood: 60% chance.
Mac Pro: No changes needed for this. 12Gb ram, Dual Dual core processors, 10k RPM hdds… Very nice. might just tweek its network connections… Likelihood: 100% chance.
God Box (AKA the V.Next): Thinking of adding a second graphics card. Maybe some more internal storage. and depending on how things go, maybe more memory. 12Gb is quite a lot at the moment, but we see how things go. If the Graphics card gets upgraded, more monitors might be on the cards also… Likelihood: 70% chance.
Opteron and Power Edge: these machines are now pretty much maxed out and out of upgrade options (Think i have a 2Gb or 4Gb upgrade for the Power Edge which never got installed… Might do that if i can find the memory). This year, i think these machines will loose their server status and might be officially deprecated… Likelihood: 65% chance
Media Centre: Over the last few years, i have had a few machines that i have called Media Centres… Now i think it is time to build a new one. Probably be a Quad Core box, few gigs of RAM, PCI-Express DVB-S2 tuners and a nice graphics card… We see how things go over the year. Likelihood: 55% chance
SAN: I have been wanting to build a SAN for a while now, mainly to store photos, videos, Ripped DVDs, Recorded TV shows, Software and VMs. With the move to Hyper-V i think getting this build sooner rather than later would come in handy. this way i can use the Failover support in Hyper-V. I am also now running at about 110Gb of photos stored on my machine, and have a lot of Music, Video, etc. If the Media Centre gets built, It will consume 2Gb per hour of SD video recorded and more for HD stuff… You can never have enough storage! Likelihood: 75% chance in one for or another…

House: Big purchase in Life. the rest are nice to haves, this would be epic! As the lads in work have started to call it “The Gadget Platform

Data De-duplication code in C#

admin — Mon, 11 Jan 2010 01:23:42 +0000

So, i have been looking at Data De-Duplication, and have built a little demo app in C# for this idea… The code is on GitHub and if you have better ideas around the code, please leave comments or update the block.

This code is not intended for Production use. it is purely experimental!

One idea i would like to look into though is using a proper backing store for this data: MSSQL DB, Amazon S3, RackSpace Cloud Storage, Azure Blob Storage, etc. the theory would go as follows:

user points de-dupe tool to a folder
folder is processed – each file is broken into 512byte blocks (could be more or less, configurable in code) and check-summed (currently using SHA512, but been told its over kill!) an Index file is created for every “real” file on the system. they contain the block IDs in order for the given real file.
after each block is check-summed, it is checked against the backing store. If the checksum exists in the backing store, the id is written to the index file. if not, a new block is added to the backing store and that ID is added to the index.
When a user wants the file, the index file is checked, all the blocks are pulled from the correct locations, and built up for the user to work with. Any changes go though the same process. if the user only changes a couple of sections at the end of a file, only those couple of sections should be changed…

You get some problems with changes, in that you dont really want to change all instances of that particular block, incase you break something else… you also need to have something cleaning blocks also, just to remove dead or orphaned blocks…

Running this on some XML files on my drive (about 3200 XMP files from Adobe Lightroom) the code finds 43k blocks duplicated, with an estimated 21mb duplicated. it does no de-duplication at the moment…

Ideally, this should be built as a FileSystem (showing as a hard drive, backed to, say, SQL server or a single large file) or as a SMB share, which can be mounted in Windows… Now to figure out if its possible…

What's your setup?

admin — Fri, 25 Dec 2009 22:07:05 +0000

Right. so OSNews has a post asking users to post their setup they are currently using… I was going to post a comment with mine, but though it would take up too much space… so, here is mine. Leave a comment if you wish with your setup…

Main workstation: There was a series of posts about this machine, and the v.next has now become the v.current. This machine currently acts as Dev Workstation, Hyper-V Server, and anything else i need a big fast machine for (Photo processing, Video Editing and conversion, etc) Specs are as follows:

Dual Xeon (Nehamam) 2.26Gz (Quad core, Hyper Threaded)
12Gb DDR 3 Memory
2 300Gb Western Digital Velociraptors (Raid 0 for boot and system drives)
2 1Tb Western Digital Caviar Black Drives (Raid 1 for Data)
NVidia GeForce 275 Graphics with 896mb ram
Running Windows Server 2008 R2.

Proliant Server (Hyper-V, Active Directory Domain Controller (secondary), DNS (Secondary), DHCP)
Intel Xeon 2.6Gz, Dual core
5 Gb Ram (DDR2 IIRC)
Currently got a 250Gb boot drive, and 2 500 Gb data drives, but this will change when the SAN gets built…
Windows 2008 Server (upgrading to R2 soon)

Mac Pro (Hyper-V Server, Remote Desktop Gateway)
Dual proc, Dual Core Xeon 2.66Gz
12Gb RAM
2 74Gb Western Digital Raptors
ATI Radion 9600GT Graphics (512mb ram)
Windows Server 2008 R2

Opteron Box (IIS, SQL Active Directory! only found out its setup for this now!)
Dual Opteron processor (old style, no Virtualization Technology, not even Dual core!)
2GB RAM (maxed out!) this is a SFF Box…
80Gb HDD
Windows Server 2008
Very basic box… Does the job, and nothing else…

Poweredge (Active Directory (Primary), DNS (Primary))
2.4Gz P4 Xeon (Hyper threaded, 64 bit and nothing else)
768Mb RAM…
80Gb HDD
Windows Server 2003 (Yup… its old!)
This used to act as the main webserver, SQL, ISA and Exchange box in the house, but it got old and needed to be replaced. now its just a nice little Active Directory box… Eventually planning on paving, upgrading the RAM and keeping just for AD and DNS. Maybe even a bit of file sharing…

Laptop (main working laptop, college notes, messing around when im board box…)
MacBook Pro
2.4Gz Core 2 Duo processor
4Gb RAM
250Gb (i think) HDD
Dual boot Win 7 X64 Ultimate and OSX 10.6 (Mostly Win7)
Its a Late 2008 UniBody MacBook Pro… More specs if yo go looking for it… Too lazy to look now…

NetBook (Messing with the MacBook is too big)
MSI Wind
Atom 1.6Gz Processor
2Gb RAM
160Gb HDD
Windows 7

So, thats about it… What have you got?

Carbon Tax Maths

admin — Thu, 10 Dec 2009 10:14:53 +0000

In yesterday’s Budget, the Irish government added a EUR15 per tonn of CO2 tax to companies… Some quick maths!

burning a Liter of Petrol produces 2.39KG of CO2 Per Liter burned. diesel = 2.62KG / L

that means for Petrol, you would need to burn 418.4 L to produce a Tonn of CO2 (381.6 for Diesel).

so, EUR15 / 418 = 3.5c (Diesel = EUR15 / 381 = 3.93).

Now, thats based on Metric toness and the the government are obviously screwing us a little more…

This maths can also be used to figure out your CO2 emissions from your car…

If your car does 25 MPG (on average) this works out at 11.3L per 100km. this works out that for every 100k you would create 27Kg of CO2. divide that by 100 and you get 270g of CO2 / KM… that’s a LOT!

the opposite is also possible to work out.. My car produces 245g of CO2 per KM…

KG of co2 produces per 100k = (245 * 100) / 1000 = 24.5KG

24.5KG / 2.39 Kg = 10.25L per 100k.

which is about 27.5MPG… change 2.39 for 2.62 for diesel, where appropriate… your millage may vary (sorry, pun intended…)

Getting the date and version of an assembly

admin — Sun, 29 Nov 2009 22:50:06 +0000

So, i have been busy working on code for Torrent Helper (still looking for a proper name and URL) and one of the things i wanted to add was a version number footer… ust for the laugh, just cause i could… But how? So, this is how you do it…

First, you need to modify the AssemblyInfo.cs file in your project (I have 2 projects, my website and my backend service). In here i set the AssemblyVersion and AssemblyFileVersion to 0.1.*. The * tells the compiler to magically build the number…

Next, how do you get it back out programatically? (click to see the larger version)

As you can see, i have 2 functions here. one (PrintBuild()) is called by the web site where i want to put the data (I know this is not pretty, but its working… maybe some dat i will changed it… the second functions only job is to get the information about the assembly and work out the date and time it was built… Any questions, leave a comment.

Announcing the release of Torrent Helper web site

admin — Sat, 28 Nov 2009 16:00:41 +0000

Today i would like to announce the release of Torrent Helper, a website i started coding about 19 hours ago (as of time of writing this post at 6pm on Satrday). The coding of this project started at 11PM on Friday, and took a short break at 6am for some sleep… It was officially released (with a pop up on Twitter) at 5.29pm…

Anyway, what does it do? You give it a Torrent file (Legal of course), either the URL or an existing torrent you have somewhere, and it will find extra trackers for that given torrent, add them to the file and send it back to you… Thats pretty much it!

Any questions, drop me a mail (tierno [at] lotas-smartman [dot] net) or leave a comment.