Using Squid as a Accelerating Proxy

I needed to accelerate some web traffic, we're using a new web system in house and wanted to make sure that the multiple users who access the system didn't clog up the busy broadband lines. 

The solution was to setup a Squid Proxy. Squid can be used in many different ways, one of the most simple solutions is for Squid to sit between your internet connection and desktop browsers and provide a local cache of content. 

Once Squid is installed edit /etc/squid/squid.conf or take a backup and paste in this one below:

 

http_port 192.168.10.108:8000 transparent

acl my_network src 192.168.10.0/24

http_access allow my_network

acl all src all

acl manager proto cache_object

acl localhost src 127.0.0.1/32

acl to_localhost dst 127.0.0.0/8 0.0.0.0/32

acl purge method PURGE

 

acl CONNECT method CONNECT

http_access allow manager localhost

http_access deny manager

http_access allow purge localhost

http_access deny purge

#http_access deny !Safe_ports

http_access allow localhost

http_access deny all

 

cache_dir ufs /cache 200 16 256

access_log /var/log/squid/access.log squid

refresh_pattern ^ftp: 1440 20% 10080

refresh_pattern ^gopher: 1440 0% 1440

refresh_pattern -i (/cgi-bin/|\?) 0 0% 0

refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880

refresh_pattern . 0 20% 4320

acl apache rep_header Server ^Apache

broken_vary_encoding allow apache

extension_methods REPORT MERGE MKACTIVITY CHECKOUT

hosts_file /etc/hosts

coredump_dir /var/spool/squid

I have a cache_dir set to /cache and have given it 200MB if you delete this Squid will use a sesible default. 

This configuration is based on the Squid server living at 192.168.10.108 and listening on port 8000. All you need to do now is setup a http proxy in your browser pointed to this IP/Port and all traffic will pass through the Squid box. 

Looking forward to Raspberry Pi (like every other geek who grew up in the 80/90's)

Rob Hauge has a nice update on the Raspberry Pi. I think like many people who grew up during the 80's and 90's with computers that didn't do an awful lot when they first came out of the box I'm excited about this concept.

I like the idea that any kid with enough determination will be able to take this and learn how to turn it into something useful. I've had a lot of conversations with people of a similar age and we all came to the same conclusion. The home brew computer culture of our childhoods was a key factor to our success as computer professionals years later.

Pcb1

Maybe kids of today will need some mentoring to grasp what is actually possible, after all so many of them these days believe computers are just Facebook/YouTube terminals. But if Raspberry Pi is a success I think we'll see a lot of software projects that make accessing the platform more universal for all.

Some projects I'd like to see come out to support this are:

  • Simple Graphical Programming environment like STOS the ability to do something graphical can capture the imagination.
  • Programming Training Courses (Open Source Training) that allows people who know about programming to teach kids who don't.
  • A web framework that makes it easy for kids to produce interactive web-apps.

More info on Raspberry Pi.

 

First Sign of the Cookie Stupidity

6152575914_a924eaa3b4_z

I Just noticed on the  ICO website the first example of someone dealing with the new Cookie legislation. Crazy to think that every single site (that has a fear of the new law) will have to do something as rubbish as this. 

What does each site have to ask: "Hey we need to use cookies, if we don't everything will be broke - what do you want to do?"

Don't you hate it when bureaucracy wins out over common sense. 

 

 

 

This Butter made me Mad!

12997544838470

Check out this Butter packet. This isn't special butter, its not super, extra, hand made butter its the cheap stuff you get in corner shops.

Made in New Zealand, shipped to Denmark to be packaged and sold on the shelf of a corner shop in the UK. 

So I used Map Crow to calculate how far its travelled and remember this is a conservative estimate as I'm sure it didn't travel in a straight line. 

New Zealand to Denmark: 11,152 miles.
Denmark to UK: 500 miles. 

So it's travelled a minimum of 12,000 miles to get to the corner shop. Come on UK, we have cows and fields here. It shouldn't have travelled more than 100 miles! 

CESG and its approach to software certification is flawed

12992362225933

A story here titled: Government open source plan hindered by lack of security clearance  at Computer Weekly talking about how Open Source software products cannot gain CESG approval (required for use in government and military uses), the issue is simply this: To get approval the product must be commericaly sponsered - that is the Vendor pays for the product to be source code level analised before it can be used. 

When someone wants to use an Open Source solution there is no one willing to foot the bill of this certification because after all its free software so there is no way to recover cost of the process. 

My opinoin this is a flawed model, in a time of austerity where governments are trying to cut back on costs, excluding free software from UK government use crazy. 

Here's an idea:  Why not apply a levy to the certification of commerical products, the money raised would be used to certify the most top requested Open Source projects. If commercial vendors are confident enough in their product to submit it for CESG accreditation then they should be confident it can stack up against Open Source offerings after all. 

 

Blackberry to Android: A week on

I've been a Blackberry user for about 2 years, it was my first smart phone and I quickly feel in love with it, I had a Bold 9000 and for a short while a Bold 9700, here's what I liked about the Blackberry:

  • Respectable battery life.
  • User interface that makes senses. 
  • Once you get used to the keyboard its fast to type on. 
The things I didn't like:
  • Terrible camera
  • Poor App selection - but those I used were very good quality. 
I also learnt how to program the Blackberry developing the mobile version of Egress Switch

So when I  our mobile contracts came up for renewal it was a tough choice to make when it came to choosing a new handset. The decision was between Android (HTC Desire HD) or Blackberry (Torch). I've wanted to try Android for a long time, so I decided to be brave and go for a change, my logic was that my Bold 9700 is still in very good condition so if I hated Android I could always drop back to that. 

So a week on I have to say I'm in love with Android and the Desire HD, but there have been problems:

  1. The battery life sucks! The Bold 9700 has been left on (and still has charge 1 week later) the HTC is lucky to do a full day. I've learnt a lot about looking after Android this week and in doing so found you can improve the battery life - but it takes a commitment. I'll write about this some other time. 
  2. The App Store is very crowded - so much so that its often hard to find a decent app buried amongst all the crap. My advice is Google the app first and find out what people are saying and if they recommend alternatives. 
  3. Touch keyboard - ok this is just me I was used to a hardware keyboard and switching to touch is hard. One guy I know who has always had BB Pearls with that strangle little 2 characters per key setup, he now has a Torch and runs the virtual keyboard in that mode! Its just what your used to. 

So after a week here are Apps I've now decided I can't be without on my HTC:

  1. Twitter - I'm using the official app right now, they are trying to make it look just like the iOS version and apart from a few annoying bugs I think its about the best client on Android .
  2. DoggCatcher - this is a Pod cast downloader. I used to hate the fact that on iPod or Blackberry I had to download podcasts in iTunes the sync (copy) them to the device. with DC I can just managed my subscriptions and download directly onto the device, its also a decent playback interface remembering the location of each show. 
  3. Audible - Just fantastic all my books are in the library and I can download and listen whenever I like. 
  4. Kindle - Just like DC and Audible its possible to see your library and download content at any time. 
  5. Tumblr - Nice app that I use for Micro-blogging updates. 
  6. Evernote - the app rocks but I'm still not convinced i'm an Evernote user yet.
  7. InstaFetch - provides "share" option to send things to Instapaper which is how I like to save Interesting Tweets that I want to read on a PC. 

So the next thing is learning the development environment and getting some of our products running on Android. Should be fun I think. I'm going to keep the Blackberry close by (especially as I still have to support our product on it) but for now I'm happy I made the switch. 

Setting up Open Source Email and Webmail in Windows

I've just completed a week long project to build an intelligent mail server solution, In the past I've always worked with Linux or BSD based mail servers but we're doing some work on building a smart server that does some automatic processing of outbound mail messages. When it’s ready I’ll post an updated - it will be part of our Egress Switch product. 

So what did I learn along the way? There are a lot of really cool free mail server options on the Windows platform these days. Here's what we've currently got running for a demo of some of our technology.

Setup Microsoft/web

Download and install the Microsoft/web platform (unless your on a Server OS in which case you've already got IIS and possibly a MSSQL server). The Microsoft/web platform is MS taking on the LAMP world with its own tight and easy to use Web stack which will install on Desktop Windows (XP, Vista, Win7). 

Make sure you install the MSSQL server too as you’re going to need this in a while. 

Image007

Setup hMailServer

Install hMailServer this is a POP, IMAP and SMTP server that runs as a service on your machine, like Microsoft/web you can install this onto a Desktop OS as well as a server. There is are plenty of docs on how to setup and configure hMailServer, and a great community of support to help you if you’re stuck. The few issues I had were resolved with a search of the forum. 

We use hMailServer because it has the ability to define server side rules that control the processing of email messages - when a rule is matched you can invoke VB Script (Yuk!) but from there jump into native code or a batch file to process the message before it is delivered. 

Image008

Setup AfterLogic Lite Webmail

AfterLogic makes some great commerical webmail solutions they also have an open source lite version of the Webmail front end. For the purpose of our demo we wanted to show people how out back end Switch technology working with a Webmail interface so we setup AfterLogic. If you’re planning to use this consider buying the commercial version or making a donation for the lite edition, it’s worth it. 

You'll need to know a little about setting up IIS server to install AfterLogic, but if you’re brave the new IIS7 that ships in Microsoft/web platform is much simpler to setup and manage that the older versions. 

Once its running point your AfterLogic install at your hMailServer install and you'll get web access to your emails stored there. 

Image009

If your sufficiently technical and enjoy hacking solutions together you’re going to have fun with these technologies. The ease of user of the new Microsoft/web platform means you can build and deploy powerful windows based web applications (even PHP is catered for), combine this with hMailServer and you could setup a complete internal solution within a few days.

Will you just delete the partition!

As part of my Eco drive I’m trying to not use CD/DVD media. When it comes to installing operating systems you can now generally put the ISO for the installer onto a USB device and have the computer install from there.

I’ve seen a problem as few times now when you try  and move a device between Mac and Windows environments. The problem is that the Mac converts the device to a GUID partition table layout. But when you take this USB device back to Windows you have a device which Windows won’t convert back to standard layout.

Even in disk manager you will see partitions that cannot be deleted (the delete option is just disabled).

I’ve found a way to get round this problem and I wanted to share it and hopefully save you some time.

Fire up a command prompt and run the following command: “diskpart”

From this tool you can select, list, delete, clean all the partitions on the device.

PLEASE make sure you select the right disk to work on first and run the “?” command to see the help.

Don’t blame me if you trash your disks, but if you’re after trashing disks this is the tool for you .

Undelete Files on Windows using Linux

A friend of mine called me up the other day in a panic - he'd accidentally deleted a set of photos of his baby and to make matters worse he'd also emptied the Recycle Bin.

I've not had to recover files for someone for a while so I dug out various boot CD's which are all a little dated now. After an hour or so I had no luck. I though I'd checkout what options existed within Linux. 

The Linux tool I used is called ntfsundelete and is part of the NtfsProgs toolset here are the steps to follow if you want to recover deleted files from a Windows machine:

1. If the machine is powered off do NOT boot windows again. The less activity that occurs on the disk the more chance you have of recovering the files. 

2. Get hold of an Ubuntu Desktop CD (Download and Burn the ISO on another PC). 

3. Boot the machine using the "Live" option which will take you to an Ubuntu desktop. 

4. If the Windows partition holding the deleted files has mounted you need to unmount it (Use the eject icon next to the disk in the file viewer).

5. Start a terminal prompt.

6. Run the following command ntfsundelete -s -m '*.jpg' /dev/XXXX - this will show the deleted files on the volume XXXX - to find out the name of the volume issue a "mount" command at the prompt and identify the disk you want to work on. 

7. If the this search command yielded results you can do a copy operation as follows

8. ntfsundelte -u -m '*.jpg' -d /mount/SomeUSBDevice - where "SomeUSBDevice" is the mounted spare USB device you've inserted to copy the files to. Issuing a "mount" command will again help you to locate the device. 

I know these aren't super user friendly steps to follow you'll need a bit of understanding of Linux to perform these steps - why not ask a friend who knows more to help you out. I hope you get your files back - I managed to recover about 270 images for my friend and he only lost about 20 to corruption. Remember nothing beats having a decent backup.

Consider using Picasa and paying a few bucks a year to sync your images to PicasaWeb.