World map stat counter
If you happen to visit this page: https://planet.gnome.org/ (disclaimer: I'm not affiliated at all with GNOME), at the bottom of the page you will see a small picture of a world map with dots. The dots represent where the people in the GNOME community come from. I have seen this in other websites too, but they use the dots on the world map to represent where the sites' visitors come from, so it is a kind of stat counter.So! I want one too, because it looks good and it's interesting. It's actually nothing new and many others have created them. One can even easily register for this widget on many web analytics sites. But I don't want to use a third party analytics - I mean, this blog already uses Google webfont and a few other online javascripts providers (which are useful), adding another one just to show a silly stat counter sounds ... well, silly.
So I rolled up my sleeves a bit and started my google-fu --- to find nothing. I really need to sharpen my google-fu.
Then I sat down and thought about it - how difficult is it? All that is needed is to convert all those IP addresses from my webserver log into some sort of location identifiers (cities, countries, etc) and from there, to find the latitudes, longitudes (popularly called as 'geocoding'), and then transform them to x,y coordinates so we can draw them on a map. Sounds simple, right?
So I rolled up my sleeves a bit higher and went on to find the information. Firstly, I need to find out how to convert a bunch of IP addresses to a 'location' (=be it cities, countries, or may directly in latitude/longitude). As it turned out, there are quite a few who provides the service to do this conversion. A few examples that I find are: MaxMind's GeoIP, HostIP, IpInfo, geonames, and a few others. Most of them offer web-services - some are RPC-based, some are REST-ful web-services with JSON or XML payload (Again, I'm not affiliated with them - this is just the result of my searches).
I won't comment on the quality of their services (which obviously vary from one another) because at the end I decided not to use any of them. The reason is simple. I need to geocode a bunch of IP addresses, not only one. Geocoding one IP address through web-service is nice and good, geocoding hundreds of them - unless they have a specific bulk-geocoding service, is going to be a hassle and slow, not to consider that I may violate the terms of service. So, no, I need a better way to do it.
Fortunately, some of the excellent sites not only offer geocoding web services; they also offer the geocoding database. That's right, a database that contains a mapping between IP addresses and locations. The 'locations' here varies, some provides the countries in which the IP addresses are located, some are based in cities, some provided the longitude/latitude directly, etc. In the end, I decided to choose MaxMind's GeoIP. They provide a 'lite' version of the geocoding database under CC 3.0 license, which is nice of them. Even this 'lite' version consists of over 50MB worth of data in the CSV format - more than enough for a silly stat counter.
With their database, the 'geocoding' service because as a simple as search problem - given an IP address, find a row in the database which includes that IP in the row's IP address range. Once done, you've got the lat/long directly. Problem solved! Hey, I could use scripts to do this, awk will do nicely, something that naively looks like this:
awk -v myip=$IP ' $1 >= myip && $2 <= myip { print $6, $7 }' geoip.dataIt does work, and it worked fast - for one IP address. But on my laptop it took me about 120ms to do one search for one IP address. With a hundred IP addresses, that will translate to 1.2s ... that's slow. I can guess why it is slow - string operations, conversion of strings to integers, etc - most of the time that only needs to be done once. So why don't we do that? In the end, I wrote a bunch of simple C programs pre-process the geoip database, converting it from text into a fixed-record binary format. Once I have *that*, I can load them into memory and access them as arrays, using binary search to look for matches. As it turned out, I don't even need to load them to memory - I can just access the data file as memory-mapped file. And that cuts down the search time from 120ms to under 2ms. Not bad!
Well after I have lat/long, I need to convert them into xy coordinate of my world map. I got the world map from Wikimedia here. I chose the 310px version. From there a simple read of Mercator Projection will tell you what you need to know on how to do the conversion.
Okay, I have the geocoding, I have the map, I have the xy coordinate markers to be put on the map. The last component I need is some scripted drawing software to put these marks into the map image. There are many ways to skin this cat, but the simplest one is to use the famous "Gd" library (old home page here, new one here. So I got it, compiled it, and ... well it is a library. There was no tool to do it from the shell, one is supposed to write a C program to use them! There are wrappers for them: for Perl (Perl::Gd), PHP (PHP-Gd), and a few others - but I'm not using any of them (except Perl, and despite the beauty of the language, I've decided to stay away from it). Oh no, the other options is to use ImageMagick, but I really prefer not to use a chainsaw just to cut a strawberry branch. Fortunately, a kind soul has written a scripting tool for Gd, called 'Fly', available here. It is dated 2009 but still works beautifully today.
The rest is just a few scripts to glue all these together, and this is the final output :)
This map will be a permanent addition to the 'Stat' section of the menu block at the right hand side of this page.
I will make the code available later after I tidy it up a little.
Comments - Edit - Delete
Fonts!
I like fonts. No, I don't like fonts - I love fonts. I collect them, especially the good ones. I like their shapes - curvy shapes, boxy and rectangular shapes, flowery shapes, and all other odd shapes too. I like their designs - monospaces, proportionals, serif, sans, blackletters, narrows, books, uprights, italics, handwritten, cursive, floral, outlines, dingbats, icons - all of them. As they say, variety is the spice of life.I have over 300 fonts installed in my system although I don't use them all obviously - but I yearn for the day that I can
I can't really explain why I like them; other than the fact that I like to read and good fonts are really pleasing to the eye: it reduces eyestrain and make materials more readable. But my love for fonts is beyond just practicality - I just like to see the pictures generated by a body of text rendered with a well designed font. Good fonts are works of art - you can see how much effort went into their creation by looking at how tidy, fluent, aesthetic and yet consistent the resulting displays are.
There are many places where you can get good quality fonts (open source or otherwise) in the Internet. Here are some of my favorite places:
• Font Squirrel (http://www.fontsquirrel.com) offers many commercial-use-friendly free fonts.
• Google webfonts (http://www.google.com/webfonts) offers many fonts with direct @font-face embedding for your own websites
(this site uses google webfonts service too).
• Da Font (http://www.dafonts.com) is a font repository with huge collection that has been around for a long time; if you are after an oldish font (from early days of truetype), chances are dafont.com will have it.
Of course, keep an eye on the designers' own websites too.
Disclaimer: I am not affiliated with any of those websites. I'm just a satisfied user.
Fatdog64 comes with the complete set of DejaVu fonts: Serif, Sans, Mono, Condensed, all the bold and italic versions. Dejavu is a versatile font that meets needs of many applications. In addition to a large repertoire of letters ("glyphs" to be precise) for many languages, it is pleasing to look at too. Dejavu Sans is the default UI font for Fatdog64 while Sans Mono is the default for text editors such as Geany and Leafpad
as well as the rxvt-unicode terminal emulator.
I have recently switched to Anonymous Pro for my text editor font, but for terminal I stick to Dejavu Sans Mono. This blog uses Kotta One for the body text, and Italianno for headings, and Cookie for categories - all courtesy of Google webfont.
Recently, Adobe and Google contributed a new hinting engine for CFF fonts (aka OTF fonts) to the Freetype project. Freetype is the open-source font rendering engine (ie the piece of software that convert fonts into bitmaps / images that gets rendered to the screen). According to many reports, it is vastly superior than the already excellent existing Freetype hinting engine.
The good news is that this new version of Freetype (2.4.12) will be in the next Fatdog, the bad news is that the new CFF engine only works with CFF fonts (that is, OTF fonts), but not TTF. In my subjective view however, the new Freetype produces clearer and more crisps rendering even of standard TTF fonts.
If, like me, you love your fonts and the pleasing displays they create, this would be something to look forward to.
Comments - Edit - Delete
SJPPLOG 1.4 Released
Scott has just updated sjpplog (the software that runs this blog) to version 1.4, see here: http://murga-linux.com/puppy/viewtopic.php?p=704592#704592. It is a relatively minor update with inputs from yours trule and Christina (forum name efiabruni), but it is still a good update.Of course, this blog now runs the updated software too. Thanks to Scott and Christina (efiabruni).
PS: Christina now runs her own version of pplog (the software from which sjpplog is based on), called pe_pplog. You can get it here: http://tine.pagekite.me/pe_pplog.pl/
Comments - Edit - Delete
Blog software updated
While we aren't looking, sc0ttman (the author of SJPPLOG, the blog software used in this site) has been working quietly under the radar to update his excellent application.Scott has recently released version 1.3 of SJPPLOG with loads of enhancements, and I have finally upgraded this blog with his latest concoction.
See what has changed here: http://murga-linux.com/puppy/viewtopic.php?p=689184#689184
Thank you Scott !
Comments - Edit - Delete
First post
Journey of a million words start with the first post.Comments - Edit - Delete
