around 500 days ago i had a crazy idea: mapping the net in a database. all domains, all hostnames, all relations of ns- and mx-servers.
i knew a few sites who should have this data but would not really let you look it up - whois.sc and netcraft.com were amongst them. that was all i knew. oh and yes, i knew mysql, i worked with sqlite and microsofts sql-server for years.
i expected this to be an adventure. a textadventure, fun.
and hence, it was fun. the database crashed, servers got blocked, i had errors in my harvesting scripts and i was overwhelmed when i got around 100 million domains with even more hosts at once.
and now i'm sitting on a bunch of data that gets older. my time is limited, my hardware-ressources are as well. data is getting old. i started updating the hostname/IP-entries these days. I should have done this earlyier, i know - but i didn't. time is limited - remember?
a bit frustrating: the "update-lag", the time between the last update of a hostentry is currently exactly at 427,167 days. most frustrating: it's still increasing.
hmpf.
tom
Saturday, June 02, 2007
Subscribe to:
Post Comments (Atom)
1 comment:
That's the first I just visited your website and I find it really interesting! Congratulations!
voyance gratuite en ligne
Post a Comment