some might have noticed: serversniffs half dead since 72 hours.
the ups failed, power failed and the raid got corrupt. time to restore from a db-dump.
and time to upgrade: our postgresql-database did run on windows 2000, which is rather fine unless your database won't grow to bloated. postgresql won't eat more than ~ 6o0MB Shared Memory on windows 2000, wich is fine, unless your database wont grow to bloated... - because the vacuum-process needs more RAM. So I ended up with an ever increasingly fragmented database.
time to switch to linux. i tried to build the system as guest on VMWare ESXi - which i was able to manage - but there must have been something horribly wrong with the filesystem: all disk-transfers were slow as hell, usually below 10MByte/s. After 60 hours of setting up ESXi, a Linux-Guest and the database i threw the stuff away and started all over installing plain Linux Sunday evening, 48 hours after the database initially crashed.
Since then i installed linux on the machine, prepared raid-array, database and everything else. currently the data is restored from a dump and the indexes are generated - 6 of ~20 are already done, the rest might be finished by tomorrow. to what i see right now the database is considerably faster using linux and 1 GB of Shared RAM.
I apologize for the downtime, especially to the folks at blackhat.
tom
Monday, August 04, 2008
Tuesday, July 29, 2008
Friday, May 30, 2008
How to check SSH and SSL Certificates for the debian flaw
I had quite a few questions from people how to check their SSH- and SSL-certificate for the recent debian-flaw. As i had to check a few hundred customer-sites too, i did a little webinterface for checking SSHCerts and SSLCerts for the PRNG-Bug.
See them at work at http://serversniff.net/sshreport.php and http://serversniff.net/sslcert.php
No magic behind - just debians ssh-vulnkey and a php-rippoff from the chksslkey-shellscript written by Michael Holzt. Maybe this will help the average rootserver-admin checking their sites.
Both scripts use standard-sets for verifying the keys, checking only standard-dsa/rsa-keys for ssh and 1024/2048-bit-keys on the ssl-check. Drop me a line to tom@serversniff.net if you really need to check for any different keysizes.
tom
tom
See them at work at http://serversniff.net/sshreport.php and http://serversniff.net/sslcert.php
No magic behind - just debians ssh-vulnkey and a php-rippoff from the chksslkey-shellscript written by Michael Holzt. Maybe this will help the average rootserver-admin checking their sites.
Both scripts use standard-sets for verifying the keys, checking only standard-dsa/rsa-keys for ssh and 1024/2048-bit-keys on the ssl-check. Drop me a line to tom@serversniff.net if you really need to check for any different keysizes.
tom
tom
Friday, May 16, 2008
Mapped the net... in parts.
"Mapping the net" did we call our little project to map as many known hosts, ips and domains as possible some two years ago. Some laughed, others smiled. And we mapped. Thousands of hosts daily, running into a steadily growing postgresql-database built out of junk-hardware, running on a single cheap dsl-connection.
I started some bencharking using search-engines to see how many hosts we really know, and i was surprised to see that we already know between 70 and 80 percent of all known hosts of major international hosts indexed on rank 1-1000 in common search-engines. And we've still far more than 10 million hostnames listed to sort in. I didn't expect to get so far when i started this funny project.
tom
I started some bencharking using search-engines to see how many hosts we really know, and i was surprised to see that we already know between 70 and 80 percent of all known hosts of major international hosts indexed on rank 1-1000 in common search-engines. And we've still far more than 10 million hostnames listed to sort in. I didn't expect to get so far when i started this funny project.
tom
Saturday, March 08, 2008
Whois dropped
Some germans consider whois via serversniff
Get your whois-info at one of the thousands of sites around the net hosted somewhere outside germany or directly at the nic listed on serversniffs-domain-report.
For the breach of privay: There was a guy, amongst others, writing me an email to "immediately remove my Name from the page http://serversniff.net/dnr-webmasterinformation.<censored>. He didn't like the realname to show up in the whois-information. Hey - I deeply understand this request: If I'd operate a site like http://www.webmasterinformation.xx, I wouldn't want to have my name assigned to it, too. LOL!
Maybe somebody's williing to tell him about whois at all?
The net's a crazy place.
Cheers,
tom
- a breach of law
- a breach of privay.
Get your whois-info at one of the thousands of sites around the net hosted somewhere outside germany or directly at the nic listed on serversniffs-domain-report.
For the breach of privay: There was a guy, amongst others, writing me an email to "immediately remove my Name from the page http://serversniff.net/dnr-webmasterinformation.<censored>. He didn't like the realname to show up in the whois-information. Hey - I deeply understand this request: If I'd operate a site like http://www.webmasterinformation.xx, I wouldn't want to have my name assigned to it, too. LOL!
Maybe somebody's williing to tell him about whois at all?
The net's a crazy place.
Cheers,
tom
Saturday, February 23, 2008
offlinetime while rebuilding db
we switched the domain-database to new, hopefully faster hdd's sponsored by roelof temmingh (and me).
since postgresql still denies a parallel installation i took the opportunity to rebuild the database, update the server and switch the stuff to the new sata-raid. it'll take a few hours until the database is rebuilt and restarted, but it's weekend - you don't work anyway, do you?
we'll be back again soon.
tom
since postgresql still denies a parallel installation i took the opportunity to rebuild the database, update the server and switch the stuff to the new sata-raid. it'll take a few hours until the database is rebuilt and restarted, but it's weekend - you don't work anyway, do you?
we'll be back again soon.
tom
facts and figures
our current lookup-lag: 237.405 days.
current number of known domains: 39.163.435
still sorting in ~100.000 domains per day from queues, mainly generic .com-domains.
tom
current number of known domains: 39.163.435
still sorting in ~100.000 domains per day from queues, mainly generic .com-domains.
tom
Wednesday, February 06, 2008
facts and figures
For the historic records:
we still lag with re-lookups of our hostnames - current time between a renewal of the IP-Lookup for a hostname is 238,749 days.
We know 36.314.321 domains, the queue with hostnames to sort in decreased to 71.000.000.
The "offline-queue" with not yet queued hostnames is around 5 million hosts.
We're still on an SCSI-Array straight out of the hardware-museum with 8 hdds, 31 of 141 GB free. Over ten year old hardware, still working fine and reasonably fast.
tom
we still lag with re-lookups of our hostnames - current time between a renewal of the IP-Lookup for a hostname is 238,749 days.
We know 36.314.321 domains, the queue with hostnames to sort in decreased to 71.000.000.
The "offline-queue" with not yet queued hostnames is around 5 million hosts.
We're still on an SCSI-Array straight out of the hardware-museum with 8 hdds, 31 of 141 GB free. Over ten year old hardware, still working fine and reasonably fast.
tom
Tuesday, February 05, 2008
Cuill
Cuill started crawling Serverniff a few days ago. It does crawl slow, but very steady.
I don't know if this is good news for serversniff, but they have a friendly and steady crawler.
I wonder, when and if they go public - and i'd bet whatever you hold against me that they will be bought by a major company (there are not too many of them left) maximum 6 months after they open their search to the general public.
Anybody willing to bet against?
If you don't know cuill - google and teccrunch will tell more.
tom
I don't know if this is good news for serversniff, but they have a friendly and steady crawler.
I wonder, when and if they go public - and i'd bet whatever you hold against me that they will be bought by a major company (there are not too many of them left) maximum 6 months after they open their search to the general public.
Anybody willing to bet against?
If you don't know cuill - google and teccrunch will tell more.
tom
Tuesday, January 22, 2008
Kick-Ass Feedback
A swedish user kicked my ass to remind me that serversniff's AS-Report is not always reporting hat it should report.
Yah. I ceased working on the stuff to get the domain-database fixed way back in Oktober 2007. Some of the mess is fixed now. Data is up-to-date again, I added more than 17.000 new subnets and i'm goin to build a complete BGP-Parser soon. I reactivadted the daily updates after i fixed database and scripts to work again.
We're currently analyzing BGP-Tables from routeviews.org and LINX once a day, we might implement KIX and DE-CIX as well.
I'd be happy to get more feedback - but it seems that most of you are plain happy with serversniff or just to bored to bother if something doesn't work out at all.
tom
Yah. I ceased working on the stuff to get the domain-database fixed way back in Oktober 2007. Some of the mess is fixed now. Data is up-to-date again, I added more than 17.000 new subnets and i'm goin to build a complete BGP-Parser soon. I reactivadted the daily updates after i fixed database and scripts to work again.
We're currently analyzing BGP-Tables from routeviews.org and LINX once a day, we might implement KIX and DE-CIX as well.
I'd be happy to get more feedback - but it seems that most of you are plain happy with serversniff or just to bored to bother if something doesn't work out at all.
tom
Subscribe to:
Posts (Atom)