The Big Big AXFR

I like the global AXFRS posted in one of the comments to my previos posting. Have a look at Maximilian Dornseifs well-hidden blogentry at http://blogs.23.nu/disLEXia/stories/10092/ to get an idea on how to automate this. Maximilian missed, that there is a hell of secondary-TLDs like co.uk, ac.uk etc. etc.
He also missed that there are a few real big hosters and providers who return far more entrys than many TLDs.

But all in all this blogentry was one of the reasons to create tomdns.net.

Thanks, Maximilian.

tom

Where do i get domains....

Dear kids,

I understand completely that you ask me where to download a list of 100.000.000 domains.

I did, and this drove a few GB traffic to a website. Can you imagine what would happen if i'd post any such url here? Boys, i guess you can. So please, have a look at http://johnny.ihackstuff.com/ and try to have fun with our favourite search-engine as well.

Another option: Since most of the URLs listed on the site are .com/.net, you might also get them directly from verisign. The TOS there are roughly the same as the TOS at www.serversniff.net, in fact i derived serversniffs Terms of Service from Versign, for the last what i wanted to do is support f*cking spammers.

Or: Create something on your own. Something to offer, me or the "community". Then come back on me and ask. I'd be happy to support you with a list of domains if you're able to explain what you're workin on. I'd be happy to team up with you if there is any win-win-situation.

If you're working for a company: I'd be happy to support you with hostnames, domainnames or known IP's, filtered by whatever you want, given that you're willing to agree to our ToS. Drop me a mail and I'm quite sure we'll negotiate a reasonable agreement.

Tom

before the flood....

We are in the process of updating our domain-database: we just started an insert of roughly 150.000.000 hostnames, bringing our db-system to the limit.

We ceased "regular" spidering and updates for a while to catch up with this bulk-data. Currently we run at a rate of about 1.000.000 new domains per day, which we consider not really bad, but still unsatisfying. We are currently testing a NAS-Array running on 6 SCSI-Disks (currently as an experiment - we will invest in more hardware if this proves to be faster than the current system).

thomas

Encryption added

Its christmas and we are working on extending and finetuning serversniff.net. We added a bunch of encryption-functions recently to check how common ciphers treat your text. While this is not designed to be used in a productive environment, it might be useful to check some other implementation of these ciphers.

Please be aware of possible keylengths and used blockmodes!

We are currently workin with the mcrypt-lib and support AES (Several Modes), Blowfish, CAST, DES, 3DES (TripleDES), GOST, LOKI97, RC2, RC4, Saferplus, Serpent, Twofish, Wake and XTEA.

Expect more Ciphers (and Blockmodes) to come when we gathered some experience with the current set.

tom

Encryption

We changed our hash-pages and updated stuff:

We do support Hashes (Strings, Files), Checksums (Strings), Several Encodings (Strings) and finally a bunch of cryptographic function. Just have a look at the new "Crypto"-Submenu on serversniff.net.

tom

Yes, we are...

we got a request to sell serversniff.net. hey, we are bribable. direct your offers to sales@serversniff.net - we'll negotiate all the rest.
If you just find serversniff to be useful or want to implement functions in your website, there is no need to buy the whole stuff. Just drop us a mail to get access to our api-functions. We still don't charge nothing for its use as long as you are using it reasonably.

tom

Finetuning and cleaning up

We started finetuning serversniff a little bit - fixing the ton of bugs still laying around, adding a bit of sorting or optics here and there or extend the explanations of a few scripts. Switched a few scripts from a generic approach to using serversniff's API - stuff like that. Not really noticable at all, but eating up enough time on our side.
We're dreaming of many many new functions to come - if only time would allow...

Tom

New HTTP-API-Functions

We updated our HTTP-checks and added new API-Funktions. You can craft your own HTTP-requests now, e.g. do a GET with HTTP 1.0 with or without Host-Header and check the response, either only the HTTP-Servers header-info or the complete file, you can even filter for some special lines e.g. to get only the Server-Header or so.
This check does support servers listening on multiple IPs and is also capable of doing https.
We started implementing this backend to Serversniffs frontend - you might expect more HTTP-Checks based on this backendscript to come.

tom

New Scripts

The IP-Scripts are coming back.

We are on our way to extend serversniff with some new IP-Scripts. We started today with a simple icmp-ping, that simply sends 4 ICMP-Echo-Requests to a given host.

Additionally, for a simple ping would be quite lame, we implented what we call a "Ping-Row", that sends about 16 ICMP-Echo-Requests with increasing packetsizes. You'll be surprised how many sites allow small pings, while their routers or firewalls sort out the hugeICMP-Packets.

tom

Extending the API

Our domain-database seems to be up and running: the scripts run very stable, we run a few background-tasks that are feeding the database with around 50.000 new domains per day.

We are working on further extensions of our API in combination with our checkomatik. Although we missed owasp's "automn of code" we are confident that we will release a full-featured version of checkomatik by the end of this year.

While we are using it internally since months, it still lacks a real user-dependent interface, security, translation and, most important, automation.

We got a few steps up the ladder with creating more api-functions, stuff like the "pingrow" or a complete ssl-check.

tom