Clever Hacking: google.js

I am learning to love Windows. Yeh it has its own problems and sometimes could be a bit insecure but in general, it just works well and every single piece of it is reusable and scriptable. Today, I quickly wrote a Google search scraper. It runs straight from the command line.

Pay attention on the size of the script and the level of clarity of the code. Moreover, it is well integrated with the host’s setup and connection settings (proxies, socks, etc). You cannot do that even with Python. Thanks Microsoft.

http://www.gnucitizen.org/blog/clever-hacking-google_js/google.js

There are two way you can run the script. If you default scripting engine is cscript, then you can just type: google.js some query here. If this is not the case then you either need to be explicit like this: cscript /nologo google.js some query here, or make cscript default like this: cscript //H:CScript. Whatever you do, the code will run and will work flawlessly.

I enjoy when things are plain and easy or just simply clever.

Comments

Stephen Bloch responds:

When running your script I’m getting:

“Access is denied” message.
The error code: 80070005,
the source: msxml3.dll

pdp responds:

cscript //H:CScript

thornade responds:

No output for me, when I debug, it seems like I have no tag <a> in the returned document but when I sniff, I could see answers from Google, furthermore the script sends 2 requests (q=xxx, q=xxx&start=10)

I wrote a similar script in perl to find all web servers crawled by google for a specific domain (ie: gnucitizen.org). I did not use the wap version, is there any reason to use it ? (speeder, smaller … ?)

nori responds:

Hi, no outpout when using no proxy. But when passing through Paros, results are returned. Maybe an issue with Accept-Encoding: gzip,deflate…

strange, it should work flawlessly… are you sure that your system proxy settings are set correctly?

if (WScript.Arguments.length == 0) {
	WScript.Echo('usage: ' + WScript.ScriptName + ' <query>');
	WScript.Echo('       ' + WScript.ScriptName + ' site:gnucitizen.org ext:js');
	WScript.Echo('');
	WScript.Echo('Google Search');
	WScript.Echo('by Petko D. Petkov (pdp) GNUCITIZEN (http://www.gnucitizen.org)');
	WScript.Quit(1);
} else {
	var tmp = [];

	for (var i = 0; i < WScript.Arguments.length; i++) {
		tmp.push(WScript.Arguments(i));
	}

	var query = tmp.join(' ');
}

var pos = 0;
var doc = WScript.CreateObject('MSXML2.DOMDocument');
var xhr = WScript.CreateObject('Microsoft.XMLHTTP');
var resp;

doc.async = false;
doc.validateOnParse = false;

do {
	var lns = [];
  xhr.open('GET','http://www.google.com/xhtml?q=' + escape(query) + (pos != 0 ? '&start=' + pos : ''),false);
  //xhr.setRequestHeader("Accept-Encoding", "text");
  xhr.send(null);
	//doc.load('http://www.google.com/xhtml?q=' + escape(query) + (pos != 0 ? '&start=' + pos : ''));
	try{
		doc.loadXML(xhr.responseText);
    //WScript.Echo(xhr.responseText);
	//WScript.Echo('XML:' + xhr.responseText);
	var as = doc.getElementsByTagName('a');

	for (var i = 0; i < as.length; i++) {
		var href = as[i].getAttribute('href');
		//WScript.Echo(href);
		var match = href.match(/^\/gwt\/.*?u=(.*?)$/);
		//WScript.Echo(match);
		if (match) {
			var ln = unescape(match[1]);

			lns.push(ln);
		}
	}

	}catch(ex){
		//WScript.Echo(ex.description);
	}
	if (pns && pns.sort().join() == lns.sort().join()) {
		break;
	}

	for (var i = 0; i < lns.length; i++) {
		WScript.Echo(lns[i]);
	}	

	var pns = lns;

	pos += 10;
} while (lns);

Yes, it sounds like MSXML2.DOMDocument could not deflate gzip. If I disable HTTP/1.1 on Internet Explorer settings, it works better, in this case no gzip encoding support.
Nevertheless, foreach page Google returns, the DTD is checked. This represents 31 documents downloaded…

nori, cheers for that.

albert arul prakash responds:

Working fine for me. Its lovely

hackathology responds:

same here. No output when not using any proxy

I don’t know. it works really well for me. Check if cscript is your default scripting engine. If not, just follow the steps described above.

followed all three steps man, still, it doesn’t work. Strange.

	Blogsecurify is a division of GNUCITIZEN. The initiative was established to provide social media security services through our free automated testing engine. The Blogsecurify team is also engaged to deliver quality content on issues concerning social media technologies.
	Netsecurify is a division of GNUCITIZEN. The initiative was established to provide network security services through our free automated testing engine. The service is still in private-beta.
	Websecurify is a division of GNUCITIZEN. The initiative was established to provide a free web application security framework for automated and manual penetration testing. The service is still in private-beta.
	Secapps serves as an application directory of all online tools which the GNUCITIZEN team has built over the years.
	Securls serves as an information security intelligence tool, combining news and articles from the best information security resources online.

GNUCITIZEN

Navigation

Clever Hacking: google.js

Comments

Respond

GNUCITIZEN Products

Random Posts

Cutting-edge Network