Hammerspace sounds cool and the idea behind it sound great too.
I’d be willing to help with developing it. I’ve been meaning to try a firefox plugin for some time now. I really don’t know the first thing about it though. I’m a quick learner though.
Parandsays:
Does it have to be a Firefox plugin? I’m guessing you can setup a proxy server that records everything it proxies. You’ll pay a bit of a performance penalty, but it’ll be a nice and simple solution.
It is a proxy solution and you will need to somehow make it persist the data (it records everything into memory), but it does the other 90% of what you need.
Scott Flinnsays:
Very cool idea. Sounds like something I might want to use myself. Personally, I’d probably favour the proxy solution because I use a variety of different browsers.
I wrote something similar while at NRC. I needed a Web crawler that could evaluate JavaScript in an authentic browser environment, complete with a DOM and all the usual embedding connections. So I wrote the crawler as a Mozilla extension. It didn’t actually write things to file (it wrote summaries to a DB), but that’s pretty easy. What it *did* do was figure out all the containment relationships so that it could fix up links for local use, the way wget does. It was actually kind of tricky figuring out how to associate independent requests with each other — e.g., was that page loaded into a top level window, or into an iframe on some page? But Mozilla (and hence Firefox) has enough hooks to do it, if only barely.
I’d love to help. I wish I had the time. If a few months go by and you’re still looking for something like this, the timing might be better for me.
Jonsays:
Um. No one mentioned Slogger? That’s exactly what you want.
Hammerspace sounds cool and the idea behind it sound great too.
I’d be willing to help with developing it. I’ve been meaning to try a firefox plugin for some time now. I really don’t know the first thing about it though. I’m a quick learner though.
Does it have to be a Firefox plugin? I’m guessing you can setup a proxy server that records everything it proxies. You’ll pay a bit of a performance penalty, but it’ll be a nice and simple solution.
Maybe ScrapBook extension could help you? http://amb.vis.ne.jp/mozilla/scrapbook/
Try Charles:
http://xk72.com/charles/index.php
It is a proxy solution and you will need to somehow make it persist the data (it records everything into memory), but it does the other 90% of what you need.
Very cool idea. Sounds like something I might want to use myself. Personally, I’d probably favour the proxy solution because I use a variety of different browsers.
I wrote something similar while at NRC. I needed a Web crawler that could evaluate JavaScript in an authentic browser environment, complete with a DOM and all the usual embedding connections. So I wrote the crawler as a Mozilla extension. It didn’t actually write things to file (it wrote summaries to a DB), but that’s pretty easy. What it *did* do was figure out all the containment relationships so that it could fix up links for local use, the way wget does. It was actually kind of tricky figuring out how to associate independent requests with each other — e.g., was that page loaded into a top level window, or into an iframe on some page? But Mozilla (and hence Firefox) has enough hooks to do it, if only barely.
I’d love to help. I wish I had the time. If a few months go by and you’re still looking for something like this, the timing might be better for me.
Um. No one mentioned Slogger? That’s exactly what you want.
http://www.kenschutte.com/slogger/
Please add firefox cookies/bad web sites immunization in next version!
Firefox 2 cannot reject third party cookies!!!!!!!!